Zero-sample license authentic identification method based on pre-training large model

Abstract

The invention belongs to the technical field of license authentication, and discloses a zero-sample license authentication method based on a pre-trained large model, which comprises the following steps: firstly, constructing a new license anti-counterfeiting description template library by self-adaptive combination of anti-counterfeiting feature description of an old license and a small amount of newly-added feature description officially; then, a text encoder of a pre-trained multi-mode large model Constrative Language-Image Pre-training (CLIP) is adopted to extract text features corresponding to the anti-counterfeiting feature text description in the anti-counterfeiting template library; in the authentic identification stage, a pre-trained segmentation large model SegmentAnything Model (SAM) is utilized to segment anti-counterfeiting features from a certificate to be detected, and an image encoder of CLIP is utilized to encode the anti-counterfeiting features to obtain image features. The cosine similarity between the image features and the text features is calculated to judge whether the input license sample conforms to the anti-counterfeiting features in the anti-counterfeiting template library, so that zero-sample authentication of the novel license is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of certificate authentication, and in particular, relates to a zero-sample certificate authentication method and system based on a pre-trained large model. Background Art

[0002] Existing certificate authentication technologies mainly rely on supervised learning-based methods, which require a large number of labeled samples to train the model to ensure effective recognition of the anti-counterfeiting features of certificates. However, certificate designs vary from country to country, and the anti-counterfeiting features of certificates are constantly changing with version updates, making it very difficult to collect and annotate enough new certificate samples in practical applications. This high reliance on labeled data has resulted in poor adaptability of existing authentication models when faced with new types of certificates, which has seriously affected the accuracy and reliability of authentication.

[0003] Traditional certificate authentication technologies are usually based on visual feature matching, and the authenticity of certificates is determined by comparing significant features such as watermarks, ultraviolet anti-counterfeiting patterns, and optically variable inks. These methods rely on the explicit design and visibility of features, so they are difficult to respond flexibly to certificates with fast version updates and high feature diversity. Especially when faced with new types of certificates that have not been seen before, these methods based on manually designed features often lack sufficient generalization capabilities. In addition, the process of manually extracting and comparing anti-counterfeiting features is cumbersome and easily affected by human subjective factors, resulting in low efficiency and high misjudgment rate in actual operation.

[0004] In recent years, with the rapid development of deep learning, certificate authentication methods based on neural networks have gradually replaced traditional methods of manually designing features. These deep learning methods can automatically extract and identify complex anti-counterfeiting features by learning image features from a large number of samples. However, these methods still face the problem of lack of sample data, especially when dealing with new types of certificates or rare types of certificates. The lack of labeled samples makes it difficult for the model to effectively learn the corresponding features, thereby reducing the accuracy of authentication. At the same time, some deep learning methods rely on specific certificate types and features, which makes them show poor generalization ability in scenarios where certificate versions change frequently.

[0005] With the development of globalization, the demand for cross-border travel and identity authentication is increasing, and the requirements for security and real-time certificate authentication are becoming increasingly higher. Existing authentication systems are unable to cope with the challenges of new certificates and are in urgent need of a new method that can reduce the reliance on a large number of labeled samples and have high adaptability and generalization capabilities to cope with the ever-changing design of certificate anti-counterfeiting features. This provides an opportunity for the development of new certificate authentication technologies based on deep learning and large-scale pre-trained models, thereby improving the automation and accuracy of certificate authentication and meeting the requirements of efficiency and reliability in practical applications.

[0006] Through the above analysis, the problems and defects of the prior art are as follows:

[0007] (1) Existing methods still face the problem of lack of sample data, especially when dealing with new types of certificates or rare types of certificates. The lack of labeled samples will make it difficult for the model to effectively learn the corresponding features, thereby reducing the accuracy of authentication.

[0008] (2) Some deep learning methods rely on specific certificate types and features, which makes them exhibit poor generalization ability in scenarios where certificate versions frequently change. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a zero-sample certificate authentication method based on a pre-trained large model.

[0010] The present invention is implemented as follows: a zero-sample certificate authentication method based on a pre-trained large model comprises:

[0011] S1: Use the adaptive combination of anti-counterfeiting feature descriptions of old certificates and licenses, combined with a small amount of descriptions provided by the authorities, to build a new version of the certificate anti-counterfeiting description template library;

[0012] S2: Use the text encoder of the pre-trained multimodal large model CLIP to extract the text features corresponding to the anti-counterfeiting feature text description in the anti-counterfeiting template library;

[0013] S3: Use the pre-trained segmentation model SAM to segment the certificate image to be tested and extract anti-counterfeiting features;

[0014] S4: Use the image encoder of the pre-trained multimodal large model CLIP to encode the extracted anti-counterfeiting features to obtain the corresponding image features;

[0015] S5: By calculating the cosine similarity between the image features and the text features, it is determined whether the input certificate sample has anti-counterfeiting features that match the anti-counterfeiting template library. When the number of anti-counterfeiting features that match is less than the target threshold, the input sample is determined to be a fake certificate.

[0016] Furthermore, the adaptive combination in step S1 includes analyzing the anti-counterfeiting feature description of the old version of the certificate and generating multiple combinations to cover various anti-counterfeiting feature combination forms in the new certificate.

[0017] Furthermore, the text features extracted by the text encoder in step S2 include feature vectors obtained through cross-modal contrastive learning, and the feature vectors are used for feature matching with image features.

[0018] Furthermore, the segmentation model SAM in step S3 locates the key anti-counterfeiting area in the certificate through an automatic segmentation method to ensure the accuracy of the subsequent encoding process.

[0019] Furthermore, the calculation of cosine similarity in step S4 is used to measure the similarity between the text features in the template library and the image features of the input certificate sample. When the similarity between the text features and the image features in the anti-counterfeiting template library is less than a preset threshold, it is determined that the anti-counterfeiting feature in the template library does not exist.

[0020] Furthermore, the zero-sample certificate authentication method based on the pre-trained large model is applicable to the authentication of various certificates including but not limited to passports, identity cards, and visas.

[0021] Another object of the present invention is to provide a zero-sample certificate authentication system based on a pre-trained large model, comprising:

[0022] A construction module is used to construct a new version of the certificate anti-counterfeiting description template library by using the adaptive combination of the anti-counterfeiting feature descriptions of the old version of the certificate and combining it with a small amount of descriptions provided by the authorities;

[0023] An extraction module is used to extract text features corresponding to the text description of anti-counterfeiting features in the anti-counterfeiting template library using a text encoder of a pre-trained multimodal large model CLIP;

[0024] The segmentation module is used to segment the image of the certificate to be tested using the pre-trained segmentation model SAM and extract anti-counterfeiting features;

[0025] An encoding module is used to encode the extracted anti-counterfeiting features using the image encoder of the pre-trained multimodal large model CLIP to obtain corresponding image features;

[0026] The judgment module is used to judge whether the input certificate sample has anti-counterfeiting features that meet the anti-counterfeiting template library by calculating the cosine similarity between the image features and the text features. When the number of anti-counterfeiting features that meet the requirements is less than the target threshold, the input sample is judged to be a fake certificate.

[0027] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the zero-sample certificate authentication method based on a pre-trained large model.

[0028] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the zero-sample certificate authentication method based on a pre-trained large model.

[0029] Another object of the present invention is to provide an information data processing terminal, which is used to implement the zero-sample certificate authentication system based on the pre-trained large model.

[0030] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0031] First, the present invention reduces the reliance on labeled data. By introducing a pre-trained large model, the present invention effectively reduces the reliance on labeled samples. Compared with the high sample requirements of traditional supervised learning methods, the present invention only needs to combine existing anti-counterfeiting features and a small amount of official description information to achieve authentication, which significantly improves adaptability and flexibility. Improve adaptability to new types of certificates and licenses: The present invention can quickly respond to changes in new types of certificates and licenses by adaptively combining anti-counterfeiting features and generating anti-counterfeiting template libraries. This method improves the generalization ability of the model for new certificates and licenses, reduces the recognition difficulties caused by insufficient samples of new certificates and licenses, and ensures that the authentication system can still maintain high accuracy in ever-changing actual scenarios. Achieve efficient and automated authentication: The present invention realizes the automatic segmentation and encoding of anti-counterfeiting features through the SAM model and the CLIP model, and all key steps are automatically completed by the model, greatly reducing the need for human intervention. Compared with traditional methods that rely on manual feature extraction, the present invention has obvious advantages in the real-time and accuracy of authentication, and can provide stable and reliable authentication results in complex application environments.

[0032] Second, the technical solution of the present invention can realize the automatic authentication of certificates under zero-sample conditions by combining a pre-trained multimodal large model (such as CLIP) and a segmentation model (such as SAM), filling the gap in the lack of generalization ability of existing technologies for new certificates. In view of the limitation of existing automated authentication technology for certificates that relies on labeled samples, the present invention proposes a technical solution based on a pre-trained large model to solve the recognition difficulties caused by insufficient samples of new certificates. At the same time, by automatically extracting key anti-counterfeiting areas through segmentation of large models, the present invention overcomes the shortcomings of traditional manually designed features in terms of diversity and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of a zero-sample certificate authentication method based on a pre-trained large model provided in an embodiment of the present invention.

[0034] Figure 2 It is a structural block diagram of a zero-sample certificate authentication system based on a pre-trained large model provided in an embodiment of the present invention.

[0035] Figure 3 It is a detailed flowchart of a zero-sample certificate authentication method based on a pre-trained large model provided in an embodiment of the present invention.

[0036] Figure 4 It is a schematic diagram of extracting anti-counterfeiting features based on segmenting a large model SAM provided by an embodiment of the present invention.

[0037] Figure 5 It is a schematic diagram of identifying anti-counterfeiting features of new certificates based on the multimodal large model CLIP provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] Example 1: Anti-counterfeiting detection of new ID cards

[0040] 1) Step S1: Construction of anti-counterfeiting feature template library

[0041] In view of the anti-counterfeiting features of the new ID card, including microtext, watermark pattern, rainbow holographic anti-counterfeiting mark, etc., we combined the anti-counterfeiting feature description of the old ID card and a small amount of feature information provided by the authorities to build an anti-counterfeiting description template library. The template library describes features such as "the microtext 'CHINA' is displayed at a specific angle at the bottom of the front" and "the optically variable ink area changes color with the viewing angle".

[0042] 2) Step S2: Text feature encoding

[0043] The CLIP text encoder is used to encode the descriptions in the template library and extract the text feature vectors of each anti-counterfeiting feature.

[0044] 3) Step S3: ID card segmentation

[0045] The ID card image to be tested is input into the system, and the pre-trained segmentation large model SAM is used to segment the ID card image and automatically extract key anti-counterfeiting areas, such as micro-text area, watermark area, optically variable area, etc.

[0046] 4) Step S4: Image feature encoding

[0047] The anti-counterfeiting feature areas extracted by SAM are encoded by the CLIP image encoder to generate image feature vectors representing the visual features of each anti-counterfeiting area.

[0048] 5) Step S5: Feature matching and authenticity judgment

[0049] By calculating the cosine similarity between the image feature vector and the text feature vector, the anti-counterfeiting features such as microtext, watermarks, and optically variable areas are matched one by one. If the number of matching features of the input ID card is less than the set threshold (such as less than 5), the ID card is judged to be a fake; otherwise, it is judged to be a real ID card.

[0050] In the detection of new version ID cards, this embodiment can accurately identify fake ID cards under zero sample training conditions, and the detection accuracy rate reaches 98%.

[0051] Example 2: Anti-counterfeiting detection of new passports

[0052] 1) Step S1: Construction of anti-counterfeiting feature template library

[0053] Describe the anti-counterfeiting features of the new version of the passport, such as transparent watermarks, anti-counterfeiting lines, ultraviolet fluorescent patterns, etc., and build an anti-counterfeiting description template library by combining the anti-counterfeiting descriptions of the old version of the passport. For example, descriptions include: "The 'national emblem' pattern is displayed under ultraviolet light" and "The central area of ​​the cover has embossed anti-counterfeiting lines."

[0054] 2) Step S2: Text feature encoding

[0055] The text description in the anti-counterfeiting template library is encoded through the CLIP text encoder to generate the corresponding text feature vector.

[0056] 3) Step S3: Passport Image Segmentation

[0057] The passport image to be tested is input into the system, and the image is automatically segmented through the SAM model to locate the anti-counterfeiting area, including ultraviolet fluorescent patterns, watermark areas, embossed lines, etc.

[0058] 4) Step S4: Image feature encoding

[0059] The anti-counterfeiting area image segmented by SAM is encoded by the CLIP image encoder to obtain the image feature vector, which is used to match the text features in the template library.

[0060] 5) Step S5: Similarity calculation and authentication

[0061] The system calculates the cosine similarity between the image features and the text features to determine whether the anti-counterfeiting features in each area meet the template description. If the number of anti-counterfeiting features that meet the target threshold (such as 4 features), the system determines that the passport is a fake; otherwise, it is a real passport.

[0062] In passport anti-counterfeiting detection, the system achieves efficient detection of new passports and can quickly identify key anti-counterfeiting features such as ultraviolet fluorescence and watermarks, with a detection accuracy rate of over 96%.

[0063] The above two embodiments perform anti-counterfeiting detection on identity cards and passports respectively. By combining CLIP with the SAM pre-trained large model, zero-sample anti-counterfeiting detection is achieved, which is suitable for authenticity identification of various new versions of certificates and licenses.

[0064] like Figure 1 As shown, a zero-sample certificate authentication method based on a pre-trained large model provided by an embodiment of the present invention includes the following steps:

[0065] S1: Use the adaptive combination of anti-counterfeiting feature descriptions of old certificates and licenses, combined with a small amount of descriptions provided by the authorities, to build a new version of the certificate anti-counterfeiting description template library;

[0066] S2: Use the text encoder of the pre-trained multimodal large model CLIP to extract the text features corresponding to the anti-counterfeiting feature text description in the anti-counterfeiting template library;

[0067] S3: Use the pre-trained segmentation model SAM to segment the certificate image to be tested and extract anti-counterfeiting features;

[0068] S4: Use the image encoder of the pre-trained multimodal large model CLIP to encode the extracted anti-counterfeiting features to obtain the corresponding image features;

[0069] S5: By calculating the cosine similarity between the image features and the text features, it is determined whether the input certificate sample has anti-counterfeiting features that match the anti-counterfeiting template library. When the number of anti-counterfeiting features that match is less than the target threshold, the input sample is determined to be a fake certificate.

[0070] The present invention first analyzes and describes the anti-counterfeiting features of the old version of certificates and licenses, and combines a small amount of text descriptions of the anti-counterfeiting features of the new version of certificates and licenses provided by the authorities to build a template library for the anti-counterfeiting description of the new version of certificates and licenses. Anti-counterfeiting features include a variety of identifiable visual and texture features, such as watermarks, reliefs, optically variable patterns, microtext, etc. By adaptively combining the feature descriptions of the old and new versions of certificates and licenses, the template library covers more types of anti-counterfeiting features, thereby meeting the requirements of zero-sample detection. This template library provides standardized reference data for subsequent text encoding and feature matching.

[0071] For the constructed anti-counterfeiting template library, the text encoder in the pre-trained multimodal large model CLIP is used to encode the anti-counterfeiting feature description. The text encoder of the CLIP model has the ability to map natural language descriptions to high-dimensional feature space through large-scale training data. In this process, the input anti-counterfeiting feature text description is processed by the encoder to extract text feature vectors. These vectors have a high degree of semantic expression and can be used as the basis for similarity calculation with image features.

[0072] For the input certificate image to be tested, the pre-trained segmentation model SAM (Segment Anything Model) is used for image segmentation. The SAM model can perform high-precision regional segmentation on the certificate image without a large number of training samples, and automatically identify and extract anti-counterfeiting feature areas, such as watermarks, background textures, micro-text and other details. Through the segmentation results, the system can effectively extract the anti-counterfeiting feature area of ​​the certificate, provide accurate input data for subsequent image feature encoding, avoid interference from irrelevant areas, and improve the accuracy of detection.

[0073] The pre-trained CLIP image encoder is used to encode the image features of the anti-counterfeiting feature area extracted by the SAM model. After large-scale image-text matching training, CLIP's image encoder has the ability to map image content to the same high-dimensional feature space as text features. In this step, the extracted image anti-counterfeiting features are processed by the image encoder to obtain an image feature vector, which can accurately express the visual information of the anti-counterfeiting features in the image and achieve cross-modal alignment with the text features.

[0074] Cross-modal anti-counterfeiting feature matching is achieved by calculating the cosine similarity between the image feature vector and the text feature vector. Specifically, the system calculates the similarity between the image features of the input certificate sample and all the anti-counterfeiting text features in the template library. When the similarity is higher than a certain threshold, the anti-counterfeiting feature is considered to exist in the certificate; otherwise, the feature is considered to be missing. The system counts the total number of anti-counterfeiting features in the template library that match the input certificate, and compares it with the set target threshold to determine the authenticity of the certificate.

[0075] After the similarity calculation is completed, the system makes a final judgment based on the matching results of all anti-counterfeiting features. When the number of features in the input sample that match the anti-counterfeiting template library is lower than the target threshold, the system determines that the input sample is a fake certificate; otherwise, it is considered a real certificate. This method combines the powerful capabilities of the pre-trained large model CLIP and SAM to quickly and accurately identify the authenticity of certificates under zero-sample conditions. It is efficient and universal, and is especially suitable for anti-counterfeiting verification and system updates for new certificates.

[0076] In summary, the method of the present invention is based on a pre-trained large model, and through technologies such as text and image feature encoding, image segmentation and cosine similarity calculation, it realizes cross-modal anti-counterfeiting feature matching and authenticity identification, solves the problem of zero-sample training in certificate anti-counterfeiting and authentication, and effectively improves the intelligence level and accuracy of anti-counterfeiting detection.

[0077] The adaptive combination in step S1 provided in the embodiment of the present invention includes analyzing the anti-counterfeiting feature description of the old version of the certificate and generating multiple combinations to cover various anti-counterfeiting feature combination forms in the new certificate.

[0078] The text features extracted by the text encoder in step S2 provided in the embodiment of the present invention include feature vectors obtained through cross-modal contrastive learning, and the feature vectors are used for feature matching with image features.

[0079] The segmentation model SAM in step S3 provided in the embodiment of the present invention locates the key anti-counterfeiting area in the certificate through an automatic segmentation method to ensure the accuracy of the subsequent encoding process.

[0080] The calculation of cosine similarity in step S4 provided in the embodiment of the present invention is used to measure the similarity between the text features in the template library and the image features of the input certificate sample. When the similarity between the text features and the image features in the anti-counterfeiting template library is less than a preset threshold, it is determined that the anti-counterfeiting feature in the template library does not exist.

[0081] The zero-sample certificate authentication method based on a pre-trained large model provided in the embodiment of the present invention is applicable to the authentication of various certificates including but not limited to passports, identity cards, and visas.

[0082] like Figure 2 As shown, a zero-sample certificate authentication system based on a pre-trained large model provided by an embodiment of the present invention includes:

[0083] A construction module is used to construct a new version of the certificate anti-counterfeiting description template library by using the adaptive combination of the anti-counterfeiting feature descriptions of the old version of the certificate and combining it with a small amount of descriptions provided by the authorities;

[0084] An extraction module is used to extract text features corresponding to the text description of anti-counterfeiting features in the anti-counterfeiting template library using a text encoder of a pre-trained multimodal large model CLIP;

[0085] The segmentation module is used to segment the image of the certificate to be tested using the pre-trained segmentation model SAM and extract anti-counterfeiting features;

[0086] An encoding module is used to encode the extracted anti-counterfeiting features using the image encoder of the pre-trained multimodal large model CLIP to obtain corresponding image features;

[0087] The judgment module is used to judge whether the input certificate sample has anti-counterfeiting features that meet the anti-counterfeiting template library by calculating the cosine similarity between the image features and the text features. When the number of anti-counterfeiting features that meet the requirements is less than the target threshold, the input sample is judged to be a fake certificate.

[0088] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the zero-sample certificate authentication method based on a pre-trained large model.

[0089] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the zero-sample certificate authentication method based on a pre-trained large model.

[0090] Another object of the present invention is to provide an information data processing terminal, which is used to implement the zero-sample certificate authentication system based on the pre-trained large model.

[0091] The present invention is specifically implemented:

[0092] like Figure 3 As shown, the present invention provides a zero-sample license authentication method based on a pre-trained large model. By combining the pre-trained large model with the adaptive combination technology, the dependence on labeled sample data is reduced. The specific implementation steps are as follows:

[0093] S1: Utilize the adaptive combination of anti-counterfeiting feature descriptions of old certificates and licenses, combined with a small amount of descriptions provided by the authorities, to build a new version of the certificate and license anti-counterfeiting description template library.

[0094] Specifically, the anti-counterfeiting features of the old version of the certificate are analyzed and summarized, and organized into corresponding text descriptions. Each anti-counterfeiting feature such as watermarks, optically variable inks, ultraviolet spectrum patterns, etc. will be summarized as a text description. For example, the "ultraviolet pattern" in the old version of the passport is described as "stripes visible under ultraviolet light." Then, these text descriptions are split and combined to generate a variety of possible anti-counterfeiting feature descriptions. For example, for the newly added ultraviolet anti-counterfeiting pattern in the new version of the passport, the "ultraviolet light wave stripes" and "plant pattern" can be split to form multiple possible anti-counterfeiting feature descriptions. Finally, these split descriptions are used to build an anti-counterfeiting template library for the new version of the certificate.

[0095] S2: Use the text encoder of the pre-trained multimodal large model CLIP to extract the text features corresponding to the anti-counterfeiting feature text description in the anti-counterfeiting template library.

[0096] Specifically, the text encoder of the pre-trained multimodal large model CLIP is used to encode the M anti-counterfeiting feature descriptions sorted out in step S1. The text encoder of the CLIP model can convert these text descriptions into feature vectors. Each vector corresponds to an anti-counterfeiting feature description in the anti-counterfeiting template library.

[0097] S3: Use the pre-trained segmentation model SAM to segment the certificate image to be tested and extract anti-counterfeiting features.

[0098] Specifically, Figure 4 As shown in the figure, the pre-trained segmentation model Segment Anything Model (SAM) is used to process the certificate image to be tested, and the key anti-counterfeiting feature area in the certificate is automatically segmented. The SAM model has zero-sample segmentation capability, which can identify and segment the area related to the anti-counterfeiting feature from the new version of the certificate sample, ensuring that the subsequent encoding process is not interfered by the background.

[0099] S4: Use the image encoder of the pre-trained multimodal large model CLIP to encode the extracted anti-counterfeiting features to obtain the corresponding image features.

[0100] Specifically, the pre-trained CLIP image encoder is used to encode the N anti-counterfeiting features segmented from the SAM model. The CLIP image encoder converts the anti-counterfeiting feature area in the image into an image feature vector The CLIP model uses cross-modal contrast learning to ensure that text features and image features can be compared in the same feature space. Therefore, after the image features are extracted, the image feature vector With text feature vector Cross-modal matching is possible.

[0101] S5: By calculating the cosine similarity between the image features and the text features, it is determined whether the input certificate sample has anti-counterfeiting features that match the anti-counterfeiting template library. When the number of anti-counterfeiting features that match is less than the target threshold, the input sample is determined to be a fake certificate.

[0102] Specifically, Figure 5 As shown, for each anti-counterfeiting feature description, calculate its corresponding text feature vector T m With all input image feature vectors I n The cosine similarity between them is calculated, and the corresponding maximum cosine similarity is taken as the probability p of the anti-counterfeiting feature. m , the calculation formula is as follows:

[0103]

[0104] Set a threshold α (0<α<1), when the anti-counterfeiting feature probability pm When it exceeds the preset threshold α, it is considered that the input sample contains the anti-counterfeiting feature. Therefore, the number of samples that meet the anti-counterfeiting feature is:

[0105]

[0106] Set the target threshold β (0<β<1). If the number of anti-counterfeiting features T that meet the requirements is less than β×M (M is the total number of features in the template library), the input sample is judged to be a fake certificate. Otherwise, it is judged to be a real certificate.

[0107] The present invention can be applied to the field of certificate inspection, and the technology can be applied in airport border inspection stations, etc. to determine the authenticity of new certificates.

[0108] Specific application examples of the present invention:

[0109] Equipment requirements:

[0110] (1) Multispectral image acquisition equipment, used to collect multiple spectral images such as visible light, ultraviolet light, and infrared light.

[0111] (2) Traditional identification and verification system, used to make a preliminary judgment on whether the certificate matches the old certificate template.

[0112] (3) A zero-sample certificate authentication system, which is used to run the authentication method based on the pre-trained large model described in the present invention to perform zero-sample authentication of certificates.

[0113] (4) Result feedback terminal, displaying the recognition result.

[0114] Specific implementation steps:

[0115] S1: The full spectrum image of the certificate is acquired through multispectral image acquisition equipment, including printed information under visible light, anti-counterfeiting patterns under ultraviolet light, hidden watermarks under infrared light, etc. After acquisition, the image is transmitted to the traditional reading and authentication system for preliminary comparison.

[0116] S2: Use the traditional identification system to analyze the collected certificate image and determine whether the certificate meets the specific anti-counterfeiting features of the old version of the certificate by template comparison. If the certificate meets the existing certificate template, it is directly judged as a genuine certificate; if the system cannot recognize it, the collected image data is input into the zero-sample certificate identification system for further identification.

[0117] S3: Use the zero-sample certificate authentication system to analyze the collected certificate image to determine whether the certificate is a genuine new type of certificate.

[0118] S4: The authentication result is transmitted to the border inspection staff through the result feedback terminal. If the passport is judged to be authentic, the passenger is released; if the passport is judged to be forged, the system marks it as abnormal, prompting the border inspection staff to conduct manual review, and generates a detailed anti-counterfeiting feature detection report, including the feature matching similarity of each segmented area.

[0119] Relevant evidence of the technical effects achieved by the embodiments of the present invention.

[0120] SAM is a pre-trained segmentation model that is trained with a large number of images and annotated data. It has strong generalization capabilities and can complete the segmentation task of new images without annotated data. This feature is particularly suitable for the extraction of anti-counterfeiting features of certificates and licenses, because the positions and shapes of anti-counterfeiting features in different certificates and licenses are diverse, but SAM can quickly identify areas with significant features based on image structural information. Without the need for special training on anti-counterfeiting features, key feature areas such as ultraviolet patterns and optically variable ink areas can be automatically segmented to ensure that the segmentation results cover all possible anti-counterfeiting information.

[0121] CLIP is a pre-trained multimodal large model that maps text descriptions and image features into a unified feature space through cross-modal contrastive learning technology, so that the similarity between text descriptions and corresponding image features can be directly calculated. This capability enables CLIP to use text descriptions in the anti-counterfeiting template library to identify corresponding features in new certificates without the need for additional annotated data. This method is not only computationally efficient, but also effectively quantifies the semantic relevance of image and text features, and is suitable for matching tasks of complex anti-counterfeiting features.

[0122] By combining the zero-sample capabilities of SAM and CLIP, the present invention can complete the task of identifying counterfeit new certificates in an efficient, flexible and accurate manner. The SAM model solves the problem of accurate segmentation of key anti-counterfeiting areas, overcomes the limitations of traditional manually designed feature segmentation, and provides high-quality input for subsequent recognition tasks. The CLIP model uses cross-modal contrastive learning technology to quantify the similarity between text and image features through cosine similarity, making it possible to identify anti-counterfeiting features based on text descriptions, significantly improving the accuracy and efficiency of recognition. This combination of technologies not only achieves efficient extraction and matching of anti-counterfeiting features under zero-sample conditions, but also has good scalability and adaptability, providing a practical and advanced solution for solving the anti-counterfeiting identification of new certificates.

[0123] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0124] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A zero-sample certificate authentication method based on a pre-trained large model, characterized in that: The following steps are involved: S1: Use the adaptive combination of anti-counterfeiting feature descriptions of old certificates and licenses, combined with a small amount of descriptions provided by the authorities, to build a new version of the certificate anti-counterfeiting description template library; S2: Use the text encoder of the pre-trained multimodal large model CLIP to extract the text features corresponding to the anti-counterfeiting feature text description in the anti-counterfeiting template library; S3: Use the pre-trained segmentation model SAM to segment the certificate image to be tested and extract anti-counterfeiting features; S4: Use the image encoder of the pre-trained multimodal large model CLIP to encode the extracted anti-counterfeiting features to obtain the corresponding image features; S5: By calculating the cosine similarity between the image features and the text features, it is determined whether the input certificate sample has anti-counterfeiting features that match the anti-counterfeiting template library. When the number of anti-counterfeiting features that match is less than the target threshold, the input sample is determined to be a fake certificate.

2. The zero-sample certificate authentication method based on a pre-trained large model as claimed in claim 1, characterized in that: The adaptive combination in step S1 includes analyzing the anti-counterfeiting feature description of the old version of the certificate and generating multiple combinations to cover various anti-counterfeiting feature combination forms in the new certificate.

3. The zero-sample certificate authentication method based on a pre-trained large model as claimed in claim 1, characterized in that: The text features extracted by the text encoder in step S2 include feature vectors obtained through cross-modal contrastive learning, and the feature vectors are used for feature matching with image features.

4. The zero-sample certificate authentication method based on a pre-trained large model as claimed in claim 1, characterized in that: The segmentation model SAM in step S3 locates the key anti-counterfeiting area in the certificate through an automatic segmentation method to ensure the accuracy of the subsequent encoding process.

5. The zero-sample certificate authentication method based on a pre-trained large model as claimed in claim 1, characterized in that: The calculation of cosine similarity in step S4 is used to measure the similarity between the text features in the template library and the image features of the input certificate sample. When the similarity between the text features and the image features in the anti-counterfeiting template library is less than a preset threshold, it is determined that the anti-counterfeiting feature in the template library does not exist.

6. The zero-sample certificate authentication method based on a pre-trained large model as claimed in claim 1, characterized in that: The zero-sample certificate authentication method based on the pre-trained large model is applicable to the authentication of multiple certificates including but not limited to passports, identity cards, and visas.

7. A zero-sample certificate authentication system based on a pre-trained large model that implements the zero-sample certificate authentication method based on a pre-trained large model as described in any one of claims 1 to 6, characterized in that: The zero-sample certificate authentication system based on the pre-trained large model includes: A construction module is used to construct a new version of the certificate anti-counterfeiting description template library by using the adaptive combination of the anti-counterfeiting feature descriptions of the old version of the certificate and combining it with a small amount of descriptions provided by the authorities; An extraction module is used to extract text features corresponding to the text description of anti-counterfeiting features in the anti-counterfeiting template library using a text encoder of a pre-trained multimodal large model CLIP; The segmentation module is used to segment the image of the certificate to be tested using the pre-trained segmentation model SAM and extract anti-counterfeiting features; An encoding module is used to encode the extracted anti-counterfeiting features using the image encoder of the pre-trained multimodal large model CLIP to obtain corresponding image features; The judgment module is used to judge whether the input certificate sample has anti-counterfeiting features that meet the anti-counterfeiting template library by calculating the cosine similarity between the image features and the text features. When the number of anti-counterfeiting features that meet the requirements is less than the target threshold, the input sample is judged to be a fake certificate.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the zero-sample certificate authentication method based on a pre-trained large model as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the zero-sample certificate authentication method based on a pre-trained large model as described in any one of claims 1 to 6.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the zero-sample certificate authentication system based on the pre-trained large model as described in claim 7.

Images