Electronic certificate anti-counterfeiting storage verification method based on fusion of OCR and micro-text features

Abstract

The application provides an electronic certificate anti-forgery storage verification method based on OCR and micro-texture feature fusion. The method combines OCR text semantic recognition and physical feature extraction: by binding text content and paper fiber distribution, ink point diffusion and other non-reproducible physical features, the single defect that traditional pure OCR technology is vulnerable to content tampering attacks and pure hash technology cannot identify semantic tampering is effectively solved. In the feature extraction stage, a dynamic semantic alignment module is designed, which can mark the high-value semantic areas such as ID number and signature in the OCR text positioning process, and accordingly guide the texture hash algorithm to preferentially process these high-risk areas, thereby improving the protection strength of the key information. A multi-scale texture enhanced hash algorithm is provided, which combines the advantages of frequency domain wavelet energy analysis and spatial LBP feature extraction, so that even under the condition of low resolution caused by mobile phone shooting, the anti-degradation extraction of texture features can still be realized, and the accuracy of forgery detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic certificate anti-counterfeiting, and specifically relates to an electronic certificate anti-counterfeiting, storage and verification method based on OCR and microscopic texture feature fusion. BACKGROUND

[0002] In the prior art, the electronic certificate anti-counterfeiting, storage and verification method includes anti-counterfeiting technologies based on digital watermarking technology, blockchain storage technology, physical anti-counterfeiting technology and deep learning-based anti-counterfeiting technology. These technologies have the following disadvantages.

[0003] The disadvantage of the anti-counterfeiting technology based on digital watermarking technology is that the watermark is easily damaged by image processing operations such as compression and format conversion, attackers can counterfeit the same watermark through reverse engineering, and verification relies on professional software or hardware, which is difficult to popularize and apply on lightweight devices such as mobile phones.

[0004] The disadvantage of the blockchain storage technology is that the technology can only verify whether the document has been tampered with, cannot identify "original replacement" attacks (such as generating a new document with the same content but different background after PS tampering), and the traditional hash algorithm only processes the text content, completely ignoring the physical characteristics of the document (such as paper texture and printing defects), in addition, the cost of storing the complete document hash chain is high, and most solutions need to rely on centralized storage and hash value.

[0005] The disadvantage of the physical anti-counterfeiting technology is that the special materials and processes result in a significant increase in document production costs, and this technology is completely unsuitable for electronic documents or verification scenarios captured by mobile phones. Even for physical documents, high-precision scanning devices can copy some anti-counterfeiting features, and the actual anti-counterfeiting effect continues to decline with technological progress.

[0006] The disadvantage of the deep learning-based anti-counterfeiting technology is that the technology relies on a large amount of labeled data for training, and the model generalization ability is limited by the diversity of training samples, which may result in performance degradation when applied across scenes. In addition, the deep learning model has high computational complexity and is difficult to run in real time on resource-constrained edge devices.

[0007] In summary, the existing electronic certificate anti-counterfeiting, storage and verification method has the disadvantages of high verification cost and low robustness. SUMMARY

[0008] The present application aims to provide an electronic certificate anti-counterfeiting, storage and verification method based on OCR and microscopic texture feature fusion, which solves the problem of high verification cost and low robustness of the existing electronic certificate anti-counterfeiting, storage and verification method.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: an electronic certificate anti-counterfeiting and verification method based on OCR and micro-texture feature fusion, comprising the following steps: S1, a preprocessing module performs grayscale conversion and geometric correction to ensure image standardization; S2, text localization and semantic parsing are performed through an OCR engine, outputting keyword coordinates and semantic tags; S3, a dynamic semantic alignment module generates a risk heatmap based on the semantic tags output by the OCR, marking high-risk areas. After obtaining the marked high-risk areas, a risk heatmap of the corresponding area is generated. The weight of each pixel in the heatmap reflects the importance of its region. The dynamic semantic alignment module includes semantic parsing and region marking, OCR engine... First, the engine performs text detection and recognition on the document, outputting the text content and its coordinate information. The dynamic semantic alignment module marks high-risk areas according to predefined semantic rules and generates a binary mask based on the normalized heatmap. The obtained normalized heatmap and mask are then input into the multi-scale texture enhancement hash algorithm module. S4: The multi-scale texture enhancement hash algorithm extracts the micro-texture features of the document based on the priority of the heatmap. The multi-scale texture enhancement hash algorithm includes wavelet multi-scale decomposition, frequency domain energy feature extraction, spatial domain adaptive LBP feature extraction, feature fusion and dimensionality reduction, and texture hash value generation. S5: The text hash and texture hash are fused to generate a joint hash value, which is then stored on the blockchain to complete the anti-counterfeiting registration.

[0010] A further technical solution is that, in step S5, the system retrieves the original joint hash value stored on the blockchain and compares it with the joint hash value. If they match, the document is determined not to have been tampered with; otherwise, a tampering alarm is triggered.

[0011] A further technical solution is that, in step S3, the dynamic semantic alignment module includes semantic parsing and region labeling. The OCR engine first performs text detection and recognition on the document, outputting the text content and its coordinate information. The dynamic semantic alignment module labels high-risk regions according to predefined semantic rules. Let the semantic label of the i-th text region in the document be... Its coordinates are within a rectangle. ,in , It is the coordinate of the top left corner of the region. , These are the width and height of the region; the set R of high-risk regions is defined as: , For a predefined set of key semantic tags, the semantic tags of high-risk regions are matched: , The regular expression for identity. A list of keywords related to the signature is generated; after obtaining the marked high-risk area R, a risk heatmap of the corresponding area is generated. The weight of each pixel in the heatmap reflects the importance of its area, and the calculation formula is as follows: ,in and For the region The center coordinates, The bandwidth parameter of the Gaussian kernel controls the weight decay rate; to avoid scale differences in heatmaps from different documents, the heatmap... Normalization is performed: , for The maximum value is normalized to a range of [0, 1]. Based on the normalized heatmap, region activation is performed through threshold segmentation, and a binary mask is generated based on the normalized heatmap. Mark the areas that need to be processed first: ,in, It is a preset threshold; the obtained normalized heatmap and mask It is input into the multi-scale texture enhancement hashing algorithm module.

[0012] A further technical solution is that the multi-scale texture enhancement hash algorithm in step S4 includes wavelet multi-scale decomposition, frequency domain energy feature extraction, spatial domain adaptive LBP feature extraction, feature fusion and dimensionality reduction, and hash generation. Specifically, wavelet multi-scale decomposition involves performing a two-dimensional discrete wavelet transform on the input standardized image to decompose the image at different scales, as shown in the following formula: Where I is the input image and k is the number of decomposition layers. It is a low-frequency component. It selects high-frequency components, then discards all low-frequency components and retains the high-frequency components. Used for frequency domain energy feature extraction calculation; in frequency domain energy feature extraction, the directional energy distribution of high-frequency components is calculated to capture the frequency domain statistical characteristics of the texture, as shown in the following formula: , where dir is Representing three high-frequency directions, The number of effective pixels within the mask area. Represented as the Kth layer subband coefficient, combining all subband energy values, a 9-dimensional frequency domain feature vector is constructed, expressed as: The inputs for spatial adaptive LBP feature extraction are the original image and the mask. Adaptive LBP calculation is performed: the improved LBP value is calculated pixel by pixel within the mask area, with the center pixel as the reference. Based on this, extract the pixel values ​​of its 3×3 neighborhood. Calculate the neighborhood mean and standard deviation , is represented as: Generate a binary pattern based on a dynamic threshold: Then, texture histograms are generated at three scales: Scale 1 (radius R=1): calculate the basic LBP value and generate a 256-dimensional histogram. Scale 2 (radius R=2): Bilinear interpolation is used to obtain neighboring pixels and generate... Scale 3 (radius R=3): Bilinear interpolation is used to obtain neighboring pixels and generate... The three histograms are concatenated to form a 768-dimensional feature vector: Finally, the eigenvectors of the spatial domain are obtained. In feature fusion and dimensionality reduction, the input frequency domain features are... airspace features Feature normalization is performed by normalizing the two feature vectors using the L2 norm: The fusion weights are calculated based on the image resolution: when the resolution is <150 PPI, the frequency domain weights are increased. When the resolution is >300 PPI, increase the spatial weight. Specific weighting formula: , ,in , where res is the image resolution, and the fused features are: By compressing features using a pre-trained PCA projection matrix, a 64-dimensional compressed feature is output: ,in For PCA projection matrix; compress features in hash generation Calculate the mean of the eigenvectors: Then generate binary hashes by dimension: It converts a 64-bit binary sequence into a hexadecimal string and outputs a 64-bit texture hash value.

[0013] Compared with the prior art, the beneficial effects of the present invention are:

[0014] 1. This invention achieves intelligent localization and enhancement of key regions in documents through a Dynamic Semantic Alignment (DSA) module. Based on OCR semantic recognition results and combined with a predefined semantic rule library (such as regular expression matching for ID card numbers and a signature keyword library), this module dynamically identifies high-risk regions (such as ID card numbers and signature fields). The core innovation lies in generating a Gaussian-weighted risk heatmap and a binary mask, using spatial weight distribution to guide subsequent texture extraction algorithms to focus on key regions. This design solves the problem of insufficient protection for tamper-sensitive regions in traditional methods, significantly improving the detection accuracy against attacks such as PS tampering and partial replacement.

[0015] 2. This invention proposes a Multi-Scale Texture Enhancement Hash (MTEH) algorithm to address the degradation of microscopic texture features in low-resolution images. This algorithm innovatively integrates frequency domain analysis and spatial domain features: it extracts the directional energy of high-frequency components through three-level wavelet decomposition, capturing frequency domain statistical features resistant to resolution degradation; in the spatial domain, it employs dynamic thresholding (LBP) technology, adaptively adjusting the binarization threshold through neighborhood standard deviation to resist illumination interference. Combined with a resolution-adaptive feature weighting mechanism and PCA dimensionality reduction quantization, a 64-bit lightweight texture hash is finally generated. This scheme can still stably extract microscopic physical features such as paper fibers and ink dot distribution even in edge scenarios such as mobile phone photography. Attached Figure Description

[0016] Fig. 1 This is a schematic diagram of the process of the present invention.

[0017] Fig. 2 This is a schematic diagram of the operation process of the present invention.

[0018] Fig. 3 This is a schematic diagram of the verification process of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] Example:

[0021] refer to Figs. 1 to 3 As shown, a method for verifying and preserving electronic certificates based on the fusion of OCR and micro-texture features is disclosed, including the following steps: S1, the preprocessing module performs grayscale conversion and geometric correction to ensure image standardization; S2, the OCR engine performs text localization and semantic parsing, outputting keyword coordinates and semantic tags; S3, the dynamic semantic alignment module generates a risk heatmap based on the semantic tags output by the OCR, marking high-risk areas; S4, the multi-scale texture enhancement hash algorithm extracts the micro-texture features of the document according to the priority of the heatmap; S5, the text hash and texture hash are fused to generate a joint hash value, which is then stored on the blockchain to complete the anti-counterfeiting registration.

[0022] In step S5, the system retrieves the original combined hash value stored on the blockchain and compares it with the combined hash value. If they match, the document is determined not to have been tampered with; otherwise, a tampering alarm is triggered.

[0023] The overall structure of the Dynamic Semantic Alignment (DSA) module is designed to dynamically identify high-risk areas (such as ID numbers and signature fields) in documents using semantic tags output by OCR, and generate a risk heatmap to guide subsequent texture feature extraction algorithms to prioritize these key areas. By combining semantic understanding and spatial localization, the DSA module significantly enhances the anti-counterfeiting system's ability to protect critical information.

[0024] The dynamic semantic alignment module in step S3 includes semantic parsing and region labeling. The OCR engine first performs text detection and recognition on the document, outputting the text content and its coordinate information. The dynamic semantic alignment module labels high-risk areas according to predefined semantic rules (such as matching ID card numbers with regular expressions, matching signature bars with keywords, etc.). Let the semantic label of the i-th text region in the document be . Its coordinates are within a rectangle. ,in , It is the coordinate of the top left corner of the region. , These are the width and height of the region; the set R of high-risk regions is defined as: , Given a predefined set of key semantic tags (such as ID numbers, signatures, official seals, etc.), semantic tags in high-risk areas are matched using: , The regular expression for identity. For the signature-related keyword list (such as "signature", "signature", etc.), appropriate high-risk area matching rules need to be used according to different certificate types; after obtaining the marked high-risk areas R, a risk heatmap of the corresponding areas is generated. The weight of each pixel in the heatmap reflects the importance of its area, and the calculation formula is: ,in and For the region The center coordinates, The bandwidth parameter of the Gaussian kernel controls the weight decay rate; to avoid scale differences in heatmaps from different documents, the heatmap... Normalization is performed: , for The maximum value is normalized to a range of [0, 1]. Based on the normalized heatmap, region activation is performed through threshold segmentation, and a binary mask is generated based on the normalized heatmap. Mark the areas that need to be processed first: ,in, It is a preset threshold; the obtained normalized heatmap and mask It is input into the multi-scale texture enhancement hashing algorithm module.

[0025] Multi-scale texture enhancement hashing (MTEH) addresses the problem of texture feature degradation in low-resolution images (such as those captured by mobile phones). By fusing frequency-domain wavelet energy analysis with spatial-domain adaptive LBP features, it achieves robust extraction of microscopic physical features such as paper fibers and ink dot distribution, generating lightweight and degradation-resistant texture hash values.

[0026] The multi-scale texture enhancement hashing algorithm described in step S4 includes wavelet multi-scale decomposition, frequency domain energy feature extraction, spatial domain adaptive LBP feature extraction, feature fusion and dimensionality reduction, and hash generation. Specifically, wavelet multi-scale decomposition involves performing a two-dimensional discrete wavelet transform on the input standardized image to decompose the image at different scales (resolutions), as shown in the following formula: Where I is the input image and k is the decomposition layer number, which is 3 in this invention. It is a low-frequency component (approximate image). It selects high-frequency components (horizontal, vertical, and diagonal details) and then discards all low-frequency components. (Retains macroscopic structure but less texture detail), retains high-frequency components. In frequency domain energy feature extraction calculations, the main function of wavelet multi-scale decomposition is to: preserve low-frequency structure while progressively amplifying high-frequency texture details through three-level decomposition, thus solving the problem of blurred microscopic features in low-resolution images; and to calculate the directional energy distribution of high-frequency components in frequency domain energy feature extraction, capturing the frequency domain statistical characteristics of texture, as shown in the following formula: , where dir is Representing three high-frequency directions, The number of effective pixels within the mask area. Represented as the Kth layer subband coefficient, combining all subband energy values, a 9-dimensional frequency domain feature vector is constructed, expressed as: The purpose of this step is to extract high-frequency energy features: high-frequency energy is not sensitive to resolution degradation and only focuses on high-risk areas, ensuring the stability of images captured by mobile phones; the input in spatial adaptive LBP feature extraction is the original image and the mask. Adaptive LBP calculation is performed: the improved LBP value is calculated pixel by pixel within the mask area, with the center pixel as the reference. Based on this, extract the pixel values ​​of its 3×3 neighborhood. Calculate the neighborhood mean and standard deviation , is represented as: Generate a binary pattern based on a dynamic threshold: Then, texture histograms are generated at three scales: Scale 1 (radius R=1): calculate the basic LBP value and generate a 256-dimensional histogram. Scale 2 (radius R=2): Bilinear interpolation is used to obtain neighboring pixels and generate... Scale 3 (radius R=3): Bilinear interpolation is used to obtain neighboring pixels and generate... The three histograms are concatenated to form a 768-dimensional feature vector: Finally, the eigenvectors of the spatial domain are obtained. In feature fusion and dimensionality reduction, the input frequency domain features are... airspace features Feature normalization is performed by normalizing the two feature vectors using the L2 norm: The fusion weights are calculated based on the image resolution: when the resolution is <150 PPI, the frequency domain weights are increased. When the resolution is >300 PPI, increase the spatial weight. Specific weighting formula: , ,in , where res is the image resolution, and the fused features are: By compressing features using a pre-trained PCA projection matrix, a 64-dimensional compressed feature is output: ,in Using the PCA projection matrix, the above steps allow for dynamic weighting to address feature degradation at different resolutions. Dimensionality reduction via PCA reduces computation by a factor of 12. Features are compressed during hash generation. Calculate the mean of the eigenvectors: Then generate binary hashes by dimension: It converts a 64-bit binary sequence into a hexadecimal string (e.g., 5A3F...E9C1) and outputs a 64-bit texture hash value.

[0027] This invention achieves intelligent localization and enhancement of key regions in documents through a Dynamic Semantic Alignment (DSA) module. Based on OCR semantic recognition results and combined with a predefined semantic rule base (such as regular expression matching for ID card numbers and a signature keyword library), this module dynamically identifies high-risk regions (such as ID card numbers and signature fields). The core innovation lies in generating a Gaussian-weighted risk heatmap and a binary mask, using spatial weight distribution to guide subsequent texture extraction algorithms to focus on key regions. This design addresses the problem of insufficient protection of tamper-sensitive areas in traditional methods, significantly improving the detection accuracy against attacks such as PS tampering and partial replacement.

[0028] This invention also proposes a multi-scale texture enhancement hashing algorithm (MTEH) to address the degradation of microscopic texture features in low-resolution images. This algorithm innovatively integrates frequency domain analysis and spatial domain features: it extracts the directional energy of high-frequency components through three-level wavelet decomposition, capturing frequency domain statistical features resistant to resolution degradation; in the spatial domain, it employs dynamic thresholding (LBP) technology, adaptively adjusting the binarization threshold through neighborhood standard deviation to resist illumination interference. Combined with a resolution-adaptive feature weighting mechanism and PCA dimensionality reduction quantization, a 64-bit lightweight texture hash is finally generated. This scheme can still stably extract microscopic physical features such as paper fibers and ink dot distribution even in edge scenarios such as mobile phone photography.

[0029] Although the invention has been described herein with reference to several illustrative embodiments, it should be understood that many other modifications and implementations can be devised by those skilled in the art, which will fall within the scope and spirit of the principles disclosed herein. More specifically, various variations and modifications can be made to the components and / or layout of the subject matter arrangement within the scope of the disclosure, drawings, and claims. Besides variations and modifications to the components and / or layout, other uses will be apparent to those skilled in the art.

Claims

1. A method for verifying and preserving electronic certificates based on the fusion of OCR and micro-texture features, characterized in that, Includes the following steps: S1. The preprocessing module performs grayscale conversion and geometric correction to ensure image standardization. S2. The OCR engine performs text localization and semantic parsing, outputting keyword coordinates and semantic labels. S3. The dynamic semantic alignment module generates a risk heatmap based on the semantic labels output by the OCR engine, marking high-risk areas. After obtaining the marked high-risk areas, a risk heatmap of the corresponding area is generated. The weight of each pixel in the heatmap reflects the importance of its area. The dynamic semantic alignment module includes semantic parsing and area marking. The OCR engine first performs text detection and recognition on the document, outputting the text content and its coordinate information. The dynamic semantic alignment module marks high-risk areas according to predefined semantic rules, generates a binary mask based on the normalized heatmap, and inputs the obtained normalized heatmap and mask into the multi-scale texture enhancement hash algorithm module. S4. The multi-scale texture enhancement hash algorithm extracts the micro-texture features of the document based on the priority of the heatmap. The multi-scale texture enhancement hash algorithm includes wavelet multi-scale decomposition, frequency domain energy feature extraction, spatial domain adaptive LBP feature extraction, feature fusion and dimensionality reduction, and texture hash value generation. S5. The text hash and texture hash are fused to generate a joint hash value, which is then stored on the blockchain to complete the anti-counterfeiting registration.

2. The electronic certificate anti-counterfeiting and verification method based on OCR and micro-texture feature fusion according to claim 1, characterized in that: In step S5, the system retrieves the original combined hash value stored on the blockchain and compares it with the combined hash value. If they match, the document is determined not to have been tampered with; otherwise, a tampering alarm is triggered.

3. The electronic certificate anti-counterfeiting and verification method based on OCR and micro-texture feature fusion according to claim 1, characterized in that: In step S3, let the semantic tag of the i-th text region in the document be... Its coordinates are within a rectangle. ,in , It is the coordinate of the top left corner of the region. , These are the width and height of the region; the set R of high-risk regions is defined as: , For a predefined set of key semantic tags, the semantic tags of high-risk regions are matched: , The regular expression for identity. A list of keywords related to the signature is generated; after obtaining the marked high-risk area R, a risk heatmap of the corresponding area is generated. The weight of each pixel in the heatmap reflects the importance of its area. The formula is as follows: ,in and For the region The center coordinates, The bandwidth parameter of the Gaussian kernel controls the weight decay rate; to avoid scale differences in heatmaps from different documents, the heatmap... Normalization is performed: , for The maximum value is normalized to a range of [0, 1]. Based on the normalized heatmap, region activation is performed through threshold segmentation, and a binary mask is generated based on the normalized heatmap. Mark the areas that need to be processed first: ,in, It is a preset threshold; the obtained normalized heatmap and mask Input into the multi-scale texture enhancement hash algorithm module.

4. The electronic certificate anti-counterfeiting and verification method based on OCR and micro-texture feature fusion according to claim 1, characterized in that: In step S4, wavelet multi-scale decomposition involves performing a two-dimensional discrete wavelet transform on the input normalized image to decompose the image at different scales, as shown in the following formula: Where I is the input image and k is the number of decomposition layers. It is a low-frequency component. It selects high-frequency components, then discards all low-frequency components and retains the high-frequency components. Used for frequency domain energy feature extraction calculation; in frequency domain energy feature extraction, the directional energy distribution of high-frequency components is calculated to capture the frequency domain statistical characteristics of the texture, as shown in the following formula: , where dir is Representing three high-frequency directions, The number of effective pixels within the mask area. Represented as the Kth layer subband coefficient, combining all subband energy values, a 9-dimensional frequency domain feature vector is constructed, expressed as: ; In spatial adaptive LBP feature extraction, the inputs are the original image and the mask. Adaptive LBP calculation is performed: the improved LBP value is calculated pixel by pixel within the mask area, with the center pixel as the reference. Based on this, extract the pixel values ​​of its 3×3 neighborhood. Calculate the neighborhood mean and standard deviation , is represented as: Generate a binary pattern based on a dynamic threshold: Then, texture histograms are generated at three scales: Scale 1, i.e., radius R=1: calculate the basic LBP value and generate a 256-dimensional histogram. Scale 2, i.e., radius R=2: Bilinear interpolation is used to obtain neighboring pixels and generate... Scale 3, i.e., radius R=3: Bilinear interpolation is used to obtain neighboring pixels and generate... The three histograms are concatenated to form a 768-dimensional feature vector: Finally, the eigenvectors of the spatial domain are obtained. ; In feature fusion and dimensionality reduction, the input frequency domain features are... airspace features Feature normalization is performed by normalizing the two feature vectors using the L2 norm: The fusion weights are calculated based on the image resolution: when the resolution is <150 PPI, the frequency domain weights are increased. When the resolution is >300 PPI, increase the spatial weight. Specific weighting formula: , ,in , where res is the image resolution, and the fused features are: By compressing features using a pre-trained PCA projection matrix, a 64-dimensional compressed feature is output: ,in For PCA projection matrix; compress features in hash generation Calculate the mean of the eigenvectors: Then generate binary hashes by dimension: It converts a 64-bit binary sequence into a hexadecimal string and outputs a 64-bit texture hash value.

Images