A parking fee reduction verification method and system based on a consumption voucher and a medium

Abstract

A parking fee reduction verification method and system based on consumption vouchers, and a medium, comprising: obtaining a consumption voucher image and inputting a multi-branch deep learning forgery detection network, including at least a splicing tampering detection branch, a photocopying detection branch, a JPEG ghost detection branch and a font consistency detection branch, and fusing the outputs of the branches to obtain a comprehensive forgery probability; calculating the image naturalness score of the consumption voucher image based on the BRISQUE algorithm, calculating the forgery probability score of the consumption voucher image based on the comprehensive forgery probability, calculating the content consistency score and the context validity score of the consumption voucher image based on the text features of the consumption voucher image, and weighting and fusing the multi-dimensional scores to obtain a comprehensive score; performing a verification decision according to the comparison result of the comprehensive score and a preset threshold; if the verification is passed, calculating the amount of reduction and updating the parking fee. The present application can not only identify physical forgery traces from the image bottom layer, but also verify the content rationality from the semantic level.

Description

Technical Field

[0001] This invention relates to the field of smart parking technology, and in particular to a method, system and medium for verifying parking fee reduction based on consumption vouchers. Background Technology

[0002] In smart parking scenarios, it has become a common service model for users to apply for parking fee reductions using proof of purchase (such as shopping receipts, electronic invoices, restaurant bills, etc.). Current technologies primarily use optical character recognition (OCR) to extract key fields such as merchant name, purchase amount, and timestamp from the proof of purchase. The authenticity and validity of the proof are then determined through methods such as encoding rule verification, QR code scanning, or comparison with a merchant database.

[0003] For example, some existing solutions verify the authenticity of vouchers by parsing specific coding rules or QR code information within them. However, these methods only focus on the correctness of the data encoding and cannot detect forgery at the image level. With the widespread use of image editing software and deep learning generation technologies (such as generative adversarial networks and diffusion models), malicious users can generate visually indistinguishable fake vouchers by splicing and altering voucher images, taking screenshots, recompressing and saving them, or replacing local text areas. Traditional optical character recognition combined with encoding verification methods lack the ability to detect these image forgery techniques.

[0004] In addition, existing technical solutions usually make decisions based on a single dimension of verification results (such as whether the QR code verification passes) without comprehensively considering multiple factors, resulting in a high false positive rate and difficulty in balancing verification accuracy and user experience.

[0005] Therefore, how to effectively detect various forgery traces in consumption voucher images and achieve accurate and robust verification of parking fee reductions based on multi-dimensional comprehensive evaluation results is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method for verifying parking fee reductions based on consumption vouchers, comprising the following steps: The image of the consumption voucher is obtained and input into a multi-branch deep learning forgery detection network. The deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghost detection branch, and a font consistency detection branch. The outputs of each branch are fused to obtain the comprehensive forgery probability. The consumption voucher image is calculated based on the BRISQUE algorithm to obtain an image naturalness score. The consumption voucher image is calculated based on the comprehensive forgery probability to obtain a forgery probability score. The consumption voucher image is calculated based on the text features to obtain a content consistency score and a context validity score. The multi-dimensional scores are then weighted and fused to obtain a comprehensive score. A verification decision is made based on the comparison between the comprehensive score and the preset threshold. If the verification is successful, the amount to be reduced will be calculated and the parking fee will be updated.

[0007] Optionally, the splicing tampering detection branch uses a dual-stream architecture to process the consumption voucher image in parallel, including the following steps: The CNN branch uses ResNet-50 as the backbone network and extracts local texture features sequentially through multiple convolutional operations. The ViT branch uses ViT-B / 16 as the backbone network to extract global contextual features; The local texture features are fused with the global context features to output a tampered region location mask.

[0008] Optionally, the re-photographing detection branch is used to perform the following steps: The consumption voucher image is subjected to mean-reducing filtering to obtain a mean-reduced image; The mean-reduced image is subjected to Mihcak filtering, noise residue components are extracted and normalized to obtain the PRNU fingerprint; Calculate the Pearson correlation coefficient between the PRNU fingerprint to be tested and the reference fingerprint; The presence of re-production behavior is determined based on the comparison result between the Pearson correlation coefficient and the preset correlation coefficient threshold.

[0009] Optionally, the JPEG ghost detection branch is used to perform the following steps: The consumption voucher image is divided into blocks and discrete cosine transform according to a preset pixel block size to obtain the DCT coefficients of each pixel block; The standard JPEG luminance quantization table is used to inverse quantize each DCT coefficient, and the difference values ​​of the AC coefficients in each pixel block are extracted to construct a global histogram. Calculate the kurtosis coefficient of the global histogram; Identify the primary and secondary peaks from the global histogram, and calculate the number of primary peaks and periodicity indicators; the periodicity indicators are specifically the ratio of the intensity of the primary peak to the intensity of the secondary peak. When the kurtosis coefficient, number of main peaks, and periodicity index meet the preset abnormal conditions, it is determined that JPEG ghost artifacts exist.

[0010] Optionally, the font consistency detection branch is used to perform the following steps: Based on optical character recognition, the text regions of the consumption voucher image are located, and sub-images of each text region are cropped out; Each sub-image is input into a pre-trained font recognition model to extract font embedding vectors; Calculate the similarity between the font embedding vectors of any two text regions. When the minimum similarity is less than a preset threshold, the fonts are determined to be inconsistent.

[0011] Optionally, after fusing the outputs of each branch to obtain the overall forgery probability, the following may also be included: Calculate the Jacobian matrix of the output function of the deep learning forgery detection network with respect to each pixel of the voucher image; A gradient saliency map is generated based on the Jacobian matrix, and the statistical characteristics of the gradient saliency map are calculated. When the statistical features meet the preset adversarial perturbation judgment conditions, randomized smoothing de-perturbation processing is performed, and the processed image is re-input into the deep learning forgery detection network.

[0012] Optionally, the multiple-dimensional scoring may also include historical behavior scoring; The weighted fusion specifically involves multiplying the scores of each dimension by their respective weight coefficients and then summing the results. The weight coefficients are pre-configured or dynamically generated based on the application scenario.

[0013] Optionally, historical behavior scores are calculated using the following steps: Query the set of all consumption voucher submission records for the user ID corresponding to the consumption voucher image within a preset historical time period. Each consumption voucher submission record includes the submission time, verification result, and forgery probability value. Based on the set of consumption voucher submission records, calculate the user's activity score, pass rate score, and historical average credibility score; The sum of the activity score, pass rate score, and historical average credibility score is calculated and used as the historical behavior score.

[0014] Corresponding to the aforementioned parking fee reduction verification method based on consumption vouchers, the present invention provides a parking fee reduction verification system based on consumption vouchers, which includes: The image acquisition module is used to acquire images of consumption vouchers and input them into a multi-branch deep learning forgery detection network; A deep learning forgery detection network is used to detect the consumption voucher image and fuse the outputs of each branch to obtain a comprehensive forgery probability; the deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghosting detection branch, and a font consistency detection branch; The comprehensive scoring module is used to calculate the naturalness score of the consumption voucher image based on the BRISQUE algorithm, calculate the forgery probability score of the consumption voucher image based on the comprehensive forgery probability, calculate the content consistency score and context validity score of the consumption voucher image based on the text features of the consumption voucher image, and weight and fuse the multi-dimensional scores to obtain the comprehensive score. The verification module is used to make verification decisions based on the comparison result between the comprehensive score and the preset threshold. The parking fee reduction module is used to calculate the amount to be reduced and update the parking fee if the verification is successful.

[0015] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a parking fee reduction verification program based on a consumption voucher, wherein when the parking fee reduction verification program based on a consumption voucher is executed by a processor, the program implements the steps of the parking fee reduction verification method based on a consumption voucher as described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) By using a deep learning forgery detection network, various forgery traces such as splicing and tampering, re-photographing, JPEG compression traces, and font inconsistencies are detected at the image level. This makes up for the lack of image forgery detection capability in traditional optical character recognition combined with encoding verification methods, and effectively identifies fake consumer vouchers that are difficult to distinguish visually. At the same time, by weighting and fusing image naturalness score, forgery probability score, content consistency score, and validity score in multiple dimensions, the problem of high false judgment rate caused by single-dimensional verification result decision is avoided, and a balance is achieved between verification accuracy and user experience. Among them, the synergistic effect of multi-branch image forgery detection and multi-dimensional semantic scoring fusion not only identifies physical forgery traces at the bottom layer of the image, but also verifies the rationality of the content at the semantic level, forming a multi-layer defense system of image layer-semantic layer-decision layer, which significantly improves the robustness against complex forgery methods.

[0017] (2) By combining the local texture features extracted by the CNN branch and the global context features extracted by the ViT branch in the splicing tampering detection branch, it can capture the subtle texture anomalies at the splicing boundary and eliminate the interference of normal areas by utilizing the semantic consistency of the global layout, thereby improving the detection accuracy of splicing tampering behavior of consumer voucher images and outputting the tampering area location mask to clarify the forgery location.

[0018] (3) By using the inherent light response non-uniformity (PRNU) noise fingerprint of the camera imaging through the photo-detection branch, the device fingerprint in the consumption voucher image is extracted by mean filtering and Mihcak filtering. Combined with the comparison results of Pearson correlation coefficient and preset correlation coefficient threshold, it can effectively identify fake vouchers generated by screen photo-detection and distinguish between real physical photo-detection vouchers and photo-detection counterfeit vouchers.

[0019] (4) By performing block discrete cosine transform and inverse quantization on the consumer voucher image, the AC coefficient difference value is extracted and a global histogram is constructed. The ghost artifacts introduced by multiple JPEG compressions or different compression histories are detected by using the kurtosis coefficient and periodicity index. This can identify counterfeit vouchers that have been compressed twice or processed after cross-image splicing.

[0020] (5) By using the font consistency detection branch, the text region is located by optical character recognition and the font embedding vector is extracted for similarity comparison. This can detect the inconsistency in font style caused by the partial text replacement and alteration of key fields (such as consumption amount and merchant name) in the consumption voucher, and identify the forgery of local content that is difficult to detect visually.

[0021] (6) By calculating the Jacobian matrix of the output function with respect to each pixel of the input image and generating a gradient saliency map, adversarial perturbations that may exist in the consumption voucher image are detected; when adversarial perturbations are determined to exist, randomization smoothing and perturbation processing is performed and the processed image is re-inputted into the network to enhance the deep learning forgery detection network’s ability to defend against adversarial samples and prevent malicious users from bypassing forgery detection by adding adversarial noise.

[0022] (7) Introduce historical behavior scores into the multi-dimensional scoring and use pre-configurable or dynamically generated weight coefficients to weight and fuse the scores of each dimension, so that the verification system can adaptively adjust the importance weight of each verification dimension according to different application scenarios or users' historical credit status, further optimize the balance between false judgment rate and verification efficiency, and improve the system's flexibility and scenario adaptability.

[0023] (8) By querying the user's historical submission records and calculating the sum of the activity score, pass rate score and historical average credibility score as the historical behavior score, the user's long-term credit behavior is quantified as a reference dimension for verification decision-making, which helps to identify malicious users who submit abnormal credentials frequently, and works in conjunction with the forgery detection results at the image level to improve the accuracy of verification decision-making. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a simplified flowchart of an embodiment of the parking fee reduction verification method based on consumption vouchers of the present invention; Figure 2 This is a framework diagram of an embodiment of the parking fee reduction verification system based on consumption vouchers of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, the present invention provides a method for verifying parking fee reduction based on a consumption voucher, which includes the following steps: The image of the consumption voucher is obtained and input into a multi-branch deep learning forgery detection network. The deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghost detection branch, and a font consistency detection branch. The outputs of each branch are fused to obtain the comprehensive forgery probability. The consumption voucher image is calculated using the BRISQUE algorithm to calculate its naturalness score, and the consumption voucher image is calculated using the comprehensive forgery probability score. The consumption voucher image is also calculated using its text features to calculate its content consistency score and context validity score. The multi-dimensional scores are then weighted and fused to obtain a comprehensive score. Verification decisions are made based on the comparison between the comprehensive score and a preset threshold. Preferably, the first preset threshold is 80 and the second preset threshold is 60. If the comprehensive score is greater than or equal to 80, the verification is automatically passed. If the comprehensive score is less than 80 but greater than or equal to 60, the system enters the manual review queue. If the comprehensive score is less than 60, the verification is rejected and a security alarm is triggered. If the verification is successful, the amount to be reduced will be calculated and the parking fee will be updated.

[0027] Preferably, users upload images of their payment receipts via the parking payment system's mobile app (such as a WeChat mini-program or Alipay mini-program) or a self-service terminal. The system supports multiple image formats, including JPEG, PNG, and BMP. The payment receipt images are pre-processed and uniformly adjusted to 512×512 pixels, and their color space is normalized.

[0028] In this embodiment, the models entering the manual review queue undergo secondary verification by reviewers, and the verification results are fed back to the model training pipeline for incremental learning.

[0029] Preferably, when calculating the amount to be reduced and updating the parking fee, a verification record is also generated and stored in the blockchain distributed ledger.

[0030] Furthermore, the method also includes notifying users of the verification results and details of parking fee reduction via mobile push notifications, SMS messages, or system messages.

[0031] This invention utilizes a deep learning-based forgery detection network to detect various forgery traces at the image level, including splicing and tampering, re-photographing, JPEG compression marks, and font inconsistencies. This overcomes the shortcomings of traditional optical character recognition combined with encoding verification methods, which lack image forgery detection capabilities, effectively identifying visually indistinguishable fake receipts. Furthermore, by weighting and fusing image naturalness scores, forgery probability scores, content consistency scores, and validity scores across multiple dimensions, it avoids the high false positive rate caused by single-dimensional verification results, achieving a balance between verification accuracy and user experience. The synergistic effect of multi-branch image forgery detection and multi-dimensional semantic scoring not only identifies physical forgery traces at the image level but also verifies content legitimacy at the semantic level, forming a multi-layered defense system from the image layer to the semantic layer to the decision layer, significantly improving robustness against complex forgery methods.

[0032] In this embodiment, the tampering detection branch uses a dual-stream architecture to process the consumption voucher image in parallel, including the following steps: The CNN branch uses ResNet-50 as the backbone network (pre-trained on the ImageNet dataset and fine-tuned with consumer voucher domain data). It extracts local texture features sequentially through multiple convolutional operations. Specifically, the first convolutional layer (3×3 kernel, stride=1) extracts edge gradient direction and intensity features (used to detect pixel-level discontinuities at splicing boundaries), the second to fourth residual blocks extract local color distribution statistics (RGB three-channel histogram moment features, used to detect color temperature deviations in different source regions), and the feature map after pooling in the fifth layer extracts local binary pattern (LBP) texture roughness features (used to distinguish Moire textures generated by screen re-photographing from real printed textures). The ViT branch uses ViT-B / 16 as the backbone network (patch size = 16×16 pixels) to extract global context features. Preferably, the 512×512 input image is divided into a sequence of 32×32=1024 image patches. After spatial location information is injected through position encoding, the following global context features are extracted through a 12-layer Transformer self-attention mechanism: long-range semantic dependencies between patches (used to detect the logical consistency of content across regions, such as whether the merchant logo matches the background scene), spatial layout features of the overall composition (used to detect abnormal cropping or unnatural perspective transformation), and implicit representation of global illumination consistency (used to detect inconsistent lighting directions in multi-source splicing regions). After fusing local texture features with global context features, the tampered region localization mask is output. Preferably, the two branch features are reduced to 256 channels by 1×1 convolution and then concatenated in the channel dimension. After passing through two fully connected layers (256→128→64) + ReLU activation, the pixel-by-pixel tampered region localization mask (size is 128×128, and each pixel value is a forgery probability value of 0-1) is output.

[0033] This invention employs a dual-stream collaborative approach: a CNN branch extracts local texture features while a ViT branch extracts global contextual features. This approach captures subtle texture anomalies at splicing boundaries and eliminates interference from normal regions by utilizing the semantic consistency of the global layout. This improves the detection accuracy of splicing tampering of consumer voucher images and outputs a tampering region location mask to clearly identify the location of the forgery.

[0034] In this embodiment, the re-photographing detection branch is used to perform the following steps: Perform mean-reduction filtering on the consumption voucher image (using a size of...) A sliding window of pixels is used to calculate the local mean at each location, resulting in a mean-free image; The mean-reduced image is subjected to Mihcak filtering to extract residual noise components and normalize them (preferably L2 norm normalization) to obtain the PRNU (Photo Response Non-Uniformity) fingerprint; wherein, the Mihcak filter parameters are set as follows: noise variance (where n represents noise), signal variance (Where s represents the signal) is adaptively estimated based on a local window; The Pearson correlation coefficient between the PRNU fingerprint to be tested and the reference fingerprint is calculated using the following formula: Where cov is the covariance operator. Fingerprint of PRNU to be tested standard deviation For reference fingerprints The standard deviation of the reference fingerprint (where z represents the index number of the reference fingerprint), preferably, is the PRNU fingerprint in a known real device PRNU library; in actual calculations, Fast Fourier Transform (FFT) is used to perform cross-correlation in the frequency domain to accelerate the operation and reduce computational complexity. The order of magnitude (where O represents the asymptotic upper bound of the algorithm complexity), where N represents the total number of reference fingerprints in the reference fingerprint database, and the maximum value of the Pearson correlation coefficient between the PRNU fingerprint to be tested and all reference fingerprints is taken as the final judgment criterion. The existence of re-production behavior is determined based on the comparison between the Pearson correlation coefficient and a preset correlation coefficient threshold (preferably, the maximum correlation coefficient threshold is 0.35 and the minimum correlation coefficient threshold is 0.15); where, when When it is determined to be direct filming with a genuine device (i.e., no copying); when At that time, it was determined to be a suspected act of reposting (further manual review is required); when When this occurs, it is determined that there is obvious copying behavior (for example, taking a screenshot of the screen or taking a second photo of a printed document, which results in the PRNU fingerprint being severely damaged or superimposed with the PRNU noise of the display device itself).

[0035] This invention utilizes the inherent light response non-uniformity (PRNU) noise fingerprint of camera imaging through a photo-detection branch. It extracts the device fingerprint from the voucher image through mean-reduction filtering and Mihcak filtering. Combined with the comparison results of Pearson correlation coefficient and preset correlation coefficient threshold, it can effectively identify fake vouchers generated by screen photo-detection and distinguish between real physical photographed vouchers and photo-detected forged vouchers.

[0036] In this embodiment, the JPEG ghost detection branch is used to perform the following steps: The consumption voucher image is arranged according to a preset pixel block size (preferably). (If the image width and height are not integer multiples of 8, zeros are padded at the bottom right.) A block-based discrete cosine transform (DCT) is performed to obtain the DCT coefficients for each pixel block. The specific formula for the block-based discrete cosine transform is as follows: ;in, These are the spatial coordinates of the pixels in the image. This represents the pixel brightness value at the corresponding coordinates. Frequency domain coordinate index and , For DCT coefficients, when At this time, F(0,0) is the DC component, and the remaining 63 DCT coefficients are the AC components; Using a standard JPEG luminance quantization table Each DCT coefficient is dequantized, and the difference values ​​of the AC coefficients in each pixel block are extracted (the difference value of the AC coefficients refers to the difference between the dequantized AC coefficients of two adjacent image blocks at the same frequency domain position). All difference values ​​are then aggregated to construct a global histogram. This is the frequency domain coordinate index of the brightness quantization table mentioned above; Let the sample set of this histogram be . ; Calculate the kurtosis coefficient of the global histogram; ;in, This represents the expected value (i.e., the average of all data within the parentheses). This represents the difference in the coefficients of exchange. express The mean (average) of. express The standard deviation (reflecting the dispersion of the data); Identify primary and secondary peaks from the global histogram, and calculate the number of primary peaks and periodicity index. The primary peak refers to the interval with the highest frequency (number of occurrences) in the global histogram, reflecting the range of most frequently occurring AC coefficient difference values. The secondary peak refers to the interval with the highest frequency other than the primary peak. The intensity of the primary peak is defined as the frequency value corresponding to the primary peak, and the intensity of the secondary peak is defined as the frequency value corresponding to the secondary peak. The periodicity index is specifically the ratio of the intensity of the primary peak to the intensity of the secondary peak; a ratio greater than 2.5 indicates a significant periodic quantization mismatch, suggesting JPEG double compression. When preset abnormal conditions are met, JPEG ghost artifacts are determined to exist. Preferably, the preset abnormal conditions are as follows: The first indicator: When Furthermore, when the number of main peaks is ≥3, it is determined that there are significant traces of double compression (the global histogram of a normal image after single compression is approximately a single-peak Laplace shape, and the Kurtosis is close to 0). The second indicator: When the periodicity index is greater than 2.5, it is determined that there is a periodic quantization mismatch (indicating that the quality factor difference between the two JPEG compressions exceeds 10). If both of the above criteria are met, it is determined to be a high-confidence JPEG ghost artifact (i.e., it is determined that a JPEG ghost artifact exists), and the JPEG ghost detection result flag G=1 is output; otherwise, G=0.

[0037] This invention utilizes a JPEG ghost detection branch to perform block-based discrete cosine transform and inverse quantization on the voucher image, extracts the AC coefficient difference value, constructs a global histogram, and uses kurtosis coefficient and periodicity index to detect ghost artifacts introduced by multiple JPEG compressions or different compression histories. It can identify counterfeit vouchers that have been compressed twice or processed after cross-image stitching.

[0038] In this embodiment, the font consistency detection branch is used to perform the following steps: Based on optical character recognition, locate each text area of the consumption voucher image (preferably extract text features based on the CRNN architecture fused with a Transformer encoder, and combine an attention mechanism to enhance the recognition ability for blurred, skewed, and low-resolution texts), and crop out sub-images of each text area (preferably pixels); the text features include merchant name features, consumption amount features, voucher timestamp features, voucher number features, and preferential type features; among them, the merchant name features are obtained by sequence decoding to output the full or abbreviated merchant name string printed on the voucher, supporting mixed recognition of Chinese (UTF-8 / GBK encoding) and English characters; the consumption amount features are obtained by recognizing the amount numbers and currency symbols (such as "¥", "￥", "yuan", etc.), and output in a standardized numerical format (retaining 2 decimal places); the voucher timestamp features are obtained by recognizing the date and time information (in the format of YYYY-MM-DD HH:MM or YYYY year MM month DD day), and converted into the Unix timestamp standard format for subsequent time series verification; the voucher number features are used to recognize unique identifier strings such as serial numbers, order numbers, and QR code encodings; the preferential type features are obtained by recognizing the text descriptions of discount types (such as classification labels like "full reduction", "discount", "deduction voucher", "gift voucher", etc.). The above structured field information is encapsulated and output in JSON format, with each field accompanied by a confidence score (a floating-point number in the range of 0-1) and character-level bounding box coordinates (left, top, right, width, height); Input each sub-image into a pre-trained font recognition model (specifically, FontRecNet is used as the basic architecture for pre-training, and FontRecNet is a font classification model based on a multi-scale residual network) to extract font embedding vectors; preferably, the pre-trained dataset includes the coverage of the following three types of fonts: the first type is common Chinese fonts, about 500 kinds (Song typeface / Black typeface / Kai typeface / FangSong typeface / Microsoft YaHei / Siyuan series / Fangzheng series / Huawen series, etc. and their variants), covering the most common print font types in parking fee consumption vouchers; the second type is common English fonts and digital fonts, about 200 kinds (Arial / Times New Roman / Courier / Helvetica / Calibri and various monospace typewriter fonts), used to recognize special font styles for amount numbers and numbers; the third type is a handwritten font sample set (from the public dataset CASIA-HWDB and custom-collected handwritten annotation data of consumption vouchers), used to recognize the handwritten style of manually filled vouchers; Calculate the similarity between the font embedding vectors of any two text regions. When the minimum similarity is less than the preset threshold (preferably 0.72), it is determined that the fonts are inconsistent (there may be some regions that have been replaced by PS or spliced ​​from different sources). Output the font consistency detection result flag C=1, otherwise C=0.

[0039] In this embodiment, the FontRecNet model uses a ResNet-34 pre-trained on ImageNet as the backbone network (with the last fully connected classification layer removed), and replaces it with a projection head (Linear-ReLU-Linear-Dropout structure, hidden layer dimension 512, output dimension 256) to map image patches into 256-dimensional font embedding vectors. .

[0040] This invention uses a font consistency detection branch to locate text regions through optical character recognition and extract font embedding vectors for similarity comparison. It can detect inconsistencies in font style caused by partial text replacement and tampering of key fields (such as consumption amount and merchant name) in consumption vouchers, and identify partial content forgery that is difficult to detect visually.

[0041] In this embodiment, after fusing the outputs of each branch to obtain the comprehensive forgery probability, the method further includes: calculating the Jacobian matrix of the output function of the deep learning forgery detection network with respect to each pixel of the voucher image; generating a gradient saliency map based on the Jacobian matrix and calculating the statistical features of the gradient saliency map; when the statistical features meet the preset adversarial perturbation judgment conditions, performing randomized smoothing and perturbation removal processing, and re-inputting the processed image into the deep learning forgery detection network. The specific data processing steps are as follows: 1) Model Input and Output Definition Let the overall output function of the multi-branch deep learning forgery detection network be: ,in, Image of the receipt. H is the image height, W is the image width, preferably H=W=512, and 3 represents the three color channels of the image (red, green, blue). This is the set of trainable parameters for the network; To comprehensively consider the probability of forgery, and ; 2) Calculate the Jacobian matrix Jacobian matrix Defined as model output ( ) for input image ( The first-order partial derivative of ) ; The dimension is Each of its elements Indicates the pixel position of the input image. The effect of minute changes in the color channel index c on the model output is considered, where c ∈ {0, 1, 2}, and c=0 corresponds to the R channel, c=1 to the G channel, and c=2 to the B channel. The calculation is performed using automatic differentiation and backpropagation: forward propagation calculation... After the value is obtained, call it directly. The backpropagation operator obtains the complete Jacobian matrix in one step, without needing to perform any further operations. Calculating the partial derivative for each pixel individually, the computational complexity is O(n). A single forward + backward propagation; 3) Generate gradient saliency map Calculate the L2 norm of the Jacobian matrix along the channel dimension to generate a gradient saliency map. The value of each pixel reflects the strength of the influence of that location on the model's decision. 4) Calculation of global statistical characteristics of gradient significance map S The global statistical features of the gradient saliency map S include: Peak density This metric represents the proportion of pixels with abnormally high response in the gradient saliency map; among which, This represents the mean of the gradient saliency plot. The standard deviation of the gradient significance plot; Spatial concentration This indicator reflects the degree of concentration of significant regions (the smaller the value, the more concentrated it is in a local area), among which, The variance of the gradient significance map is represented. This represents the expectation of the gradient saliency map; Edge response ratio ;in, This represents the mean of the saliency values ​​of the gradient saliency map in the image edge region. This represents the mean saliency value of the gradient saliency map in regions other than the image edge (center); preferably, the edge regions can be determined by edge detection operators such as Canny or Sobel. 5) Comprehensive judgment criteria and anti-disturbance processing The preset anti-disturbance judgment conditions are: (This means there are a lot of pixels with abnormally high response times.) (This implies that the significant regional distribution is highly concentrated, rather than dispersed), and (This means that the significant areas are mainly concentrated in the background / smooth areas rather than the object edges). When the above-mentioned preset anti-perturbation judgment condition is met, it is determined that there is an adversarial perturbation (i.e., the adversarial perturbation is usually uniformly distributed across the entire domain rather than concentrated on the object outline). At this time, randomized smoothing deperturbation processing is performed, and the processed image is re-inputted into the deep learning forgery detection network.

[0042] Preferably, the randomized smoothing and perturbation processing includes: a) adding Gaussian random noise to X; b) performing denoising processing through a denoising network; c) repeating a)-b) K times; d) taking the average of the K denoised images as the final output image (i.e., the processed image). 6) Model Training During the training phase, adversarial training strategies FGSM (Fast Gradient Signed Method) and PGD (Projected Gradient Descent) are integrated into the training process of the deep learning forgery detection network. Joint loss function. Defined as: ;in, Represents cross-entropy loss, This represents adversarial robustness loss (used to enhance the model's ability to defend against adversarial examples). This represents the balance coefficient.

[0043] This invention detects adversarial perturbations in consumption voucher images by calculating the Jacobian matrix of the output function with respect to each pixel of the input image and generating a gradient saliency map. When adversarial perturbations are determined to exist, randomized smoothing and perturbation removal processing is performed, and the processed image is re-inputted into the network. This enhances the defense capability of the deep learning forgery detection network against adversarial examples and prevents malicious users from bypassing forgery detection by adding adversarial noise.

[0044] In this embodiment, the naturalness score of the consumption voucher image is calculated based on the BRISQUE algorithm, including: First, the consumption voucher image is converted to grayscale space and normalized by mean subtraction (MSCN) to obtain the normalized coefficient image; Shape and scale parameters are extracted by fitting a generalized Gaussian distribution (GGD) model to the MSCN coefficients. At the same time, asymmetric generalized Gaussian distribution (AGGD) parameters are extracted in four directions: horizontal, vertical, diagonal, and anti-diagonal, resulting in a total of 18-dimensional natural scene statistics (NSS) feature vectors. The 18-dimensional Natural Scene Statistics (NSS) feature vector is input into a pre-trained SVR (Support Vector Regression) model to predict the raw BRISQE score. (The typical range is 0-100, the lower the better); Finally, the BRISQE raw score was calculated. A linear mapping to the 0-20 score range is used as the image naturalness score, and the specific formula is as follows: ; when When D1 scores a maximum of 20 points (for high-quality images), At that time, D1 is set to 0 (extremely low quality image), and the middle segment is linearly interpolated.

[0045] In this embodiment, the calculation of the forgery probability score of the consumption voucher image based on the comprehensive forgery probability specifically includes the following steps: 1) Calculation of branch forgery probability The multi-branch deep learning forgery detection network outputs forgery probability values ​​for four branches, as follows: , which represents the forgery probability value output by the splicing tampering detection branch, is calculated as follows: based on the 128×128 tampering mask, the average value of the mask pixel values ​​is taken; This represents the forgery probability value output from the re-image detection branch. It is calculated based on the Pearson correlation coefficient between the PRNU fingerprint to be tested and the reference fingerprint, and then... Mapped to obtain; This represents the forgery probability value output by the JPEG ghost detection branch. It is calculated as a weighted sum based on histogram anomaly indicators, using the following formula: ,in, , All are 0 / 1 indicator functions (1 is taken when the corresponding abnormal indicator meets the preset threshold, and 0 is taken otherwise); This represents the forgery probability value output from the font consistency detection branch. It is calculated by calculating the cosine similarity of font features between all compared character pairs and taking the minimum value. ;based on Obtained through linear mapping ; 2) Branch fusion and overall forgery probability The probability values ​​of the four branches are weighted and aggregated through the attention fusion module to obtain the comprehensive forgery probability. : ; in, This represents the forgery probability value of the output of the i-th branch (corresponding to the above). , , , ), Let the attention weights corresponding to the i-th branch satisfy the following condition: ,and ; It is dynamically generated by a lightweight MLP (the input is the global average pooling features of the image, and the output is a 4-dimensional softmax vector); 3) Final score mapping Based on the overall probability of forgery Calculate the final forgery probability score The formula is as follows; That is, the lower the overall forgery probability, the higher the D3 score (indicating a more credible image); specifically, a score of 30 corresponds to complete credibility. A score of 0 corresponds to a highly suspicious result. ).

[0046] In this embodiment, the content consistency score and context validity score (0-25 points) of the consumption voucher image are calculated based on the text features of the consumption voucher image, including: Based on the text features of the receipt image, the degree of consistency between key fields such as merchant name, amount, and time and the OCR recognition results, as well as the business rationality, are verified. The specific calculation formula is as follows: ; Where, when k=1, The merchant name matching score (0-8 points) is calculated based on the Levenshtein edit distance between the merchant name recognized by OCR and the whitelist of partner merchants. Specifically, a full score of 8 points is awarded when the Levenshtein edit distance is ≤2; a score of 5 points is awarded when the Levenshtein edit distance is ≤5; and 0 points are awarded in all other cases. express The corresponding weight is preferably 0.32; When k=2 This is represented by a reasonableness score (0-7 points), determined based on the transaction amount and the historical transaction amount for the corresponding merchant type; specifically, when the transaction amount falls within a certain percentage of the historical transaction amount for the corresponding merchant type... interval ( This represents the average of historical transaction amounts. A perfect score of 7 points is awarded if the transaction amount falls within the standard deviation of the historical transaction amount for the corresponding merchant type. Within, or when the consumption amount falls within the historical transaction amount of the corresponding merchant type. Within the specified range, 4 points are awarded; otherwise, 0 points are awarded. express The corresponding weight is preferably 0.28; When k=3 The score is expressed as a time validity score (0-5 points), determined based on the time interval between the voucher's timestamp and the current time. Specifically, a score of 5 is awarded when the time interval is ≤72 hours; 3 points are awarded when the time interval is >72 hours but ≤168 hours; and 0 points are awarded when the time interval is >168 hours. express The corresponding weight is preferably 0.2; When k=4 The score is represented as a format compliance score (0-5 points), determined based on the voucher number format. Specifically, 3 points are awarded when the voucher number format conforms to a preset regular expression (e.g., a combination of 8 or more consecutive alphanumeric characters); 2 points are awarded when the discount type corresponding to the voucher number belongs to a preset valid category; and 0 points are awarded for all other cases. express The corresponding weight is preferably 0.2.

[0047] The content consistency score and contextual validity score D4 (0-15 points) of the consumption voucher image are calculated based on the textual features of the image. The specific calculation formula is as follows: ; in, The time consistency factor (0-5 points) is based on the timestamp of the voucher image. The corresponding user's vehicle entry time (From the barrier gate system record) and current verification time Determined; specifically, when (That is, the voucher time is between the entry time and the current time) and Hour, 5 points; when ,but Hour, 2 points; when (i.e., the voucher timestamp is earlier than the vehicle entry time), which is directly judged as unreasonable and scores 0 points; The geographic consistency factor (0-5 points) is based on the location coordinates when the consumption voucher image is uploaded. and parking lot registration coordinates Calculate the Haversine spherical distance Determined; preferably, the location coordinates when the consumption voucher image is uploaded. It can be obtained through mobile GPS positioning or WiFi fingerprint positioning; specifically, when (When within the parking lot area or adjacent to the commercial area), 5 points are awarded; when At that time, 3 points were scored; when At this time, you will get 0 points (which means that there is a high risk of uploading the consumption voucher image from a different location). The business logic consistency factor (0-5 points) is determined based on the matching degree between the consumption type corresponding to the consumption voucher image and the business formats around the parking lot (the matching degree is defined as whether the consumption type on the consumption voucher image belongs to the list of cooperative merchant types within a 3km radius of the parking lot). Specifically, if the consumption type corresponding to the consumption voucher image belongs to the list of cooperative merchant types, 5 points are awarded; if the consumption type corresponding to the consumption voucher image does not belong to the list of cooperative merchant types, but belongs to general consumption (such as e-commerce platform vouchers, online top-ups, etc.), 2 points are awarded; other cases (such as long-distance cross-regional consumption) receive 0 points.

[0048] In this embodiment, the multi-dimensional scoring also includes historical behavior scoring; The weighted fusion process involves multiplying the scores of each dimension by their respective weight coefficients and then summing the results. These weight coefficients are either pre-configured or dynamically generated based on the application scenario. Preferably, the weight coefficients are pre-configured based on the application scenario, or automatically and dynamically generated by the system based on the features of the current consumption voucher image.

[0049] This invention introduces historical behavior scoring into multi-dimensional scoring and uses pre-configurable or dynamically generated weight coefficients to weight and fuse the scores of each dimension. This enables the verification system to adaptively adjust the importance weight of each verification dimension according to different application scenarios or users' historical credit status, further optimizing the balance between false positive rate and verification efficiency, and improving the system's flexibility and scenario adaptability.

[0050] In this embodiment, the historical behavior score is calculated through the following steps: Query the set of all consumption voucher submission records (up to the most recent 50 records) for the user ID corresponding to the consumption voucher image within a preset historical period (preferably the past 90 days). Each consumption voucher submission record includes the submission time, verification result, and forgery probability value. Calculate the user's activity score, pass rate score, and historical average credibility score based on the set of consumption voucher submission records; Calculate the sum of activity score, pass rate score, and historical average credibility score, and use it as the historical behavior score D5.

[0051] Preferred, Activity score (0-3 points), based on the number of times the consumption voucher was submitted. Determined; specifically, when At that time, 3 points were scored; when When, 2 points are scored; when When M=0, you get 1 point; when M=0 (new user's initial score), you get 0 points. The pass rate score (0-4 points) is based on historical validation. (That is, the number of consumption voucher submission records with a verification result of PASS / the number of consumption voucher submission records) is used to determine the value. Specifically, when At that time, he scored 4 points; when At that time, 3 points were scored; when When, 2 points are scored; when At that time, 0 points were awarded; This represents the j-th consumption voucher submission record by the user within a preset historical time period, and ; This represents the verification result corresponding to the j-th consumption voucher submission record within a preset historical time period. ; The historical average credibility score (0-3 points) is used to calculate the user's historical average forgery probability. Sure: , ; in, This represents a set (M elements) consisting of the combined forgery probabilities of all consumption voucher submission records of the user within a preset historical time period. express For all in the set The arithmetic mean of the values, i.e., the average historical forgery probability of this user within a preset historical period, is obtained through... Linearly map Ravg to scores of 0-3.

[0052] Final historical behavior score D5 D5 has a maximum score of 10 (representing long-term trustworthy users) and a minimum score of 0 (representing high-risk users).

[0053] This invention queries a user's historical submission records and calculates the sum of activity score, pass rate score, and historical average credibility score as a historical behavior score. It quantifies the user's long-term credit behavior as a reference dimension for verification decisions, assists in identifying malicious users who frequently submit abnormal credentials, and works in conjunction with image-level forgery detection results to improve the accuracy of verification decisions.

[0054] like Figure 2 As shown, the present invention also provides a parking fee reduction verification system based on consumption vouchers, which includes: Image acquisition module 10 is used to acquire the image of the consumption voucher and input it into the multi-branch deep learning counterfeit detection network 20; A deep learning forgery detection network 20 is used to detect forgery images of consumer vouchers and fuse the outputs of each branch to obtain a comprehensive forgery probability. The deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghosting detection branch, and a font consistency detection branch. The comprehensive score calculation module 30 is used to calculate the naturalness score of the consumption voucher image based on the BRISQUE algorithm, calculate the forgery probability score of the consumption voucher image based on the comprehensive forgery probability, calculate the content consistency score and context validity score of the consumption voucher image based on the text features of the consumption voucher image, and weight and fuse the multi-dimensional scores to obtain the comprehensive score. The verification module 40 is used to make verification decisions based on the comparison results between the comprehensive score and the preset threshold. The parking fee reduction module 50 is used to calculate the amount to be reduced and update the parking fee if the verification passes.

[0055] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The method for verifying parking fee reduction based on consumption vouchers is shown. The computer-readable storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0056] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments and storage medium embodiments, since they are basically similar to method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0057] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for verifying parking fee reduction based on consumption vouchers, characterized in that, Includes the following steps: The image of the consumption voucher is obtained and input into a multi-branch deep learning forgery detection network. The deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghost detection branch, and a font consistency detection branch. The outputs of each branch are fused to obtain the comprehensive forgery probability. The consumption voucher image is calculated based on the BRISQUE algorithm to obtain an image naturalness score. The consumption voucher image is calculated based on the comprehensive forgery probability to obtain a forgery probability score. The consumption voucher image is calculated based on the text features to obtain a content consistency score and a context validity score. The multi-dimensional scores are then weighted and fused to obtain a comprehensive score. A verification decision is made based on the comparison between the comprehensive score and the preset threshold. If the verification is successful, the amount to be reduced will be calculated and the parking fee will be updated.

2. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, The splicing tampering detection branch uses a dual-stream architecture to process the consumption voucher image in parallel, including the following steps: The CNN branch uses ResNet-50 as the backbone network and extracts local texture features sequentially through multiple convolutional operations. The ViT branch uses ViT-B / 16 as the backbone network to extract global contextual features; The local texture features are fused with the global context features to output a tampered region location mask.

3. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, The re-photograph detection branch is used to perform the following steps: The consumption voucher image is subjected to mean-reducing filtering to obtain a mean-reduced image; The mean-reduced image is subjected to Mihcak filtering, noise residue components are extracted and normalized to obtain the PRNU fingerprint; Calculate the Pearson correlation coefficient between the PRNU fingerprint to be tested and the reference fingerprint; The presence of re-production behavior is determined based on the comparison result between the Pearson correlation coefficient and the preset correlation coefficient threshold.

4. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, The JPEG ghost detection branch is used to perform the following steps: The consumption voucher image is divided into blocks and discrete cosine transform according to a preset pixel block size to obtain the DCT coefficients of each pixel block; The standard JPEG luminance quantization table is used to inverse quantize each DCT coefficient, and the difference values ​​of the AC coefficients in each pixel block are extracted to construct a global histogram. Calculate the kurtosis coefficient of the global histogram; Identify the primary and secondary peaks from the global histogram, and calculate the number of primary peaks and periodicity indicators; the periodicity indicators are specifically the ratio of the intensity of the primary peak to the intensity of the secondary peak. When the kurtosis coefficient, number of main peaks, and periodicity index meet the preset abnormal conditions, it is determined that JPEG ghost artifacts exist.

5. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, The font consistency detection branch is used to perform the following steps: Based on optical character recognition, the text regions of the consumption voucher image are located, and sub-images of each text region are cropped out; Each sub-image is input into a pre-trained font recognition model to extract font embedding vectors; Calculate the similarity between the font embedding vectors of any two text regions. When the minimum similarity is less than a preset threshold, the fonts are determined to be inconsistent.

6. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, After fusing the outputs of each branch to obtain the overall forgery probability, it also includes: Calculate the Jacobian matrix of the output function of the deep learning forgery detection network with respect to each pixel of the voucher image; A gradient saliency map is generated based on the Jacobian matrix, and the statistical characteristics of the gradient saliency map are calculated. When the statistical features meet the preset adversarial perturbation judgment conditions, randomized smoothing de-perturbation processing is performed, and the processed image is re-input into the deep learning forgery detection network.

7. The parking fee reduction verification method based on consumption vouchers according to claim 1, characterized in that, The multi-dimensional scoring also includes historical behavior scoring; The weighted fusion specifically involves multiplying the scores of each dimension by their respective weight coefficients and then summing the results. The weight coefficients are pre-configured or dynamically generated based on the application scenario.

8. The parking fee reduction verification method based on consumption vouchers according to claim 7, characterized in that, Historical behavior scores are calculated through the following steps: Query the set of all consumption voucher submission records for the user ID corresponding to the consumption voucher image within a preset historical time period. Each consumption voucher submission record includes the submission time, verification result, and forgery probability value. Based on the set of consumption voucher submission records, calculate the user's activity score, pass rate score, and historical average credibility score; The sum of the activity score, pass rate score, and historical average credibility score is calculated and used as the historical behavior score.

9. A parking fee reduction verification system based on consumption vouchers, characterized in that, include: The image acquisition module is used to acquire images of consumption vouchers and input them into a multi-branch deep learning forgery detection network; A deep learning forgery detection network is used to detect the consumption voucher image and fuse the outputs of each branch to obtain a comprehensive forgery probability; the deep learning forgery detection network includes at least a splicing tampering detection branch, a re-photographing detection branch, a JPEG ghosting detection branch, and a font consistency detection branch; The comprehensive scoring module is used to calculate the naturalness score of the consumption voucher image based on the BRISQUE algorithm, calculate the forgery probability score of the consumption voucher image based on the comprehensive forgery probability, calculate the content consistency score and context validity score of the consumption voucher image based on the text features of the consumption voucher image, and weight and fuse the multi-dimensional scores to obtain the comprehensive score. The verification module is used to make verification decisions based on the comparison result between the comprehensive score and the preset threshold. The parking fee reduction module is used to calculate the amount to be reduced and update the parking fee if the verification is successful.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a parking fee reduction verification program based on a consumption voucher, which, when executed by a processor, implements the steps of the parking fee reduction verification method based on a consumption voucher as described in any one of claims 1 to 8.

Images