Electronic ticket processing method based on image recognition

Abstract

The application discloses an image recognition-based electronic bill processing method, which extracts spatial semantics and frequency domain texture features of bills in parallel through frequency-space double flow, introduces local texture inconsistency analysis to generate an abnormal activation map, thereby guiding cross-domain attention fusion, and realizing deep analysis of bill underlying data structure. In this way, the traditional global uniformity assumption is abandoned, the false positive rate caused by complex backgrounds such as seals and texts is greatly reduced, and the detection robustness and accuracy of the system in a real business scenario are significantly improved.

Description

Technical Field

[0001] This application relates to the field of intelligent analysis, and more specifically, to an electronic ticket processing method based on image recognition. Background Technology

[0002] With the widespread application of electronic invoices in enterprise expense control and reimbursement scenarios, ensuring their authenticity and integrity has become a critical requirement. Therefore, developing efficient and accurate automated methods for processing electronic invoices is particularly important, given the potential risks of forgery and tampering with invoice images.

[0003] Existing electronic ticket processing solutions largely rely on traditional convolutional neural networks (such as ResNet and VGG) for feature extraction in the RGB spatial domain, combined with fixed high-pass filters for frequency domain analysis. However, these solutions face significant technical bottlenecks when dealing with complex forgery scenarios. First, the multiple downsampling and pooling operations during spatial domain feature extraction often filter out microscopic high-frequency noise (such as pixel grid effects) introduced by screen re-photographing or interpolation traces left by digital tampering, preventing the model from capturing subtle forgery traces. Simultaneously, existing solutions typically couple the semantic content of the ticket with the carrier texture features, making the model highly susceptible to being misled by clear text content and ignoring anomalies in the background carrier. Furthermore, traditional techniques often rely on the assumption of global uniformity, ignoring the fact that electronic tickets contain multiple legitimate texture modalities such as seals, table lines, and text. Forcibly calculating the global background distribution can lead to misjudgments of legitimate complex areas due to significant differences from the average distribution, resulting in a high false positive rate. In summary, existing electronic ticket processing methods generally face technical bottlenecks such as loss of spatial domain details and interference from multiple feature coupling when dealing with highly concealed screen photography and digital tampering, which restricts further improvement in the identification effect.

[0004] Therefore, an optimized method for processing electronic tickets based on image recognition is needed. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an electronic ticket processing method based on image recognition.

[0006] According to one aspect of this application, an electronic ticket processing method based on image recognition is provided, comprising: Acquire the image data of the electronic invoice to be processed; The electronic ticket image data to be processed is preprocessed and decomposed in the frequency domain to obtain the normalized ticket spatial domain image and the initial ticket frequency domain components. Parallel feature extraction of frequency and space streams is performed on the normalized spatial domain image of the bill and the initial frequency domain component of the bill to obtain the spatial domain feature vector, the frequency domain feature vector, the intermediate features of the spatial stream, and the intermediate features of the frequency stream. Local texture inconsistency analysis is performed on the intermediate features of spatial flow and the intermediate features of frequency domain flow to obtain local anomalous activation maps; Based on the local anomaly activation map, cross-domain attention fusion is performed on the spatial domain feature vector and the frequency domain feature vector of the bill to obtain the bill fusion identification feature vector. The fused identification feature vector of the invoice is used for identification and classification to obtain the invoice status category.

[0007] Compared with existing technologies, this application provides an image-based electronic invoice processing method that extracts spatial semantics and frequency domain texture features of invoices through parallel extraction of frequency and spatial dual-stream data. It also introduces local texture inconsistency analysis to generate anomaly activation maps, thereby guiding cross-domain attention fusion and achieving deep analysis of the underlying data structure of the invoices. This approach abandons the traditional assumption of global uniformity, significantly reducing the false alarm rate caused by complex backgrounds such as seals and text, thus significantly improving the system's robustness and accuracy in real-world business scenarios. Attached Figure Description

[0008] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 This is a flowchart of an image-based electronic ticket processing method according to an embodiment of this application; Figure 2 This is a schematic diagram of the data flow of an image-based electronic ticket processing method according to an embodiment of this application. Detailed Implementation

[0010] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0011] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0012] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0013] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0014] The technical solution of this application proposes an electronic ticket processing method based on image recognition. Figure 1 This is a flowchart of an image-based electronic ticket processing method according to an embodiment of this application. Figure 2 This is a system architecture diagram of an image-based electronic ticket processing method according to an embodiment of this application. Figure 1 and Figure 2 As shown, the image-based electronic ticket processing method according to an embodiment of this application includes the following steps: S1, acquiring electronic ticket image data to be processed; S2, preprocessing and frequency domain decomposing the electronic ticket image data to be processed to obtain a normalized ticket spatial domain image and initial ticket frequency domain components; S3, performing parallel feature extraction of frequency and space streams on the normalized ticket spatial domain image and initial ticket frequency domain components to obtain a ticket spatial domain feature vector, a ticket frequency domain feature vector, intermediate features of the spatial stream, and intermediate features of the frequency stream; S4, performing local texture inconsistency analysis on the intermediate features of the spatial stream and the intermediate features of the frequency stream to obtain a local abnormal activation map; S5, based on the local abnormal activation map, performing cross-domain attention fusion on the ticket spatial domain feature vector and the ticket frequency domain feature vector to obtain a ticket fusion identification feature vector; S6, performing identification and classification on the ticket fusion identification feature vector to obtain the ticket status category.

[0015] Specifically, S1 involves acquiring the electronic invoice image data to be processed. With the deepening development of enterprise financial and tax digitalization, electronic invoices, such as VAT invoices, have become the core carrier of commercial transactions. However, this has also given rise to highly concealed invoice forgery and tampering, especially the increasingly common practice of modifying key information using digital image processing software and then taking high-quality screen re-images. This forgery method is extremely difficult to distinguish with the human eye and traditional algorithms. To ensure financial and tax security and to deeply analyze the underlying features of images to identify illegal tampering, the original image data is first acquired as the basic input for the subsequent identification system. The electronic invoice image data to be processed refers to the digital matrix that carries the complete visual information of the invoice and is used for authenticity verification. This data has significant multimodal characteristics; its surface is not a single homogeneous texture, but rather a composite of multiple legitimate texture modalities, such as blank paper areas, red stamp areas, black printed text areas, and table line printing edges. The key second-order statistic describing the texture structure is the covariance matrix, which is a symmetric positive definite (SPD) matrix, forming a Riemannian manifold with a specific curved structure. This is the core basis for distinguishing the original image from the high-quality re-image.

[0016] In practical implementation, the source channels of the electronic ticket image data to be processed are first clarified. These channels typically include, but are not limited to: digital image files generated by document scanners scanning physical tickets, screenshots of electronic tickets generated by computer operating systems or applications, and photos of tickets taken by smartphones or dedicated cameras. The implementation of this step mainly relies on the communication protocol between the system and the corresponding hardware devices (such as scanners, cameras) or software interfaces (such as screen capture APIs). Next, by calling the corresponding functions of image processing libraries (e.g., OpenCV, PIL), image files (common formats such as JPEG, PNG, BMP, etc.) from the above data sources are read and decoded into a pixel matrix representation in memory. For example, an RGB color image will be decoded into a three-dimensional array with dimensions [height, width, 3], corresponding to the image's height pixel value, width pixel value, and the intensity values ​​of the red, green, and blue color channels, respectively. This step is crucial in converting static data stored in the file system into dynamic data that can be directly processed by the program. Furthermore, the decoded pixel data is loaded into the system-specified memory area or data buffer and uniformly encapsulated into the data structure expected by subsequent processing modules.

[0017] Specifically, step S2 involves preprocessing and frequency domain decomposition of the electronic ticket image data to be processed to obtain a normalized spatial domain image of the ticket and initial frequency domain components. It should be understood that electronic tickets in actual business scenarios often originate from different acquisition devices, resulting in diverse original resolutions and image sizes. Size normalization through preprocessing ensures the standardization of input to the subsequent feature extraction network. More importantly, high-quality screen re-photographs or digital tampering often leave extremely subtle high-frequency anomalies in the underlying image, such as pixel grid effects or interpolation traces. Traditional convolutional neural networks, due to multiple downsampling and pooling operations, easily filter out these crucial high-frequency details as noise when processing images. Therefore, in the technical solution of this application, frequency domain decomposition decouples and extracts semantic information from the spatial domain and textural details from the frequency domain, thereby providing the model with a data foundation capable of capturing microscopic evidence of forgery and solving the identification problem caused by feature coupling or information loss in existing technologies.

[0018] In practical implementation, the first step is to perform spatial domain size normalization on the electronic invoice image data to be processed, resulting in a normalized spatial domain image of the invoice. It should be understood that in actual business scenarios, the electronic invoice image data to be processed often comes from different acquisition terminals, such as different models of mobile phone cameras, scanners, or screen capture software. This leads to significant differences in resolution, pixel scale, and aspect ratio of the original images. Furthermore, since subsequent frequency-space dual-stream parallel feature extraction requires a unified feature mapping space, and the microscopic high-frequency noise left by high-quality screen re-photographs (such as pixel grid effects) or interpolation traces left by digital tampering are extremely sensitive to spatial scale, the technical solution of this application eliminates scale interference through size normalization, providing a standardized data input foundation for subsequent deep analysis of the image's underlying features.

[0019] Specifically, resampling techniques can be used to unify ticket images of different specifications to a preset pixel dimension, thereby eliminating scale differences. In this process, firstly, the scaling ratios for height and width are determined; secondly, an interpolation algorithm (such as bilinear interpolation) is used to calculate the original pixel value corresponding to each coordinate point in the normalized image. This process ensures that the electronic ticket image data to be processed is accurately reshaped into a normalized ticket spatial domain image with uniform specifications, thus enabling subsequent spatial high-frequency extraction based on Gaussian differences to be performed on a stable frequency reference.

[0020] Furthermore, Gaussian difference-based spatial high-frequency extraction is performed on the normalized spatial domain image of the invoice to obtain the initial frequency domain components of the invoice. It should be understood that the normalized spatial domain image of the invoice, obtained after size normalization, mainly contains low- and mid-frequency information such as the overall layout and text content of the invoice. However, many subtle forgery traces, such as local texture anomalies caused by smearing, cloning, or erasing using image editing software, or artificial traces such as moiré patterns and jagged edges introduced during screen reproduction, often manifest as abrupt changes or distortions in the high-frequency information of specific areas of the image. These high-frequency signals may be difficult to detect directly in the original spatial domain image due to their weak energy. Therefore, in the technical solution of this application, a bandpass filtering effect is used to effectively suppress the gently changing low-frequency background in the image while highlighting rapidly changing high-frequency details, thereby making potential, visually imperceptible forgery clues explicit and providing a high signal-to-noise ratio input for subsequent frequency domain-based feature analysis.

[0021] Specifically, first, two parameters with different standard deviations are set. The Gaussian kernel function, where This is used for smoothing images, and the process is expressed by the formula: in, and Represents spatial coordinates, indicating the offset distance of a pixel relative to the center of the Gaussian kernel in the horizontal and vertical directions. It is the normalization constant (amplitude term) of the Gaussian distribution, used to ensure that the integral of the Gaussian kernel function in the spatial domain is always equal to 1, thereby ensuring that the total energy of the image remains unchanged after smoothing.

[0022] Next, the normalized spatial domain image of the ticket is... Convolution operations are performed with Gaussian kernels of these two different scales to obtain two images with different levels of blur. Then, these two images are subtracted to extract specific high-frequency information, thus obtaining the initial frequency domain components of the ticket. The complete spatial high-frequency extraction process can be expressed by the following formula: in, The initial frequency domain components of the ticket are used. Through this Gaussian difference-based mechanism, the system can adaptively perceive the differentiated frequency domain characteristics of different devices, providing a clean high-frequency signal source for subsequent frequency domain stream feature extraction.

[0023] Specifically, in S3, parallel feature extraction of the normalized spatial domain image and the initial frequency domain components of the ticket is performed to obtain the spatial domain feature vector, the frequency domain feature vector, intermediate features of the spatial stream, and intermediate features of the frequency stream. It should be understood that existing electronic ticket processing technologies, such as traditional CNN backbone networks (e.g., ResNet or VGG), primarily extract features in the RGB spatial domain. Even high-frequency noise (such as pixel grid effects) or interpolation traces left by digital tampering introduced by screen re-photographing are invisible to the human eye. After multiple downsampling and pooling operations by the convolutional neural network, these microscopic high-frequency details are filtered out as noise, causing the model to fail to detect forgery traces. Furthermore, existing feature coupling processing methods can lead to discrimination interference because ticket images contain semantic content (such as textual amounts) and carrier features (such as paper or screen texture). Existing methods typically mix these two and input them into the same network, causing the model to be easily misled by clear textual content and ignore subtle anomalies in the background carrier. Meanwhile, a simple fixed high-pass filter cannot adapt to the differentiated frequency domain characteristics of different display devices. Therefore, in the technical solution of this application, parallel dual-stream feature extraction is used to extract deep features that can characterize the semantic content of the ticket and high-frequency texture details, while retaining intermediate layer feature maps for subsequent refined analysis. By constructing two parallel processing streams, the unique advantages of information from different domains are fully utilized. Specifically, the spatial stream focuses on understanding the global semantics and structural layout of the ticket, while the frequency domain stream focuses on capturing subtle texture anomalies and forgery traces, thereby comprehensively improving the model's discrimination capability through complementary fusion at the feature level.

[0024] In practice, the system first extracts spatial semantic features from the normalized spatial domain image of the invoice using a lightweight CNN to obtain the spatial domain feature vector and intermediate features of the spatial flow. During this process, a lightweight convolutional neural network is constructed, typically consisting of multiple stacked convolutional layers, pooling layers, and activation function layers. First, the normalized spatial domain image of the invoice is input into the pre-convolutional layers of the lightweight CNN, and primary visual features are extracted through cross-correlation operations between the convolutional kernels and local image regions. Second, the tensor data output from the intermediate layers of the network is extracted as intermediate features of the spatial flow. These features preserve the spatial correspondence of the original image and play a crucial role in accurately locating counterfeit regions. Subsequently, the feature map undergoes dimensionality reduction and compression through a global average pooling layer, outputting the final spatial domain feature vector of the invoice. Through this layer-by-layer mapping mechanism, the system can efficiently extract semantic expressions describing the macro-level business information of the invoice.

[0025] Furthermore, frequency domain texture features are extracted from the initial document's frequency domain components using dilated residual convolution to obtain the document's frequency domain feature vector and intermediate features of the frequency domain flow. Specifically, firstly, a convolutional layer with a dilation rate is used to perform multi-scale feature perception on the initial document's frequency domain components. In the dilated convolution operation, by introducing a gap between the convolution kernel elements, the convolution kernel can cover a wider pixel area, thereby capturing the global texture structure without losing spatial resolution. Secondly, tensor data output from the intermediate layers of this network, which retains the original image spatial correspondence, is extracted as intermediate features of the frequency domain flow. Subsequently, these features are nonlinearly mapped through a residual connection structure to solve the gradient vanishing problem in deep networks and enhance the expressive power of texture features. Finally, the feature map is processed by a global average pooling layer to output the final document frequency domain feature vector used for identification and classification. This process achieves a high-dimensional numerical vector representation of the micro-texture of the document's underlying carrier.

[0026] Specifically, in S4, local texture inconsistency analysis is performed on the intermediate features of the spatial flow and the intermediate features of the frequency domain flow to obtain a local abnormal activation map. It should be understood that since real electronic documents (such as VAT invoices) are not uniformly textured, but contain multiple legitimate texture modalities such as blank paper, red stamps, black text, and table lines, forcibly calculating a single global background distribution would lead to high false positives in areas with dense legitimate stamps or text due to their large difference from the average background. Furthermore, attackers can often mimic the background mean in screen copying or digital tampering, but it is difficult to reproduce the second-order statistical correlation structure between pixels. KL divergence ignores the geometric relationship of the Riemannian manifold where the covariance matrix resides, and cannot accurately reflect the real handling costs distributed on the manifold surface. Specifically, the surface of real electronic documents, such as VAT invoices, is not a single homogeneous texture, but naturally composed of multiple legitimate texture modalities, such as the fine fiber texture of blank paper areas, the ink smudge texture of red stamp areas, the toner particle texture of black printed text areas, and the printed edge texture of table lines. Existing techniques forcibly statistically average these distinct texture regions into a single global background distribution, ignoring the inherent multimodal characteristics of documents. The direct consequence is that when a legitimate but inherently complex textured region (such as a stamp area) is compared to this oversimplified average background, it will be incorrectly identified as an anomaly due to its inherent significant differences, resulting in a large number of false positives (i.e., a high false positive rate). Furthermore, the covariance matrix, a key second-order statistic describing texture structure, is a symmetric positive definite (SPD) matrix, and its set constitutes a Riemannian manifold with a specific curved structure. KL divergence, when measuring the difference between two Gaussian distributions, cannot adequately capture the true distance or degree of deformation of the covariance matrix in the manifold space. In advanced forgery scenarios, attackers may manipulate the forged region to make its mean (such as average brightness and color) highly consistent with the background, but it is difficult to perfectly reproduce the subtle correlations between pixels, i.e., the structure of the covariance matrix. KL divergence is not sensitive enough to cases where the mean matches but the structure (shape) does not, thus leaving an opportunity for highly covert forgery methods such as high-quality screen remakes or AI content generation.

[0027] To address the aforementioned technical shortcomings, an adaptive manifold prototype matching and Wasserstein anomaly measurement mechanism is proposed. This mechanism abandons the single global background model and purely information-theoretic distance metrics, instead employing an analytical framework more aligned with physical and geometric intuition. Specifically, firstly, multi-dimensional feature alignment and probability distribution modeling are performed on the intermediate features of the spatial flow and the intermediate features of the frequency domain flow to obtain the local region noise probability distribution and the global background noise probability distribution. In other words, through multi-dimensional feature alignment and probability distribution modeling, the high-dimensional distribution characteristics are accurately reflected at the geometric level to identify highly concealed forgery methods.

[0028] In this process, firstly, multi-dimensional feature alignment is performed on the intermediate features of the spatial flow and the intermediate features of the frequency domain flow to ensure that the feature tensors from different flows are accurately matched in both spatial and channel dimensions, thereby comprehensively representing the semantic and textural information of the ticket. Secondly, local noise features at all locations in the image are collected. For any local region in the image, the system extracts the set of feature vectors within that region, calculates its mean vector and covariance matrix, and thus defines the local noise probability distribution at that location. Through the above modeling process, the system not only obtains the local noise probability distribution at each sampling point, but also calculates the global background noise probability distribution representing the comprehensive statistical characteristics of the entire ticket by performing global statistics on the features of the entire image. This process transforms the original feature tensors into a set of measurable probability distributions in the Riemannian manifold space, laying the data foundation for subsequent background texture prototype clustering.

[0029] Next, background texture prototype clustering and manifold mapping are performed on the local noise probability distribution to obtain a set of background texture prototypes. This step aims to discard the invalid global uniformity assumption and instead allow the system to autonomously learn and understand the legitimate texture composition contained in the current document to be inspected. In this process, firstly, the set of local noise probability distributions (defined by the mean vector and covariance matrix) at all locations in the image is collected. Then, on the Riemannian manifold space formed by these distributions, an unsupervised clustering algorithm suitable for manifold data (such as the Riemann K-Means algorithm) is applied to cluster the massive local self-distributions into K representative clusters. The center of each cluster constitutes a background texture prototype, and this process is expressed by the formula: in, This represents the set of extracted background texture prototypes, where each prototype is represented by a mean vector. and a covariance matrix By definition, this set is defined by the clustering function. Set of local noise distributions at all locations in the image This is obtained after processing. In this way, by establishing such a multimodal benchmark set, subsequent anomaly judgments will no longer be compared with a fuzzy global average, but will be matched with a series of clear and specific legitimate texture templates, thus laying the foundation for accurate identification and effectively avoiding misjudgments caused by the complexity of the tickets themselves.

[0030] Furthermore, the minimum transport cost is calculated on the background texture prototype set and the local noise probability distribution to obtain the manifold geometric anomaly score set. That is, a metric that can deeply understand the geometric changes in the probability distribution is used to compensate for the shortcomings of KL divergence in geometric blind spots. In this process, for any coordinate point on the image... Local noise probability distribution at [location] Calculate its set with the background texture prototype. Each prototype The 2-Wasserstein distance between the two points is used. This distance has a closed-form solution for a Gaussian distribution, which not only considers the difference in means but, more importantly, includes a term measuring the structural difference between the two covariance matrices. Finally, the minimum distance between the local region and all valid prototypes is taken as the geometric anomaly score for that point. This process is expressed by the formula: This formula represents two Gaussian distributions. and The square of the 2-Wasestein distance between them The calculation method. Among them, This represents the square of the Euclidean distance. This represents the trace of a matrix. The formula represents the minimum cost required to move or transform the noise distribution of a region into another legitimate background distribution. Specifically, the trace term in the formula... The geometric cost of altering the covariance matrix (i.e., the shape, orientation, and magnitude of the distribution) was precisely quantified. Subsequently, a geometric anomaly score was calculated for that point. Due to its local noise distribution With all background prototypes Calculate Wasserstein distance The minimum value is then determined. Thus, by utilizing the sensitivity of the Wasserstein distance to structural deformation of the covariance matrix, forged regions with similar means but abnormal texture structures can be accurately captured. For example, the moiré patterns introduced by screen re-photographing cause periodic changes in the eigenvalues ​​of the covariance matrix. Even if the color difference is imperceptible to the human eye, this geometric distance will significantly increase, effectively identifying extremely subtle forgery traces.

[0031] Subsequently, based on the manifold geometric anomaly score set, a local anomaly activation map is generated. During this process, the geometric anomaly scores obtained in the previous step are... This is multiplied by a weighting factor determined by the local texture entropy. This weighting factor is a negative exponential function of the Shannon entropy of the local covariance matrix, thereby differentially modulating regions with different properties. This process is expressed by the formula: Among them, the final local abnormal activation map It is in the geometric scoring Entropy weighting followed by normalization Obtained. It is a hyperparameter used to control the strength of entropy suppression. The Shannon entropy, representing the local covariance matrix, is calculated as follows: in,, It is the dimension of the feature vector. It is the determinant of the local covariance matrix. This step constructs a smart confidence gate. For example, a region faked using Photoshop's smoothing tool will have an unusually uniform texture, resulting in a lower determinant of its covariance matrix. The exponent HH decreases sharply. At this point, the negative exponent term... The entropy H becomes very large, thus significantly amplifying the geometric anomaly score of the region. Conversely, a legitimate region with inherently complex textures (such as dense text areas) will have a high entropy H, resulting in a smaller weighting factor and effectively suppressing its original geometric anomaly score. In this way, through entropy weight fusion, the mechanism can clearly separate geometrically mismatched and structurally overly simple regions (highly suspected of forgery) from geometrically mismatched but structurally reasonably complex regions (potentially legitimate but unique backgrounds), ultimately generating a high signal-to-noise ratio anomaly activation map that can accurately locate forged regions.

[0032] In summary, this mechanism fundamentally improves the algorithm's ability to detect forgery traces by abandoning overly simplistic assumptions and introducing more profound geometric and information theory tools. Specifically, by adaptively learning the multimodal texture prototype of the document, it significantly reduces the false alarm rate caused by the complexity of the document itself, improving the system's usability in real-world business scenarios. On the other hand, by employing the Wasserstein distance, which is extremely sensitive to distributed geometry, and combining it with entropy weights for intelligent modulation, the system can effectively detect sophisticated forgeries that appear visually "perfect" to blend into the background but have flaws in their underlying data structure. This significantly improves the detection rate of covert attacks such as screen copying and digital tampering. Ultimately, it achieves a leap from traditional statistical comparison to in-depth geometric structure analysis, resulting in a more reliable, accurate, and intelligent electronic document security auditing solution.

[0033] Specifically, in step S5, based on the local anomaly activation map, cross-domain attention fusion is performed on the spatial domain feature vector and the frequency domain feature vector of the invoice to obtain the invoice fusion identification feature vector. It should be understood that existing electronic invoice processing schemes often couple the semantic content of the invoice with the texture features of the carrier, making the model highly susceptible to being misled by clear text content and ignoring subtle anomalies in the background carrier, and making it difficult to identify advanced forgery methods with similar means but disrupted correlation structures. In the technical solution of this application, by introducing a local anomaly activation map as a guide, the system can construct an intelligent confidence gate, clearly separating geometrically mismatched and structurally overly simple regions (such as PS smoothed regions) from geometrically mismatched but structurally reasonably complex regions. This cross-domain attention fusion mechanism fundamentally improves the algorithm's ability to perceive forgery traces. Utilizing the Wasserstein distance and entropy weight modulation results, which are extremely sensitive to the geometrical distribution, the system can identify sophisticated forgeries that are visually flawless but have flaws in their underlying data structure, thereby significantly improving the detection rate of covert attack methods such as screen re-photographing and digital tampering.

[0034] In practice, firstly, QKV projection is performed on the spatial domain feature vector and the frequency domain feature vector of the invoice to obtain the query vector, key vector, and value vector. That is, features from different domains are mapped to a unified attention space through linear transformation. In this process, firstly, the spatial domain feature vector and the frequency domain feature vector of the invoice are concatenated to form a joint feature vector; secondly, matrix multiplication is performed on this concatenated joint feature vector using three independent, trainable linear transformation matrices (query weight matrix, key weight matrix, and value weight matrix). The query weight matrix projects the input features onto a new vector space, generating the so-called query vector, which represents an active inquiry signal; the key weight matrix projects the input features onto another vector space, generating the key vector, which represents a matching identifier signal; and the value weight matrix projects the input features onto a third vector space, generating the value vector, which carries the original information to be aggregated. This process transforms the original two-stream features into a set of intermediate representations more suitable for attention calculation.

[0035] Next, an attention mask is generated based on the local anomaly activation map. The local anomaly activation map is a two-dimensional matrix, where the value at each position reflects the probability of anomalies in the corresponding image region. To apply this spatial information to the fusion modulation of the global feature vector, the activation map is aggregated into one or more global anomaly saliency indices. In this process, firstly, the arithmetic mean of all pixel values ​​in the entire anomaly activation map is calculated to obtain a scalar score representing the average anomaly severity of the entire image. Secondly, this scalar score is mapped using an sigmoid function (e.g., a sigmoid function) to compress its value into a fixed interval (e.g., between 0 and 1). Then, this mapped score is multiplied by a negative scaling factor to finally generate an attention mask value. This negative mask value means that when the overall anomaly severity of the image is higher, it will have a negative bias effect on certain attention scores in subsequent calculations, thereby changing the distribution of attention weights.

[0036] Furthermore, based on attention masks, masked attention fusion is performed on the query vector, key vector, and value vector to obtain the fused identification feature vector of the ticket. Specifically, masked attention fusion is performed on the query vector, key vector, and value vector using the following formula: in, For attention masking, For query vector, For key vectors, Let be the dimension of the key vector. For value vectors, for Activation function. Through this mechanism, the system utilizes the sensitivity of Wasserstein distance to the structural deformation of the covariance matrix to accurately capture the features of forged regions with similar means but abnormal texture structure, and condenses them into the final discriminative feature vector.

[0037] Specifically, in step S6, the fused identification feature vector of the invoice is classified to obtain the invoice status category. It should be understood that after a series of complex transformations such as preprocessing, frequency domain decomposition, dual-stream feature extraction, local anomaly analysis, and cross-domain attention fusion, the original electronic invoice image data has been transformed into a highly refined fused identification feature vector. Although this vector contains all the key information for distinguishing the invoice status, it is still a high-dimensional, continuous numerical vector that cannot be directly understood by humans or used for business logic judgment. Therefore, in the technical solution of this application, the highly abstract and information-rich fused identification feature vector of the invoice generated in the preceding steps is mapped to a specific category label representing the authenticity or status of the invoice, thereby completing the final task of automated identification.

[0038] In practice, the fused identification feature vector of the invoice is input into a fully connected classification layer to obtain the invoice status category. During this process, the fully connected classification layer performs nonlinear mapping on the semantic, frequency domain, and geometric anomaly signals contained in the fused identification feature vector, transforming these complex texture features captured in the high-dimensional Riemannian manifold space (such as the deformation cost of the covariance matrix reflecting the distribution shape) into specific classification scores. Then, the classification layer extracts key patterns that can distinguish between legitimate multimodal textures (such as seals and dense text areas) and illegal structural anomalies (such as PS smoothing marks and photocopy moiré patterns) through multiplication of the weight matrix and feature vector, ultimately outputting the corresponding status category. The final generated invoice status category refers to the final review result output by the system, including statuses such as normal invoices, suspected photocopy invoices, and illegally tampered invoices.

[0039] In summary, the image-based electronic invoice processing method according to the embodiments of this application is explained. It extracts the spatial semantics and frequency-domain texture features of the invoice through parallel extraction of frequency and spatial dual-stream data, and introduces local texture inconsistency analysis to generate anomaly activation maps, thereby guiding cross-domain attention fusion and achieving deep analysis of the underlying data structure of the invoice. In this way, the traditional assumption of global uniformity is abandoned, greatly reducing the false alarm rate caused by complex backgrounds such as seals and text, thus significantly improving the system's robustness and accuracy in real-world business scenarios.

[0040] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An electronic ticket processing method based on image recognition, characterized in that, include: Acquire the image data of the electronic invoice to be processed; The electronic ticket image data to be processed is preprocessed and decomposed in the frequency domain to obtain the normalized ticket spatial domain image and the initial ticket frequency domain components. Parallel feature extraction of frequency and space streams is performed on the normalized spatial domain image of the bill and the initial frequency domain component of the bill to obtain the spatial domain feature vector, the frequency domain feature vector, the intermediate features of the spatial stream, and the intermediate features of the frequency stream. Local texture inconsistency analysis is performed on the intermediate features of spatial flow and the intermediate features of frequency domain flow to obtain local anomalous activation maps; Based on the local anomaly activation map, cross-domain attention fusion is performed on the spatial domain feature vector and the frequency domain feature vector of the bill to obtain the bill fusion identification feature vector. The fused identification feature vector of the bill is used for identification and classification to obtain the bill status category.

2. The electronic ticket processing method based on image recognition according to claim 1, characterized in that, The electronic ticket image data to be processed undergoes preprocessing and frequency domain decomposition to obtain a normalized ticket spatial domain image and initial ticket frequency domain components, including: The electronic ticket image data to be processed is subjected to spatial domain size normalization to obtain a normalized ticket spatial domain image; Gaussian difference-based spatial high-frequency extraction is performed on the normalized ticket spatial domain image to obtain the initial ticket frequency domain components.

3. The electronic ticket processing method based on image recognition according to claim 1, characterized in that, Parallel feature extraction of frequency and space domains is performed on the normalized spatial domain image of the document and the initial frequency domain components of the document to obtain the document spatial domain feature vector, document frequency domain feature vector, intermediate features of the spatial flow, and intermediate features of the frequency flow, including: Spatial semantic features are extracted from normalized ticket spatial domain images using a lightweight CNN to obtain the ticket spatial domain feature vector and intermediate features of spatial flow. Frequency domain texture features are extracted from the initial ticket frequency domain components using dilated residual convolution to obtain the ticket frequency domain feature vector and intermediate features of the frequency domain flow.

4. The electronic ticket processing method based on image recognition according to claim 1, characterized in that, Local texture inconsistency analysis is performed on intermediate features of spatial flow and intermediate features of frequency domain flow to obtain local anomalous activation maps, including: Multidimensional feature alignment and probability distribution modeling are performed on the intermediate features of spatial flow and frequency domain flow to obtain the local noise probability distribution and the global background noise probability distribution. The background texture prototype set is obtained by performing background texture prototype clustering and manifold mapping on the local noise probability distribution. Minimum transport cost calculation is performed on the background texture prototype set and the local region noise probability distribution to obtain the manifold geometric anomaly score set; Based on the manifold geometric anomaly score set, a local anomaly activation map is generated.

5. The electronic ticket processing method based on image recognition according to claim 1, characterized in that, Based on the local anomaly activation map, cross-domain attention fusion is performed on the spatial domain feature vector and the frequency domain feature vector of the bill to obtain the bill fusion identification feature vector, including: QKV projection is performed on the spatial domain feature vector and the frequency domain feature vector of the bill to obtain the query vector, key vector and value vector; Generate attention masks based on local anomaly activation maps; Based on attention masking, the query vector, key vector, and value vector are subjected to masked attention fusion to obtain the fused identification feature vector of the ticket.

6. The electronic ticket processing method based on image recognition according to claim 5, characterized in that, Based on attention masks, a masked attention fusion is performed on the query vector, key vector, and value vector to obtain the fused identification feature vector of the document. This includes performing masked attention fusion on the query vector, key vector, and value vector using the following formula: in, For attention masking, For query vector, For key vectors, Let be the dimension of the key vector. For value vectors, for Activation function.

7. The electronic ticket processing method based on image recognition according to claim 1, characterized in that, To obtain the bill status category, the fusion identification feature vector of the bill is classified and identified, including: inputting the fusion identification feature vector of the bill into a fully connected classification layer to obtain the bill status category.

Images