An enterprise finance and tax data processing method based on image recognition

Abstract

The application relates to the technical field of image data processing, in particular to an enterprise financial tax data processing method based on image recognition, which comprises the following steps: performing quality evaluation on financial tax bill images through a lightweight neural network and dynamically scheduling the images to different processing channels; accurately positioning and extracting a key information region by using a target detection and attention mechanism segmentation network; constructing a multi-modal error correction model, combining original text recognized by a character recognition engine with image layout features and financial tax field semantic logic to perform collaborative verification and intelligent error correction; automatically filling corrected information into a structured template, and performing verification on a hooking relationship of an embedded rule base to output data results or mark abnormalities for manual review. Therefore, the problems of rigid enterprise financial tax data processing flow, single template information and poor adaptability to complex scenarios are solved, and the overall efficiency, accuracy and robustness of financial tax bill processing are improved.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, and in particular to a method for processing enterprise financial and tax data based on image recognition. Background Technology

[0002] With the deepening of enterprise digital transformation, the automation and intelligentization of financial and tax data processing have become key aspects of improving financial management efficiency. In daily operations, enterprises face massive amounts of heterogeneous invoices, including VAT invoices, expense reimbursement forms, and bank statements. Information extraction and structured data entry are core pre-processing tasks for financial and tax systems. Current mainstream solutions generally rely on Optical Character Recognition (OCR) technology to automate the processing of invoice images, replacing traditional manual data entry. However, this method faces multiple technical bottlenecks in practical applications, making it difficult to balance processing efficiency, recognition accuracy, and system robustness.

[0003] While image recognition-based methods for processing financial and tax data have been widely deployed, their underlying architecture still has significant limitations. Systems in related technologies typically employ a fixed process to perform uniform recognition operations on all input images, failing to dynamically adjust processing strategies based on image quality. This results in redundant calculations of high-resolution documents, while low-quality images—blurry, tilted, or unevenly lit—suffer from high recognition failure rates due to a lack of targeted enhancement methods. Furthermore, traditional OCR engines rely solely on pixel-level visual features for text recognition, neglecting the inherent semantic logic and layout structure information of documents. They struggle to perform contextual correction for easily confused characters (such as "0" and "O", "1" and "I") and cannot utilize logical relationships between fields (such as the mathematical relationship between amount and tax rate) for automatic error correction.

[0004] The existing technologies still struggle to effectively handle variations in document shape under complex physical conditions. When documents are stuck together, folded, partially occluded, or subject to photographic distortion, conventional region segmentation algorithms often fail to accurately define the boundaries of key information regions, causing input distortion in subsequent recognition modules. Furthermore, the lack of a multimodal information fusion mechanism prevents the system from collaboratively utilizing image layout, textual semantics, and financial and tax business rules for joint reasoning, resulting in significant fluctuations in overall accuracy in real-world business environments, particularly prone to fatal errors in key numerical fields.

[0005] Therefore, there is an urgent need for an intelligent processing method for enterprise financial and tax data that can combine adaptive scheduling of image quality, multimodal collaborative error correction, and structured verification, in order to overcome the comprehensive bottlenecks in efficiency, accuracy, and reliability of related technologies. Summary of the Invention

[0006] This application provides an image recognition-based method for processing corporate financial and tax data to address issues such as rigid corporate financial and tax data processing workflows, limited template information, and poor adaptability to complex scenarios, thereby improving the overall efficiency, accuracy, and robustness of financial and tax invoice processing.

[0007] The first aspect of this application provides a method for processing enterprise financial and tax data based on image recognition, including the following steps: The system receives images of tax invoices to be processed, performs quality assessment on the images using a lightweight convolutional neural network, and outputs a comprehensive quality score. The comprehensive quality score reflects the sharpness, noise level, and integrity of the tax invoice images. Based on the comprehensive quality score, the tax invoice images are dynamically scheduled to different processing channels; In each processing channel, an object detection model is used to locate and identify the tax invoice image to obtain the layout features and invoice type of the tax invoice image, and a segmentation network with an attention mechanism is used to accurately segment the key information regions in the tax invoice image. The segmented regions are processed by an image recognition engine to obtain the original text sequence. At the same time, a multimodal error correction model is constructed. The multimodal error correction model receives the original text sequence and the layout features, uses a pre-trained financial and tax domain language model to perform semantic understanding on the original text sequence, and performs verification and error correction on the original text sequence based on the layout features and semantic logic, and outputs the corrected text information. Based on the invoice type, the corrected text information is automatically mapped and filled into a predefined structured data template. The built-in financial and tax rule library is called to verify the consistency of the structured data. If logical errors or numerical anomalies are found, the corresponding data is marked as data to be manually reviewed, and a confidence report is output.

[0008] Optionally, in some embodiments, dynamically scheduling the tax invoice image to different processing channels includes: If the overall quality score is higher than the set high threshold, the tax invoice image will be dynamically scheduled to a fast recognition channel equipped with a basic text recognition engine and simple post-processing. If the overall quality score is lower than the set low threshold, the tax invoice image will be dynamically scheduled to a fine recognition channel that includes image enhancement operations. If the overall quality score is higher than the set low threshold and lower than the set high threshold, the tax invoice image will be dynamically scheduled to the standard recognition channel configured with standard image preprocessing operations.

[0009] Optionally, in some embodiments, the image enhancement operations in the fine recognition channel include one or more of generative adversarial network-based super-resolution reconstruction, multi-scale denoising, and perspective transformation correction.

[0010] Optionally, in some embodiments, the lightweight convolutional neural network employs a depth-separable convolutional structure, comprising 8 convolutional layers and 3 fully connected layers; Among them, the sharpness assessment uses the Laplacian variance algorithm, the noise level assessment uses the block signal-to-noise ratio calculation, and the integrity assessment is based on the combined results of image edge detection and template matching.

[0011] Optionally, in some embodiments, the target detection model adopts a single-stage detection architecture, with a deep residual network as the backbone network and a feature pyramid network for multi-scale feature fusion. The segmentation network that introduces an attention mechanism integrates a channel attention module and a spatial attention module in the encoder part, and the decoder adopts a skip connection structure, achieving pixel-level segmentation accuracy.

[0012] Optionally, in some embodiments, the multimodal error correction model adopts an encoder-decoder architecture, wherein the encoder part includes a visual feature encoding branch and a text feature encoding branch; The layout features extracted by the visual feature encoding branch include the center coordinates, width, height, and relative positional relationship of the text region with neighboring regions; The text feature encoding branch uses the pre-trained language model in the fiscal and tax domain to perform semantic understanding of the text sequence; The decoder adopts a gated recurrent unit (GRU) structure, which fuses visual and text features through an attention mechanism, and uses a beam search algorithm at the output layer to generate the corrected text information.

[0013] Optionally, in some embodiments, the predefined structured data template is defined using a structured data format, including required fields and optional fields, and the field mapping rules are dynamically loaded based on the invoice type; The built-in tax and finance rule library covers numerical calculation rules, logical association rules, and format specification rules, and the built-in tax and finance rule library supports online updates and incremental learning. The verification of the reconciliation relationship includes verification of the mathematical relationship between the amount and the tax amount, checking the compliance of the date format, and verifying the taxpayer identification number.

[0014] A second aspect of this application provides an enterprise financial and tax data processing apparatus based on image recognition, comprising: The image quality assessment module is used to receive the tax invoice image to be processed, perform quality assessment on the tax invoice image through a lightweight convolutional neural network, and output a comprehensive quality score, which reflects the sharpness, noise level and integrity of the tax invoice image. The dynamic channel scheduling module is used to dynamically schedule the financial and tax invoice images to different processing channels based on the comprehensive quality score. The invoice location and region segmentation module uses an object detection model to locate and identify the tax invoice image in each processing channel, obtains the layout features and invoice type of the tax invoice image, and uses a segmentation network with an attention mechanism to accurately segment the key information region in the tax invoice image. The multimodal text correction module is used to perform text recognition on each segmented region using a text recognition engine to obtain the original text sequence. At the same time, it constructs a multimodal error correction model. The multimodal error correction model receives the original text sequence and the layout features, uses a pre-trained financial and tax domain language model to perform semantic understanding on the original text sequence, and performs verification and error correction on the original text sequence based on the layout features and semantic logic, and outputs the corrected text information. The structured data verification module is used to automatically map and fill the corrected text information into a predefined structured data template according to the invoice type, call the built-in financial and tax rule library to verify the consistency of the structured data, and if logical errors or numerical anomalies are found, the corresponding data is marked as data to be manually reviewed and a confidence report is output.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an image recognition-based enterprise financial and tax data processing method as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer program product having a computer program stored thereon, which is executed by a processor to implement an image recognition-based enterprise financial and tax data processing method as described in the above embodiments.

[0017] The beneficial effects of the embodiments of this application are as follows: (1) Through the dynamic task scheduling mechanism, the resources of the financial and tax data processing method of this application are reasonably allocated, avoiding the over-processing of high-quality images and the under-processing of low-quality images by the "one-size-fits-all" processing method, thereby significantly improving the throughput of massive invoice processing, i.e. improving processing efficiency. (2) This application effectively solves the problem of misrecognition caused by image quality or character similarity in traditional OCR by using multimodal information (image, text, layout) collaborative error correction. In particular, the recognition accuracy (especially key numerical fields) is significantly improved in complex scenarios such as handwritten characters, blurry characters, and small fonts. (3) This application adopts an improved segmentation network with attention mechanism, which can more accurately handle complex situations such as sticking, folding, and obscuring of invoices, ensuring the complete extraction of key information areas and expanding the scope of application of the financial and tax data processing method of this application in practical applications. (4) This application can automatically detect and prompt potential data logic errors through structured verification and adaptive output, providing high-quality and reliable data input for enterprise financial and tax systems, and reducing financial and audit risks from the source.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a corporate financial and tax data processing method based on image recognition, according to an embodiment of this application. Figure 2 This is a block diagram of an image recognition-based enterprise financial and tax data processing device according to an embodiment of this application; Figure 3 This is a block diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0021] The following description, with reference to the accompanying drawings, illustrates an image recognition-based method for processing enterprise financial and tax data according to an embodiment of this application. Addressing the problems mentioned in the background art, such as rigid enterprise financial and tax data processing workflows, limited template information, and poor adaptability to complex scenarios, this application first provides an image recognition-based method for processing enterprise financial and tax data. Specifically, Figure 1 This is a flowchart illustrating an image recognition-based enterprise financial and tax data processing method provided in an embodiment of this application.

[0022] like Figure 1 As shown, this image recognition-based method for processing enterprise financial and tax data includes the following steps: In step S101, the tax invoice image to be processed is received, and the quality of the tax invoice image is evaluated by a lightweight convolutional neural network. A comprehensive quality score is output, which reflects the clarity, noise level and integrity of the tax invoice image.

[0023] Among them, financial and tax document images refer to digital images of various financial and tax documents acquired through devices such as scanners and mobile phone cameras, such as value-added tax invoices, train tickets, and expense reimbursement forms. The comprehensive quality score in this application embodiment is a quantitative numerical output used to comprehensively and objectively evaluate the overall quality of a document image. It is not a reflection of a single indicator, but rather a fusion result of multiple quality dimensions.

[0024] Understandably, to enable subsequent recognition algorithms (such as object detection and text recognition) to process high-quality images faster and more accurately, this application embodiment identifies high-quality images in advance and assigns them to the "fast channel," skipping unnecessary enhancement steps, thereby significantly improving the throughput of the processing system (which integrates an image recognition-based enterprise financial and tax data processing method according to this application embodiment). For poor-quality images, direct recognition would result in extremely low accuracy. This application embodiment assesses their poor quality in advance and schedules them to the "fine channel," which includes "image enhancement," to repair them before recognition, improving the accuracy of the final result from the source. In other words, the comprehensive quality score is the decision-making basis for subsequent "dynamic task scheduling" and is the "perception" link for achieving intelligence in the entire automated process.

[0025] Optionally, in some embodiments, the lightweight convolutional neural network employs a depth-separable convolutional structure containing eight convolutional layers and three fully connected layers.

[0026] Optionally, the number of network parameters of the lightweight convolutional neural network is controlled to within 500,000, the inference speed reaches 30 images per second, and the quality score output range is 0 to 1.

[0027] Specifically, in step S101, the processing system integrating an image recognition-based enterprise financial and tax data processing method according to an embodiment of this application receives raw RGB (Red, Green, Blue) or grayscale invoice images from sources such as file directories, network interfaces, or scanning devices. First, the financial and tax invoice images are standardized, such as being uniformly scaled to a fixed size (e.g., 224×224) and pixel values ​​are normalized.

[0028] Then, the preprocessed image is input into a trained lightweight convolutional neural network. In this embodiment, an 8-layer convolutional network with a depth separable convolutional structure is used to extract image features, and finally a comprehensive score is output by regression through 3 fully connected layers.

[0029] It should be noted that the lightweight convolutional neural network in this application embodiment needs to be trained on a large-scale manually annotated bill image dataset. Each image in the dataset is labeled with a quality rating (e.g., 0-1 points) by human experts based on the criteria of "sharpness, noise, and integrity".

[0030] Thus, the lightweight convolutional neural network outputs a floating-point number between 0 and 1. For example, 0.9 points indicates excellent image quality, while 0.3 points indicates very poor quality that is difficult to recognize directly.

[0031] Optionally, in some embodiments, sharpness assessment uses the Laplacian variance algorithm, noise level assessment uses block signal-to-noise ratio calculation, and integrity assessment is based on the combined results of image edge detection and template matching.

[0032] Among them, the Laplacian operator is an edge detection operator. After performing the Laplacian operation on the image, the regions with rich edges and details will obtain higher response values. In this embodiment, the variance of this response result is calculated. The larger the variance, the more high-frequency components (i.e. details) there are in the image, and the clearer the image is.

[0033] This application's embodiments divide the image into multiple small blocks. Ideally, pixel values ​​should be consistent within smooth-textured blocks, and the degree to which actual pixel values ​​deviate from this ideal value reflects the noise level. This application's embodiments calculate the signal-to-noise ratio of all smooth blocks and take the average to estimate the overall noise level.

[0034] This application embodiment assesses the integrity of tax invoice images through edge detection and template matching. Specifically, the entire outer contour of the invoice is extracted using algorithms such as Canny (Canny Edge Detector) and compared with a standard, complete invoice template contour. If the detected contour matches the template contour very well and has no major missing parts, the integrity score is high; conversely, if the contour is severely incomplete or deformed, the score is low.

[0035] Optionally, in this embodiment of the application, the three components of sharpness, noise level and integrity can be normalized and then linearly weighted and fused, with weight coefficients of 0.4, 0.3 and 0.3 respectively, and finally output a single comprehensive quality score.

[0036] It should be noted that the lightweight convolutional neural network in this application embodiment uses more than 50,000 manually labeled ticket images with quality levels for supervised learning during the training phase. The labels are in the form of triples (sharpness level, noise level, integrity level), and the network parameters are optimized through the cross-entropy loss function.

[0037] Therefore, in step S101, the embodiment of this application performs a quality assessment on the financial and tax invoice image, providing a key decision signal for the subsequent processing flow. This achieves a leap from the clumsy "one-size-fits-all" processing to the intelligent "quality-based" processing, thus laying a solid foundation for improving overall efficiency and ensuring recognition accuracy from the very beginning of the processing method.

[0038] In step S102, the images of financial and tax invoices are dynamically scheduled to different processing channels based on the comprehensive quality score.

[0039] The enterprise financial and tax data processing method in this application does not employ a fixed processing flow. Instead, it flexibly and automatically allocates appropriate computing paths and algorithm resources based on image quality. Each processing channel in this application embodiment differs in processing speed and enhancement intensity.

[0040] Optionally, in some embodiments, the tax invoice image is dynamically scheduled to different processing channels, including: if the overall quality score is higher than a set high threshold, the tax invoice image is dynamically scheduled to a fast recognition channel configured with a basic text recognition engine and simple post-processing; if the overall quality score is lower than a set low threshold, the tax invoice image is dynamically scheduled to a fine recognition channel containing image enhancement operations; if the overall quality score is higher than a set low threshold and lower than a set high threshold, the tax invoice image is dynamically scheduled to a standard recognition channel configured with standard image preprocessing operations.

[0041] Optionally, in some embodiments, the image enhancement operations in the fine recognition channel include one or more of generative adversarial network-based super-resolution reconstruction, multi-scale denoising, and perspective transformation correction.

[0042] Optionally, the high threshold can be set to 0.85 and the low threshold can be set to 0.45.

[0043] In actual execution, the processing latency of the fast recognition channel is controlled within 200 milliseconds; the processing latency of the standard recognition channel is controlled within 500 milliseconds; and the resolution of the fine recognition channel is increased by 4 times, with a processing latency of 2 seconds.

[0044] Understandably, most ticket images are clearly identifiable. Processing these clearly identifiable images quickly in the fast channel avoids them waiting for ticket images requiring image enhancement, thus reducing the average processing time of the entire batch processing task. Furthermore, for images of extremely poor quality, forcing them to be recognized in the fast or standard channels would reduce accuracy, rendering all subsequent work useless. Sending them to the fine channel, where powerful enhancement algorithms "save" them, ensures the lower limit of the overall accuracy of the final output. Additionally, image enhancement algorithms (such as super-resolution) have huge computational overhead. Using them on every image (including high-quality images) would be a huge waste of computing power. The dynamic scheduling in this application ensures that valuable computing resources are used only on those images most in need of enhancement, achieving economical computation.

[0045] Specifically, the processing system receives the comprehensive quality score of the current tax invoice image from step S101, compares the comprehensive quality score with a set high threshold and a set low threshold. If the comprehensive quality score is higher than the set high threshold, the processing system directs its path to the fast recognition channel. This channel may only contain a basic OCR engine and simple post-processing such as space filtering, pursuing the ultimate speed.

[0046] If the overall quality score is lower than a set low threshold, the processing system directs its path to the fine-grained recognition channel. The super-resolution reconstruction module in the fine-grained recognition channel adopts an SRGAN (Super-Resolution Generative Adversarial Network) architecture. The generator network contains 16 residual blocks, and the discriminator network adopts a PatchGAN (Patch-based Generative Adversarial Network) structure. During the training phase, it uses pairs of low-resolution and high-resolution ticket images for adversarial training. The loss function includes three terms: content loss, adversarial loss, and perceptual loss, with a weight ratio of 1:0.001:0.006. The multi-scale denoising algorithm uses a cascaded structure of nonlocal mean filtering and wavelet thresholding. First, a global similarity block search is performed in the spatial domain, and then soft thresholding is applied to high-frequency coefficients in the wavelet domain. The perspective transformation correction module detects the coordinates of the four corner points of the ticket, constructs a homography matrix H, and performs inverse perspective transformation on the image to restore the frontal view of the ticket.

[0047] If the overall quality score is higher than the set low threshold but lower than the set high threshold, meaning the quality of the tax invoice image is medium, the processing system will direct its path to the standard recognition channel, which performs routine operations, such as image sharpening to enhance edges and contrast enhancement to make the text more distinct from the background.

[0048] Therefore, in step S102, this embodiment of the application introduces a dynamic scheduling mechanism based on quality scoring to form a multimodal processing method. High-quality financial and tax invoice images are sent to the fast track to improve efficiency, while low-quality financial and tax invoice images are sent to the repair room to ensure quality. This achieves a balance between processing speed and recognition accuracy on a global scale, and solves the contradiction in related technologies when a single process faces a large number of invoices with uneven quality.

[0049] In step S103, in each processing channel, a target detection model is used to locate and identify the tax invoice image, obtain the layout features and invoice type of the tax invoice image, and a segmentation network with an attention mechanism is used to accurately segment the key information regions in the tax invoice image.

[0050] In this application embodiment, the task of the target detection model is to find the main body of the tax invoice in the image and determine its specific type.

[0051] Optionally, in some embodiments, the target detection model adopts a single-stage detection architecture, with a deep residual network as the backbone network and a feature pyramid network for multi-scale feature fusion.

[0052] Optionally, the training data for the object detection model includes more than 100,000 labeled images of tax invoices, covering 15 common invoice types.

[0053] It is understandable that the images of tax invoices provided by users may contain irrelevant backgrounds such as desktops or fingers. Therefore, this embodiment of the application uses an object detection model to first "select" the main body of the invoice from the complex background, providing a clean working area for subsequent processing. Considering that different types of invoices have completely different fields and format rules, the processing system can only load the corresponding parsing template and verification rules to achieve adaptive processing if it first knows "what kind of invoice this is".

[0054] The layout features of the tax invoice image in this application embodiment refer to the position and spatial relationship information of various key elements (such as "amount", "invoice date", and "seller") in the tax invoice image. The invoice type of the tax invoice image refers to the specific type of tax invoice, such as "value-added tax special invoice", "taxi ticket", "general machine-printed invoice", etc.

[0055] The segmentation network with an attention mechanism in this embodiment is a deep learning model capable of pixel-level classification. It classifies each pixel in a tax invoice image, determining whether it belongs to the "background" or a "key information region" (such as an amount region or a date region). The "attention mechanism" acts like a "focusing lens," making the network pay more attention to information-rich areas and ignore irrelevant background.

[0056] Optionally, in some embodiments, the segmentation network that introduces an attention mechanism integrates a channel attention module and a spatial attention module in the encoder part, and the decoder adopts a skip connection structure, achieving pixel-level segmentation accuracy.

[0057] Those skilled in the art will understand that traditional OCR engines, which locate text regions automatically, are prone to errors in complex layouts. However, the embodiments of this application first perform pixel-level segmentation, which can accurately delineate key areas such as "amount" and "tax rate," and then send this "small image" to OCR for recognition. This is equivalent to telling OCR "only look at this area," thereby significantly improving the accuracy and reliability of the recognition.

[0058] It should be noted that the embodiments of this application perform precise image segmentation to obtain the layout features of the tax invoice image. These layout features provide the precise coordinates and spatial relationships of each key region. These "layout features" are important inputs to the subsequent multimodal error correction model and are used for logical verification (e.g., determining whether a number is located within the "tax amount" region).

[0059] Specifically, the encoder of the segmentation network that introduces the attention mechanism adopts the U-Net (Convolutional Networks for Biomedical Image Segmentation) backbone structure. The downsampling path contains four max pooling layers. After each encoding stage, the CBAM (Convolutional Block Attention Module) attention module is connected. This module performs channel attention and spatial attention calculations in sequence: channel attention generates two 1-dimensional channel descriptors through global average pooling and global max pooling, which are then added after passing through a shared multilayer perceptron and activated by Sigmoid to obtain the channel weights; spatial attention performs average and max pooling in the channel dimension, and after concatenation, it generates a spatial weight map through a 7x7 convolutional layer.

[0060] The segmentation network incorporating an attention mechanism upsamples through transposed convolutions and concatenates the features from the corresponding encoder layers, ultimately outputting a segmentation mask of the same size as the input image. Each pixel is classified into a specific field category (e.g., invoice code, invoice number, invoice date, buyer's name, seller's name, amount, tax, total price including tax, etc.). Optionally, for VAT invoices, the system defines 12 key regions; for train tickets, it defines 8 key regions. During training, the segmentation network uses a weighted sum of Dice loss and cross-entropy loss as the optimization objective, with a weight ratio of 0.7:0.3, to alleviate the class imbalance problem.

[0061] In actual implementation, the segmentation network with attention mechanism introduced in this application embodiment has an intersection-union ratio of over 0.95 for key information regions, and a recall rate of over 99.8% is guaranteed for the monetary value region.

[0062] Intersection over Union (IoU) is an indicator used to measure the accuracy of segmentation or detection. It is calculated as the ratio of the area of ​​intersection of the "model-predicted region" and the area of ​​union of the "human-labeled real region". The closer the ratio is to 1, the more accurate the segmentation. Recall measures the model's ability to find all targets. In this application, an image recognition-based enterprise financial and tax data processing method achieves a "99.8% recall rate for monetary regions". In 1000 real monetary regions, the model can successfully find at least 998, greatly reducing the risk of missed detections.

[0063] Therefore, in step S103, this embodiment of the application achieves accurate parsing of financial and tax documents from macro-type to micro-field through a two-level sequential perception strategy of "target detection and overall localization" and "attention segmentation and local refinement." This provides a high-purity input region and key structured layout information for subsequent OCR recognition and multimodal error correction, forming the basis for the high accuracy of the entire processing method. Furthermore, its intersection-over-union ratio (IoU) of nearly 0.95 and recall rate of over 99.8% ensure that key financial and tax information is rarely missed or misdefined, guaranteeing the reliability of automated processing.

[0064] In step S104, the segmented regions are processed by a text recognition engine to obtain the original text sequence. At the same time, a multimodal error correction model is constructed. The multimodal error correction model receives the original text sequence and layout features, uses a pre-trained language model in the financial and tax domain to perform semantic understanding on the original text sequence, and verifies and corrects the original text sequence based on layout features and semantic logic, and outputs the corrected text information.

[0065] In this embodiment, the text recognition engine employs a CRNN (Convolutional Recurrent Neural Network) architecture. The front-end uses a ResNet-34 feature extraction network, and the back-end uses a bidirectional LSTM (Long Short-Term Memory) sequence modeling network. The output layer uses a CTC (Connectionist Temporal Classification) loss function. The recognition result is the original text string for each region and its character-level confidence score.

[0066] The multi-modal error correction model of the embodiments of the present application is a deep learning model that can simultaneously process and fuse different types of information. "Multi-modal" refers to the visual modality (layout features) and the text modality (original text sequence). The goal of this multi-modal error correction model is to output a more accurate, corrected text sequence.

[0067] It can be understood that there are some limitations in the single text modality. For example, a pure text model cannot know that the number "385.60" appears in the "tax amount" area, so it cannot trigger the reconciliation logic. There are also some limitations in the single visual modality. For example, a pure visual model can determine that an area is the "tax amount", but it cannot determine what the specific number recognized in it is, nor can it perform logical calculations. Therefore, to avoid the above limitations, the embodiments of the present application combine the visual modality (layout features) and the text modality (original text sequence) to construct a multi-modal error correction model, enabling it to simultaneously determine "what is seen where", and thus be able to perform reasoning based on business rules.

[0068] Optionally, the multi-modal error correction model of the embodiments of the present application is: ; Where, is the visual feature vector, is the text feature vector, is the hidden state at the previous moment, is the output weight matrix.

[0069] The pre-trained language model in the field of finance and taxation of the embodiments of the present application is a Transformer model trained on a super-large-scale finance and taxation text (such as invoice details, financial reports, policies and regulations), such as domain variants of BERT (Bidirectional Encoder Representations from Transformers, bidirectional encoder representations based on Transformer) and RoBERTa (Robustly Optimized BERT Pretraining Approach, robustly optimized BERT pre-training method). The pre-trained language model in the field of finance and taxation of the embodiments of the present application is well-versed in the vocabulary, syntax and common sense of the finance and taxation field, and can understand the context meanings of professional terms such as "input tax" and "output tax".

[0070] It can be understood that a general language model may not be able to judge the subtle differences between "tax exemption" and "zero tax rate" in finance and taxation. The pre-trained language model in the field of finance and taxation of the embodiments of the present application has built-in domain knowledge and can better judge whether a word is reasonable in the context of finance and taxation. For example, this model can judge that "咨洵" is a typo or recognition error of "咨询".

[0071] This application embodiment utilizes a pre-trained language model in the financial and tax domain to perform semantic understanding on the original text sequence. That is, in the context, the pre-trained language model in the financial and tax domain performs in-depth analysis on the original text sequence to determine its rationality and authenticity in the financial and tax scenario, rather than just its grammatical correctness.

[0072] Optionally, in some embodiments, the multimodal error correction model adopts an encoder-decoder architecture. The encoder part includes a visual feature encoding branch and a text feature encoding branch. The layout features extracted by the visual feature encoding branch include the center coordinates, width, height, and relative positional relationship with neighboring regions of the text region. The text feature encoding branch uses a pre-trained language model in the financial and tax domain to perform semantic understanding of the text sequence. The decoder adopts a gated recurrent unit structure, fuses visual features and text features through an attention mechanism, and uses a beam search algorithm in the output layer to generate corrected text information.

[0073] Optionally, the pre-trained language model for the tax and finance domain is trained on more than 100 million tax and finance text data.

[0074] The encoder-decoder architecture is a classic framework for sequence-to-sequence tasks. The encoder is responsible for compressing and encoding the input information (text and layout) into a semantically rich intermediate representation; the decoder then generates (decodes) the correct output sequence step by step based on this intermediate representation.

[0075] Gated recurrent units (RNNs) are a variant of RNNs that excel at processing sequential data and can remember contextual information as decoders. Beam search is a decoding strategy that retains multiple most likely candidate sequences when generating each word, ultimately selecting the sequence with the highest overall probability as the output, resulting in a smoother and more accurate result.

[0076] Specifically, the multimodal error correction model in this application adopts an encoder-decoder architecture. The encoder part includes a visual feature encoding branch and a text feature encoding branch. The visual feature encoding branch extracts layout features including the center coordinates, width, height, and relative positional relationship with neighboring regions of the text area. Specifically, a coordinate system is established with the upper left corner of the bill image as the origin, and the center point (x, y), width w, and height h of each segmented region are calculated and normalized to the range of 0 to 1. At the same time, the Euclidean distance, angle, and bounding box overlap between the region and the predefined anchor point region (such as the "amount" region) are calculated to form a 12-dimensional layout feature vector.

[0077] The text feature encoding branch is based on a pre-trained language model for the tax and finance domain. This model, trained on over 100 million tax and finance text data points, possesses strong semantic understanding capabilities. It adopts the BERT architecture with a vocabulary size of 30,000 words, a hidden layer dimension of 768, and 12 layers. The feature vectors of the two branches are mapped to a unified dimension through independent fully connected layers and then concatenated. The decoder uses a gated recurrent unit structure, fusing visual and text features through an attention mechanism. A beam search algorithm is used in the output layer to generate the corrected text sequence, with a beam width of 5. In actual execution, the error correction accuracy reached 98.7% on the test set, with a particularly high accuracy exceeding 99.9% for distinguishing easily confused characters such as the number 0 from the letter O, and the number 1 from the letter I. The error correction process follows tax and finance semantic constraints. For example, when the "amount" field contains the letter "O", the model corrects it to the number "0" based on context and layout features (this area is usually located in the lower right corner of the document and has a larger font). If the "taxpayer identification number" field contains a non-numeric character, it is forcibly replaced with the closest numeric character.

[0078] For example, the "multimodal collaborative error correction" in step S104 of this application embodiment includes the following steps: First, prepare the input data of the model. For text input, the processing system combines the original text sequence (such as "Shenqi") with its field labels into a context sequence, for example, [CLS] company name: Shenqi [SEP]; For visual input, prepare the layout feature vector (such as the 12-dimensional vector defined above) representing the location of the text region.

[0079] Then, the encoder starts working. For the text feature encoding branch, the context sequence [CLS] company name: Shenqi [SEP] is input into the pre-trained financial and tax domain language model. The model outputs a text feature vector containing deep semantics. The model can determine that "Shenqi" is not a common or reasonable word in the context of "company name". For the visual feature encoding branch, the layout feature vector is encoded into a visual feature vector through a fully connected neural network. This vector represents that "this text block is located in the upper left part of the invoice and belongs to the company name area".

[0080] Furthermore, the decoder begins to work, first performing feature fusion. That is, when generating each character, the decoder dynamically "looks up" relevant information from the two feature vectors mentioned above, namely the text and the visual feature vectors, through an attention mechanism.

[0081] Furthermore, the decoder generates the corrected text step by step in an autoregressive manner. Specifically, based on the fused features, the decoder calculates the probabilities of all words in the vocabulary. For the first character, it calculates a high probability for "shen" (深), but there are also certain probabilities for "shang" (上), "guang" (广), "bei" (北), etc.; it selects "shen" as the output and uses "shen" as the input for the next step; when generating the second character, the model combines the already output character "shen", the strong text semantics (knowing that "shen qi" (深期) is unreasonable), and the visual layout (confirming that this is a company name), and calculates that the probability of "zhen" (圳) is much higher than that of "qi" (期); through beam search, it finally determines that "Shenzhen" is the optimal sequence.

[0082] Thus, in step S104, the embodiment of the present application constructs a multimodal error correction model that fuses visual layout and text semantics, elevates the OCR recognition process from "isolated character recognition" to "context understanding and reasoning", can intelligently discover and correct errors that are difficult to detect solely by relying on images or texts, and finally outputs text information that is highly credible in terms of business logic and spatial layout, which is a decisive link to achieve ultra-high accuracy in fiscal and tax data processing.

[0083] In step S105, according to the bill type, the corrected text information is automatically mapped and filled into a predefined structured data template, and the built-in fiscal and tax rule library is called to perform a check on the hook-up relationship of the structured data. If a logical error or numerical anomaly is found, the corresponding data is marked as data to be manually reviewed, and a confidence report is output.

[0084] Among them, the structured data template is a predefined and machine-readable data architecture that stipulates the format and fields of the output data. The predefined structured data template of the embodiment of the present application is dynamically loaded based on the bill type to ensure the pertinence of the output.

[0085] Optionally, in some embodiments, the predefined structured data template is defined in a structured data format, including mandatory fields and optional fields, and the field mapping rules are dynamically loaded based on the bill type.

[0086] It can be understood that unstructured text cannot be directly used by downstream financial software, ERP (Enterprise Resource Planning) systems or declaration systems. Structured data (such as JSON) is the "common language" for inter-system communication. The embodiment of the present application adopts a structured data template to achieve system integration. And through the structured data template, it can be ensured that regardless of the format of the input bill, the output data fields are unified and standardized, which is convenient for subsequent storage, analysis and auditing.

[0087] The built-in financial and tax rule base of this application embodiment is an extensible, coded enterprise financial and tax knowledge base that transforms financial and tax regulations, accounting standards, and business logic into rules that can be executed by computers.

[0088] Optionally, in some embodiments, the built-in tax and finance rule base covers numerical calculation rules, logical association rules, and format specification rules, and the built-in tax and finance rule base supports online updates and incremental learning.

[0089] Optionally, the built-in financial and tax rule base of this application embodiment contains more than 1,000 business rules.

[0090] The verification of the relevance in this application refers to the inherent logical connections and mathematical relationships between different fields in financial and tax data. Verifying these relationships is the "touchstone" for verifying the accuracy of the data.

[0091] Optionally, in some embodiments, the reconciliation verification includes verifying the mathematical relationship between the amount and the tax amount, checking the compliance of the date format, and verifying the taxpayer identification number check digit.

[0092] It is understandable that even if the OCR and multimodal error correction in the above embodiments fail, as long as the data violates the basic business logic (such as amount + tax ≠ total amount), the reconciliation relationship can be verified through the financial and tax rule base of this application to automatically capture the anomaly. In other words, deep errors can be found by verifying the reconciliation relationship through the financial and tax rule base.

[0093] In addition, the process of verifying the reconciliation relationship through the financial and tax rules database simulates the thought process of senior financial personnel when reviewing vouchers, that is, checking whether the internal logic of the data is consistent.

[0094] The confidence report in this application is a comprehensive evaluation document. It does not simply give a binary judgment of "right" or "wrong". Instead, it provides a quantitative confidence score and detailed diagnostic information for each processing result to guide the priority of subsequent manual intervention.

[0095] Optionally, in some embodiments, the confidence report generation module adopts a multi-index weighted evaluation, including: image quality score, text recognition confidence, semantic consistency score and rule compliance, with weights of 0.25, 0.25, 0.3 and 0.2 respectively, and the final confidence score ranges from 0 to 100; when the score is below 80, a manual review process is automatically triggered, and a detailed analysis of the reasons for the anomaly is provided.

[0096] The multi-index weighted evaluation in this application is a decision-making mechanism that integrates evidence from multiple dimensions (such as image quality, recognition confidence, etc.) and assigns different weights to each piece of evidence to calculate a comprehensive score, making the final judgment more scientific and comprehensive.

[0097] Understandably, the confidence level mechanism enables "gray-scale decision-making," meaning high-scoring results are directly approved, while low-scoring results are automatically transferred to manual review. This frees up most of the manpower while ensuring controllable risks. The anomaly analysis in the report directly guides reviewers to the problem, eliminating the need for a complete overhaul and significantly improving the efficiency of manual review.

[0098] Specifically, step S105 of this application embodiment includes the following process: First, data mapping and filling are performed. The processing system loads the corresponding structured template according to the identified invoice type. The corrected text information is automatically filled into the corresponding fields of the template according to the preset mapping rules. For example, the identified string "total price and tax" is filled into the "total_amount" field of the template.

[0099] Next, rule base validation is performed. The processing system calls the rule subset corresponding to the invoice type and scans the populated structured data object. Rule base validation includes: numerical calculation rule validation, logical association rule validation, and format specification rule validation. For example, numerical calculation rule validation automatically executes the following logic code: if (amount_before_tax + tax_amount != total_amount) { throw Exception;}; logical association rule validation checks whether the length and check digits of the "buyer's taxpayer identification number" conform to national standards; and format specification rule validation checks whether the "invoice date" is a legal and reasonable date (not a future date).

[0100] Then, a confidence report and decision are generated. Specifically, according to step S101, the image quality score is obtained. The text recognition confidence score is obtained through the OCR engine and error correction model's understanding of the output text. The semantic consistency score is obtained through the multimodal error correction model based on the degree of matching between the text and the layout semantics. The rule compliance score is obtained through the degree of cross-reference verification. Thus, according to the image quality score, text recognition confidence score, semantic consistency score, rule compliance score, and the weight coefficients corresponding to each indicator, the final comprehensive score can be obtained. For example, the comprehensive score = (image quality score * 0.25) + (text recognition confidence score * 0.25) + (semantic consistency score * 0.3) + (rule compliance score * 0.2).

[0101] Furthermore, if the overall score is greater than or equal to the preset score threshold (e.g., 80 points), the processing system considers the current data to be highly reliable and automatically proceeds to the next stage (e.g., importing into the database); if the overall score is less than the preset score threshold, the processing system marks the data as "pending manual review". Optionally, the system will list the deduction items in detail in the confidence report, such as "blurred image (-5 points)" and "mismatch in tax amount reconciliation (-15 points)".

[0102] Therefore, in step S105, this application achieves data standardization through "template-based output", simulates expert logic review through "rule base verification", and introduces a "quantified confidence mechanism" as an intelligent switch for human-machine collaboration. Ultimately, while ensuring a high degree of automation in the data processing process, it constructs a flexible, reliable and efficient quality control closed loop, ensuring the business availability and extremely high credibility of the output results.

[0103] Optionally, the enterprise financial and tax data processing method based on image recognition in this application embodiment is deployed on a distributed computing platform, adopts a microservice architecture, and the image processing service, text recognition service and rule engine service are deployed independently and communicate asynchronously through message queues; the control system supports horizontal scaling, and a single cluster can concurrently process 1,000 invoice images.

[0104] Specifically, the enterprise financial and tax data processing method based on image recognition in this application can decouple the complex processing flow into independently manageable and elastically scalable components through microservice architecture and asynchronous message communication, thereby building a robust system platform that can cope with enterprise-level high concurrency and high load requirements, while having high availability and good maintainability.

[0105] Optionally, the enterprise financial and tax data processing method based on image recognition in this application embodiment also includes a continuous learning mechanism, which collects manual review results and user feedback, and regularly updates the parameters of the lightweight convolutional neural network, the weights of the object detection model, and the pre-trained financial and tax domain language model, with each model updating every 7 days.

[0106] Specifically, the processing system automatically collects all invoice images marked "awaiting manual review," the model's original output, and the final correct results confirmed by financial personnel, thus forming a high-quality error-correct sample pair dataset. Simultaneously, it collects correction feedback submitted by users through the interface. Furthermore, this embodiment uses a new dataset to incrementally train (fine-tune) the existing lightweight convolutional neural network, object detection model, and pre-trained financial and tax domain language model, respectively, instead of training from scratch, to save resources and retain existing knowledge.

[0107] Therefore, the enterprise financial and tax data processing method / processing system integrating the image recognition based embodiment of this application can proactively adapt to changes in the business environment and continuously correct its own errors, thereby achieving a leap from "static tool" to "dynamic intelligent agent" and ensuring the long-term effectiveness and vitality of the solution.

[0108] To enable those skilled in the art to further understand the enterprise financial and tax data processing method based on image recognition according to the embodiments of this application, the following specific embodiments will be listed to illustrate the technical effects of the method.

[0109] In this embodiment, a large manufacturing enterprise needs to process approximately 2,000 invoices of various types daily, of which VAT invoices account for 60%, expense reimbursement forms account for 25%, and other invoices account for 15%. The processing system integrating the enterprise financial and tax data processing method based on image recognition implemented in this application is deployed on a private cloud platform and configured with 3 server nodes, each node equipped with an NVIDIA T4 GPU (NVIDIA's high-performance inference graphics card for data centers and edge computing).

[0110] The invoice processing flow on a certain day is as follows: First, the scanner or mobile APP uploads the invoice image to the processing system entry point. In steps 1 and 2, a lightweight CNN model completes the quality assessment within 50 milliseconds, outputting a VAT invoice with a score of 0.92 to enter the fast recognition channel, and a fuzzy reimbursement form with a score of 0.38 to enter the fine recognition channel.

[0111] In step 3, the target detection model accurately identified the invoice type as "value-added tax special invoice", and the segmentation network accurately extracted 12 key regions, among which the IoU (Intersection over Union) of the amount region segmentation reached 0.97.

[0112] In step 4, the OCR engine initially identifies "total price and tax (lowercase)" as "¥1130.00". The multimodal error correction model combines the layout and semantic information such as the area being located in the lower right corner of the invoice, the font being Arial Bold, and the adjacent "tax amount" field being "130.00" to determine that the last character should be the number "0", and corrects it to "¥1130.00".

[0113] In step 5, the processing system maps the data to an XML template and verifies that "Amount 1000.00 + Tax 130.00 = Total Price and Tax 1130.00" is true, the taxpayer identification number verification bit is correct, the final confidence score is 96.5, and the data is automatically entered into the database without manual intervention.

[0114] For a reimbursement form partially obscured due to folding, the quality score was 0.42. After entering the fine recognition channel, through super-resolution reconstruction and perspective correction, the segmentation network successfully restored the obscured "amount" area, and the OCR recognized it as "850.00". Based on the business common sense that reimbursement forms usually have integer amounts without letters, the multimodal error correction model corrected it to "850.00". However, due to the low confidence level of the date field recognition (only 0.65), the final confidence score was 76.3. The processing system automatically marked it as pending review and pushed it to the finance staff's workbench, along with an error message: "The confidence level of the date field recognition is too low. Please confirm whether 'May 12, 2023' should be 'May 12, 2023'."

[0115] This application proposes an image recognition-based method for processing enterprise financial and tax data. It employs a lightweight neural network to assess the quality of financial and tax invoice images and dynamically allocates them to different processing channels. Object detection and attention mechanisms are used to segment the network for precise localization and extraction of key information regions. A multimodal error correction model is constructed, combining the original text recognized by the character recognition engine with image layout features and semantic logic from the financial and tax domain for collaborative verification and intelligent error correction. The corrected information is automatically filled into a structured template, and after verification using a built-in rule base, the data results are output or anomalies are marked for manual review. This method solves problems such as rigid enterprise financial and tax data processing workflows, limited template information, and poor adaptability to complex scenarios, thereby improving the overall efficiency, accuracy, and robustness of financial and tax invoice processing.

[0116] Next, referring to the accompanying drawings, an image recognition-based enterprise financial and tax data processing device is described according to an embodiment of this application.

[0117] Figure 2 This is a block diagram of an image recognition-based enterprise financial and tax data processing device according to an embodiment of this application.

[0118] like Figure 2 As shown, the enterprise financial and tax data processing device 10 based on image recognition includes: an image quality assessment module 100, a dynamic channel scheduling module 200, a document positioning and region segmentation module 300, a multimodal text error correction module 400, and a structured data verification module 500.

[0119] Specifically, the image quality assessment module 100 is used to receive the tax invoice image to be processed, perform quality assessment on the tax invoice image through a lightweight convolutional neural network, and output a comprehensive quality score. The comprehensive quality score reflects the clarity, noise level and integrity of the tax invoice image. The dynamic channel scheduling module 200 is used to dynamically schedule financial and tax invoice images to different processing channels based on the comprehensive quality score. The invoice localization and region segmentation module 300 uses an object detection model to locate and identify tax invoice images in each processing channel, obtains the layout features and invoice type of the tax invoice images, and uses a segmentation network with an attention mechanism to accurately segment the key information regions in the tax invoice images. The multimodal text correction module 400 is used to perform text recognition on each segmented region using a text recognition engine to obtain the original text sequence. At the same time, a multimodal error correction model is constructed. The multimodal error correction model receives the original text sequence and layout features, uses a pre-trained language model in the financial and tax domain to perform semantic understanding on the original text sequence, and performs verification and error correction on the original text sequence based on layout features and semantic logic, and outputs the corrected text information. The structured data verification module 500 is used to automatically map and fill the corrected text information into a predefined structured data template according to the type of invoice. It calls the built-in financial and tax rule library to verify the consistency of the structured data. If logical errors or numerical anomalies are found, the corresponding data is marked as data to be manually reviewed and a confidence report is output.

[0120] It should be noted that the foregoing explanation of an embodiment of an image recognition-based enterprise financial and tax data processing method also applies to an image recognition-based enterprise financial and tax data processing device of the same embodiment, and will not be repeated here.

[0121] According to an embodiment of this application, an image recognition-based enterprise financial and tax data processing device includes: an image quality assessment module that uses a lightweight neural network to assess the quality of financial and tax invoice images; a dynamic channel scheduling module that dynamically schedules financial and tax invoice images to different processing channels; an invoice localization and region segmentation module that uses object detection and attention mechanisms to segment networks to accurately locate and extract key information regions; a multimodal text correction module that constructs a multimodal error correction model that combines the original text recognized by the text recognition engine with image layout features and semantic logic in the financial and tax domain for collaborative verification and intelligent error correction; and a structured data verification module that automatically fills the corrected information into a structured template, verifies the consistency of the data through a built-in rule base, and outputs data results or marks anomalies for manual review. This solves the problems of rigid enterprise financial and tax data processing workflows, limited template information, and poor adaptability to complex scenarios, thereby improving the overall efficiency, accuracy, and robustness of financial and tax invoice processing.

[0122] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0123] When the processor 302 executes the program, it implements the enterprise financial and tax data processing method based on image recognition provided in the above embodiments.

[0124] Furthermore, electronic devices also include: Communication interface 303 is used for communication between memory 301 and processor 302.

[0125] The memory 301 is used to store computer programs that can run on the processor 302.

[0126] The memory 301 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0127] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0128] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0129] Processor 302 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0130] This application also provides a computer program product on which a computer program is stored, which, when executed by a processor, implements the above-described image recognition-based enterprise financial and tax data processing method.

[0131] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0133] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0134] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0135] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0136] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for processing enterprise financial and tax data based on image recognition, characterized in that, Includes the following steps: The system receives images of tax invoices to be processed, performs quality assessment on the images using a lightweight convolutional neural network, and outputs a comprehensive quality score. The comprehensive quality score reflects the sharpness, noise level, and integrity of the tax invoice images. Based on the comprehensive quality score, the tax invoice images are dynamically scheduled to different processing channels; In each processing channel, an object detection model is used to locate and identify the tax invoice image to obtain the layout features and invoice type of the tax invoice image, and a segmentation network with an attention mechanism is used to accurately segment the key information regions in the tax invoice image. The segmented regions are processed by an image recognition engine to obtain the original text sequence. At the same time, a multimodal error correction model is constructed. The multimodal error correction model receives the original text sequence and the layout features, uses a pre-trained financial and tax domain language model to perform semantic understanding on the original text sequence, and performs verification and error correction on the original text sequence based on the layout features and semantic logic, and outputs the corrected text information. Based on the invoice type, the corrected text information is automatically mapped and filled into a predefined structured data template. The built-in financial and tax rule library is called to verify the consistency of the structured data. If logical errors or numerical anomalies are found, the corresponding data is marked as data to be manually reviewed, and a confidence report is output.

2. The enterprise financial and tax data processing method based on image recognition according to claim 1, characterized in that, The step of dynamically scheduling the tax invoice images to different processing channels includes: If the overall quality score is higher than the set high threshold, the tax invoice image will be dynamically scheduled to a fast recognition channel equipped with a basic text recognition engine and simple post-processing. If the overall quality score is lower than the set low threshold, the tax invoice image will be dynamically scheduled to a fine recognition channel that includes image enhancement operations. If the overall quality score is higher than the set low threshold and lower than the set high threshold, the tax invoice image will be dynamically scheduled to the standard recognition channel configured with standard image preprocessing operations.

3. The enterprise financial and tax data processing method based on image recognition according to claim 2, characterized in that, The image enhancement operations in the fine recognition channel include one or more of the following: super-resolution reconstruction based on generative adversarial networks, multi-scale denoising, and perspective transformation correction.

4. The enterprise financial and tax data processing method based on image recognition according to claim 1, characterized in that, The lightweight convolutional neural network adopts a depthwise separable convolutional structure, which includes 8 convolutional layers and 3 fully connected layers. Among them, the sharpness assessment uses the Laplacian variance algorithm, the noise level assessment uses the block signal-to-noise ratio calculation, and the integrity assessment is based on the combined results of image edge detection and template matching.

5. The enterprise financial and tax data processing method based on image recognition according to claim 1, characterized in that, The target detection model adopts a single-stage detection architecture, with a deep residual network as the backbone network and a feature pyramid network for multi-scale feature fusion. The segmentation network that introduces an attention mechanism integrates a channel attention module and a spatial attention module in the encoder part, and the decoder adopts a skip connection structure, achieving pixel-level segmentation accuracy.

6. The enterprise financial and tax data processing method based on image recognition according to claim 1, characterized in that, The multimodal error correction model adopts an encoder-decoder architecture, with the encoder part including a visual feature encoding branch and a text feature encoding branch; The layout features extracted by the visual feature encoding branch include the center coordinates, width, height, and relative positional relationship of the text region with neighboring regions; The text feature encoding branch uses the pre-trained language model in the fiscal and tax domain to perform semantic understanding of the text sequence; The decoder adopts a gated recurrent unit structure, fuses visual features and text features through an attention mechanism, and uses a beam search algorithm in the output layer to generate the corrected text information.

7. The enterprise financial and tax data processing method based on image recognition according to claim 1, characterized in that, The predefined structured data template is defined using a structured data format, including required fields and optional fields, and the field mapping rules are dynamically loaded based on the invoice type; The built-in tax and finance rule library covers numerical calculation rules, logical association rules, and format specification rules, and the built-in tax and finance rule library supports online updates and incremental learning. The verification of the reconciliation relationship includes verification of the mathematical relationship between the amount and the tax amount, checking the compliance of the date format, and verifying the taxpayer identification number.

8. A device for processing enterprise financial and tax data based on image recognition, characterized in that, include: The image quality assessment module is used to receive the tax invoice image to be processed, perform quality assessment on the tax invoice image through a lightweight convolutional neural network, and output a comprehensive quality score, which reflects the sharpness, noise level and integrity of the tax invoice image. The dynamic channel scheduling module is used to dynamically schedule the financial and tax invoice images to different processing channels based on the comprehensive quality score. The invoice location and region segmentation module uses an object detection model to locate and identify the tax invoice image in each processing channel, obtains the layout features and invoice type of the tax invoice image, and uses a segmentation network with an attention mechanism to accurately segment the key information region in the tax invoice image. The multimodal text correction module is used to perform text recognition on each segmented region using a text recognition engine to obtain the original text sequence. At the same time, it constructs a multimodal error correction model. The multimodal error correction model receives the original text sequence and the layout features, uses a pre-trained financial and tax domain language model to perform semantic understanding on the original text sequence, and performs verification and error correction on the original text sequence based on the layout features and semantic logic, and outputs the corrected text information. The structured data verification module is used to automatically map and fill the corrected text information into a predefined structured data template according to the invoice type, call the built-in financial and tax rule library to verify the consistency of the structured data, and if logical errors or numerical anomalies are found, the corresponding data is marked as data to be manually reviewed and a confidence report is output.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an image recognition-based enterprise financial and tax data processing method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement an image recognition-based enterprise financial and tax data processing method as described in any one of claims 1-7.

Images