Artificial intelligence-based automatic financial document information input system

Abstract

The application belongs to the field of intelligent ticket management, and specifically relates to a financial ticket information automatic input system based on artificial intelligence, which comprises a ticket collection module, a multi-scale self-adaptive noise reduction module, an intelligent classification module and an intelligent partition input module; the application adopts a feature double-domain self-adaptive enhancement method based on transpose self-attention, improves noise reduction robustness, specifically processes different noise types, overcomes the limitation of a single domain, restores the global structure while retaining the details, increases the noise reduction capability of the ticket image, and improves the text detection efficiency; the application adopts a document intelligent partition method, unifies heterogeneous tasks into the combination of instance and semantic segmentation, avoids model redundancy, maps the category name into a semantic query, supports open set classification and zero sample migration, realizes the dynamic interaction of instance and semantic query through hybrid query, enhances the understanding of the model to the complex document structure, and realizes efficient ticket image partition information extraction and input.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of intelligent ticket management, and specifically relates to a financial ticket information automatic input system based on artificial intelligence. BACKGROUND

[0002] With the digitization of enterprise finance and tax, paper tickets need to be converted into electronic data through scanning, OCR (optical character recognition) and other technologies for system processing. Manual input is inefficient and prone to errors, and automated input technology can improve efficiency and accuracy. Existing information input systems rely on rule templates to locate fields (such as fixed coordinate amount extraction), which cannot be identified when the ticket format changes, and the text detection efficiency and quality are low under complex backgrounds or noise interference. SUMMARY

[0003] To overcome the defects of the prior art, the present application provides a financial ticket information automatic input system based on artificial intelligence. To solve the problem of low text detection efficiency and quality of existing information input systems under complex backgrounds or noise interference, the application creatively adopts a feature dual-domain adaptive enhancement method based on transpose self-attention. The image pyramid uses the clean context information of low-resolution images to improve the noise reduction robustness. The learnable mask dynamically separates the high-frequency (details / noise) and low-frequency (structure) components, and processes different noise types. Through dual-domain feature deep fusion, the limitations of traditional methods relying on a single domain are overcome, and cross-scale feature enhancement is performed on the image to preserve details while restoring the global structure, increasing the system's noise reduction capability for ticket images and improving text detection efficiency. For the problem that existing information input systems rely on rule templates to locate fields (such as fixed coordinate amount extraction), which cannot be identified when the ticket format changes, the application adopts a document intelligent partitioning method based on Transformer, which unifies heterogeneous tasks (layout analysis, text detection, table recognition) into a combination of instance and semantic segmentation, avoids redundant special models, maps category names to semantic queries using Sentence-BERT, supports open set classification and zero-shot transfer, and realizes dynamic interaction between instance and semantic queries through hybrid queries, enhances the model's understanding of complex document structures, and finally realizes efficient ticket image partition information extraction and input.

[0004] The technical solutions adopted by the application are as follows: The financial ticket information automatic input system based on artificial intelligence provided by the application includes a ticket collection module, a multi-scale adaptive noise reduction module, an intelligent classification module and an intelligent partition input module.

[0005] The bill acquisition module comprises a scanner and an image preprocessing unit, the scanner scans the financial bill and acquires a bill image, and the image preprocessing unit removes the background of the bill image by using a U-Net segmentation network;

[0006] The multi-scale adaptive denoising module removes the background of the bill image to obtain a denoised image, and comprises a multi-scale image construction unit, a primary feature extraction unit, an adaptive learning unit and an image denoising unit;

[0007] The multi-scale image construction unit performs an interpolation operation on the bill image from which the background is removed to generate multi-scale images of different scales;

[0008] The primary feature extraction unit performs primary feature extraction on all multi-scale images by a convolution layer to obtain primary features;

[0009] The adaptive learning unit enhances the primary features by using a feature dual-domain adaptive enhancement method based on transposed self-attention to obtain enhanced features;

[0010] The feature dual-domain adaptive enhancement method based on transposed self-attention specifically comprises the following steps:

[0011] Step S1: after the primary features are subjected to convolution and ReLU activation, the primary features are converted to a frequency domain by fast Fourier transform to obtain frequency domain features:

[0012] ;

[0013] In the formula, represents the primary features, represents a convolution operation, represents an activation operation, represents fast Fourier transform, represents the frequency domain features;

[0014] Step S2: the high-frequency component and the low-frequency component of the frequency domain features are separated by using a learnable mask Mask;

[0015] Step S3: the high-frequency component and the low-frequency component are respectively subjected to inverse Fourier transform, multilayer perceptron processing and transposed self-attention enhancement to obtain high-frequency features and low-frequency features:

[0016] ;

[0017] In the formula, represents the high-frequency component and the low-frequency component, represents inverse Fourier transform, represents multilayer perceptron processing, represents transposed self-attention enhancement, represent high-frequency features and low-frequency features;

[0018] Step S4: merging the high-frequency features and the low-frequency features to obtain enhanced frequency features, using transposed self-attention to capture long-range dependencies between channels of the enhanced frequency features to obtain adaptive spatial frequency enhanced features;

[0019] Step S5: performing gated feed-forward network processing on the adaptive spatial frequency enhanced features to obtain enhanced features;

[0020] The image denoising unit generates residual images through convolution of the enhanced features of each scale, and adds all the residual images to the bill image without background to obtain a denoised image.

[0021] The intelligent classification module includes a primary classification unit and a secondary classification unit. The primary classification unit classifies the denoised image of the bill to divide the bill into categories, including invoices, receipts, contracts, and bank documents. The secondary classification unit further subdivides the categories of the bill.

[0022] The intelligent partitioning and inputting module uses a document intelligent partitioning method based on Transformer to intelligently partition and input information of the bill.

[0023] Further, in the intelligent partitioning and inputting module, a document intelligent partitioning method based on Transformer is used to intelligently partition and input information of the bill, which specifically includes the following steps:

[0024] Step Q1: multi-scale feature encoding based on vision, using Swin-Transformer to extract multi-scale features of the denoised image of the bill, and fusing the multi-scale features into mask features through an FPN network;

[0025] Step Q2: semantic query and instance query, using a text extraction technique to extract all the table header texts of the denoised image, using Sentence-BERT to map the table header texts into semantic vectors, i.e., semantic queries, and randomly initializing learnable instance position embedding parameters for all table headers, i.e., instance queries;

[0026] Step Q3: information interaction, performing information interaction on the semantic queries and the instance queries through a cooperative attention mechanism to obtain semantic interaction features and instance interaction features;

[0027] Step Q4: decoding feature acquisition, performing feature decoding on the semantic interaction features and the instance interaction features with the multi-scale features through cross-scale attention to obtain semantic refined features and instance refined features;

[0028] Step Q5: Mask prediction, matrix multiplication is performed between the semantic refinement feature and the instance refinement feature and the mask feature respectively and activation is performed, to obtain semantic mask and instance mask;

[0029] Step Q6: Class prediction, similarity calculation is performed on the semantic refinement feature and the instance refinement feature, and normalization processing is performed using a softmax function, to obtain class scores, a score threshold is set, and instance queries with class scores lower than the threshold are removed;

[0030] Step Q7: Initial bounding box initialization, for each instance query, an initial bounding box is calculated according to the instance mask corresponding thereto;

[0031] Step Q8: Bounding box coordinate calculation, the instance query and the mask feature are fused, and the instance position embedding coordinates of the bounding box are predicted through a fully connected layer, the initial bounding box is adjusted according to the instance position embedding coordinates, and the table bounding box partition result of the bill denoising image is obtained;

[0032] Step Q9: Information entry, according to the partition result, a character recognition extraction model is used to extract and store the information in the partition.

[0033] The beneficial results obtained by the above scheme are as follows:

[0034] (1) In view of the problem that the existing information entry system has low text detection efficiency and quality under complex background or noise interference, the application creatively adopts a feature dual-domain self-adaptive enhancement method based on transpose self-attention, uses the clean context information of low-resolution images through image pyramid to improve the denoising robustness, dynamically separates high-frequency (details / noise) and low-frequency (structure) components through a learnable mask, processes different noise types, overcomes the limitation of traditional methods which only rely on a single domain, and performs cross-scale feature enhancement on the image, retains details while restoring the global structure, increases the denoising ability of the system for bill images, and improves the efficiency of text detection;

[0035] (2) In view of the problem that the existing information entry system relies on rule template positioning fields (such as fixed coordinate extraction of amount), and cannot recognize when the bill format changes, the application adopts a document intelligent partition method based on Transformer, unifies heterogeneous tasks (layout analysis, text detection, table recognition) into a combination of instance and semantic segmentation, avoids special model redundancy, maps category names to semantic queries using Sentence-BERT, supports open set classification and zero-shot transfer, realizes dynamic interaction between instance and semantic queries through hybrid queries, enhances the understanding of the model for complex document structures, and finally realizes efficient information extraction and entry of bill image partition. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A module diagram of the financial bill information automatic input system based on artificial intelligence provided by the present application is provided.

[0037] Figure 2 A flowchart of a document intelligent partitioning method based on a Transformer is provided.

[0038] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0040] Embodiment one, refer to Figure 1 The financial bill information automatic input system based on artificial intelligence provided by the present application comprises a bill collection module, a multi-scale adaptive noise reduction module, an intelligent classification module and an intelligent partitioning input module.

[0041] The bill collection module comprises a scanner and an image preprocessing unit. The scanner scans the financial bill and obtains a bill image. The image preprocessing unit removes the background of the bill image by using a U-Net segmentation network.

[0042] The multi-scale adaptive noise reduction module reduces the noise of the bill image from which the background is removed to obtain a noise reduction image, and comprises a multi-scale image construction unit, a primary feature extraction unit, an adaptive learning unit and an image noise reduction unit.

[0043] The intelligent classification module comprises a primary classification unit and a secondary classification unit. The primary classification unit classifies the noise reduction image of the bill to divide the bill into categories, including invoices, receipts, contracts and bank documents. The secondary classification unit classifies the categories of the bill according to subcategories, for example, invoices can be divided into value-added tax invoices, electronic general invoices, air ticket itineraries, etc.

[0044] The intelligent partitioning input module uses a document intelligent partitioning method based on a Transformer to intelligently partition and input information of the bill.

[0045] Embodiment two, this embodiment is based on the above-mentioned embodiment, and in the multi-scale adaptive noise reduction module:

[0046] The multi-scale image construction unit performs an interpolation operation on the background-removed bill image to generate multi-scale images of different scales;

[0047] The primary feature extraction unit performs primary feature extraction on all multi-scale images through a convolution layer to obtain primary features;

[0048] The adaptive learning unit adopts a feature dual-domain adaptive enhancement method based on transposed self-attention to enhance the primary features to obtain enhanced features;

[0049] The image denoising unit generates residual images through convolution of the enhanced features of each scale, and adds all residual images to the background-removed bill image to obtain a denoised image.

[0050] Embodiment Three, based on the above-mentioned embodiments, in the adaptive learning unit, a feature dual-domain adaptive enhancement method based on transposed self-attention, specifically includes the following steps:

[0051] Step S1: After the primary features are convolved and activated by ReLU, they are converted to the frequency domain through fast Fourier transform to obtain frequency domain features:

[0052] ;

[0053] In the formula, represents the primary features, represents the convolution operation, represents the activation operation, represents the fast Fourier transform, represents the frequency domain features;

[0054] Step S2: Use a learnable mask Mask to separate the high-frequency component and the low-frequency component of the frequency domain features;

[0055] Step S3: Perform inverse Fourier transform, multilayer perceptron processing, and transposed self-attention enhancement on the high-frequency component and the low-frequency component respectively to obtain high-frequency features and low-frequency features:

[0056] ;

[0057] In the formula, represents the high-frequency component and the low-frequency component, represents the inverse Fourier transform, represents the multilayer perceptron processing, represents the transposed self-attention enhancement, represents the high-frequency features and the low-frequency features;

[0058] Step S4: merge the high-frequency features and the low-frequency features to obtain enhanced frequency features, use transpose self-attention to capture the long-range dependence between channels of the enhanced frequency features, and obtain adaptive spatial frequency enhanced features;

[0059] Step S5: performing gated feed-forward network processing on the adaptive spatial frequency enhanced features to obtain enhanced features.

[0060] Embodiment four, see Figure 2 The embodiment is based on the above-mentioned embodiment. In the intelligent partition input module, a document intelligent partition method based on Transformer includes the following steps:

[0061] Step Q1: use Swin-Transformer to extract multi-scale features of the denoised image of the bill, and fuse the multi-scale features into mask features through FPN network;

[0062] Step Q2: extract all the table header text of the denoised image using the text extraction technology, map the table header text into semantic vectors using Sentence-BERT, i.e. semantic query, and randomly initialize the learnable instance position embedding parameters for all table headers, i.e. instance query;

[0063] Step Q3: information interaction between semantic query and instance query is performed through cooperative attention mechanism to obtain semantic interaction features and instance interaction features;

[0064] Step Q4: the semantic interaction features and the instance interaction features are respectively decoded with the multi-scale features through cross-scale attention to obtain semantic refinement features and instance refinement features;

[0065] Step Q5: the semantic refinement features and the instance refinement features are respectively multiplied with the mask features and activated to obtain semantic masks and instance masks;

[0066] Step Q6: similarity calculation is performed on the semantic refinement features and the instance refinement features, and the class scores are obtained by using the softmax function for normalization processing. A score threshold is set, and the instance queries with class scores lower than the threshold are removed;

[0067] Step Q7: for each instance query, the initial bounding box is calculated according to the corresponding instance mask;

[0068] Step Q8: fuse the instance query and the mask feature, and predict the instance position embedding coordinates of the bounding box through the full connection layer. Adjust the coordinates of the initial bounding box according to the instance position embedding coordinates to obtain the table bounding box partition result of the denoised image of the bill;

[0069] Step Q9: According to the partition result, the text recognition extraction model is used to extract and store the information in the partition.

[0070] Example Five, based on the above examples, the multi-scale adaptive noise reduction module is applied to the actual application of invoice image noise reduction:

[0071] Input image:

[0072] The invoice contaminated by scanning noise contains Gaussian noise and salt and pepper noise (simulating low-quality scanner output);

[0073] Resolution: 600 dpi, grayscale image;

[0074] Processing flow:

[0075] Multi-scale input: Generate 0.5x and 0.25x resolution pyramids, low-resolution images are used to capture clear table structure;

[0076] Frequency domain separation: high frequency branch: enhance the strokes of the text "invoice number: 2024-001";

[0077] Low-frequency branch: smooth background noise, preserve the red area of the seal;

[0078] Output result: text PSNR improved by 12dB, background SSIM improved from 0.65 to 0.92.

[0079] Example Six, based on the above examples, the multi-scale adaptive noise reduction module is applied to the actual application of handwritten check image noise reduction:

[0080] Input image:

[0081] Handwritten check, with ink diffusion and paper wrinkle noise;

[0082] Key steps:

[0083] (1) Adaptive mask learning:

[0084] Ink diffusion area is identified as high-frequency noise and suppressed by mask;

[0085] Wrinkle texture as low-frequency noise, corrected by MLP of low-frequency branch;

[0086] (2) Space-frequency domain collaboration:

[0087] Enhance the continuity of handwritten "￥5000.00" through a feature dual-domain adaptive enhancement method based on transpose self-attention.

[0088] Example Seven, based on the above examples, the intelligent partition entry module is applied to the actual application of value-added tax invoice image:

[0089] enter:

[0090] Scanned image of a VAT invoice (600dpi, PNG format);

[0091] Processing flow:

[0092] (1) Partitioning stage (a document intelligent partitioning method based on Transformer)

[0093] Optimize semantic query configuration:

[0094] invoice_queries=["invoice_code","invoice_number","seller_name","tax_amount","total_amount","goods_table"]

[0095] Dynamic segmentation: After detecting the goods_table area, a 640×640 sliding window (overlap rate 30%) is enabled.

[0096] Output:

[0097] {"type":"invoice_code","bbox":[120,45,280,80],"confidence":0.98},

[0098] {"type":"goods_table","bbox":[30,320,550,700],"tiles":[{"coord":[x1,y1,x2,y2], "page": 1}]}

[0099] (2) Information extraction stage

[0100] Text field: The TroOCR model recognizes invoice codes (accuracy rate 99.7%).

[0101] Table fields: TableNet parses the goods list → outputs a structured CSV:

[0102] Serial number, name, specifications, quantity, unit price, amount;

[0103] 1. Laptop, ThinkPad X1, 2, 8999, 17998.

[0104] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other present or future technologies can provide. Specifically, it is contemplated that some embodiments of the present application can be practiced without the specific details set forth in the preceding description.

[0105] While the embodiments of the application have been illustrated and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the spirit and scope of the application, which is defined by the appended claims and their equivalents.

[0106] The above description of the application and its embodiments is not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the application.

Claims

1. An AI-based automatic financial invoice information entry system, characterized by: It includes a ticket acquisition module, a multi-scale adaptive noise reduction module, an intelligent classification module, and an intelligent partitioning input module; The invoice acquisition module includes a scanner and an image preprocessing unit. The scanner scans financial invoices and acquires invoice images. The image preprocessing unit uses a U-Net segmentation network to remove the background of the invoice images. The multi-scale adaptive denoising module denoises the background-removed ticket image to obtain a denoised image, including a multi-scale image construction unit, a primary feature extraction unit, an adaptive learning unit, and an image denoising unit. In the multi-scale adaptive noise reduction module, the multi-scale image construction unit performs interpolation on the background-removed ticket image to generate multi-scale images of different scales. The primary feature extraction unit performs primary feature extraction on all multi-scale images through convolutional layers to obtain primary features. The adaptive learning unit employs a feature dual-domain adaptive enhancement method based on transposed self-attention to enhance the primary features and obtain enhanced features. In the adaptive learning unit, a feature dual-domain adaptive enhancement method based on transposed self-attention specifically includes the following steps: Step S1: After convolution and ReLU activation, the primary features are transformed to the frequency domain by fast Fourier transform to obtain frequency domain features; Step S2: Use a learnable mask to separate the high-frequency and low-frequency components of the frequency domain features; Step S3: Perform inverse Fourier transform, multilayer perceptron processing, and transpose self-attention enhancement on the high-frequency and low-frequency components respectively to obtain high-frequency features and low-frequency features. Step S4: Merge high-frequency and low-frequency features to obtain enhanced frequency features, and use transpose self-attention to capture the long-range inter-channel dependence of the enhanced frequency features to obtain adaptive spatial frequency enhancement features; Step S5: Gated feedforward network processing is applied to the adaptive spatial frequency enhancement features to obtain the enhanced features; The image denoising unit generates residual images by convolution of the enhancement features at each scale, and adds all residual images to the background-removed ticket image to obtain the denoised image. The intelligent classification module includes a primary classification unit and a secondary classification unit. The primary classification unit performs basic classification on the denoised images of the invoices, dividing the invoices into major categories, including invoices, receipts, contracts, and bank documents. The secondary classification unit further classifies the major categories of invoices into subcategories. The intelligent partitioning and data entry module uses a Transformer-based intelligent document partitioning method to intelligently partition and input information into invoices. In the intelligent partitioning input module, a document intelligent partitioning method based on Transformer specifically includes the following steps: Step Q1: Use Swin-Transformer to extract multi-scale features from the denoised image of the ticket, and fuse the multi-scale features into mask features through FPN network; Step Q2: Use text extraction technology to extract all the header text in the denoised image, use Sentence-BERT to map the header text into semantic vectors, i.e. semantic query, and randomly initialize learnable instance position embedding parameters for all headers, i.e. instance query; Step Q3: Use a collaborative attention mechanism to perform information interaction between semantic queries and instance queries to obtain semantic interaction features and instance interaction features; Step Q4: Use cross-scale attention to decode the semantic interaction features and instance interaction features with the multi-scale features to obtain semantic refinement features and instance refinement features. Step Q5: Perform matrix multiplication and activation on the semantic refinement features and instance refinement features with the mask features respectively to obtain the semantic mask and instance mask; Step Q6: Calculate the similarity between the semantic refinement features and the instance refinement features, and use the softmax function to normalize them to obtain the category score. Set a score threshold and remove instance queries with category scores below the threshold. Step Q7: For each instance, query and calculate the initial bounding box based on its corresponding instance mask; Step Q8: Fuse the instance query and mask features, and predict the instance location embedding coordinates of the bounding box through a fully connected layer. Adjust the coordinates of the initial bounding box according to the instance location embedding coordinates to obtain the table bounding box partitioning result of the invoice denoising image. Step Q9: Based on the partitioning results, use a text recognition extraction model to extract and store the information within each partition.

Images