An automatic financial document information input system based on optical character recognition

Abstract

The application belongs to the field of optical character recognition, and particularly discloses a financial bill information automatic input system based on optical character recognition, which comprises a bill image acquisition module, a bill image correction module, a bill image preprocessing module and a bill information input module. Through precise preprocessing based on linear attention, the scheme adopts a two-stage training strategy of pre-training and fine-tuning to realize efficient removal of bill image noise; through optimization of the optical character recognition and input process, the Transformer model is integrated as a text recognizer, the local features of each character are accurately extracted through the sliding window technology, the joint loss function combining the reconstruction loss and the recognition loss is adopted, the image reconstruction capability and the text recognizer are deeply fused for joint fine-tuning, and the synchronous optimization of the image quality and the recognition accuracy is realized.

Description

Technical Field

[0001] This invention relates to the field of optical character recognition, specifically to an automatic financial document information entry system based on optical character recognition. Background Technology

[0002] Automatic entry of financial document information utilizes OCR technology to convert text, numbers, symbols, and other information on various financial documents into text data that computers can recognize and process, and automatically enters it into relevant databases, reducing human error. However, in the actual collection process of financial documents, due to multiple factors such as scanning equipment, ambient lighting, and the physical condition of the documents themselves, there are generally problems such as severe noise interference, tilting and deformation, wrinkles and occlusion, and loss of low-resolution details. These problems combine to significantly reduce the text feature recognition accuracy, seriously affecting the efficiency and accuracy of financial data processing. At the same time, traditional optical character recognition methods have limited recognition accuracy, a high error rate, and insufficient data integration capabilities, requiring manual verification and entry, which is not only time-consuming and labor-intensive, but also prone to entry errors due to human negligence, failing to meet the core requirements of automation and efficiency in financial work. Summary of the Invention

[0003] To address the aforementioned issues and overcome the shortcomings of existing technologies, this invention provides an automatic financial document information entry system based on optical character recognition. Addressing the common problems encountered during the actual acquisition of financial documents—including severe noise interference, tilting and distortion, wrinkles and occlusion, and loss of low-resolution details—caused by multiple factors such as scanning equipment, ambient light, and the physical condition of the document itself, the system resolves these issues. These problems, when combined, significantly reduce text feature recognition, severely impacting the efficiency and accuracy of financial data processing. This solution utilizes precise preprocessing based on linear attention to achieve accurate text feature focusing. A two-stage training strategy of pre-training and fine-tuning is employed. Forward diffusion simulates noise pollution, time-step embedding captures temporal features, noise prediction updates model parameters, and backward diffusion is used for gradual denoising and residual learning to achieve efficient removal of noise from document images, significantly improving image clarity and character recognition. This solution ensures complete preservation of key information on invoices. Addressing the limitations of traditional optical character recognition (OCR) methods—limited accuracy, high error rates, insufficient data integration capabilities, and reliance on manual verification and entry (which is time-consuming, labor-intensive, and prone to human error)—this solution optimizes the OCR and entry process. It integrates a Transformer model as the text recognizer, uses sliding window technology to accurately extract local features of each character, and employs a joint loss function combining reconstruction and recognition losses. This deep integration and fine-tuning of image reconstruction capabilities and the text recognizer achieves simultaneous optimization of image quality and recognition accuracy, minimizing human error rates. Furthermore, it is compatible with various formats and types of financial invoices, significantly improving data entry efficiency and shortening the financial data processing cycle.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides an automatic financial document information entry system based on optical character recognition. The automatic financial document information entry system based on optical character recognition includes a document image acquisition module, a document image correction module, a document image preprocessing module, and a document information entry module.

[0005] The document image acquisition module uses a scanning device to acquire document text images and records the acquisition parameters of the document text images, including resolution, supplementary lighting mode, and scanning time.

[0006] The ticket image correction module detects the edge lines of the ticket based on Hough transform, calculates the tilt angle, and uses interpolation rotation correction for slight tilt and block rotation and edge cropping correction for large tilt to avoid image stretching distortion. It also identifies the folded areas of the ticket through gray value gradient analysis, performs slight smoothing on the folded areas to eliminate gray value abrupt changes, and completes the character edges occluded by the folds based on neighborhood texture features to ensure character integrity.

[0007] The ticket image preprocessing module uses a linear attention-based method to remove noise from the ticket text image, improve the quality and clarity of the ticket text image, and obtain the final reconstructed image;

[0008] The invoice information entry module constructs a text recognizer, which uses optical character recognition (OCR) to recognize invoice information and automatically save it.

[0009] Furthermore, in the ticket image preprocessing module, the linear attention-based method specifically includes the following steps:

[0010] Step A1: Acquire high-resolution text images, establish and initialize a customized conditional convolutional network and a feature extractor. The customized conditional convolutional network contains 10 residual blocks and 2 skip connections. Linear attention blocks are embedded in the middle layer of the network to achieve precise focusing of text features. The feature extractor contains 5 residual blocks and 1 skip connection.

[0011] Step A2: A two-stage training strategy combining pre-training and fine-tuning is adopted. By optimizing the target in stages, image quality enhancement and text recognition are achieved. The pre-training specifically includes the following steps:

[0012] Step A21: Forward diffusion process. Gaussian noise is gradually added to the high-resolution text image to generate a noisy image, simulating noise pollution in low-resolution images. Through the forward diffusion process, as the iterations near the end, the noisy image approaches pure Gaussian noise. By learning the inverse process, the ability to recover a clear image from a noisy image is obtained. The formula used is as follows: ;

[0013] In the formula, It is a time step Noisy images at that time, It is a high-resolution text image. It was before The cumulative product of the diffusion coefficients, It is Gaussian noise;

[0014] Step A22: Time step embedding. The time steps are embedded into a high-dimensional vector to capture the temporal features of different diffusion stages. The formula used is as follows: ;

[0015] In the formula, It is a high-dimensional vector. It is the single-dimensional length of the time step embedding. Yes It is a traversal;

[0016] Step A23: Low-resolution image preprocessing and feature extraction. The low-resolution text image is upsampled by bilinear interpolation and denoted as the upsampled text image. The upsampled text image is input into the feature extractor. The text image features are obtained by feature extraction of 5 residual blocks and feature fusion of skip connections.

[0017] Step A24: Noise prediction and parameter update. The noisy image, time step embedding, and text image features are used as inputs to a customized conditional convolutional network to obtain noise estimation. The mean square error of the real noise and the noise estimation is used as the loss function, and the parameters of the customized conditional convolutional network and the feature extractor are updated through gradient descent.

[0018] Step A3: The fine-tuning combines the image reconstruction capabilities learned in the pre-training stage with the text recognizer, generating high-quality, high-resolution text images through a back-diffusion process and optimizing text recognition accuracy. Specifically, it includes the following steps:

[0019] Step A31: The reverse diffusion process is the inverse of the forward diffusion process. Starting with pure Gaussian noise, it gradually removes noise to reconstruct a clear, high-resolution text image. The formula used is as follows: ;

[0020] In the formula, It is a time step A clear, high-resolution text image reconstructed in time. It is predicted Gaussian noise. It is a text image feature. It is a time step The standard deviation of noise during back diffusion. It is Gaussian noise added during the reverse diffusion process;

[0021] Step A32: Residual learning and image reconstruction. A residual learning strategy is used to calculate the output residual image of the backdiffusion process, and the final reconstructed image is obtained.

[0022] Furthermore, in the invoice information entry module, the optical character recognition method specifically includes the following steps:

[0023] Step B1: Text Recognizer Integration. The Transformer model is used as the text recognizer. The parameters of the Transformer model are kept frozen and used only as a tool for feature extraction and label prediction. The final reconstructed image is input into the Transformer model, and the predicted label of each character is output. Finally, the overall predicted label is obtained by concatenating the images.

[0024] Step B2: Joint fine-tuning and parameter update, using a joint loss function of reconstruction loss and recognition loss to optimize text image reconstruction quality and text recognition accuracy;

[0025] Step B3: The Transformer model extracts local features of each character through a sliding window, integrates structured and unstructured data, outputs the results, and saves them automatically.

[0026] The beneficial effects achieved by the present invention using the above solution are as follows:

[0027] (1) In the actual collection process of financial documents, due to multiple factors such as scanning equipment, ambient light and the physical state of the document itself, there are common problems such as serious noise interference, tilting and deformation, wrinkles and occlusion and loss of low resolution details. These problems are superimposed and lead to a significant reduction in text feature recognition, which seriously affects the efficiency and accuracy of financial data processing. This solution achieves precise focus of text features through precise preprocessing based on linear attention. It adopts a two-stage training strategy of pre-training and fine-tuning. It simulates noise pollution through forward diffusion, captures temporal features through time step embedding, updates model parameters through noise prediction, and uses backward diffusion to gradually remove noise and residual learning to achieve efficient removal of document image noise, significantly improve image clarity and character integrity, and fully retain the key information on the document.

[0028] (2) In view of the technical problems of limited recognition accuracy, high recognition error rate, insufficient data integration capability, and reliance on manual verification and input in traditional optical character recognition methods, which are not only time-consuming and labor-intensive, but also prone to input errors due to human negligence, and cannot meet the core needs of automation and efficiency in financial work, this solution optimizes the optical character recognition and input process, integrates the Transformer model as a text recognizer, accurately extracts the local features of each character through sliding window technology, and adopts a joint loss function that combines reconstruction loss and recognition loss. It deeply integrates image reconstruction capability with text recognizer for joint fine-tuning, realizes synchronous optimization of image quality and recognition accuracy, minimizes human error rate, and adapts to various formats and types of financial documents, greatly improves data input efficiency and shortens the financial data processing cycle. Attached Figure Description

[0029] Figure 1 This invention provides a module connection diagram for an automatic financial document information entry system based on optical character recognition.

[0030] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0032] Example 1, see Figure 1 The present invention provides an automatic financial invoice information entry system based on optical character recognition. The automatic financial invoice information entry system based on optical character recognition includes an invoice image acquisition module, an invoice image correction module, an invoice image preprocessing module, and an invoice information entry module.

[0033] The document image acquisition module uses a scanning device to acquire document text images and records the acquisition parameters of the document text images, including resolution, supplementary lighting mode, and scanning time.

[0034] The ticket image correction module detects the edge lines of the ticket based on Hough transform, calculates the tilt angle, and uses interpolation rotation correction for slight tilt and block rotation and edge cropping correction for large tilt to avoid image stretching distortion. It also identifies the folded areas of the ticket through gray value gradient analysis, performs slight smoothing on the folded areas to eliminate gray value abrupt changes, and completes the character edges occluded by the folds based on neighborhood texture features to ensure character integrity.

[0035] The ticket image preprocessing module uses a linear attention-based method to remove noise from the ticket text image, improve the quality and clarity of the ticket text image, and obtain the final reconstructed image;

[0036] The invoice information entry module constructs a text recognizer, which uses optical character recognition (OCR) to recognize invoice information and automatically save it.

[0037] Example 2, see Figure 1 This embodiment is based on the above embodiment. In the ticket image preprocessing module, the linear attention-based method specifically includes the following steps:

[0038] Step A1: Acquire high-resolution text images, establish and initialize a customized conditional convolutional network and a feature extractor. The customized conditional convolutional network contains 10 residual blocks and 2 skip connections. Linear attention blocks are embedded in the middle layer of the network to achieve precise focusing of text features. The feature extractor contains 5 residual blocks and 1 skip connection.

[0039] Step A2: A two-stage training strategy combining pre-training and fine-tuning is adopted. By optimizing the target in stages, image quality enhancement and text recognition are achieved. The pre-training specifically includes the following steps:

[0040] Step A21: Forward diffusion process. Gaussian noise is gradually added to the high-resolution text image to generate a noisy image, simulating noise pollution in low-resolution images. Through the forward diffusion process, as the iterations near the end, the noisy image approaches pure Gaussian noise. By learning the inverse process, the ability to recover a clear image from a noisy image is obtained. The formula used is as follows: ;

[0041] In the formula, It is a time step Noisy images at that time, It is a high-resolution text image. It was before The cumulative product of the diffusion coefficients, It is Gaussian noise;

[0042] Step A22: Time step embedding. The time steps are embedded into a high-dimensional vector to capture the temporal features of different diffusion stages. The formula used is as follows: ;

[0043] In the formula, It is a high-dimensional vector. It is the single-dimensional length of the time step embedding. Yes It is a traversal;

[0044] Step A23: Low-resolution image preprocessing and feature extraction. The low-resolution text image is upsampled by bilinear interpolation and denoted as the upsampled text image. The upsampled text image is input into the feature extractor. The text image features are obtained by feature extraction of 5 residual blocks and feature fusion of skip connections.

[0045] Step A24: Noise prediction and parameter update. The noisy image, time step embedding, and text image features are used as inputs to a customized conditional convolutional network to obtain noise estimation. The mean square error of the real noise and the noise estimation is used as the loss function, and the parameters of the customized conditional convolutional network and the feature extractor are updated through gradient descent.

[0046] Step A3: The fine-tuning combines the image reconstruction capabilities learned in the pre-training stage with the text recognizer, generating high-quality, high-resolution text images through a back-diffusion process and optimizing text recognition accuracy. Specifically, it includes the following steps:

[0047] Step A31: The reverse diffusion process is the inverse of the forward diffusion process. Starting with pure Gaussian noise, it gradually removes noise to reconstruct a clear, high-resolution text image. The formula used is as follows: ;

[0048] In the formula, It is a time step A clear, high-resolution text image reconstructed in time. It is predicted Gaussian noise. It is a text image feature. It is a time step The standard deviation of noise during back diffusion. It is Gaussian noise added during the reverse diffusion process;

[0049] Step A32: Residual learning and image reconstruction. A residual learning strategy is used to calculate the output residual image of the backdiffusion process, and the final reconstructed image is obtained.

[0050] Example 3, see Figure 1 This embodiment is based on the above embodiment. In the invoice information entry module, the optical character recognition method specifically includes the following steps:

[0051] Step B1: Text Recognizer Integration. The Transformer model is used as the text recognizer. The parameters of the Transformer model are kept frozen and used only as a tool for feature extraction and label prediction. The final reconstructed image is input into the Transformer model, and the predicted label of each character is output. Finally, the overall predicted label is obtained by concatenating the images.

[0052] Step B2: Joint fine-tuning and parameter update, using a joint loss function of reconstruction loss and recognition loss to optimize text image reconstruction quality and text recognition accuracy;

[0053] Step B3: The Transformer model extracts local features of each character through a sliding window, integrates structured and unstructured data, outputs the results, and saves them automatically.

[0054] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0056] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An automatic financial document information entry system based on optical character recognition, characterized in that, It includes a ticket image acquisition module, a ticket image correction module, a ticket image preprocessing module, and a ticket information entry module; The document image acquisition module uses a scanning device to acquire document text images and records the acquisition parameters of the document text images; The ticket image correction module detects the edge lines of the ticket based on Hough transform, calculates the tilt angle, and uses interpolation rotation correction for slight tilt and block rotation and edge cropping correction for large tilt to avoid image stretching distortion. It also identifies the folded areas of the ticket through gray value gradient analysis, performs slight smoothing on the folded areas to eliminate gray value abrupt changes, and completes the character edges occluded by the folds based on neighborhood texture features to ensure character integrity. The ticket image preprocessing module uses a linear attention-based method to remove noise from the ticket text image, improve the quality and clarity of the ticket text image, and obtain the final reconstructed image; The invoice information entry module constructs a text recognizer, which uses optical character recognition (OCR) to recognize invoice information and automatically save it.

2. The automatic financial document information entry system based on optical character recognition according to claim 1, characterized in that, In the ticket image preprocessing module, the linear attention-based method specifically includes the following steps: Step A1: Acquire high-resolution text images, establish and initialize a customized conditional convolutional network and a feature extractor. The customized conditional convolutional network contains 10 residual blocks and 2 skip connections. Linear attention blocks are embedded in the middle layer of the network to achieve precise focusing of text features. The feature extractor contains 5 residual blocks and 1 skip connection. Step A2: A two-stage training strategy combining pre-training and fine-tuning is adopted to achieve image quality enhancement and text recognition. The pre-training specifically includes the following steps: Step A21: Forward diffusion process, Gaussian noise is gradually added to the high-resolution text image to generate a noisy image, simulating the noise pollution of low-resolution images; Step A22: Time step embedding, embedding the time steps into a high-dimensional vector to capture the temporal features of different diffusion stages; Step A23: Low-resolution image preprocessing and feature extraction. The low-resolution text image is upsampled by bilinear interpolation and denoted as the upsampled text image. The upsampled text image is input into the feature extractor. The text image features are obtained by feature extraction of 5 residual blocks and feature fusion of skip connections. Step A24: Noise prediction and parameter update. The noisy image, time step embedding, and text image features are used as inputs to a customized conditional convolutional network to obtain noise estimation. The mean square error of the real noise and the noise estimation is used as the loss function, and the parameters of the customized conditional convolutional network and the feature extractor are updated through gradient descent. Step A3: The fine-tuning combines the image reconstruction capabilities learned in the pre-training stage with the text recognizer, generating high-quality, high-resolution text images through a back-diffusion process and optimizing text recognition accuracy. Specifically, it includes the following steps: Step A31: The reverse diffusion process is the inverse of the forward diffusion process. Starting from pure Gaussian noise, the noise is gradually removed to reconstruct a clear, high-resolution text image. Step A32: Residual learning and image reconstruction. A residual learning strategy is used to calculate the output residual image of the backdiffusion process, and the final reconstructed image is obtained.

3. The automatic financial document information entry system based on optical character recognition according to claim 1, characterized in that, In the invoice information entry module, the optical character recognition method specifically includes the following steps: Step B1: Text Recognizer Integration. The Transformer model is used as the text recognizer. The parameters of the Transformer model are kept frozen and used only as a tool for feature extraction and label prediction. The final reconstructed image is input into the Transformer model, and the predicted label of each character is output. Finally, the overall predicted label is obtained by concatenating the images. Step B2: Joint fine-tuning and parameter update, using a joint loss function of reconstruction loss and recognition loss to optimize text image reconstruction quality and text recognition accuracy; Step B3: The Transformer model extracts local features of each character through a sliding window, integrates structured and unstructured data, outputs the results, and saves them automatically.

Images