RPA automatic bill information processing system based on CRNN and improved CTC

Abstract

The invention discloses an RPA automatic bill information processing system based on CRNN and improved CTC. The RPA automatic bill information processing system comprises a bill data preprocessing unit, a data analysis unit and a data automatic input unit. The bill data preprocessing unit is used for extracting convolutional features of the bill image by using a convolutional neural network; the convolutional features are coded into a feature vector sequence; further learning the feature vector sequence by taking a bidirectional long-short-term memory network as a circulation layer, and preliminarily predicting a probability distribution matrix of a feature vector sequence tag; the data analysis unit is used for decoding the probability distribution matrix into a specific text label by adopting a CTC added with an attention mechanism; and the data automatic input unit automatically inputs the final identification result obtained by the data analysis unit into a service system by using RPA. According to the method, the high-efficiency and high-accuracy recognition of the bill information is realized by utilizing the strong feature extraction and sequence modeling capabilities of the deep learning technology and combining the automatic processing advantage of the RPA.

Description

Technical Field

[0001] The present invention relates to the field of automated bill information processing, and in particular to an RPA automated bill information processing system based on CRNN and improved CTC. Background Art

[0002] In the field of automated bill information processing, although a variety of technologies such as optical character recognition (OCR) have been proposed and applied in large numbers, existing solutions still have some significant pain points. The diversity of bill formats, the unevenness of image quality, the low processing efficiency, the lack of real-time performance and the limited accuracy have always been the bottleneck of the industry's development.

[0003] First, the diversity of bill formats makes it difficult to generalize the automated system. Since bills in different industries and regions have significant differences in format and layout, a general automated processing system often requires expensive and time-consuming customized development.

[0004] Secondly, the inhomogeneity of image quality, such as blur, tilt, uneven lighting or noise, seriously affects the accuracy of text recognition. Although existing technologies can provide acceptable recognition rates in some cases, the accuracy often drops significantly when faced with complex layouts or handwritten texts.

[0005] Then, the lack of real-time performance is another key issue, especially in industries such as financial services, healthcare and retail, which have strict requirements on the speed of bill processing. However, existing systems often cannot achieve fast processing without compromising accuracy.

[0006] In addition, the efficiency of the existing processing flow cannot be ignored. Many systems need to go through complex image pre-processing, feature extraction, text recognition and post-processing when processing bills. These steps not only increase processing time, but also increase the possibility of errors.

[0007] Lack of real-time performance is another key issue, especially in industries such as financial services, healthcare, and retail, which have strict requirements on the speed of bill processing. However, existing systems often cannot achieve fast processing without compromising accuracy. Finally, the issue of accuracy remains a core challenge in bill information processing. Although existing technologies can provide acceptable recognition rates in some cases, performance often drops significantly when faced with complex layouts or handwritten text. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide an RPA automated bill information processing system based on CRNN and improved CTC. The present invention utilizes the powerful feature extraction and sequence modeling capabilities of deep learning technology, combined with the automated processing advantages of RPA, to achieve high-efficiency and high-accuracy recognition of bill information.

[0009] The purpose of the present invention is achieved through the following technical solutions:

[0010] The RPA automated bill information processing system based on CRNN and improved CTC includes bill data preprocessing unit, data parsing unit and data automatic entry unit; among them,

[0011] The bill data preprocessing unit performs deep learning and feature extraction on the input bill image: first, the convolutional neural network CRNN is used to extract the convolutional features of the bill image. The convolutional features capture the key visual information in the image; then, the convolutional features are encoded into a feature vector sequence; then, the bidirectional long short-term memory network BiLSTM is used as a recurrent layer to further learn the feature vector sequence and preliminarily predict the probability distribution matrix of the feature vector sequence label;

[0012] The data parsing unit uses the connection temporal classification algorithm (CTC) with an attention mechanism to decode the probability distribution matrix of the feature vector sequence labels output by the preprocessing unit into specific text labels;

[0013] The automatic data entry unit uses the robotic process automation technology RPA to automatically enter the final recognition results obtained by the data analysis unit into the business system.

[0014] The input bill image is a two-dimensional matrix I, I∈R h×ω , h is the height of the bill image, ω is the width of the bill image, R is the real number set of the bill image, indicating that each pixel value in the image is a real number; I indicates that the input bill image is a two-dimensional real number matrix with a height of h and a width of ω, where each element I in the I matrix ij Represents the pixel value of the image at the position coordinate (i, j), where i represents the row index and j represents the column index; after passing through the k-layer convolutional layer of the convolutional neural network CNN, the convolution feature map of the output bill image is F k , F k ∈R hk×ωk×dk , where h k Represented as feature map F k The spatial height, ω k Represented as feature map F k The spatial width, d k Represented as feature map F k The number of channels, k represents the kth convolutional layer;

[0015] The feature map F is transformed into k Flattening into a one-dimensional sequence represents the feature vector. This process transforms the feature map F k The spatial information of is converted from a two-dimensional matrix into a one-dimensional feature vector. This one-dimensional vector contains all the information in the original two-dimensional matrix, but no longer has the original two-dimensional spatial structure. The processed feature map F k is encoded as a one-dimensional feature vector sequence {x1, x2, ..., x t-1 , x t}, where t is the sequence length.

[0016] The bidirectional long short-term memory network BiLSTM is used to model the feature vector sequence to fully capture the dependency relationship between the context. The bidirectional long short-term memory network BiLSTM is used to model the feature vector sequence through the forward {x1, x2, ..., x t-1 , x t} and the reverse {x t , x t-1 , ..., x2, x1} are passed into two LSTM structures, and then the output results of the two LSTMs are merged to obtain {y'1, y'2, ..., y' t-1 , y' t}, the merged result is then passed into the continuously stacked bidirectional long short-term memory network BiLSTM structure to form a loop structure; then the Softmax function is used for normalization, and the probability distribution matrix of the feature vector sequence is obtained after normalization. The probability distribution matrix of the feature vector sequence reflects the potential character labels and their confidence corresponding to different positions of the bill image.

[0017] The attention mechanism calculates the weight α of the feature vector corresponding to time step t t , determine the decoding feature sequence {y'1, y'2, ..., y' t-1 , y' t} should be concerned about the characteristic value; among them,

[0018]

[0019] In the formula, u t Represents the input feature y' t The vector value of T v is the hidden state vector of the decoder; α t represents the weight of the feature vector corresponding to time step t, e is a natural constant, and T is the number of elements in the feature sequence;

[0020] The calculation formula of the context vector C is as follows:

[0021]

[0022] Where C represents the weighted sum of the most relevant parts of the context vector input feature sequence.

[0023] The calculation formula of the improved CTC algorithm is as follows:

[0024] A = δ[W·C+b];

[0025] Among them, δ represents the Softmax activation function, which is used to convert the output value into a probability distribution, ensuring that all output values ​​are between 0 and 1 and the sum is 1; W is a learnable weight matrix, which is used to calculate the correlation between the context vector C and the target output; C is the context vector, which is generated by the previous bidirectional long short-term memory network BiLSTM and contains important information about the bill image; b is a learnable bias term, which is used to adjust the output of the attention layer; A is the final output, which is obtained by linearly transforming the context vector C and the weight matrix W, then adding the bias term b, and finally performing a nonlinear transformation through the δ activation function.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] 1. Application of deep learning technology: The CRNN network can effectively extract text features from bill images without being restricted by format and layout, thus improving the versatility and adaptability of the system.

[0028] 2. Introduction of improved CTC algorithm: Through CTC technology, the model can directly learn accurate label sequences from image sequences without complex post-processing steps, simplifying the recognition process.

[0029] 3. Integration of RPA technology: The application of RPA technology enables the system to automatically perform tasks such as scanning, identifying and data entry of bills, reducing manual intervention and improving the overall level of automation.

[0030] 4. End-to-end recognition process: The system provided by the present invention realizes end-to-end automated processing from image input to text recognition result output, which greatly shortens the processing time and improves efficiency.

[0031] 5. Improvement of real-time processing capabilities: The system is designed with real-time requirements in mind, and can quickly respond to bill processing needs to meet high-speed business processing scenarios.

[0032] 6. Significant improvement in accuracy: The application of deep learning models has significantly improved the accuracy of text recognition, and even handwritten or complex-format bills can be recognized with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the workflow diagram of the RPA automated ticket information processing system based on CRNN and improved CTC.

[0034] Figure 2 Workflow diagram for the automatic data entry unit. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0036] like Figure 1 , the RPA automated bill information processing system based on CRNN and improved CTC includes a bill data preprocessing unit (CRNN), a data parsing unit (improved CTC) and an automatic data entry unit (RPA):

[0037] (1) Bill data preprocessing (CRNN) unit: This module is the first layer processing unit of the system, responsible for deep learning and feature extraction of the input bill image. It first uses a convolutional neural network to extract the convolution features of the bill image, which captures the key visual information in the image. Then, these features are encoded into a feature vector sequence to provide input for subsequent processing. Then, the bidirectional long short-term memory network (BiLSTM) is used as a recurrent layer to further learn these feature sequences and preliminarily predict the probability distribution of the feature sequence labels. The role of this module is to convert the original bill image into a feature sequence that can be further processed by the system, laying the foundation for accurate recognition of bill information.

[0038] (2) Data parsing (improved CTC) unit: The data parsing unit is the second-layer processing unit of the system, which is mainly responsible for decoding the probability distribution of the feature sequence labels output by the preprocessing unit into specific text labels. This module adopts the improved connection temporal classification (CTC) algorithm, which can convert the preliminary predicted sequence label distribution output by BiLSTM into the final recognition result. On the basis of the traditional CTC algorithm, the attention mechanism is added to form an improved CTC algorithm. The improved CTC algorithm optimizes the decoding process, improves the accuracy and efficiency of recognition, and ensures the accurate conversion from feature sequence to text information. The function of this module is to convert the abstract feature information extracted from the bill image into readable text information to complete the bill information recognition process.

[0039] (3) Data automatic entry (RPA) unit: The data automatic entry unit is the third-layer processing unit of the system. It is responsible for automatically entering the final recognition results obtained by the data analysis unit into the business system. This module uses robotic process automation (RPA) technology to simulate the operations of human users to complete data filling, submission, confirmation and other operations. The application of RPA technology realizes the automatic processing of bill information, greatly improves work efficiency, and reduces manual operation errors and time costs. The function of this module is to seamlessly integrate the recognized bill information into the business process of the enterprise, ensuring the continuity of information flow and the automation of business operations.

[0040] The following is a detailed description of each part:

[0041] 1. Bill data preprocessing (CRNN) unit

[0042] The bill data preprocessing unit is the front-end processing module of the system, which is mainly responsible for converting the input bill image into a feature vector sequence suitable for subsequent text parsing. The core architecture of this module is based on the convolutional recurrent neural network (CRNN). Through the joint processing of the convolutional neural network (CNN) and the bidirectional long short-term memory network (BiLSTM), efficient image feature extraction and serialization representation are achieved. The output of this module is a probability distribution of a set of feature sequences, which serves as the input of the subsequent parsing unit.

[0043] The bill image is first processed by a multi-layer convolutional neural network. The function of CNN is to extract local features from the image, such as edges, corners, and texture information. The input image is a two-dimensional matrix with a height of h and a width of . After k convolutional layers, the output feature map is , where represents the spatial dimension of the feature map and represents the number of channels.

[0044] After the convolution feature extraction is completed, the feature map output by CNN is flattened into a one-dimensional sequence representation, converting the two-dimensional spatial information into time series data that can be processed by a recurrent neural network. The feature map is encoded as a feature vector sequence {x1, x2, ..., xt-1, xt}, where t is the sequence length. This sequence captures the structural information of the bill image on the time axis, which serves as the basis for subsequent recognition.

[0045] Next, the bidirectional long short-term memory network (BiLSTM) is used to model the feature vector sequence to fully capture the dependencies between the context. The BiLSTM is passed into two LSTM structures in the form of forward {x1, x2, ..., xt-1, xt} and reverse {xt, xt-1, ..., x2, x1}. The output results of the two LSTMs are then merged to obtain {y1, y2, ..., yt-1, yt}. The merged results are then passed into the continuously stacked BiLSTM structure to form a loop structure. Before the feature sequence is passed to the data parsing unit, the result of the BiLSTM output needs to be normalized using the Softmax function. After normalization, the probability distribution matrix of the feature sequence is obtained. These probability distributions reflect the potential character labels and their confidence corresponding to different positions of the bill image, which provides a basis for the subsequent improved CTC algorithm parsing.

[0046] 2. Data analysis (improved CTC) unit

[0047] The data parsing unit is the core module of the system, responsible for decoding the probability distribution of the feature sequence output by the bill data preprocessing unit into specific text labels. The traditional CTC algorithm plays a fundamental role in this process, but it has great limitations when processing blurred or low-quality bill images, resulting in insufficient decoding correctness in these non-standard scenarios. In order to improve the accuracy of recognition, this patent proposes an improved CTC algorithm based on the attention mechanism to optimize the decoding performance and improve the accuracy and efficiency of bill information recognition.

[0048] 2.1 Reasons for introducing the attention mechanism

[0049] Although the traditional CTC algorithm can effectively deal with the problem of sequence misalignment, it often ignores or misrecognizes key characters when dealing with long sequences or bills with complex background interference. This is because CTC relies on serialized decoding from beginning to end, and lacks the ability to pay attention to certain key areas in the sequence. This shortcoming is more obvious when the text layout of the bill is complex or the character spacing is uneven. Therefore, in order to solve this problem, we propose an attention-enhanced CTC algorithm. The model dynamically pays attention to different parts of the input sequence, calculates the correlation between image features and text labels, and dynamically adjusts the weight of the features, so that the model pays more attention to the text information in the bill image, thereby enhancing the ability to capture key characters.

[0050] 2.2 How the Attention Mechanism Works

[0051] The attention mechanism assigns different weights to each time step by calculating the similarity or correlation between the input feature sequence and the decoder output. That is, when decoding a character, the model no longer relies solely on global information, but can selectively focus on the part of the input feature sequence that is most relevant to the current decoding step.

[0052] The attention mechanism calculates the weight α of the feature vector corresponding to time step t t , determine the decoding feature sequence {y'1, y'2, ..., y' t-1 , y' t} should be concerned about the characteristic value; among them,

[0053]

[0054] In the formula, u t Represents the input feature y' t The vector value of T v is the hidden state vector of the decoder; α t represents the weight of the feature vector corresponding to time step t, e is a natural constant, and T is the number of elements in the feature sequence;

[0055] The calculation formula of the context vector C is as follows:

[0056]

[0057] Where C represents the weighted sum of the most relevant parts of the context vector input feature sequence.

[0058] 2.3 Combination of Attention Mechanism and Traditional CTC

[0059] The traditional CTC algorithm decodes the sequence by finding the maximum probability path, but its limitation is the lack of selective attention to different time steps. To overcome this problem, this patent introduces an improved CTC decoding strategy, which embeds the attention mechanism into the CTC decoding process and combines the attention weights between time steps to guide the path selection of CTC decoding.

[0060] In the improved CTC algorithm, the decoding process not only considers the character probability distribution of the BiLSTM output, but also combines the attention weight α at each time step. t When processing bill images, the feature map F is extracted by the convolutional layer, while the sequence feature S is generated by the recurrent layer. The attention layer A combines these two features to generate a new weighted feature map that focuses more on the text area. The weighted feature map is then fed into the data parsing unit to generate the final recognition result.

[0061] The calculation formula of the improved CTC algorithm is as follows:

[0062] A = δ[W·C+b];

[0063] Among them, δ represents the Softmax activation function, which is used to convert the output value into a probability distribution, ensuring that all output values ​​are between 0 and 1 and the sum is 1; W is a learnable weight matrix, which is used to calculate the correlation between the context vector C and the target output; C is the context vector, which is generated by the previous bidirectional long short-term memory network BiLSTM and contains important information about the bill image; b is a learnable bias term, which is used to adjust the output of the attention layer; A is the final output, which is obtained by linearly transforming the context vector C and the weight matrix W, then adding the bias term b, and finally performing a nonlinear transformation through the δ activation function.

[0064] 2.4 Improve the operation mechanism of CTC

[0065] By introducing the attention mechanism, the improved CTC algorithm can more accurately identify key text information on bills, and maintain good performance even in the case of low image quality or small text areas. In addition, the addition of the attention mechanism also improves the model's ability to adapt to changes in bill format, thereby improving the robustness and practicality of the entire system.

[0066] 2.5 Innovations of the Improved CTC Algorithm Compared with the Traditional CTC Algorithm

[0067] Compared with the traditional CTC algorithm, the improved CTC algorithm can focus more on the key text areas in the image by introducing attention weights in the decoding process when processing blurred bill images. This attention weight is dynamically generated by learning the alignment relationship between the input sequence and the output sequence, allowing the model to automatically ignore interference information and focus on text features when processing blurred or low-contrast bill images, significantly improving recognition accuracy.

[0068] In the recognition task of a dataset containing 1,000 blurred bill images, the average recognition accuracy of the traditional CTC algorithm was 85%, while the average recognition accuracy of the improved CTC algorithm was increased to 95%. In terms of processing time, the traditional CTC algorithm takes an average of 18 seconds to process an image, while the improved CTC algorithm only takes 7.5 seconds, reducing the processing time by about 58.3%.

[0069] 3. Data automatic entry (RPA) unit

[0070] The main responsibility of the automatic data entry unit is to automatically enter the final recognition results output by the data analysis unit into the corresponding business system, complete the data filling, submission, confirmation and other operations, so as to realize the fully automated processing of bill information. In this process, the automatic data entry unit first needs to perform necessary data conversion and structured processing on the recognition results provided by the data analysis unit, and convert the unstructured text information into a structured data format through predefined rules or templates, so as to ensure that the data output by the system can be correctly understood and processed by the business system. Then, the data is verified and corrected to ensure the accuracy and completeness of the data. The reliability of the data is ensured by checking whether the format of the field value is correct and whether the required fields are missing. If necessary, the automatic error correction mechanism is triggered or an error log is generated for manual processing. Finally, the execution of the task is monitored and feedback is provided. The detailed information of each step of the operation is tracked through log recording and real-time monitoring, and an alarm is generated when an abnormal situation occurs, and feedback is promptly given to relevant personnel for processing to ensure the smooth operation of the business process.

[0071] like Figure 2 , the workflow of the data automatic entry (RPA) unit is as follows:

[0072] 1) Data conversion and structuring

[0073] The automatic data entry unit needs to perform necessary data conversion and structural processing on the recognition results provided by the data analysis unit. Since the recognition results may contain unstructured text information, these text information are converted into structured data formats through predefined rules or templates. This step ensures that the data output by the system can be correctly understood and processed by the business system. Structured bill data includes field names, field value pairs, and other metadata related to business processes.

[0074] 2) Data verification and error correction

[0075] Before entering data into the business system, data verification and error correction are required to ensure the accuracy and completeness of the data. Data verification includes checking whether the format of the field value is correct and whether the required fields are missing. For example, for the amount field, the automatic data entry unit will verify whether it contains valid numbers and currency symbols; for the date field, it will verify whether it conforms to the specific date format. For the detected erroneous data, the system can trigger the automatic error correction mechanism or generate an error log for subsequent manual processing.

[0076] 3) Data entry and submission

[0077] The structured data after verification and error correction is automatically entered into the target business system through RPA technology. RPA robots simulate manual operation steps according to predefined operation processes, including opening the business system interface, filling out forms, submitting data, etc. For example, in the banking system, the RPA robot automatically logs in to the customer account, fills out the bill information form, and submits it for confirmation; in the e-commerce system, the RPA robot automatically enters the order information and completes the order processing.

[0078] 4) Task monitoring and feedback

[0079] The "task monitoring and feedback" link is responsible for data monitoring and operation feedback of the entire task process, ensuring the controllability and traceability of the data entry process. The system tracks the execution of each step of the entry operation through log recording and real-time monitoring, and records detailed information on task execution, including operation time, operation content, operation results, etc. Once an abnormal situation occurs or entry fails, the system will automatically generate an alarm and promptly feedback to relevant personnel for processing. Through the task monitoring and feedback mechanism, the system can promptly discover and solve problems to ensure the smooth operation of business processes.

[0080] CRNN: Convolutional Recurrent Neural Network (CRNN) is a deep learning model that integrates the spatial feature extraction capability of convolutional neural network (CNN) and the temporal modeling capability of recurrent neural network (RNN). It is particularly suitable for processing text recognition tasks in images. Its advantage is that it can process text sequences of arbitrary length and does not require the positioning and segmentation of each character in the image in advance. Instead, it directly inputs the entire text line as a sequence, thereby simplifying the recognition process.

[0081] Improved CTC: Connectionist Temporal Classification (CTC) is a neural network model for processing sequence data, which is widely used in natural language processing, image recognition and other fields. Its core idea is to achieve classification or recognition of sequence data by establishing a mapping relationship between the input sequence and the output sequence. However, traditional CTC has the limitation of conditional independence assumption, that is, it assumes that the output labels are independent of each other, which is often not true in actual scenarios, limiting its performance. The improved CTC proposed in this patent enhances the model's ability to capture the dependency relationship of sequence data by introducing the attention mechanism, thereby improving recognition accuracy and the generalization ability of the model.

[0082] RPA: Robotic process automation (RPA) is a technology that uses software to execute business processes. It can execute the same processes according to human execution rules and operating procedures. It helps to reduce manpower input in work, avoid human operation errors, greatly reduce processing time, and improve work efficiency and accuracy.

[0083] In terms of RPA technology, deep learning algorithms can be used to complete the execution of automated tasks, improve the efficiency and adaptability of automated processes, and achieve the purpose of replacing RPA. However, although deep learning provides powerful data processing capabilities and pattern recognition functions, RPA still has its unique advantages and application scenarios. The main advantages of RPA are its rapid deployment, low cost, easy maintenance, and non-intrusiveness to existing systems. Therefore, although deep learning can be used as an alternative to RPA, RPA may still be a more practical choice when considering cost-effectiveness, implementation speed, and ease of operation.

[0084] CRNN is actually a combination of CNN+RNN. The feature map is first processed by CNN and then enters RNN for processing.

[0085] The present invention not only solves the accuracy and efficiency problems in the existing bill information processing technology, but also provides a highly adaptable and practical solution, bringing innovative changes to the field of bill information processing. This can not only improve the operational efficiency of enterprises and reduce costs, but also provide users with a more accurate and convenient service experience, and promote technological progress and business development in related industries.

[0086] The present invention brings significant beneficial effects to the field of bill information processing through its innovative structure and method:

[0087] 1. Improve processing efficiency: Traditional bill information processing relies on manual input and multi-step processing, which is inefficient. The present invention realizes end-to-end automated processing from bill image input to data entry through an automated bill data preprocessing unit, a data parsing unit, and an automatic data entry unit, significantly improving the speed of bill information processing.

[0088] 2. Enhanced recognition accuracy: Different from the traditional OCR image recognition processing scheme, the CRNN network proposed in the technical solution of the present invention combined with the improved CTC algorithm can enable the system to more accurately recognize the text information on the bill, including handwriting and complex layouts. The application of this deep learning technology reduces human errors and improves the accuracy of recognition results.

[0089] 3. Reduce operating costs: The technical solution of the present invention organically integrates RPA technology, reduces the need for manual operation, and reduces labor costs. RPA robots automatically process large amounts of data generated in business processes, reducing dependence on professional operators, and also reducing training and maintenance costs.

[0090] 4. Improve the accuracy of data entry: The data verification and error correction mechanism of the automatic data entry unit in the present invention ensures the accuracy and completeness of the entered data and reduces subsequent processing problems caused by data errors.

[0091] 5. Enhance user experience: Automated and efficient processing procedures provide users with a smoother and more convenient service experience, reduce user waiting time, and improve user satisfaction.

[0092] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. The RPA automated bill information processing system based on CRNN and improved CTC is characterized by: It includes a bill data preprocessing unit, a data parsing unit and an automatic data entry unit; wherein, The bill data preprocessing unit performs deep learning and feature extraction on the input bill image: first, the convolutional neural network CRNN is used to extract the convolutional features of the bill image. The convolutional features capture the key visual information in the image; then, the convolutional features are encoded into a feature vector sequence; then, the bidirectional long short-term memory network BiLSTM is used as a recurrent layer to further learn the feature vector sequence and preliminarily predict the probability distribution matrix of the feature vector sequence label; The data parsing unit uses the connection temporal classification algorithm (CTC) with an attention mechanism to decode the probability distribution matrix of the feature vector sequence labels output by the preprocessing unit into specific text labels; The automatic data entry unit uses the robotic process automation technology RPA to automatically enter the final recognition results obtained by the data analysis unit into the business system.

2. According to claim 1, the RPA automated bill information processing system based on CRNN and improved CTC is characterized in that: The input bill image is a two-dimensional matrix I, I∈R h×ω , h is the height of the bill image, ω is the width of the bill image, R is the real number set of the bill image, indicating that each pixel value in the image is a real number; I indicates that the input bill image is a two-dimensional real number matrix with a height of h and a width of ω, where each element I in the I matrix ij Represents the pixel value of the image at the position coordinate (i, j), where i represents the row index and j represents the column index; after passing through the k-layer convolutional layer of the convolutional neural network CNN, the convolution feature map of the output bill image is F k , F k ∈R hk×ωk×dk , where h k Represented as feature map F k The spatial height, ω k Represented as feature map F k The spatial width, d k Represented as feature map F k The number of channels, k represents the kth convolutional layer; The feature map F is transformed into k Flattened into a one-dimensional sequence to represent the feature vector, this process transforms the feature map F k The spatial information of is converted from a two-dimensional matrix into a one-dimensional feature vector. This one-dimensional vector contains all the information in the original two-dimensional matrix, but no longer has the original two-dimensional spatial structure. The processed feature map F k is encoded as a one-dimensional feature vector sequence {x1, x2, ..., x t-1 , x t }, where t is the sequence length.

3. The RPA automated bill information processing system based on CRNN and improved CTC according to claim 1 is characterized in that: The bidirectional long short-term memory network BiLSTM is used to model the feature vector sequence to fully capture the dependency relationship between the context. The bidirectional long short-term memory network BiLSTM is used to model the feature vector sequence through the forward {x1, x2, ..., x t-1 , x t } and the reverse {x t , x t-1 , ..., x2, x1} are passed into two LSTM structures, and then the output results of the two LSTMs are merged to obtain {y'1, y'2, ..., y' t-1 , y' t }, the merged result is then passed into the continuously stacked bidirectional long short-term memory network BiLSTM structure to form a loop structure; then the Softmax function is used for normalization, and the probability distribution matrix of the feature vector sequence is obtained after normalization. The probability distribution matrix of the feature vector sequence reflects the potential character labels and their confidence corresponding to different positions of the bill image.

4. According to claim 1, the RPA automated bill information processing system based on CRNN and improved CTC is characterized in that: The attention mechanism calculates the weight α of the feature vector corresponding to time step t t , determine the decoding feature sequence {y'1, y'2, ..., y' t-1 , y' t } should be concerned about the characteristic value; among them, In the formula, u t Represents the input feature y' t The vector value of T v is the hidden state vector of the decoder; α t represents the weight of the feature vector corresponding to time step t, e is a natural constant, and T is the number of elements in the feature sequence; The calculation formula of the context vector C is as follows: Where C represents the weighted sum of the most relevant parts of the context vector input feature sequence.

5. According to claim 4, the RPA automated bill information processing system based on CRNN and improved CTC is characterized in that: The calculation formula of the improved CTC algorithm is as follows: A = δ[W·C+b]; Among them, δ represents the Softmax activation function, which is used to convert the output value into a probability distribution, ensuring that all output values ​​are between 0 and 1 and the sum is 1; W is a learnable weight matrix, which is used to calculate the correlation between the context vector C and the target output; C is the context vector, which is generated by the previous bidirectional long short-term memory network BiLSTM and contains important information about the bill image; b is a learnable bias term, which is used to adjust the output of the attention layer; A is the final output, which is obtained by linearly transforming the context vector C and the weight matrix W, then adding the bias term b, and finally performing a nonlinear transformation through the δ activation function.

Images