Automatic bill information processing method based on capsule network

Abstract

The invention discloses an automatic bill information processing method based on a capsule network. The automatic bill information processing method comprises the following steps of converting image features of a bill image into feature blocks with fixed sizes; the initial capsule layer of the capsule network projects each feature block into an initial capsule vector, and then nonlinear transformation is carried out on the initial capsule vector; the connection weight between the capsules is updated, the contribution value of each low-layer capsule vector to the high-layer capsule vector is calculated, and the low-layer capsules are weighted and summed and then transmitted to the high-layer capsules; the output vector of the high-level capsule is transmitted to an independent full connection layer, the full connection layer serves as a follow-up processing module of the capsule network, capsule features are mapped into probability distribution of character categories, and accurate recognition of the character categories in the bill image is achieved. The method has high adaptability to the diversity of bill patterns, can significantly improve the efficiency and real-time performance of bill information processing, and has strong anti-noise capability and recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the field of bill information processing, and in particular to an automatic bill information processing method based on capsule network. Background Art

[0002] In the field of automated bill information processing, although a variety of technologies such as optical character recognition (OCR) have been proposed and applied in large numbers, existing solutions still face a series of pain points that need to be addressed, mainly including the diversity of bill formats, uneven image quality, insufficient real-time processing capabilities, and low recognition accuracy.

[0003] First, the diversity of bill formats makes it difficult for automated systems to be universal. Since bills in different industries and regions have significant differences in format and layout, general automated processing systems often require expensive and time-consuming customized development to meet the bill recognition needs of specific scenarios.

[0004] Secondly, the inhomogeneity of image quality also has a significant impact on the accuracy of bill recognition. Due to the limitations of shooting equipment and environmental conditions, bill images often have problems such as blur, tilt, uneven lighting or noise, which will significantly reduce the recognition effect of bill information.

[0005] Again, the lack of real-time processing capabilities is another key issue, especially in industries such as financial services, healthcare and retail, which have strict requirements on the speed of bill processing. Existing solutions are usually unable to meet the real-time requirements without sacrificing accuracy due to high computational complexity or lengthy processing procedures.

[0006] Finally, the recognition accuracy is always the core challenge of bill information processing. Although existing technologies can achieve high recognition rates in certain situations, their performance often drops significantly when faced with bills with complex layouts, handwritten text, or special characters, increasing the possibility of errors. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide an automatic bill information processing method based on capsule networks.

[0008] The purpose of the present invention is achieved through the following technical solutions:

[0009] The automatic bill information processing method based on capsule network includes the following steps:

[0010] S1. Input the bill image, perform normalization processing, and then convert the image features of the bill image into feature blocks of fixed size through the convolution operation of convolution preprocessing;

[0011] S2, the initial capsule layer of the capsule network projects each feature block into an initial capsule vector; the initial capsule vector is nonlinearly transformed through the Squash activation function;

[0012] S3, through the dynamic routing mechanism of the capsule network, update the connection weights between capsules and calculate the contribution value of each low-level capsule vector to the high-level capsule vector;

[0013] The contribution value of each low-level capsule vector to the high-level capsule vector is weighted and summed up, and then passed to the high-level capsule.

[0014] S4. The output vector of the high-level capsule is passed to an independent fully connected layer. The fully connected layer serves as the subsequent processing module of the capsule network, mapping the capsule features into the probability distribution of character categories to achieve accurate recognition of character categories in the bill image.

[0015] In step S1, the bill image is represented as a two-dimensional matrix , , is the height of the bill image, is the width of the bill image, R is the real number set of the bill image, indicating that each pixel value in the image is a real number; Indicates that the input bill image is a two-dimensional real number matrix with height multiplied by width;

[0016] Ticket Image After normalization, the image features of the bill image are converted into a fixed-size feature block P through the convolution operation of the convolution preprocessing, and flattened into a vector as the input of the subsequent capsule layer:

[0017] ;

[0018] Among them, Conv2D is a module that performs two-dimensional convolution to learn and extract bill images The important features in the image; P is the result of the convolution operation, which is used to capture the bill image The local features in .

[0019] In step S2, the initial capsule layer of the capsule network projects each feature tile into an initial capsule vector:

[0020] ;

[0021] In the formula, For the Capsules The projection result of the feature blocks represents the output vector of the lower-level capsule; is a learnable weight matrix used to transform the input vector of the feature block Projection into capsule space; For the The vector formed by flattening the feature blocks; For feature blocks to The bias vector of each capsule is initially the output vector of each low-level capsule Applies a simple linear offset.

[0022] In step S3, the core of the capsule network is a dynamic routing mechanism, which is used to update the connection weights between capsules; the dynamic routing mechanism calculates the contribution of each low-level capsule vector to the high-level capsule vector:

[0023] ;

[0024] In the formula, is a probability value, indicating the lower layer Capsules for high-rise The contribution weight of each capsule; For feature blocks to The bias vector of the capsule; For feature blocks to The bias vector of each capsule is updated dynamically; Exponentialize the connection weights, amplifying larger weights; To normalize all weights, ensure that each coupling coefficient is a valid probability value.

[0025] The contributions of lower-level capsules are aggregated in the dynamic routing mechanism to get the input of the higher-level glue:

[0026] ;

[0027] In the formula, is the input vector of the high-level capsule;

[0028] The output vector of the lower-level capsule After weighted summation and passing it to the high-level capsule, the capsule vector is nonlinearly transformed through the Squash activation function to ensure that the norm of the vector is compressed to the range of [0, 1]:

[0029] ;

[0030] In the formula, For high-level The output vector of each capsule; For the The modulus of the capsule vector.

[0031] In step S4, the output vector of the high-level capsule , the capsule features are mapped to the probability distribution of character categories using the fully connected layer to achieve accurate recognition of character categories in bill images:

[0032] ;

[0033] In the formula, is the weight matrix of the fully connected layer; is the bias term of the fully connected layer; is the probability distribution of the character categories in the bill image.

[0034] At the same time, the present invention provides:

[0035] A server comprises a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned automatic ticket information processing method based on capsule network.

[0036] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned automatic ticket information processing method based on capsule network.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] 1. The present invention has high adaptability to the diversity of bill formats.

[0039] The present invention is based on the dynamic routing mechanism of capsule networks, which can automatically capture the spatial relationship between characters and local features in bills, and integrate global representations through high-level capsules to adapt to diverse bill layouts. Traditional OCR methods often rely on templated bill formats, requiring complex manual template design and adaptation development, and are difficult to meet the diverse needs of bills in different industries and regions. In contrast, the present invention can quickly adapt to bill formats in different languages, regions and industries, significantly reducing the customized development costs caused by format differences.

[0040] 2. The present invention can significantly improve the efficiency and real-time performance of bill information processing.

[0041] By optimizing the dynamic routing mechanism of the capsule network and combining it with an efficient convolution preprocessing module, the present invention reduces the complex image preprocessing and postprocessing steps in the traditional method, thereby reducing the overall computational overhead. In addition, the structure of the capsule network can extract multiple feature relationships at the same time, without the need for a multi-stage processing flow in the traditional CNN + LSTM architecture. Compared with the traditional OCR method that takes a long time to process bills, the present invention can complete the extraction of bill information in a shorter time, especially in high-concurrency scenarios. It can achieve real-time processing, meeting the needs of industries such as finance, medical care, and retail for efficient bill processing.

[0042] 3. The present invention has strong anti-noise capability and recognition accuracy.

[0043] The present invention utilizes the feature representation capability and spatial relationship capture mechanism of capsule networks to extract key features and ensure recognition accuracy in blurred, tilted, unevenly illuminated or noisy bill images. Compared with the shortcomings of traditional OCR methods that are extremely sensitive to image quality, the present invention significantly enhances the robustness of character shape and spatial arrangement. In addition, through the dynamic adjustment of feature contribution values ​​by a dynamic routing mechanism, the present invention exhibits higher recognition capabilities in complex layouts, handwritten texts or bills containing special characters, effectively reducing the possibility of misidentification and missed recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The flowchart of the automatic bill information processing method based on capsule network. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0046] like Figure 1 , an automatic bill information processing method based on capsule network, comprising the following steps:

[0047] 1. Image input and preprocessing

[0048] The input bill image is represented as a two-dimensional matrix , , is the height of the bill image, is the width of the bill image, R is the real number set of the bill image, indicating that each pixel value in the image is a real number; Indicates that the input bill image is a A two-dimensional real matrix of height by width.

[0049] Image of the receipt After normalization and convolution preprocessing, the image features are converted into fixed-size feature blocks P through convolution operations and flattened into vectors as input to the subsequent capsule layer: , where Conv2D is a module that performs two-dimensional convolution to learn and extract the bill image The important features in the image; P is the result of the convolution operation, which is used to capture the bill image The local features in .

[0050] 2. Capsule Network Feature Extraction

[0051] The initial capsule layer of the capsule network projects each feature patch P into an initial capsule vector:

[0052] ;

[0053] In the formula, For the Capsules Input of feature blocks; is the weight matrix that projects the feature block into a capsule vector; For the The vector formed by flattening the feature blocks; A fixed bias, initially for each lower-level capsule output vector Applies a simple linear offset.

[0054] The capsule vector is nonlinearly transformed through the Squash activation function to ensure that the norm of the vector is compressed to the range of [0, 1]:

[0055] ;

[0056] In the formula, For the The output vector of each capsule; For the The modulus of the capsule vector.

[0057] 3. Dynamic routing mechanism

[0058] Through the dynamic routing mechanism of the capsule network, the connection weights between capsules are updated, and the contribution value of each low-level capsule vector to the high-level capsule vector is calculated;

[0059] The contribution value of each low-level capsule vector to the high-level capsule vector is weighted and summed up, and then passed to the high-level capsule.

[0060] The core of the capsule network is a dynamic routing mechanism, which is used to update the connection weights between capsules. The routing mechanism calculates the contribution of each low-level capsule vector to the high-level capsule vector:

[0061] ;

[0062] In the formula, is a probability value, indicating the lower layer Capsules for high-rise The contribution weight of each capsule; For feature blocks to The bias vector of the capsule; For feature blocks to The bias vector of each capsule is updated dynamically; Exponentialize the connection weights, amplifying larger weights; To normalize all weights, ensure that each coupling coefficient is a valid probability value.

[0063] The contributions of lower-level capsules are aggregated in the dynamic routing mechanism to get the input of the higher-level glue:

[0064] ;

[0065] In the formula, is the input vector of the high-level capsule.

[0066] The lower capsule After weighted summation and passing it to the high-level capsule, the capsule vector is nonlinearly transformed through the Squash activation function to ensure that the norm of the vector is compressed to the range of [0, 1]:

[0067] ;

[0068] In the formula, For high-level The output vector of each capsule; For the The modulus of the capsule vector.

[0069] 4. Output and decoding

[0070] The output vector of the high-level capsule is passed to an independent fully connected layer. The fully connected layer serves as the subsequent processing module of the capsule network, mapping the capsule features into the probability distribution of character categories, thereby achieving accurate recognition of character categories in bill images.

[0071] The output vector of the high-level capsule , the capsule features are mapped to the probability distribution of character categories using the fully connected layer to achieve accurate recognition of character categories in bill images:

[0072] ;

[0073] In the formula, is the weight matrix of the fully connected layer; is the bias term of the fully connected layer; is the probability distribution of the character categories in the bill image.

[0074] At the same time, the present invention provides:

[0075] A server comprises a processor and a memory, wherein the memory stores at least one program, and the program is loaded and executed by the processor to implement the above-mentioned automatic ticket information processing method based on capsule network.

[0076] A computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the above-mentioned automatic ticket information processing method based on capsule network.

[0077] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. The automatic bill information processing method based on capsule network is characterized by: The following steps are involved: S1. Input the bill image, perform normalization processing, and then convert the image features of the bill image into feature blocks of fixed size through the convolution operation of convolution preprocessing; S2, the initial capsule layer of the capsule network projects each feature block into an initial capsule vector; the initial capsule vector is nonlinearly transformed through the Squash activation function; S3, through the dynamic routing mechanism of the capsule network, update the connection weights between capsules and calculate the contribution value of each low-level capsule vector to the high-level capsule vector; The contribution value of each low-level capsule vector to the high-level capsule vector is weighted and summed up, and then passed to the high-level capsule. S4. The output vector of the high-level capsule is passed to an independent fully connected layer. The fully connected layer serves as the subsequent processing module of the capsule network, mapping the capsule features into the probability distribution of character categories to achieve accurate recognition of character categories in the bill image.

2. The method for automatic bill information processing based on capsule network according to claim 1, characterized in that: In step S1, the bill image is represented as a two-dimensional matrix , , is the height of the bill image, is the width of the bill image, R is the real number set of the bill image, indicating that each pixel value in the image is a real number; Indicates that the input bill image is a two-dimensional real number matrix with height multiplied by width; Ticket Image After normalization, the image features of the bill image are converted into a fixed-size feature block P through the convolution operation of the convolution preprocessing, and flattened into a vector as the input of the subsequent capsule layer: ； Among them, Conv2D is a module that performs two-dimensional convolution to learn and extract bill images The important features in the image; P is the result of the convolution operation, which is used to capture the bill image The local features in .

3. The method for automatic bill information processing based on capsule network according to claim 1, characterized in that: In step S2, the initial capsule layer of the capsule network projects each feature tile into an initial capsule vector: ； In the formula, For the Capsules The projection result of the feature blocks represents the output vector of the lower-level capsule; is a learnable weight matrix used to transform the input vector of the feature block Projection into capsule space; For the The vector formed by flattening the feature blocks; For feature blocks to The bias vector of each capsule is initially the output vector of each low-level capsule Applies a simple linear offset.

4. The method for automatic bill information processing based on capsule network according to claim 1, characterized in that: In step S3, the core of the capsule network is a dynamic routing mechanism, which is used to update the connection weights between capsules; the dynamic routing mechanism calculates the contribution of each low-level capsule vector to the high-level capsule vector: ； In the formula, is a probability value, indicating the lower layer Capsules for high-rise The contribution weight of each capsule; For feature blocks to The bias vector of the capsule; For feature blocks to The bias vector of each capsule is updated dynamically; Exponentialize the connection weights, amplifying larger weights; To normalize all weights, ensure that each coupling coefficient is a valid probability value; The contributions of lower-level capsules are aggregated in the dynamic routing mechanism to get the input of the higher-level glue: ； In the formula, is the input vector of the high-level capsule; The output vector of the lower-level capsule After weighted summation and passing it to the high-level capsule, the capsule vector is nonlinearly transformed through the Squash activation function to ensure that the norm of the vector is compressed to the range of [0, 1]: ； In the formula, For high-level The output vector of each capsule; For the The modulus of the capsule vector.

5. The method for automatic bill information processing based on capsule network according to claim 1, characterized in that: In step S4, the output vector of the high-level capsule , the capsule features are mapped to the probability distribution of character categories using the fully connected layer to achieve accurate recognition of character categories in bill images: ； In the formula, is the weight matrix of the fully connected layer; is the bias term of the fully connected layer; is the probability distribution of the character categories in the bill image.

6. A server, comprising a processor and a memory, characterized in that: At least one program is stored in the memory, and the program is loaded and executed by the processor to implement the automatic ticket information processing method based on capsule network as described in any one of claims 1 to 5.

7. A computer-readable storage medium, wherein at least one program is stored in the storage medium, characterized in that: The program is loaded and executed by a processor to implement the automatic ticket information processing method based on capsule network as described in any one of claims 1 to 5.

Images