A method and system for invoice text recognition based on multi-scale feature fusion

Abstract

This invention discloses an invoice text recognition method and system based on multi-scale feature fusion. The method includes: acquiring invoice images and preprocessing them; constructing an attention-embedded dilated convolutional neural network, including three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives the preprocessed invoice image, and its output is connected to the attention layer; training the attention-embedded dilated convolutional neural network; and using the trained attention-embedded dilated convolutional neural network to classify characters in each single-character image. The advantage of this invention is that it improves the accuracy of invoice text recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more specifically to a method and system for recognizing invoice text based on multi-scale feature fusion. Background Technology

[0002] In the invoice reimbursement process, it is necessary to identify the information in each field of the invoice for system entry. Traditional manual entry methods suffer from problems such as long processing time, high cost, and low accuracy. With the development of computer vision technology, image recognition algorithms can perform text recognition on scanned invoice images and extract field information. When recognizing the information in each field of an invoice, it is necessary to first divide the invoice image into different text regions, and then perform optical character recognition on the text in a single text region. Traditional image recognition methods, such as template matching, require algorithm and parameter design for each type of character or manual feature extraction, resulting in low accuracy and universality. Deep learning methods, through convolutional neural networks, can directly extract image features, transforming text recognition into a classification problem, classifying the characters contained in the image to obtain the recognition result. For example, Chinese Patent Publication No. CN109977957A discloses a deep learning-based invoice recognition method and system.

[0003] However, variations in the tilt angle, pixel count, and size ratio of invoice photographs lead to different data distributions in invoice images. Paper invoices may also have folds, misalignments, or stains in the printed area, introducing noise into the image data. Due to the limited receptive field of ordinary convolutional neural network models, their stability and accuracy are easily affected by changes in invoice image data, resulting in poor generalization and adaptability in invoice text recognition and ultimately, low accuracy in the recognition results. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that the existing invoice text recognition methods are easily affected by changes in invoice image data, resulting in poor generalization ability and adaptability in recognizing invoice text, and thus low accuracy of recognition results.

[0005] This invention solves the above-mentioned technical problems through the following technical means: an invoice text recognition method based on multi-scale feature fusion, comprising the following steps:

[0006] Step 1: Collect invoice images and preprocess them;

[0007] Step 2: Construct an attention-embedded dilated convolutional neural network, including three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives a pre-processed invoice image and its output is connected to the attention layer.

[0008] Step 3: Train the dilated convolutional neural network with the attention embedding;

[0009] Step 4: Use the trained attention-embedded dilated convolutional neural network to classify the characters in each single-character image. Combine the classification results of the single-character images segmented from each invoice image according to the spatial position of the single-character images to obtain the text recognition result of each invoice.

[0010] Beneficial effects: This invention designs an attention-embedded dilated convolutional neural network, using multiple dilated convolutions with different dilation rates to extract features from invoice images. While expanding the receptive field, it can effectively extract multi-scale contextual information, achieving effective image feature extraction and thus obtaining accurate character recognition results for invoice images. By adaptively fusing the captured multi-scale features through the attention layer, it improves the robustness and universality in the face of changes in invoice image data, solving the problem of insufficient adaptability of existing deep learning methods, thereby further improving the accuracy of recognition results.

[0011] Further, step one includes:

[0012] Step 1.1: Use a camera to take pictures of invoices. The collected invoice image dataset is denoted as V = {v1, v2, ..., v...} m ,...,v M}, v m Let m represent the m-th invoice image, where 1 ≤ m ≤ M, and M represents the number of invoice images.

[0013] Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially;

[0014] Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as P. m ={p m1 ,p m2 ,...,p mn ,...,p mN}, p mn Let N represent the nth single-character image segmented from the mth invoice image, where 1 ≤ n ≤ N, and N represents the number of single-character images segmented from each invoice image.

[0015] Furthermore, the working process of the multi-scale dilated convolutional layer in step two includes:

[0016] Step 2.1: For each dilated convolution branch, the feature extraction result is obtained using equation (1):

[0017]

[0018] In equation (1), f mn,d This represents the output after the dilated convolution operation on the d-th branch. W represents the dilated convolution operation. c,d and b c,d These represent the kernel parameters and bias of dilated convolution, respectively.

[0019] Step 2.2: Use equation (2) to process the extracted features f mn,d Perform activation and residual connections to obtain the output h of the d-th dilated convolution branch. mn,d :

[0020]

[0021] In equation (2), tanh is the hyperbolic tangent activation function, · represents a 1×1 convolution operation, and W cr,d It is the 1×1 convolution parameter in the d-th dilated convolution branch;

[0022] Step 2.3: Concatenate the output features of each dilated convolution branch according to channels to obtain the multi-scale feature H of the multi-scale dilated convolution layer output. mn =[h mn,1 ,h mn,2 ,h mn,3 ].

[0023] Furthermore, the working process of the attention layer in step two includes:

[0024] Step 2.4: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer, and obtains the first attention matrix α by using the first global average pooling. Then, it uses equation (3) to process the multi-scale features H. mn The output of the first attention matrix is ​​obtained by weighting by channel and using residual connections:

[0025] t mn,1 =αH mn +H mn (3)

[0026] Step 2.5: The attention layer further employs a second global average pooling to obtain a second attention matrix β, and uses equation (4) to process the multi-scale features H. mn The output of the second attention matrix is ​​obtained by weighting the features by feature points and using residual connections:

[0027] t mn,2 =βH mn +H mn (4).

[0028] Furthermore, the working process of the serial residual module and character classification module in step two is as follows:

[0029] Step 2.6: Use equation (5) to obtain the output of the first residual module:

[0030] T mn,1 =t mn,1 +t mn,2 +W r,1 ·p mn (5)

[0031] In equation (5), T mn,1 W is the output of the first residual module. r,1 The matrix parameters are the 1×1 convolution parameters of the first residual module;

[0032] Step 2.7, T mn,1 Input two serial residual modules with the same structure as the first residual module to obtain feature T. mn,3 ;

[0033] Step 2.8, for feature T mn,3 Perform global max pooling on each channel to obtain the feature vector O. mn Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6):

[0034] y mn =softmax(W o O mn +b o (6)

[0035] In equation (6), y mn This represents the probability distribution of the category to which the nth single-character image segmented from the mth invoice image belongs. `softmax` is the normalized activation function, and `W` represents the probability distribution of the category to which the nth single-character image belongs. o and b o These are the matrix parameters and biases for the fully connected operation, respectively.

[0036] This invention also provides an invoice text recognition system based on multi-scale feature fusion, comprising:

[0037] The preprocessing section is used to collect invoice images and preprocess them.

[0038] The network construction part is used to build an attention-embedded dilated convolutional neural network, which includes three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives a pre-processed invoice image and its output is connected to the attention layer.

[0039] The network training section is used to train the dilated convolutional neural network with the attention embedding.

[0040] The recognition result output section is used to classify characters in each single-character image using a pre-trained attention-embedded dilated convolutional neural network. The classification results of the single-character images segmented from each invoice image are combined according to the spatial position of the single-character images to obtain the text recognition result of each invoice.

[0041] Furthermore, the preprocessing section is also used for:

[0042] Step 1.1: Use a camera to take pictures of invoices. The collected invoice image dataset is denoted as V = {v1, v2, ..., v...} m ,...,v M}, v m Let m represent the m-th invoice image, where 1 ≤ m ≤ M, and M represents the number of invoice images.

[0043] Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially;

[0044] Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as P. m ={p m1 ,p m2 ,...,p mn ,...,p mN}, p mn Let N represent the nth single-character image segmented from the mth invoice image, where 1 ≤ n ≤ N, and N represents the number of single-character images segmented from each invoice image.

[0045] Furthermore, the network construction portion is also used for:

[0046] Step 2.1: For each dilated convolution branch, the feature extraction result is obtained using equation (1):

[0047]

[0048] In equation (1), f mn,d This represents the output after the dilated convolution operation on the d-th branch. W represents the dilated convolution operation.c,d and b c,d These represent the kernel parameters and bias of dilated convolution, respectively.

[0049] Step 2.2: Use equation (2) to process the extracted features f mn,d Perform activation and residual connections to obtain the output h of the d-th dilated convolution branch. mn,d :

[0050]

[0051] In equation (2), tanh is the hyperbolic tangent activation function, · represents a 1×1 convolution operation, and W cr,d It is the 1×1 convolution parameter in the d-th dilated convolution branch;

[0052] Step 2.3: Concatenate the output features of each dilated convolution branch according to channels to obtain the multi-scale feature H of the multi-scale dilated convolution layer output. mn =[h mn,1 ,h mn,2 ,h mn,3 ].

[0053] Furthermore, the network construction portion is also used for:

[0054] Step 2.4: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer, and obtains the first attention matrix α by using the first global average pooling. Then, it uses equation (3) to process the multi-scale features H. mn The output of the first attention matrix is ​​obtained by weighting by channel and using residual connections:

[0055] t mn,1 =αH mn +H mn (3)

[0056] Step 2.5: The attention layer further employs a second global average pooling to obtain a second attention matrix β, and uses equation (4) to process the multi-scale features H. mn The output of the second attention matrix is ​​obtained by weighting the features by feature points and using residual connections:

[0057] t mn,2 =βH mn +H mn (4).

[0058] Furthermore, the network construction portion is also used for:

[0059] Step 2.6: Use equation (5) to obtain the output of the first residual module:

[0060] T mn,1 =t mn,1 +tmn,2 +W r,1 ·p mn (5)

[0061] In equation (5), T mn,1 W is the output of the first residual module. r,1 The matrix parameters are the 1×1 convolution parameters of the first residual module;

[0062] Step 2.7, T mn,1 Input two serial residual modules with the same structure as the first residual module to obtain feature T. mn,3 ;

[0063] Step 2.8, for feature T mn,3 Perform global max pooling on each channel to obtain the feature vector O. mn Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6):

[0064] y mn =softmax(W o O mn +b o (6)

[0065] In equation (6), y mn This represents the probability distribution of the category to which the nth single-character image segmented from the mth invoice image belongs. `softmax` is the normalized activation function, and `W` represents the probability distribution of the category to which the nth single-character image belongs. o and b o These are the matrix parameters and biases for the fully connected operation, respectively.

[0066] The advantages of this invention are as follows: This invention designs an attention-embedded dilated convolutional neural network, which uses multiple dilated convolutions with different dilation rates to extract features from invoice images. While expanding the receptive field, it can effectively extract multi-scale contextual information, achieving effective image feature extraction and thus obtaining accurate character recognition results for invoice images. By adaptively fusing the captured multi-scale features through the attention layer, it improves the robustness and universality in the face of changes in invoice image data, solving the problem of insufficient adaptability of existing deep learning methods, thereby further improving the accuracy of recognition results. Attached Figure Description

[0067] Figure 1 This is a flowchart of an invoice text recognition method based on multi-scale feature fusion disclosed in Embodiment 1 of the present invention;

[0068] Figure 2 This is a schematic diagram of the dilated convolutional neural network structure with attention embedding in an invoice text recognition method based on multi-scale feature fusion disclosed in Embodiment 1 of the present invention.

[0069] Figure 3 This is a schematic diagram of the attention layer structure in an invoice text recognition method based on multi-scale feature fusion disclosed in Embodiment 1 of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1

[0072] like Figure 1 As shown, this invention provides a method for recognizing invoice text based on multi-scale feature fusion, the method comprising the following steps:

[0073] Step 1: Collect the invoice image dataset and preprocess the invoice images. The specific process is as follows:

[0074] Step 1.1: Use a camera to take pictures of invoices. The collected invoice image dataset is denoted as V = {v1, v2, ..., v...} m ,...,v M}, v m Let m represent the m-th invoice image, where 1 ≤ m ≤ M, and M represents the number of invoice images.

[0075] Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially;

[0076] Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as P. m ={p m1 ,p m2 ,...,p mn ,...,p mN}, p mn Let N represent the nth single-character image segmented from the mth invoice image, where 1 ≤ n ≤ N, and N represents the number of single-character images segmented from each invoice image.

[0077] Step 2, as follows Figure 2 As shown, an attention-embedded dilated convolutional neural network is constructed, comprising three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The specific process is as follows:

[0078] Step 2.1: The multi-scale dilated convolutional layer includes three parallel dilated convolutional branches. Each branch contains dilated convolutions with different dilation rates to capture multi-scale features. The feature extraction result of each dilated convolutional branch is obtained using equation (1):

[0079]

[0080] In equation (1), f mn,d This represents the output after the dilated convolution operation in the d-th branch. The dilation rates used in the dilated convolutions in the three branches are 2, 3, and 4, respectively. W represents the dilated convolution operation. c,d and b c,d These represent the kernel parameters and bias of dilated convolution, respectively.

[0081] Step 2.2: Use equation (2) to process the extracted features f mn,d Perform activation and residual connections to obtain the output h of the d-th dilated convolution branch. mn,d :

[0082]

[0083] In equation (2), tanh is the hyperbolic tangent activation function, · represents a 1×1 convolution operation, and W cr,d It is the 1×1 convolution parameter in the d-th dilated convolution branch;

[0084] Step 2.3: Concatenate the output features of each convolutional branch according to channels to obtain the multi-scale feature H of the output of the multi-scale dilated convolutional layer. mn =[h mn,1 ,h mn,2 ,h mn,3 ];

[0085] Step 2.4, as follows Figure 3 As shown, the attention layer receives multi-scale features output by the multi-scale dilated convolutional layer, and obtains the first attention matrix α∈R using the first global average pooling. C×1 The first global average pooling is applied to multi-scale features H mn ∈R C×A×B Global average pooling is performed on each channel. Then, equation (3) is used to perform multi-scale feature H. mn The output t of the first attention matrix is ​​obtained by weighting by channel and using residual connections. mn,1 ∈R C×A×B :

[0086] t mn,1 =αH mn +H mn (3)

[0087] Step 2.5: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer and uses a second global average pooling to obtain the second attention matrix β∈R. A×B The second global average pooling is applied to multi-scale features H mn ∈R C×A×B Global average pooling is performed on feature points at the same location in different channels. Then, equation (4) is used to process the multi-scale feature H. mn The output t of the second attention matrix is ​​obtained by weighting the features and using residual connections. mn,2 ∈R C×A×B :

[0088] t mn,2 =βH mn +H mn (4)

[0089] Step 2.6, t mn,1 and t mn,2 The fused multi-scale features are obtained by summing them, and the output of the residual module is obtained using equation (5):

[0090] T mn,1 =t mn,1 +t mn,2 +W r,1 ·p mn (5)

[0091] In equation (5), T mn,1 This represents the output of the first residual module, where · indicates a 1×1 convolution operation, and W... r,1 The matrix parameters are the 1×1 convolution parameters of the first residual module;

[0092] Step 2.7, T mn,1 Input two serial residual modules with the same structure as the first residual module to obtain feature T. mn,3 ;

[0093] Step 2.8, for feature T mn,3 Perform global max pooling on each channel to obtain the feature vector O. mn Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6):

[0094] y mn =softmax(W o O mn +b o (6)

[0095] In equation (6), y mnLet W represent the probability distribution of the category of the nth single-character image segmented from the mth invoice image. The index of the position with the highest probability is the classification result. Softmax is the normalized activation function. o and b o These are the matrix parameters and biases for the fully connected operation, respectively.

[0096] Step 3: Using the following loss function, train the attention-embedded dilated convolutional neural network through backpropagation algorithm, and use the Adam optimizer to speed up parameter updates. Stop training when the loss function value is minimized or the training iterations are reached.

[0097]

[0098] In equation (7), y mn,k It represents the probability of classifying the nth single-character image segmented from the mth invoice image into the kth character category. γ represents the frequency of the k-th text category in all single-character images, and γ is the focusing parameter. This loss function can alleviate the overfitting problem that may be caused by the imbalance in the distribution of character data categories.

[0099] Step 4: Use the trained attention-embedded dilated convolutional neural network to classify the characters in each single-character image. Combine the classification results of the single-character images segmented from each invoice image according to the spatial position of the single-character images to obtain the text recognition result of each invoice.

[0100] Through the above technical solutions, this invention designs an attention-embedded dilated convolutional neural network, using multiple dilated convolutions with different dilation rates to extract features from invoice images. While expanding the receptive field, it can effectively extract multi-scale contextual information, achieving effective image feature extraction and thus obtaining accurate character recognition results for invoice images. By adaptively fusing the captured multi-scale features through the attention layer, it improves the robustness and universality in the face of changes in invoice image data, solving the problem of insufficient adaptability of existing deep learning methods, thereby further improving the accuracy of recognition results.

[0101] Example 2

[0102] Based on Embodiment 1, Embodiment 2 of the present invention also provides an invoice text recognition system based on multi-scale feature fusion, comprising:

[0103] The preprocessing section is used to collect invoice images and preprocess them.

[0104] The network construction part is used to build an attention-embedded dilated convolutional neural network, which includes three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives a pre-processed invoice image and its output is connected to the attention layer.

[0105] The network training section is used to train the dilated convolutional neural network with the attention embedding.

[0106] The recognition result output section is used to classify each single character image using a pre-trained attention-embedded dilated convolutional neural network. The classification results of the single character images segmented from each invoice image are combined according to the spatial position of the single character images to obtain the text recognition result of each invoice.

[0107] Specifically, the preprocessing section is also used for:

[0108] Step 1.1: Use a camera to take pictures of invoices. The collected invoice image dataset is denoted as V = {v1, v2, ..., v...} m ,...,v M}, v m Let m represent the m-th invoice image, where 1 ≤ m ≤ M, and M represents the number of invoice images.

[0109] Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially;

[0110] Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as P. m ={p m1 ,p m2 ,...,p mn ,...,p mN}, p mn Let N represent the nth single-character image segmented from the mth invoice image, where 1 ≤ n ≤ N, and N represents the number of single-character images segmented from each invoice image.

[0111] More specifically, the network construction component is also used for:

[0112] Step 2.1: For each dilated convolution branch, the feature extraction result is obtained using equation (1):

[0113]

[0114] In equation (1), f mn,d This represents the output after the dilated convolution operation on the d-th branch. W represents the dilated convolution operation. c,dand b c,d These represent the kernel parameters and bias of dilated convolution, respectively.

[0115] Step 2.2: Use equation (2) to process the extracted features f mn,d Perform activation and residual connections to obtain the output h of the d-th dilated convolution branch. mn,d :

[0116]

[0117] In equation (2), tanh is the hyperbolic tangent activation function, · represents a 1×1 convolution operation, and W cr,d It is the 1×1 convolution parameter in the d-th dilated convolution branch;

[0118] Step 2.3: Concatenate the output features of each dilated convolution branch according to channels to obtain the multi-scale feature H of the multi-scale dilated convolution layer output. mn =[h mn,1 ,h mn,2 ,h mn,3 ].

[0119] More specifically, the network construction component is also used for:

[0120] Step 2.4: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer, and obtains the first attention matrix α by using the first global average pooling. Then, it uses equation (3) to process the multi-scale features H. mn The output of the first attention matrix is ​​obtained by weighting by channel and using residual connections:

[0121] t mn,1 =αH mn +H mn (3)

[0122] Step 2.5: The attention layer further employs a second global average pooling to obtain a second attention matrix β, and uses equation (4) to process the multi-scale features H. mn The output of the second attention matrix is ​​obtained by weighting the features by feature points and using residual connections:

[0123] t mn,2 =βH mn +H mn (4).

[0124] More specifically, the network construction component is also used for:

[0125] Step 2.6: Use equation (5) to obtain the output of the first residual module:

[0126] T mn,1 =t mn,1 +t mn,2+W r,1 ·p mn (5)

[0127] In equation (5), T mn,1 W is the output of the first residual module. r,1 The matrix parameters are the 1×1 convolution parameters of the first residual module;

[0128] Step 2.7, T mn,1 Input two serial residual modules with the same structure as the first residual module to obtain feature T. mn,3 ;

[0129] Step 2.8, for feature T mn,3 Perform global max pooling on each channel to obtain the feature vector O. mn Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6):

[0130] y mn =softmax(W o O mn +b o (6)

[0131] In equation (6), y mn This represents the probability distribution of the category to which the nth single-character image segmented from the mth invoice image belongs. `softmax` is the normalized activation function, and `W` represents the probability distribution of the category to which the nth single-character image belongs. o and b o These are the matrix parameters and biases for the fully connected operation, respectively.

[0132] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing invoice text based on multi-scale feature fusion, characterized in that, Includes the following steps: Step 1: Collect invoice images and preprocess them; Step 1 includes: Step 1.1: Use a camera to take pictures of invoices. Record the collected invoice image dataset as follows: , Indicates the first Image of an invoice. , Indicates the number of invoice images; Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially; Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as . , Indicates from the first The first segment of the invoice image A single-character image, , This indicates the number of single-character images segmented from each invoice image; Step 2: Construct an attention-embedded dilated convolutional neural network, including three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives a pre-processed invoice image, and its output is connected to the attention layer. The operation of the multi-scale dilated convolutional layers in Step 2 includes: Step 2.1: For each dilated convolution branch, the feature extraction result is obtained using equation (1): (1) In equation (1), This represents the output after the dilated convolution operation on the d-th branch. This indicates a dilated convolution operation. and These represent the kernel parameters and bias of dilated convolution, respectively. Step 2.2: Use equation (2) to extract the features. Perform activation and residual connections to obtain the output of the d-th dilated convolution branch. : (2) In equation (2), It is the hyperbolic tangent activation function. This represents a 1×1 convolution operation. It is the 1×1 convolution parameter in the d-th dilated convolution branch; Step 2.3: Concatenate the output features of each dilated convolution branch according to channels to obtain the multi-scale features output by the multi-scale dilated convolution layer. ; The working process of the attention layer in step two includes: Step 2.4: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer and obtains the first attention matrix using the first global average pooling. Using equation (3) for multi-scale features The output of the first attention matrix is ​​obtained by weighting by channel and using residual connections: (3) Step 2.5: The attention layer further employs a second global average pooling method to obtain a second attention matrix. Using equation (4) for multi-scale features The output of the second attention matrix is ​​obtained by weighting the features by feature points and using residual connections: (4)； Step 3: Train the dilated convolutional neural network with the attention embedding; Step 4: Use the trained attention-embedded dilated convolutional neural network to classify the characters in each single-character image. Combine the classification results of the single-character images segmented from each invoice image according to the spatial position of the single-character images to obtain the text recognition result of each invoice.

2. The invoice text recognition method based on multi-scale feature fusion according to claim 1, characterized in that, The working process of the serial residual module and character classification module in step two is as follows: Step 2.6: Use equation (5) to obtain the output of the first residual module: (5) In equation (5), This is the output of the first residual module. The matrix parameters are the 1×1 convolution parameters of the first residual module; Step 2.7, Input two serial residual modules with the same structure as the first residual module to obtain the features. ; Step 2.8, for features Perform global max pooling on each channel to obtain the feature vector. Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6): (6) In equation (6), Let represent the probability distribution of the category to which the nth single-character image segmented from the mth invoice image belongs. It is a normalized activation function. and These are the matrix parameters and biases for the fully connected operation, respectively.

3. An invoice text recognition system based on multi-scale feature fusion, characterized in that, include: The preprocessing section is used to collect invoice images and preprocess them. The preprocessing section is also used for: Step 1.1: Use a camera to take pictures of invoices. Record the collected invoice image dataset as follows: , Indicates the first Image of an invoice. , Indicates the number of invoice images; Step 1.2: Perform grayscale and binarization processing on each invoice image sequentially; Step 1.3: Perform single-character segmentation on each invoice image to obtain a set of single-character images to be recognized, denoted as . , Indicates from the first The first segment of the invoice image A single-character image, , This indicates the number of single-character images segmented from each invoice image; The network construction section is used to build an attention-embedded dilated convolutional neural network, including three identical serial residual modules and a character classification module. The residual modules include multi-scale dilated convolutional layers and attention layers. The multi-scale dilated convolutional layers include multiple dilated convolutional branches with different dilation rates. Each dilated convolutional branch receives a pre-processed invoice image, and its output is connected to the attention layer. The network construction section is also used for: Step 2.1: For each dilated convolution branch, the feature extraction result is obtained using equation (1): (1) In equation (1), This represents the output after the dilated convolution operation on the d-th branch. This indicates a dilated convolution operation. and These represent the kernel parameters and bias of dilated convolution, respectively. Step 2.2: Use equation (2) to extract the features. Perform activation and residual connections to obtain the output of the d-th dilated convolution branch. : (2) In equation (2), It is the hyperbolic tangent activation function. This represents a 1×1 convolution operation. It is the 1×1 convolution parameter in the d-th dilated convolution branch; Step 2.3: Concatenate the output features of each dilated convolution branch according to channels to obtain the multi-scale features output by the multi-scale dilated convolution layer. ; Step 2.4: The attention layer receives the multi-scale features output by the multi-scale dilated convolutional layer and obtains the first attention matrix using the first global average pooling. Using equation (3) for multi-scale features The output of the first attention matrix is ​​obtained by weighting by channel and using residual connections: (3) Step 2.5: The attention layer further employs a second global average pooling method to obtain a second attention matrix. Using equation (4) for multi-scale features The output of the second attention matrix is ​​obtained by weighting the features by feature points and using residual connections: (4)； The network training section is used to train the dilated convolutional neural network with the attention embedding. The recognition result output section is used to classify each single character image using a pre-trained attention-embedded dilated convolutional neural network. The classification results of the single character images segmented from each invoice image are combined according to the spatial position of the single character images to obtain the text recognition result of each invoice.

4. The invoice text recognition method based on multi-scale feature fusion according to claim 3, characterized in that, The network construction component is also used for: Step 2.6: Use equation (5) to obtain the output of the first residual module: (5) In equation (5), This is the output of the first residual module. The matrix parameters are the 1×1 convolution parameters of the first residual module; Step 2.7, Input two serial residual modules with the same structure as the first residual module to obtain the features. ; Step 2.8, for features Perform global max pooling on each channel to obtain the feature vector. Then, it is input into the character classification module to obtain the output of the attention-embedded dilated convolutional neural network using equation (6): (6) In equation (6), Let represent the probability distribution of the category to which the nth single-character image segmented from the mth invoice image belongs. It is a normalized activation function. and These are the matrix parameters and biases for the fully connected operation, respectively.

Images