A method for recognizing bill text

Abstract

The invention relates to image processing, and in particular to a bill text recognition method, which comprises the following steps: reading a bill image to be recognized, determining feature points in the bill image to be recognized, and performing rotation correction on the bill image to be recognized according to the feature points; obtaining RGB three-channel values ​​of each pixel point in the bill image to be recognized, delimiting a sub-image in a preset size area with each pixel point as the center, and calculating a variance matrix and a maximum value matrix of the bill image to be recognized with respect to an R channel; removing a seal area in the bill image to be recognized according to the variance matrix and the maximum value matrix of the bill image to be recognized with respect to the R channel, and performing image restoration on the bill image to be recognized; performing image interpolation on the bill image to be recognized by using an interpolation method based on wavelet transform; and performing text recognition on a text box area by using a text recognition model. The technical scheme provided by the invention can overcome the defect of the prior art that it is difficult to perform accurate text recognition on bills due to seal interference and image rotation.

Description

Technical Field

[0001] The invention relates to image processing, and in particular to a bill text recognition method. Background Art

[0002] OCR technology converts various bills, newspapers, books, manuscripts and other printed materials into image data through optical input methods such as scanning, and then uses text recognition technology to perform text recognition on the image data. For example, OCR technology can be used to obtain information from bills, achieving recognition and extraction of bill information in seconds.

[0003] However, the format of bills is usually complex. For example, there are usually at least two seals on the face of a bill (one is the pre-stamp of the bill, and the other is the stamp of an individual or unit). These seals may cover part of the text to be recognized. Therefore, when using OCR technology to obtain information in the bill, these seals may affect the accuracy of text recognition. In addition, due to the rotation of the bill image, the accuracy of text recognition is also affected to a certain extent. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method for recognizing text on bills, which can effectively overcome the defect of the prior art that it is difficult to accurately recognize text on bills due to seal interference and image rotation.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] A method for bill text recognition comprises the following steps:

[0007] S1, reading the bill image to be identified, determining the feature points in the bill image to be identified, and performing rotation correction on the bill image to be identified according to the feature points;

[0008] S2, obtaining the RGB three-channel values ​​of each pixel in the bill image to be identified, demarcating a sub-image in a preset size area with each pixel as the center, and calculating the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel;

[0009] S3, removing the seal area in the bill image to be identified according to the variance matrix and the maximum value matrix of the bill image to be identified with respect to the R channel, and performing image restoration on the bill image to be identified;

[0010] S4, using an interpolation method based on wavelet transform to perform image interpolation on the bill image to be identified;

[0011] S5. Determine a text box area in the bill image to be recognized, and use a text recognition model to perform text recognition on the text box area.

[0012] Preferably, in S1, the bill image to be identified is read, feature points in the bill image to be identified are determined, and rotation correction is performed on the bill image to be identified according to the feature points, including:

[0013] S11, reading the bill image to be identified through the image processing library;

[0014] S12, performing Gaussian smoothing on the bill image to be recognized, gradually downsampling, and constructing a Gaussian pyramid;

[0015] S13, determining a suitable resolution, and performing edge detection on the to-be-recognized bill image of the corresponding layer in the Gaussian pyramid using an edge detection algorithm;

[0016] S14, determining feature points in the bill image to be identified, and calculating the tilt angle of the bill image to be identified;

[0017] S15, determining the rotation center of the bill image to be identified, and performing rotation correction on the bill image to be identified based on the tilt angle.

[0018] Preferably, determining the feature points in the bill image to be identified and calculating the tilt angle of the bill image to be identified in S14 includes:

[0019] S141, performing connected domain analysis on the to-be-recognized bill image after edge detection, and marking all connected domains;

[0020] S142, selecting feature points that can reflect the tilt of the bill image to be identified from the connected domain;

[0021] S143, by analyzing the orientation relationship between the feature points, selecting target feature points belonging to the same frame from the feature points;

[0022] S144, fitting the target feature points, and calculating the angle between the fitting result and the horizontal direction to obtain the tilt angle of the recognized bill image;

[0023] The feature points are the corner points of the frame or the midpoints of the long sides.

[0024] Preferably, in S2, the RGB three-channel values ​​of each pixel in the to-be-recognized bill image are obtained, a sub-image is delineated in a preset size area with each pixel as the center, and the variance matrix and the maximum value matrix of the to-be-recognized bill image with respect to the R channel are calculated, including:

[0025] S21, separating the RGB three channels of the bill image to be identified, obtaining three single-channel images of R, G, and B respectively, and obtaining the RGB three-channel value of each pixel point in the bill image to be identified;

[0026] S22, delimiting a sub-image in a region of a preset size with the pixel as the center, and obtaining R channel values ​​of all the pixels in the sub-image;

[0027] S23, calculating the variance of the R channel values ​​of all pixels in the sub-image, and extracting the maximum value of the R channel values ​​of all pixels in the sub-image, to obtain the R channel variance and the R channel maximum value of the central pixel;

[0028] S24, repeat S22-S23 until the R channel variance and R channel maximum value of each pixel in the bill image to be identified are obtained;

[0029] S25, constructing a variance matrix based on the R channel variance of each pixel in the to-be-recognized bill image, and constructing a maximum value matrix based on the R channel maximum value of each pixel in the to-be-recognized bill image;

[0030] Among them, the preset size area is 3*3.

[0031] Preferably, in S3, the seal area in the bill image to be identified is removed according to the variance matrix and the maximum value matrix of the bill image to be identified with respect to the R channel, and image restoration is performed on the bill image to be identified, including:

[0032] S31. According to the distribution of the variance matrix and the maximum value matrix, set an appropriate variance threshold and a maximum value threshold;

[0033] S32, comparing the R channel variance of each pixel in the bill image to be identified with the variance threshold, and the R channel maximum value with the maximum value threshold, respectively; if the R channel variance is less than the variance threshold, the R channel variance is set to 0, otherwise the R channel variance is set to 1; if the R channel maximum value is less than the maximum value threshold, the R channel maximum value is set to 0, otherwise the R channel maximum value is set to 1;

[0034] S33, performing a logical OR operation on the R channel variance and the R channel maximum value of each pixel in the to-be-recognized bill image, and determining the pixel whose calculation result is 1 as a potential seal area in the to-be-recognized bill image;

[0035] S34, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region, and then removing the seal region;

[0036] S35. Perform image restoration on the bill image to be recognized by using a restoration method based on texture synthesis.

[0037] Preferably, in S34, morphological operations are used to connect potential seal areas in the bill image to be identified to form a complete seal area, and the seal area is removed, including:

[0038] S331, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region;

[0039] S332, generating a binary image mask based on the stamp area, setting the stamp area to white and other areas to black;

[0040] S333: Use the image mask to perform a mask operation on the bill image to be identified, and replace the pixel values ​​of the pixel points in the bill image to be identified corresponding to the white pixel points in the image mask with the background color.

[0041] Preferably, in S4, an interpolation method based on wavelet transform is used to perform image interpolation on the bill image to be identified, including:

[0042] S41, performing wavelet transform on the bill image to be identified, so as to decompose the bill image to be identified into a low-frequency part reflecting the smooth area of ​​the image and a high-frequency part containing image detail information;

[0043] S42, for the low-frequency part, using bilinear interpolation to perform image interpolation;

[0044] S43. For the high frequency part, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation;

[0045] S44, performing inverse wavelet transform on the interpolated low-frequency part and high-frequency part to reconstruct the interpolated bill image to be recognized;

[0046] Among them, inter-scale persistence refers to the fact that wavelet coefficients have a certain correlation between different scales, that is, larger or smaller wavelet coefficients may maintain larger or smaller values ​​between adjacent scales. This correlation is captured by the state transition probability matrix of the hidden Markov model;

[0047] Exponential decay means that as the scale increases, the value of the wavelet coefficient shows an exponential decay trend. This characteristic is reflected in the observation probability matrix of the hidden Markov model.

[0048] Preferably, for the high frequency part, in S43, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to describe the probability structure of the wavelet coefficients to achieve image interpolation, including:

[0049] S431, construct a hidden Markov model, use the known observation sequence to train the model, and estimate the initial state probability vector, state transition probability matrix and observation probability matrix;

[0050] S432. Input the wavelet coefficient sequence reflecting the high-frequency part into the trained hidden Markov model, use the forward-backward algorithm to calculate the probability of the wavelet coefficient sequence under the given model, use the Viterbi algorithm to find the hidden state sequence that is most likely to produce the wavelet coefficient sequence, and predict the unknown wavelet coefficients based on the observed probability matrix and the hidden state sequence.

[0051] Preferably, the text recognition model is constructed based on an improved convolutional recurrent neural network, the recurrent layer in the improved convolutional recurrent neural network adopts a bidirectional gated recurrent unit, and the improved convolutional recurrent neural network sequentially includes an input layer, a convolutional layer, a bidirectional gated recurrent unit, a hidden state merging and an output layer;

[0052] The convolution layer extracts the local features of the input data in the input layer and generates a feature map by sliding the convolution kernel on the input data.

[0053] A bidirectional gated recurrent unit, including a forward gated recurrent unit and a backward gated recurrent unit. The forward gated recurrent unit processes the input data in the positive order to capture the forward context information. The backward gated recurrent unit processes the input data in the reverse order to capture the backward context information. The forward gated recurrent unit and the backward gated recurrent unit generate their own hidden states at each time step.

[0054] Hidden state merging: At each time step, the hidden states of the forward gated recurrent unit and the backward gated recurrent unit are merged. The merged hidden state contains the complete context information at that time step.

[0055] The output layer maps the merged hidden states to the required output space to generate the final output sequence.

[0056] Preferably, before determining the text box area in the bill image to be recognized in S5 and performing text recognition on the text box area using the text recognition model, the following steps are included:

[0057] S51, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0058] S52, setting the loss function and optimizer of the text recognition model;

[0059] S53, inputting the training set into the text recognition model for model training;

[0060] S54, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0061] S55. If the loss value is less than the preset threshold, the model training ends, and the current text recognition model is the trained text recognition model. Otherwise, the process returns to S53 and continues to use the training set for model training.

[0062] S56. Input the validation set into the trained text recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0063] S57. Input the test set into the tuned text recognition model to evaluate the model performance.

[0064] Compared with the prior art, the bill text recognition method provided by the present invention has the following beneficial effects:

[0065] 1) Read the bill image to be identified, determine the feature points in the bill image to be identified, perform rotation correction on the bill image to be identified based on the feature points, construct a Gaussian pyramid, and use the edge detection algorithm to perform edge detection on the bill image to be identified in the corresponding layer of the Gaussian pyramid, and then calculate the tilt angle of the bill image to be identified by extracting the feature points in the bill image to be identified, thereby realizing rotation correction of the bill image to be identified;

[0066] 2) Obtain the RGB three-channel values ​​of each pixel in the bill image to be identified, delineate a sub-image in a preset size area with each pixel as the center, and calculate the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel. Remove the seal area in the bill image to be identified based on the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel, and perform image restoration on the bill image to be identified. By constructing the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel, and performing logical OR operations on the corresponding elements in the variance matrix and the maximum value matrix, the seal area and its edge image can be effectively determined, thereby realizing accurate removal of the seal area;

[0067] 3) The interpolation method based on wavelet transform is used to interpolate the image of the bill to be identified. For the low-frequency part, bilinear interpolation is used for image interpolation. For the high-frequency part, by analyzing the inter-scale continuity and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation. The low-frequency part and the high-frequency part after interpolation are subjected to inverse wavelet transform to reconstruct the interpolated bill image to be identified, thereby achieving effective repair of the bill image to be identified and improving the image quality of the bill image to be identified, providing image support for the text recognition task of the subsequent text recognition model;

[0068] 4) The text recognition model is built based on an improved convolutional recurrent neural network. The recurrent layer in the improved convolutional recurrent neural network adopts a bidirectional gated recurrent unit. The use of the bidirectional gated recurrent unit can better capture the complete contextual information at each time step, effectively improving the accuracy of text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 It is a schematic diagram of the process of the present invention;

[0071] Figure 2 A schematic diagram of a process of performing rotation correction on a bill image to be identified in the present invention;

[0072] Figure 3 A schematic diagram of the process of removing the seal area in the bill image to be identified in the present invention;

[0073] Figure 4 It is a schematic diagram of the process of performing image interpolation on the bill image to be recognized in the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] A bill text recognition method, such as Figure 1 and Figure 2 As shown, S1, read the bill image to be identified, determine the feature points in the bill image to be identified, and perform rotation correction on the bill image to be identified according to the feature points, specifically including:

[0076] S11, reading the bill image to be identified through the image processing library;

[0077] S12, performing Gaussian smoothing on the bill image to be recognized, gradually downsampling, and constructing a Gaussian pyramid;

[0078] S13, determining a suitable resolution, and performing edge detection on the to-be-recognized bill image of the corresponding layer in the Gaussian pyramid using an edge detection algorithm;

[0079] S14, determining feature points in the bill image to be identified, and calculating the tilt angle of the bill image to be identified;

[0080] S15, determining the rotation center of the bill image to be identified, and performing rotation correction on the bill image to be identified based on the tilt angle.

[0081] Specifically, determining the feature points in the bill image to be identified and calculating the tilt angle of the bill image to be identified in S14 includes:

[0082] S141, performing connected domain analysis on the to-be-recognized bill image after edge detection, and marking all connected domains;

[0083] S142, selecting feature points that can reflect the tilt of the bill image to be identified from the connected domain;

[0084] S143, by analyzing the orientation relationship between the feature points, selecting target feature points belonging to the same frame from the feature points;

[0085] S144, fitting the target feature points, and calculating the angle between the fitting result and the horizontal direction to obtain the tilt angle of the recognized bill image;

[0086] The feature points are the corner points of the frame or the midpoints of the long sides.

[0087] The above technical solution reads the bill image to be identified, determines the feature points in the bill image to be identified, performs rotation correction on the bill image to be identified based on the feature points, constructs a Gaussian pyramid, and uses an edge detection algorithm to perform edge detection on the bill image to be identified in the corresponding layer of the Gaussian pyramid, and then calculates the inclination angle of the bill image to be identified by extracting the feature points in the bill image to be identified, thereby realizing rotation correction of the bill image to be identified.

[0088] like Figure 1 and Figure 3 As shown, S2, obtain the RGB three-channel value of each pixel in the bill image to be identified, delineate a sub-image in a preset size area with each pixel as the center, and calculate the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel, specifically including:

[0089] S21, separating the RGB three channels of the bill image to be identified, obtaining three single-channel images of R, G, and B respectively, and obtaining the RGB three-channel value of each pixel point in the bill image to be identified;

[0090] S22, delimiting a sub-image in a region of a preset size with the pixel as the center, and obtaining R channel values ​​of all the pixels in the sub-image;

[0091] S23, calculating the variance of the R channel values ​​of all pixels in the sub-image, and extracting the maximum value of the R channel values ​​of all pixels in the sub-image, to obtain the R channel variance and the R channel maximum value of the central pixel;

[0092] S24, repeat S22-S23 until the R channel variance and R channel maximum value of each pixel in the bill image to be identified are obtained;

[0093] S25, constructing a variance matrix based on the R channel variance of each pixel in the to-be-recognized bill image, and constructing a maximum value matrix based on the R channel maximum value of each pixel in the to-be-recognized bill image;

[0094] Among them, the preset size area is 3*3.

[0095] S3, removing the seal area in the bill image to be identified according to the variance matrix and the maximum value matrix of the bill image to be identified with respect to the R channel, and performing image restoration on the bill image to be identified, specifically including:

[0096] S31. According to the distribution of the variance matrix and the maximum value matrix, set an appropriate variance threshold and a maximum value threshold;

[0097] S32, comparing the R channel variance of each pixel in the bill image to be identified with the variance threshold, and the R channel maximum value with the maximum value threshold, respectively; if the R channel variance is less than the variance threshold, the R channel variance is set to 0, otherwise the R channel variance is set to 1; if the R channel maximum value is less than the maximum value threshold, the R channel maximum value is set to 0, otherwise the R channel maximum value is set to 1;

[0098] S33, performing a logical OR operation on the R channel variance and the R channel maximum value of each pixel in the to-be-recognized bill image, and determining the pixel whose calculation result is 1 as a potential seal area in the to-be-recognized bill image;

[0099] S34, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region, and then removing the seal region;

[0100] S35. Perform image restoration on the bill image to be recognized by using a restoration method based on texture synthesis.

[0101] The following is an analysis of the logical OR operation between the R channel variance and the R channel maximum value of the pixel in the bill image to be identified:

[0102] 1) When the variance of the R channel is 0 and the maximum value of the R channel is 0, this indicates that the pixel is most likely not red and the surrounding color does not change much, indicating that the pixel is a non-stamp area and does not need to be removed;

[0103] 2) When the variance of the R channel is 0 and the maximum value of the R channel is 1, this indicates that the pixel is most likely red and the surrounding colors do not change much, indicating that the pixel is a stamp area and needs to be removed;

[0104] 3) When the variance of the R channel is 1 and the maximum value of the R channel is 0, this indicates that the pixel is most likely not red and the surrounding color changes greatly, indicating that the pixel is at the edge of the seal and needs to be removed;

[0105] 4) When the variance of the R channel is 1 and the maximum value of the R channel is 1, this indicates that the pixel is most likely red and the surrounding color changes greatly, indicating that the pixel is at the edge of the seal and needs to be removed.

[0106] Specifically, in S34, morphological operations are used to connect potential seal regions in the bill image to be identified to form a complete seal region, and the seal region is removed, including:

[0107] S331, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region;

[0108] S332, generating a binary image mask based on the stamp area, setting the stamp area to white and other areas to black;

[0109] S333: Use the image mask to perform a mask operation on the bill image to be identified, and replace the pixel values ​​of the pixel points in the bill image to be identified corresponding to the white pixel points in the image mask with the background color.

[0110] The above technical scheme obtains the RGB three-channel values ​​of each pixel in the bill image to be identified, delineates a sub-image in a preset size area with each pixel as the center, and calculates the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel. The seal area in the bill image to be identified is removed according to the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel, and image restoration is performed on the bill image to be identified. By constructing the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel, and performing logical OR operations on the corresponding elements in the variance matrix and the maximum value matrix, the seal area and its edge image can be effectively determined, thereby realizing accurate removal of the seal area.

[0111] like Figure 1 and Figure 4 As shown, S4, using the interpolation method based on wavelet transform to perform image interpolation on the bill image to be identified, specifically including:

[0112] S41, performing wavelet transform on the bill image to be identified, so as to decompose the bill image to be identified into a low-frequency part reflecting the smooth area of ​​the image and a high-frequency part containing image detail information;

[0113] S42, for the low-frequency part, using bilinear interpolation to perform image interpolation;

[0114] S43. For the high frequency part, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation;

[0115] S44, performing inverse wavelet transform on the interpolated low-frequency part and high-frequency part to reconstruct the interpolated bill image to be recognized;

[0116] Among them, inter-scale persistence refers to the fact that wavelet coefficients have a certain correlation between different scales, that is, larger or smaller wavelet coefficients may maintain larger or smaller values ​​between adjacent scales. This correlation is captured by the state transition probability matrix of the hidden Markov model;

[0117] Exponential decay means that as the scale increases, the value of the wavelet coefficient shows an exponential decay trend. This characteristic is reflected in the observation probability matrix of the hidden Markov model.

[0118] Specifically, for the high-frequency part, in S43, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation, including:

[0119] S431, construct a hidden Markov model, use the known observation sequence to train the model, and estimate the initial state probability vector, state transition probability matrix and observation probability matrix;

[0120] S432. Input the wavelet coefficient sequence reflecting the high-frequency part into the trained hidden Markov model, use the forward-backward algorithm to calculate the probability of the wavelet coefficient sequence under the given model, use the Viterbi algorithm to find the hidden state sequence that is most likely to produce the wavelet coefficient sequence, and predict the unknown wavelet coefficients based on the observed probability matrix and the hidden state sequence.

[0121] The above technical scheme adopts an interpolation method based on wavelet transform to perform image interpolation on the bill image to be identified. For the low-frequency part, bilinear interpolation is used for image interpolation. For the high-frequency part, by analyzing the inter-scale continuity and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation. The low-frequency part and the high-frequency part after interpolation are subjected to wavelet inverse transform to reconstruct the interpolated bill image to be identified, thereby achieving effective restoration of the bill image to be identified and improving the image quality of the bill image to be identified, providing image support for the text recognition task of the subsequent text recognition model.

[0122] like Figure 1 As shown, S5, determine the text box area in the bill image to be recognized, and use the text recognition model to perform text recognition on the text box area.

[0123] In the technical solution of the present application, the text recognition model is constructed based on an improved convolutional recurrent neural network, in which the recurrent layer of the improved convolutional recurrent neural network adopts a bidirectional gated recurrent unit, and the improved convolutional recurrent neural network sequentially includes an input layer, a convolutional layer, a bidirectional gated recurrent unit, a hidden state merging and an output layer;

[0124] The convolution layer extracts the local features of the input data in the input layer and generates a feature map by sliding the convolution kernel on the input data.

[0125] A bidirectional gated recurrent unit, including a forward gated recurrent unit and a backward gated recurrent unit. The forward gated recurrent unit processes the input data in the positive order to capture the forward context information. The backward gated recurrent unit processes the input data in the reverse order to capture the backward context information. The forward gated recurrent unit and the backward gated recurrent unit generate their own hidden states at each time step.

[0126] Hidden state merging: At each time step, the hidden states of the forward gated recurrent unit and the backward gated recurrent unit are merged. The merged hidden state contains the complete context information at that time step.

[0127] The output layer maps the merged hidden states to the required output space to generate the final output sequence.

[0128] Specifically, in S5, before determining the text box area in the bill image to be recognized and performing text recognition on the text box area using the text recognition model, the following steps are included:

[0129] S51, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0130] S52, setting the loss function and optimizer of the text recognition model;

[0131] S53, inputting the training set into the text recognition model for model training;

[0132] S54, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0133] S55. If the loss value is less than the preset threshold, the model training ends, and the current text recognition model is the trained text recognition model. Otherwise, the process returns to S53 and continues to use the training set for model training.

[0134] S56. Input the validation set into the trained text recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0135] S57. Input the test set into the tuned text recognition model to evaluate the model performance.

[0136] In the above technical solution, the text recognition model is constructed based on an improved convolutional recurrent neural network. The recurrent layer in the improved convolutional recurrent neural network adopts a bidirectional gated recurrent unit. The use of the bidirectional gated recurrent unit can better capture the complete context information at each time step, effectively improving the accuracy of text recognition.

[0137] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for bill text recognition, characterized in that: The following steps are involved: S1, reading the bill image to be identified, determining the feature points in the bill image to be identified, and performing rotation correction on the bill image to be identified according to the feature points; S2, obtaining the RGB three-channel values ​​of each pixel in the bill image to be identified, demarcating a sub-image in a preset size area with each pixel as the center, and calculating the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel; S3, removing the seal area in the bill image to be identified according to the variance matrix and the maximum value matrix of the bill image to be identified with respect to the R channel, and performing image restoration on the bill image to be identified; S4, using an interpolation method based on wavelet transform to perform image interpolation on the bill image to be identified; S5, determining a text box area in the bill image to be recognized, and performing text recognition on the text box area using a text recognition model; In S2, the RGB three-channel values ​​of each pixel in the bill image to be identified are obtained, a sub-image is delineated in a preset size area with each pixel as the center, and the variance matrix and maximum value matrix of the bill image to be identified with respect to the R channel are calculated, including: S21, separating the RGB three channels of the bill image to be identified, obtaining three single-channel images of R, G, and B respectively, and obtaining the RGB three-channel value of each pixel point in the bill image to be identified; S22, delimiting a sub-image in a region of a preset size with the pixel as the center, and obtaining R channel values ​​of all the pixels in the sub-image; S23, calculating the variance of the R channel values ​​of all pixels in the sub-image, and extracting the maximum value of the R channel values ​​of all pixels in the sub-image, to obtain the R channel variance and the R channel maximum value of the central pixel; S24, repeat S22-S23 until the R channel variance and R channel maximum value of each pixel in the bill image to be identified are obtained; S25, constructing a variance matrix based on the R channel variance of each pixel in the to-be-recognized bill image, and constructing a maximum value matrix based on the R channel maximum value of each pixel in the to-be-recognized bill image; Among them, the preset size area is 3*3.

2. The method for bill text recognition according to claim 1, characterized in that: S1 reads the bill image to be identified, determines the feature points in the bill image to be identified, and performs rotation correction on the bill image to be identified according to the feature points, including: S11, reading the bill image to be identified through the image processing library; S12, performing Gaussian smoothing on the bill image to be recognized, gradually downsampling, and constructing a Gaussian pyramid; S13, determining a suitable resolution, and performing edge detection on the to-be-recognized bill image of the corresponding layer in the Gaussian pyramid using an edge detection algorithm; S14, determining feature points in the bill image to be identified, and calculating the tilt angle of the bill image to be identified; S15, determining the rotation center of the bill image to be identified, and performing rotation correction on the bill image to be identified based on the tilt angle.

3. The method for bill text recognition according to claim 2, characterized in that: In S14, feature points in the bill image to be identified are determined, and the tilt angle of the bill image to be identified is calculated, including: S141, performing connected domain analysis on the to-be-recognized bill image after edge detection, and marking all connected domains; S142, selecting feature points that can reflect the tilt of the bill image to be identified from the connected domain; S143, by analyzing the orientation relationship between the feature points, selecting target feature points belonging to the same frame from the feature points; S144, fitting the target feature points, and calculating the angle between the fitting result and the horizontal direction to obtain the tilt angle of the recognized bill image; The feature points are the corner points of the frame or the midpoints of the long sides.

4. The method for bill text recognition according to claim 1, characterized in that: In S3, the seal area in the bill image to be identified is removed according to the variance matrix and the maximum value matrix of the bill image to be identified with respect to the R channel, and image restoration is performed on the bill image to be identified, including: S31. According to the distribution of the variance matrix and the maximum value matrix, set an appropriate variance threshold and a maximum value threshold; S32, comparing the R channel variance of each pixel in the bill image to be identified with the variance threshold, and the R channel maximum value with the maximum value threshold, respectively; if the R channel variance is less than the variance threshold, the R channel variance is set to 0, otherwise the R channel variance is set to 1; if the R channel maximum value is less than the maximum value threshold, the R channel maximum value is set to 0, otherwise the R channel maximum value is set to 1; S33, performing a logical OR operation on the R channel variance and the R channel maximum value of each pixel in the to-be-recognized bill image, and determining the pixel whose calculation result is 1 as a potential seal area in the to-be-recognized bill image; S34, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region, and then removing the seal region; S35. Perform image restoration on the bill image to be recognized by using a restoration method based on texture synthesis.

5. The method for bill text recognition according to claim 4, characterized in that: In S34, morphological operations are used to connect potential seal regions in the bill image to be identified to form a complete seal region, and the seal region is removed, including: S331, using morphological operations to connect potential seal regions in the bill image to be identified to form a complete seal region; S332, generating a binary image mask based on the stamp area, setting the stamp area to white and other areas to black; S333: Use the image mask to perform a mask operation on the bill image to be identified, and replace the pixel values ​​of the pixel points in the bill image to be identified corresponding to the white pixel points in the image mask with the background color.

6. The method for bill text recognition according to claim 1, characterized in that: S4 uses an interpolation method based on wavelet transform to interpolate the image of the bill to be identified, including: S41, performing wavelet transform on the bill image to be identified, so as to decompose the bill image to be identified into a low-frequency part reflecting the smooth area of ​​the image and a high-frequency part containing image detail information; S42, for the low-frequency part, using bilinear interpolation to perform image interpolation; S43. For the high frequency part, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation; S44, performing inverse wavelet transform on the interpolated low-frequency part and high-frequency part to reconstruct the interpolated bill image to be recognized; Among them, inter-scale persistence refers to the fact that wavelet coefficients have a certain correlation between different scales, that is, larger or smaller wavelet coefficients may maintain larger or smaller values ​​between adjacent scales. This correlation is captured by the state transition probability matrix of the hidden Markov model; Exponential decay means that as the scale increases, the value of the wavelet coefficient shows an exponential decay trend. This characteristic is reflected in the observation probability matrix of the hidden Markov model.

7. The method for bill text recognition according to claim 6, characterized in that: For the high-frequency part in S43, by analyzing the inter-scale persistence and exponential decay of the wavelet coefficients, a hidden Markov model is constructed to characterize the probability structure of the wavelet coefficients to achieve image interpolation, including: S431, construct a hidden Markov model, use the known observation sequence to train the model, and estimate the initial state probability vector, state transition probability matrix and observation probability matrix; S432. Input the wavelet coefficient sequence reflecting the high-frequency part into the trained hidden Markov model, use the forward-backward algorithm to calculate the probability of the wavelet coefficient sequence under the given model, use the Viterbi algorithm to find the hidden state sequence that is most likely to produce the wavelet coefficient sequence, and predict the unknown wavelet coefficients based on the observed probability matrix and the hidden state sequence.

8. The method for bill text recognition according to claim 1, characterized in that: The text recognition model is constructed based on an improved convolutional recurrent neural network, in which the recurrent layer of the improved convolutional recurrent neural network adopts a bidirectional gated recurrent unit, and the improved convolutional recurrent neural network sequentially includes an input layer, a convolutional layer, a bidirectional gated recurrent unit, a hidden state merging and an output layer; The convolution layer extracts the local features of the input data in the input layer and generates a feature map by sliding the convolution kernel on the input data. A bidirectional gated recurrent unit, including a forward gated recurrent unit and a backward gated recurrent unit. The forward gated recurrent unit processes the input data in the positive order to capture the forward context information. The backward gated recurrent unit processes the input data in the reverse order to capture the backward context information. The forward gated recurrent unit and the backward gated recurrent unit generate their own hidden states at each time step. Hidden state merging: At each time step, the hidden states of the forward gated recurrent unit and the backward gated recurrent unit are merged. The merged hidden state contains the complete context information at that time step. The output layer maps the merged hidden states to the required output space to generate the final output sequence.

9. The method for bill text recognition according to claim 8, characterized in that: In S5, before determining the text box area in the bill image to be recognized and using the text recognition model to perform text recognition on the text box area, the process includes: S51, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio; S52, setting the loss function and optimizer of the text recognition model; S53, inputting the training set into the text recognition model for model training; S54, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information; S55. If the loss value is less than the preset threshold, the model training ends, and the current text recognition model is the trained text recognition model. Otherwise, the process returns to S53 and continues to use the training set for model training. S56. Input the validation set into the trained text recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model; S57. Input the test set into the tuned text recognition model to evaluate the model performance.

Images