Method and system for identifying and correcting bill orientation

Abstract

The present invention relates to a method and system for identifying and correcting bill orientation. The method comprises the following steps: extracting multi-directional features from an input bill image to obtain a multi-directional feature vector; performing anomaly detection and local correction on the bill image based on the multi-directional feature vector to obtain a corrected bill image; using an orientation recognition network to identify the bill orientation based on the multi-directional feature vector and the corrected bill image to obtain an initial orientation recognition result; performing connectivity detection on the corrected bill image to obtain a connectivity detection result, and performing orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result; performing text recognition on the corrected bill image based on the target orientation recognition result to obtain bill text information. The implementation of the present invention comprehensively considers the integration of bill orientation recognition, anomaly detection, text line detection and text recognition, and improves the automation level and accuracy of bill processing.

Description

Technical Field

[0001] The present invention relates to the technical field of bill recognition, and in particular to a bill orientation-based recognition and correction method and system. Background Art

[0002] Automatic recognition and processing of bills. In practical applications, the acquisition of bill images often has problems such as uncertain orientation, tilt, and wrinkles, which brings huge challenges to subsequent text recognition and information extraction.

[0003] Traditional bill recognition methods usually rely on fixed templates and rules, which are difficult to adapt to diverse bill formats and complex deformations. At the same time, the diversity of elements such as text layout, fonts, and seals on bills also increases the difficulty of recognition. Orientation recognition of bill images is a multimodal problem that requires considering both the global features and local text features of the image. The existing single model is difficult to effectively capture these complex feature relationships. Secondly, common anomalies in bill images, such as local wrinkles and stains, are prone to recognition errors, requiring a more robust anomaly detection and correction mechanism. In addition, bill text recognition also faces challenges such as special character recognition and context understanding. Traditional OCR technology often performs poorly when dealing with bill-specific text formats and professional terms. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and system for identifying and correcting bill orientation based on bill orientation, so as to realize the integration of bill orientation identification, anomaly detection, text line detection and text recognition, and improve the automation level and accuracy of bill processing.

[0005] To achieve the above object, the present invention provides a method for identifying and correcting bill orientation, comprising the following steps:

[0006] Perform multi-directional feature extraction on the input bill image to obtain a multi-directional feature vector;

[0007] Based on the multi-directional feature vector, abnormality detection and local correction are performed on the bill image to obtain a corrected bill image;

[0008] According to the multi-directional feature vector and the corrected bill image, using an orientation recognition network to perform bill orientation recognition to obtain an initial orientation recognition result;

[0009] Performing connectivity detection on the corrected bill image to obtain a connectivity detection result, and performing orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result;

[0010] Based on the target orientation recognition result, text recognition is performed on the corrected bill image to obtain bill text information.

[0011] The present invention also provides a bill orientation-based identification and correction system, comprising:

[0012] An extraction module is used to extract multi-directional features from the input bill image to obtain a multi-directional feature vector;

[0013] A correction module, used for performing abnormality detection and local correction on the bill image based on the multi-directional feature vector to obtain a corrected bill image;

[0014] An orientation recognition module, used to perform bill orientation recognition using an orientation recognition network according to the multi-directional feature vector and the corrected bill image, to obtain an initial orientation recognition result;

[0015] A detection module, configured to perform connectivity detection on the corrected bill image to obtain a connectivity detection result, and perform orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result;

[0016] The text recognition module is used to perform text recognition on the corrected bill image based on the target orientation recognition result to obtain the bill text information.

[0017] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0018] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0019] In summary, the technical solution provided by the present invention improves the accuracy of orientation recognition through multi-directional feature extraction technology: by using the ResNet50 structure and multi-directional convolution operation, combined with the autoencoder network and residual connection, all-round and multi-scale feature extraction of the bill image is realized, and the directional information of the bill is effectively captured. The abnormal detection and local correction mechanism enhances the robustness of the system: the edge-guided feature enhancement module and the complementary feature attention module are adopted, combined with the abnormal detection network of the U-Net structure, the abnormal area in the bill image can be accurately located and corrected, and the accuracy of subsequent recognition is improved. Multimodal feature fusion improves the effect of orientation recognition: combining multi-directional feature vectors and image features extracted by deep convolutional neural networks, more accurate orientation recognition is achieved through feature fusion and orientation classification network. The combination of connectivity detection and text line detection optimizes the orientation correction process: through edge detection, connected domain analysis and text line detection, the overall structure and text arrangement of the bill are comprehensively considered, and the accuracy of orientation correction is improved. The combination of deep learning and traditional image processing technology improves the adaptability of the system: in text line detection and direction calculation, traditional technologies such as Gabor filters and Hough transforms are combined to complement deep learning methods and enhance the system's ability to process different types of bills. The text recognition method based on SikuBERT improves the recognition of special characters and professional terms: by using a BERT variant optimized for ancient Chinese books and bill texts, combined with BiLSTM and multi-head attention mechanisms, the accuracy of text recognition is significantly improved. The end-to-end integrated solution improves the overall efficiency of the system: the full process design from orientation recognition to text recognition reduces intermediate steps and data conversion, improves processing efficiency and system scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the steps of a method for identifying and correcting the bill orientation according to an embodiment of the present invention;

[0021] Figure 2 is a structural block diagram of a bill orientation-based identification and correction system in one embodiment of the present invention;

[0022] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0023] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] Reference Figure 1 This embodiment provides a method for identifying and correcting a bill based on its orientation, comprising the following steps:

[0026] S1, extract multi-directional features from the input bill image to obtain a multi-directional feature vector;

[0027] Among them, the input bill image is preprocessed to obtain a standardized bill image. The preprocessing steps include adjusting the image size, grayscale processing, denoising and other operations to ensure that the subsequent feature extraction process is carried out under a unified standard. The standardized bill image is input into a pre-trained convolutional neural network, which adopts the ResNet50 structure. ResNet50 consists of 5 convolution blocks, each of which contains 3 convolution layers, and directly passes the input to the output through a short-circuit connection to alleviate the gradient vanishing problem in deep neural networks. Through the convolution operation of the network, a basic feature map is obtained. The basic feature map is subjected to multi-directional convolution operations to capture feature information from different directions in the image. The multi-directional convolution operation is processed by 4 parallel convolution branches, each of which is for the four directions of 0°, 45°, 90° and 135°. Each branch consists of three convolutional layers, using convolution kernels of different sizes, namely 1×3, 3×1 and 3×3. This design ensures that feature information in different directions is fully extracted and expressed to form a directional feature atlas. Global average pooling is performed on each directional feature map in the directional feature atlas to compress the directional feature map into a directional feature vector set. Global average pooling can not only effectively reduce the number of parameters, but also enhance the generalization ability of the model. The directional feature vector set combines features from different directions through feature fusion to generate a fused feature vector. The fused feature vector is nonlinearly transformed to extract deep features. The nonlinear transformation uses two fully connected layers, each followed by a BatchNormalization operation and a ReLU activation function, which improves the training speed of the model and reduces overfitting. The intermediate feature vector is input into an autoencoder network for feature reconstruction. The autoencoder consists of an encoder and a decoder, each of which contains three fully connected layers. The encoder encodes the intermediate feature vector into a representation of a low-dimensional feature space, and the decoder reconstructs this low-dimensional representation into a high-dimensional feature vector. To avoid overfitting, each layer of the autoencoder uses the LeakyReLU activation function and Dropout regularization. The reconstructed feature vector is residually connected with the fused feature vector to enhance the robustness of feature expression by retaining the original feature information. After residual connection, a multi-directional feature vector is finally obtained.

[0028] S2, based on the multi-directional feature vector, perform abnormality detection and local correction on the bill image to obtain the corrected bill image;

[0029] Among them, the multi-directional feature vector is dimensionally transformed to obtain a feature map. The dimension transformation is achieved through a deconvolution operation. The deconvolution operation uses three deconvolution layers, and each deconvolution layer is followed by a BatchNormalization and ReLU activation function. The function of the deconvolution layer is to expand the low-dimensional feature vector into a high-dimensional feature map by upsampling, so that these features can be aligned with the original bill image. The feature map obtained by deconvolution is input into the edge-guided feature enhancement module for processing to obtain an edge-enhanced feature map. The edge-guided feature enhancement module contains three dilated convolution layers with dilated rates of 1, 2, and 4, respectively. The design of dilated convolution can expand the receptive field, so that the network can better capture the edge information and other structured features in the image. Each dilated convolution layer is followed by a LeakyReLU activation function, which reduces the occurrence of dead neurons while maintaining the nonlinear transformation capability, and enhances the network's ability to capture the edge details of complex bill images. The edge-enhanced feature map is input into the complementary feature attention module for processing to obtain an attention-weighted feature map. The complementary feature attention module contains a channel attention submodule and a spatial attention submodule. These two attention mechanisms are fused through a gating mechanism. The channel attention submodule focuses on selecting which feature channels are more important for the current task, while the spatial attention submodule is used to locate more informative areas in the image. Through the complementary effect of these two attention mechanisms, the network's ability in feature selection and positioning is effectively improved, ensuring that the key areas in the bill image are more accurately represented. The attention-weighted feature map is input into the multi-scale feature extraction module, which uses a pyramid pooling network structure. The pyramid pooling network contains 4 average pooling layers of different scales, which can extract multi-scale information from the feature map. By using different pooling scales, the network can capture global and local information in the bill image at the same time. The multi-scale feature extraction mechanism ensures that the detection of abnormal areas can be performed at different spatial scales, improving the robustness of the detection. The multi-scale feature atlas is input into the anomaly detection network to obtain anomaly detection heatmaps. The anomaly detection network adopts the U-Net structure. The U-Net network contains an encoder and a decoder, each containing 4 convolutional blocks, and uses jump connections to directly pass the encoder's intermediate layer features to the decoder. The jump connection can retain fine-grained feature information, ensuring that key information is not lost during feature reconstruction, and generating high-quality anomaly detection heat maps. Based on the anomaly detection heat map, the bill image is segmented to obtain candidate anomaly regions. Morphological processing and connected domain analysis are performed on the candidate anomaly regions to improve the precision of detection. Morphological processing is used to remove noise and smooth regional edges, while connected domain analysis can accurately identify adjacent anomaly regions and determine fine anomaly regions.The bill image is locally corrected based on the fine abnormal areas. The abnormalities in the bill image are corrected by adjusting the positions or shapes of the pixels in these areas, and finally a corrected bill image is obtained.

[0030] S3, using the orientation recognition network to perform bill orientation recognition based on the multi-directional feature vector and the corrected bill image, and obtaining an initial orientation recognition result;

[0031] Among them, by enhancing the features of multi-directional feature vectors, more representative information is extracted. The feature enhancement process adopts the self-attention mechanism, which can help the model capture global context information more effectively when processing bill images. The self-attention mechanism includes a multi-head self-attention layer and a feedforward neural network layer, each of which is followed by LayerNormalization. The multi-head self-attention layer allows the model to focus on different parts of the input data at the same time, enhancing the ability to capture complex features. The feedforward neural network layer helps improve the expression ability of features through nonlinear transformation and feature combination. The role of LayerNormalization is to ensure that stable gradient propagation can be maintained during network training, thereby accelerating convergence and preventing gradient explosion or disappearance. At the same time, the corrected bill image is input into a pre-trained deep convolutional neural network to extract the feature vector of the image. The convolutional neural network adopts the DenseNet structure. DenseNet effectively enhances information flow and reduces the number of parameters by reusing features between layers in each dense block. The DenseNet network contains 4 dense blocks, each of which contains 6 convolutional layers. Each convolutional layer uses batch normalization and ReLU activation function. Batch normalization is used to stabilize the training process and improve the generalization ability of the network, while the ReLU activation function enhances the network's expressiveness by introducing nonlinearity. Through the layer-by-layer feature reuse of DenseNet, the rich details of the bill image can be fully preserved and the image feature vector can be generated. The enhanced feature vector is fused with the image feature vector to obtain a multimodal feature vector. The feature fusion process allows the global semantic information in the multi-directional feature vector to be combined with the fine-grained visual information in the image features, thereby providing a more comprehensive representation for orientation recognition. The multimodal feature vector can contain both the spatial features and the orientation information of the bill image, thereby providing richer contextual support for the orientation recognition task. The multimodal feature vector is input into the orientation classification network for processing. The orientation classification network contains 3 fully connected layers. In order to prevent overfitting and enhance the generalization ability of the network, the fully connected layer uses Dropout regularization, and the LeakyReLU activation function is used after each fully connected layer to improve the nonlinear expressiveness of the model and maintain the fluidity of the gradient. In the last layer of the orientation classification network, the Softmax activation function is used to convert the network output into a probability distribution of orientation. The probability distribution represents the possible orientation categories of the input bill image and provides the predicted probability of each orientation. In order to optimize the orientation recognition results, conditional random field optimization is performed based on the orientation probability distribution. Conditional random field is a common sequence labeling model that can adjust the probability distribution according to the relationship between local and global features, making the orientation output by the model more consistent and in line with actual logic. Through conditional random field optimization, the accuracy of orientation prediction is significantly improved.Based on the optimized probability distribution of the orientation, the maximum likelihood estimation is performed to obtain the preliminary orientation recognition result.

[0032] Specifically, the corrected bill image is preprocessed, the input image is adjusted to a uniform scale and format, and a normalized image is obtained. The normalized image is input into the first dense block in the pre-trained deep convolutional neural network. The network adopts the DenseNet structure, in which the first dense block contains 6 convolutional layers, and the output of each convolutional layer not only depends on its own input, but also connects with the output of all previous layers to form a dense connection structure. Through dense connection, each layer can directly access the feature information of the previous layer, enhance the fluidity and reusability of the features, and effectively improve the expression ability of the network. After the first dense block generates the first feature map, the feature map is feature recalibrated to adjust the weight and importance of the features. The feature recalibration process includes a global average pooling operation, followed by further refining the features through two fully connected layers, and normalizing each feature through the sigmoid activation function, so that the network can automatically adjust the weights of different features. The feature map after feature recalibration is input into the second dense block and the third dense block. The structures of these two dense blocks are the same as the first dense block, both of which contain 6 convolutional layers and use the same dense connection mechanism. After being processed by these two dense blocks, the network generates an intermediate feature map. Multi-scale feature extraction is performed on the intermediate feature map, and a spatial pyramid pooling module is used. This module contains four different pooling scales of 1×1, 2×2, 3×3 and 6×6. Through these different pooling layers, the network can simultaneously capture feature information at different spatial scales, ensuring a comprehensive analysis of the global and local information of the bill image. The multi-scale feature map set is input into the fourth dense block for processing. In addition to the same convolutional layer structure as the previous dense blocks, the fourth dense block also introduces an attention mechanism between each convolutional layer. The attention mechanism uses a combination of channel attention and spatial attention. The channel attention mechanism helps the network select important information on the feature channel, while the spatial attention mechanism enhances the key areas in the image space dimension. The dual attention mechanism can focus on the important information in the bill image more accurately and improve the accuracy of orientation recognition. Global average pooling and global maximum pooling operations are performed on the high-level feature map. Global average pooling extracts global information by calculating the average value of all pixels on the feature map, while global maximum pooling captures important local features by extracting the maximum value. Through these two pooling methods, two global feature vectors are generated, representing global and local information respectively. After concatenating these two feature vectors, the feature vector of the image is finally obtained through dimensionality reduction.

[0033] Furthermore, the multimodal feature vectors are subjected to feature normalization to ensure that the numerical range of the data is uniform, so that the gradient propagation in the neural network is more stable, and a normalized feature vector is obtained. The normalized feature vector is input into the first fully connected layer of the orientation classification network, and the network performs a linear transformation on the feature vector to obtain an intermediate feature vector. In order to prevent overfitting and enhance the generalization ability of the model, the intermediate feature vector is subjected to a Dropout operation. Dropout obtains a sparse feature vector by randomly shielding the output of some neurons, thereby reducing the model's excessive dependence on certain specific neurons and improving its robustness. The sparse feature vector is input into the second fully connected layer of the orientation classification network, which has 2048 neurons and can effectively capture feature representations in high-dimensional space. The second fully connected layer uses the LeakyReLU activation function. One feature of the LeakyReLU activation function is that the slope parameter for negative input is 0.2, allowing a small number of negative values ​​to pass through the activation function, thereby alleviating the common "dead neuron" problem in the ReLU activation function and ensuring that the nonlinear expression ability of the network will not be limited during training. A high-dimensional feature vector is generated in the second fully connected layer network. The high-dimensional feature vector is enhanced to improve the expressive power of the feature. Feature enhancement highlights the key orientation-related information and weakens the unimportant redundant information by reprocessing the features at the deep learning level. The enhanced feature vector is input into the third fully connected layer of the orientation classification network. The output of the third fully connected layer is a logits vector, which is the original prediction value of the last layer of the model before the Softmax function is processed. In order to improve the stability of the model in uncertain scenarios, the logits vector is temperature scaled. The role of temperature scaling is to smooth or sharpen the output probability distribution by adjusting the temperature parameter of the Softmax function, which helps to calibrate the output probability of the model, especially the classification bias problem that may exist in the classification task. Through temperature scaling, the confidence of the network in predicting different orientations is more reasonable and better reflects the actual uncertainty of the input data. The calibrated logits vector is subjected to a Softmax operation. The Softmax function converts the logits vector into an orientation probability distribution, which represents the predicted probability value of each orientation. In this way, the orientation classification network can output specific orientation information about the bill image.

[0034] S4, performing connectivity detection on the corrected bill image to obtain a connectivity detection result, and performing orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result;

[0035] Among them, edge detection is performed on the corrected bill image to extract the contour information in the image and obtain the edge image. Binarization is performed on the edge image to distinguish the edges in the image into black and white to form a binary edge image. Morphological processing is performed on the binary edge image to optimize the edge image. Morphological processing includes two operations: opening and closing. The opening operation can effectively remove small noise points in the image by corroding first and then expanding, while the closing operation can fill small holes in the image by expanding first and then corroding. Morphological processing uses a 3x3 rectangular structural element to make the details of the image clearer and more coherent, and obtain an optimized edge image. Connected domain analysis is performed on the optimized edge image to identify the pixel blocks connected to each other in the image and mark them as different connected regions. In this way, a connected domain labeled image is generated. Based on the connected domain labeled image, the direction and aspect ratio of the largest connected region are calculated. The geometric features of the connected domain, such as the direction and aspect ratio, can reflect the placement of the overall structure of the bill, and then generate a connectivity feature vector. At the same time, text line detection is performed on the corrected bill image to identify the regions of each text line in the bill image. Through text line detection, the position information and arrangement pattern of the main text lines in the bill are obtained. Based on the text line area, the main text direction is calculated. Through these calculations, the text direction vector is generated to provide supplementary information about the bill orientation. The connectivity feature vector, the text direction vector and the initial orientation recognition result are weighted fused. The weighted fusion process comprehensively considers the geometric features provided by the connected domain analysis, the text direction information obtained by the text line detection, and the results of the initial orientation recognition by setting different weights. The weights of each vector can be adjusted based on experience or training data to ensure that accurate orientation correction results can be obtained in various complex situations. After the above steps, the target orientation recognition result is obtained.

[0036] Specifically, the corrected bill image is preprocessed to obtain an enhanced image. The enhanced image is input into the text region detection network, which adopts a fully convolutional network structure. The network includes an encoder and a decoder module. The encoder extracts high-level semantic features of the image, while the decoder restores these features to the same resolution as the input image. In order to ensure that the network can capture text features of different scales, the network uses a dilated convolution and a feature pyramid network to extract multi-scale features to obtain a text region probability map. The text region probability map is post-processed. The probability map is converted into a clearer binary image, namely, a text region binary map. In the binary map, the text region and the non-text region are clearly distinguished. The text region binary map is subjected to a connected region analysis, which can identify adjacent pixel groups as candidate text line regions. In this way, the connectivity structure in the text region is identified, and a candidate region that may contain text is initially formed. The candidate text line regions are merged to obtain text line regions. The text line merging is based on the distance threshold and direction consistency. By analyzing the spatial distance between the candidate regions and the similarity of the text direction, it is determined which regions can be merged into one text line. A hierarchical clustering algorithm is used to gradually aggregate adjacent regions to obtain a complete text line region. Based on the text line region, the original image is perspective corrected to correct the geometric distortion in the image, ensuring that the text lines can be arranged horizontally and vertically to obtain a corrected text line image; the directional features of the corrected text line image are extracted. A Gabor filter group is used, which is suitable for extracting the direction information of the strokes of the text. A Gabor filter group containing 8 different directions is used to analyze the response of the image in different directions to extract the direction information of the text strokes. Each filter corresponds to a specific direction and can detect the distribution of the strokes of the text in this direction. The main text direction is calculated based on the directional feature map. By comprehensively analyzing the responses in each direction, the arrangement direction of the main text in the image is obtained, and the text direction vector is obtained to describe the overall direction of the text line.

[0037] S5, based on the target orientation recognition result, perform text recognition on the corrected bill image to obtain the bill text information.

[0038] Among them, the corrected bill image is rotated and corrected according to the target orientation recognition result to ensure that the text in the image is in the correct direction. The rotation correction is performed based on the angle adjustment of the target orientation recognition result, and a direction-corrected image is obtained through the rotation operation. The direction-corrected image is adaptively binarized. Adaptive binarization converts the grayscale information in the image into black and white pixels by dynamically adjusting the threshold according to the local illumination conditions of the image to generate a binary image. The binarization operation can enhance the contrast between the text and the background, which is convenient for subsequent text detection and recognition. Based on the binarized image, text line segmentation is performed to separate the text line in the image from the overall image to obtain a text line image set. Each text line image in the text line image set is tilted and corrected to correct the tilt angle in the text line so that the text line can be arranged horizontally. The corrected text line image set is input into the SikuBERT model for feature extraction. The SikuBERT model is a deep learning model based on the Transformer architecture, which contains 12 Transformer encoder layers, each of which can capture the semantic information and contextual relationship in the text line image. In this way, SikuBERT can convert text line images into text feature sequences, which contain deep semantic features of each character in the image and its context. In order to enhance the contextual information of the feature sequence, the text feature sequence is input into a bidirectional long short-term memory network (BiLSTM) for feature extraction. BiLSTM can capture the contextual dependencies in the text sequence through bidirectional processing. The bidirectional long short-term memory network contains 2 layers, each with 128 hidden units, and uses residual connections, so that the network can not only effectively learn sequence features, but also maintain the integrity of feature information and the stability of training through residual connections. Through BiLSTM processing, a context-enhanced feature sequence is generated. The context-enhanced feature sequence is input into the multi-head attention module for feature weighting. The multi-head attention mechanism contains 8 attention heads, each with a dimension of 64. The multi-head attention mechanism weights the text features through multiple attention points, so that the model can capture more fine-grained semantic information and structural information in different dimensions. After being processed by the multi-head attention module, the obtained attention-weighted feature sequence can more effectively represent the text features in the bill. Sequence labeling is performed based on the attention-weighted feature sequence, label prediction is performed on the text features character by character, and the text information in the bill is output.

[0039] In one example, multi-directional feature extraction is performed on the input bill image to obtain a multi-directional feature vector, including:

[0040] Preprocess the input bill image to obtain a standardized bill image;

[0041] The standardized bill image is input into the pre-trained convolutional neural network to obtain the basic feature map; the pre-trained convolutional neural network adopts the ResNet50 structure, which contains 5 convolutional blocks, each of which contains 3 convolutional layers and a short-circuit connection;

[0042] Perform multi-directional convolution operations on the basic feature maps to obtain a set of directional feature maps; the multi-directional convolution operation includes the use of 4 parallel convolution branches, corresponding to 0°, 45°, 90° and 135° directions, each branch contains 3 convolution layers, and the convolution kernel sizes are 1×3, 3×1 and 3×3 respectively;

[0043] Performing a global average pooling operation on each directional feature map in the directional feature map set to obtain a directional feature vector set, and performing feature fusion on the directional feature vector set to obtain a fused feature vector;

[0044] Perform a nonlinear transformation on the fused feature vector to obtain an intermediate feature vector; the nonlinear transformation includes using two fully connected layers, each followed by a BatchNormalization and ReLU activation function;

[0045] The intermediate feature vector is input into the autoencoder network to obtain the reconstructed feature vector; the autoencoder network includes an encoder and a decoder, each of which contains 3 fully connected layers, using the LeakyReLU activation function and Dropout regularization;

[0046] The reconstructed feature vector is residually connected with the fused feature vector to obtain a multi-directional feature vector.

[0047] In this example, the bill image is standardized, and bill images of different sizes, formats, and lighting conditions are uniformly converted into a standardized image format. This includes adjusting the size of the image, contrast enhancement, grayscale processing, and denoising, etc., to obtain a standardized bill image. The standardized bill image is input into a pre-trained convolutional neural network. The network adopts the ResNet50 structure, which is a deep residual network with good feature extraction capabilities. The network contains 5 convolution blocks, each of which contains 3 convolution layers and a short-circuit connection. The function of the short-circuit connection is to pass the input directly to the subsequent layer, alleviate the gradient vanishing problem that may occur in the deep neural network, and ensure that the model can be trained efficiently. In the convolution layer of ResNet50, the core of the convolution operation is to apply the convolution kernel to filter the input image layer by layer, extract local features in the image, and gradually build global semantic information. Through this process, the basic feature map is obtained. The basic feature map is subjected to multi-directional convolution operations, so that the model can accurately recognize the distribution of text or lines in the image at different angles. Four parallel convolution branches are used, corresponding to the four directions of 0°, 45°, 90° and 135° respectively. Each convolution branch contains three convolution layers, and the sizes of the convolution kernels are 1×3, 3×1 and 3×3 respectively. This design can capture features of different scales and directions, ensuring that all kinds of information in the image are fully extracted. The mathematical representation of directional convolution is:

[0048] ;

[0049] in, Indicates The feature map of the direction, is the basic feature map of the input, is the convolution kernel, is the bias term, is the activation function. Each convolution branch extracts information from different angles. In this way, the network can better capture the rotated or tilted text features that may exist in the bill. After completing the multi-party convolution operation, a set of directional feature maps is obtained, each of which contains features extracted from different angles. A global average pooling operation is performed on each directional feature map, the information in each feature map is compressed in the spatial dimension, the global features are extracted, and a set of directional feature vectors is obtained. The pooling formula can be expressed as:

[0050] ;

[0051] in, Indicates The global eigenvectors in the directions, and Represent the height and width of the feature map respectively, Represents the position in the feature map pixel values. Through global average pooling, each directional feature map is compressed into a feature vector. The directional feature vectors are feature fused to generate a fused feature vector. The features from different directions are effectively combined so that the model can capture global feature information in multiple directions. In order to introduce stronger nonlinear expression capabilities, the fused feature vector is nonlinearly transformed. The nonlinear transformation is achieved through two fully connected layers, each of which is followed by BatchNormalization and ReLU activation functions. The role of BatchNormalization is to re-standardize the input of each layer, reduce the difference between different samples, and help the model converge quickly. The ReLU activation function enables the model to learn more complex features by introducing nonlinearity. After this step of processing, an intermediate feature vector is generated. The intermediate feature vector is input into the autoencoder network. The autoencoder network consists of an encoder and a decoder. The encoder encodes the input intermediate feature vector into a low-dimensional representation, and the decoder re-decodes this low-dimensional representation into high-dimensional features. The role of the autoencoder is to extract the key features of the input data and remove redundant information by reconstruction. The encoder and decoder of the autoencoder each contain 3 fully connected layers, using the LeakyReLU activation function and Dropout regularization. The LeakyReLU activation function is more robust than the ordinary ReLU because it allows negative values ​​to pass through, thereby alleviating the common "neuron death" problem in ReLU activation. Dropout regularization reduces overfitting and improves the generalization ability of the model by randomly blocking some connections of neurons. Through the encoding and decoding process of the autoencoder, the reconstructed feature vector is finally obtained. In order to enhance the feature expression ability, the reconstructed feature vector is residually connected with the fused feature vector. The residual connection directly adds the input and output, so that the feature information can be effectively transmitted between different layers and avoid information loss. After the residual connection, a multi-directional feature vector is obtained.

[0052] In one example, based on the multi-directional feature vector, anomaly detection and local correction are performed on the bill image to obtain a corrected bill image, including:

[0053] The multi-directional feature vectors are transformed into feature maps. The dimension transformation is achieved through deconvolution operation, using 3 deconvolution layers, each followed by BatchNormalization and ReLU activation function.

[0054] The feature map is input into the edge-guided feature enhancement module for processing to obtain an edge-enhanced feature map; the edge-guided feature enhancement module contains three dilated convolutional layers with dilation rates of 1, 2, and 4, respectively, and each layer is followed by a LeakyReLU activation function;

[0055] The edge enhancement feature map is input into the complementary feature attention module for processing to obtain the attention weighted feature map; the complementary feature attention module includes a channel attention submodule and a spatial attention submodule, and the two attention weights are fused through a gating mechanism;

[0056] Perform multi-scale feature extraction on the attention-weighted feature map to obtain a multi-scale feature atlas; multi-scale feature extraction uses a pyramid pooling network, which includes 4 average pooling layers of different scales;

[0057] The multi-scale feature atlas is input into the anomaly detection network to obtain the anomaly detection heat map; the anomaly detection network adopts the U-Net structure, the encoder and decoder each contain 4 convolution blocks, and use skip connections;

[0058] Based on the anomaly detection heat map, the bill image is segmented to obtain candidate anomaly regions, and the candidate anomaly regions are subjected to morphological processing and connected domain analysis to obtain fine anomaly regions;

[0059] The bill image is locally corrected based on the fine abnormal area to obtain a corrected bill image.

[0060] In this example, the multi-directional feature vectors are transformed in dimension and restored to feature maps. The dimension transformation is achieved through deconvolution, which is a common upsampling method used to restore low-dimensional feature vectors to high-dimensional feature maps. Three deconvolution layers are used, and each deconvolution layer is followed by BatchNormalization and ReLU activation functions. BatchNormalization can effectively stabilize the training process and accelerate the convergence of the model; the ReLU activation function makes the model more expressive by introducing nonlinearity. The mathematical expression of deconvolution is:

[0061] ;

[0062] in, represents the generated feature map, is the input feature vector, is the deconvolution kernel, is the bias term, is the activation function ReLU. The deconvolution operation restores the spatial information of the feature map through the reverse convolution process to ensure that the features of the high-dimensional space are fully preserved. After the dimensionality conversion is completed, the obtained feature map is input to the edge-guided feature enhancement module for processing. The goal of the edge-guided feature enhancement module is to enhance the edge features of the image. The module contains 3 dilated convolution layers, each of which is followed by the LeakyReLU activation function. The role of dilated convolution is to expand the receptive field while keeping the size of the convolution kernel unchanged. By setting different dilation rates (1, 2, and 4 respectively), the model can capture edge information at different scales. Compared with the ordinary RelU, the LeakyReLU activation function allows negative values ​​to pass, which helps to alleviate the "neuron death" phenomenon and thus improve the robustness of the model. After being processed by the edge-guided feature enhancement module, the generated edge-enhanced feature map can more clearly present the edge and structural information in the bill image. The feature map is input into the complementary feature attention module. The complementary feature attention module improves the expressiveness of the feature map by combining channel attention and spatial attention. The channel attention submodule adjusts the weight of each channel by selecting important feature channels, while the spatial attention submodule assigns weights to different regions according to the spatial position of the feature map. By fusing the two attention weights through a gating mechanism, the complementary feature attention module can more accurately capture important detail information in the image and generate an attention-weighted feature map. The attention-weighted feature map is input into the multi-scale feature extraction module. This module uses a pyramid pooling network to perform multi-scale processing on the feature map through 4 average pooling layers of different scales (1×1, 2×2, 3×3, and 6×6, respectively). The purpose of pyramid pooling is to capture features of different scales in the image, from global to local, to ensure that the model has an accurate understanding of both the global structure and local details of the bill image. Multi-scale feature extraction can be expressed as:

[0063] ;

[0064] in, Indicates The feature vector of scale, For the The window size of the pooling layer is is the pixel value in the feature map. Through pooling operations at different scales, a multi-scale feature atlas can be generated to ensure that image information at different levels is captured. The multi-scale feature atlas is input into the anomaly detection network. The anomaly detection network adopts the U-Net structure, which is a network commonly used for image segmentation tasks. It has a symmetrical encoder and decoder structure and uses jump connections between the encoder and decoder. The jump connection allows low-level features in the encoder to be directly passed to the decoder, so that fine-grained feature information is not lost during the decoding process. In the U-Net structure, the encoder and decoder each contain 4 convolution blocks, and anomaly detection heat maps are generated by downsampling and upsampling the image layer by layer. The anomaly detection heat map identifies the abnormal areas that may exist in the bill image. Based on the anomaly detection heat map, the bill image is segmented to obtain candidate abnormal areas, the results of anomaly detection are converted into clear image regions, and these areas are marked as candidate abnormal areas. In order to accurately locate the abnormal areas, morphological processing and connected domain analysis are used. Morphological processing includes operations such as expansion and corrosion, which can remove noise and correct regional edges; connected domain analysis identifies adjacent pixel groups in the image, confirms the boundaries of abnormal regions, and obtains fine abnormal regions. Local correction is performed on fine abnormal regions to correct abnormalities in the bill image, such as text tilt, image noise, or structural damage. Through local correction, the corrected bill image is obtained.

[0065] In one example, based on the multi-directional feature vector and the corrected bill image, an orientation recognition network is used to perform bill orientation recognition to obtain an initial orientation recognition result, including:

[0066] Perform feature enhancement on multi-directional feature vectors to obtain enhanced feature vectors; feature enhancement uses a self-attention mechanism, including a multi-head self-attention layer and a feedforward neural network layer, each layer is followed by LayerNormalization;

[0067] The corrected bill image is input into the pre-trained deep convolutional neural network to obtain the image feature vector; the deep convolutional neural network adopts the DenseNet structure, which contains 4 dense blocks, each dense block contains 6 convolutional layers, and uses batch normalization and ReLU activation function;

[0068] Perform feature fusion on the enhanced feature vector and the image feature vector to obtain a multimodal feature vector;

[0069] The multimodal feature vector is input into the orientation classification network to obtain the orientation probability distribution; the orientation classification network contains 3 fully connected layers, using Dropout regularization and LeakyReLU activation function, and the last layer uses Softmax activation function;

[0070] Based on the orientation probability distribution, conditional random field optimization is performed to obtain the optimized orientation probability distribution, and maximum likelihood estimation is performed on the optimized orientation probability distribution to obtain a preliminary orientation recognition result.

[0071] In this example, feature enhancement is performed on multi-directional feature vectors. Feature enhancement uses a self-attention mechanism and is processed through a multi-head self-attention layer and a feedforward neural network layer, each of which is followed by a LayerNormalization layer. The self-attention mechanism can focus on different parts of the input vector through different attention heads and capture long-range dependencies between input features. In the multi-head self-attention layer, the input vector is linearly transformed to generate a query matrix, a key matrix, and a value matrix, and multiple attention outputs are obtained by calculating the similarity weights between the query matrix and the key matrix and then adding the weighted value matrix. The formula can be expressed as:

[0072] ;

[0073] in, represents the query matrix, is the key matrix, is the value matrix, is the dimension of the vector. Through the self-attention mechanism, the enhanced feature vector is obtained. The feedforward neural network performs nonlinear transformation on the features, and ensures the stability of training and the normalization of features through LayerNormalization. At the same time, the corrected bill image is input into the pre-trained deep convolutional neural network to extract the feature vector of the image. The convolutional neural network adopts the DenseNet structure. DenseNet is a densely connected neural network. Its notable feature is that in each layer of the network, the input includes not only the output of the current layer, but also the output of all previous layers. The dense connection design enhances the reusability of features and reduces the risk of gradient disappearance. The DenseNet network contains 4 dense blocks, each of which has 6 convolutional layers. Each convolutional layer is followed by batch normalization (BatchNormalization) and ReLU activation function. Batch normalization can effectively improve training efficiency. ReLU introduces nonlinearity, thereby enhancing the expression ability of the network. The inter-layer connection of DenseNet can be expressed as:

[0074] ;

[0075] in, Indicates The output of the layer, is the convolution operation of this layer, Represents the output from all previous layers. Through this mechanism, DenseNet can effectively extract complex features from bill images and generate image feature vectors. The enhanced feature vector is fused with the image feature vector, and features from different sources are effectively combined, so that the model can not only utilize the global information of multi-directional feature vectors, but also capture the detailed information of image features. Feature fusion can be achieved through simple weighting or connection operations. The fused multimodal feature vector contains all-round information of the bill. The multimodal feature vector is input into the orientation classification network, which is used to output the orientation probability distribution of the bill. The network consists of 3 fully connected layers, each of which is followed by Dropout regularization and LeakyReLU activation function. Dropout regularization reduces the risk of overfitting and improves the generalization ability of the model by randomly shielding the output of some neurons; the LeakyReLU activation function allows negative values ​​to pass, thereby alleviating the "neuron death" problem of the ReLU function. In the last layer of the orientation classification network, the Softmax activation function is used to convert the output of the network into a probability distribution, representing the predicted probability of each possible bill orientation. The formula of the Softmax function is:

[0076] ;

[0077] in, For the predicted category The probability of For the The raw scores (logits) of each category. Through Softmax, the predicted probability of each orientation output by the network can be normalized, and the sum of the probabilities of all orientations is 1. In order to optimize the orientation recognition results, conditional random field optimization is performed based on the orientation probability distribution. Conditional random field is a probabilistic graph model for sequence labeling that can adjust the classification results based on contextual information to obtain more consistent predictions. The optimization process of the conditional random field is adjusted by maximizing the global probability distribution to ensure that the final orientation prediction not only considers local information but also captures global structural relationships. The optimization formula of the conditional random field can be expressed as:

[0078] ;

[0079] in, Represents a given input feature sequence Output position label sequence The probability of is the local energy function, is a normalization term. Through conditional random field optimization, the optimized orientation probability distribution is obtained. The maximum likelihood estimation is performed on the optimized orientation probability distribution to determine the final orientation prediction result. The maximum likelihood estimation can output the most likely orientation recognition result by selecting the parameter estimation value that maximizes the probability of the observed data.

[0080] In one example, the corrected bill image is input into a pre-trained deep convolutional neural network to obtain an image feature vector; the deep convolutional neural network adopts a DenseNet structure, which contains 4 dense blocks, each dense block contains 6 convolutional layers, uses batch normalization and ReLU activation function, including:

[0081] Preprocessing the corrected bill image to obtain a normalized image;

[0082] Input the normalized image into the first dense block in the pre-trained deep convolutional neural network to obtain the first feature map; the first dense block contains 6 convolutional layers, and the output of each convolutional layer is connected with the output of all previous layers to form a dense connection structure;

[0083] Performing feature recalibration on the first feature map to obtain a recalibrated feature map; the feature recalibration includes global average pooling, two fully connected layers and a sigmoid activation function;

[0084] The recalibrated feature map is input into the second dense block and the third dense block to obtain an intermediate feature map; the structures of the second dense block and the third dense block are the same as the first dense block;

[0085] Multi-scale feature extraction is performed on the intermediate feature map to obtain a multi-scale feature atlas. Multi-scale feature extraction uses a spatial pyramid pooling module, which includes four pooling scales: 1×1, 2×2, 3×3, and 6×6.

[0086] The multi-scale feature map set is input into the fourth dense block to obtain a high-level feature map; the fourth dense block adds an attention mechanism between each convolution layer, using a combination of channel attention and spatial attention;

[0087] Global average pooling and global maximum pooling are performed on the high-level feature map to obtain two global feature vectors, and the two global feature vectors are concatenated and dimensionally reduced to obtain the image feature vector.

[0088] In this example, the corrected bill image is preprocessed to obtain a normalized image. The preprocessing steps include adjusting the size of the image and normalizing the pixel values ​​so that the numerical range of the image is unified so that it can meet the input requirements of the subsequent neural network. The image can also undergo operations such as denoising and grayscale to remove excess noise information and ensure that the image quality reaches an ideal level. After completing these operations, a normalized image is obtained. The normalized image is input into the first dense block in the pre-trained deep convolutional neural network, which contains 6 convolutional layers. The salient feature of the dense block structure is that the output of each layer depends not only on the input of the current layer, but also on the output of all previous layers, that is, a densely connected structure is formed. The formula for dense connection is:

[0089] ;

[0090] in, Indicates The output of the layer, is the convolution operation of this layer, Represents the output from all previous layers. Through dense connections, information flows more smoothly, reducing the problem of gradient vanishing and improving the reusability of features. In the 6 convolutional layers of the first dense block, each convolution operation depends on the feature information of the previous layer, and finally the first feature map is obtained. The first feature map is recalibrated to enhance the expressiveness of the model by redistributing the weights of the features. The feature recalibration process includes global average pooling, two fully connected layers, and a sigmoid activation function. A global average pooling operation is performed to compress the spatial information of the feature map into a global feature vector, helping the model understand the characteristics of the image as a whole. The formula is as follows:

[0091] ;

[0092] in, is the feature vector after pooling, and are the height and width of the feature map, respectively. Represents the position in the feature map The pixel value of the image. The pooled feature vector is further processed by two fully connected layers, and the output is normalized to the range of [0, 1] by the sigmoid activation function. Feature recalibration can effectively enhance valuable features, suppress redundant information, and obtain a recalibrated feature map. The recalibrated feature map is input into the second dense block and the third dense block. The structures of the second and third dense blocks are the same as the first dense block, both containing 6 convolutional layers, and continue to maintain the dense connection characteristics of DenseNet. Through these two dense blocks, the network extracts deep features in the bill image and generates an intermediate feature map. Multi-scale feature extraction is performed on the intermediate feature map. Multi-scale feature extraction uses a spatial pyramid pooling module, which contains four pooling scales of 1×1, 2×2, 3×3, and 6×6. Through different pooling scales, the model can capture feature information at different spatial levels at the same time. For example, a larger pooling window can capture global features, while a smaller pooling window can retain fine-grained local information. The formula for the pooling operation is:

[0093] ;

[0094] in, For the The pooling features of the scale, represents the size of the pooling window, is the pixel value in the feature map. Through multi-scale pooling operations, a multi-scale feature atlas is generated, which can better represent the global and local features of the image. The multi-scale feature atlas is input into the fourth dense block to obtain a high-level feature map. The fourth dense block adds an attention mechanism between each convolution layer, using a combination of channel attention and spatial attention. The channel attention mechanism helps the network focus on important channel information by assigning weights to each feature channel; the spatial attention mechanism enables the model to focus on important areas by assigning weights to each spatial position in the image. The combination of the two can effectively improve the model's ability to capture key information in the image and generate a high-level feature map. Global average pooling and global maximum pooling operations are performed on the high-level feature map to obtain two global feature vectors. Global average pooling can summarize the image features as a whole, while global maximum pooling can capture the salient features in the image. Through these two pooling operations, two different global feature vectors are generated, representing the global information and local salient features of the image respectively. The two global feature vectors are concatenated and the final image feature vector is obtained through dimensionality reduction operations. Dimensionality reduction operations reduce computational complexity by reducing the dimension of features while maintaining important feature information.

[0095] In one example, a multimodal feature vector is input into a position classification network to obtain a position probability distribution; the position classification network contains three fully connected layers, uses Dropout regularization and LeakyReLU activation function, and the last layer uses Softmax activation function, including:

[0096] Perform feature normalization processing on the multimodal feature vector to obtain a normalized feature vector;

[0097] The normalized feature vector is input into the first fully connected layer of the orientation classification network to obtain an intermediate feature vector, and a Dropout operation is performed on the intermediate feature vector to obtain a sparse feature vector;

[0098] The sparse feature vector is input into the second fully connected layer of the orientation classification network to obtain a high-dimensional feature vector; the number of neurons in the second fully connected layer is 2048, and the LeakyReLU activation function is used with a slope parameter of 0.2;

[0099] Perform feature enhancement on the high-dimensional feature vector to obtain an enhanced feature vector, and input the enhanced feature vector into the third fully connected layer of the orientation classification network to obtain a logits vector;

[0100] The logits vector is temperature scaled to obtain a calibrated logits vector, and the Softmax function is performed on the calibrated logits vector to obtain the orientation probability distribution.

[0101] In this example, the multimodal feature vector is normalized to map each dimension of the feature vector to a uniform numerical range, usually between [0, 1] or [-1, 1]. The normalization operation helps to avoid excessive numerical differences between different features, which causes some features to dominate the model update process during network training, thereby improving the training efficiency and stability of the network. Through normalization, a normalized feature vector is obtained. The normalized feature vector is input into the first fully connected layer of the orientation classification network, and the input features are linearly transformed to map the high-dimensional features to the new feature space. In this layer, the features are linearly transformed to obtain an intermediate feature vector, and its mathematical expression is:

[0102] ;

[0103] in, is the intermediate eigenvector, is the weight matrix of the first fully connected layer, is the bias term, is the normalized feature vector. Through linear transformation, the intermediate feature vector retains the main information of the input features while performing dimensional transformation. In order to prevent the model from overfitting during training, the intermediate feature vector is subjected to Dropout operation. Dropout is a commonly used regularization method that makes the model more robust by randomly masking the output of some neurons. The output after Dropout is a sparse feature vector. The sparse feature vector is input into the second fully connected layer of the orientation classification network. The number of neurons in this layer is set to 2048. Through the larger network capacity, deep information in the input data can be extracted. The activation function of the second fully connected layer adopts LeakyRelU, and the expression of its activation function is:

[0104] ;

[0105] in, is the slope parameter, set to 0.2, allowing negative values ​​to pass, thereby alleviating the common "neuron death" problem in the ReLU activation function. Through the LeakyReLU activation function, the network can capture positive signals while not completely ignoring negative signals, so that the network's nonlinear expression ability is enhanced. After the second fully connected layer, a high-dimensional feature vector is output, which contains the deep semantic information of the input features. The high-dimensional feature vector is subjected to feature enhancement processing, and the model's ability to capture orientation information is improved through nonlinear transformation and feature combination. Through this process, an enhanced feature vector is obtained. The enhanced feature vector is input into the third fully connected layer of the orientation classification network to obtain the logits vector. The logits vector represents the original score of each category in the orientation classification task. Temperature scaling is performed on logits. Temperature scaling is a commonly used probability calibration method that adjusts the model's prediction confidence by scaling the logits value. Its mathematical expression is:

[0106] ;

[0107] in, is the temperature scaled logits vector, are the original logits, is the temperature parameter. The role of the temperature parameter is to control the smoothness of logits. A higher temperature will make logits smoother, thereby reducing the confidence of the model; a lower temperature will make logits sharper and increase the confidence of the model. Through temperature scaling, the output probability of the model can be better calibrated. After completing the temperature scaling, the calibrated logits vector is operated by the Softmax function. The role of the Softmax function is to convert logits into a probability distribution, ensuring that the sum of the probabilities of all categories is 1. The formula for Softmax is:

[0108] ;

[0109] in, Represents input features Belongs to category The probability of Yes Category The corresponding logits value. The Softmax function maps loqits to the probability value of each category through exponential operation and normalization operation, generating a direction probability distribution.

[0110] In one example, connectivity detection is performed on the corrected bill image to obtain a connectivity detection result, and orientation correction is performed based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result, including:

[0111] Performing edge detection on the corrected bill image to obtain an edge image, and performing binarization processing on the edge image to obtain a binary edge image;

[0112] Perform morphological processing on the binary edge image to obtain an optimized edge image; the morphological processing includes opening and closing operations, using a 3x3 rectangular structure element to remove noise and fill small holes;

[0113] Performing connected domain analysis on the optimized edge image to obtain a connected domain labeled image, and calculating the direction and aspect ratio of the maximum connected area based on the connected domain labeled image to obtain a connectivity feature vector;

[0114] Perform text line detection on the corrected bill image to obtain the text line area, and calculate the main text direction based on the text line area to obtain the text direction vector;

[0115] The connectivity feature vector, text direction vector and initial orientation recognition result are weightedly fused to obtain the target orientation recognition result.

[0116] In this example, edge detection is performed on the corrected bill image to extract the main structural features in the image. The goal of edge detection is to identify the boundaries and contours in the image, especially key elements such as text, frames, and patterns in the bill image. Common edge detection algorithms include Canny and Sobel, which locate edges by calculating the gradient change of pixel grayscale in the image. For bill images, Canny edge detection is a common method that can effectively capture clear edges while removing noise. The output of Canny edge detection is a binary edge image with edge pixels as 1 and non-edge pixels as 0. Binarization is performed on the edge image to enhance the boundary contrast in the image and further simplify the grayscale information in the image so that each pixel has only a value of 0 or 1, highlighting the edge information. Global threshold or adaptive threshold methods are often used for binarization. Adaptive threshold performs better in images with uneven illumination and can dynamically adjust the threshold according to the brightness of the local image to ensure that the edge can be accurately detected. After binarization, a binary edge image is obtained. Morphological processing is performed on the binary edge image to optimize the structure of the image. Morphological processing includes opening and closing operations. The opening operation is to erode first and then expand, which is mainly used to remove small noise points in the image, so as to make the edge smoother; the closing operation is to expand first and then erode, which can fill small holes or voids in the image, so as to enhance the continuity of the edge. These two operations use 3x3 rectangular structural elements to optimize the morphological characteristics of the edge image by processing the image pixel by pixel. After morphological processing, the optimized edge image is smoother and cleaner, with clearer edges and reduced noise interference. Connected domain analysis is performed on the optimized edge image to identify the groups of pixels connected to each other in the image and mark them as different regions. During the connected domain analysis process, the algorithm identifies all connected pixel regions and assigns a unique label to each region. Each connected region in the image is clearly identified to generate a connected domain label image. By analyzing the connected domain, important features related to the image structure are extracted. For example, the direction and aspect ratio of each connected region are calculated. The largest connected region usually corresponds to the main part of the image, such as the frame or text area of ​​the bill. The direction of the connected area can be calculated by the angle of its minimum circumscribed rectangle, and the aspect ratio is the ratio of the width to the height of the circumscribed rectangle. Through calculation, a connectivity feature vector is generated. Describe the overall geometric shape of the image. At the same time, the text line detection is performed on the corrected bill image to analyze the text structure in the image. Text line detection is to segment the text area in the image, which is usually achieved through pixel-based density analysis or deep learning methods. Common deep learning methods include convolutional neural networks or connection temporal classification algorithms, which can effectively identify the text area in the image and segment the text lines. By analyzing the text line area, the main text direction is calculated.The text direction usually indicates the angle of text arrangement, especially in bill images, where the text is usually arranged horizontally. Therefore, the text direction vector can be determined by analyzing the overall tilt angle of the text line. The connectivity feature vector and the text direction vector are weighted fused with the initial orientation recognition result. The initial orientation recognition result may come from a previous orientation classification algorithm, providing a preliminary judgment on the overall orientation of the bill image. In order to more accurately determine the final orientation of the bill, it is necessary to combine the geometric structure and text direction information in the image. The weighted fusion process can give weights to different features, comprehensively consider the importance of these features in orientation recognition, and fuse their orientation information to obtain a more accurate target orientation recognition result.

[0117] In one example, text line detection is performed on the corrected bill image to obtain a text line area, and the main text direction is calculated based on the text line area to obtain a text direction vector, including:

[0118] Preprocessing the corrected bill image to obtain an enhanced image;

[0119] The enhanced image is input into the text region detection network to obtain the text region probability map; the text region detection network adopts a fully convolutional network structure, including an encoder and a decoder, and uses dilated convolution and feature pyramid network to extract multi-scale features;

[0120] Post-processing the text area probability map to obtain a text area binary map, and performing connected area analysis based on the text area binary map to obtain candidate text line areas;

[0121] Merge candidate text line regions to obtain text line regions; text line merging is based on distance threshold and direction consistency, and a hierarchical clustering algorithm is used to combine adjacent candidate regions into a complete text line;

[0122] Performing perspective correction on the original image based on the text line area to obtain a corrected text line image;

[0123] Directional feature extraction is performed on the corrected text line image to obtain a directional feature map. The directional feature extraction uses a Gabor filter group, which includes 8 filters in different directions, to extract the directional information of the text strokes, and the main text direction is calculated based on the directional feature map to obtain the text direction vector.

[0124] In this example, the rectified bill image is preprocessed to obtain an enhanced image. Preprocessing usually includes operations such as adjusting contrast, denoising, and image smoothing. These steps can effectively enhance the details and edge information in the image. The enhanced image is input into the text region detection network. The text region detection network adopts a fully convolutional network structure. The fully convolutional network structure consists of an encoder and a decoder. The encoder is used to gradually downsample the image and extract high-level semantic features, while the decoder restores these features to the same resolution as the input image through upsampling. In order to enhance the detection of multi-scale text regions, the text region detection network uses dilated convolution and feature pyramid network. Dilated convolution expands the receptive field by inserting "holes" in the convolution kernel, and can capture a wider range of contextual information without increasing the amount of computation. The feature pyramid network effectively captures text regions of different sizes by extracting features at different scales, improving the detection accuracy of small and large text regions. The network finally outputs a text region probability map, which shows the probability of each pixel belonging to the text region. The text region probability map is post-processed to obtain a text region binary map. The information in the probability map is converted into clear binary information, that is, the pixel values ​​in the image are classified as 0 (non-text area) or 1 (text area). Common post-processing methods include using a fixed threshold or an adaptive threshold to mark pixels with probability values ​​higher than a certain set threshold as text areas. After post-processing, the generated text area binary map clearly shows the text area in the image. Connected region analysis is performed on the text area binary map to connect adjacent text areas to form a complete candidate text line area. Connected region analysis is a method to determine different areas by detecting the connectivity of adjacent pixel groups in an image. In this process, all adjacent text pixels are combined into candidate text line areas. The result of the connected region analysis is a set of connected regions in the image that may contain text lines. Text line merging operation is performed on the candidate text line areas. Since text lines may be divided into multiple small blocks in the image due to noise, occlusion or other factors, it is necessary to merge adjacent candidate areas into complete text lines. The merging process is based on two criteria: distance threshold and direction consistency. The distance threshold is used to determine whether the distance between two candidate areas is close enough. If the distance between adjacent areas is less than the set threshold, they will be considered to belong to the same row. Direction consistency is used to check whether the text directions of the two candidate areas are consistent to ensure that the direction of the merged area is reasonable. The merging operation uses a hierarchical clustering algorithm, which gradually aggregates similar candidate areas to eventually form a complete text line area. Based on the complete text line area, the original image is perspective corrected to correct the geometric distortion in the image so that the tilted text line can be restored to a horizontal state. Perspective distortion is more common in practical applications, especially when scanning bills or documents. The angle of the scanning device may cause the text lines in the image to tilt.Perspective correction is performed by aligning the vertices of the text line with the standard horizontal line to ensure that each line of text is arranged horizontally, thereby obtaining a corrected text line image. Directional feature extraction is performed on the corrected text line image to analyze the direction information of the strokes in the text line. Directional feature extraction is performed using a Gabor filter group. The Gabor filter is a tool commonly used in texture analysis and edge detection, which can capture different directional information of an image. A Gabor filter with 8 different directions is used, corresponding to different angles. These filters can identify the stroke direction of the text and generate a directional feature map. Directional feature extraction can be expressed as the following formula:.

[0125] ;

[0126] in, is the output of the Gabor filter, and are the rotated coordinates, is the direction angle of the filter, is the spatial aspect ratio, is the standard deviation of the Gaussian kernel, is the wavelength of the filter, is the phase shift. By responding to Gabor filters in different directions, the direction information of the text strokes can be extracted to generate a directional feature map. The main text direction is calculated based on the directional feature map. The main text direction is determined according to the overall arrangement direction of the strokes in the text line. The main direction of each line of text is obtained by analyzing the main response direction in the directional feature map. The direction information is integrated to generate a text direction vector, which represents the main text direction in the entire bill image.

[0127] In one example, based on the target orientation recognition result, text recognition is performed on the corrected bill image to obtain bill text information, including:

[0128] Performing rotation correction on the corrected bill image according to the target orientation recognition result to obtain a direction-corrected image;

[0129] Adaptively binarizing the direction-corrected image to obtain a binarized image, and performing text line segmentation based on the binarized image to obtain a text line image set;

[0130] Performing tilt correction on each text line image in the text line image set to obtain a corrected text line image set;

[0131] The corrected text line image set is input into the SikuBERT model to obtain the text feature sequence; the SikuBERT model contains 12 Transformer encoder layers;

[0132] The text feature sequence is input into the bidirectional long short-term memory network for feature extraction to obtain a context-enhanced feature sequence; the bidirectional long short-term memory network contains 2 layers, each layer has 128 hidden units, and uses residual connections;

[0133] The context-enhanced feature sequence is input into the multi-head attention module to obtain the attention-weighted feature sequence; the multi-head attention module contains 8 attention heads, and the dimension of each head is 64;

[0134] Sequence labeling is performed based on the attention-weighted feature sequence to obtain the bill text information.

[0135] In this example, the corrected bill image is rotated to ensure that the text and table structure in the image are in the correct direction. The rotation correction is based on the previously identified target orientation angle. By performing a corresponding rotation transformation on the image, the content in the bill image is restored to the standard level. This process can be achieved through affine transformation. The rotation formula is:

[0136] ;

[0137] in, is the rotation angle, and are the pixel coordinates in the original image, and is the new coordinate after rotation. Through the rotation correction operation, a direction-corrected image is obtained. The direction-corrected image is subjected to adaptive binarization to obtain a binary image. Adaptive binarization is a local area-based binarization method that can dynamically adjust the threshold according to the brightness of different areas of the image and convert the grayscale information in the image into a binary image (black and white image). Text line segmentation is performed based on the binary image. By analyzing the density and arrangement pattern of pixels in the image, the start and end positions of the text line can be identified, and each line of text in the image can be segmented to form a text line image set. The segmentation process can be achieved through a projection method or contour detection to ensure that each line of text can be accurately extracted. Each text line image in the text line image set is tilted, and the text lines that may be tilted are adjusted to a horizontal position to ensure that the text is neatly arranged. Tilt correction can be achieved through the angle of the minimum circumscribed rectangle, that is, the rotation angle of the text line is calculated according to its direction, and reverse rotation correction is performed. This operation makes the corrected text line image more regular and reduces recognition errors caused by text tilt. The corrected text line image set is input into the SikuBERT model for feature extraction. The SikuBERT model is based on the Transformer architecture and contains 12 encoder layers. Each encoder layer captures contextual information in the text sequence through the self-attention mechanism. In the SikuBERT model, the input text line image is converted into a text feature sequence. The self-attention mechanism extracts meaningful features by focusing on different parts of the input sequence. The calculation formula for self-attention is:

[0138] ;

[0139] in, , , are query, key, and value matrices respectively, is the dimension of the key. Through the self-attention mechanism, SikuBERT can capture the long-range dependencies in the text and generate a text feature sequence containing contextual information. The text feature sequence is input into the bidirectional long short-term memory network (BiLSTM) for feature extraction. The bidirectional long short-term memory network can capture richer contextual information by processing the forward and backward information of the sequence at the same time. Each layer of BiLSTM contains 128 hidden units, and residual connections are used to ensure the effective transmission of information. Residual connections avoid the problem of information loss that may occur in deep networks by directly adding input and output. BiLSTM uses bidirectional processing to enable the text feature sequence to better reflect the forward and backward dependencies between words and obtain a context-enhanced feature sequence. The context-enhanced feature sequence is input into the multi-head attention module for processing. The multi-head attention mechanism allows the model to process different parts of the features in parallel on different heads, so that the model can capture different contextual relationships in different dimensions. The multi-head attention module contains 8 attention heads, each with a dimension of 64. Multi-head attention can focus on different parts of the input in parallel, thereby improving the processing power of the model. Each attention head calculates its own independent attention score and performs weighted average of the results. After multi-head attention processing, an attention weighted feature sequence is obtained. Based on the attention weighted feature sequence, sequence annotation is performed to identify the text information in the bill image. Sequence annotation assigns a corresponding label to each input character and finally outputs the complete bill text information.

[0140] Reference Figure 2 This embodiment provides a bill orientation-based identification and correction system, including:

[0141] Extraction module 1, used for performing multi-directional feature extraction on the input bill image to obtain a multi-directional feature vector;

[0142] Correction module 2, used for performing abnormality detection and local correction on the bill image based on the multi-directional feature vector to obtain a corrected bill image;

[0143] A direction recognition module 3 is used to perform bill direction recognition using a direction recognition network according to the multi-directional feature vector and the corrected bill image to obtain an initial direction recognition result;

[0144] A detection module 4 is used to perform connectivity detection on the corrected bill image to obtain a connectivity detection result, and perform orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result;

[0145] The text recognition module 5 is used to perform text recognition on the corrected bill image based on the target orientation recognition result to obtain bill text information.

[0146] In this embodiment, for the specific implementation of each unit in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0147] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input system, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0148] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0149] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0151] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, system, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, system, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, system, article or method including the element.

[0152] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying and correcting bill orientation, characterized in that: The following steps are involved: The method comprises the following steps: performing multi-directional feature extraction on the input bill image to obtain a multi-directional feature vector; specifically comprising: preprocessing the input bill image to obtain a standardized bill image; inputting the standardized bill image into a pre-trained convolutional neural network to obtain a basic feature map; the pre-trained convolutional neural network adopts a ResNet50 structure, comprising 5 convolution blocks, each of which comprises 3 convolution layers and a short-circuit connection; performing a multi-directional convolution operation on the basic feature map to obtain a directional feature atlas; the multi-directional convolution operation comprises using 4 parallel convolution branches, corresponding to 0°, 45°, 90° and 135° directions respectively, each branch comprising 3 convolution layers, and the convolution kernel sizes are 1×3, 3×1 and 3×3 respectively; performing a multi-directional convolution operation on the basic feature map to obtain a directional feature atlas; Performing a global average pooling operation on each directional feature map in the image to obtain a directional feature vector set, and performing feature fusion on the directional feature vector set to obtain a fused feature vector; performing a nonlinear transformation on the fused feature vector to obtain an intermediate feature vector; the nonlinear transformation includes using two fully connected layers, each layer is followed by a BatchNormalization and a ReLU activation function; inputting the intermediate feature vector into an autoencoder network to obtain a reconstructed feature vector; the autoencoder network includes an encoder and a decoder, each of which includes 3 fully connected layers, using a LeakyReLU activation function and a Dropout regularization; performing a residual connection between the reconstructed feature vector and the fused feature vector to obtain a multi-directional feature vector; Based on the multi-directional feature vector, abnormality detection and local correction are performed on the bill image to obtain a corrected bill image; According to the multi-directional feature vector and the corrected bill image, the bill orientation is recognized using the orientation recognition network to obtain an initial orientation recognition result; specifically, the multi-directional feature vector is enhanced to obtain an enhanced feature vector; the feature enhancement uses a self-attention mechanism, including a multi-head self-attention layer and a feedforward neural network layer, each layer is followed by LayerNormalization; the corrected bill image is input into a pre-trained deep convolutional neural network to obtain an image feature vector; the deep convolutional neural network adopts a DenseNet structure, including 4 dense blocks, each dense block contains 6 convolutional layers, using batch normalization and ReLU activation function; performing feature fusion on the enhanced feature vector and the image feature vector to obtain a multimodal feature vector; inputting the multimodal feature vector into an orientation classification network to obtain an orientation probability distribution; the orientation classification network comprises three fully connected layers, using Dropout regularization and LeakyReLU activation function, and the last layer uses a Softmax activation function; performing conditional random field optimization based on the orientation probability distribution to obtain an optimized orientation probability distribution, and performing maximum likelihood estimation on the optimized orientation probability distribution to obtain a preliminary orientation recognition result; Performing connectivity detection on the corrected bill image to obtain a connectivity detection result, and performing orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result; Based on the target orientation recognition result, text recognition is performed on the corrected bill image to obtain bill text information.

2. The method for identifying and correcting bill orientation according to claim 1, characterized in that: The method of performing abnormality detection and local correction on the bill image based on the multi-directional feature vector to obtain a corrected bill image includes: Performing dimension transformation on the multi-directional feature vector to obtain a feature map; the dimension transformation is achieved by a deconvolution operation, using three deconvolution layers, each followed by a BatchNormalization and a ReLU activation function; Inputting the feature map into an edge-guided feature enhancement module for processing to obtain an edge-enhanced feature map; the edge-guided feature enhancement module comprises three dilated convolutional layers, with dilation rates of 1, 2, and 4, respectively, and each layer is followed by a LeakyReLU activation function; Inputting the edge enhancement feature map into a complementary feature attention module for processing to obtain an attention weighted feature map; the complementary feature attention module includes a channel attention submodule and a spatial attention submodule, and the two attention weights are fused through a gating mechanism; Performing multi-scale feature extraction on the attention weighted feature map to obtain a multi-scale feature atlas; the multi-scale feature extraction uses a pyramid pooling network, including 4 average pooling layers of different scales; Inputting the multi-scale feature atlas into an anomaly detection network to obtain an anomaly detection heat map; the anomaly detection network adopts a U-Net structure, and the encoder and decoder each contain 4 convolution blocks and use skip connections; Based on the abnormality detection heat map, the bill image is segmented to obtain candidate abnormal regions, and the candidate abnormal regions are subjected to morphological processing and connected domain analysis to obtain fine abnormal regions; The bill image is locally corrected based on the fine abnormal area to obtain a corrected bill image.

3. The method for identifying and correcting bill orientation according to claim 1, characterized in that: The corrected bill image is input into a pre-trained deep convolutional neural network to obtain an image feature vector; the deep convolutional neural network adopts a DenseNet structure, including 4 dense blocks, each dense block includes 6 convolutional layers, and uses batch normalization and ReLU activation function, including: Preprocessing the corrected bill image to obtain a normalized image; Inputting the normalized image into a first dense block in a pre-trained deep convolutional neural network to obtain a first feature map; the first dense block includes 6 convolutional layers, and the output of each convolutional layer is connected with the outputs of all previous layers to form a dense connection structure; Performing feature recalibration on the first feature map to obtain a recalibrated feature map; the feature recalibration includes global average pooling, two fully connected layers and a sigmoid activation function; Inputting the recalibrated feature map into a second dense block and a third dense block to obtain an intermediate feature map; the structures of the second dense block and the third dense block are the same as those of the first dense block; Performing multi-scale feature extraction on the intermediate feature map to obtain a multi-scale feature atlas; the multi-scale feature extraction uses a spatial pyramid pooling module, including four pooling scales of 1×1, 2×2, 3×3 and 6×6; Inputting the multi-scale feature map set into a fourth dense block to obtain a high-level feature map; the fourth dense block adds an attention mechanism between each convolution layer, using a combination of channel attention and spatial attention; Global average pooling and global maximum pooling are performed on the high-level feature map to obtain two global feature vectors, and the two global feature vectors are concatenated and dimensionally reduced to obtain an image feature vector.

4. The method for identifying and correcting bill orientation according to claim 1, characterized in that: The multimodal feature vector is input into the orientation classification network to obtain the orientation probability distribution; the orientation classification network comprises three fully connected layers, uses Dropout regularization and LeakyReLU activation function, and the last layer uses Softmax activation function, including: Performing feature normalization processing on the multimodal feature vector to obtain a normalized feature vector; Inputting the normalized feature vector into the first fully connected layer of the orientation classification network to obtain an intermediate feature vector, and performing a Dropout operation on the intermediate feature vector to obtain a sparse feature vector; Input the sparse feature vector into the second fully connected layer of the orientation classification network to obtain a high-dimensional feature vector; the number of neurons in the second fully connected layer is 2048, and the LeakyReLU activation function is used with a slope parameter of 0.2; Performing feature enhancement on the high-dimensional feature vector to obtain an enhanced feature vector, and inputting the enhanced feature vector into a third fully connected layer of an orientation classification network to obtain a logits vector; A temperature scaling operation is performed on the logits vector to obtain a calibrated logits vector, and a Softmax function operation is performed on the calibrated logits vector to obtain an orientation probability distribution.

5. The method for identifying and correcting bill orientation according to claim 1, characterized in that: The performing connectivity detection on the corrected bill image to obtain a connectivity detection result, and performing orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result, including: Performing edge detection on the corrected bill image to obtain an edge image, and performing binarization processing on the edge image to obtain a binarized edge image; Performing morphological processing on the binary edge image to obtain an optimized edge image; the morphological processing includes an opening operation and a closing operation, using a 3x3 rectangular structure element to remove noise points and fill small holes; Performing a connected domain analysis on the optimized edge image to obtain a connected domain labeled image, and calculating the direction and aspect ratio of the maximum connected area based on the connected domain labeled image to obtain a connectivity feature vector; Performing text line detection on the corrected bill image to obtain a text line area, and calculating a main text direction based on the text line area to obtain a text direction vector; The connectivity feature vector, the text direction vector and the initial orientation recognition result are weightedly fused to obtain a target orientation recognition result.

6. The method for identifying and correcting bill orientation according to claim 5, characterized in that: The step of performing text line detection on the corrected bill image to obtain a text line area, and calculating a main text direction based on the text line area to obtain a text direction vector includes: Preprocessing the corrected bill image to obtain an enhanced image; Inputting the enhanced image into a text region detection network to obtain a text region probability map; the text region detection network adopts a fully convolutional network structure, including an encoder and a decoder, and uses a dilated convolution and a feature pyramid network to extract multi-scale features; Post-processing the text region probability map to obtain a text region binary map, and performing connected region analysis based on the text region binary map to obtain candidate text line regions; Merging the candidate text line regions to obtain text line regions; the text line merging is based on a distance threshold and direction consistency, and a hierarchical clustering algorithm is used to combine adjacent candidate regions into a complete text line; Performing perspective correction on the original image based on the text row area to obtain a corrected text row image; Directional feature extraction is performed on the corrected text line image to obtain a directional feature map; the directional feature extraction uses a Gabor filter group, which includes 8 filters in different directions, to extract directional information of text strokes, and the main text direction is calculated based on the directional feature map to obtain a text direction vector.

7. The method for identifying and correcting bill orientation according to claim 1, characterized in that: The step of performing text recognition on the corrected bill image based on the target orientation recognition result to obtain bill text information includes: Performing rotation correction on the corrected bill image according to the target orientation recognition result to obtain a direction-corrected image; Adaptively binarizing the direction-corrected image to obtain a binarized image, and performing text line segmentation based on the binarized image to obtain a text line image set; Performing tilt correction on each text line image in the text line image set to obtain a corrected text line image set; Inputting the corrected text line image set into a SikuBERT model to obtain a text feature sequence; the SikuBERT model comprises 12 Transformer encoder layers; Inputting the text feature sequence into a bidirectional long short-term memory network for feature extraction to obtain a context-enhanced feature sequence; the bidirectional long short-term memory network comprises 2 layers, each layer has 128 hidden units, and uses residual connection; Inputting the context-enhanced feature sequence into a multi-head attention module to obtain an attention-weighted feature sequence; the multi-head attention module comprises 8 attention heads, each of which has a dimension of 64; Sequence labeling is performed based on the attention weighted feature sequence to obtain the bill text information.

8. A bill orientation-based recognition and correction system, characterized in that: For performing the steps of the method according to any one of claims 1 to 7, the system comprises: An extraction module is used to extract multi-directional features from the input bill image to obtain a multi-directional feature vector; A correction module, used for performing abnormality detection and local correction on the bill image based on the multi-directional feature vector to obtain a corrected bill image; An orientation recognition module, used to perform bill orientation recognition using an orientation recognition network according to the multi-directional feature vector and the corrected bill image, to obtain an initial orientation recognition result; A detection module, configured to perform connectivity detection on the corrected bill image to obtain a connectivity detection result, and perform orientation correction based on the connectivity detection result and the initial orientation recognition result to obtain a target orientation recognition result; The text recognition module is used to perform text recognition on the corrected bill image based on the target orientation recognition result to obtain the bill text information.

Images