Check image correction method, device and equipment

Abstract

The embodiment of the invention provides a check image correction method, device and equipment. The method comprises the following steps: acquiring an original deformation check image; processing the original deformation check image through a backbone network, a distortion encoder, a correction decoder and an output predictor in an image correction model in sequence to obtain a target pixel displacement field representing a pixel corresponding relation from the original deformation check image to a corrected check image; and according to the target pixel displacement field, performing bilinear difference distortion operation on the original deformed check image to obtain a corrected check image after the original deformed check image is corrected. An original deformed check image is encoded, an encoded feature map is learned, a corresponding pixel displacement field is generated by a correction decoder, and pixels are sampled from the original deformed check image using the predicted pixel displacement field to reconstruct a corrected check image. Automatic and high-quality deformation check image automatic correction is realized, the labor cost is reduced, and the correction accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a check image rectification method, device and equipment. BACKGROUND

[0002] With the rapid development of banking business, check automatic scanning and recognition technology has become an important means to improve business efficiency and reduce labor costs. However, in actual business, due to the influence of factors such as shooting angle, light condition, paper quality, etc., the check image often appears deformation problems such as distortion, tilt, blur, etc., which seriously affects the subsequent optical character recognition (OCR) and information extraction. Therefore, developing a method that can efficiently rectify various deformed check images is of great significance to improve the efficiency of bank check verification processing.

[0003] In recent years, a large number of researches have made great efforts in document image rectification, but the existing advanced algorithms are limited to processing limited document images, that is, the input image must contain a complete document. Once the captured image only involves a local text area, the rectification quality will decrease. SUMMARY

[0004] The embodiments of the present application provide a check image rectification method, device and equipment, which realizes automatic, high-quality deformation check image automatic rectification, reduces labor costs, improves rectification accuracy, and improves business efficiency.

[0005] In a first aspect, the embodiments provide a check image rectification method, which comprises:

[0006] obtaining an original deformed check image;

[0007] processing the original deformed check image through a backbone network, a distortion encoder, a rectification decoder and an output predictor in the image rectification model in sequence to obtain a target pixel displacement field representing the pixel correspondence relationship from the original deformed check image to a rectified check image;

[0008] performing a bilinear difference warping operation on the original deformed check image according to the target pixel displacement field to obtain a rectified check image rectified from the original deformed check image.

[0009] In a second aspect, the embodiments provide a check image rectification device, which comprises:

[0010] an image acquisition module configured to acquire an original deformed check image;

[0011] The displacement field determination module is used to process the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

[0012] The image correction module is used to perform a bilinear interpolation distortion operation on the original deformed check image based on the target pixel displacement field to obtain a corrected check image after correcting the original deformed check image.

[0013] Thirdly, this embodiment provides an electronic device, including:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the check image correction method according to any embodiment of the present invention.

[0017] This invention provides a method, apparatus, and device for correcting check images. The method includes: acquiring an original deformed check image; processing the original deformed check image sequentially through a backbone network, a distortion encoder, a correction decoder, and an output predictor in an image correction model to obtain a target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image; and performing a bilinear interpolation distortion operation on the original deformed check image based on the target pixel displacement field to obtain a corrected check image. This technical solution first encodes the original deformed check image, then learns the encoded feature map, generates a corresponding pixel displacement field using the correction decoder, and finally uses the predicted pixel displacement field to sample pixels from the original deformed check image to reconstruct the corrected check image. This achieves automated, high-quality automatic correction of deformed check images, reduces labor costs, improves correction accuracy, and enhances business efficiency.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a check image correction method provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is an example diagram of an image correction model in the execution of a check image correction method according to Embodiment 1 of the present invention;

[0022] Figure 3 This is a flowchart illustrating another check image correction method provided in Embodiment 2 of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of an improved self-attention module with two-dimensional position encoding in an image correction model provided in Embodiment 2 of the present invention;

[0024] Figure 5 This is a schematic diagram of a two-dimensional position encoding module in an image correction model provided in Embodiment 2 of the present invention;

[0025] Figure 6 This is a schematic diagram of the structure of a check image correction device provided in Embodiment 3 of the present invention;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Existing document image correction methods are mainly divided into two categories: (1) 3D reconstruction-based methods. 3D reconstruction methods usually first reconstruct the 3D shape of the deformed document page, and then flatten it into a planar shape. These methods have high hardware requirements for scanning deformed documents and are not suitable for individuals. In addition, 3D reconstruction can be performed through multi-view images of the document, but the complex image acquisition steps still limit their further application. (2) Deep learning-based methods. Most existing deep learning-based methods predict the pixel-by-pixel displacement field by detecting text lines and document boundaries, and obtain the corrected document image by resampling pixels from the deformed image into the corrected image. These methods can only process document images with obvious boundaries, which has limitations in practical applications.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a check image correction method according to Embodiment 1 of the present invention. This method is applicable to the correction of deformed check images. The method can be executed by a check image correction device, which can be implemented in hardware and / or software and is generally integrated into an electronic device.

[0032] like Figure 1 As shown, the check image correction method provided in this embodiment may specifically include the following steps:

[0033] S101. Obtain the original deformed check image.

[0034] The check image correction method provided in this embodiment aims to solve the image quality degradation problem caused by non-rigid deformation (such as folding, curling, and perspective distortion) of check images in bank branch check scanning and recognition scenarios. It improves OCR recognition accuracy to a set standard of ≥99.5%, while also improving the flatness of check signatures and anti-counterfeiting marks (such as watermarks and fluorescent markings), thereby enhancing check verification accuracy. OCR: A technology that detects and converts text regions in an image into editable text. It uses image processing and pattern recognition algorithms to identify characters in an image, such as letters, numbers, and symbols, and converts them into computer-readable text format. Non-rigid deformation: Non-linear geometric deformations caused by folding and curling of documents, which cannot be corrected by affine transformations (such as rotation and translation).

[0035] Specifically, the original deformed check image can be understood as a check image that has undergone deformation and needs to be corrected. In this embodiment, the original deformed check image can be directly input into the image correction model for processing. Preferably, before processing the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder, and output predictor in the image correction model, the method further includes: slicing the original deformed check image to obtain processed deformed check image slices, and inputting the deformed check image slices into the image correction model.

[0036] Considering that the original deformed check image may be large, it can be sliced. The resulting slices are then recorded as deformed check image slices. These multiple deformed check image slices are subsequently input into an image correction model for further processing.

[0037] S102. The original deformed check image is processed sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image.

[0038] In this embodiment, the image correction model can be specifically understood as a deep neural network model capable of establishing a pixel displacement field from a deformed check image to a corrected check image. The input to the image correction model is the original deformed check image, and the output is a pixel displacement field representing the pixel correspondence between the deformed and corrected check images. The image correction model is obtained by training the deep neural network model using a sample training set. During network training, the loss function is defined as the predicted pixel displacement field f. b and its given true pixel displacement field f gt The distance L1 between them is given by the following formula: L1=||f gt -f bWhen the distance L1 is less than or equal to the set distance threshold, it means that the model training is complete and the trained model is obtained as the image correction model.

[0039] Specifically, the displacement field can be understood as a vector field describing the positional offset of each pixel in the image, used for deformation correction. The target pixel displacement field can be understood as a vector field describing the positional offset of each pixel in the original deformed image.

[0040] In this embodiment, the image correction model consists of a backbone network, a distortion encoder, a correction decoder, and an output predictor. For example, Figure 2 This is an example diagram of an image correction model in the execution of a check image correction method according to Embodiment 1 of the present invention, as shown below. Figure 2 As shown, the image correction model employs a hierarchical architecture that combines a distortion encoder for multi-level feature extraction and a correction decoder for predicting pixel displacement fields. After the original deformed check image is input into the image correction model, multi-level feature extraction is performed through the backbone network and the distortion encoder architecture. Subsequently, the correction decoder processes the output of the distortion encoder using a fixed number of learnable queries as input. These learnable queries are transformed in parallel into target pixel displacement fields pointing to the original deformed check image. The target pixel displacement field represents the pixel-level correspondence from the original deformed check image to the corrected image.

[0041] It should be noted that the backbone network uses a convolutional network for feature extraction and consists of 6 residual blocks, with 1 / 2 downsampling performed every 2 residual blocks. An improved self-attention module with 2D positional encoding is used to construct the network blocks for the distortion encoder and the correction decoder. In check images, structural information often resides in non-local regions, such as curved text lines and textures with gradually changing light appearances. To encode this structural information for correction, this embodiment introduces an improved self-attention mechanism to capture long-range dependencies and constructs a hierarchical distortion encoder. The distortion encoder consists of three encoding blocks, each employing an improved self-attention module with 2D positional encoding. The correction decoder consists of three decoding blocks, each employing an improved self-attention module with 2D positional encoding.

[0042] Following the above description, an improved self-attention mechanism is used to capture the global dependencies in check images, performing global context modeling and local feature enhancement to address the correction issues of incomplete and borderless checks. Specifically, a distortion encoder captures the global context of the check image, such as check borders and table line distribution, resolving tilt and perspective distortion issues caused by handheld shooting. The correction decoder fuses global features with local details, such as the amount field, to generate a displacement field for key areas of the check, prioritizing the correction of financially sensitive information areas. A multi-head self-attention mechanism models long-distance dependencies, improving the correction accuracy for complex deformations.

[0043] Specifically, the original deformed check image is processed sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

[0044] S103. Based on the target pixel displacement field, perform a bilinear interpolation distortion operation on the original deformed check image to obtain a corrected check image after correcting the original deformed check image.

[0045] In this embodiment, the image obtained after correcting the original deformed check image is denoted as the corrected check image. Specifically, a corrected, distortion-free check image is obtained using a predicted target pixel displacement field and a correction operation based on bilinear sampling. The predicted target pixel displacement field f... b The corrected check image I is obtained based on the following bilinear interpolation distortion operation. r ∈R H×W×3 : in and f b The mapping matrix between the horizontal and vertical coordinates, where (u0, v0) represents the pixel coordinates.

[0046] The above technical solution first encodes the original deformed check image, then learns the encoded feature map, and generates a corresponding pixel displacement field by the correction decoder. Finally, using the predicted pixel displacement field, pixels are sampled from the original deformed check image to reconstruct the corrected check image. This achieves automated, high-quality automatic correction of deformed check images, reduces labor costs, improves correction accuracy, and enhances business efficiency.

[0047] Example 2

[0048] Figure 3This is a flowchart illustrating another check image correction method provided in Embodiment 2 of the present invention. This embodiment is a further optimization of the above embodiment. In this embodiment, the "inputting the original deformed check image into a pre-trained image correction model, and processing it sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image" is further optimized.

[0049] like Figure 3 As shown in the figure, this embodiment two provides a check image correction method, which specifically includes the following steps:

[0050] S201. Obtain the original deformed check image.

[0051] S202. Input the original deformed check image into the backbone network of the image correction model for feature extraction to obtain the initial feature map of the original deformed check image.

[0052] The backbone network consists of 6 residual blocks, and downsampling is performed once for every 2 residual blocks.

[0053] In this embodiment, the backbone network uses a convolutional network for feature extraction, consisting of six residual blocks. Every two residual blocks undergo a 1 / 2 downsampling. This variant of CNN introduces residual blocks to train deeper neural networks, achieving a Residual Convolutional Network (ResNet) as the backbone network. CNN is a deep learning model primarily used to process data with a grid structure, such as images and videos. Its core is to extract local features through convolutional operations and gradually abstract high-level semantic information through multi-layer stacking. Convolutional layers can extract local features from check images, such as the edges of numerals, signature handwriting, and security line texture, providing high-precision low-level features for subsequent deformation modeling. Residual connections alleviate the gradient vanishing problem and support deeper network design.

[0054] In addition, the backbone network can also use deep learning models EfficientNet and HRNet. EfficientNet is lightweight and has strong feature extraction capabilities, but the network structure needs to be adjusted to adapt to the Transformer block; HRNet maintains high-resolution features and is suitable for complex deformation modeling.

[0055] Specifically, the original deformed check image is input into the backbone network of the image correction model for feature extraction, and the extracted feature map is denoted as the initial feature map. For example, continue to refer to... Figure 2 Given an image of a check with arbitrary deformation I d ∈R H ×W×3Where H and W represent the height and width of the RGB image, after feature extraction by the backbone network, a feature map is finally generated. Where C b This indicates the number of convolution kernels, set to 256.

[0056] S203. Input the initial feature map into the distortion encoder of the image correction model for multi-level feature extraction, and output three levels of encoded feature maps.

[0057] In this embodiment, the distortion encoder includes three coding blocks. Each coding block employs an improved self-attention module with two-dimensional position coding, and adjacent coding blocks are connected by downsampling modules. The improved self-attention module includes: a normalization layer, a two-dimensional position coding module, a multi-head self-attention module, learnable residual connections, and a feedforward network.

[0058] It should be noted that Transformer is a deep learning model based on a self-attention mechanism, originally used in natural language processing and widely applied in computer vision, speech recognition, and other fields. Its core principle is to model long-range dependencies in input data through self-attention. Unlike standard self-attention network blocks (Transform Blocks), this embodiment uses an improved self-attention module with two-dimensional positional encoding to construct the distortion encoder and correction decoder network blocks. The learnable two-dimensional positional encoding is used to analyze pixel value changes in local regions of the image, helping the model flexibly capture local positional relationships and understand details and structures in the image. For example, Figure 4 This is a schematic diagram of the structure of an improved self-attention module with two-dimensional positional encoding in an image correction model provided in Embodiment 2 of the present invention, as shown below. Figure 4 As shown, the improved self-attention module includes: a normalization layer, a two-dimensional position encoding module, a multi-head self-attention module, a learnable residual connection, and a feedforward network.

[0059] Figure 5 This is a schematic diagram of a two-dimensional position encoding module in an image correction model provided in Embodiment 2 of the present invention, as shown below. Figure 5 As shown, two-dimensional positional encoding is achieved using depthwise convolution with a 7×7 kernel and 3-level zero padding. Channel-dimensional feature extraction is performed independently on each channel using a smaller kernel to preserve relative positional information in the input feature map. This module is plug-and-play and introduces only a small number of additional parameters (7×7×C), where C represents the number of filters.

[0060] To correct distorted check images, their structural information must be captured. However, structural information in check images is often found in non-local regions, such as curved text lines and textures with gradually changing light appearances. To encode this structural information for correction, this embodiment introduces the self-attention mechanism from the Transformer to capture long-range dependencies and constructs a hierarchical distortion encoder to achieve multi-scale feature extraction.

[0061] Continue to refer to Figure 2 The distortion encoder consists of three coding blocks, each employing an improved self-attention module with 2D positional encoding. To generate high-resolution fine features and low-resolution strong semantic features, the feature maps are downsampled at half resolution using convolutional layers (stride = 2) after the first and second network blocks. Because the Transformer architecture is permutation-invariant, fixed 2D positional encoding is used to supplement the image features before the Transformer encoder layers. Each encoder layer contains a multi-head self-attention module and a feedforward network. The output representation in each layer can be computed as follows:

[0062] E0 = E c +P e

[0063] E′ i-1 =CPosE(LN(E i-1 ))+LN(E i-1 )

[0064] E” i-1 =MA(E′) i-1 ,E′ i-1 )+ResScale(E′ i-1 )

[0065] E i =FFN((LN(E”) i-1 ))+ResScale(E” i-1 ), i = {1, ..., 6}

[0066] Where CPosE(·), LN(·), MA(·), ResScale(·), and FFN(·) represent the two-dimensional positional encoding module, the normalization layer, the multi-head self-attention module, the learnable residual connection, and the feedforward network, respectively, and represent the output features of the i-th encoder layer. E0 represents the input of the first encoder, including the initial feature map E output by the backbone network. c and two-dimensional position encoding P e .

[0067] As a specific implementation, the step of inputting the initial feature map into the distortion encoder of the image correction model for multi-level feature extraction to obtain the three-level encoded feature map output by the distortion encoder can be optimized, including:

[0068] a1) Input the initial feature map into the first coding block of the distortion encoder of the image correction model for encoding processing, and output the first coded feature map.

[0069] In this embodiment, the three coding blocks of the distortion encoder, which are sequentially connected to the backbone network, are denoted as the first coding block, the second coding block, and the third coding block, respectively. Specifically, the initial feature map extracted by the backbone network is input into the first coding block of the distortion encoder in the image correction model for encoding processing, and the output coded feature map is denoted as the first coded feature map. For example, continue to refer to... Figure 2 The first encoding block (Encoder Block 1) is generated by processing the initial feature map. Encode the feature map and output the first encoded feature map (E2).

[0070] b1) The first encoded feature map is downsampled by the first downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the second encoding block of the distortion encoder for encoding processing, and the second encoded feature map is output.

[0071] In this embodiment, the downsampling module between the first and second coding blocks is denoted as the first downsampling module. The downsampling module between the second and third coding blocks is denoted as the second downsampling module. Both the first and second downsampling modules perform 1 / 2 downsampling. Specifically, the first coded feature map output from the first coding block is processed by the first downsampling module and then input into the second coding block for encoding processing. The coded feature map output from the second coding block is denoted as the second coded feature map. Through downsampling module processing, image patches can be made smaller and smaller, allowing for the learning of the relationships between very small regions and focusing more on detailed information. For example, continue to refer to... Figure 2 The first encoded feature map (E2) is downsampled by the first downsampling module (Patch Merging), and the output is... The second encoding block (Encoder Block 2) then performs encoding processing, outputting the second encoded feature map (E4).

[0072] c1) The second encoded feature map is downsampled by the second downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the third encoding block of the distortion encoder for encoding processing, and the third encoded feature map is output.

[0073] Specifically, the second encoded feature map output from the second encoded block is processed by the second downsampling module and then input into the third encoded block for encoding. The encoded feature map output from the third encoded block is denoted as the third encoded feature map. For example, continue to refer to... Figure 2 The second encoded feature map (E4) is downsampled by the second downsampling module (PatchMerging), and the output is... The third encoding block (Encoder Block 3) then performs encoding processing, outputting the third encoded feature map (E6).

[0074] d1) The first coding feature map, the second coding feature map, and the third coding feature map are used as the three levels of coding feature maps output by the distortion encoder.

[0075] The above technical solution specifies the steps for multi-level feature extraction of images through the distortion encoder of the image correction model, realizing the generation of high-resolution fine features and low-resolution strong semantic features, which provides a foundation for subsequent correction accuracy.

[0076] S204. Input the encoded feature maps of the three levels into the correction decoder of the image correction model for processing, and output the decoded feature map.

[0077] Continue to refer to Figure 2 The corrective decoder takes the hierarchical feature maps {E2, E4, E6} from the distortion encoder, and a fixed number of learnable embeddings. As input, it outputs the pixel displacement field f. b ∈R H×W×2 Features, f b Used to correct the distorted input image I d In this context, the learnable embedding is called a correction query, and each correction query is responsible for correcting the corresponding region in the input image.

[0078] The corrective decoder consists of three decoding blocks, each employing an improved self-attention module with two-dimensional positional encoding. For the first decoding block, a learnable embedding is used. and the feature map extracted from the third coding block As input. For the second decoding block, the output of the first decoding block and... As input, the third decoding block combines the output of the second decoding block with... As input, the output of the first and second blocks is upsampled by a factor of two based on bilinear interpolation when feeding the output of the first block to the next. Each decoder layer consists of a multi-head self-attention module, an encoder-decoder multi-head attention module, and a feedforward network. The formulas in each layer are shown below:

[0079] D0 = D l +P d

[0080] D′ i-1 =CPosE(LN(D i-1 ))+LN(D i-1 )

[0081] D” i-1 =MA(D′) i-1 ,D′ i-1 )+ResScale(D′ i-1 ), i = {1, ..., 6}

[0082] D′ i =MA(D”) i-1 E k )+ResScale(D” i-1 ), i = {1, ..., 6}, k = {2, 4, 6}

[0083] D i =FFN(LN(D′) i ))+ResScale(D′ i ), i = {1, ..., 6}

[0084] Where D i P represents the output feature of the i-th decoder layer. d E represents the learnable embedding vector (i.e., the learnable positional embedding). k This represents the feature representation from the corresponding encoder block.

[0085] In this embodiment, the correction decoder includes three decoding blocks, each employing an improved self-attention module with two-dimensional positional encoding. Upsampling modules connect adjacent decoding blocks. The coded feature maps at these three levels are input into the correction decoder of the image correction model for processing, outputting a decoded feature map, thus enabling the processing of deformed images from multiple scale features.

[0086] As a specific implementation, the step of inputting the three-level encoded feature maps into the correction decoder of the image correction model for processing to obtain the decoded feature map output by the correction decoder can be optimized, including:

[0087] a2) Input the third encoded feature map and a set number of learnable embedding vectors into the first decoding block of the image correction model for decoding processing, and output the first decoded feature map.

[0088] Learnable embeddings are a method that generates embedding vectors by dynamically adjusting parameters. They are primarily used to handle the numerical representation of unstructured data (such as text and images). The core idea is to allow the model to automatically learn the inherent structure of the data, thereby generating more accurate vector representations. During the model training phase, the parameters of the learnable embeddings are adjusted; during the model prediction phase, the iterated parameters are used directly.

[0089] In this embodiment, the three improved self-attention modules of the distortion encoder, which are sequentially connected to the output predictor, are denoted as the third decoding block, the second decoding block, and the first decoding block, respectively. Specifically, the learnable embedding vector and the third encoded feature map output by the third encoding block are input into the first decoding block of the correction decoder of the image correction model for decoding processing, and the output decoded feature map is denoted as the first decoded feature map. For example, continue to refer to... Figure 2 Learnable embeddings and the third coded feature map output by the third coded block The input is fed into the first decoding block (Decoder Block 1) for decoding, and the first decoded feature map is output.

[0090] b2) The first decoded feature map is upsampled by the first upsampling module in the correction decoder, and the upsampled decoded feature map and the second encoded feature map are input to the second decoding block in the correction decoder for decoding, and the second decoded feature map is output.

[0091] In this embodiment, the upsampling module between the first decoding block and the second decoding block is denoted as the first upsampling module. The upsampling module between the second decoding block and the third decoding block is denoted as the second upsampling module. Both the first and second upsampling modules perform a 2x upsampling. Specifically, the first decoded feature map output by the first decoding block is processed by the first upsampling module and then compared with the second encoded feature map output by the second encoding block. Figure 1 The input is taken as the first element and fed into the second decoding block for decoding processing. The decoded feature map output by the second decoding block is denoted as the second decoded feature map. For example, continue to refer to... Figure 2 The output of the first decoded feature map after processing by the first upsampling module, and the output of the second coded block. Together they are output to the second decoding block (Decoder Block 2) for decoding processing, and the second decoded feature map is output.

[0092] c2) The second decoded feature map is upsampled by the second upsampling module in the correction decoder, and the upsampled decoded feature map and the first encoded feature map are input into the third decoding block in the correction decoder for decoding, and the third decoded feature map is output.

[0093] Specifically, the second decoded feature map output from the second decoding block is processed by the second upsampling module, and then compared with the first encoded feature map output from the first encoding block. Figure 1 The input is taken as the first element and fed into the third decoding block for decoding processing. The decoded feature map output by the third decoding block is denoted as the third decoded feature map. For example, continue to refer to... Figure 2 The output of the second decoded feature map after processing by the second upsampling module is compared with the output of the first coded block. Together they are output to the second decoding block (Decoder Block 3) for decoding processing, and the third decoded feature map is output.

[0094] d2) Use the third decoding feature map as the decoding feature map output by the correction decoder.

[0095] The above technical solution specifies the steps for decoding the features output by the distortion encoder through the correction decoder of the image correction model, providing a basis for outputting the predicted pixel displacement field.

[0096] S205. The decoded feature map is input into the output predictor of the image correction model for processing, and the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image is output.

[0097] In this embodiment, the output predictor of the image correction model uses convolutional layers, weight matrices, and a softmax activation function to generate the pixel displacement field. The convolutional layers are used to locate local deformation regions, such as text distortion caused by folding; the weight matrix is ​​used to combine prior knowledge of the check template, such as fixing field positions, to optimize the displacement field prediction; the softmax function normalizes the displacement field output, ensuring that the corrected check image maintains geometric continuity and avoids breaks in the amount numerals. The softmax function is a commonly used activation function, primarily used to convert each element of the input tensor into a probability value between 0 and 1, with the sum of these probabilities being 1. It is typically used in the last layer of multi-class classification problems to output the probability of each class.

[0098] In this embodiment, the output predictor (flow head) is given the decoded features. This embodiment introduces a learnable upsampling module, namely a predictor output, to predict the pixel displacement field f. b ∈R H×W×2 To correct distorted images Id .

[0099] As a specific implementation, the step of inputting the decoded feature map into the output predictor of the image correction model for processing to obtain the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image can be optimized, including:

[0100] a3) Input the decoded feature map into the output predictor of the image correction model, and process it through two convolutional layers of the output predictor to generate an initial pixel displacement field.

[0101] Convolutional layers are used to locate localized deformation regions, such as text distortion caused by folding. Specifically, two convolutional layers are first applied to decode the feature map. And generate a coarse twisted flow pixel displacement field Let this be denoted as the initial pixel displacement field.

[0102] b3) The decoded feature map is processed through the other two convolutional layers of the output predictor to obtain the output feature map, and the size of the output feature map is adjusted to obtain the resized output feature map.

[0103] Then, two more convolutional layers are used to process D6, and the output feature map size is adjusted to...

[0104] c3) The activation function is used to solve the last layer of the resized output feature map to obtain the weight matrix.

[0105] The weight matrix is ​​used to incorporate prior knowledge of the check template, such as fixed field positions, to optimize displacement field prediction. Specifically, Softmax is performed on the last dimension of feature map D6 to obtain the weight matrix.

[0106] d3) The weight matrix is ​​used to perform a weighted summation on the set neighborhood of each pixel in the initial pixel displacement field to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

[0107] In this embodiment, the Softmax function is used to normalize the displacement field output, ensuring that the corrected check image maintains geometric continuity and avoiding breaks in the amount digits. Using the obtained weight matrix, in the initial pixel displacement field f... m Weighted combinations are performed on the 3×3 neighborhood of each pixel, which is equivalent to smoothing and preventing gradient explosion. Finally, the obtained size is... The feature map is adjusted to the final pixel displacement field f b ∈R H×W×2 Let be the target pixel displacement field.

[0108] The above technical solution specifies the steps for determining the target pixel displacement field to establish the pixel correspondence between the deformed check image and the corrected check image. A convolutional layer is used to locate the local deformation region; a weight matrix is ​​used to optimize the displacement field prediction by incorporating prior knowledge of the check template; and a softmax function is used to normalize the displacement field output, ensuring that the corrected check image maintains geometric continuity and avoiding breaks in the amount digits. The determination of the target pixel displacement field provides a basis for subsequent image correction.

[0109] S206. Based on the target pixel displacement field, perform a bilinear interpolation distortion operation on the original deformed check image to obtain a corrected check image after correcting the original deformed check image.

[0110] The aforementioned technical solution details the steps involved in processing deformed check images using the backbone network, distortion encoder, correction decoder, and output predictor within the image correction model. It achieves automated, high-quality automatic correction of deformed check images, reducing labor costs, improving correction accuracy, and enhancing business efficiency. It eliminates hardware dependence, supporting non-standardized image inputs such as handheld shooting, low-light conditions, and tilted perspectives, without requiring 3D scanners or multi-view shooting, significantly lowering the barrier to entry and making it suitable for individual users and bank branch business scenarios. Simultaneously, it overcomes boundary dependency limitations by using Transformer for image encoding and decoding. Leveraging the Transformer's global context modeling capabilities, it can infer text boundaries in missing regions. It supports the correction and recognition of incomplete check images; furthermore, the multi-head self-attention mechanism within the Transformer can directly model the spatial relationships of text regions without relying on check edge features, supporting the localization of text regions without clear boundaries and adapting to complex geometric deformations.

[0111] Example 3

[0112] Figure 6 This is a schematic diagram of a check image correction device provided in Embodiment 3 of the present invention. This device is applicable to situations requiring correction of deformed check images. The check image correction device can be implemented in hardware and / or software and is generally integrated into electronic devices. For example... Figure 6 As shown, the device includes: an image acquisition module 31, a displacement field determination module 32, and an image correction module 33, wherein,

[0113] Image acquisition module 31 is used to acquire the original deformed check image;

[0114] The displacement field determination module 32 is used to process the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

[0115] The image correction module 33 is used to perform a bilinear interpolation distortion operation on the original deformed check image according to the target pixel displacement field to obtain a corrected check image after correcting the original deformed check image.

[0116] The above technical solution first encodes the original deformed check image, then learns the encoded feature map, and generates a corresponding pixel displacement field by the correction decoder. Finally, using the predicted pixel displacement field, pixels are sampled from the original deformed check image to reconstruct the corrected check image. This achieves automated, high-quality automatic correction of deformed check images, reduces labor costs, improves correction accuracy, and enhances business efficiency.

[0117] Optionally, the displacement field determination module 32 includes:

[0118] The backbone extraction unit is used to input the original deformed check image into the backbone network of the image correction model for feature extraction to obtain the initial feature map of the original deformed check image;

[0119] The feature encoding unit is used to input the initial feature map into the distortion encoder of the image correction model for multi-level feature extraction and output a three-level encoded feature map.

[0120] The feature decoding unit is used to input the encoded feature maps of the three levels into the correction decoder of the image correction model for processing and output the decoded feature map.

[0121] The displacement field determination unit is used to input the decoded feature map into the output predictor of the image correction model for processing, and output the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image.

[0122] Optionally, the backbone network includes 6 residual blocks, and downsampling is performed once for every 2 residual blocks.

[0123] Optionally, the distortion encoder includes three coding blocks, each coding block employing an improved self-attention module with two-dimensional position coding, and a downsampling module is connected between adjacent coding blocks;

[0124] The feature coding unit is specifically used for:

[0125] The initial feature map is input into the first coding block of the distortion encoder of the image correction model for encoding processing, and the first coded feature map is output.

[0126] The first encoded feature map is downsampled by the first downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the second encoding block of the distortion encoder for encoding processing, and the second encoded feature map is output.

[0127] The second encoded feature map is downsampled by the second downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the third encoding block of the distortion encoder for encoding processing, and the third encoded feature map is output.

[0128] The first coded feature map, the second coded feature map, and the third coded feature map are used as the three levels of coded feature maps output by the distortion encoder.

[0129] Optionally, the correction decoder includes three decoding blocks, each employing an improved self-attention module with two-dimensional position coding, and an upsampling module is connected between adjacent decoding blocks;

[0130] The feature decoding unit is specifically used for:

[0131] The third encoded feature map and a set number of learnable embedding vectors are input into the first decoding block of the image correction model's correction decoder for decoding processing, and the first decoded feature map is output.

[0132] The first decoded feature map is upsampled by the first upsampling module in the corrective decoder. The upsampled decoded feature map and the second encoded feature map are then input into the second decoding block in the corrective decoder for decoding, and the second decoded feature map is output.

[0133] The second decoded feature map is upsampled by the second upsampling module in the corrective decoder. The upsampled decoded feature map and the first encoded feature map are then input into the third decoding block in the corrective decoder for decoding, and the third decoded feature map is output.

[0134] The third decoding feature map is used as the decoding feature map output by the correction decoder.

[0135] Optionally, the improved self-attention module includes: a normalization layer, a two-dimensional position encoding module, a multi-head self-attention module, a learnable residual connection, and a feedforward network.

[0136] Optionally, the displacement field determining element is specifically used for:

[0137] The decoded feature map is input into the output predictor of the image correction model, and processed by the two convolutional layers of the output predictor to generate an initial pixel displacement field.

[0138] The decoded feature map is processed through two more convolutional layers of the output predictor to obtain an output feature map, and the size of the output feature map is adjusted to obtain a resized output feature map.

[0139] The weight matrix is ​​obtained by solving the last layer of the resized output feature map using an activation function;

[0140] The weight matrix is ​​used to perform a weighted summation on a set neighborhood of each pixel in the initial pixel displacement field to obtain a target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

[0141] Optionally, the device also includes a slicing module, which, before processing the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder, and output predictor in the image correction model, is used to:

[0142] The original deformed check image is sliced ​​to obtain processed deformed check image slices, which are then input into the image correction model.

[0143] The check image correction device provided in the embodiments of the present invention can execute the check image correction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0144] Example 4

[0145] Figure 7 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0146] like Figure 7As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded into the RAM 43 from storage unit 48. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0147] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0148] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as the check image correction method.

[0149] In some embodiments, the check image correction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the check image correction method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the check image correction method by any other suitable means (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0155] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0156] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the check image correction method as provided in any embodiment of this invention.

[0157] In implementing a computer program product, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for correcting a check image, characterized in that, include: Obtain the original image of the deformed check; The original deformed check image is processed sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image. Based on the target pixel displacement field, a bilinear interpolation distortion operation is performed on the original deformed check image to obtain a corrected check image after correcting the original deformed check image.

2. The method according to claim 1, characterized in that, The process involves inputting the original deformed check image into a pre-trained image correction model, which then processes the image through the backbone network, distortion encoder, correction decoder, and output predictor in sequence to obtain a target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image. This includes: The original deformed check image is input into the backbone network of the image correction model for feature extraction to obtain the initial feature map of the original deformed check image; The initial feature map is input into the distortion encoder of the image correction model for multi-level feature extraction, and three levels of encoded feature maps are output. The encoded feature maps of the three levels are input into the correction decoder of the image correction model for processing, and the decoded feature maps are output. The decoded feature map is input into the output predictor of the image correction model for processing, and the target pixel displacement field representing the pixel correspondence between the original deformed check image and the corrected check image is output.

3. The method according to claim 2, characterized in that, The backbone network consists of 6 residual blocks, and downsampling is performed once for every 2 residual blocks.

4. The method according to claim 2, characterized in that, The distortion encoder includes three coding blocks, each employing an improved self-attention module with two-dimensional position coding, and a downsampling module is connected between adjacent coding blocks; The initial feature map is input into the distortion encoder of the image correction model for multi-level feature extraction, resulting in three levels of encoded feature maps output by the distortion encoder, including: The initial feature map is input into the first coding block of the distortion encoder of the image correction model for encoding processing, and the first coded feature map is output. The first encoded feature map is downsampled by the first downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the second encoding block of the distortion encoder for encoding processing, and the second encoded feature map is output. The second encoded feature map is downsampled by the second downsampling module in the distortion encoder, and the downsampled encoded feature map is input into the third encoding block of the distortion encoder for encoding processing, and the third encoded feature map is output. The first coded feature map, the second coded feature map, and the third coded feature map are used as the three levels of coded feature maps output by the distortion encoder.

5. The method according to claim 4, characterized in that, The correction decoder includes three decoding blocks, each employing an improved self-attention module with two-dimensional positional encoding, and an upsampling module is connected between adjacent decoding blocks; Specifically, the three levels of encoded feature maps are input into the correction decoder of the image correction model for processing to obtain the decoded feature map output by the correction decoder, including: The third encoded feature map and a set number of learnable embedding vectors are input into the first decoding block of the image correction model's correction decoder for decoding processing, and the first decoded feature map is output. The first decoded feature map is upsampled by the first upsampling module in the corrective decoder. The upsampled decoded feature map and the second encoded feature map are then input into the second decoding block in the corrective decoder for decoding, and the second decoded feature map is output. The second decoded feature map is upsampled by the second upsampling module in the corrective decoder. The upsampled decoded feature map and the first encoded feature map are then input into the third decoding block in the corrective decoder for decoding, and the third decoded feature map is output. The third decoding feature map is used as the decoding feature map output by the correction decoder.

6. The method according to claim 4 or 5, characterized in that, The improved self-attention module includes: a normalization layer, a two-dimensional position encoding module, a multi-head self-attention module, learnable residual connections, and a feedforward network.

7. The method according to claim 2, characterized in that, The step of inputting the decoded feature map into the output predictor of the image correction model for processing to obtain the target pixel displacement field representing the pixel correspondence from the original deformed check image to the corrected check image includes: The decoded feature map is input into the output predictor of the image correction model, and processed by the two convolutional layers of the output predictor to generate an initial pixel displacement field. The decoded feature map is processed through two more convolutional layers of the output predictor to obtain an output feature map, and the size of the output feature map is adjusted to obtain a resized output feature map. The weight matrix is ​​obtained by solving the last layer of the resized output feature map using an activation function; The weight matrix is ​​used to perform a weighted summation on a set neighborhood of each pixel in the initial pixel displacement field to obtain a target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image.

8. The method according to claim 1, characterized in that, Before processing the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder, and output predictor in the image correction model, the following steps are also included: The original deformed check image is sliced ​​to obtain processed deformed check image slices, which are then input into the image correction model.

9. A check image correction device, characterized in that, include: The image acquisition module is used to acquire the original image of the deformed check; The displacement field determination module is used to process the original deformed check image sequentially through the backbone network, distortion encoder, correction decoder and output predictor in the image correction model to obtain the target pixel displacement field that represents the pixel correspondence between the original deformed check image and the corrected check image. The image correction module is used to perform a bilinear interpolation distortion operation on the original deformed check image based on the target pixel displacement field to obtain a corrected check image after correcting the original deformed check image.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the check image correction method as described in any one of claims 1-8.

Images