Progressive non-uniform illumination text image correction method and device based on size change, equipment and medium

Abstract

The invention relates to a progressive non-uniform illumination correction method based on size change. A correction model comprises a small-size non-uniform illumination field learning network used for learning a rough non-uniform illumination field; the original-size fine illumination field correction network is used for learning a precise non-uniform illumination field and outputting a corrected image; the downsampling module is used for reducing the size of the image; the multi-head attention module is used for extracting global information from the down-sampled image; the multi-scale feature up-sampling module is used for carrying out up-sampling on features in a plurality of branches; the adaptive feature fusion module is used for fusing features of the small-size non-uniform illumination field learning network and the original-size fine illumination field correction network; according to the method, the uneven illumination field is gradually learned and corrected in different stages by utilizing progressive network processing, so that the correction process is finer.

Description

Technical Field

[0001] The technology belongs to the field of computer vision and image processing, and more specifically, it relates to a method, device, electronic device and computer storage medium for progressive non-uniform illumination text image correction based on size change. Background Art

[0002] With the rapid development of optical technology, the impact of non-uniform illumination on fields such as text recognition and natural language processing has gradually become prominent. In practical applications, text scenes are often affected by various lighting factors such as uneven light intensity and shadow projection, which causes the text in the image to appear different in brightness and darkness. This non-uniform illumination phenomenon seriously challenges the accuracy and stability of text recognition. Under complex and changing lighting conditions, existing text recognition systems often find it difficult to accurately capture the details of the text, resulting in incorrect recognition results and low-reliability text output. To address this problem, researchers are committed to developing innovative text correction technologies to eliminate or reduce the impact of non-uniform illumination on text recognition. These technologies aim to perform brightness equalization, contrast enhancement and other operations on the text area in the image, so that the text can present a clear and consistent appearance under various lighting conditions.

[0003] However, non-uniform illumination in different scenes shows diversity, so it is necessary to design flexible correction strategies for different illumination modes and text styles. In addition, text correction methods also need to maintain efficiency and stability when processing large-scale and diverse data.

[0004] In general, text correction technology plays a key role in text recognition under uneven lighting conditions. By improving the accuracy and robustness of text recognition, these methods are expected to play an important role in document scanning, autonomous driving, mobile applications and other fields, providing users with a better text recognition and understanding experience. Summary of the invention

[0005] In view of this, it is necessary to provide a method, device, electronic device and computer storage medium for progressive non-uniform illumination text image correction based on size change, so as to solve the problem that it is difficult to effectively remove non-uniform illumination in the prior art.

[0006] In order to solve the above problems, in a first aspect, the present invention provides a method for correcting text images with progressive non-uniform illumination based on size change, comprising the following steps.

[0007] Construct training and test sets of simulated degraded images of text images with non-uniform illumination.

[0008] The training set images are input into the size-changing progressive correction network model for training until the loss function of the size-changing progressive correction network model reaches a preset convergence condition, wherein the structure of the size-changing progressive correction network model includes a small-size non-uniform illumination field learning network S1 and an original-size fine illumination field correction network S2, wherein the small-size non-uniform illumination field learning network S1 includes a downsampling module, a multi-head attention module, and a multi-scale feature upsampling module. The original-size fine illumination field correction network S2 includes a multi-head attention module and an adaptive feature fusion module.

[0009] The test set images are input into the trained size-changing progressive network model and the test rectified images are output.

[0010] Furthermore, the construction of a training set and a test set of non-uniformly illuminated text image simulated degraded images includes.

[0011] Get the text image set.

[0012] Light, medium and heavy non-uniform illuminations of different illumination directions and illumination types are added to the text images in the text image set to form low, medium and high non-uniform illumination text image simulation degradation images.

[0013] The text image simulated degraded image and the clear image are spliced, and the spliced ​​image is cropped and normalized to obtain a training set and a test set of the text image simulated degraded image.

[0014] The method of constructing a training set of text image simulation degraded images also includes.

[0015] The low, medium and high text image simulated degraded images in the training set are augmented to obtain the final training set, and the proportion of low, medium and high text image simulated degraded images increases progressively.

[0016] Furthermore, the downsampling module of the input of the small-size non-uniform illumination field learning network S1 adopts a bilinear interpolation downsampling method.

[0017] It can be understood that the size of the image is reduced by the downsampling module, thereby reducing the amount of calculation for subsequent rough calculation of the uneven illumination field.

[0018] Furthermore, the multi-head attention module in the small-size non-uniform illumination field learning network includes a normalization layer, a multi-head attention layer and a gated feedforward layer.

[0019] It can be understood that the multi-head attention module extracts feature maps of smaller size and original size containing global information respectively.

[0020] Furthermore, the small-size non-uniform illumination field learning network includes three stages, each stage is composed of a three-scale UNet, each scale is composed of two ResBlock blocks, and multi-supervised L1 content loss constraints and frequency domain loss constraints are added after each UNet.

[0021] The UNet network is composed of a downsampling path and an upsampling path. The downsampling path includes convolution blocks and maximum pooling layers for extracting features of different scales and reducing the size. The upsampling path (decoder) includes upsampling layers and skip connections for gradually restoring the size of the feature map to the original size and integrating multi-scale feature information.

[0022] The ResBlock is a residual block that usually contains multiple convolutional layers, activation functions, and skip connections, which helps capture more complex features and speeds up the training process.

[0023] The multi-supervised L1 constraint is about subtracting the uneven illumination field learned by the network in each stage from the degraded text image to obtain a corrected image, and the L1 content loss constraint and frequency domain loss constraint of the clear image.

[0024] It can be understood that a smaller-scale rough non-uniform illumination field is extracted through the small-scale non-uniform illumination field learning network.

[0025] Furthermore, the multi-scale feature upsampling module extracts spatial information of two different scales, small size and original size, through multiple branches to achieve upsampling of input features.

[0026] It can be understood that the multi-scale feature upsampling module not only takes into account the spatial and channel factors, but also retains the multi-scale information of the original features, and outputs the upsampled small-size non-uniform illumination field to the original-size fine illumination field correction network and the adaptive feature fusion module respectively.

[0027] Furthermore, the structure of the original-size fine illumination field correction network is the same as that of the small-size non-uniform illumination field learning network, but the input is different. The input of the original-size fine illumination field correction network is an upsampled image of the output of the small-size non-uniform illumination field learning network.

[0028] The accurate non-uniform illumination field is learned through the original scale fine illumination field correction network.

[0029] Furthermore, the adaptive feature fusion module adaptively fuses and redistributes the output features of the two sub-networks S1 and S2 to efficiently utilize the output features of the two sub-networks of different scales.

[0030] It can be understood that the features of the small-size uneven illumination field learning network and the original-size fine illumination field correction network are fused through the adaptive feature fusion module to generate a corrected image.

[0031] Furthermore, the training process of the size-changing progressive correction network model includes.

[0032] Downsample the size of the text-degraded image to a smaller size.

[0033] A multi-head attention module is used to extract smaller feature maps containing global information.

[0034] Learning a coarse inhomogeneous illumination field via a small-scale inhomogeneous illumination field learning network.

[0035] A multi-scale feature upsampling module is used to upsample the coarse inhomogeneous illumination field to the original size.

[0036] A multi-head attention module is used to extract feature maps of the original size containing global information.

[0037] The upsampled result is input into the original-size fine illumination field correction network to progressively learn an accurate uneven illumination field.

[0038] An adaptive feature fusion module is used to fuse the features of the first sub-network S1 and the second sub-network S2, two non-uniform illumination fields of different scales, and output the corrected image during the training process.

[0039] The Adam algorithm is used to train the size-changing progressive correction network model, set training parameters, and adopt multi-supervised L1 content loss constraints and frequency domain loss constraints.

[0040] The weights of the initial progressive network model are iteratively updated until the loss function reaches a preset convergence condition.

[0041] In a second aspect, the present invention further provides a device for correcting text images with progressive non-uniform illumination based on size change, comprising:

[0042] The dataset construction module constructs the training set and test set of non-uniform illumination text image simulation degradation images.

[0043] The model training module is used to input the training set images into the progressive correction network model with size changes for training until the loss function of the progressive correction network model with size changes reaches a preset convergence condition, wherein the structure of the progressive correction network model with size changes includes a small-size non-uniform illumination field learning network S1 and an original-size fine illumination field correction network S2, wherein the small-size non-uniform illumination field learning network S1 includes a downsampling module, a multi-head attention module, and a multi-scale feature upsampling module. The original-size fine illumination field correction network S2 includes a multi-head attention module and an adaptive feature fusion module.

[0044] The model testing module is used to input the test set images into the trained size-changing progressive network model and output the test correction images.

[0045] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the above-mentioned progressive non-uniform illumination text image correction method based on size change are implemented.

[0046] In a fourth aspect, the present invention further provides a computer storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned progressive non-uniform illumination text image correction method based on size change are implemented.

[0047] The beneficial effect of adopting the above embodiment is: the present invention trains a progressive network model based on size change based on the training set of non-uniform illumination text image simulation degraded images, and after the model training is completed, the non-uniform illumination effect in the text image degraded image can be removed. Then the test set image is input into the trained progressive network model based on size change, and the features of the non-uniform illumination field of the degraded image are extracted through the small-size non-uniform illumination field learning network, while suppressing certain noise; then the features are input into the original-size fine illumination field correction network to quickly extract richer non-uniform illumination field features and output the test correction image. The present invention can effectively correct the non-uniform illumination text image simulation degraded image to obtain a clear image. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention is a flowchart of an embodiment of a method for progressive non-uniform illumination text image correction based on size change provided by the present invention.

[0049] Figure 2 A multi-head attention module is provided for one embodiment of the present invention.

[0050] Figure 3 A multi-scale upsampling module is provided in one embodiment of the present invention.

[0051] Figure 4 An adaptive feature fusion module is provided for an embodiment of the present invention.

[0052] Figure 5 A flowchart of a training process of a progressive non-uniform network model based on size change is provided in accordance with an embodiment of the present invention.

[0053] Figure 6 A schematic diagram of the structure of a small-size non-uniform illumination field learning network and an original-size fine illumination field correction network provided in one embodiment of the present invention.

[0054] Figure 7 The present invention is a schematic structural diagram of an embodiment of a device for progressive non-uniform illumination correction of text images provided by the present invention.

[0055] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically limited. Reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] The present invention provides a method, device, equipment and medium for progressive non-uniform illumination text image correction based on size change, which are described below respectively.

[0059] See also Figure 1 , Figure 1 A flow chart of an embodiment of a progressive non-uniform illumination text image correction method based on size change provided by the present invention is provided. A specific embodiment of the present invention discloses an image denoising and deblurring method, including:

[0060] Step S101: constructing a training set and a test set of simulated degraded images of text images with non-uniform illumination.

[0061] Among them, during the imaging process of text image degradation, factors such as lens shape, refraction and scattering of light, aperture and shutter speed, backlight and strong light source will affect the distribution of light on the lens and sensor, thereby affecting the final image quality.

[0062] Therefore, it is necessary to perform image correction operation on the degraded text image. Before performing image correction on the degraded text image, it is necessary to construct a training set and a test set of simulated degraded text image images. It can be understood that the training set and the test set of simulated degraded text image images include image pairs consisting of degraded images and clear images.

[0063] In one embodiment of the present invention, a training set and a test set of text image simulated degraded images are constructed, including.

[0064] Get the text image set.

[0065] Light, medium and heavy non-uniform illuminations of different illumination directions and illumination types are added to the text images in the text image set to form text image simulated degradation images of three degrees: low, medium and high.

[0066] The text image simulated degraded image and the clear image are spliced, and the spliced ​​image is cropped and normalized to obtain a training set and a test set of the text image simulated degraded image.

[0067] The method of constructing a training set of text image simulation degraded images also includes.

[0068] The low, medium and high text image simulated degraded images in the training set are augmented to obtain the final training set, and the proportion of low, medium and high text image simulated degraded images increases progressively.

[0069] Optionally, the text images may adopt the non-uniform illumination dataset in the SynthText dataset, which includes various illumination conditions, including uniform and non-uniform illumination. The training set and test set of text image simulation degradation images may be produced based on the text images in the SynthText dataset.

[0070] By setting different parameters, images with different degrees of degradation can be obtained. After obtaining the simulated degraded image of the text image, the degraded image can be spliced ​​with the clear image to form a degraded-clear image pair, and 700 image pairs are randomly selected as the training set, and the remaining 300 image pairs are used as the test set.

[0071] Furthermore, it is understandable that in order to achieve the purpose of fast training, the image pairs can be cropped into the same size, such as 256×256, and normalized to [-1,1]; in addition, in order to improve the effect of network training, it is necessary to expand the number of training sets and perform data augmentation on the training sets. Specifically, data similar to but different from the original training set can be randomly generated through operations such as horizontal / vertical flipping and rotation. The low, medium, and high non-uniform lighting simulation degraded images in the training set can also be augmented to obtain the final training set, and the proportion of the number of low, medium, and high non-uniform lighting simulation degraded images increases progressively.

[0072] Step S102: input the training set images into the size-changing progressive correction network model for training until the loss function of the size-changing progressive correction network model reaches a preset convergence condition, wherein the structure of the size-changing progressive correction network model includes a small-size non-uniform illumination field learning network and an original-size fine illumination field correction network.

[0073] After the training set and test set of non-uniform illumination text image simulated degraded images are constructed, the training set needs to be trained, so a size-changing progressive correction network model can be built. The network structure of the model includes two parts, namely, a small-size non-uniform illumination field learning network and an original-size fine illumination field correction network.

[0074] It should be noted that the small-scale non-uniform illumination field learning network consists of three stages, each of which is composed of a three-scale UNet, and each scale is composed of two ResBlock blocks, which can extract features of different scales of smaller-scale images and capture more complex features, and learn smaller-scale rough non-uniform illumination fields; the original-size fine illumination field correction network has the same structure as the non-uniform illumination field learning network, but it can further learn accurate non-uniform illumination fields and output corrected images.

[0075] In the process of training the size-changing progressive correction network model according to the training set, the parameters in the model can be iteratively modified until the loss function of the size-changing progressive correction network model reaches a preset convergence condition until the model training is completed.

[0076] Step S103: input the test set image into the trained network model and output the test corrected image.

[0077] It can be understood that after the model training is completed, the best fitting solution of a model can be obtained. At this time, the test set image is used as the input of the model, and the output test correction image is compared with the clear image. It can be seen that the test correction image can have a high-quality correction result.

[0078] The present invention is based on constructing a training set and a test set of non-uniformly illuminated text image simulation degraded images to train a progressive correction network model with size changes. After the model training is completed, the non-uniform illumination in the degraded text image can be removed. Then the test set image is input into the trained progressive correction network model with size changes, and the small-scale non-uniform illumination field learning network is used to extract the features of different scales of the smaller-scale image and capture more complex features, and learn the coarse non-uniform illumination field of the smaller scale; then the non-uniform illumination field learning network input is output to the original-scale fine illumination field correction network after multi-scale upsampling, which can further learn the precise non-uniform illumination field and output the test correction image. The present invention can effectively correct the degraded image of the non-uniformly illuminated text image to obtain a clear correction image.

[0079] In one embodiment of the present invention, the small-size non-uniform illumination field learning network includes three stages, each stage includes a 3-level UNet structure, and each level is composed of two ResBlock blocks.

[0080] It can be understood that the small-scale non-uniform illumination field learning network can learn a rough non-uniform illumination field from the input degraded image while being able to suppress noise to a certain extent.

[0081] Specifically, the non-uniform illumination field learning network consists of a 3-level UNet, each level consists of two ResBlock blocks, and the convolution kernel size of the convolution layer is 5×5; the residual block consists of two convolutions with a convolution kernel size of 3×3 and a ReLU activation function, and the number of convolution kernels is 64.

[0082] For example, given a degraded image Y of size H×W, it is downsampled to obtain an input of size H / 2×W / 2, and the degraded image is recorded as Y↓. Each stage of the S1 subnetwork obtains an output, which is then subtracted from Y↓ to obtain the output clear image of each stage. It can be seen that the estimated thermal radiation bias field is closer to the real thermal radiation bias field as the stage goes deeper, and the correction image is closer to the real clear image.

[0083] In one embodiment of the present invention, the structure of the original-size fine illumination field correction network is consistent with that of the small-size non-uniform illumination field learning network.

[0084] It can be understood that the original size fine illumination field correction network can learn a more accurate precise non-uniform illumination field from the input upsampled coarse non-uniform illumination field, while being able to suppress noise to a certain extent.

[0085] See also Figure 2 , Figure 2A multi-head attention module provided for an embodiment of the present invention includes components such as layer normalization, multi-head self-attention, a gated Feed-Forward module and a GELU activation function.

[0086] Specifically, we first use layer normalization to normalize the features and use multi-head self-attention MSA to calculate the weight of each position, thereby introducing global information and obtaining features containing more useful information. Then, we use a gated feed-forward module (GFM) to control the information flow, allowing useful information to propagate deeper. Given a feature , MFSFEM can be expressed as: .

[0087] Furthermore, GFM contains two parallel paths. Given a feature , first use Convolution ups the channel, and then uses depthwise convolution to encode information from spatially adjacent pixel locations. We implement gating by element-wise product of two parallel paths of linear transformation layers, one of which is activated by the GELU nonlinearity, and finally by a Convolution reduces the channels to the original input dimension. GFM can be expressed as: .

[0088] in represents the GELU activation function, , Respectively represent the first channel for ascending and descending convolution, represents the depthwise convolution, Represents element-wise multiplication.

[0089] In one embodiment of the present invention, see Figure 3 , Figure 3 A multi-scale upsampling module provided in an embodiment of the present invention extracts spatial information of different scales through multiple branches to achieve upsampling of input features. The method includes:

[0090] Multi-branch spatial information extraction, targeting input features In a parallel way, each branch uses convolution kernels of different sizes to capture spatial information of different scales. In order to reduce the number of parameters, grouped convolution is used, and the convolution kernel sizes and grouping settings of the four branches are 3, 5, 7, 9 and 1, 4, 5, 10 respectively.

[0091] Channel shuffling enhances information interaction. The channel shuffling mechanism is used to enhance the information interaction between different branches, which helps to integrate multi-scale feature information.

[0092] Pixel shuffle upsampling, perform pixel shuffle upsampling on the features of each branch to obtain the upsampling result on each branch: .

[0093] Multi-scale feature stacking: stacking upsampled features of different branches in the channel dimension to form multi-scale upsampled information , .

[0094] The SE attention module uses the channel attention module to obtain a Attention: .

[0095] Channel shuffling enhances interaction. Channel shuffling is used to further enhance the information interaction between multi-scale features to obtain the final upsampled features. : .

[0096] Where S stands for channel-shuffle, Represents element-wise multiplication.

[0097] The multi-scale feature upsampling module not only takes into account the spatial and channel factors, but also retains the multi-scale information of the original features, and outputs the upsampled small-size non-uniform illumination field to the original-size fine illumination field correction network and the adaptive feature fusion module respectively.

[0098] In one embodiment of the present invention, see Figure 4 , Figure 4 An embodiment of the present invention provides an adaptive feature fusion module that adaptively fuses and redistributes the output features of the two sub-networks S1 and S2 to efficiently utilize the output features of the two sub-networks at different scales, including:

[0099] Perform element-wise addition on the two features L1 and L2 to obtain the feature L=L1+L2.

[0100] Perform a GAP operation on feature L to obtain a statistic : .

[0101] Using a descending channel Convolution processes the pooled features to obtain compressed feature representations .

[0102] Use a multi-layer perceptron (MLP) to map the compressed features to a higher-dimensional feature space to obtain .

[0103] Using two parallel Convolution combined with softmax activation to obtain the attention weight map corresponding to the two features and

[0104] .

[0105] Apply the attention weight map to the two features separately, recalibrate and add them together to obtain the adaptively fused feature map .

[0106] In one embodiment of the present invention, see Figure 5 , Figure 5 A flowchart of a training process of a size-changing progressive correction network model provided by an embodiment of the present invention includes:

[0107] Step S501: down-sample the size of the text-degraded image to a smaller size.

[0108] Step S502: Use a multi-head attention module to extract a smaller feature map containing global information.

[0109] Step S503: learning a rough non-uniform illumination field through a small-size non-uniform illumination field learning network.

[0110] Step S504: using a multi-scale feature upsampling module to upsample the rough uneven illumination field to the original size.

[0111] Step S505: Use a multi-head attention module to extract a feature map of the original size containing global information.

[0112] Step S506: The up-sampled result is input into the original size fine illumination field correction network to progressively learn the accurate uneven illumination field.

[0113] Step S507: Adopting an adaptive feature fusion module, the features of the first sub-network S1 and the features of the second sub-network S2, which are two features of non-uniform illumination fields of different scales, are fused to output a corrected image in the training process.

[0114] Step S508: using the Adam algorithm to train the size-changing progressive correction network model, setting training parameters, and adopting multi-supervised L1 content loss constraints and frequency domain loss constraints.

[0115] Step S509: iteratively updating the weights of the size-changing progressive correction network model until the loss function reaches a preset convergence condition.

[0116] First of all, it should be noted that the Adam algorithm is used in the embodiment of the present invention to train the progressive correction network model with size changes, and the number of training epochs is 700. The initial learning rate of the network in the embodiment of the present invention is , when the training is halfway through, the learning rate decays linearly until , the batch size is 2.

[0117] For a better understanding of the present invention, please refer to Figure 6 , Figure 6 A schematic diagram of the structure of a small-size non-uniform illumination field learning network and an original-size fine illumination field correction network provided in one embodiment of the present invention.

[0118] The progressive correction network models of size change are small-size non-uniform illumination field learning network and original-size fine illumination field correction network. The small-size non-uniform illumination field learning network S1 includes a downsampling module, a multi-head attention module, and a multi-scale feature upsampling module. The original-size fine illumination field correction network S2 includes a multi-head attention module and an adaptive feature fusion module.

[0119] The training set images are downsampled through the downsampling module in the small-size non-uniform illumination field learning network, and then input into the small-size non-uniform illumination field learning network. The multi-head attention mechanism first extracts the features of the small-scale image, and then the coarse illumination field is learned through the three-scale UNet of the three stages of the small-size non-uniform illumination field learning network, where each scale consists of two ResBlock blocks, and then multi-supervised L1 content loss constraints and frequency domain loss constraints are added after each UNet to obtain the learned coarse illumination field. The coarse illumination field learned by the small-size non-uniform illumination field learning network is upsampled to its original size through the multi-scale upsampling module to obtain the coarse illumination field of the original size.

[0120] The structure of the original size fine illumination field correction network is consistent with the small size non-uniform illumination field learning network, but the input is different. The input is the rough illumination field of the original size obtained by multi-scale sampling. The multi-head attention mechanism first extracts the features of the original size image, and then the original size fine illumination field correction network learns the precise illumination field. Finally, the output features of the two sub-networks S1 and S2 are adaptively fused and redistributed through the adaptive feature fusion module to efficiently utilize the output features of these two different scales to obtain the final illumination field and output the final corrected image.

[0121] The embodiment of the present invention uses two types of loss functions to train SPNet: content loss and frequency reconstruction loss. Content loss is defined as:

[0122] Where GT is the ground-truth clear image and GT↓ is its downsampled version. is the normalization factor, which we set here. The frequency reconstruction loss restores the high-frequency details of the degraded image by minimizing the difference between the degraded image and the frequency domain ground-truth. The frequency reconstruction loss is defined as:

[0123] where is the Fourier transform. Finally we get the loss as where .

[0124] The training is completed by continuously adjusting the weights of the progressively corrected network model with varying sizes until the loss function reaches the preset convergence condition.

[0125] In order to better implement the progressive non-uniform illumination text image correction method based on size change in the embodiment of the present invention, based on the progressive non-uniform illumination text image correction method based on size change, correspondingly, please refer to Figure 7 , Figure 7 This is a schematic structural diagram of an embodiment of a non-uniform illumination text image correction device provided by the present invention. The embodiment of the present invention provides a non-uniform illumination text image correction device 700, comprising.

[0126] The data set construction module 701 is used to construct a training set and a test set of non-uniformly illuminated text image simulated degraded images.

[0127] The model training module 702 trains the initial progressive network model with the training set image input size change until the loss function of the initial progressive network model reaches the preset convergence condition, wherein the structure of the progressive network model includes a small-size non-uniform illumination field learning network S1 and an original-size fine illumination field correction network S2, wherein the small-size non-uniform illumination field learning network includes a downsampling module, a multi-head attention module, and a multi-scale feature upsampling module. The original-size fine illumination field correction network includes a multi-head attention module and an adaptive feature fusion module.

[0128] The model testing module 703 inputs the test set image into the trained size-changing progressive network model and outputs the test correction image.

[0129] It should be noted here that: the device 700 provided in the above embodiment can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above method embodiments, which will not be repeated here.

[0130] Based on the above-mentioned non-uniform illumination text image correction method, an embodiment of the present invention also provides an electronic device accordingly, including: a processor and a memory, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps in the non-uniform illumination degraded image denoising and deblurring method in the above-mentioned embodiments are implemented.

[0131] Figure 8 8 is a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present invention. The electronic device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0132] The electronic device includes: a memory and a processor, wherein the processor here can be referred to as the processing device 801 below, and the memory can include at least one of the read-only memory (ROM) 802, the random access memory (RAM) 803 and the storage device 808 below, as shown below.

[0133] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0134] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0135] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present invention are executed.

[0136] Based on the above-mentioned progressive non-uniform illumination text image correction method, an embodiment of the present invention also provides a computer-readable storage medium accordingly, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the non-uniform illumination text image correction method as described in the above-mentioned embodiments.

[0137] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0138] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A progressive non-uniform illumination text image correction method based on size change, characterized in that: include: Construct training and test sets of simulated degraded images of text images with non-uniform illumination; Input the training set images into the size-changing progressive correction network model for training until the loss function of the size-changing progressive correction network model reaches a preset convergence condition, wherein the structure of the size-changing progressive correction network model includes a small-size non-uniform illumination field learning network S1 and an original-size fine illumination field correction network S2, wherein the small-size non-uniform illumination field learning network S1 includes a downsampling module, a multi-head attention module, and a multi-scale feature upsampling module. The original-size fine illumination field correction network S2 includes a multi-head attention module and an adaptive feature fusion module; The test set images are input into the trained size-changing progressive network model and the test rectified images are output.

2. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The method of constructing a training set and a test set of non-uniformly illuminated text image simulated degraded images includes: Get the text image set; Adding light, medium and heavy non-uniform illumination of different illumination directions and different illumination types to the text images in the text image set to form low, medium and high levels of non-uniform illumination text image simulation degradation images; Splicing the simulated degraded image of the text image with the clear image, and cropping and normalizing the spliced ​​image to obtain a training set and a test set of the simulated degraded image of the text image; The step of constructing a training set of text image simulation degraded images also includes: The low, medium and high text image simulated degraded images in the training set are augmented to obtain the final training set, and the proportion of low, medium and high text image simulated degraded images increases progressively.

3. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The downsampling module of the input of the small-size non-uniform illumination field learning network S1 adopts a bilinear interpolation downsampling method; The downsampling module is used to reduce the size of the image, thereby reducing the amount of calculation for subsequent rough calculation of the uneven illumination field.

4. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The multi-head attention module in the small-size non-uniform illumination field learning network includes a normalization layer, a multi-head attention layer, and a gated feedforward layer; The multi-head attention module extracts feature maps of smaller size and original size containing global information respectively.

5. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The small-scale non-uniform illumination field learning network includes three stages, each of which is composed of a three-scale UNet, each scale is composed of two ResBlock blocks, and multi-supervisory L1 content loss constraints and frequency domain loss constraints are added after each UNet; The UNet network is composed of a downsampling path and an upsampling path. The downsampling path includes a convolution block and a maximum pooling layer for extracting features of different scales and reducing the size. The upsampling path includes an upsampling layer and a skip connection for gradually restoring the size of the feature map to the original size and integrating multi-scale feature information. The ResBlock is a residual block that usually contains multiple convolutional layers, activation functions, and skip connections, which helps capture more complex features and speed up the training process; The multi-supervisory L1 constraint is about the degraded text image minus the uneven illumination field learned by the network at each stage to obtain a corrected image, and the L1 content loss constraint and frequency domain loss constraint of the clear image; A coarse non-uniform illumination field of a smaller scale is extracted through the small-scale non-uniform illumination field learning network.

6. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The multi-scale feature upsampling module extracts spatial information of two different scales, small size and original size, through multiple branches to achieve upsampling of input features; The multi-scale feature upsampling module not only takes into account spatial and channel factors, but also retains the multi-scale information of the original features, and outputs the upsampled small-size non-uniform illumination field to the original-size fine illumination field correction network and the adaptive feature fusion module respectively.

7. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The structure of the original-size fine illumination field correction network is the same as that of the small-size non-uniform illumination field learning network, but the input is different. The input of the original-size fine illumination field correction network is an upsampled image of the output of the small-size non-uniform illumination field learning network. The accurate non-uniform illumination field is learned through the original scale fine illumination field correction network.

8. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The adaptive feature fusion module adaptively fuses and redistributes the output features of the two sub-networks S1 and S2 to efficiently utilize the output features of the two sub-networks at different scales; The adaptive feature fusion module is used to fuse the features of the small-size uneven illumination field learning network and the original-size fine illumination field correction network to generate a corrected image.

9. The method for correcting text images with progressive non-uniform illumination based on size change according to claim 1, characterized in that: The training process of the size-changing progressive correction network model includes: Downsample the size of the text-degraded image to a smaller size; A multi-head attention module is used to extract smaller feature maps containing global information; Learn a rough inhomogeneous illumination field through a small-scale inhomogeneous illumination field learning network; A multi-scale feature upsampling module is used to upsample the rough uneven illumination field to the original size; A multi-head attention module is used to extract feature maps of original size containing global information; The upsampled result is input into the original size fine illumination field correction network to progressively learn the accurate uneven illumination field; Adopting an adaptive feature fusion module, the features of the first sub-network S1 and the features of the second sub-network S2, two features of non-uniform illumination fields of different scales, are fused to output a corrected image during the training process; Using the Adam algorithm to train the size-changing progressive correction network model, setting training parameters, and adopting multi-supervised L1 content loss constraints and frequency domain loss constraints; The weights of the initial progressive network model are iteratively updated until the loss function reaches a preset convergence condition.

10. An electronic device, characterized in that: It includes a memory and a processor, wherein the memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the progressive non-uniform illumination correction method based on size change as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the progressive non-uniform illumination correction method based on size change described in any one of claims 1 to 9.

Images