A text image super-resolution method and system based on mask reconstruction paradigm
Through the method based on the mask reconstruction paradigm, large-size low-resolution text images are processed, which solves the problems of text reconstruction distortion, large memory usage and low computing efficiency, and realizes high-quality image reconstruction, which is suitable for a variety of application scenarios.
Patent Information
- Application Number
- CN202411055108.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-02
AI Technical Summary
The prior art has problems such as text reconstruction distortion, excessive video memory usage and low computing efficiency when processing large-size low-resolution text images, especially in real-time processing or high-resolution image scenes, and it is easy to occur during image cutting and recombination. Non-smoothing affects image quality.
Using the text image super-resolution method based on the mask reconstruction paradigm, by dividing large-size low-resolution text images into equal-sized image blocks, randomly mask and match them using sliding windows, combining backbone network and decoding module for features Extract and combine, use self-attention mechanism and CNN layer for smoothing processing, reduce video memory usage and improve image quality.
Significantly improves the efficiency and quality of large-size low-resolution text image processing, optimizes video memory usage, improves image reconstruction quality, especially high-quality reconstruction capabilities in text areas, overcomes unsmoothing, and is suitable for a wide range of application scenarios .
Smart Images

Figure CN118570066B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image reconstruction, and in particular to a text image super-resolution method and system based on a mask reconstruction paradigm. Background Art
[0002] Super-resolution reconstruction of text images is widely used in document scanning, historical document restoration, text recognition, autonomous driving, etc. This technology is used to reconstruct low-resolution text images with low resolution and blurred text in real scenes due to inaccurate focus and jitter of camera equipment.
[0003] Existing large-scale, low-quality images usually use image super-resolution models, which are not optimized for large-scale, low-resolution text images. Using general image super-resolution models, it is impossible to finely reconstruct the text area in the image, and the reconstructed text may even be distorted and twisted, resulting in the problem that it is still unrecognizable. Therefore, existing solutions for processing large-scale, low-resolution text images usually first obtain a small local image with text based on a detector, and then super-resolve the small local image through a text image super-resolution algorithm. This method can improve the quality of subsequent recognition, but it is difficult to obtain an accurate super-resolution text image based on a large-scale, low-resolution image.
[0004] On the other hand, when super-resolution processing is performed on large-size images, the computing resources required are quite large, especially the demand for video memory is extremely significant. This is because a large amount of image details need to be processed and stored during the processing to achieve the purpose of improving image quality. As the image size grows, the amount of computing required increases dramatically, and accordingly, the usage of video memory will also increase significantly. This challenge not only limits the size of images that can be processed, but also places high demands on computing hardware, especially in scenarios that pursue real-time processing or processing of high-resolution images. Summary of the invention
[0005] Based on the technical problems existing in the background technology, the present invention proposes a text image super-resolution method and system based on a mask reconstruction paradigm, an algorithm that reduces video memory usage while maintaining high image quality.
[0006] The present invention proposes a text image super-resolution method based on a mask reconstruction paradigm, which divides a large-sized low-resolution text image into equal-sized parts and inputs them into a trained image reconstruction model to output a super-resolution image corresponding to the low-resolution text image.
[0007] The training process of the image reconstruction model is as follows:
[0008] Step 1: Get a large low-resolution text image and cut it into equal-sized image blocks to form a matrix , randomly mask multiple image blocks according to a set ratio to obtain a covered image block;
[0009] Step 2: Matrix Each image block in is assigned a unique number, and a sliding window of a set shape is used to map the matrix Slide the image blocks in the window to obtain multiple window sets, each of which represents the image block coding set in the window. Determine whether each window set has an intersection with the covered image block set and the uncovered image block set. If so, save the intersection area in the dictionary , Represents an index, when , A dictionary representing the connected areas of uncovered image blocks. , A dictionary representing connected areas of covered image blocks, wherein the covered image block set is a set consisting of each covered image block, and the uncovered image block set is a set consisting of each uncovered image block;
[0010] Step 3: extracting features of image blocks in the set of uncovered image blocks based on the backbone network to obtain features of the uncovered image blocks;
[0011] Step 4: Based on the decoding module, the features of the covered image block and the uncovered image block are combined in the original order and then position encoded. Then, the predicted pixel block set of the covered image block is obtained based on the self-attention mechanism and smoothed through the CNN layer. , the pixel block collection The corresponding features are based on the dictionary The first combined area is obtained by combining, and a loss function is constructed based on the first combined area and the low-resolution text image. , to train the image reconstruction model;
[0012] Step 5: Based on the super-resolution module, all uncovered image blocks are super-resolved and smoothed through the CNN layer to obtain the super-resolution image blocks of each uncovered image block, and the features corresponding to the super-resolution image blocks are calculated according to the dictionary. The second combined area is obtained by combining, and the loss function is constructed based on the second combined area and the high-resolution text image. , to train the image reconstruction model.
[0013] Furthermore, in step 4, the loss function The build is as follows:
[0014] ;
[0015] in, Represents a large low-resolution text image, Indicates the number of connected areas of the covered image blocks, represents the first Connected areas, Indicates The predicted sub-region of the mask image block corresponds to the first Connected sub-regions, represents the parameters of the convolutional layer, represents the convolutional layer, represents a set of image blocks selected in the low-resolution text image, the selected set of image blocks and the first image block in the first combined area corresponding to consecutive sub-regions.
[0016] Further, in step five, the super-resolution module includes a first convolution module, an upsampling module and a second convolution module;
[0017] The first convolution module performs dimension upgrading on the feature dimension of the uncovered image block to obtain the feature dimension after dimension upgrading.
[0018] The upsampling module rearranges the feature dimensions after the first dimension upgrade by the number of channels to obtain the feature dimensions after the second dimension upgrade;
[0019] The second convolution module performs dimensionality reduction reconstruction on the feature dimension after the secondary dimensionality increase to obtain a super-resolution image block of the uncovered image block.
[0020] Furthermore, the loss function of step 5 is The build is as follows:
[0021] ;
[0022] in, represents the super-resolution image patch of the unmasked image patch, represents a real high-resolution image, Indicates the number of connected areas of uncovered image blocks, represents the first Connected areas, represents a set of image blocks selected in the super-resolution text image obtained by the image reconstruction model, and the selected set of image blocks is combined with the first Corresponding to continuous sub-regions, represents a set of image blocks selected in a real high-resolution text image, and the selected set of image blocks is combined with the first Corresponding to continuous sub-regions, represents the parameters of the convolutional layer, Represents a convolutional layer.
[0023] Furthermore, the verification phase after the training of the image reconstruction model is specifically as follows:
[0024] After the low-resolution text image to be super-resolved is cut into equal sizes, it is sent to the backbone network for feature extraction to obtain the extracted features;
[0025] The extracted features are input into the super-resolution module to obtain a super-resolution text image;
[0026] The resolution difference between the super-resolution text image and the real high-resolution text image corresponding to the low-resolution text image to be super-resolved is compared. If the difference is within the set range, the image reconstruction model training is completed. If the difference is not within the set range, the image reconstruction model training continues until the difference is within the set range.
[0027] A text image super-resolution system based on mask reconstruction paradigm, which pre-processes a large-size low-resolution text image and inputs it into a trained image reconstruction model to output a super-resolution image corresponding to the low-resolution text image;
[0028] The image reconstruction model includes a preprocessing module, a backbone network, a decoding module and a super-resolution module. The training process of the image reconstruction model is as follows:
[0029] The preprocessing module is used to obtain a large-size low-resolution text image and cut it into equal-sized image blocks to form a matrix , randomly mask multiple image blocks according to the set ratio to obtain the covered image blocks; Each image block in is assigned a unique number, and a sliding window of a set shape is used to map the matrix Slide the image blocks in the window to obtain multiple window sets, each of which represents the image block coding set in the window. Determine whether each window set has an intersection with the covered image block set and the uncovered image block set. If so, save the intersection area in the dictionary , Represents an index, when , A dictionary representing the connected areas of uncovered image blocks. , A dictionary representing connected areas of covered image blocks, wherein the covered image block set is a set consisting of each covered image block, and the uncovered image block set is a set consisting of each uncovered image block;
[0030] The backbone network is used to extract features of image blocks in the set of uncovered image blocks to obtain features of the uncovered image blocks;
[0031] The decoding module is used to combine the features of the covered image block and the uncovered image block in the original order and add the position code, and then process the predicted pixel block set of the covered image block based on the self-attention mechanism. , the pixel block collection Corresponding features and dictionaries The first combined area is obtained by combining, and a loss function is constructed based on the first combined area and the low-resolution text image. , to train the image reconstruction model;
[0032] The super-resolution module is used to super-resolve all uncovered image blocks to obtain super-resolution image blocks for each uncovered image block, and to map the features and dictionary corresponding to the super-resolution image blocks. The second combined area is obtained by combining, and the loss function is constructed based on the second combined area and the low-resolution text image. , to train the image reconstruction model.
[0033] Furthermore, the verification phase after the training of the image reconstruction model is specifically as follows:
[0034] After the low-resolution text image to be super-resolved is cut into equal sizes, it is sent to the backbone network for feature extraction to obtain the extracted features;
[0035] The extracted features are input into the super-resolution module to obtain a super-resolution text image;
[0036] The resolution difference between the super-resolution text image and the real super-resolution text image corresponding to the low-resolution text image to be super-resolved is compared. If the difference is within the set range, the image reconstruction model training is completed. If the difference is not within the set range, the image reconstruction model training continues until the difference is within the set range.
[0037] The advantages of the text image super-resolution method and system based on the mask reconstruction paradigm provided by the present invention are: the text image super-resolution method and system based on the mask reconstruction paradigm provided in the structure of the present invention are aimed at solving the problems of text reconstruction distortion, excessive video memory occupancy and low computational efficiency encountered when processing large-size low-resolution text images, while overcoming the non-smoothness phenomenon that may occur during image cutting and recombination, significantly improving the efficiency and quality of large-size low-resolution text image processing, optimizing video memory occupancy, and improving image reconstruction quality, especially being able to perform high-quality reconstruction of text areas, and having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 Schematic diagram of the training process of the image reconstruction model of the present invention;
[0039] Figure 2Schematic diagram of the verification and execution process of the trained image reconstruction model. DETAILED DESCRIPTION
[0040] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.
[0041] This is especially true when processing large-size images. Current super-resolution technologies often encounter problems such as text reconstruction distortion, excessive video memory usage, and low computational efficiency when processing large-size images. In addition, when an image is cut into small blocks for processing, when these blocks are combined, non-smoothness often occurs, such as the checkerboard effect, which seriously affects the image quality and visual effect.
[0042] This embodiment proposes Figure 1 and 2 A text image super-resolution method based on mask reconstruction paradigm is shown to reduce the algorithm memory usage while maintaining high image quality;
[0043] Specifically, the large-size low-resolution text image is cut into equal-size parts and then input into the trained image reconstruction model to output a super-resolution image corresponding to the low-resolution text image; wherein, the equal-size cutting can directly refer to the image cutting in step one.
[0044] The image reconstruction model includes a backbone network, a decoding module, and a super-resolution module. The decoding module is only used during the training process and does not participate in the subsequent execution process of the trained image reconstruction model. The training process of the image reconstruction model includes steps one to five:
[0045] Step 1: Get a large low-resolution text image and cut it into equal-sized image blocks to form a matrix , randomly mask multiple image blocks according to a set ratio to obtain a covered image block;
[0046] Assume that the input is a large-sized low-resolution text image ,in and are the height and width of the low-resolution text image, and divide it into image blocks, each of which has a size of , are the height and width of the image block, respectively. If the height or width of the low-resolution text image cannot be divided by the image block, the size of the image block is adjusted by padding with zeros. The total number of image blocks ,in Indicates rounding up. The resulting image block set is used express, is the first image blocks.
[0047] Next, a masking strategy is applied to these image blocks, randomly selecting a certain proportion of image blocks and masking them. Set the masking ratio to , then The image block will be masked. After masking, if the first If the image block is covered, it is represented as The entire image block can be represented as in , is the set of uncovered image patches, is a set of masked image blocks. In subsequent processing It does not directly participate in the feature extraction of the backbone network. However, it will be used in the decoding module to enable the image reconstruction model to learn how to fill in the missing information, thereby enhancing its generalization ability.
[0048] Step 2: Matrix Each image block in is assigned a unique number, and a sliding window of a set shape is used to map the matrix Slide the image blocks in the window to obtain multiple window sets, each of which represents the image block coding set in the window. Determine whether each window set has an intersection with the covered image block set and the uncovered image block set. If so, save the intersection area in the dictionary , Represents an index, when , A dictionary representing the connected areas of uncovered image blocks. , A dictionary representing connected areas of covered image blocks, wherein the covered image block set is a set consisting of each covered image block, and the uncovered image block set is a set consisting of each uncovered image block;
[0049] The covered image blocks and the uncovered image blocks are matched by category. This embodiment proposes an image block matching strategy based on intersection operation, which aims to deal with the reconstruction and combination of covered image blocks and uncovered image blocks in the image super-resolution process. , whose size is , where each position corresponds to an image block and each position is assigned a unique number (such as The matrices are numbered from 1 to 12). After the above masking, use the set Represents two parts, m=1 represents a set of covered image blocks composed of covered image blocks and m=0 is the set of uncovered image blocks composed of uncovered image blocks Then, in the image block matrix Sliding window operations are performed on the , and the obtained window sets are distinguished by set operations, which can effectively determine whether these window sets can form a rectangular or long strip structure. For example, for a rectangular connected area, The sliding window will be rectangular Divide into different window sets }, each window set represents the set of image block numbers within the window, where Indicates A set of image block numbers within a set of windows.
[0050] Through the set operation, it can be determined whether these window sets are respectively consistent with the cover image block set or a collection of uncovered image patches There is an intersection.
[0051] like but ;
[0052] Specifically, for each sliding window, the window set , if satisfied This means that the window set is completely composed of covered (uncovered) image blocks, and the intersection area is obtained. The intersection area is saved in the dictionary For subsequent use, m=1 indicates a dictionary of the connected areas of the covered image blocks, and m=0 indicates a dictionary of the connected areas of the uncovered image blocks. In this example, the connected areas of the image blocks can be obtained based on this, and different connected areas can be obtained by setting sliding windows of different shapes and sizes.
[0053] Steps 1 and 2 mainly preprocess the low-resolution text image before inputting it into the image reconstruction model, thereby improving the training effect of the image reconstruction model.
[0054] Step 3: extracting features of image blocks in the set of uncovered image blocks based on the backbone network to obtain features of the uncovered image blocks;
[0055] The backbone network uses the text image super-resolution model RTSRN to extract features from the set of uncovered image blocks.
[0056] ;
[0057] is the unmasked image block feature, is the parameter of the RTSRN model. RTSRN is an existing text super-resolution model that designs an optimized network structure to efficiently and accurately extract features from local details of low-resolution text images, enabling it to achieve better performance in super-resolution reconstruction. After the backbone model extracts features, the unmasked image block features are input into the decoding module and the super-resolution module respectively.
[0058] Step 4: Based on the decoding module, the features of the covered image block and the uncovered image block are combined in the original order and then position encoded. Then, the predicted pixel block set of the covered image block is obtained based on the self-attention mechanism and smoothed through the CNN layer. , the pixel block collection The corresponding features are based on the dictionary The first combined area is obtained by combining, and a loss function is constructed based on the first combined area and the low-resolution text image. , to train the image reconstruction model;
[0059] The decoding module is a key component in the mask reconstruction paradigm, responsible for reconstructing the original image from partially masked image features. The input of the decoding module includes the unmasked image block features and the masked image blocks. All image blocks are combined in the original order and the position encoding is added to ensure that the decoder can understand the specific location of each mark in the image.
[0060] ;
[0061] represents the set of all input image patches plus position encoding, It is the position code obtained based on the image block. Represents a combination function that combines the encoded unmasked image block features and the masked image block's mask mark into a set in the original order. A lightweight Transformer layer is introduced to process features. The Transformer’s self-attention mechanism enables the network to capture complex dependencies between covered and uncovered regions.
[0062] ;
[0063] in Represents the features processed by Transformer, where Represents the features of the masked image patch. are the parameters of the Transformer layer. The features processed by the Transformer layer Input to the linear prediction layer to predict the pixel value of the masked image block:
[0064] ;
[0065] A set of pixel blocks representing the predicted covered image blocks , Indicates predicted sub-regions of the masked image patch, It is the feature of the masked image block obtained through lightweight Transform. is the parameter of the linear layer. Then according to the dictionary obtained in step 2 , the pixel block collection Corresponding features according to the dictionary Combine and pass a layer of CNN for smoothing to obtain the first combined area. These first combined areas are compared with the low-resolution text image The corresponding area calculates the loss function based on L1 loss .
[0066] ;
[0067] in, Represents a large low-resolution text image, Indicates the number of connected areas of the covered image blocks, represents the first Connected sub-regions, Indicates The predicted sub-region of the mask image block corresponds to the first Connected sub-regions, represents the parameters of the convolutional layer, represents the convolutional layer, represents a set of image blocks selected in the low-resolution text image, the selected set of image blocks and the first image block in the first combined area corresponding to consecutive sub-regions.
[0068] Step 5: Based on the super-resolution module, all uncovered image blocks are super-resolved and smoothed through the CNN layer to obtain the super-resolution image blocks of each uncovered image block, and the features corresponding to the super-resolution image blocks are calculated according to the dictionary. The second combined area is obtained by combining, and the loss function is constructed based on the second combined area and the high-resolution text image provided by the dataset , to train the image reconstruction model.
[0069] In the super-resolution module, focus on the features of the unmasked image patches These unmasked image blocks already contain local detail information of the original image and can be reconstructed with high quality. The super-resolution module includes a first convolution module, an upsampling module, and a second convolution module.
[0070] The first convolution module has the following feature dimensions for the unmasked image block: Perform dimension upgrade to obtain the feature dimension after dimension upgrade , Indicates the number of channels, Respectively represent the height and width of the low-resolution text image, Represents the scaling factor of the upsampling module;
[0071] The upsampling module uses Pixel Shuffle, which is used to scale the feature dimensions after one dimension upgrade. By rearranging the number of channels, the feature dimension after secondary dimensionality increase is obtained , thereby increasing the image resolution; the Pixel Shuffle operation converts it into an Super-resolution image feature map with times resolution;
[0072] The second convolution module is used to upgrade the feature dimension after the second dimension Perform dimensionality reduction reconstruction to obtain the super-resolution image block of the uncovered image block. The feature dimension of the super-resolution image block is ;
[0073] That is, the generation of super-resolution image blocks can be described as follows:
[0074] ;
[0075] is the super-resolution image patch of the unmasked image patch, represents the first convolutional module, are the parameters of the first convolution module, is the upsampling module, is the scale parameter of the upsampling module, is the second convolution module, are the parameters of the second convolution module.
[0076] Then according to the dictionary obtained by matching , the features corresponding to the super-resolution image blocks are calculated according to the dictionary Combined, and then smoothed through a layer of CNN to obtain the second combined area. The second combined area is compared with the real high-resolution text image provided by the dataset The corresponding area constructs the loss function based on L1 loss .
[0077] ;
[0078] in, represents the super-resolution image patch of the unmasked image patch, Represents a low-resolution text image The corresponding real high-resolution image, Indicates the number of connected areas of uncovered image blocks, represents the first Connected areas, represents a set of image blocks selected in the super-resolution text image obtained by the image reconstruction model, and the selected set of image blocks is combined with the first Corresponding to continuous sub-regions, represents a set of image blocks selected in the real high-resolution text image, and the selected image block set is combined with the first Corresponding to continuous sub-regions, represents the parameters of the convolutional layer, Represents a convolutional layer.
[0079] Loss based on the decoding module and super-resolution module and , supervise the image reconstruction model training and obtain the final image reconstruction model.
[0080] After the image reconstruction model is trained, it is verified through the verification set to determine the training effect of the image reconstruction model. The verification stage after the image reconstruction model is trained is as follows:
[0081] After the low-resolution text image to be super-resolved is cut into equal sizes, it is sent to the backbone network for feature extraction to obtain the extracted features;
[0082] The extracted features are input into the super-resolution module to obtain a super-resolution text image;
[0083] The resolution difference between the super-resolution text image and the real high-resolution text image corresponding to the input low-resolution text image is compared. If the difference is within the set range, the image reconstruction model training is completed. If the difference is not within the set range, the image reconstruction model training continues until the difference is within the set range.
[0084] It can be understood that the random mask in step 1, the local matching in step 2, and the decoding module in step 4 are only applied to the training process of the image reconstruction model and do not participate in the subsequent verification and execution process of the image reconstruction model.
[0085] Through steps one to five, this embodiment aims to solve the problems of text reconstruction distortion, excessive video memory usage and low computational efficiency encountered when processing large-size low-resolution text images, while overcoming the unevenness that may occur during image cutting and recombination, significantly improving the efficiency and quality of large-size low-resolution text image processing, optimizing video memory usage, and improving image reconstruction quality, especially enabling high-quality reconstruction of text areas, and has broad application prospects.
[0086] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A text image super-resolution method based on mask reconstruction paradigm, characterized in that: The large-sized low-resolution text image is cut into equal sizes and then input into the trained image reconstruction model to output a super-resolution image corresponding to the low-resolution text image; The training process of the image reconstruction model is as follows: Step 1: Get a large low-resolution text image and cut it into equal-sized image blocks to form a matrix , randomly mask multiple image blocks according to a set ratio to obtain a covered image block; Step 2: Matrix Each image block in is assigned a unique number, and a sliding window of a set shape is used to map the matrix Slide the image blocks in the window to obtain multiple window sets, each of which represents the image block coding set in the window. Determine whether each window set has an intersection with the covered image block set and the uncovered image block set. If so, save the intersection area in the dictionary , Represents an index, when , A dictionary representing the connected areas of uncovered image blocks. , A dictionary representing connected areas of covered image blocks, wherein the covered image block set is a set consisting of each covered image block, and the uncovered image block set is a set consisting of each uncovered image block; Step 3: extracting features of image blocks in the set of uncovered image blocks based on the backbone network to obtain features of the uncovered image blocks; Step 4: Based on the decoding module, the features of the covered image block and the uncovered image block are combined in the original order and the position code is added. Then, the pixel block set of the predicted covered image block is obtained based on the self-attention mechanism and smoothed through the CNN layer. , the pixel block collection The corresponding features are based on the dictionary The first combined area is obtained by combining the first combined area and the low-resolution text image. Constructing the loss function , to train the image reconstruction model; Step 5: Based on the super-resolution module, all uncovered image blocks are super-resolved and smoothed through the CNN layer to obtain the super-resolution image blocks of each uncovered image block, and the features corresponding to the super-resolution image blocks are calculated according to the dictionary. The second combined region is obtained by combining the second combined region and the real high-resolution text image. Constructing the loss function , to train the image reconstruction model.
2. The text image super-resolution method based on mask reconstruction paradigm according to claim 1, characterized in that: In step 4, the loss function The build is as follows: ; in, represents a low-resolution text image, Indicates the number of connected areas of the covered image blocks, represents the first Connected areas, Indicates The predicted sub-region of the mask image block corresponds to the first Connected sub-regions, represents the parameters of the convolutional layer, represents the convolutional layer, represents a set of image blocks selected in the low-resolution text image, the selected set of image blocks and the first image block in the first combined area corresponding to consecutive sub-regions.
3. The text image super-resolution method based on mask reconstruction paradigm according to claim 1, characterized in that: In step 5, the super-resolution module includes a first convolution module, an upsampling module and a second convolution module; The first convolution module performs dimension upgrading on the feature dimension of the uncovered image block to obtain the feature dimension after dimension upgrading. The upsampling module rearranges the feature dimensions after the first dimension upgrade by the number of channels to obtain the feature dimensions after the second dimension upgrade; The second convolution module performs dimensionality reduction reconstruction on the feature dimension after the secondary dimensionality increase to obtain a super-resolution image block of the uncovered image block.
4. The text image super-resolution method based on mask reconstruction paradigm according to claim 3, characterized in that: Loss function of step 5 The build is as follows: ; in, represents the super-resolution image patch of the unmasked image patch, represents a real high-resolution text image, Indicates the number of connected areas of uncovered image blocks, represents the first Connected areas, represents a set of image blocks selected in the super-resolution text image obtained by the image reconstruction model, and the selected set of image blocks is combined with the first Corresponding to continuous sub-regions, represents a set of image blocks selected in a real high-resolution text image, and the selected set of image blocks is combined with the first Corresponding to continuous sub-regions, represents the parameters of the convolutional layer, Represents a convolutional layer.
5. The text image super-resolution method based on mask reconstruction paradigm according to claim 1, characterized in that: The verification phase after the training of the image reconstruction model is completed is specifically as follows: After the low-resolution text image to be super-resolved is cut into equal sizes, it is sent to the backbone network for feature extraction to obtain the extracted features; The extracted features are input into the super-resolution module to obtain a super-resolution text image; The resolution difference between the super-resolution text image and the real super-resolution text image corresponding to the low-resolution text image to be super-resolved is compared. If the difference is within the set range, the image reconstruction model training is completed. If the difference is not within the set range, the image reconstruction model training continues until the difference is within the set range.
6. A text image super-resolution system based on mask reconstruction paradigm, characterized in that: The large-size low-resolution text image is preprocessed and input into the trained image reconstruction model to output the super-resolution image corresponding to the low-resolution text image; The image reconstruction model includes a preprocessing module, a backbone network, a decoding module and a super-resolution module. The training process of the image reconstruction model is as follows: The preprocessing module is used to obtain a large-size low-resolution text image and cut it into equal-sized image blocks to form a matrix , randomly mask multiple image blocks according to a set ratio to obtain a covered image block; Pair Matrix Each image block in is assigned a unique number, and a sliding window of a set shape is used to map the matrix Slide the image blocks in the window to obtain multiple window sets, each of which represents the image block coding set in the window. Determine whether each window set has an intersection with the covered image block set and the uncovered image block set. If so, save the intersection area in the dictionary , Represents an index, when , A dictionary representing the connected areas of uncovered image blocks. , A dictionary representing connected areas of covered image blocks, wherein the covered image block set is a set consisting of each covered image block, and the uncovered image block set is a set consisting of each uncovered image block; The backbone network is used to extract features of image blocks in the set of uncovered image blocks to obtain features of the uncovered image blocks; The decoding module is used to combine the features of the covered image block and the uncovered image block in the original order and add the position code, and then process the predicted pixel block set of the covered image block based on the self-attention mechanism. , the pixel block collection Corresponding features and dictionaries The first combined area is obtained by combining, and a loss function is constructed based on the first combined area and the low-resolution text image. , to train the image reconstruction model; The super-resolution module is used to super-resolve all uncovered image blocks to obtain super-resolution image blocks for each uncovered image block, and to map the features and dictionary corresponding to the super-resolution image blocks. The second combined region is obtained by combining the second combined region and the real high-resolution text image provided by the dataset. Constructing the loss function , to train the image reconstruction model.
7. The text image super-resolution method based on mask reconstruction paradigm according to claim 6, characterized in that: The verification phase after the training of the image reconstruction model is completed is specifically as follows: After the low-resolution text image to be super-resolved is cut into equal sizes, it is sent to the backbone network for feature extraction to obtain the extracted features; The extracted features are input into the super-resolution module to obtain a super-resolution text image; The resolution difference between the super-resolution text image and the real super-resolution text image corresponding to the low-resolution text image to be super-resolved is compared. If the difference is within the set range, the image reconstruction model training is completed. If the difference is not within the set range, the image reconstruction model training continues until the difference is within the set range.
