Text image super-resolution method and system based on lightweight convolutional neural network

Abstract

The present invention discloses a text image super-resolution method and system based on a lightweight convolutional neural network, belonging to the field of image super-resolution technology. A lightweight convolutional neural network is trained using low-quality text image-true value image pairs to achieve super-resolution reconstruction of low-quality text images; during the training process, multi-level feature maps of the reconstructed high-resolution text image and its true value image are extracted, and the memory information of the feature map with the smallest spatial resolution in the multi-level feature map is extracted using an LSTM network to generate an embedding vector of the memory information; the errors at each level between the multi-level feature maps of the reconstructed high-resolution text image and its true value image, as well as the memory errors between the embedding vectors of the memory information, are calculated respectively, and the weighted result between the errors at each level and the memory errors is used as the training loss. The present invention can achieve excellent reconstruction effects with very low parameter count and computing resource consumption.

Description

Technical Field

[0001] The present invention belongs to the field of image super-resolution technology, and in particular relates to a text image super-resolution method and system based on a lightweight convolutional neural network. Background Art

[0002] Optical Character Recognition (OCR) refers to the process of using a device (such as a scanner or digital camera) to record text or text information in various scenes. It then determines the position and shape of the text by detecting light and dark patterns or edges, and then uses character recognition methods to translate the shape into computer text. This technology optically converts the text in the scene into a black and white dot image file. Recognition software then converts the text in the image into text format for further editing and processing in word processing software. Improving character recognition accuracy is a crucial issue in OCR.

[0003] Current optical character recognition (OCR) technology has achieved excellent results on high-quality text images. However, when recognizing low-quality blurred text images, OCR performance drops dramatically. Reasons for this low quality include low resolution, noise, blur, JPEG compression, artifacts, and ringing. The primary difficulty in recognizing blurred text images lies in the lack of detailed character information and edge texture information in the image. Super-resolution is a reasonable approach to address this problem. However, traditional natural scene super-resolution methods reconstruct the global image without distinguishing between foreground and background, while text super-resolution focuses on text information and fails to leverage the contextual relevance of text to facilitate better information recovery and reconstruction. Therefore, natural scene super-resolution is not suitable for directly applying to blurred text tasks. Compared to natural images, scene text has arbitrary poses, lighting, and blur, making super-resolution of super-blurred text images even more challenging. Therefore, a content-aware text super-resolution network is needed to generate clear and recognizable text images for recognition.

[0004] The current text super-resolution network model has a large number of parameters and many floating-point operations, requiring a large amount of computing resources and complex operators, making it difficult to strike a balance between performance and resources. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention discloses a text image super-resolution method and system based on a lightweight convolutional neural network. The technical solutions adopted are as follows:

[0006] In a first aspect, the present invention proposes a text image super-resolution method based on a lightweight convolutional neural network, comprising:

[0007] A lightweight convolutional neural network is trained using low-quality text image-ground truth image pairs to achieve super-resolution reconstruction of low-quality text images.

[0008] The lightweight convolutional neural network includes a first network for extracting shallow features of low-quality text images, a second network for extracting first deep features and second deep features from semantic information of the shallow features, a first fusion network for fusing the semantic information and its two deep features, a second fusion network for fusing the shallow features and the output of the first fusion network, and a third network for reconstructing a high-resolution text image using the output of the second fusion network;

[0009] The process of constructing the loss function for training lightweight convolutional neural networks includes:

[0010] The multi-level feature maps of the reconstructed high-resolution text image and its true value image are extracted, and the memory information of the feature map with the smallest spatial resolution in the multi-level feature map is extracted using the LSTM network to generate an embedding vector of the memory information. The errors at each level between the multi-level feature maps of the reconstructed high-resolution text image and its true value image, as well as the memory errors between the embedding vectors of the memory information are calculated respectively, and the weighted results between the errors at each level and the memory errors are used as the training loss.

[0011] Furthermore, when training the lightweight convolutional neural network, the step of image enhancement for low-quality text images is also included.

[0012] Furthermore, the first network first expands the number of channels of the low-quality text image from 3 to 64, and then sequentially uses a convolutional layer and several stacked first residual blocks to extract shallow features of the image.

[0013] Furthermore, the second network includes a convolutional layer, several stacked second residual blocks and two parallel deep extraction branches with the same structure. The convolutional layer and the several stacked second residual blocks are first used in sequence to extract the semantic information of the shallow features, and then the two deep extraction branches are used to extract the first deep features and the second deep features from the semantic information respectively.

[0014] Furthermore, the deep extraction branch is a multi-level structure. In each layer, a convolution operation is performed on the output features of the previous layer as input features. After the convolution operation results are nonlinearly activated, they are spliced ​​with the input features in the channel dimension to obtain the output features of the current layer.

[0015] Furthermore, the first fusion network first splices the semantic information, the first deep features and the second deep features in the channel dimension, and then extracts the first fusion features from the splicing results using a plurality of stacked third residual blocks.

[0016] Furthermore, the second fusion network first splices the shallow features and the output results of the first fusion network in the channel dimension, performs a convolution operation on the splicing result, and then performs a 1×1 convolution on the convolution operation result after nonlinear activation to obtain a second fusion feature containing global information.

[0017] Furthermore, the third network first performs a sub-pixel convolution operation on the output result of the second fusion network to improve the resolution, and then performs a convolution operation to generate a reconstructed high-resolution text image.

[0018] Furthermore, when training a lightweight convolutional neural network, a multi-level text recognition network is used to extract the multi-level feature map of the image. In each layer, the output features of the previous layer are used as input features to perform a convolution operation. The results of the convolution operation are sequentially nonlinearly activated and pooled to obtain the output features of the current layer. The output features of different layers constitute a multi-level feature map, and the feature map of the last level is the feature map with the smallest spatial resolution.

[0019] In the second aspect, the present invention proposes a text image super-resolution system based on a lightweight convolutional neural network, which is used to implement the above-mentioned text image super-resolution method based on a lightweight convolutional neural network.

[0020] In a third aspect, the present invention provides a computer electronic device, characterized in that it includes a memory and a processor;

[0021] The memory is used to store computer programs;

[0022] The processor is used to implement the above-mentioned text image super-resolution method based on a lightweight convolutional neural network when executing the computer program.

[0023] The beneficial effects of the present invention are:

[0024] The present invention proposes a text image super-resolution method based on a lightweight convolutional neural network. The parameters of the lightweight convolutional neural network used are less than 1M, and the floating-point operation is less than 4G. The method inputs a low-quality text image into a lightweight convolutional neural network for super-resolution, and inputs the obtained high-resolution image and the true value image into the text recognition network to calculate the feature difference under different feature dimensions and the memory information embedding vector difference of the feature map with the minimum spatial resolution as the text loss function, and the text loss function is used as part of the loss to participate in the reverse gradient propagation. The present invention utilizes a lightweight image super-resolution network in series with a text recognition network as a text prior to correct the super-resolution quality of the text image, so that the super-resolved text image has rich detail information and excellent visual effects. The lightweight convolutional neural network trained by the semantic information present invention is very convenient to be integrated on mobile devices as an upstream task of the OCR task to improve the recognition rate of the OCR task. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flowchart of an image super-resolution method based on a lightweight convolutional neural network in an embodiment of the present invention;

[0026] Figure 2 This is a structural diagram of the components of the lightweight convolutional neural network proposed in the present invention;

[0027] Figure 3 This is a schematic diagram of an electronic device proposed by the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described and illustrated below in conjunction with specific embodiments. The embodiments are merely illustrative of the present disclosure and do not limit its scope. The technical features of the various embodiments of the present invention may be combined accordingly, provided that there is no conflict between them.

[0029] The accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to actual circumstances.

[0031] The present invention realizes text image super-resolution based on a lightweight convolutional neural network. A low-quality text image dataset is used as a training set, and the true value image corresponding to the low-quality text image is used as a label image to train a lightweight convolutional neural network. The trained lightweight convolutional neural network is used to generate super-resolution results of the low-quality text image. The lightweight convolutional neural network takes the low-quality text image as input, and uses a first network, a second network, a first fusion network, a second fusion network, and a third network connected in series to extract shallow features, semantic information, deep features, and fusion features from the text image in turn and reconstruct it into a high-resolution text image. The present invention uses a lightweight image super-resolution network connected in series with a text recognition network as a text prior to correct the super-resolution quality of the text image, so that the super-resolved text image has rich detail information and excellent visual effects.

[0032] like Figure 1 As shown in Figure 2, the main steps of the text image super-resolution method based on lightweight convolutional neural network include:

[0033] S01, using the first network to extract shallow features of the low-quality text image after data enhancement.

[0034] S02, using the second network to extract semantic information from the shallow features of the low-quality text image, and using a multi-layer dual-branch network to extract the first deep features and the second deep features from the semantic information respectively.

[0035] S03: Utilize a first fusion network to fuse the semantic information, the first deep feature, and the second deep feature as a first fusion feature.

[0036] S04: Using the second fusion network to fuse the shallow features and the first fusion features as the second fusion features.

[0037] S05: Reconstruct the second fused features into a high-resolution text image using a third network.

[0038] S06, extracting the multi-level feature maps of the reconstructed high-resolution text image and the true value image, using the LSTM network to extract the memory feature map of the feature map with the smallest spatial resolution in the multi-level feature map, respectively calculating the hierarchical error between the multi-level feature maps of the high-resolution text image and the true value image, and the memory error between the memory feature maps, and using the weighted result between the memory error and the multiple hierarchical errors as the training loss to update the model parameters of the lightweight convolutional neural network.

[0039] S07, taking low-quality text images as input, uses the trained lightweight convolutional neural network model to generate high-resolution text images as super-resolution results.

[0040] For example, image enhancement in step S01 includes mirror symmetry, 90° horizontal flipping, and 90° vertical flipping. Image enhancement refers to the process of improving the visual effects of an image or highlighting specific information through a series of technical means, aiming to improve image quality, enhance key features, or adapt to subsequent processing requirements. Its core goal is to address defects in the original image (such as blur, low contrast, and noise interference) caused by factors such as the shooting environment and equipment limitations, or to specifically enhance detailed information in specific areas.

[0041] It should be noted that the original text image is a three-channel image, and the number of channels remains unchanged after image enhancement processing. Before extracting the shallow features of the low-quality text image after data enhancement, the number of channels of the low-quality text image after data enhancement is first expanded, and the 3 channels are converted into 64 channels, and then shallow feature extraction is performed. In a specific implementation of the present invention, the first network first expands the number of channels of the low-quality text image, and then sequentially uses the convolution layer and several stacked first residual blocks to extract the shallow features of the image. Since these features are obtained through simple convolution, the information of the original image can be retained more completely. These shallow features serve as guiding information in the subsequent calculation process of reconstructing features, which can help the network converge quickly while achieving good performance.

[0042] The second network includes a convolutional layer, several stacked second residual blocks and two parallel deep extraction branches with the same structure. The convolutional layer and several stacked second residual blocks are used in sequence to extract the semantic information of the shallow features, and then the two deep extraction branches are used to extract the first deep features and the second deep features from the semantic information respectively. Here, the deep extraction branch is a multi-level structure. In each layer, the output features of the previous layer are used as input features to perform a convolution operation. After the convolution operation results are nonlinearly activated, they are spliced ​​with the input features in the channel dimension to obtain the output features of the current layer. Since the network parameters and floating-point operations are required to be small while ensuring performance, the present invention uses the structure of the residual splicing block to extract and calculate features, and splices the calculated output features with the input features in the channel dimension as a new structural input. The above process is repeated to obtain the semantic information and deep features of the low-resolution text image.

[0043] The first fusion network first splices the semantic information, the first deep feature and the second deep feature in the channel dimension, and then uses several stacked third residual blocks to extract the splicing result to obtain the first fusion feature. The second fusion network first splices the shallow features and the output result of the first fusion network in the channel dimension, and performs a convolution operation on the splicing result, and then performs a 1×1 convolution on the convolution operation result after nonlinear activation to obtain a second fusion feature containing global information. The present invention executes two fusion networks in sequence, first performing multi-scale feature fusion from the semantic information scale and the deep feature scale after several residual splicing blocks, and then splicing the feature channels of the fused features and the shallow features and fusing the information of 1×1 convolution to obtain a global feature containing the information of the entire image.

[0044] The third network first performs a sub-pixel convolution operation on the output of the second fusion network to increase the resolution, and then performs a convolution operation to generate a reconstructed high-resolution text image. The global features are converted into a high-resolution image through sub-pixel convolution and ordinary convolution operations.

[0045] This paper uses real images as labels and does not use the traditional VGG network to calculate perception. This is because the VGG network is not designed for text recognition tasks and cannot understand the unique information in text images. To calculate text loss, this paper uses a multi-level text recognition network to extract multi-level feature maps of the image. In each layer, the output features of the previous layer are used as input features to perform convolution operations. The results of the convolution operations are sequentially nonlinearly activated and pooled to obtain the output features of the current layer. The output features of different layers form a multi-level feature map, and the feature map of the last layer is the feature map with the lowest spatial resolution.

[0046] For example, the multi-level text recognition network has a pyramid-like structure. As the network continues to deepen, the size of the resulting feature map continues to shrink, the receptive field continues to expand, and the feature map contains more global information. After passing through the four-layer network structure, the high-resolution image and the ground truth each generate four feature maps of different sizes. The resulting feature maps are input into the LSTM unit to obtain the final feature map. The weighted difference between the high-resolution image and the ground truth, a total of five corresponding feature maps, is calculated as the text loss function for gradient backpropagation.

[0047] Based on the same inventive concept, the present invention proposes a text image super-resolution system based on a lightweight convolutional neural network, such as Figure 2 As shown in the figure, the system mainly includes six modules: a text image data preprocessing module, a shallow feature calculation module, a semantic information and deep feature calculation module, a fusion feature calculation module, a super-resolution reconstruction module, and a text prior-based training module. The shallow feature calculation module, the semantic information and deep feature calculation module, the fusion feature calculation module, and the super-resolution reconstruction module constitute the lightweight convolutional neural network module. Each module is used to execute the steps of the text image super-resolution method based on the lightweight convolutional neural network.

[0048] In a specific implementation of the present invention, the text image data preprocessing module is used to preprocess the input original low-resolution text image. Specifically, it executes the method in the following step S11.

[0049] S11, obtain the original low-resolution text image and perform image enhancement, and record the low-resolution text image after image enhancement as .

[0050] Here, the image enhancement process may adopt the method in S01 above.

[0051] The shallow feature calculation module is used to extract shallow features of the low-resolution text image. Specifically, it performs the method in the following step S12.

[0052] S12, the low-resolution text image obtained in step S11 Convolution is used to change the channel dimension of the text image from 3 to 64, and then shallow feature extraction is performed.

[0053] For example, the shallow feature extraction process is expressed as:

[0054]

[0055] in, Represents a low-resolution text image The shallow characteristics of represents the first residual block network of the stack for extracting shallow features, represents the activation function, Represents a convolutional layer that maintains the spatial resolution while performing shallow feature computations.

[0056] Semantic information and deep feature calculation module is used to extract shallow features from low-quality text images Specifically, it performs the method in the following step S13.

[0057] S13, the low-resolution text image obtained in S12 Further feature extraction is performed on the shallow features to obtain a low-resolution text image Semantic information:

[0058]

[0059] in, Represents a low-resolution text image The semantic information of represents the second residual block network of the stack for extracting semantic information, represents the activation function, represents the convolutional layer, Represents a low-resolution text image shallow features.

[0060] Convert low-resolution text images Semantic information Input two parallel deep extraction branches for deep feature extraction to obtain low-resolution text images The first deep feature and the second deep feature:

[0061]

[0062]

[0063] in, 、 Represent low-resolution text images The first deep feature and the second deep feature, 、 Represents two parallel deep extraction branches, both of which have the same structure. In this embodiment, 、 A five-layer calculation structure is adopted, and the calculation process is shown as follows:

[0064]

[0065] in, express The intermediate state, For low-resolution text images Semantic information , For low-resolution text images The deep features of or ; Indicates that the features are spliced ​​in the channel dimension. represents the activation function, Represents the convolution layer. The deep extraction branch is a multi-layer structure. The overall calculation idea of ​​each layer is to combine the features of the previous layer Input the convolution layer, and the result is nonlinearly activated through the activation function, and finally It splices itself in the channel to obtain new features .

[0066] The fusion feature calculation module is used to calculate the low-resolution text image The semantic information, the first deep features and the second deep features are multi-scale fused to obtain fused features to assist in super-resolution of low-resolution text images. Specifically, the method in step S14 is performed.

[0067] S14, the low-resolution text image obtained in step S13 The semantic information of , the first deep features and the second deep features are fused as follows:

[0068]

[0069] in, Represents a low-resolution text image The first fusion feature, represents the third residual block network of the stack for extracting features, Indicates concatenating features in the channel dimension.

[0070] The super-resolution reconstruction module is used to transform shallow features and fusion features The fusion is performed to obtain a second fusion feature, which is a global feature and can be further reconstructed into a high-resolution text image. Specifically, the method in step S15 is described below.

[0071] S15, the low-resolution text image obtained in step S12 Shallow features and the fusion features obtained in step S14 Further fusion to obtain low-resolution text image The global features of . For example, the fusion process is expressed as:

[0072]

[0073] in, Represents a low-resolution text image The global characteristics of represents the 1×1 convolution used to fuse features, represents the activation function, represents the convolutional layer, Indicates that the features are concatenated in the channel dimension.

[0074] Convert low-resolution text images Global characteristics of Perform convolution reconstruction to obtain a low-resolution text image The third deep characteristic:

[0075]

[0076] in, Represents a low-resolution text image The third deep feature of the reconstructed high-resolution text image has a resolution of 2 times; represents the convolutional layer, Represents sub-pixel convolution.

[0077] A training module based on text priors for computing reconstructed high-resolution text images The difference between multiple pairs of feature maps and the true value image GT, and the weighted sum of the difference between the feature maps is used as the text loss Specifically, it executes the method in S16 below.

[0078] S16, reconstructing the high-resolution text image obtained in step S15 The ground truth image GT corresponding to the low-quality text image is input into the multi-level text recognition network to extract the multi-level feature map. For example, the multi-level text recognition network is a four-layer pyramid convolutional network. The calculation process of the multi-level feature map is as follows:

[0079]

[0080]

[0081] in, Represents the reconstructed high-resolution text image Multi-level feature map, when i=1, for ; Represents the multi-level feature map of the true value image GT. When i=1, For GT. Indicates pooling, with a downsampling rate of 2; represents the activation function, Represents a convolutional layer.

[0082] Reconstructed high-resolution text image The feature map with the smallest spatial resolution And the feature map with the minimum spatial resolution of the ground truth image GT Input the same LSTM network to calculate the memory feature map and extract the word embedding:

[0083]

[0084]

[0085] in, 、 Represent the reconstructed high-resolution text images And the memory information embedding vector of the true value image GT; represents word embedding operation, Represents a long short-term memory unit.

[0086] Using the high-resolution text image obtained above Multi-level feature maps , multi-level feature map of the true value image GT } and memory feature map 、 Calculate the loss:

[0087]

[0088]

[0089]

[0090] in, Representing high-resolution text images in the context of text recognition networks The text loss between the real image GT, is the weighted weight given when calculating the difference between different features. Represents the mean square error loss, which is used to measure the similarity between two features. Backward gradient propagation is performed as part of the overall loss function.

[0091] When using the trained lightweight convolutional neural network to achieve super-resolution reconstruction of low-quality text images, the final super-resolution result That is, a high-resolution text image is outputted by the low-resolution text image through the image super-resolution method based on a lightweight convolutional neural network of the present invention.

[0092] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0093] Embodiments of the system of the present invention can be applied to any device with data processing capabilities, such as a computer or other device. System embodiments can be implemented through software, hardware, or a combination of software and hardware. For example, a software implementation, as a logical device, is implemented by a processor of any device with data processing capabilities, reading corresponding computer program instructions from non-volatile memory into internal memory and executing them.

[0094] It should also be noted that the text image super-resolution method based on a lightweight convolutional neural network in the above embodiment can essentially be executed by a computer program. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the method provided in the above embodiment, such as Figure 3 As shown, it includes a memory and a processor;

[0095] The memory is used to store computer programs;

[0096] The processor is used to implement the text image super-resolution method based on lightweight convolutional neural network in the above embodiment when executing the computer program.

[0097] Furthermore, the logic instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0098] The embodiment described above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A text image super-resolution method based on a lightweight convolutional neural network, characterized in that: include: Use low-quality text image-ground truth image pairs to train a lightweight convolutional neural network to achieve super-resolution reconstruction of low-quality text images; The lightweight convolutional neural network includes a first network for extracting shallow features of low-quality text images, a second network for extracting first deep features and second deep features from the semantic information of shallow features, a first fusion network for fusing semantic information and its two deep features, a second fusion network for fusing shallow features and output results of the first fusion network, and a third network for reconstructing high-resolution text images using the output results of the second fusion network; the second network includes a convolution layer, several stacked second residual blocks and two parallel deep extraction branches with the same structure, firstly extracting semantic information of shallow features by using the convolution layer and several stacked second residual blocks in sequence, and then extracting semantic information of shallow features by using the two deep extraction branches from the semantic information. The first deep feature and the second deep feature are extracted from the information respectively; the deep extraction branch is a multi-level structure, and in each layer, the output feature of the previous layer is used as the input feature to perform a convolution operation, and the result of the convolution operation is nonlinearly activated and then spliced ​​with the input feature in the channel dimension to obtain the output feature of the current layer; the first fusion network first splices the semantic information, the first deep feature and the second deep feature in the channel dimension, and then uses several stacked third residual blocks to extract the first fusion feature from the splicing result; the second fusion network first splices the shallow feature and the output result of the first fusion network in the channel dimension, and performs a convolution operation on the splicing result, and then performs a 1×1 convolution on the convolution operation result after nonlinear activation to obtain the second fusion feature containing global information; The process of constructing the loss function for training lightweight convolutional neural networks includes: Extract the multi-level feature maps of the reconstructed high-resolution text image and its ground-truth image, use the LSTM network to extract the memory information of the feature map with the smallest spatial resolution in the multi-level feature map, and generate an embedding vector for the memory information; The errors at each level between the multi-level feature maps of the reconstructed high-resolution text image and its true value image, as well as the memory errors between the embedded vectors of the memory information are calculated respectively, and the weighted results between the errors at each level and the memory errors are used as the training loss.

2. The text image super-resolution method based on lightweight convolutional neural network according to claim 1 is characterized in that The first network first expands the number of channels of the low-quality text image, and then sequentially uses a convolutional layer and several stacked first residual blocks to extract shallow features of the image.

3. The text image super-resolution method based on lightweight convolutional neural network according to claim 1 is characterized in that The third network first performs a sub-pixel convolution operation on the output result of the second fusion network to improve the resolution, and then performs a convolution operation to generate a reconstructed high-resolution text image.

4. The text image super-resolution method based on lightweight convolutional neural network according to claim 1 is characterized in that When training a lightweight convolutional neural network, a multi-level text recognition network is used to extract the multi-level feature map of the image. In each layer, the output features of the previous layer are used as input features to perform a convolution operation. The results of the convolution operation are sequentially nonlinearly activated and pooled to obtain the output features of the current layer. The output features of different layers constitute a multi-level feature map, and the feature map of the last level is the feature map with the smallest spatial resolution.

5. A text image super-resolution system based on a lightweight convolutional neural network, used to implement the text image super-resolution method according to claim 1, characterized in that the system include: A lightweight convolutional neural network module comprising a first network for extracting shallow features of low-quality text images, a second network for extracting first and second deep features from semantic information of the shallow features, a first fusion network for fusing the semantic information and its two deep features, a second fusion network for fusing the shallow features with the output of the first fusion network, and a third network for reconstructing a high-resolution text image using the output of the second fusion network; A text prior-based training module is used to train a lightweight convolutional neural network using low-quality text image-ground truth image pairs to achieve super-resolution reconstruction of low-quality text images. The loss function used to train the lightweight convolutional neural network includes the following steps: The multi-level feature maps of the reconstructed high-resolution text image and its true value image are extracted, and the memory information of the feature map with the smallest spatial resolution in the multi-level feature map is extracted using the LSTM network to generate an embedding vector of the memory information. The errors at each level between the multi-level feature maps of the reconstructed high-resolution text image and its true value image, as well as the memory errors between the embedding vectors of the memory information are calculated respectively, and the weighted results between the errors at each level and the memory errors are used as the training loss.

6. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the text image super-resolution method based on a lightweight convolutional neural network as described in any one of claims 1 to 4 when executing the computer program.

Images