Motion blurred flower text image restoration method and system
Through the method of combining neural networks with convolutional neural networks and bidirectional long and short-term memory networks, the problem of unsatisfactory motion blur restoration effect in flower text images is solved, and a more efficient text image defuzzy effect is achieved, especially in rich font shapes. , in natural scenes with complex colors.
Patent Information
- Application Number
- CN202411317300.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-31
AI Technical Summary
The prior art is difficult to effectively remove motion blur in flower text images, especially in natural scenes with rich font shapes and complex colors. The traditional method has poor restoration effect and has not fully considered the strong correlation between text pixels.
Using a neural network-based method, combining convolutional neural networks and bidirectional long and short-term memory networks, a generator network model is built. By generating an adversarial network, the generator network model can learn the strong correlation of pixel points of text pictures and generate clear image.
The effect of removing motion blur of flower text images is improved, and complex scenes in text images can be better processed, overcome the shortcomings of the prior art, and achieve more efficient image restoration.
Smart Images
Figure CN119228685A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to machine vision technology, and in particular to image deblurring technology in computer vision. Background Art
[0002] An important research direction in computer vision is image enhancement. Image motion blur removal is a hot topic. With the popularity of lightweight devices such as mobile phones, motion blur is more likely to occur, and important information that users need to record, such as object names and road signs, is difficult to recognize. Therefore, the restoration of blurred cursive text is an urgent problem that needs to be solved.
[0003] Motion deblurring is a blind deblurring algorithm. In the deblurring process, it is impossible to make a priori assumptions about the blur kernel or the image, and it is necessary to achieve true image deblurring. Typical representatives of traditional methods are feature methods based on image edges and maximum a posteriori probability methods. These algorithms have a long recovery time and cannot meet most needs at this stage. With the development of machine learning, more and more scholars use neural networks to achieve image deblurring, such as SRN-DeblurNet, DeblurGAN, etc. However, these algorithms focus on deblurring natural scenes and have not paid attention to the removal of motion blur of text.
[0004] In addition, there is not enough research on text images. Most studies use the dataset constructed by M. Hradis. The original images of this dataset are cut from published journals, and the blur kernel is artificially constructed and convolved with the original image to generate a blurred image. However, the text in the images of this dataset is relatively neat, and the font and color are very simple. These algorithms are effective for text restoration in a single scene, but they are difficult to deal with the restoration of cursive text in natural scenes with rich font shapes and complex colors. On the other hand, most restoration methods simply treat text images as simple images, without considering the strong correlation between adjacent pixels of the text. The restoration effect is not ideal and cannot meet actual needs. Summary of the invention
[0005] In view of the problem that the existing de-motion blurring effect of cursive text is not ideal, the purpose of the present invention is to provide a motion blurred cursive text image restoration method, which restores the motion blurred cursive text image based on a neural network, thereby improving the deblurring effect; and on this basis, the present invention further provides a motion blurred cursive text image restoration system that can implement the restoration method.
[0006] In order to achieve the above object, the present invention provides a method for restoring a motion blurred cursive text image, comprising:
[0007] Model training phase
[0008] First, a dataset of blurred pictures of cursive text is generated. A generator network model is constructed based on a combination of a convolutional neural network and a bidirectional long short-term memory network, and the generator is supervised by a generative adversarial network. On this basis, the constructed generator network model is trained based on the generated dataset of blurred pictures of cursive text to obtain a blurred cursive text restoration model.
[0009] Recovery Phase
[0010] Input the blurred cursive text image to be restored into the trained blurred cursive text restoration model to generate a clear image.
[0011] In some embodiments of the present invention, when generating a data set of blurred pictures of cursive text, the method includes:
[0012] (1) Select an open source image dataset, extract the text content in the image in the dataset, and obtain a clear cursive text image;
[0013] (2) Randomly generate the blur kernel size and blur kernel jitter intensity, and randomly generate a blur path based on these two values, and then add Gaussian blur to form a blur kernel;
[0014] (3) Convolve the clear cursive text image obtained in step (1) with the blur kernel formed in step (2) to obtain a cursive text blur image dataset.
[0015] In some embodiments of the present invention, the generator network model constructed by the method mainly includes:
[0016] An input module, wherein the input module is configured to perform multiple convolution module operations on the input blurred picture of cursive text to obtain a feature map through the convolution operation;
[0017] A feature learning module, wherein the feature learning module is composed of a plurality of residual-bidirectional long short-term memory networks, and is configured to perform feature learning on a feature map formed by an input module based on the plurality of residual-bidirectional long short-term memory networks, and learn the strong correlation of pixel points of a cursive text image;
[0018] The image reconstruction module is configured to interact with the feature learning module data and can generate a clear image by passing the feature vector through multiple deconvolution modules based on the correlation determined by the feature learning module.
[0019] In some embodiments of the present invention, each residual-bidirectional long short-term memory network module in the feature learning module is connected through a densely connected network.
[0020] In some embodiments of the present invention, residual connections are added before the input module and after the output module in the generator network model.
[0021] In some embodiments of the present invention, during the model training stage, when the generator network model is trained, the restored image generated and output by the generator network model and its corresponding clear image are input into the discriminator to determine whether the input image is true or false; then a loss function is generated based on the feature distance between the clear image and the restored image and the judgment result of the discriminator; and the network parameters of the generator network model and the discriminator are iterated through the loss function to optimize the network parameters until the model converges, thereby obtaining a fuzzy cursive text restoration model.
[0022] In order to achieve the above object, the present invention provides a motion blurred cursive text image restoration system, comprising:
[0023] Cursive text blurred image dataset;
[0024] A fuzzy cursive text restoration model, wherein the fuzzy cursive text restoration model is constructed by combining a convolutional neural network and a bidirectional long short-term memory network to form a generator network model, and is trained based on a fuzzy cursive text image dataset.
[0025] In some embodiments of the present invention, the generator network model mainly includes:
[0026] An input module, wherein the input module is configured to perform multiple convolution module operations on the input blurred picture of cursive text to obtain a feature map through the convolution operation;
[0027] A feature learning module, wherein the feature learning module is composed of a plurality of residual-bidirectional long short-term memory networks, and is configured to perform feature learning on a feature map formed by an input module based on the plurality of residual-bidirectional long short-term memory networks, and learn the strong correlation of pixel points of a cursive text image;
[0028] The image reconstruction module is configured to interact with the feature learning module data and can generate a clear image by passing the feature map through multiple deconvolution modules.
[0029] In some embodiments of the present invention, each residual-bidirectional long short-term memory network module in the feature learning module is connected through a densely connected network.
[0030] In some embodiments of the present invention, a generative adversarial network is constructed for the generator network model in the restoration system.
[0031] The motion blurred cursive text image restoration solution provided by the present invention aims at the problem of de-motion blurring of cursive text, provides an end-to-end motion blurred cursive text image restoration method that fully utilizes text relevance, can improve the effect of image restoration, and overcome the problems existing in the prior art.
[0032] Furthermore, the solution of the present invention introduces BLSTM, an effective model for processing time series, to deal with text image problems, which can specifically extract features of a special type of image, text, and effectively learn the strong correlation between pixels.
[0033] Furthermore, the solution of the present invention makes full use of residual networks and densely connected networks, which can effectively improve the training results of parameters and improve the model restoration effect.
[0034] Furthermore, the solution of the present invention adopts a generative adversarial network to supervise the generation effect of the generator, which helps to further improve the texture effect of the generated image.
[0035] Furthermore, the solution of the present invention also provides a method for generating a data set of fuzzy images of cursive text, and the expansion of the data set helps to further improve the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0037] Figure 1 A flowchart of restoring a motion blurred cursive text image in an example of the present invention;
[0038] Figure 2 is a structural example diagram of a generator network in an example of the present invention;
[0039] Figure 3 is an example diagram of the RSBLblock structure in the example of the present invention;
[0040] Figure 4 This is an example diagram of the structure of a generative adversarial network in the example of the present invention. DETAILED DESCRIPTION
[0041] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further described below with reference to specific diagrams.
[0042] In order to solve the problem of deblurring cursive text, the present invention provides a restoration method based on a neural network. The restoration method first generates a cursive text blurred image dataset that meets the application scenario; on this basis, the dataset that meets the application scenario is used for model training. Then, restoration is performed to convert image deblurring into an end-to-end image problem (i.e., output is generated directly from input through network calculation, without artificial stage-by-stage restoration). Based on a convolutional neural network and taking full account of the particularity of text, a time series model Bi-directional LSTM (BLSTM, bidirectional long short-term memory network) is introduced into the model to learn the strong correlation between adjacent pixels of text images, thereby forming a blurred image restoration model for text.
[0043] Accordingly, the motion blurred cursive text image restoration method provided by the present invention mainly includes two stages, namely, a model training stage and a restoration stage, when implemented.
[0044] In the model training stage, the basic model is constructed and the model training is completed to obtain the fuzzy cursive text restoration model.
[0045] Specifically, in the model training stage, a data set of blurred pictures of cursive text is first generated for model training; the data set of blurred pictures of cursive text meets the application scenario and can ensure the performance of model training;
[0046] Next, a generator network model is constructed based on a combination of a convolutional neural network and a bidirectional long short-term memory network, and the constructed generator network model is trained based on the generated fuzzy cursive text image dataset to obtain a fuzzy cursive text restoration model. The fuzzy cursive text restoration model thus formed can effectively improve the effect of image restoration.
[0047] In the restoration stage, the fuzzy cursive text image is restored based on the trained fuzzy cursive text restoration model. It is only necessary to input the fuzzy cursive text image to be restored into the trained fuzzy cursive text restoration model to generate a clear image.
[0048] The following specifically describes the implementation scheme and corresponding technical features of the motion blurred cursive text image restoration method provided by the present invention.
[0049] In some embodiments of the present invention, when generating a fuzzy picture dataset of cursive text, the following steps are included:
[0050] (1) First, an open source image dataset is selected and the text content in the image is captured to obtain clear cursive text images.
[0051] (2) Next, a blur kernel size and a blur kernel jitter intensity are randomly generated, and a blur path is randomly generated based on the two randomly generated values, and then Gaussian blur is added to form a blur kernel.
[0052] (3) Then, the clear cursive text image obtained in step (1) is convolved with the blur kernel formed in step (2) to obtain a cursive text blur image dataset.
[0053] As a further explanation, when generating a data set of blurred pictures of cursive text, each clear picture can be convolved with multiple different convolution kernels to obtain multiple blurred text pictures.
[0054] The provided cursive text blurred image dataset can provide more text scenes and further improve the robustness of the text image deblurring algorithm.
[0055] In some embodiments of the present invention, when constructing a generator network model, the generator network is based on a convolutional neural network and mainly includes three parts: an input module, a feature learning module, and an image reconstruction module.
[0056] Among them, the input module in the generator network is configured to be able to subject the input blurred image (i.e., the blurred image in the cursive text blurred image dataset) to multiple convolution module operations, and obtain a large number of feature maps through convolution operations for feature learning.
[0057] As a further explanation, when the input module performs multiple convolution module operations, one of the convolution modules includes three procedures: convolution, normalization, and activation.
[0058] As a further illustration, the solution of the present invention adds a residual connection before the input module and after the output module to improve the training effect.
[0059] The feature learning module in this generator network is composed of multiple residual-bidirectional long short-term memory networks, and is configured to be able to perform feature learning on the feature map formed by the input module based on multiple residual-bidirectional long short-term memory networks.
[0060] Furthermore, the feature learning module is preferably composed of 9 residual-bidirectional long short-term memory networks (RSBLblock), and each residual-bidirectional long short-term memory network is connected through a dense connection network.
[0061] The bidirectional long short-term memory network module in this feature learning module can input the special graph in the horizontal and vertical dimensions of the feature graph when training the current feature graph, thereby learning the relationship between a certain feature value and adjacent feature values in the feature graph, so that the network can more fully consider the context information of the current feature.
[0062] When learning the feature graph, this feature learning module will be configured to perform bidirectional long short-term memory network feature extraction on the row vector and column vector of the feature graph respectively.
[0063] In addition, in order to optimize the training effect of the parameters, the 9 residual-bidirectional long short-term memory network modules in this feature learning module are connected through a densely connected network to optimize the learning and training effect. The densely connected network and the residual connection network are combined to effectively improve the back propagation of the gradient during the training process, thereby improving the accuracy of the network parameters.
[0064] As a further explanation, the feature learning module in the solution of the present invention is based on the residual network (ResNets) gradient passed from high level to low level, and innovatively further adopts densely connected networks (DenseNets) to directly connect the output of each residual-bidirectional long short-term memory network module in the feature learning module to the subsequent residual-bidirectional long short-term memory network module, thereby improving the gradient transfer between layers and further improving the propagation of the gradient.
[0065] As a further explanation, in the feature learning module in the scheme of the present invention, for each residual-bidirectional long short-term memory network module, a combination of the residual network and the bidirectional long short-term memory network is used to subject the feature map to two convolutions for feature learning, and then the feature map is input into the long short-term memory network in both the horizontal and vertical directions for feature learning, that is, the sequence of the horizontal quantities of the feature map is trained by the long short-term memory network, and then the vertical quantities of the feature map are trained by the long short-term memory network, thereby learning the strong correlation between the pixels of the text.
[0066] The image reconstruction module in this generator network is configured to interact with the feature learning module data, and can pass the feature vector (i.e., feature map) through multiple deconvolution modules to extract useful information and finally generate a clear picture.
[0067] In some embodiments of the present invention, a generative adversarial network is further formed for the generator network, thereby improving the effect of image restoration.
[0068] Furthermore, the generative adversarial network here is mainly composed of a discriminator network and a generator network.
[0069] Specifically, when training the generator network model, the restored image generated and output by the generator network and its corresponding clear image are input into the discriminator to determine whether the input image is real or fake; then a loss function is generated based on the feature distance between the clear image and the restored image and the judgment result of the discriminator; the network parameters of the generator network model and the discriminator are iterated through the loss function to optimize the network parameters until the model converges, thereby obtaining a fuzzy cursive text restoration model, thereby further improving the efficiency of image restoration.
[0070] As a further explanation, the discriminator network is constructed in the solution of the present invention, and is configured to perform high-dimensional feature extraction through multiple convolution layers, and finally perform dimensionality reduction through a convolution, and then use the Sigmoid() activation function to obtain a judgment on the authenticity of the image, thereby ensuring the accuracy of the discriminator network in distinguishing true from false.
[0071] As a further explanation, the loss function of the generator in the solution of the present invention is specifically composed of two parts: one is the adversarial loss, and the other is the content loss of image quality.
[0072] As an example, the loss function here can be constructed by the following formula:
[0073] L=L ad +λL p ;
[0074] Where L ad To generate the adversarial loss, L p is the content loss of image quality, and λ is a weighting parameter, which is adjusted according to the size of the two losses.
[0075] Furthermore, the content loss in this solution uses perceptual loss, which is specifically the distance between the feature map obtained by inputting the restored image and the clear image into a pre-trained CNN network model (such as the VGG19 model).
[0076] As an example, here the content loss of image quality is L p It can be determined by the following formula:
[0077]
[0078] Among them, Ф i,j is the feature map after the jth convolution before the i-th pooling in the network pre-trained by the VGG19 model; W i,j and H i,j The dimension of the corresponding feature map; I S For clear pictures; I B is a blurred image; G represents the deblurring operation of the generator.
[0079] Furthermore, the adversarial loss in this scheme is a universal loss function for generating adversarial networks.
[0080] As an example, here is the adversarial loss L ad It can be determined by the following formula:
[0081]
[0082] Among them, I S For a clear picture, I Bis a blurred image, and D is the discriminant operation of the discriminator.
[0083] Accordingly, when generating the loss function, the solution of the present invention extracts features of the image deblurred by the generator through a pre-trained CNN network (such as the VGG19 model), and uses the generated feature map to calculate the loss function.
[0084] In the solution of the present invention, by adding adversarial loss into the loss function, the effect of image restoration can be further improved.
[0085] Based on the motion blurred cursive text image restoration method formed by the above scheme, the cursive text is deblurred based on deep learning. Its training model uses convolutional neural network and bidirectional long short-term memory network to fully extract the special information of the text. The idea of residual and densely connected network is used before each module of the model to increase the gradient propagation between modules and improve the training effect; on this basis, the generative adversarial network is further introduced, and the discriminator's discrimination result is used to supervise the effect of the generator network, further narrowing the gap between the generated image and the real image, which can effectively improve the deblurring effect.
[0086] The motion blurred cursive text image restoration method provided by the present invention can be used as a corresponding software program to form a corresponding motion blurred cursive text image restoration system. When the software program is running, it will execute the motion blurred cursive text image restoration method and store it in a corresponding storage medium for the processor to retrieve and execute.
[0087] The motion blurred cursive text image restoration system thus formed mainly includes two parts in terms of functions: a cursive text blurred image dataset and a blurred cursive text restoration model.
[0088] Among them, the blurred image dataset of cursive text is used to train the blurred cursive text restoration model.
[0089] The fuzzy cursive text restoration model is a generator network model constructed by combining a convolutional neural network and a bidirectional long short-term memory network, and is trained based on a dataset of fuzzy cursive text images.
[0090] For the fuzzy image dataset of cursive text,
[0091] The construction of the cursive text blurred image dataset is as mentioned above and will not be elaborated here.
[0092] In some embodiments of the present invention, the generator network model used to train the fuzzy cursive text restoration model is mainly composed of an input module, a feature learning module and an image reconstruction module.
[0093] The input module is configured to be able to subject the input blurred picture of cursive text to multiple convolution module operations, and obtain a feature map through the convolution operation;
[0094] The feature learning module is composed of a plurality of residual-bidirectional long short-term memory networks, and is configured to perform feature learning on the feature map formed by the input module based on the plurality of residual-bidirectional long short-term memory networks, and learn the strong correlation of the pixels of the cursive text image;
[0095] The image reconstruction module is configured to interact with the feature learning module data, and can generate a clear picture by passing the feature map through multiple deconvolution modules.
[0096] As a further explanation, in this feature learning module, each residual-bidirectional long short-term memory network module is connected through a densely connected network.
[0097] As a further explanation, in this feature learning module, a residual connection is added between the module input and output to improve the training efficiency.
[0098] In some embodiments of the present invention, a generative adversarial network is constructed for the generator network model in the restoration system to further improve the restoration effect.
[0099] Here, the discriminator is used to form a corresponding generative adversarial network with the generator, and the discriminator's discrimination results are used to supervise the effect of the generator network, further narrowing the gap between the generated images and the real images.
[0100] Specifically, when the generator network model is trained, the restored image generated and output by the generator network model and its corresponding clear image are input into the discriminator to determine whether the input image is real or fake; then a loss function is generated based on the feature distance between the clear image and the restored image and the judgment result of the discriminator; the network parameters of the generator network model and the discriminator are iterated through the loss function to optimize the network parameters until the model converges, thereby obtaining a fuzzy cursive text restoration model.
[0101] The loss function here consists of content loss and adversarial loss. The content loss is the distance between the feature map obtained by inputting the restored image and the clear image into the pre-trained VGG19, and the adversarial loss is the general loss function of the generative adversarial network.
[0102] The implementation process of the motion blurred cursive text image restoration method provided by the present invention is further illustrated below through specific application examples.
[0103] This example uses deep learning to restore motion blurred images of cursive text. In the early stage, the blurred cursive text restoration model is trained, and in the later stage, the blurred cursive text restoration model is based on the trained one.
[0104] When constructing and training the fuzzy cursive text restoration model in the early stage, first generate a fuzzy cursive text image dataset. As an example, in this example, the following steps are used to generate a fuzzy cursive text image dataset:
[0105] (1) Extract text from open-source entity scene images;
[0106] (2) Generate a blur kernel, and generate a blur kernel path by randomizing the size and strength. The greater the strength, the greater the angle and length of the path jitter.
[0107] (3) Then, based on the blur kernel path generated in step (2), Gaussian blur is added to form a blur kernel;
[0108] (4) Based on the blur kernel formed in step (3), each image is convolved with multiple different blur kernels to obtain a generated blurry image-clear image pair, so that the corresponding data set can be used to train a text image deblurring model.
[0109] Next, after generating the corresponding dataset, the dataset is preprocessed to make it suitable for the model.
[0110] As an example, since the size of the captured data set image is relatively small, in this example the width and height of the image are proportionally enlarged or reduced to make its width 256px or height 128px; then the image is filled with a solid color around it to fill its size to 256*128px.
[0111] Next, based on the generated cursive text blurred image dataset, model training set and model testing are carried out.
[0112] For example, see Figure 1 , the specific process of model training set and model testing in this example includes the following steps:
[0113] (1) Input the blurred image of the training set into the generator to generate the restored image;
[0114] (2) Input the restored image output by the generator and the corresponding clear image into the discriminator;
[0115] (3) Based on the feature distance between the clear image and the restored image and the discrimination result output by the discriminator in step (2), a loss function is generated to iterate the parameters of the generator and the discriminator, and the network parameters are optimized until the model converges to obtain a fuzzy cursive text restoration model;
[0116] (4) Input the blurred images in the test set into the trained blurred cursive text restoration model to obtain the restored clear images.
[0117] Further, see Figure 2, which shows an example of the structure of the generator in this example. When the generator in this example processes an image, the image first passes through the input layer, which is a convolution of size 7×7 and stride 1; then it passes through two 3×3 convolutions with a stride of 2 to extract the feature map, and each convolution is followed by a normalization and activation operation; then the feature map passes through 9 densely connected RSBLblocks; and then passes through three deconvolution layers to obtain the output image.
[0118] Further, see Figure 3 , which shows an example of the structure of RSBLblock in this example. This RSBLblock uses a combination of residual network and bidirectional long short-term memory network. First, the feature map is subjected to two convolutions for feature learning, and then the feature map is input into the long short-term memory network in both horizontal and vertical directions for feature learning. That is, the order of the horizontal quantity of the feature map is trained by the long short-term memory network, and then the vertical quantity of the feature map is trained by the long short-term memory network, and a residual connection is added between the module input and output, thereby learning the strong correlation of text pixels.
[0119] Furthermore, the discriminator in this example is specifically composed of five convolution operations, which extract high-dimensional features through five convolutions to determine whether the input image is true or false.
[0120] On this basis, in this example, a generative adversarial network is formed based on the discriminator and the generator. Figure 4 As shown, it shows the network structure of the generated adversarial network formed in this example.
[0121] As shown in the figure, in this example, the blurred image is passed through the generator to obtain the restored image. The restored image and the clear image are then input into the discriminator to obtain a judgment on the authenticity of the image; then, the loss function is calculated; further, the network parameters of the generator and the discriminator are iterated through the loss function to optimize the network parameters until the model converges. The blurred cursive text restoration model is obtained.
[0122] The loss function formed here consists of content loss and adversarial loss, where the content loss is the distance between the feature map obtained by inputting the restored image and the clear image into the pre-trained VGG19 model, and the adversarial loss is the general loss function of the generative adversarial network.
[0123] Furthermore, after the training of the blurred cursive text restoration model is completed in this example, corresponding blurred cursive text images for testing are constructed based on the generated blurred cursive text image dataset to form a corresponding test set.
[0124] On this basis, the blurred images of cursive text in the test set are input into the trained fuzzy cursive text restoration model, and the fuzzy cursive text restoration model processes them to obtain the corresponding clear images. The original images corresponding to the input blurred cursive text images are then compared with the clear images restored by the fuzzy cursive text restoration model to determine the restoration performance of the fuzzy cursive text restoration model.
[0125] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0126] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0130] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0133] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0134] The above-mentioned method of the present invention, or a specific system unit, or a part of the same, is a pure software architecture, which can be arranged on a physical medium, such as a hard disk, an optical disk, or any electronic device (such as a smart phone, a computer-readable storage medium) through program code. When the machine loads the program code and executes it (such as a smart phone loading and executing it), the machine becomes a device for implementing the present invention. The above-mentioned method and device of the present invention can also be transmitted in the form of program code through some transmission media, such as cables, optical fibers, or any transmission mode. When the program code is received, loaded and executed by a machine (such as a smart phone), the machine becomes a device for implementing the present invention.
[0135] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A motion blurred cursive text image restoration method, characterized in that: include: Model training phase First, a dataset of blurred pictures of cursive text is generated. A generator network model is constructed based on a combination of a convolutional neural network and a bidirectional long short-term memory network, and the generator is supervised by a generative adversarial network. On this basis, the constructed generator network model is trained based on the generated dataset of blurred pictures of cursive text to obtain a blurred cursive text restoration model. Recovery Phase Input the blurred cursive text image to be restored into the trained blurred cursive text restoration model to generate a clear image.
2. The method for restoring motion blurred cursive text images according to claim 1, characterized in that: When generating a data set of blurred pictures of cursive text, the method includes: (1) Select an open source image dataset, extract the text content in the image in the dataset, and obtain a clear cursive text image; (2) Randomly generate the blur kernel size and blur kernel jitter intensity, and randomly generate a blur path based on these two values, and then add Gaussian blur to form a blur kernel; (3) Convolve the clear cursive text image obtained in step (1) with the blur kernel formed in step (2) to obtain a cursive text blur image dataset.
3. The method for restoring motion blurred cursive text images according to claim 1, characterized in that: The generator network model constructed by the method mainly includes: An input module, wherein the input module is configured to perform multiple convolution module operations on the input blurred picture of cursive text to obtain a feature map through the convolution operation; A feature learning module, wherein the feature learning module is composed of a plurality of residual-bidirectional long short-term memory networks, and is configured to perform feature learning on a feature map formed by an input module based on the plurality of residual-bidirectional long short-term memory networks, and learn the strong correlation of pixel points of a cursive text image; The image reconstruction module is configured to interact with the feature learning module data and can generate a clear image by passing the feature vector through multiple deconvolution modules based on the correlation determined by the feature learning module.
4. The method for restoring motion blurred cursive text images according to claim 3, characterized in that: Each residual-bidirectional long short-term memory network module in the feature learning module is connected through a dense connection network.
5. The method for restoring motion blurred cursive text images according to claim 3, characterized in that: In the generator network model, residual connections are added before the input module and after the output module.
6. The method for restoring motion blurred cursive text images according to claim 3, characterized in that: In the model training stage, when the generator network model is trained, the restored image generated and output by the generator network model and its corresponding clear image are input into the discriminator to judge the authenticity of the input image; then, based on the feature distance between the clear image and the restored image and the judgment result of the discriminator, a loss function is generated; and then the network parameters of the generator network model and the discriminator are iterated through the loss function to optimize the network parameters until the model converges, thereby obtaining a fuzzy cursive text restoration model.
7. Motion blurred cursive text image restoration system, characterized by: include: Cursive text blurred image dataset; A fuzzy cursive text restoration model, wherein the fuzzy cursive text restoration model is constructed by combining a convolutional neural network and a bidirectional long short-term memory network to form a generator network model, and is trained based on a fuzzy cursive text image dataset.
8. The motion blurred cursive text image restoration system according to claim 7, characterized in that: The generator network model mainly includes: An input module, wherein the input module is configured to perform multiple convolution module operations on the input blurred picture of cursive text to obtain a feature map through the convolution operation; A feature learning module, wherein the feature learning module is composed of a plurality of residual-bidirectional long short-term memory networks, and is configured to perform feature learning on a feature map formed by an input module based on the plurality of residual-bidirectional long short-term memory networks, and learn the strong correlation of pixel points of a cursive text image; The image reconstruction module is configured to interact with the feature learning module data and can generate a clear image by passing the feature map through multiple deconvolution modules.
9. The motion blurred cursive text image restoration system according to claim 8, characterized in that: Each residual-bidirectional long short-term memory network module in the feature learning module is connected through a dense connection network.
10. The motion blurred cursive text image restoration system according to claim 7 or 8, characterized in that: In the restoration system, a generative adversarial network is constructed for the generator network model.