Character repairing method and product based on conditional generative adversarial network

Abstract

The embodiment of the invention provides a character restoration method and device based on a conditional generative adversarial network, equipment, a medium and a product, relates to the technical field of artificial intelligence, and can be applied to the field of financial science and technology. The method comprises the steps of obtaining a training data set; a conditional generative adversarial network is constructed, the conditional generative adversarial network comprises a first generator, a second generator and a decision device, and input data in the training process of the first generator are a first font type and a fuzzy character image of the first font type; input data in the training process of the second generator are a second font type and a fuzzy character image of the second font type, and output data of the first generator and the second generator, clear character images corresponding to the output data and the font type are input data of the decision device; training a condition generation adversarial network based on the training data set; and inputting a to-be-repaired character image into the trained conditional generative adversarial network to obtain a repaired character image. According to the method, the fuzzy character repairing effect is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text repair method, device, equipment, computer-readable storage medium and computer program product based on a conditional generative adversarial network. Background Art

[0002] The current daily work of the banking industry involves a large number of scenarios such as customer signatures and document materials printed through devices. In the scenario of customer signatures, it is easy for some customers to have sloppy handwriting or use special fonts when signing, resulting in blurred signatures that are difficult to identify; or when the bank provides signature services, if the signature area is not well lit, the paper quality is poor, or the signature handwriting is easy to dry, the signature may also be blurred. If the signature is blurred, it is difficult for bank staff to accurately identify it, which may increase the risk of fraud such as forgery or forgery, thereby threatening the security and legality of the transaction. When the customer's signature is blurred, bank staff may need to spend more time to compare and confirm, and may even require the customer to sign again, which will undoubtedly reduce the efficiency of business processing and increase customer waiting time.

[0003] When printing documents, there may be special situations where the printed materials are blurred or intermittent due to equipment failure. Blurred printed materials require more time to check and confirm, and some materials may not be reprinted. Blurred handwriting may lead to misunderstanding or omission of information, which in turn affects the accuracy and compliance of business processing.

[0004] Therefore, in view of the above-mentioned situation in the banking industry, how to more effectively repair blurred text has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] The embodiments of the present application provide a text repair method, apparatus, device, computer-readable storage medium and computer program product based on a conditional generative adversarial network, which has a good text repair effect on blurred text.

[0006] In a first aspect, an embodiment of the present application provides a text restoration method based on a conditional generative adversarial network, comprising:

[0007] Acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image;

[0008] Constructing a conditional generative adversarial network, the conditional generative adversarial network comprising a first generator, a second generator and a judge, the first generator and the second generator share the judge, the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge;

[0009] Iteratively training the conditional generative adversarial network based on the training data set;

[0010] The text image to be repaired and the target font type of the text image to be repaired are input into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type; wherein the target font type is the first font type or the second font type.

[0011] In a second aspect, an embodiment of the present application provides a text repair device based on a conditional generative adversarial network, comprising:

[0012] A data acquisition module, used to acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image;

[0013] A network construction module, used to construct a conditional generative adversarial network, wherein the conditional generative adversarial network includes a first generator, a second generator and a judge, wherein the first generator and the second generator share the judge, wherein the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, wherein the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, wherein the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge;

[0014] A network training module, used for iteratively training the conditional generative adversarial network based on the training data set;

[0015] A text repair module is used to input the text image to be repaired and the target font type of the text image to be repaired into the trained conditional generative adversarial network to obtain a repaired text image that conforms to the target font type; wherein the target font type is the first font type or the second font type.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the text repair method based on the conditional generative adversarial network provided in the first aspect of the embodiment of the present application are implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the text repair method based on the conditional generative adversarial network provided in the first aspect of the embodiment of the present application are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product on which a computer program is stored. When the computer program is executed by a processor, the steps of the text repair method based on the conditional generative adversarial network provided in the first aspect of the embodiment of the present application are implemented.

[0019] The technical solution provided in the embodiment of the present application obtains a training data set, constructs a conditional generative adversarial network, and the conditional generative adversarial network includes a first generator, a second generator and a judge. The first generator and the second generator share the judge. The conditional generative adversarial network is iteratively trained using the training data set, and the trained conditional generative adversarial network is used to repair the text image to be repaired. The conditional generative adversarial network can capture more abstract and robust feature representations when processing blurred text, and can retain the detail information in the image to the greatest extent, thereby improving the blurred text repair effect; and the conditional generative adversarial network includes two parallel generators and a shared judge. This structure can repair blurred text in two different font types, meeting the text repair needs of different font types in the banking industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an application environment of a text repair method based on a conditional generative adversarial network provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a text repair method based on a conditional generative adversarial network provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of a training data set acquisition process provided in an embodiment of the present application;

[0023] Figure 4 Another flowchart of the training data set acquisition process provided in the embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of a text repair device based on a conditional generative adversarial network provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application are further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Those skilled in the art can make adjustments to them as needed to suit specific application scenarios. It should also be noted that, for ease of description, only the parts related to the present application are shown in the accompanying drawings, rather than all structures.

[0027] Figure 1 A schematic diagram of the application environment of the text repair method based on the conditional generative adversarial network provided in the embodiment of the present application. Figure 1 , the application environment may include a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless network or a wired network. The terminal 101 may be a desktop terminal or a mobile terminal. Optionally, the mobile terminal may be at least one of a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (such as a vehicle navigation terminal), and a mobile phone. The server 102 may be an independent server or a server cluster consisting of multiple servers.

[0028] It should be noted that the text repair method based on the conditional generative adversarial network in the embodiment of the present application can be executed separately by the terminal 101 or the server 102, or can be jointly executed by the terminal 101 and the server 102. The following method embodiment is described by taking the execution subject as an electronic device (the electronic device is the terminal 101 and / or the server 102) as an example.

[0029] Figure 2 A flowchart of a text repair method based on a conditional generative adversarial network provided in an embodiment of the present application. Figure 2 As shown, the method may include:

[0030] S201. Obtain a training data set.

[0031] The training data set includes clear text images of a first font type and corresponding fuzzy text images, and clear text images of a second font type and corresponding fuzzy text images.

[0032] A clear text image refers to an image containing clear text, that is, the text in the image is clear. A blurred text image refers to an image containing blurred text, that is, the text in the image is blurred. In order to adapt to the two more common text scenarios in the banking industry, handwriting and printing, the first font type can be set to handwriting, and the second font type can be set to printing. Of course, the above font types can also be divided into other forms, for example, official script, small seal script, Song style or Fangsong style, etc., which is not limited in the embodiments of the present application.

[0033] For handwriting, you can select an open source handwriting dataset as a clear text image. For printed text, use a general printed Chinese character dataset and an open source printed Chinese character dataset as a clear text image. Of course, you can also use an open source image processing library to automatically generate images containing clear text. These texts can be randomly selected texts to ensure the diversity of clear text images. In addition, you can also process the clear text image through corresponding technical means to obtain the fuzzy text image corresponding to the clear text image, thereby forming a training data set.

[0034] S202: Construct a conditional generative adversarial network.

[0035] Among them, the conditional generative adversarial network includes a first generator, a second generator and a judge, the first generator and the second generator share the judge, the input data of the first generator during the training process is the first font type and the blurred text image of the first font type, the input data of the second generator during the training process is the second font type and the blurred text image of the second font type, the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge.

[0036] The first generator is used to extract features from a random noise vector, conditional information (i.e., a first font type) and a blurred text image of the first font type, and generate a restored text image that conforms to the first font type based on the extracted features. The second generator is used to extract features from a random noise vector, conditional information (i.e., a second font type) and a blurred text image of the second font type, and generate a restored text image that conforms to the second font type based on the extracted features. The judge is used to distinguish whether the input text image is a real clear text image or a restored text image generated by the generator.

[0037] The above random noise vector: as the initial input of the first generator and the second generator, is usually a low-dimensional vector that obeys a normal distribution.

[0038] The above-mentioned condition information: indicates the font type (such as the first font type or the second font type) of the blurred text image to be repaired, and is represented in a one-hot encoding or other form.

[0039] The above-mentioned blurred text image: serves as a conditional input of the generator (including the first generator and the second generator), providing blur information of the text to be repaired.

[0040] Optionally, the first generator and the second generator sequentially include a plurality of convolutional layers, upsampling layers, and activation function layers, for gradually extracting features from the input and generating a repaired text image, wherein the repaired text image is the same size as the input blurred text image, but the text in the image is clearer and conforms to the specified font type.

[0041] Optionally, in the early layers of the generator, the conditional information (i.e., font type) and the random noise vector are fused to guide the generator to generate corresponding text according to the font type. This fusion method ensures that the generator can perform targeted repairs according to different font types. The early layers here can be understood as the first few layers of the generator.

[0042] Optionally, the discriminator adopts a convolutional neural network structure, which includes multiple convolutional layers, pooling layers and fully connected layers in sequence, for extracting features of the input image. In the input layer or early layer of the discriminator, the conditional information and image data are fused to distinguish text images of different font types. The output result of the discriminator is a scalar value indicating the probability that the input image is a real and clear text image. The closer the value is to 1, the more likely the input image is real; the closer the value is to 0, the more likely it is generated.

[0043] S203, iteratively training conditional generative adversarial network based on the training data set.

[0044] During the training process, the first generator, the second generator, and the judge in the adversarial network are iterated based on the training data set until the preset training rounds or convergence conditions are reached. At this point, the generator can generate sufficiently realistic and font-compliant restored text images, and the judge has difficulty distinguishing between real text and generated text.

[0045] Optionally, the process of iteratively training a conditional generative adversarial network based on a training data set may include the following steps:

[0046] Step 1: Initialize the parameters of the first generator, the second generator and the judge.

[0047] Step 2: Fix the parameters of the first generator and the second generator, take the output data of the first generator and the second generator as negative samples, the clear text images corresponding to the output data as positive samples, and the font type corresponding to the output data as conditional information, and train the discriminator. The training goal is to maximize the classification ability of the discriminator for the output data and the clear text images corresponding to the output data.

[0048] Specifically, the training process of the decision maker may include:

[0049] Forward propagation: the random noise vector, conditional information (i.e., the first font type) and the blurred text image of the first font type are input into the first generator, and the first generator outputs a repaired text image that conforms to the first font type; the random noise vector, conditional information (i.e., the second font type) and the blurred text image of the second font type are input into the second generator, and the second generator outputs a repaired text image that conforms to the second font type.

[0050] Back propagation: The judge receives a restored text image that conforms to the first font type, a restored text image that conforms to the second font type, conditional information (i.e., the first font type and the second font type), and a corresponding clear text image, extracts features through a convolutional neural network, and outputs a probability value that the input text image is a real clear text image; based on the probability value obtained above, the loss value of the loss function of the judge is determined, and the parameters of the judge are adjusted based on the loss value to better distinguish between real text and generated text.

[0051] Step 3: Fix the parameters of the judge, train the first generator based on the judgement result of the judge on the output data of the first generator, and train the second generator based on the judgement result of the judge on the output data of the second generator. The training goal is to minimize the probability that the judge judges the output data of the first generator and the output data of the second generator as false samples.

[0052] The training process of the generator can include:

[0053] Forward propagation: When the decision parameter is fixed, the random noise vector, conditional information (i.e., the first font type) and the blurred text image of the first font type are input into the first generator, and the first generator outputs a repaired text image that conforms to the first font type; the random noise vector, conditional information (i.e., the second font type) and the blurred text image of the second font type are input into the second generator, and the second generator outputs a repaired text image that conforms to the second font type.

[0054] Back propagation: According to the judgment result of the judge on the repaired text image generated by the first generator and the loss function of the first generator, the parameters of the first generator are adjusted through the optimization algorithm to improve the generation effect of the first generator; according to the judgment result of the judge on the repaired text image generated by the second generator and the loss function of the second generator, the parameters of the second generator are adjusted through the optimization algorithm to improve the generation effect of the second generator.

[0055] The training process of the discriminator, the first generator and the second generator is executed repeatedly until a preset training round or convergence condition is reached. At this point, the generator can generate sufficiently realistic restored text images, and the discriminator has difficulty distinguishing between real text and generated text.

[0056] S204: Input the text image to be repaired and the target font type of the text image to be repaired into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type.

[0057] The text image to be repaired refers to an image that contains blurred text and needs to be repaired. In the banking industry, the text image to be repaired can be a text image that is blurred due to handwriting, printing quality, or storage conditions. These images can be repaired through the trained conditional generative adversarial network to obtain clear text content.

[0058] Specifically, the text image to be repaired and the target font type corresponding to the text image to be repaired can be input into the trained conditional generative adversarial network, and the corresponding generator can be used to generate the corresponding repaired text image. The target font type here can be the first font type or the second font type. For example, if the target font type corresponding to the text image to be repaired is the first font type, then the text image to be repaired and the first font type can be input into the conditional generative adversarial network, and the first generator can be used to generate a repaired text image that conforms to the first font type. For another example, if the target font type corresponding to the text image to be repaired is the second font type, then the text image to be repaired and the second font type can be input into the conditional generative adversarial network, and the second generator can be used to generate a repaired text image that conforms to the second font type.

[0059] The text repair method based on conditional generative adversarial network provided in the embodiment of the present application obtains a training data set, constructs a conditional generative adversarial network, and the conditional generative adversarial network includes a first generator, a second generator and a judge. The first generator and the second generator share the judge. The conditional generative adversarial network is iteratively trained using the training data set, and the trained conditional generative adversarial network is used to repair the text image to be repaired. The conditional generative adversarial network can capture more abstract and robust feature representations when processing blurred text, and can retain the detail information in the image to the greatest extent, thereby improving the blurred text repair effect; and the conditional generative adversarial network includes two parallel generators and a shared judge. This structure can repair blurred text in two different font types, meeting the text repair needs of different font types in the banking industry.

[0060] In one embodiment, optionally, Figure 3 As shown, the above-mentioned acquisition of the training data set may include:

[0061] S301, obtaining an image data set; the image data set includes clear text images of two font types.

[0062] The two font types here refer to a first font type and a second font type. Optionally, the first font type may be handwriting, and the second font type may be printing.

[0063] In the specific implementation, an open source image processing library is used to automatically generate images containing clear texts in two font types. These texts can be randomly selected texts to ensure the diversity of clear text images.

[0064] S302: Blurring the clear text image to obtain a corresponding blurred text image.

[0065] Exemplarily, an image masking method may be used to mask a partial area of ​​a clear text image, thereby obtaining a corresponding blurred text image.

[0066] As an optional implementation, Figure 4 As shown, the above S302 may include:

[0067] S3021. Grayscale the clear text image.

[0068] The grayscale processing methods here include but are not limited to component method, maximum value method, average value method, weighted average method, etc. By grayscale processing the clear text image, the training efficiency of the conditional generative adversarial network can be improved because the grayscale image processing speed is faster.

[0069] S3022, generating a non-uniform blur kernel, and using the non-uniform blur kernel to blur the grayscale clear text image to obtain a corresponding blurred text image.

[0070] Specifically, some different non-uniform blur kernels can be generated by artificial sampling, and the corresponding blurred text images can be generated by convolving the non-uniform blur kernels with the grayscale clear text image. The non-uniform blur kernel here can be understood as different blur kernels in different areas of the image. In this way, a variety of blurred text images can be generated.

[0071] Optionally, after the above S3022, the following operations S3023-S3025 may also be included:

[0072] S3023. Perform a convolution operation on the blurred text image using a Laplace operator to obtain a response graph.

[0073] S3024. Determine the variance of the response graph.

[0074] Specifically, the Laplacian operator can be a 3*3 convolution kernel, through which the gradient change of pixels around each pixel in the blurred text image is calculated to detect whether the blur degree meets the requirements. The Laplacian operator is convolved with the blurred text image to obtain a response map, and the variance of the response map is calculated. If the variance is lower than a preset threshold, it can be considered that the blurred text image meets the requirements, and the obtained blurred text image is added to the training data set.

[0075] S3025. If the variance is higher than the preset threshold, adjust the parameters of the non-uniform blur kernel, and based on the adjusted non-uniform blur kernel, continue to use the non-uniform blur kernel to blur the grayscale clear text image to obtain the corresponding blurred text image, until the variance is lower than the preset threshold, and the final blurred text image is obtained.

[0076] That is, when the variance of the response graph is higher than the preset threshold, it can be considered that the blurred text image does not meet the requirements. At this time, the parameters of the non-uniform blur kernel can be adjusted, and based on the adjusted non-uniform blur kernel, the grayscale clear text image is continued to be blurred until the variance of the response graph is lower than the preset threshold, thereby obtaining the final blurred text image. Among them, the above-mentioned preset threshold can be set based on actual needs, and this embodiment does not specifically limit the value of the preset threshold.

[0077] By performing variance verification and parameter adjustment on the generated blurred text images, the quality of the blurred text images can be improved. High-quality blurred text images help improve the training effect of the conditional generative adversarial network, and thus improve the blurred text repair effect.

[0078] S303: Combine the clear text image and the fuzzy text image to obtain a target data set.

[0079] The clear text images and the corresponding blurred text images are paired, and each pair of images includes a clear version and a blurred version of the same text content, thereby forming a target dataset.

[0080] S304: Perform data enhancement on the target data set to obtain a training data set.

[0081] In order to further enrich the target dataset, data enhancement can be performed on the target dataset, including image rotation, scaling or cropping, etc., to increase the diversity of the training dataset and thus improve the robustness of the conditional generative adversarial network.

[0082] In this embodiment, the clear text image is blurred by a non-uniform blur kernel, which enriches the data volume of the blurred text image and increases the diversity of the training data, which helps to improve the training effect of the conditional generative adversarial network and further improves the blurred text repair effect.

[0083] Figure 5 A structural diagram of a text repair device based on a conditional generative adversarial network provided in an embodiment of the present application. Figure 5 As shown, the device may include: a data acquisition module 501, a network construction module 502, a network training module 503 and a text repair module 504.

[0084] Specifically, the data acquisition module 501 is used to acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image;

[0085] The network construction module 502 is used to construct a conditional generative adversarial network, which includes a first generator, a second generator and a judge. The first generator and the second generator share the judge. The input data of the first generator during the training process is the first font type and the blurred text image of the first font type. The input data of the second generator during the training process is the second font type and the blurred text image of the second font type. The output data of the first generator and the second generator are the input data of the judge. The clear text image and the font type corresponding to the output data of the first generator and the second generator are the input data of the judge.

[0086] The network training module 503 is used to iteratively train the conditions to generate an adversarial network based on the training data set;

[0087] The text repair module 504 is used to input the text image to be repaired and the target font type of the text image to be repaired into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type; wherein the target font type is the first font type or the second font type.

[0088] Based on the above embodiment, optionally, the data acquisition module 501 is specifically used to acquire an image data set; the image data set includes clear text images of two font types; the clear text images are blurred to obtain corresponding blurred text images; the clear text images and the blurred text images are combined to obtain a target data set; and the target data set is data enhanced to obtain a training data set.

[0089] On the basis of the above embodiment, optionally, the data acquisition module 501 is also used to grayscale the clear text image; generate a non-uniform blur kernel, and use the non-uniform blur kernel to blur the grayscale clear text image to obtain a corresponding blurred text image.

[0090] On the basis of the above embodiment, optionally, the data acquisition module 501 is also used to perform a convolution operation on the blurred text image through a Laplace operator to obtain a response map; determine the variance of the response map; if the variance is higher than a preset threshold, adjust the parameters of the non-uniform blur kernel, and based on the adjusted non-uniform blur kernel, continue to use the non-uniform blur kernel to blur the grayscale clear text image to obtain the corresponding blurred text image, until the variance is lower than the preset threshold, and the final blurred text image is obtained.

[0091] Optionally, the first generator and the second generator include multiple convolutional layers, upsampling layers and activation function layers in sequence; the judge includes multiple convolutional layers, pooling layers and fully connected layers in sequence.

[0092] On the basis of the above embodiment, optionally, the network training module 503 is also used to initialize the parameters of the first generator, the second generator and the judge; fix the parameters of the first generator and the second generator, take the output data of the first generator and the second generator as negative samples, the clear text images corresponding to the output data as positive samples, and the font type corresponding to the output data as conditional information, and train the judge, wherein the training goal is to maximize the judge's classification ability for the output data and the clear text images corresponding to the output data; fix the parameters of the judge, train the first generator based on the judgement result of the judge on the output data of the first generator, and train the second generator based on the judgement result of the judge on the output data of the second generator, wherein the training goal is to minimize the probability that the judge judges the output data of the first generator and the output data of the second generator as false samples.

[0093] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of the processor 60 in the device can be one or more. Figure 6 A processor 60 is taken as an example; the processor 60, memory 61, input device 62 and output device 63 in the device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0094] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the text repair method based on conditional generative adversarial networks in the embodiments of the present application (for example, the data acquisition module 501, the network construction module 502, the network training module 503 and the text repair module 504 in the text repair device based on conditional generative adversarial networks). The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the text repair method based on conditional generative adversarial networks provided in any of the above embodiments.

[0095] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created during the text repair process based on the conditional generative adversarial network, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device / terminal / server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 63 may include a display device such as a display screen.

[0097] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0098] Acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image;

[0099] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a first generator, a second generator and a judge, the first generator and the second generator share the judge, the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge;

[0100] Iteratively train conditional generative adversarial networks based on training datasets;

[0101] The text image to be repaired and the target font type of the text image to be repaired are input into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type; wherein the target font type is the first font type or the second font type.

[0102] In one embodiment, a computer program product is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0103] Acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image;

[0104] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a first generator, a second generator and a judge, the first generator and the second generator share the judge, the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge;

[0105] Iteratively train conditional generative adversarial networks based on training datasets;

[0106] The text image to be repaired and the target font type of the text image to be repaired are input into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type; wherein the target font type is the first font type or the second font type.

[0107] The text repair device based on conditional generative adversarial network, electronic device, computer-readable storage medium and computer program product provided in the above embodiments can execute the text repair method based on conditional generative adversarial network provided in any embodiment of the present application, and have the corresponding functional modules and beneficial effects of executing the method. For technical details not described in detail in the above embodiments, please refer to the text repair method based on conditional generative adversarial network provided in any embodiment of the present application.

[0108] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0109] It is worth noting that the various units and modules included in the above embodiments are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0110] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A text restoration method based on conditional generative adversarial network, characterized in that: include: Acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image; Constructing a conditional generative adversarial network, the conditional generative adversarial network comprising a first generator, a second generator and a judge, the first generator and the second generator share the judge, the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge; Iteratively training the conditional generative adversarial network based on the training data set; The text image to be repaired and the target font type of the text image to be repaired are input into the trained conditional generative adversarial network to obtain a repaired text image that meets the target font type; wherein the target font type is the first font type or the second font type.

2. The method according to claim 1, characterized in that The step of obtaining a training data set includes: Acquire an image data set; the image data set includes clear text images of two font types; Performing blur processing on the clear text image to obtain a corresponding blurred text image; Combining the clear text image and the fuzzy text image to obtain a target data set; The target data set is enhanced to obtain a training data set.

3. The method according to claim 2, characterized in that The blurring the clear text image to obtain a corresponding blurred text image includes: grayscale the clear text image; A non-uniform blur kernel is generated, and the grayscaled clear text image is blurred using the non-uniform blur kernel to obtain a corresponding blurred text image.

4. The method according to claim 3, characterized in that Also includes: Performing a convolution operation on the blurred text image through a Laplace operator to obtain a response graph; determining a variance of the response graph; If the variance is higher than the preset threshold, the parameters of the non-uniform blur kernel are adjusted, and based on the adjusted non-uniform blur kernel, the step of blurring the grayscaled clear text image using the non-uniform blur kernel to obtain the corresponding blurred text image is continued until the variance is lower than the preset threshold to obtain the final blurred text image.

5. The method according to claim 1, characterized in that The first generator and the second generator sequentially include a plurality of convolutional layers, upsampling layers, and activation function layers; The decision device includes multiple convolutional layers, pooling layers and fully connected layers in sequence.

6. The method according to claim 1, characterized in that The iterative training of the conditional generative adversarial network based on the training data set includes: Initialize parameters of the first generator, the second generator, and the judgement device; The parameters of the first generator and the second generator are fixed, the output data of the first generator and the second generator are used as negative samples, the clear text images corresponding to the output data are used as positive samples, and the font type corresponding to the output data is used as conditional information, and the discriminator is trained, wherein the training goal is to maximize the classification ability of the discriminator for the output data and the clear text images corresponding to the output data; The parameters of the judger are fixed, and the first generator is trained based on the judgment result of the judger on the output data of the first generator, and the second generator is trained based on the judgment result of the judger on the output data of the second generator. The training goal is to minimize the probability that the judger judges the output data of the first generator and the output data of the second generator as false samples.

7. A text repair device based on conditional generative adversarial network, characterized in that: include: A data acquisition module, used to acquire a training data set; wherein the training data set includes a clear text image of a first font type and a corresponding fuzzy text image, and a clear text image of a second font type and a corresponding fuzzy text image; A network construction module, used to construct a conditional generative adversarial network, wherein the conditional generative adversarial network includes a first generator, a second generator and a judge, wherein the first generator and the second generator share the judge, wherein the input data of the first generator during the training process is a first font type and a blurred text image of the first font type, wherein the input data of the second generator during the training process is a second font type and a blurred text image of the second font type, wherein the output data of the first generator and the second generator are the input data of the judge, and the clear text image and font type corresponding to the output data of the first generator and the second generator are the input data of the judge; A network training module, used for iteratively training the conditional generative adversarial network based on the training data set; A text repair module is used to input the text image to be repaired and the target font type of the text image to be repaired into the trained conditional generative adversarial network to obtain a repaired text image that conforms to the target font type; wherein the target font type is the first font type or the second font type.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images