An intelligent generation method of an artistic painting

Abstract

This invention discloses an intelligent method for generating artwork. The method includes the following steps: constructing a deep convolutional generative adversarial network (DGAN); inputting an image dataset into the DGAN for network training, and saving the generator module with the lowest FID value in the output image; using the generator module to input random Gaussian noise to generate and output the image; performing super-resolution upscaling on the output image, enlarging it to twice its original size; constructing a Gaussian filtering module to perform Gaussian filtering on the upscaled image from step S3 to reduce the image sharpening effect; repeating steps S3 and S4 several times to obtain a high-resolution artwork of a set size, and displaying it on a monitor. The intelligent artwork generation method provided by this invention solves the problems of low clarity, cluttered images, and poor image quality in large-size images generated by traditional deep convolutional generative adversarial networks.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method for intelligently generating artistic paintings. Background Technology

[0002] With the advancement of science and the improvement of computing power, artificial intelligence has developed rapidly. In the field of computer vision, since the emergence of Generative Adversarial Networks (GANs) in 2014, their neural network models have been widely used in image generation. In order to reduce the number of network parameters and improve the image generation effect, a variant of GAN—Deep Convolutional Generative Adversarial Network (DCGAN)—was proposed in 2015. DCGAN networks have been proven to outperform traditional GAN ​​networks in terms of image generation quality on multiple occasions. However, both traditional GAN ​​networks and DCGANs generate relatively small images, mostly 64*64 and 128*128. If they are used to generate large-sized images, it is easy to make it difficult to train the adversarial relationship between the generator module and the discriminator module. Therefore, the generated large-sized images have low clarity, messy images, and poor image quality. For the generation of high-resolution images, existing technologies employ a method of cascading multiple generative adversarial networks to amplify the resolution of the generated images. However, this cascading method also results in excessively deep and complex networks, making them more difficult to train than single networks. Furthermore, it significantly increases the number of network parameters, places excessive demands on equipment, and makes reproduction difficult (CN111563841A - A high-resolution image generation method based on generative adversarial networks). Summary of the Invention

[0003] The purpose of this invention is to address the problems of low resolution, cluttered images, and poor image quality in large-size images generated by deep convolutional generative adversarial networks (DCGANs). It proposes a solution for an intelligent method of generating artistic paintings. By using a small-size image output from a trained generator and then repeatedly performing super-resolution upscaling and Gaussian filtering on the image, a high-resolution image can be generated. The image generated by this system is clearer and of higher quality than large-size images generated simply using DCGANs.

[0004] The objective of this invention is achieved by at least one of the following technical solutions.

[0005] A method for intelligently generating artwork includes the following steps:

[0006] S1. Construct a deep convolutional generative adversarial network consisting of a generator module and a discriminator module, and set the network learning rate and the number of batches of data to be processed. The generator module takes randomly generated noise as input, and outputs the image of the last sample after multiple deconvolutions. The discriminator module judges the authenticity of the image output by the generator module.

[0007] S2. Input the image dataset into the deep convolutional generative adversarial network for network training, and save the generator module with the lowest FID (Fréchet Inception Distance) value of the output image; the image dataset is a pre-stored set of multiple images or a set of multiple images acquired by an image acquisition device;

[0008] S3. Use the generator module saved in step S2 to input random Gaussian noise to complete the generation and output of the image. Then, perform super-resolution upscaling on the output image to double the size of the original output image.

[0009] S4. Construct a Gaussian filter module to perform Gaussian filtering on the magnified image in step S3 to reduce the image sharpening effect.

[0010] S5. Repeat steps S3 and S4 several times on the image after Gaussian filtering in step S4 to obtain a high-resolution artwork of a set size, and display it on the monitor.

[0011] Furthermore, in step S1, the generator module uses multiple deconvolutional layers to upsample and enlarge the image, ultimately amplifying the random noise into an output image of a set size; the discriminator module uses multiple convolutional layers, taking real image samples and image samples output by the generator module as input, and outputs a numerical value, the magnitude of which represents the probability that the image sample output by the generator module is a real image sample.

[0012] Furthermore, in step S1, both the generator module and the discriminator module use batch normalization to address the vanishing gradient problem during training and accelerate module convergence.

[0013] Furthermore, in step S1, the generator module uses the ReLU function as the activation function, with the following expression:

[0014]

[0015] The last convolutional layer in the generator module uses the tanh function, expressed as follows:

[0016]

[0017] Furthermore, in step S1, the discriminator module uses the LeakyReLU function as the activation function, with the following expression:

[0018]

[0019] Where α is a fixed hyperparameter;

[0020] In the discriminator module, the last convolutional layer uses the sigmoid function, expressed as follows:

[0021]

[0022] Furthermore, the objective function Loss of the adversarial generative network is:

[0023]

[0024] Where D(·) is the discriminator; G(·) is the generator; p data (x) represents the true image distribution; p z (z) represents a Gaussian distribution; E(·) denotes the calculation of the expected value. This represents taking the logarithm of the discriminator module's output and calculating its expectation, where x is the real image and follows the distribution p of the real images. data (x), denoted as x~p data (x), This represents the process of inverting, incrementing, and logarithming the output of the discriminator, followed by expectation calculation. Here, G(z) represents the output image obtained by inputting random Gaussian noise into the generator, where z is the random Gaussian noise following a Gaussian distribution p. z (z), denoted as z~p z (z).

[0025] Furthermore, in step S3, the size is super-resolution enlarged using Waifu2x super-resolution software.

[0026] Furthermore, in step S4, the image with magnified resolution is filtered using a Gaussian filter kernel with a set radius.

[0027] A computer device includes a processor and a storage medium, wherein a computer program is stored on the storage medium, and the processor executes the computer program to implement the aforementioned method for intelligent generation of artistic paintings.

[0028] An intelligent art painting generation device includes an image acquisition device, a display device, and a computer device. The image acquisition device and the display device are respectively communicatively connected to the computer device. The image acquisition device stores captured image data in the storage medium of the computer device. When the processor of the computer device executes the intelligent art painting generation method, the display device displays the method operation flow and results, and displays a high-resolution art painting image.

[0029] Compared with the prior art, the advantages of this invention are:

[0030] This invention uses a deep convolutional generative adversarial network to generate low-resolution images before upscaling them, avoiding the training difficulties that arise when directly generating high-resolution images. This reduces the difficulty of achieving adversarial balance between the generator and discriminator modules. Furthermore, Gaussian filtering is incorporated into the repeated upscaling operation. When the image resolution is low, a small-radius Gaussian filter kernel is used because upscaling at this resolution can easily lead to severe sharpening, while using a large-radius Gaussian filter on a low-resolution image can cause blurring and reduce image quality. Therefore, a small-radius Gaussian filter reduces sharpening and distortion. When the image resolution is upscaled to a certain level, a large-radius Gaussian filter is used because the image size is larger, and a large-radius Gaussian filter reduces noise and improves image quality. The image generated based on this invention still has a small FID (Fréchet Inception Distance) value, indicating that the generated image is similar to the real image and has good generation quality. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a method for intelligently generating artistic paintings according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the generator module structure of the deep convolutional generative adversarial network in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the discriminator module structure of a deep convolutional generative adversarial network in an embodiment of the present invention. Detailed Implementation

[0034] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings.

[0035] A method for intelligently generating artwork, such as Figure 1 As shown, it includes the following steps:

[0036] S1. In this embodiment, a deep convolutional generative adversarial network consisting of a generator module and a discriminator module is constructed using Python in the PyCharm integrated development environment and stored in a computer-readable storage medium. The network learning rate and the number of batches of data to be processed are set. The generator module takes randomly generated noise as input, and outputs the image of the last sample after multiple deconvolutions. The discriminator module determines the authenticity of the image output by the generator module.

[0037] In this embodiment, as Figure 2 , Figure 3 As shown;

[0038] The generator module uses multiple deconvolutional layers to upsample and enlarge the image, ultimately amplifying random noise of dimension 100 into an output image of size 128*128. The discriminator module uses multiple convolutional layers, taking real image samples and image samples output by the generator module as input, and outputs a numerical value, the magnitude of which represents the probability that the image sample output by the generator module is a real image sample.

[0039] Both the generator and discriminator modules use batch normalization to address the vanishing gradient problem during training and accelerate module convergence.

[0040] The generator module uses the ReLU function as the activation function, as shown in the following expression:

[0041]

[0042] The last convolutional layer in the generator module uses the tanh function, expressed as follows:

[0043]

[0044] The discriminator module uses the LeakyReLU function as the activation function, as shown in the following expression:

[0045]

[0046] Wherein, α is a fixed hyperparameter, and in this embodiment, α is set to 0.2;

[0047] In the discriminator module, the last convolutional layer uses the sigmoid function, expressed as follows:

[0048]

[0049] S2. Input the image dataset into the deep convolutional generative adversarial network for network training, and save the generator module with the lowest FID (Fréchet Inception Distance) value of the output image in a computer-readable storage medium; the image dataset is a pre-stored set of multiple images or a set of multiple images acquired by an image acquisition device.

[0050] The objective function Loss of the adversarial generative network is:

[0051]

[0052] Where D(·) is the discriminator; G(·) is the generator; p data (x) represents the true image distribution; p z(z) represents a Gaussian distribution; E(·) denotes the calculation of the expected value. This represents taking the logarithm of the discriminator module's output and calculating its expectation, where x is the real image and follows the distribution p of the real images. data (x), denoted as x~p data (x), This represents the process of inverting, incrementing, and logarithming the output of the discriminator, followed by expectation calculation. Here, G(z) represents the output image obtained by inputting random Gaussian noise into the generator, where z is the random Gaussian noise following a Gaussian distribution p. z (z), denoted as z~p z (z).

[0053] S3. Using the generator module saved in step S2, random Gaussian noise is input to complete the generation and output of the image. In this embodiment, the output image is loaded into Waifu2x super-resolution software for size super-resolution magnification, magnifying it to twice the size of the original output image.

[0054] S4. In this embodiment, a Gaussian filtering module is constructed using Python language to perform Gaussian filtering on the magnified image in step S3, thereby reducing the image sharpening effect.

[0055] S5. Repeat steps S3 and S4 on the image after Gaussian filtering in step S4, repeating the process four times to finally generate a high-resolution artwork and display it on the monitor.

[0056] In this embodiment, the radius of the Gaussian filter used in the first and second super-resolution amplification loop operations of steps S3 and S4 is 1, and the radius of the Gaussian filter used in the third and fourth super-resolution amplification loop operations of steps S3 and S4 is 3.

[0057] In this embodiment, high-resolution art images are generated using a simple deep convolutional generative adversarial network and the steps of this invention, respectively, and the FID values ​​of the results are statistically analyzed. First, deep convolutional generative adversarial networks with generated image sizes of 128*128, 256*256, and 512*512 are constructed. The networks are trained iteratively for 100 epochs, with the training dataset coming from the Landscape subset of the Wikiart dataset. The FID values ​​of the generated images from 50 generator modules in epochs 51 to 100 are calculated and averaged, as shown in Table 1. It can be seen that the larger the generated image size, the higher the FID value, but the lower the quality of the generated image. Due to the large generated image size, the network training is prone to non-convergence. In this example, for the 512*512 image, because the network training struggles to balance the capabilities of the generator and discriminator modules, the discriminator module initially acquires strong discriminative power, leading to insufficient learning ability in the generator. Therefore, the calculated FID value is much larger than the previous two.

[0058] Table 1. Statistical results of FID values ​​of images of different sizes generated by different size generator modules.

[0059]

[0060] Table 2 shows the FID value calculation results of the 2048*2048 image generated using the steps of this invention. The specific process for generating the image is as follows: a deep convolutional generator adversarial network is trained using the Landscape subset from the Wikiart dataset as input. The 128*128 image generated by the trained generator module is then loaded into Waifu2x super-resolution software for size enlargement by a factor of 2, followed by Gaussian filtering. This process is repeated four times (the Gaussian filtering radius is set to 1 for the first and second times, and to 3 for the third and fourth times), ultimately generating a 2048*2048 high-resolution art painting image. Compared with Table 1, the 2048*2048 high-resolution art painting image generated by the steps of this invention still has a relatively low FID value, indicating that the distribution of the generated image is close to the distribution of the dataset images, and the image quality is good.

[0061] Table 2. Statistical results of FID values ​​of the 2048*2048 image generated by the steps of this invention.

[0062] Image size 2048*2048 FID (Fréchet Inception Distance) 230.7015

[0063] In another embodiment, the present invention can be a method or a system. The system includes a storage medium, a computer device, an image acquisition device, and a display device. The storage medium is a tangible device that stores instructions used by a processor, including but not limited to: a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a read-only memory, a random access memory, and any suitable combination thereof, and carries computer-readable program instructions and the application program Waifu2x for enabling the processor to implement various aspects of the present invention. The computer device can be a personal computer, a server, or a network device, etc., for executing the computer-readable program instructions and the application program Waifu2x stored in the storage medium, and is communicatively connected to the image acquisition device and the display device. The image acquisition device is used for acquiring image datasets in the present invention, including but not limited to cameras, camcorders, cameras, scanners, and other devices with photographic capabilities, and is communicatively connected to the computer device. When performing an image acquisition operation, the image acquisition device acquires the captured images and stores them in the storage medium. The display device is used to visualize the operation flow and results of the computer device running the computer-readable program instructions and application program, including but not limited to display output devices such as computer monitors, television screens, and projectors.

[0064] In another embodiment, the computer-readable program instructions used for the operation of the present invention may be source code written in any combination of one or more programming languages, which can be executed on a user's computer. The programming languages ​​include object-oriented programming languages ​​such as Python, C++, and Matlab.

[0065] In addition, the various functional units of the present invention can be integrated into one processing unit, or each unit can exist independently, or any two or more units can be integrated into one unit.

Claims

1. An intelligent artistic painting generation method, characterized in that, Includes the following steps: S1. Construct a deep convolutional generative adversarial network consisting of a generator module and a discriminator module, and set the network learning rate and the number of batches of data to be processed. The generator module takes randomly generated noise as input, and outputs the image of the last sample after multiple deconvolutions. The discriminator module judges the authenticity of the image output by the generator module. S2. Input the image dataset into the deep convolutional generative adversarial network for network training, and save the generator module with the lowest FID (Fréchet Inception Distance) value of the output image; the image dataset is a pre-stored set of multiple images or a set of multiple images acquired by an image acquisition device; S3. Use the generator module saved in step S2 to input random Gaussian noise to complete the generation and output of the image. Then, perform super-resolution upscaling on the output image to double the size of the original output image. S4. Construct a Gaussian filter module to perform Gaussian filtering on the magnified image in step S3 to reduce the image sharpening effect. S5. Repeat steps S3 and S4 several times on the image after Gaussian filtering in step S4 to obtain a high-resolution artwork of a set size, and display it on the monitor. 2.The method of claim 1, wherein: In step S1, the generator module uses multiple deconvolutional layers to upsample and enlarge the image, ultimately amplifying the random noise into an output image of a set size; the discriminator module uses multiple convolutional layers, taking real image samples and image samples output by the generator module as input, and outputs a numerical value, the magnitude of which represents the probability that the image sample output by the generator module is a real image sample.

3. The method for intelligently generating artistic paintings according to claim 1, characterized in that: In step S1, both the generator module and the discriminator module use batch normalization to solve the vanishing gradient problem during training and accelerate module convergence.

4. The method for intelligently generating artistic paintings according to claim 1, characterized in that: In step S1, the generator module uses the ReLU function as the activation function, as shown in the following expression: The last convolutional layer in the generator module uses the tanh function, expressed as follows:

5. The method for intelligently generating artistic paintings according to claim 1, characterized in that: In step S1, the discriminator module uses the LeakyReLU function as the activation function, with the following expression: Where α is a fixed hyperparameter; In the discriminator module, the last convolutional layer uses the sigmoid function, expressed as follows:

6. The method for intelligently generating artistic paintings according to claim 1, characterized in that: The objective function Loss of the generative adversarial network is: Where D(·) is the discriminator; G(·) is the generator; p data (x) represents the true image distribution; p z (z) represents a Gaussian distribution; E(·) denotes the calculation of the expected value. This represents taking the logarithm of the discriminator module's output and calculating its expectation, where x is the real image and follows the distribution p of the real images. data (x), denoted as x~p data (x), This represents the process of inverting, incrementing, and logarithming the output of the discriminator, followed by expectation calculation. Here, G(z) represents the output image obtained by inputting random Gaussian noise into the generator, where z is the random Gaussian noise following a Gaussian distribution p. z (z), denoted as z~p z (z).

7. The method for intelligently generating artistic paintings according to claim 1, characterized in that: In step S3, Waifu2x super-resolution software is used to perform super-resolution scaling of the dimensions.

8. The method for intelligently generating artistic paintings according to claim 1, characterized in that: In step S4, the image with magnified resolution is filtered using a Gaussian filter kernel with a set radius.

9. A computer device comprising a processor and a storage medium, wherein a computer program is stored on the storage medium, characterized in that, When the processor executes the computer program, it implements the intelligent generation method for artistic paintings according to any one of claims 1-8.

10. An intelligent art painting generation device, comprising an image acquisition device and a display device, characterized in that it further comprises a computer device as described in claim 9, wherein the image acquisition device and the display device are respectively communicatively connected to the computer device, the image acquisition device stores captured image data in the storage medium of the computer device; when the processor of the computer device executes the intelligent art painting generation method according to any one of claims 1-8, the display device displays the method operation flow and results, and displays a high-resolution art painting image.

Images