A landscape painting style transfer method based on an attention mechanism

Abstract

The application discloses a landscape painting style transfer method based on an attention mechanism, which is composed of data preprocessing, construction of a style transfer network, construction of a loss function, training of the style transfer network and generation of a landscape painting and has the advantages of information diversity, high calculation efficiency, good generated image quality, high speed and the like and can be used for landscape painting style transfer.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for style transfer in landscape painting. Technical Background

[0002] Paintings, created by artists based on their understanding of the objective world and their own psychological feelings, have long been considered a uniquely human art form. In recent years, with the development of deep learning technologies, the intelligent generation of paintings using computers has attracted researchers' attention. Image style transfer technology refers to using machine learning algorithms to give one image the style of another. Image style transfer technology has wide applications in fields such as artistic creation, photographic post-processing, and online marketing. It can help people quickly change the style of an image without altering its content, achieving excellent visual effects. Image style transfer technology is currently a hot research topic and an important direction in the fields of computer vision and image processing.

[0003] Landscape painting, a traditional Chinese art form, uses mountains and water as its themes to depict the beauty of nature. The backgrounds of landscape paintings typically feature natural elements such as mountains, rivers, lakes, trees, and clouds, conveying a sense of nature and tranquility, and expressing respect and gratitude for nature.

[0004] Landscape painting is a unique art form with typical stylistic characteristics. It typically revolves around natural themes such as mountains, rivers, lakes, clouds, and trees. Real-world objective scenery forms the basis of landscape painting creation, and similarly, image style transfer techniques are usually based on real-world images. Landscape paintings are often presented in the form of long scrolls, with scenes arranged from far to near. Varying shades of color and lines are used to express a sense of space. The fluidity and vividness of the brushstrokes are important characteristics of landscape painting, and the use of color is crucial in conveying the spatial sense and natural beauty of the landscape.

[0005] Research on style transfer in landscape painting began in the 1990s, when researchers mainly studied methods for representing and transferring image styles. With the development of deep learning technology, many researchers have begun to use convolutional neural networks to achieve style transfer in landscape painting.

[0006] Traditional deep learning-based style transfer methods are not satisfactory in terms of production quality and artistic appeal, and suffer from technical problems such as slow generation speed, poor generation quality, and weak network transfer capabilities.

[0007] In the field of landscape painting style transfer technology, the urgent technical problem to be solved is to provide a method for generating high-quality landscape painting style transfer maps quickly. Summary of the Invention

[0008] The technical problems to be solved by the present application are to overcome the shortcomings of the above-mentioned prior art, and to provide a landscape painting style transfer method based on attention mechanism, which has good image processing effect, various information, high calculation efficiency, good quality of generated landscape painting style transfer graph, and fast generation speed.

[0009] The technical solution adopted to solve the above technical problems is composed of the following steps.

[0010] (1) Data preprocessing

[0011] Select landscape painting images and artistic images of landscape paintings as a data set, divide the data set into a training set and a test set, and the number ratio of the training set to the test set is 10:1.

[0012] Adjust the landscape painting photos and landscape painting works to 256x256 pixel images and save them in png format.

[0013] (2) Constructing a style transfer network

[0014] The style transfer network is composed of an image transformation network and a loss network in series;

[0015] The image transformation network is composed of a downsampling module, an attention mechanism module, an image conversion module, and an upsampling module in series.

[0016] The downsampling module is composed of a first convolutional layer and a second convolutional layer, a first pooling layer, a third convolutional layer, a fourth convolutional layer, a second pooling layer, a first Inception module, and a second Inception module in series.

[0017] The image conversion module is composed of eight residual blocks connected in series.

[0018] The upsampling module is composed of a first deconvolutional layer and a fifth convolutional layer, a third Inception module, a residual block, a second deconvolutional layer, a sixth convolutional layer, a fourth Inception module, a seventh convolutional layer, a third deconvolutional layer, and an eighth convolutional layer in series.

[0019] The loss network is composed of the first eight layers of the first vgg16 network and the first eight layers of the second InceptionV3 network in parallel.

[0020] (3) Constructing a loss function

[0021] The loss function is constructed according to formula (1)

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] wherein, is the original content picture feature map, is the style picture feature map, is the generated picture feature map, l c represents the feature map on the 2nd layer; c1, c2, α, β are coefficients, taking values from 0 to 1, c1+c2=1, α+β=1; w l1 , w l2 is a coefficient, w l1 and w l2 are is the response of the i-th filter of the convolutional layer l of the generated picture in the first eight layers of the first vgg16 network at position j, is the response of the i-th filter of the convolutional layer l of the content picture in the first eight layers of the first vgg16 network at position j; is the response of the i-th filter of the convolutional layer l of the generated picture in the first eight layers of the second InceptionV3 network at position j, is the response of the i-th filter of the convolutional layer l of the content picture in the first eight layers of the second InceptionV3 network at position j; l1 represents the feature map on the 5th, 6th, and 7th layers, and l2 represents the feature map on the 5th, 6th, and 7th layers; N is the number of feature maps of the generated picture, and M is the width by height of the picture; is the Gram matrix of the generated picture at position j of the i-th filter of the convolutional layer l, is the Gram matrix of the style picture at position j of the i-th filter of the convolutional layer l.

[0032] (4) Training the style transfer network

[0033] The training set is input into the style transfer network for training, the initial learning rate is set to 0.001, the learning rate decay strategy is that the learning rate is halved every 30 epochs of training, the maximum gradient of gradient clipping is 2.0, the batch size of each round of training is 32, the dropout rate is 0.5, and the training is stopped until the loss function converges.

[0034] (5) generating landscape paintings

[0035] The test set is input into the trained landscape painting style transfer network for testing, and the corresponding stylized images are output, and the images generated by the test set are evaluated by using an artificial evaluation method.

[0036] In the step (2) of constructing the style transfer network in the application, the convolution kernel size of the first convolution layer in the down-sampling module is 3*3, the step is 1, and the output channel number is 64; the second convolution layer is the same as the first convolution layer; the size of the pooling kernel of the first pooling layer is 2*2; the convolution kernel size of the third convolution layer is 3*3, the step is 1, and the output channel number is 128; the fourth convolution layer is the same as the third convolution layer; the size of the pooling kernel of the second pooling layer is 2*2; the convolution kernel size of the first Inception module is 1*1, 3*3 and 5*5, the step is 1, and the output channel is 128; the second Inception module is the same as the first Inception module.

[0037] The size of the deconvolution kernel of the first deconvolution layer of the up-sampling module is 2*2, the step is 1, and the output channel number is 64; the size of the convolution kernel of the fifth convolution layer is 3*3, the step is 1, and the output channel number is 64; the size of the convolution kernel of the third Inception module is 1*1, 3*3 and 5*5, the step is 1, and the output channel is 64; the size of the deconvolution kernel of the second deconvolution layer is 2*2, the step is 1, and the output channel number is 32; the size of the convolution kernel of the sixth convolution layer is 3*3, the step is 1, and the output channel number is 32; the size of the convolution kernel of the fourth Inception module is 1*1, 3*3 and 5*5, the step is 1, and the output channel is 32; the seventh convolution layer is the same as the sixth convolution layer; the size of the deconvolution kernel of the third deconvolution layer is 2*2, the step is 1, and the output channel number is 1; the size of the convolution kernel of the eighth convolution layer is 1*1, the step is 1, and the output channel number is 1.

[0038] In the (2) style transfer network construction step of the present application, the first eight layers of the first VGG16 network are: the ninth convolutional layer and the tenth convolutional layer, the first max pooling layer, the eleventh convolutional layer, the twelfth convolutional layer, the second max pooling layer, the thirteenth convolutional layer, and the fourteenth convolutional layer, which are sequentially connected in series; the first eight layers of the second InceptionV3 network are: the fifteenth convolutional layer and the sixteenth convolutional layer, the seventeenth convolutional layer, the third max pooling layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the fourth max pooling layer, and the fifth Inception module, which are sequentially connected in series.

[0039] In the (2) style transfer network construction step of the present application, the convolutional kernel size of the ninth convolutional layer is 3x3, the step is 1, and the output channel number is 64; the tenth convolutional layer is the same as the ninth convolutional layer; the pooling kernel size of the first max pooling layer is 2x2, and the step is 2; the convolutional kernel size of the eleventh convolutional layer is 3x3, the step is 1, and the output channel number is 128; the twelfth convolutional layer is the same as the eleventh convolutional layer; the second max pooling layer, the third max pooling layer, and the fourth max pooling layer are the same as the first max pooling layer; the convolutional kernel size of the thirteenth convolutional layer is 3x3, the step is 1, and the output channel number is 256; the fourteenth convolutional layer is the same as the thirteenth convolutional layer; the convolutional kernel size of the fifteenth convolutional layer is 3x3, the step is 2, and the output channel number is 32; the convolutional kernel size of the sixteenth convolutional layer is 3x3, the step is 1, and the output channel number is 32; the convolutional kernel size of the seventeenth convolutional layer is 3x3, the step is 1, and the output channel number is 64; the convolutional kernel size of the eighteenth convolutional layer is 1x1, the step is 1, and the output channel number is 80; the convolutional kernel size of the nineteenth convolutional layer is 3x3, the step is 1, and the output channel number is 192; the fifth Inception module is the same as the first Inception module.

[0040] Since the style transfer network of the present application adopts the downsampling module, the technical problem of insufficient feature extraction is solved, and the feature extraction efficiency of the image is improved; the attention mechanism module is adopted to solve the technical problem of low attention of the feature map, and the accuracy of the conversion network is improved; the conversion network module is adopted to avoid the problems of blurring and distortion of the image in the conversion process, and the quality of the converted image is improved; the upsampling module solves the problem of feature map information loss through operations such as deconvolution and Inception module, and improves the image reconstruction effect; the parallel loss network is adopted to solve the technical problem of large network parameters, and the calculation speed is improved. The present application has the advantages of good image processing effect, diverse information, high calculation efficiency, good quality of generated landscape painting style transfer image, fast generation speed, etc., and can be used for style transfer of landscape paintings. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the flowchart of embodiment 1 of the present application.

[0042] Figure 2 is a structural schematic diagram of the image conversion network.

[0043] Figure 3 is Figure 2 is a structural schematic diagram of the middle down-sampling module.

[0044] Figure 4 is Figure 2 is a structural schematic diagram of the middle up-sampling module.

[0045] Figure 5 is a structural schematic diagram of the loss network.

[0046] Figure 6 is an original landscape painting photo.

[0047] Figure 7 is a landscape painting image output by the convolutional neural network style transfer method.

[0048] Figure 8 is a landscape painting image output by the method of embodiment 1 of the present application.

[0049] The present application will be further described in detail below in combination with the drawings and embodiments, but the present application is not limited to the following embodiments.

[0050] Embodiment 1

[0051] In Figure 1 , the landscape painting style transfer method based on the attention mechanism of the present embodiment is composed of the following steps.

[0052] (1) Data preprocessing

[0053] Select a landscape painting image and an artistic image of a landscape painting as a data set, and divide the data set into a training set and a test set, with a training set to test set quantity ratio of 10:1.

[0054] Adjust the landscape painting photo and the landscape painting work to a 256x256 pixel image, and save it in png format.

[0055] (2) Constructing a style transfer network

[0056] The style transfer network is composed of an image transformation network and a loss network in series.

[0057] In Figure 2 , the image transformation network of the present embodiment is composed of a down-sampling module and an attention mechanism module, an image conversion module, and an up-sampling module in series.

[0058] In Figure 3In the downsampling module of the embodiment, the first convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 64; the second convolutional layer is the same as the first convolutional layer; the first pooling layer has a pooling kernel size of 2x2; the third convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 128; the fourth convolutional layer is the same as the third convolutional layer; the second pooling layer has a pooling kernel size of 2x2; the first Inception module has convolution kernel sizes of 1x1, 3x3, and 5x5, a step of 1, and an output channel number of 128; and the second Inception module is the same as the first Inception module.

[0059] In the downsampling module of the embodiment, the first convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 64; the second convolutional layer is the same as the first convolutional layer; the first pooling layer has a pooling kernel size of 2x2; the third convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 128; the fourth convolutional layer is the same as the third convolutional layer; the second pooling layer has a pooling kernel size of 2x2; the first Inception module has convolution kernel sizes of 1x1, 3x3, and 5x5, a step of 1, and an output channel number of 128; and the second Inception module is the same as the first Inception module.

[0060] The downsampling module is adopted, the technical problem of insufficient feature extraction is solved, and the feature extraction efficiency of the image is improved.

[0061] The image conversion module of the embodiment is composed of eight residual blocks connected in sequence.

[0062] In Figure 4 In the upsampling module of the embodiment, the first deconvolutional layer has a deconvolution kernel size of 4x4, a step of 2, and an output channel number of 256; the fifth convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 256; the third Inception module has convolution kernel sizes of 1x1, 3x3, and 5x5, a step of 1, and an output channel number of 256; the residual block has a convolution kernel size of 3x3, a step of 1, and an output channel number of 256; the second deconvolutional layer has a deconvolution kernel size of 4x4, a step of 2, and an output channel number of 128; the sixth convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 128; the fourth Inception module has convolution kernel sizes of 1x1, 3x3, and 5x5, a step of 1, and an output channel number of 128; the seventh convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 64; the third deconvolutional layer has a deconvolution kernel size of 4x4, a step of 2, and an output channel number of 3; and the eighth convolutional layer has a convolution kernel size of 3x3, a step of 1, and an output channel number of 3.

[0063] The first deconvolution layer of the upsampling module of the embodiment has a deconvolution kernel size of 2*2, a step of 1, and 64 output channels; the fifth convolution layer has a convolution kernel size of 3*3, a step of 1, and 64 output channels; the third Inception module has a convolution kernel size of 1*1, 3*3, and 5*5, a step of 1, and 64 output channels; the second deconvolution layer has a deconvolution kernel size of 2*2, a step of 1, and 32 output channels; the sixth convolution layer has a convolution kernel size of 3*3, a step of 1, and 32 output channels; the fourth Inception module has a convolution kernel size of 1*1, 3*3, and 5*5, a step of 1, and 32 output channels; the seventh convolution layer is the same as the sixth convolution layer; the third deconvolution layer has a deconvolution kernel size of 2*2, a step of 1, and 1 output channel; and the eighth convolution layer has a convolution kernel size of 1*1, a step of 1, and 1 output channel. The upsampling module is adopted in the embodiment, and the feature map information loss problem is solved through operations such as deconvolution and Inception module, so that the image reconstruction effect is improved.

[0064] In Figure 5 the loss network of the embodiment is composed of the first eight layers of the first vgg16 network and the first eight layers of the second InceptionV3 network in parallel. The first eight layers of the first vgg16 network of the embodiment are sequentially connected in series and include the ninth convolution layer and the tenth convolution layer, the first maximum pooling layer, the eleventh convolution layer, the twelfth convolution layer, the second maximum pooling layer, the thirteenth convolution layer, and the fourteenth convolution layer. The first eight layers of the second InceptionV3 network of the embodiment are sequentially connected in series and include the fifteenth convolution layer and the sixteenth convolution layer, the seventeenth convolution layer, the third maximum pooling layer, the eighteenth convolution layer, the nineteenth convolution layer, the fourth maximum pooling layer, and the fifth Inception module.

[0065] The convolution kernel size of the ninth convolution layer of the embodiment is 3*3, the step is 1, and the output channel number is 64; the tenth convolution layer is the same as the ninth convolution layer; the pooling kernel size of the first maximum pooling layer is 2*2, and the step is 2; the convolution kernel size of the eleventh convolution layer is 3*3, the step is 1, and the output channel number is 128; the twelfth convolution layer is the same as the eleventh convolution layer; the second maximum pooling layer, the third maximum pooling layer and the fourth maximum pooling layer are the same as the first maximum pooling layer; the convolution kernel size of the thirteenth convolution layer is 3*3, the step is 1, and the output channel number is 256; the fourteenth convolution layer is the same as the thirteenth convolution layer; the convolution kernel size of the fifteenth convolution layer is 3*3, the step is 2, and the output channel number is 32; the convolution kernel size of the sixteenth convolution layer is 3*3, the step is 1, and the output channel number is 32; the convolution kernel size of the seventeenth convolution layer is 3*3, the step is 1, and the output channel number is 64; the convolution kernel size of the eighteenth convolution layer is 1*1, the step is 1, and the output channel number is 80; the convolution kernel size of the nineteenth convolution layer is 3*3, the step is 1, and the output channel number is 192; the fifth Inception module is the same as the first Inception module.

[0066] The loss network in parallel is adopted in the application, the technical problem of large network parameters is solved, and the calculation speed is improved.

[0067] (3) constructing a loss function

[0068] The loss function is constructed according to formula (1)

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] wherein, is the original content picture feature map, is the style picture feature map, is the generated picture feature map, l cThe feature map on the 2nd layer is represented; c1, c2, alpha, beta are coefficients, the values of which are 0-1, c1+c2=1, alpha+beta=1, the values of c1, c2, alpha, beta in the embodiment are 0.5; w l1 , w l2 is a coefficient, w l1 and w l2 is the response of the i-th filter of the convolutional layer l in the first eight layers of the first vgg16 network to the position j of the generated picture, is the response of the i-th filter of the convolutional layer l in the first eight layers of the first vgg16 network to the position j of the content picture; is the response of the i-th filter of the convolutional layer l in the first eight layers of the second InceptionV3 network to the position j of the generated picture, is the response of the i-th filter of the convolutional layer l in the first eight layers of the second InceptionV3 network to the position j of the content picture; l1 represents the feature map on the 5th, 6th and 7th layers, and l2 represents the feature map on the 5th, 6th and 7th layers; N is the number of feature maps of the generated picture, and M is the width and height of the picture; is the Gram matrix of the generated picture in the convolutional layer l, the i-th filter is in the position j, is the Gram matrix of the style picture in the convolutional layer l, the i-th filter is in the position j.

[0079] Since the down-sampling module is used in the step, the technical problem of insufficient feature extraction is solved, and the feature extraction efficiency of the image is improved; the attention mechanism module is used to solve the technical problem of low attention of the feature map, and the accuracy of the conversion network is improved; the conversion network module is used to avoid the problems of blur and distortion of the image in the conversion process, and the quality of the converted image is improved; the up-sampling module solves the problem of feature map information loss through deconvolution and Inception module operations, and improves the image reconstruction effect; the parallel loss network is used to solve the technical problem of large network parameters, and the calculation speed is improved. The present application has the advantages of good image processing effect, diverse information, high calculation efficiency, good quality of generated landscape painting style transfer image, fast generation speed and the like, and can be used for style transfer of landscape paintings.

[0080] (4) Training of the style transfer network

[0081] The training set is input into the style transfer network for training, the initial learning rate is set to 0.001, the learning rate decay strategy is to reduce the learning rate by half every 30 epochs, the maximum gradient for gradient clipping is 2.0, the batch size for each round of training is 32, the dropout rate is 0.5, and the training is performed until the loss function converges.

[0082] ​(5) generating landscape paintings

[0083] The test set is input into the trained landscape painting style transfer network for testing, and the corresponding stylized image is output. The images generated by the test set are evaluated by a manual evaluation method.

[0084] The attention mechanism-based landscape painting style transfer method is completed.

[0085] Embodiment 2

[0086] The attention mechanism-based landscape painting style transfer method of this embodiment consists of the following steps.

[0087] (1) Data preprocessing

[0088] This step is the same as in Embodiment 1.

[0089] (2) Constructing a style transfer network

[0090] This step is the same as in Embodiment 1.

[0091] (3) Constructing a loss function

[0092] In formula (1) of this step, c1, c2, α, β are coefficients, and take values of 0-1, c1+c2=1, α+β=1, c1 of this embodiment takes a value of 0, c2 takes a value of 1, α takes a value of 0, and β takes a value of 1; w l1 and w l2 Take the same values as in Embodiment 1. The other steps of this step are the same as in Embodiment 1.

[0093] The other steps are the same as in Embodiment 1, and the attention mechanism-based landscape painting style transfer method is completed.

[0094] Embodiment 3

[0095] The attention mechanism-based landscape painting style transfer method of this embodiment consists of the following steps.

[0096] (1) Data preprocessing

[0097] This step is the same as in Embodiment 1.

[0098] (2) Constructing a style transfer network

[0099] This step is the same as in Embodiment 1.

[0100] (3) Constructing a loss function

[0101] In formula (1) of this step, c1, c2, α, β are coefficients, and take values of 0-1, c1+c2=1, α+β=1, c1 of this embodiment takes a value of 1, c2 takes a value of 0, α takes a value of 1, and β takes a value of 0; w l1and w l2 The values are the same as in Example 1. The other steps of this step are the same as in Example 1.

[0102] The other steps are the same as in Example 1, and the landscape painting style transfer method based on attention mechanism is completed.

[0103] In order to verify the beneficial effects of the present application, the inventors used the landscape painting style transfer method based on attention mechanism of Example 1 of the present application (hereinafter referred to as Example 1 method) and the style transfer model based on convolutional neural network (hereinafter referred to as comparative experiment method) to carry out a comparative simulation experiment, and the experimental conditions are as follows:

[0104] 1. Simulation conditions

[0105] The hardware conditions are: Ubuntu 16.04, a server composed of 2 NVIDIA 1080Ti GPUs.

[0106] The software platform is: python language and Pytorch deep network framework.

[0107] 2. Simulation content and results

[0108] The landscape real scene photos are selected as Figure 6 , the method of Example 1 of the present application and the comparative experiment method are used to carry out simulation experiments, the experimental results of the comparative experiment method are shown in Figure 7 , the experimental results of the method of Example 1 are shown in Figure 8 . From Figure 7 and Figure 8 , it can be seen that Figure 7 only mechanically migrates the structure of the landscape photo, and the sky and the mountain are not well processed, only the color of the original photo is retained, which is not the color picture that should appear in the landscape painting. Figure 8 The generated image has a stronger landscape style, and the sky and the mountain are well stylized. The style transfer effect is realistic, the image generation time is reduced from 14 seconds to 12 seconds, and the image generation speed is faster.

Claims

1. A method for style transfer in landscape painting based on an attention mechanism, characterized in that... It consists of the following steps: (1) Data preprocessing Landscape painting images and artistic images of landscape paintings were selected as the dataset. The dataset was divided into a training set and a test set, with a ratio of 10:1 between the training set and the test set. Adjust the landscape painting photos and works to 256×256 pixel images and save them as PNG format; (2) Constructing a style transfer network Style transfer networks consist of an image transformation network and a loss network connected in series. The image transformation network is composed of a downsampling module, an attention mechanism module, an image transformation module, and an upsampling module connected in series. The downsampling module is composed of a first convolutional layer, a second convolutional layer, a first pooling layer, a third convolutional layer, a fourth convolutional layer, a second pooling layer, a first Inception module, and a second Inception module connected in series. The image conversion module consists of eight residual blocks connected in series. The upsampling module is composed of a first deconvolution layer and a fifth convolution layer, a third Inception module, a residual block, a second deconvolution layer, a sixth convolution layer, a fourth Inception module, a seventh convolution layer, a third deconvolution layer, and an eighth convolution layer connected in series. The loss network is composed of the first eight layers of the first vgg16 network and the first eight layers of the second InceptionV3 network connected in parallel; (3) Constructing the loss function Construct the loss function according to equation (1) in, It is the feature map of the original content image. It is a style image feature map. It generates image feature maps, l c This represents the feature map at layer 2; c1, c2, α, and β are coefficients, taking values ​​from 0 to 1, where c1 + c2 = 1 and α + β = 1; w l1 w l2 It is a coefficient, w l1 and w l2 for It is the response of the i-th filter in the convolutional layer l that generates the image in the first eight layers of the first VGG16 network at position j. It is the response of the i-th filter in the convolutional layer l of the content image at position j in the first eight layers of the first VGG16 network. It is the response of the i-th filter in the convolutional layer l that generates the image in the first eight layers of the second InceptionV3 network at position j. This is the response of the i-th filter in the convolutional layer l of the content image at position j in the first eight layers of the second InceptionV3 network; l1 represents the feature maps in layers 5, 6, and 7, l2 represents the feature maps in layers 5, 6, and 7; N is the number of feature maps in the generated image, and M is the image width by height; It is the Gram matrix of the i-th filter at position j in the convolutional layer l that generates the image. It is the Gram matrix of the style image at position j of the i-th filter in convolutional layer l; (4) Training Style Transfer Network The training set is input into the style transfer network for training. The initial learning rate is set to 0.

001. The learning rate decay strategy is to halve the learning rate every 30 epochs. The maximum gradient clipping is 2.

0. The batch size for each training round is 32. The dropout rate is 0.

5. Training continues until the loss function converges. (5) Generate landscape paintings The test set is input into the trained landscape painting style transfer network for testing, and the corresponding stylized images are output. The images generated from the test set are evaluated by human evaluation methods.

2. The method for style transfer of landscape painting based on attention mechanism according to claim 1, characterized in that: In step (2) of constructing the style transfer network, the kernel size of the first convolutional layer in the downsampling module is 3×3, the stride is 1, and the number of output channels is 64; the second convolutional layer is the same as the first convolutional layer; the kernel size of the first pooling layer is 2×2; the kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of output channels is 128; the fourth convolutional layer is the same as the third convolutional layer; the kernel size of the second pooling layer is 2×2; the kernel sizes of the first Inception module are 1×1, 3×3, and 5×5, the stride is 1, and the number of output channels is 128; the second Inception module is the same as the first Inception module. The upsampling module has the following configurations: the first deconvolution layer has a 2×2 kernel size, a stride of 1, and 64 output channels; the fifth convolution layer has a 3×3 kernel size, a stride of 1, and 64 output channels; the third Inception module has kernel sizes of 1×1, 3×3, and 5×5, a stride of 1, and 64 output channels; the second deconvolution layer has a 2×2 kernel size, a stride of 1, and 32 output channels; the sixth convolution layer has a 3×3 kernel size, a stride of 1, and 32 output channels; the fourth Inception module has kernel sizes of 1×1, 3×3, and 5×5, a stride of 1, and 32 output channels; the seventh convolution layer is the same as the sixth convolution layer; and the third deconvolution layer has a 2×2 kernel size, a stride of 1, and 1 output channel. The eighth convolutional layer has a kernel size of 1×1, a stride of 1, and 1 output channel.

3. The method for style transfer in landscape painting based on an attention mechanism according to claim 1, characterized in that... In step (2) of constructing the style transfer network, the first eight layers of the first VGG16 network are: the ninth and tenth convolutional layers, the first max pooling layer, the eleventh convolutional layer, the twelfth convolutional layer, the second max pooling layer, the thirteenth convolutional layer, and the fourteenth convolutional layer are connected in series; the first eight layers of the second InceptionV3 network are: the fifteenth and sixteenth convolutional layers, the seventeenth convolutional layer, the third max pooling layer, the eighteenth convolutional layer, the nineteenth convolutional layer, the fourth max pooling layer, and the fifth Inception module are connected in series.

4. The method for style transfer of landscape painting based on attention mechanism according to claim 3, characterized in that: The ninth convolutional layer has a kernel size of 3×3, a stride of 1, and 64 output channels; the tenth convolutional layer is the same as the ninth; the first max-pooling layer has a kernel size of 2×2 and a stride of 2; the eleventh convolutional layer has a kernel size of 3×3, a stride of 1, and 128 output channels; the twelfth convolutional layer is the same as the eleventh; the second, third, and fourth max-pooling layers are the same as the first max-pooling layer; the thirteenth convolutional layer has a kernel size of 3×3, a stride of 1, and 256 output channels; the fourteenth convolutional layer is the same as the tenth. The three convolutional layers are identical; the fifteenth convolutional layer has a kernel size of 3×3, a stride of 2, and 32 output channels; the sixteenth convolutional layer has a kernel size of 3×3, a stride of 1, and 32 output channels; the seventeenth convolutional layer has a kernel size of 3×3, a stride of 1, and 64 output channels; the eighteenth convolutional layer has a kernel size of 1×1, a stride of 1, and 80 output channels; the nineteenth convolutional layer has a kernel size of 3×3, a stride of 1, and 192 output channels; the fifth Inception module is identical to the first Inception module.

Images