A deep learning-based fast image style conversion method

Abstract

The application belongs to the technical field of computer vision, and provides a rapid image style conversion method based on deep learning, comprising: constructing a double-network architecture through a generation network and a loss network; the loss network receives an original image, a reference image and a generated image, extracts features and calculates a perceptual loss to provide gradient information; when training a specific style model, input a single original image into the generation network to obtain a generated image; then input the three types of images into the loss network, extract content and style loss features and calculate a perceptual loss; adjust the generation network weight by minimizing the loss through gradient descent; iterate the data set for multiple rounds until the perceptual loss converges and the parameters are stable, and then the specific style model is obtained; when used, the original image is cropped, rotated and de-meaned to be standardized, and then input into the model generation network to output a specific style image; the application can realize the conversion through the double architecture of "generation network + loss network", and only one model training is needed to complete the conversion through one-time forward propagation of a single image.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and provides a fast image style transfer method based on deep learning. Background Technology

[0002] With the development of computer and artificial intelligence technologies, the demand for images as information carriers and entertainment tools is growing. Human image art creation relies on the combination of "content (semantic elements)" and "style (texture, color)". In the fast-paced Internet environment, from everyday filters to painting AI, there is a need to efficiently achieve the image style conversion function of "preserving content and transferring style". This technology has also become a key direction to meet the needs of digital entertainment, design and other fields.

[0003] Currently, there are two main types of algorithms for image style transfer based on deep learning: one is the image iteration-based method proposed by Gatys et al., which uses VGG19 as the core, starts with a white noise image, extracts deep features of the content image and the full-layer Gram matrix of the style image through CNN, calculates the perceptual loss (content loss + style loss), and iteratively optimizes pixels through gradient descent until the loss converges; the other is the fast method based on model iteration proposed by Johnson et al., which constructs a dual architecture of "generative network + VGG16 loss network", uses perceptual loss to train the parameters of the generative network, and after training, a single image can be transformed through a single forward propagation.

[0004] Existing technologies have obvious drawbacks: image-based iterative methods require repeated loss calculations for each transformation, which is time-consuming and cannot meet real-time requirements; while fast methods based on model iteration are faster, a single model only corresponds to a fixed style and cannot be tuned, and training new models requires large-scale data and high-performance hardware, which takes a long time; moreover, when processing high-resolution or complex content images, both types of methods are prone to edge noise and content distortion, making it difficult to balance efficiency and quality. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention provides a fast image style transfer method based on deep learning. This method utilizes a dual architecture of "generative network + loss network" to achieve subsequent single-image "one forward propagation" conversion with only one model training, meeting the needs of real-time applications.

[0006] The technical solution of the present invention includes the following steps: A dual-network architecture is constructed using a generator network and a loss network: the loss network is used to receive the original image, the reference image, and the generated image and extract features, calculate perceptual loss to provide gradient information, and the generator network is used to convert the original image into abstract content features, restore the resolution of the original image, and fuse style features.

[0007] A specific style model is generated by training a dual-network architecture: a single original image from the original image dataset is input into the generator network to obtain the generated image. The original image, reference image, and generated image are input into the loss network to extract the content loss and style loss features of the generated image compared to the original and reference images. The perceptual loss is calculated based on the extracted content loss and style loss features. The perceptual loss is minimized by gradient descent, thereby adjusting the weight parameters of the generator network. The above steps are repeated to traverse all original images in the original image dataset for multiple iterations until the perceptual loss converges and the generator network parameters stabilize, at which point the iteration stops, resulting in a specific style model.

[0008] Generate a specific reference image using a specific style model: The original image to be transformed is cropped, rotated, and mean-removed to obtain a standardized data format required by the generator network. This data is then input into the specific style model, and the generator network of the specific style model outputs an original image of a specific style, thus completing the image style transfer.

[0009] Furthermore, the generating network is a convolutional neural network, comprising an input layer, an encoding layer, a decoding layer, and an output layer. The encoding layer consists of multiple convolutional layers and is used to convert the original image into abstract content features. The decoding layer includes deconvolutional layers and upsampling layers, which are used to restore the image resolution and fuse style features.

[0010] Furthermore, the loss network is a pre-trained VGG16 convolutional neural network, including convolutional layers and pooling layers.

[0011] Furthermore, the original image dataset includes images of landscapes, people, and animals, with at least 1,000 images of each.

[0012] Furthermore, the number of iterations is no less than 50 rounds. After each iteration, the perceptual loss of the current round is calculated. When the rate of change of the perceptual loss is less than 0.001 for 5 consecutive rounds, the perceptual loss is determined to have converged, and the iteration is stopped.

[0013] Furthermore, the mean-removal method is as follows: calculate the pixel mean of the RGB three channels of the original image training set and the pixel mean of the RGB three channels of the original image to be converted, and subtract the pixel mean of the RGB three channels of the original image training set from the pixel mean of the RGB three channels of the original image to be converted to obtain the mean-removed value.

[0014] Furthermore, the gradient descent method employs the Adam optimizer, with an initial learning rate of 0.001. After every 10 iterations, the learning rate decays to 0.5 of the current value, in order to balance the convergence speed and stability of model training.

[0015] Furthermore, the perceptual loss is the sum of content loss and style loss.

[0016] The content loss is calculated as follows: the degree of difference between the original image and the generated image is quantified, feature information is extracted using shallow convolutional layers in the convolutional neural network, and then the content loss is calculated by measuring the distance between the feature representations of the generated image and the original image on that layer. The style loss is calculated as follows: the similarity between the generated image and the reference image is quantified by comparing their statistical features. The Gram matrix is ​​used to represent the style information of each feature layer, and the Gram matrix difference between the generated image and the style image is calculated, which is the style loss.

[0017] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: A dual-network architecture is constructed using a generator network and a loss network: the loss network receives the original image, reference image, and generated image, extracts features, and calculates perceptual loss to provide gradient information; the generator network converts the original image into abstract content features, restores the resolution of the original image, and fuses style features. A specific style model is trained using this dual-network architecture: a single original image from the original image dataset is input into the generator network to obtain the generated image; the original image, reference image, and generated image are input into the loss network to extract content loss and style loss features of the generated image compared to the original and reference images; the perceptual loss is calculated based on the extracted content loss and style loss features; the perceptual loss is minimized using gradient descent, thereby adjusting the weight parameters of the generator network; this process is repeated for multiple iterations across all original images in the original image dataset until the perceptual loss converges and the generator network parameters stabilize, resulting in a specific style model. A specific reference image is generated using this specific style model: the original image to be transformed is cropped, rotated, and mean-removed to obtain a standardized data format required by the generator network input, which is then input into the specific style model. The generator network of the specific style model outputs an original image of a specific style, completing the image style conversion. Compared with existing technologies, this invention can achieve the transformation of a single graph with only one forward propagation by using a dual architecture of "generative network + loss network", thus meeting the needs of real-time applications.

[0018] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a dual-network architecture consisting of a generator network and a loss network, as described in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the image style conversion process according to an embodiment of the present invention. Detailed Implementation

[0022] The following detailed description of a specific embodiment of the present invention is provided in conjunction with the accompanying drawings. However, it should be understood that the scope of protection of the present invention is not limited to the specific embodiment.

[0023] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the technical solution of this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0024] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0025] like Figure 1 and Figure 2 As shown, the present invention provides a fast image style transfer method based on deep learning, comprising the following steps: A dual-network architecture is constructed using a generator network and a loss network: the loss network is a pre-trained VGG16 convolutional neural network (with fully connected layers removed), used to receive the original image (content map), the reference image (style map), and the generated image and extract features, and calculate perceptual loss to provide gradient information; the generator network is a convolutional neural network used to convert the original image into abstract content features, restore the resolution of the original image, and fuse style features.

[0026] A style-specific model is generated by training a dual-network architecture: a single original image from the original image dataset is input into the generator network to obtain an initial generated image; the original image, reference image, and initial generated image are input into the loss network to extract deep semantic content features of the original image, full-layer style features of the reference image, and corresponding features of the initial generated image, and the perceptual loss is calculated based on the features (perceptual loss = content loss + style loss); the perceptual loss is minimized using gradient descent, and the weight parameters of the generator network are adjusted; the above steps are repeated, traversing the original image dataset for multiple iterations (no less than 50 iterations), until the rate of change of the perceptual loss is less than 0.001 for 5 consecutive iterations (determining that the perceptual loss has converged) and the generator network parameters are stable, at which point the iteration stops, and the style-specific model is obtained.

[0027] A specific style image is generated by a specific style model: the original image to be transformed is cropped (unified to the preset size of the generator network), rotated (correcting the image orientation), and mean removed (the mean of the RGB three channels of the image to be transformed is subtracted from the mean of the RGB three channels of the original image training set) to obtain the standardized data format required by the generator network input; the standardized data is input into the specific style model, and the generator network of the specific style model outputs an image of the specific style after one forward propagation, thus completing the image style conversion.

[0028] To address the pain point of traditional image-based iterative methods that require repeated calculations of perceptual loss and are time-consuming for each conversion, a dual architecture of "generative network + loss network" is adopted. This allows subsequent single images to be converted with "one forward propagation" after only one model training, meeting the needs of real-time applications (such as real-time filters and video style conversion).

[0029] The preprocessing steps (cropping, rotation, and mean removal) ensure that the data format input to the generator network is standardized, avoiding fluctuations in conversion quality caused by differences in the size and brightness of the original images. At the same time, the conversion accuracy of the generator model is guaranteed through multiple rounds of iterative convergence conditions (perceptual loss stability), balancing efficiency and quality.

[0030] In the embodiments provided by this invention, the generating network is a convolutional neural network, comprising an input layer, an encoding layer, a decoding layer, and an output layer. The encoding layer consists of multiple convolutional layers, used to convert the original image into abstract content features. The decoding layer includes deconvolutional layers and upsampling layers, used to restore the image resolution and fuse style features. Specifically:

[0031] Input layer: Contains 1 to 2 convolutional layers, which convert the RGB three channels of the original image into the number of feature channels (such as 64 channels) processed internally by the generative network, thus completing the data format adaptation.

[0032] The encoding layer consists of multiple stacked units: convolutional layer → ReLU nonlinearization layer → pooling layer (maximum pooling or average pooling). The convolutional layer extracts local features from the original image, the ReLU layer introduces nonlinearity to improve feature representation, and the pooling layer downsamples and compresses the feature map size to reduce computation. Finally, it outputs abstract content features.

[0033] Decoding layer: It consists of multiple "deconvolution layer → ReLU nonlinearization layer" units stacked together. The deconvolution layer upsamples to gradually restore the image resolution, the convolution layer refines the feature details, and at the same time, it integrates the content features transmitted by the encoding layer and the style features of the reference image.

[0034] Output layer: Contains a 1×1 convolutional layer, which converts the high-dimensional feature channels output by the decoding layer into RGB three channels, and outputs the style-transformed image.

[0035] In the embodiments provided by this invention, the loss network is a pre-trained VGG16 convolutional neural network, including convolutional layers and pooling layers (fully connected layers are removed to reduce computation). The core function of the loss network is to extract features from the original image, reference image, and generated image (deep semantic content features of the original image, full-layer style features of the reference image, and corresponding features of the generated image), calculate the perceptual loss, and feed the loss gradient back to the generation network for iterative adjustment of the generation network weight parameters.

[0036] Using a pre-trained VGG16 loss network, its mature feature extraction capabilities can accurately extract deep semantic content features and full-layer style features of images, improving the accuracy of perceptual loss calculation. Removing fully connected layers reduces the computational cost of feature extraction, improves the operating efficiency of the loss network, provides efficient gradient support for optimizing the parameters of the generator network, and indirectly shortens the model training cycle.

[0037] In the embodiments provided by the present invention, the original image dataset needs to cover diverse semantic scenes, including landscape, people and animal images, and there should be at least 1,000 landscape, people and animal images respectively, in order to ensure the data generalization during the training process of the generative network and avoid model overfitting.

[0038] The original image dataset covers diverse semantic scenes such as landscapes, people, and animals, and has a sufficient number of images for each category (≥1000 images). This avoids overfitting problems caused by limited data during model training, improves the model's adaptability to different types of content images, and ensures that the model can achieve high-quality style transfer for various everyday images after training.

[0039] In the embodiments provided by the present invention, the number of iterations is no less than 50 rounds. After each iteration traverses the original image dataset, the perceptual loss of the current round is calculated. When the rate of change of the perceptual loss is less than 0.001 for 5 consecutive rounds, the perceptual loss is determined to have converged, and the iteration is stopped to ensure that the generated network parameters are stable and that training resources are not wasted.

[0040] Set a basic training quantity of "no less than 50 iterations" to ensure that the generative network fully learns the content features of the original image and the style features of the reference image; use "the rate of change of perceptual loss for 5 consecutive iterations < 0.001" as the convergence criterion to avoid unstable model parameters due to insufficient iterations or waste of training resources due to excessive iterations, thus balancing the model training effect and efficiency.

[0041] In the embodiments provided by the present invention, the mean removal processing method is as follows: first, calculate the mean value of the RGB three-channel pixels of the original image training set, then calculate the mean value of the RGB three-channel pixels of the original image to be converted, and subtract the mean value of the original image training set for each pixel value of the RGB three-channel of the original image to be converted; the mean removal processing is used to eliminate the interference of image brightness on the feature extraction of the generating network and ensure the consistency of the input data distribution.

[0042] By subtracting the RGB mean of the training set from the RGB mean of the image to be converted, the pixel value distribution deviation caused by differences in shooting brightness and ambient light in different images can be eliminated, so that all image data input to the generator network are uniformly distributed, avoiding brightness interference in the feature extraction process and improving the stability of style transfer of the generator network.

[0043] In the embodiments provided by the present invention, the gradient descent method uses the Adam optimizer, the initial learning rate of the Adam optimizer is set to 0.001, and the learning rate decays to 0.5 of the current value after every 10 iterations; the initial learning rate ensures the convergence speed in the early stage of training, and the learning rate decay avoids parameter oscillation in the later stage of training, thus balancing the convergence speed of model training and the stability of generated network parameters.

[0044] An initial learning rate of 0.001 ensures that the parameter adjustment range is appropriate in the early stage of model training, quickly approaching the optimal solution; the learning rate is reduced to 0.5 of the current value every 10 iterations, which can avoid parameter oscillation caused by excessively high learning rates in the later stage of training, balancing model training speed and parameter stability, and ensuring the accuracy of the final parameters of the generated network.

[0045] In the embodiments provided by the present invention, the perceptual loss is the sum of content loss and style loss.

[0046] The content loss is calculated as follows: the difference between the original image and the generated image is quantified, feature information is extracted using shallow convolutional layers in a convolutional neural network, and then the content loss is calculated by determining the distance between the feature representations of the generated image and the original image at that layer. Specifically:

[0047] The feature representations of the original image and the generated image are extracted using shallow convolutional layers (such as the conv1_2 layer) of a lossy network (VGG16). The difference is quantified by calculating the mean squared error (MSE) of the feature representations of the two images, as shown in the formula: in, These are the pixel values ​​of the feature maps in the first layer of the generated image and the original image, respectively.

[0048] The style loss is calculated as follows: The similarity between the generated image and the reference image is quantified by comparing their statistical features. A Gram matrix is ​​used to represent the style information of each feature layer, and the difference in the Gram matrix between the generated image and the style image is calculated, i.e., the style loss. Specifically:

[0049] The generated and reference images are extracted using fully convolutional layers (such as conv1_1, conv1_2, and conv2_1) of a lossy network (VGG16). The Gram matrix of the feature representation of each layer is calculated, and the difference is quantified by calculating the mean square error of the corresponding layer's Gram matrix and then weighting and summing them. The formula is as follows: in, These are the Gram matrix elements of the generated image and the reference image in layer 1, respectively, where L is the total number of convolutional layers. For the weight parameters of the I-th layer, This represents the number of channels in the first layer feature map. This represents the number of pixels in the first layer feature map.

[0050] Content loss, through feature comparison of shallow convolutional layers, can accurately quantify the content difference between the generated image and the original image, ensuring that the generated image retains the core semantics of the original image (such as object outlines and structures); style loss, through comparison of style features across all layers using the Gram matrix, can completely restore the texture and color patterns of the reference image; the perceptual loss formed by the combination of the two can simultaneously constrain the "content retention" and "style restoration" of the generated image, achieving the core requirement of "changing style without changing content".

[0051] Example: I. Data Preparation Content image dataset preparation: Select an image dataset covering three types of scenes: "landscape, people, and animals". The number of images in each type should not be less than 1000. Preprocess all images to a resolution of 512×512 and save them to a specified path for use in generating network training. Style map preparation: Obtain a high-resolution image of "The Great Wave off Kanagawa", and crop it to a size of 512×512 based on the center of the image according to the original file 3.2.3 "Crop Processing" logic, and save it to the style data path as a reference image (style map) for style transfer; Test content image preparation: Obtain the image of "The Scream" (as the original content image to be converted), retain its original resolution (e.g., 1200×800), and save it to the test input path for subsequent style conversion testing.

[0052] II. Configuration File Settings To create a configuration file (such as wave_config.yml) for a model in the style of "The Great Wave off Kanagawa", the key parameters are set as follows: Model basic information: The model is named "wave" (corresponding to the style of "The Great Wave off Kanagawa"), and the model is saved in a specified folder to ensure that the parameters can be saved in real time during training; Image parameters: The input size of the generating network is set to 256×256, and the number of image channels is RGB three channels; Loss layer configuration: The content loss layer is specified as the conv1_2 layer of the VGG16 convolutional neural network (original file 3.2.4 "Content loss with shallow convolutional layers"); the style loss layer is specified as the conv1_1, conv1_2, conv2_1, and conv2_2 layers of VGG16; Training parameters: no less than 50 iterations; Adam optimizer is used, with an initial learning rate of 0.001, which decays to 0.5 of the current value after every 10 iterations; batch size is set to 4 to optimize training efficiency for the content dataset. Convergence condition: Set "the rate of change of perception loss for 5 consecutive rounds is less than 0.001" as the iteration stopping condition to ensure the stability of the generated network parameters.

[0053] III. Training in Models in the Style of "The Great Wave off Kanagawa" 3.1 Building a Dual Network Architecture Generative Network Construction: Constructing a convolutional neural network containing an input layer, an encoding layer, a decoding layer, and an output layer. Input layer: Contains one 3×3 convolutional layer (64 convolutional kernels), which converts the original RGB three-channel image into a 64-channel feature map to complete the data format adaptation; The encoding layer consists of two stacked sets of "3×3 convolutional layers (using 64 and 128 convolutional kernels respectively) → ReLU nonlinearization layer → 2×2 max pooling layer". The feature map size is compressed to 64×64×128 by downsampling, thereby achieving content feature abstraction. Decoding layer: It consists of two sets of "3×3 deconvolution layers (with 128 and 64 convolution kernels respectively) → ReLU nonlinearization layer" stacked together. The feature map size is restored to 256×256×64 by upsampling, and the content features and the style features of "The Great Wave off Kanagawa" are fused at the same time. Output layer: Contains one 1×1 convolutional layer (3 convolutional kernels), which converts the number of high-dimensional feature channels into RGB three channels and outputs a generated image of size 256×256; Loss network construction: A pre-trained VGG16 convolutional neural network is used to extract features from the content map, style map, and generated map and calculate the perceptual loss, providing gradient information for optimizing the parameters of the generation network.

[0054] Single-round training process: Randomly read 4 content images (batch size 4) from the content dataset and input them into the generator network to obtain 4 initial generated images. Input the content images, the "The Great Wave off Kanagawa" style image, and the initial generated images into the loss network: Extract features from the content graph and the generated graph at conv1_2 layer, and calculate the content loss formula: in, To generate graph features, This is a content graph feature.

[0055] Extract the Gram matrices of the style map and the generated map at the four style layers, and calculate the style loss formula: in, To generate the graph Gram matrix, For style map Gram matrix, For the number of channels, This represents the number of pixels.

[0056] Summing yields the perceived loss: The Adam optimizer adjusts the network weight parameters based on the perceptual loss through backpropagation, thus completing single-batch training.

[0057] Multi-round iteration and convergence determination: Repeat the single-round training process, and save the model parameters after traversing all content images (1000 images / class × 3 classes ÷ 4 batches = 750 batches / round) in each round; when training reaches the 45th round, the perceptual loss change rate of 5 consecutive rounds is 0.0008, 0.0007, 0.0006, 0.0005, and 0.0004 (all < 0.001), and the perceptual loss is determined to have converged. The iteration is stopped, and the "The Great Wave off Kanagawa" style model is obtained (e.g., wave.ckpt-45).

[0058] IV. Execution of Image Style Transfer in "The Scream" 4.1 Preprocessing of Test Content Image (Image from "The Scream") Cropping: Based on the image center, crop to a size of 256×256 (matching the input size of the network); Rotation: Read the image EXIF ​​information to confirm that the image orientation is positive, and no rotation is needed (the purpose is to eliminate orientation interference). Mean Removal: Calculate the RGB three-channel mean of the content dataset (e.g., R=123.68, G=116.779, B=103.939), and subtract the corresponding channel mean from the RGB value of each pixel in the "The Scream" image (e.g., a pixel (190,150,110) becomes (66.32,33.221,6.061)). Digitization: Convert pixel values ​​from int8 (0-255) to float32, and add batch dimensions through dimension expansion to obtain standardized data in the format [1,256,256,3].

[0059] 4.2 Style Transfer and Output Load the trained Kanagawa Great Wave off-the-fly style model (wave.ckpt-45); The preprocessed image data of "The Scream" was input into the generator network, and after one forward propagation, a style transfer map of size 256×256 was output. Perform a reverse mean-reduction operation on the transformed image (add the pixel values ​​back to the RGB three-channel mean of the content dataset), and crop the pixel values ​​to the range of [0,255], saving it as the final style conversion result (such as the "The Great Wave off Kanagawa" style image of the "The Scream" image), thus completing the entire style conversion process.

[0060] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0061] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Other modifications can be readily implemented by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.

Claims

1. A fast image style transfer method based on deep learning, characterized in that, Includes the following steps: A specific style model is generated by training a dual-network architecture: a single original image from the original image dataset is input into the generator network to obtain the generated image. The original image, reference image, and generated image are input into the loss network to extract the content loss and style loss features of the generated image compared with the original image and reference image. The perceptual loss is calculated based on the extracted content loss and style loss features. The perceptual loss is minimized by the gradient descent method, thereby adjusting the weight parameters of the generator network. The above steps are repeated to traverse all original images in the original image dataset for multiple rounds of iteration until the perceptual loss converges and the generator network parameters stabilize, and the iteration stops to obtain the specific style model. Generate a specific reference image using a specific style model: The original image to be transformed is cropped, rotated, and mean-removed to obtain a standardized data format required by the generator network. This data is then input into the specific style model, and the generator network of the specific style model outputs an original image of a specific style, thus completing the image style transfer.

2. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The generating network is a convolutional neural network, which includes an input layer, an encoding layer, a decoding layer and an output layer. The encoding layer consists of multiple convolutional layers and is used to convert the original image into abstract content features. The decoding layer includes deconvolutional layers and upsampling layers and is used to restore the image resolution and fuse style features.

3. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The loss network is a pre-trained VGG16 convolutional neural network, which includes convolutional layers and pooling layers.

4. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The original image dataset includes images of landscapes, people, and animals, with at least 1,000 images of each.

5. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The number of iterations is no less than 50. After each iteration, the perceptual loss of the current iteration is calculated. When the rate of change of the perceptual loss is less than 0.001 for 5 consecutive iterations, the perceptual loss is considered to have converged and the iteration is stopped.

6. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The method for removing the mean is as follows: calculate the pixel mean of the RGB three channels of the original image training set and the pixel mean of the RGB three channels of the original image to be converted. Subtract the pixel mean of the RGB three channels of the original image training set from the pixel mean of the RGB three channels of the original image to be converted to obtain the mean.

7. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The gradient descent method employs the Adam optimizer, with an initial learning rate of 0.

001. After every 10 iterations, the learning rate decays to 0.5 of the current value, balancing the convergence speed and stability of model training.

8. The fast image style transfer method based on deep learning as described in claim 1, characterized in that, The perceptual loss is the sum of content loss and style loss; The content loss is calculated as follows: the degree of difference between the original image and the generated image is quantified, feature information is extracted using shallow convolutional layers in the convolutional neural network, and then the content loss is calculated by measuring the distance between the feature representations of the generated image and the original image on that layer. The style loss is calculated as follows: the similarity between the generated image and the reference image is quantified by comparing their statistical features. The Gram matrix is ​​used to represent the style information of each feature layer, and the Gram matrix difference between the generated image and the style image is calculated, which is the style loss.

Images