Infrared ship image expansion method based on unreal engine and StyleGAN3

Abstract

The invention provides an infrared ship image expansion method based on an unreal engine and StyleGAN3. The method comprises the following steps: calculating ship surface radiance distribution field texture and atmospheric radiation transmission data; constructing a simulated infrared sea surface based on an unreal engine; obtaining a three-dimensional dynamic infrared simulation scene based on an unreal engine; obtaining a training sample set; performing iterative training on the pattern generative adversarial network; and obtaining an expansion result of the infrared ship image. According to the method, iterative training is carried out on the StyleGAN3 through the simulated infrared sea surface and three-dimensional dynamic infrared simulation scene information obtained based on the unreal engine, so that the simulation rendering capability of the engine is fully utilized, and meanwhile, more high-resolution characteristics can be reserved; and the training sample set contains more feature changes brought by the feature distribution simulation image of the real shot infrared ship image, so that the resolution and fidelity of the expanded image are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and relates to an infrared ship image augmentation method based on Unreal Engine and StyleGAN3, which can be applied to ship target recognition and detection in the infrared field. Background Art

[0002] Image augmentation refers to the technology of artificially increasing the number of training samples by performing a series of transformations on image data or synthesizing new images using a generative model. This method can enrich the diversity of the dataset and improve the generalization ability of the model, and is particularly suitable for scenarios with limited original data volume.

[0003] Infrared ship image augmentation refers to generating diverse infrared ship images through different technical means to increase the number of training samples, mainly including two methods: image augmentation based on generative adversarial networks and image augmentation based on simulation generation.

[0004] The augmentation method based on generative adversarial networks learns the feature distribution of infrared ship images and generates new infrared ship images through a generator. Among them, a typical augmentation method based on style generative adversarial networks, for example, the patent document with the authorization announcement number CN111814875B and the name "Infrared image ship sample augmentation method based on style generative adversarial networks" of Xidian University. This invention adopts a style generative adversarial network in which a discriminator and a generator are alternately trained, and a method of modulating each layer of the generator with a random feature vector to realize the output of infrared image ship samples by the generator through the random feature vector for augmentation, solving the problems of poor authenticity due to complex simulation modeling, difficult acquisition of visible-light to infrared image optoelectronic conversion training samples, and lack of diversity in the augmented infrared images due to the small number of training sets. However, when the augmented image exceeds a resolution of 256×256, the image loses detailed features, the ships in the augmented image are distorted and the details are blurred, that is, there is an image resolution bottleneck, which will affect the recognition success rate of downstream ship target recognition tasks. In addition, this method requires at least 2000 actual photographed infrared ship images as a training set during the training process, and it is difficult to obtain such a large-scale actual photographed infrared ship images in practice.

[0005] Typical image augmentation based on simulation, such as the patent application with the publication number CN118334472A and the title "Method and System for Generating Infrared Ship Target Sample Library and Infrared Ship Target Recognition Based on CG Simulation" applied by Zhejiang University of Technology, discloses an infrared ship image augmentation method based on CG simulation. This method combines MORTAN and Unity 3D to realize the simulation of infrared ship target images on the sea surface. A large number of infrared augmented ship images can be obtained from the simulation scene, solving the problem of the lack of real-shot infrared ship image data. Its deficiencies are that due to the limitations of the Unity 3D engine, the realism of the infrared sea surface and ships is insufficient, and there is a loss of accuracy under long-distance observation. At the same time, only simulation is used for infrared ship image augmentation, lacking the distribution characteristics of real infrared ship images. Summary of the Invention

[0006] The object of the present invention is to overcome the defects existing in the above-mentioned prior art, and propose an infrared ship image augmentation method based on the Unreal Engine and StyleGAN3, which is used to solve the technical problems of low resolution and poor realism of the augmented images in the prior art.

[0007] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0008] (1) Calculate the texture of the radiance distribution field on the ship's surface and the data of atmospheric radiative transfer:

[0009] Calculate the texture of the radiance distribution field on the ship's surface in the simulation scene and the data of atmospheric radiative transfer including the atmospheric transmittance texture and the path radiance texture through the FBX ship model parameters and the infrared real sea scene parameters;

[0010] (2) Construct a simulated infrared sea surface based on the Unreal Engine:

[0011] Based on the Unreal Engine, convert the sea surface radiance calculated by the surface seawater temperature and seawater emissivity of the infrared real sea scene into a simulated infrared sea surface;

[0012] (3) Obtain a three-dimensional dynamic infrared simulation scene based on the Unreal Engine:

[0013] Assign the texture of the radiance distribution field on the ship's surface to the material of the FBX ship model, and render the FBX ship model with the assigned material, the data of atmospheric radiative transfer, and the simulated infrared sea surface based on the Unreal Engine to obtain a three-dimensional dynamic infrared simulation scene;

[0014] (4) Obtain a training sample set:

[0015] Preprocess multiple infrared real and infrared simulation images of real and virtual targets at different angles in the infrared real sea scene and infrared simulation scene obtained by the infrared camera, and form a training sample set with the preprocessed infrared real images and infrared simulation images;

[0016] (5) Iteratively train the style generation adversarial network:

[0017] Iteratively train the style generation adversarial network StyleGAN3 with the training sample set to obtain a trained style generation adversarial network;

[0018] (6) Obtain the expansion result of the infrared ship image:

[0019] Input the randomly sampled latent vector into the trained style generation adversarial network, and generate an infrared ship image similar to the characteristics of the training samples through the generator.

[0020] 1. The present invention iteratively trains the style generation adversarial network StyleGAN3 by including the simulated infrared sea surface and three-dimensional dynamic infrared simulation scene information obtained based on the Unreal Engine, making full use of the simulation rendering ability of the engine to obtain a more realistic ocean scene, and at the same time being able to retain more high-resolution features. Compared with the prior art, it effectively improves the resolution and realism of the expanded images.

[0021] 2. Since the training sample set of the present invention contains real-shot and simulated infrared ship images, during the training process of StyleGAN3, it can not only learn the feature distribution of the real-shot infrared ship images, but also consider the more feature variations brought by the simulation images, making the expanded images more diverse. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the implementation flowchart of the present invention.

[0023] Figure 2 is the resolution-expanded image obtained in the embodiment of the present invention.

[0024] Figure 3 is the simulation comparison diagram of the fidelity of the expanded images between the present invention and the prior art. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments.

[0026] Refer to Figure 1 , the present invention includes the following steps:

[0027] Step 1) Calculate the texture of the radiance distribution field on the ship's surface and the atmospheric radiation transmission data:

[0028] Calculate the texture of the radiance distribution field on the ship's surface in the simulation scenario and the atmospheric radiation transmission data including the atmospheric transmittance texture and the path radiation texture through the FBX ship model parameters and the infrared real sea scene parameters;

[0029] Calculate the texture of the radiance distribution field on the ship's surface in the simulation scenario. The implementation steps are as follows:

[0030] Use Ansys software, and through the FBX ship model and its unfolded UV texture including I = P × Q pixels, as well as the time of shooting each ship target in the infrared real sea scene and the longitude and latitude coordinates of the ship target, calculate the temperature field texture on the surface of the ship target; then based on Planck's blackbody radiation law, the calculated temperature field texture on the surface of the ship target and the emissivity texture of the FBX ship model and its unfolded including I = P × Q pixels, calculate the texture of the radiance distribution field on the ship's surface in the simulation scenario, where the value L of the i-th pixel point in the radiance distribution field texture i (T i , ε i ) is calculated by the formula:

[0031]

[0032] where P ≥ 512, Q ≥ 512; T i and ε i respectively represent the values of the i-th pixel point of the temperature field texture and the emissivity texture on the surface of the ship target, and L λ (T i , ε i ) represents the spectral radiance value of the i-th pixel point, λ1 and λ2 respectively represent the starting wavelength and the ending wavelength of the infrared camera imaging, e is the base of the natural logarithm, h is Planck's constant, c is the speed of light, and k B is the Boltzmann constant;

[0033] The FBX model is a common 3D file format that can not only store model mesh data such as vertices, edges, and faces, but also contains the texture maps and materials of the model. The ship model is from an open-source website, and the model selected is similar to the ship type in the infrared real sea scene being photographed;

[0034] The UV texture is a method for mapping a 2D texture image onto the surface of a 3D model. The U coordinate represents the horizontal direction of the texture, and the V coordinate represents the vertical direction of the texture. These two coordinates determine which part of the texture image will be mapped to a specific position on the model surface; the emissivity texture is a UV texture that stores the emissivity at each position on the model surface, and a coordinate position corresponds to the emissivity value at a specific position on the model surface;

[0035] Calculate the atmospheric transmittance texture and the path radiation texture. The implementation steps are as follows:

[0036] For the range of the distance d of the ship target observed by the camera in the infrared real sea scene (d min , d max ) and the range of the zenith angle θ (θ min , θ max ), N uniform samplings are respectively carried out, and the MODTRAN atmospheric transmission calculation software is used to calculate the atmospheric transmittance texture and the path radiance texture through the collected parameters and the infrared real sea scene parameters. The infrared real sea scene parameters include the geographical location and shooting date of the shooting area, and the meteorological condition parameters including wind speed and cloud cover, where N ≥ 32;

[0037] MODTRAN is a widely used atmospheric radiation transmission calculation software that can accurately calculate path radiance and atmospheric transmittance and is often used in the simulation of infrared imaging systems;

[0038] The zenith angle refers to the angle between the camera and directly above. When the ship target is on the horizontal line, the zenith angle is 90°; when the target is directly below, the zenith angle is 180°. In atmospheric transmission and infrared imaging, the zenith angle directly affects the path length of the radiation passing through the atmosphere, thus having a significant impact on the atmospheric transmittance and path radiance.

[0039] Step 2) Construct a simulated infrared sea surface based on the Unreal Engine:

[0040] Based on the Unreal Engine, convert the sea surface radiance calculated by the surface seawater temperature and seawater emissivity of the infrared real sea scene into a simulated infrared sea surface;

[0041] The method for constructing a simulated infrared sea surface based on the Unreal Engine is as follows:

[0042] (2a) Based on Planck's blackbody radiation law, calculate the sea surface radiance through the surface seawater temperature and seawater emissivity of the infrared real sea scene;

[0043] (2b) Based on the Oceannology dynamic ocean simulation plug-in in the Unreal Engine, construct a sea surface model in the visible light band, and output the sea surface radiance value through the self-illumination channel of the sea surface model material to obtain a simulated infrared sea surface;

[0044] The Oceannology plug-in is a dynamic ocean simulation plug-in developed specifically for the Unreal Engine. It can calculate the undulation and waveform of sea waves in real time based on a physical model and can render a realistic sea surface effect. With the help of this plug-in, a highly realistic visible light sea surface scene can be quickly built and rendered;

[0045] Step 3) Obtain a three-dimensional dynamic infrared simulation scene based on the Unreal Engine:

[0046] Assign the texture of the ship surface radiance distribution field to the material of the FBX ship model, and render the FBX ship model with the assigned material, the atmospheric radiation transmission data, and the simulated infrared sea surface based on the Unreal Engine to obtain a three-dimensional dynamic infrared simulation scene;

[0047] The steps to render the FBX ship model with the assigned material, the atmospheric radiation transmission data, and the simulated infrared sea surface based on the Unreal Engine are as follows:

[0048] Place the FBX ship model with the assigned material in the simulated infrared sea surface, and use the post-processing material system of the Unreal Engine to perform real-time sampling of the atmospheric radiation transmission data according to the distance d and the zenith angle θ of the camera observing the ship target, simulate the atmospheric effects at different spatial positions in the real scene, and then render the sampling results, the FBX ship model with the assigned material, and the simulated infrared sea surface to obtain a three-dimensional dynamic infrared simulation scene;

[0049] With its high-fidelity rendering, physical simulation accuracy, and real-time performance, the Unreal Engine has become an ideal tool for infrared sea surface ship simulation. Moreover, the Unreal Engine can render a wave model with tens of millions of geometric patches in real time, and use GPU-accelerated wave Niagara particle simulation to drive, enabling high-resolution and high-precision real-time three-dimensional infrared sea scene rendering.

[0050] Step 4) Obtain the training sample set:

[0051] Preprocess multiple infrared real and infrared simulation images of multiple real and virtual targets at different angles in the infrared real sea scene and infrared simulation scene obtained by the infrared camera, and form a training sample set with the preprocessed infrared real images and infrared simulation images;

[0052] The steps to preprocess multiple infrared real and infrared simulation images of multiple real and virtual targets at different angles in the infrared real sea scene and infrared simulation scene obtained by the infrared camera are as follows:

[0053] Upsample each infrared real image, and at the same time add Gaussian noise to each infrared simulation image and then adjust the contrast. Then, perform channel dimension expansion on each upsampled infrared real image with the same resolution as the simulation image and each adjusted infrared simulation image respectively to obtain multiple preprocessed infrared real images and infrared simulation images in RGB three-channel format. The training sample set contains M infrared real images and N infrared simulation images, where M ≥ 30 and 20 ≤ N ≤ M;

[0054] Step 5) Iteratively train the style generation adversarial network:

[0055] Iteratively train the style generation adversarial network StyleGAN3 through the training sample set to obtain a trained style generation adversarial network;

[0056] Style generation adversarial network StyleGAN3, including: a cascaded mapping network M and a synthesis network G S The generator G composed of G S The output end is connected to the discriminator D; where M includes multiple stacked fully connected layers and a LeakyReLU activation function; G S Includes multiple stacked SynthesisLayer layers and a ToRGB modulation convolution layer; D includes multiple stacked convolution layers and a fully connected layer;

[0057] The mapping network M consists of 8 fully connected layers, each of which uses LeakyReLU activation;

[0058] The synthesis network S passes through the initial input module and 9 resolution stages, with a total of 16 layers of SynthesisLayer. Among them, the 9 resolution stages start from the initial 4×4, and the network upsamples the feature map in turn. The target resolutions are:

[0059] 4×4→8×8→16×16→32×32→64×64→128×128→256×256→512×512→1024×1024

[0060] The discriminator includes multiple layers of convolution and a fully connected layer. The convolution kernel size is 3×3, the step size is generally 2, the padding is 1, and the number of channels gradually increases from 64 to 512. Finally, the fully connected layer outputs the true and false discrimination results. During training, the R1 gradient penalty is applied as a regularization constraint.

[0061] StyleGAN3's cycle consistency constraint and noise injection mechanism enable the generated images to meet physical consistency while maintaining rich details. It adopts a progressive training method, starting from low resolution and gradually increasing the resolution of the generated images. At the same time, it adopts an adaptive learning rate strategy to accelerate the training process and shorten the training time. StyleGAN3 performs well in generating high-resolution images of complex scenes and objects such as ships.

[0062] Iterative training of the style generative adversarial network StyleGAN3 is implemented as follows:

[0063] (5a) The number of initial iterations is t, the maximum number of iterations is T, T ≥ 500, and the network parameters of the generator G and the discriminator D are θ G ,θ D , and let t = 0;

[0064] (5b) The mapping network M in the generator G maps the randomly sampled latent vector z to the intermediate latent space w, and the synthesis network G S Performs modulated convolution, filtering, and layer-by-layer upsampling on the obtained intermediate latent space w through the SynthesisLayer. The ToRGB layer performs convolutional mapping on the upsampled result to generate an infrared ship image G(z) with features similar to the training sample x; the discriminator D performs feature extraction on x and G(z) respectively and then performs linear mapping to obtain the scores D(x) and D(G(z)) of x and G(z);

[0065] (5c) Calculate the loss value L of the generator G through D(x) and D(G(z)) G and the loss value L of the discriminator D D , and through L G and L D update the generator parameters θ G , discriminator parameters θ D to obtain the style generative adversarial network for this iteration;

[0066] (5d) Determine whether t≥T holds. If so, obtain the trained style generative adversarial network. Otherwise, set t = t + 1 and execute step (5b);

[0067] The loss value L of the generator G G and the loss value L of the discriminator D D , and the calculation formulas are respectively:

[0068]

[0069] where represents the expectation value, λ pl , L pl respectively represent the path length regularization weight and loss of the generator G, λ r1 , L r1 respectively represent the R1 regularization weight and loss of the discriminator D, ||▽ z G(z)||2 represents the second-order norm of the gradient of G(z) with respect to w, const is a constant, defaulting to 1, represents the square of the second-order norm of the gradient of D(x) with respect to x.

[0070] Step 6) Obtain the augmented result of the infrared ship image:

[0071] Input the randomly sampled latent vector into the trained style generative adversarial network, and generate an infrared ship image with features similar to the training sample through the generator;

[0072] Next, in combination with the experimental results, the technical effects of the present invention will be further described:

[0073] 1. Experimental conditions and content:

[0074] The hardware platform for the experiment is as follows: The processor is an Intel(R) Core i9-12900K CPU, the memory is 64GB, and the graphics card is an NVIDIA GeForce RTX 3080Ti. The experimental simulation platform is: Unreal Engine 4.27. The experimental software platform is: Windows 11 operating system, Python version is 3.9.20, Pytorch version is 2.0.0, and CUDA version is 11.8.

[0075] Simulation 1. A comparative simulation was carried out on the fidelity of the expanded images of the present invention and the prior art, and the results are as Figure 2 shown.

[0076] Simulation 2. A simulation was carried out on the resolution-expanded images obtained in the embodiments of the present invention, and the results are as Figure 3 shown.

[0077] 2. Analysis of experimental results:

[0078] Referring to Figure 2 , in which Figure 2 (a), Figure 2 (b) are respectively three-dimensional dynamic infrared simulation scene diagrams of the prior art and the present invention based on the Unreal Engine, including simulated infrared sea surfaces and ships. It can be seen from the figures that the sea skyline of the images obtained by the present invention is clearly demarcated, the wave changes on the sea surface are natural and in line with reality, and the detailed features of the ship's masts are obvious. The above effects are more realistic than the images of the prior art. Therefore, the present invention effectively improves the realism of the simulated ocean scene and solves the technical problem of poor realism in the prior art;

[0079] Referring to Figure 3 , which is a 1024×1024 resolution infrared ship expansion image obtained by iterative training through the StyleGAN3 of the style generation adversarial network. From Figure 3 it can be seen that for the infrared ship images expanded by the present invention, the sea horizon is still clearly visible, the infrared sea surface still retains the undulation of the waves, and the edge features of the infrared ships are clear without blurring or distortion. It is proved that the present invention effectively solves the problem of the resolution bottleneck existing in the images expanded by the prior art.

Claims

1. A method for expanding infrared ship images based on Unreal Engine and StyleGAN3, characterized in that: The steps include: (1) Calculate the ship surface radiance distribution field texture and atmospheric radiation transmission data: The FBX ship model parameters and infrared real sea scene parameters are used to calculate the ship surface radiance distribution field texture and the atmospheric radiation transmission data including the atmospheric transmittance texture and path radiation texture in the simulation scene; (2) Building a simulated infrared sea surface based on Unreal Engine: Based on Unreal Engine, the sea surface radiance calculated by the surface sea temperature and sea emissivity of the infrared real sea scene is converted into a simulated infrared sea surface; (3) Obtaining 3D dynamic infrared simulation scene based on Unreal Engine: The ship surface radiance distribution field texture is assigned to the material of the FBX ship model, and the FBX ship model assigned with the material, the atmospheric radiation transmission data and the simulated infrared sea surface are rendered based on the Unreal Engine to obtain a three-dimensional dynamic infrared simulation scene; (4) Obtain training sample set: Preprocess multiple infrared real and infrared simulated images of multiple real and virtual targets at different angles in infrared real sea scenes and infrared simulated scenes acquired by infrared cameras, and form a training sample set with the preprocessed infrared real images and infrared simulated images; (5) Iterative training of the style generative adversarial network: Iteratively train the style generation adversarial network StyleGAN3 through the training sample set to obtain a trained style generation adversarial network; (6) Obtain the expanded results of infrared ship images: The randomly sampled latent vectors are input into the trained style generative adversarial network, and the generator is used to generate infrared ship images with similar features to the training samples.

2. The method according to claim 1, characterized in that The calculation of the ship surface radiance distribution field texture in the simulation scene described in step (1) is implemented by: The temperature field texture of the ship target surface is calculated by using Ansys software, the FBX ship model and its unfolded UV texture including I=P×Q pixels, as well as the time of shooting each ship target in the infrared real sea scene and the longitude and latitude coordinates of the ship target; Then, based on Planck's blackbody radiation law, the temperature field texture of the calculated ship target surface and the FBX ship model and its expanded emissivity texture including I = P × Q pixels are used to calculate the radiance distribution field texture of the ship surface in the simulation scene, where the i-th pixel value L in the radiance distribution field texture is i (T i ,ε i ) is calculated as: Among them, P≥512, Q≥512; T i and ε i They represent the temperature field texture and emissivity texture of the ship target surface, and L λ (T i ,ε i ) represents the spectral radiance value of the i-th pixel, λ1 and λ2 represent the starting wavelength and ending wavelength of the infrared camera imaging, e is the base of the natural logarithm, h is Planck's constant, c is the speed of light, k B is the Boltzmann constant.

3. The method according to claim 1, characterized in that The steps for calculating the atmospheric transmittance texture and path radiation texture described in step (1) are as follows: The range of the distance d of the camera observing the ship target in the infrared real sea scene (d min ,d max ) and the range of the zenith angle θ (θ min ,θ max ) were uniformly sampled N times respectively, and the atmospheric transmittance texture and path radiation texture were calculated by MODTRAN atmospheric transmission calculation software through the collected parameters and infrared real sea scene parameters, where the infrared real sea scene parameters include the geographical location and shooting date of the shooting area, as well as meteorological condition parameters including wind speed and cloud cover, N ≥ 32.

4. The method according to claim 1, characterized in that: The specific method of constructing a simulated infrared sea surface based on Unreal Engine in step (2) is as follows: (2a) Based on Planck's blackbody radiation law, the sea surface radiance is calculated using the surface sea temperature and seawater emissivity of the infrared real sea scene; (2b) Based on the Oceannology dynamic ocean simulation plug-in in the Unreal Engine, a sea surface model in the visible light band is constructed, and the sea surface radiance value is output through the self-luminous channel of the sea surface model material to obtain a simulated infrared sea surface.

5. The method according to claim 1, characterized in that The Unreal Engine is used to render the FBX ship model with material, the atmospheric radiation transmission data and the simulated infrared sea surface in step (3), and the implementation steps are as follows: The FBX ship model with material is placed in the simulated infrared sea surface, and the atmospheric radiation transmission data is sampled according to the distance d and zenith angle θ of the camera observing the ship target based on the post-processing material system of Unreal Engine. Then, the sampling results, the FBX ship model with material and the simulated infrared sea surface are rendered to obtain a three-dimensional dynamic infrared simulation scene.

6. The method according to claim 1, characterized in that The preprocessing of the multiple infrared real and infrared simulated images of multiple real and virtual targets at different angles in the infrared real sea scene and the infrared simulated scene obtained by the infrared camera in step (4) is implemented by: Each infrared real image is upsampled, and Gaussian noise is added to each infrared simulated image for contrast adjustment. The channel dimension of each upsampled infrared real image with the same resolution as the simulated image and each adjusted infrared simulated image are expanded to obtain multiple preprocessed infrared real images and infrared simulated images in RGB three-channel format.

7. The method according to claim 6, characterized in that The style generation adversarial network StyleGAN3 described in step (5) includes a cascaded mapping network M and a synthesis network G S The generator G composed of G S The output end is connected to the discriminator D; where M includes multiple stacked fully connected layers and a Leaky ReLU activation function; G S It includes multiple stacked SynthesisLayer layers and a ToRGB modulation convolution layer; D includes multiple stacked convolution layers and a fully connected layer.

8. The method according to claim 7, characterized in that The iterative training of the style generative adversarial network StyleGAN3 described in step (5) is implemented as follows: (5a) The number of initial iterations is t, the maximum number of iterations is T, T ≥ 500, and the network parameters of the generator G and the discriminator D are θ G ,θ D , and let t = 0; (5b) The mapping network M in the generator G maps the randomly sampled latent vector z to the intermediate latent space w, the synthetic network G S The intermediate latent space w obtained by mapping is modulated, convolved, filtered and upsampled layer by layer through SynthesisLayer. The ToRGB layer performs convolution mapping on the upsampled result to generate an infrared ship image G(z) with similar features to the training sample x. The discriminator D extracts features from x and G(z) respectively and then performs linear mapping to obtain the scores D(x) and D(G(z)) of x and G(z). (5c) Calculate the loss value L of the generator G through D(x) and D(G(z)) G And the loss value L of the discriminator D D , and through L G and L D For the generator parameter θ G , discriminator parameters θ D Update to obtain the style generation adversarial network of this iteration; (5d) Determine whether t≥T holds. If so, obtain the trained style generative adversarial network. Otherwise, set t=t+1 and execute step (5b).

9. The method according to claim 8, characterized in that The loss value L of the generator G described in step (5c) G And the loss value L of the discriminator D D , the calculation formulas are: in, represents the expected value, λ pl , L pl Respectively represent the path length regularization weight and loss of the generator G, λ r1 , L r1 Respectively represent the R1 regularization weight and loss of the discriminator D, Represents the second-order norm of the gradient of G(z) to w. const is a constant, the default value is 1. Denotes the square of the second-order norm of the gradient of D(x) with respect to x.

Images