Image processing method, device and computer-readable storage medium

Abstract

The present disclosure proposes a method, device and computer-readable storage medium for image processing, and relates to the field of image processing technology. The present disclosure provides a method for image processing, comprising: inputting the image features extracted by the feature extractor of the first domain into the generator of the first generative adversarial network to obtain the completed features; inputting the completed features into the generator of the second generative adversarial network to obtain the style transfer image; wherein, the images taken in the real environment used for the first generative adversarial network training and the images generated after the training are attributed to the first domain, the images used for the second generative adversarial network training and the images generated after the training are attributed to the second domain, and the latent spaces of the first domain and the second domain are aligned. Through such a method, based on the generation capability of the generative adversarial network, it is possible to complete the completion of character features and introduce the style transfer task, thereby improving the scope of application and credibility of the image style transfer effect.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a picture processing method, device and computer-readable storage medium. Background Art

[0002] In the person recognition system, changes in background, lighting, angle, camera parameters and other factors often lead to missing or erroneous character feature extraction, resulting in poor performance in direct application scenarios, and the effect of further processing based on such images will also be affected. Summary of the invention

[0003] One purpose of the present disclosure is to improve the applicability and credibility of image style transfer effects.

[0004] According to one aspect of some embodiments of the present disclosure, a method for processing an image is proposed, including: inputting image features extracted by a feature extractor of a first domain into a generator of a first generative adversarial network to obtain completed features, and the generator of the first generative adversarial network is trained to complete the missing features of images taken in a real environment; inputting the completed features into a generator of a second generative adversarial network to obtain style transfer images, and the generator of the second generative adversarial network is trained to restore the input features to images with a consistent style with the second domain images; wherein the images taken in a real environment used for training the first generative adversarial network and the images generated after the training belong to the first domain, the images used for training the second generative adversarial network and the images generated after the training belong to the second domain, and the latent spaces of the first domain and the second domain are aligned.

[0005] In some embodiments, the image processing method also includes: training a first generative adversarial network based on images with known features; obtaining data completed by the generator of the first generative adversarial network and a first domain to which the images with known features belong; training a second generative adversarial network based on images belonging to the second domain; and aligning the latent spaces of the first domain and the second domain.

[0006] In some embodiments, training the first generative adversarial network based on pictures with known features includes: obtaining a character feature vector through a feature extractor of the first domain based on pictures taken in a real environment; randomly omitting one or more features in the character feature vector, and completing the features through the generator of the first generative adversarial network; the discriminator of the first generative adversarial network determines the authenticity of the features completed by the generator based on the known character feature vector; and repeatedly performing the operation of completing the features through the generator of the first generative adversarial network until the output of the discriminator reaches a predetermined first target.

[0007] In some embodiments, training a second generative adversarial network based on pictures belonging to different styles includes: obtaining style picture features from pictures belonging to the second domain through a feature extractor of the second domain; inputting the style picture features into a generator of a second generative adversarial network to change the style of the picture; a discriminator of the second generative adversarial network determines whether the picture output by the generator of the second generative adversarial network belongs to the second domain; and repeatedly inputting pictures belonging to the second domain into the generator of the second generative adversarial network until the output of the discriminator reaches a predetermined second target.

[0008] In some embodiments, aligning the latent space of the first domain with that of the second domain includes: introducing data of the encoding encoder layer of the network structure of the first domain and the network structure of the second domain into the hidden layer, and binding the data of the hidden layer introduced in the first generative adversarial network and the second generative adversarial network; inputting the first image belonging to the first domain into the network structure of the first domain, and obtaining the first vector after passing through the hidden layer; the decoding decoder layer of the network structure of the second domain obtains the second image according to the first vector; inputting the second image into the encoder layer of the network structure of the second domain, and obtaining the second vector after passing through the hidden layer; obtaining the distance between the first vector and the second vector; and repeatedly performing the operation of inputting the same or different first image belonging to the first domain into the first generative adversarial network until the distance between the first vector and the second vector is less than a predetermined loss value.

[0009] In some embodiments, the algorithm based on which the generation layer of the first generative adversarial network is based includes a recurrent neural network RNN, a gated recurrent unit GRU or a long short-term memory network LSTM; the algorithm based on which the discriminator of the first adversarial network is based includes a multilayer perceptron MLP, a support vector machine SVM, and naive Bayes.

[0010] In some embodiments, the algorithm based on which the generator of the second generative adversarial network is based includes a neural network; the algorithm based on which the discriminator of the second adversarial network is based includes MLP, SVM, and Naive Bayes.

[0011] Through this method, based on the generation ability of GAN, it is possible to associate the remaining features of the character through partial features, complete the completion of the character features, and introduce the style transfer task, thereby improving the scope of application and credibility of the image style transfer effect.

[0012] According to one aspect of some embodiments of the present disclosure, a picture processing device is proposed, including: a completion unit, configured to input picture features extracted by a feature extractor of a first domain into a generator of a first generative adversarial network to obtain the completed features, and the generator of the first generative adversarial network is trained to complete the missing features of pictures taken in a real environment; a style transfer unit, configured to input the completed features into a generator of a second generative adversarial network to obtain a style transferred picture, and the generator of the second generative adversarial network is trained to modify the picture to which the input features belong to the style of the picture of the second domain; wherein the pictures taken in the real environment used for training the first generative adversarial network and the pictures generated after the training belong to the first domain, the pictures used for training the second generative adversarial network and the pictures generated after the training belong to the second domain, and the latent spaces of the first domain and the second domain are aligned.

[0013] In some embodiments, the image processing device also includes: a first training unit, configured to train a first generative adversarial network based on images with known features; a first domain acquisition unit, configured to acquire data completed by a generator of the first generative adversarial network, and the first domain to which the images with known features belong; a second training unit, configured to train a second generative adversarial network based on images belonging to the second domain; and a latent space alignment unit, configured to align the latent spaces of the first domain and the second domain.

[0014] According to one aspect of some embodiments of the present disclosure, a picture processing device is proposed, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the above picture processing methods based on instructions stored in the memory.

[0015] Such a device can, based on the generation capability of GAN, associate the remaining features of a person with some of their features, complete the completion of the person's features, and introduce the style transfer task, thereby improving the scope of application and credibility of the image style transfer effect.

[0016] According to one aspect of some embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the steps of any one of the above-mentioned image processing methods are implemented.

[0017] By executing the instructions on such a computer-readable storage medium, it is possible to, based on the generation capability of GAN, associate the remaining features of the character through partial features, complete the completion of the character's features, and introduce the style transfer task to improve the scope of application and credibility of the image style transfer effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0019] Figure 1 The following are flowcharts of some embodiments of the image processing method disclosed herein.

[0020] Figure 2 The present invention is a flowchart of some embodiments of a method for generating an adversarial network in the image processing method of the present invention.

[0021] Figure 3 Schematic diagrams of some embodiments of the image processing method disclosed herein.

[0022] Figure 4 Schematic diagrams of some embodiments of the image processing device disclosed herein.

[0023] Figure 5 Schematic diagrams of other embodiments of the image processing device disclosed herein.

[0024] Figure 6 Schematic diagrams of some further embodiments of the image processing device disclosed herein. DETAILED DESCRIPTION

[0025] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.

[0026] The flowcharts of some embodiments of the picture processing method disclosed in the present invention are as follows: Figure 1 shown.

[0027] In step 101, the features of the image are input into the generator of the first generative adversarial network to obtain the completed features of the image. In some embodiments, the image to be processed is first input into the feature extractor of the first domain, and the extracted features are input into the generator of the first generative adversarial network. In some embodiments, the generator of the first generative adversarial network is trained to complete the missing features of the images taken in the real environment.

[0028] In step 102, the completed features of the image are input into the generator of the second generative adversarial network to obtain a style-transferred image. The generator of the second generative adversarial network is trained to restore the input features to an image with the same style as the image in the second domain.

[0029] In some embodiments, pictures taken in a real environment and generated pictures used for training a first generative adversarial network belong to a first domain, pictures taken in a real environment and generated pictures used for training a second generative adversarial network belong to a second domain, and the latent spaces of the first domain and the second domain are aligned.

[0030] GAN (Generative Adversarial Network) is a deep learning framework proposed by Ian Goodfellow in 2014. It constructs two models, the generator and the discriminator. Its main goal is to use the discriminator network to assist in training a generative network that can accurately learn the distribution characteristics of the original data. The generative network and the discriminator network continuously improve their performance in mutual competition and confrontation, reaching a dynamic balance and optimizing their performance.

[0031] Since the proposal of generative adversarial networks, related research and application scenarios have spawned various types of variants, which are mainly used in fields such as image data generation, text and speech generation.

[0032] Through the method in the above embodiment, based on the generation ability of GAN, the remaining features can be associated with partial features of the character, the character features can be completed, and the style transfer task can be introduced, thereby improving the scope of application and credibility of the image style transfer effect.

[0033] The flowcharts of some embodiments of the method for generating adversarial networks in the image processing method of the present disclosure are as follows: Figure 2 shown.

[0034] In step 201, a first generative adversarial network is trained based on images with known features.

[0035] In some embodiments, a feature extractor can be used to extract character features from real-world images. Character features include, but are not limited to: basic facial features, age, gender, height, hairstyle, whether there are glasses, whether there is a beard, gait features, etc., and a combination of several appropriate features is selected to form a feature vector representation space. Due to the limitations of real-world lighting, equipment, angles and other factors, the extracted features cannot be guaranteed to be complete or accurate.

[0036] A known complete and accurate model is used to randomly hide some features or several items of some features, which are filled by the generator. The discriminator determines the authenticity of the corresponding accurate features, thereby realizing the training of the first generative adversarial network.

[0037] In some embodiments, an adaptive network structure can be selected according to the specific data form, and a generator and a discriminator can be built to form an adversarial generative network architecture.

[0038] 1) Generator:

[0039] a. Training objectives:

[0040]

[0041] b. Since the data combined with character motion capture in real scenes is usually time series data, RNN and its variants are preferred as generators, including but not limited to RNN, GRU, LSTM, etc.

[0042] 2) Discriminator:

[0043] a. Training objectives:

[0044] maxDVD(D,G)=E x～Pdata ( x) [lOgD(x)]+E z～Pz(z) [log(1-D(G(z)))]

[0045] b. For the output data of the generator, select an appropriate model as the discriminator, including but not limited to: MLP, SVM, Naive Bayes and other common classifiers.

[0046] Where z is random noise data, and x is the data sampled from real samples. The goal of the generator is to convert random noise into synthetic features and obtain a higher score at the discriminator. The goal of the discriminator is to make the real data score higher and the synthetic data score lower for real data and synthetic data.

[0047] The generator and the discriminator are continuously and iteratively trained through an adversarial game mechanism until a predetermined first goal is reached. In some embodiments, the predetermined first goal may be a predetermined proportion, probability, or continuous number of things that the discriminator judges to be true.

[0048] In step 202, the data completed by the generator of the first generative adversarial network and the first domain to which the picture of the known feature belongs are obtained. Figure 3 As shown, Figure 3 The half above the dotted line, i.e., the X domain (first domain), illustrates the process of obtaining the output of the encoder Encoder X in the network structure of the first domain and then using the generative adversarial network to fill it in, thereby obtaining the filled or reconstructed vector. The synthetic data of the first domain is obtained using a fully trained generator.

[0049] In step 203, a second generative adversarial network is trained based on the features of the image belonging to the second domain. In some embodiments, the completed features can be input into the generator of the second generative adversarial network to obtain a style transfer image, and then the style image features can be input into the generator of the second generative adversarial network to change the style of the image. Figure 3As shown in FIG, the adversarial generative network training method for the second domain is as follows: select several images of the target domain, i.e., the style domain, perform feature extraction through Encoder Y of the Y domain (the second domain), obtain the feature vector, use the generator to generate the image, and the discriminator combines the original style domain image to make a true or false judgment (i.e., whether it is a picture of the style of this domain).

[0050] In some embodiments, the input data of the second domain's generative adversarial network model is the output in step 203. During the second generative adversarial network training process, a suitable network structure is selected to build a generator and a discriminator to form a generative adversarial network architecture.

[0051] 1) Generator:

[0052] a. Training objectives:

[0053]

[0054] b. Since the target result of character image style transfer in real scenes is image data, it is preferred to use neural networks and their variants as generators.

[0055] 2) Discriminator:

[0056] a. Training objectives:

[0057]

[0058] b. For the output image of the generator, the task of the discriminator is to distinguish whether the image is synthesized by the generator or a real image of Domain Y. Select an adapted model as the discriminator. Including but not limited to: MLP, SVM, Naive Bayes and other common classifiers.

[0059] Where z is random noise data, and x is data sampled from real samples. The goal of the generator is to convert random noise into synthetic features and obtain a higher score at the discriminator. The goal of the discriminator is to make the real data score higher and the synthetic data score lower for real data and synthetic data.

[0060] The generator and the discriminator are continuously and iteratively trained through an adversarial game mechanism until a predetermined second goal is reached. In some embodiments, the predetermined second goal may be a predetermined proportion, probability, or continuous number of things that the discriminator determines to be true.

[0061] In step 204, the latent spaces of the first domain and the second domain are aligned. In order to achieve smooth conversion between real domain images and specific style images, it is necessary to ensure that the latent spaces of the two domains are shared. By introducing a shared data layer through a cyclic generation-alignment method, and then reducing the distance between the vectors generated by the two domains in the latent space, the alignment and sharing of the latent space are achieved.

[0062] In some embodiments, after the two generative adversarial networks are trained separately, the method for completing the composite process of the two models through the latent space alignment step can be as follows:

[0063] In some embodiments, the latent space data of the first domain and the second domain can be obtained based on the same picture, and since the above two data come from the same picture, the latent space data of the two domains should be as consistent as possible, or retain the same key information. Vector distance or neural network is used here to measure the distance between the two latent space data. The goal of the training process is to minimize the distance. In some embodiments, the latent space data of the first domain can be obtained based on processing the real picture data through the encoding layer Encoder X of the network structure of the first domain; in some embodiments, the real picture data can be input from Encoder X, and the decoding layer Decoder Y (second generative adversarial network) of the network structure of the second domain generates a picture, and then the picture is processed by the encoding layer Encoder Y of the second generative adversarial network to obtain the latent space data of the second domain.

[0064] In some embodiments, two hidden layers can be introduced after the feature vector generation layer of the two GANs, respectively, to align the size of the vectors generated by the two encoding layer encoders and fuse the information. After the hidden layer is introduced, the following operations are performed: first, the hidden layer parameters used by the two GANs are bound, that is, the two models share two hidden layers. Then the encoding layer Encoder X of the network structure of the first domain completes the information extraction, completes the completion through the first generative adversarial network, and obtains vector A after propagating through the two newly added hidden layers. The generative adversarial network of the second domain generates a picture of the style domain, and finally the style domain picture passes through the encoding layer Encoder Y and two hidden layers of the network structure of the second domain to obtain vector B. The distance between vector A and vector B is used as the loss function of the training step, and the loss function is reduced to achieve the alignment of the latent space.

[0065] This method can combine two adversarial generative networks. By introducing two hidden layers with mutually bound parameters, the latent spaces of the two networks can be aligned and reused, thereby achieving the purpose of joint training and supporting multiple tasks, and improving the robustness of the style transfer task.

[0066] Schematic diagrams of some embodiments of the image processing device disclosed in the present invention are as follows Figure 4 shown.

[0067] The completion unit 401 can input the image features extracted by the feature extractor of the first domain into the generator of the first generative adversarial network to obtain a feature-completed image. In some embodiments, the generator of the first generative adversarial network is trained to complete the missing features of images taken in a real environment.

[0068] The style transfer unit 402 can input the padded features into the generator of the second generative adversarial network to obtain a style transfer picture, and the generator of the second generative adversarial network is trained to restore the input features to a picture with the same style as the picture of the second domain. In some embodiments, the pictures taken in the real environment for the first generative adversarial network training and the pictures generated after the training are completed belong to the first domain, the pictures used for the second generative adversarial network training and the pictures generated after the training are completed belong to the second domain, and the latent spaces of the first domain and the second domain are aligned.

[0069] Such a device can, based on the generation capability of GAN, associate the remaining features of a person with some of their features, complete the completion of the person's features, and introduce the style transfer task, thereby improving the scope of application and credibility of the image style transfer effect.

[0070] In some embodiments, Figure 4 As shown, the image processing apparatus may further include a first training unit 403 , a first domain acquisition unit 404 , a second training unit 405 and a latent space alignment unit 406 .

[0071] The first training unit 403 can train the first generative adversarial network on pictures with known features, such as using a known complete and accurate model, randomly hiding some features or several items of some features, using a generator to fill in, and the discriminator combines the corresponding accurate features to determine the true or false, thereby realizing the training of the first generative adversarial network.

[0072] The first domain acquisition unit 404 can obtain the data completed by the generator of the first generative adversarial network and the first domain to which the picture with known features belongs.

[0073] The second training unit 405 can train a second generative adversarial network based on pictures belonging to the second domain.

[0074] The latent space alignment unit 406 can align the latent spaces of the first domain and the second domain.

[0075] In some embodiments, the data processing method of the first training unit 403, the first domain acquisition unit 404, the second training unit 405 and the latent space alignment unit 406 can be as follows: Figure 2 As shown in the examples.

[0076] Such an image processing device can combine two adversarial generative networks. By introducing two hidden layers with mutually bound parameters, the latent spaces of the two networks can be aligned and reused, thereby achieving the purpose of joint training and supporting multiple tasks, and improving the robustness of the style transfer task.

[0077] A structural diagram of an embodiment of the image processing device disclosed in the present invention is as follows Figure 5 As shown. The image processing device includes a memory 501 and a processor 502. Among them: the memory 501 can be a disk, a flash memory or any other non-volatile storage medium. The memory is used to store the instructions in the corresponding embodiment of the image processing method above. The processor 502 is coupled to the memory 501 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 502 is used to execute the instructions stored in the memory, which can improve the scope of application and credibility of the image style transfer effect.

[0078] In one embodiment, it is also possible to Figure 6 As shown, the image processing device 600 includes a memory 601 and a processor 602. The processor 602 is coupled to the memory 601 via a BUS bus 603. The image processing device 600 can also be connected to an external storage device 605 via a storage interface 604 to call external data, and can also be connected to a network or another computer system (not shown) via a network interface 606. No further detailed description will be given here.

[0079] In this embodiment, by storing data instructions in a memory and processing the instructions through a processor, the applicability and credibility of the image style transfer effect can be improved.

[0080] In another embodiment, a computer-readable storage medium stores thereon computer program instructions, which, when executed by a processor, implement the steps of the method in the corresponding embodiment of the image processing method. Those skilled in the art should understand that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0085] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present disclosure rather than to limit it. Although the present disclosure has been described in detail with reference to the preferred embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present disclosure, which should be included in the scope of the technical solution for protection of the present disclosure.

Claims

1. A method for processing an image, comprising: Inputting the image features extracted by the feature extractor of the first domain into a generator of a first generative adversarial network to obtain the completed features, wherein the generator of the first generative adversarial network is trained to complete the missing features of the images taken in the real environment; Inputting the padded features into a generator of a second generative adversarial network to obtain a style-transferred image, wherein the generator of the second generative adversarial network is trained to restore the input features to an image having a style consistent with that of an image in the second domain; Among them, the pictures taken in the real environment for training the first generative adversarial network and the pictures generated after the training belong to the first domain, the pictures used for training the second generative adversarial network and the pictures generated after the training belong to the second domain, and the latent spaces of the first domain and the second domain are aligned.

2. The method according to claim 1, further comprising: Train the first generative adversarial network based on images with known features; Obtaining data completed by the generator of the first generative adversarial network and a first domain to which the picture of the known feature belongs; training a second generative adversarial network based on the images belonging to the second domain; The latent spaces of the first domain and the second domain are aligned.

3. The method according to claim 2, wherein: The generating a first generative adversarial network based on the picture training of known features comprises: Obtaining a character feature vector through a feature extractor in the first domain according to a picture taken in a real environment; Randomly omitting one or more features from the character feature vector, and completing the features through the generator of the first generative adversarial network; The discriminator of the first generative adversarial network judges the authenticity of the features complemented by the generator based on the known character feature vector; The operation of completing features through the generator of the first generative adversarial network is repeatedly performed until the output of the discriminator reaches a predetermined first target.

4. The method according to claim 2, wherein: The generating a second generative adversarial network based on training of pictures belonging to different styles includes: Obtaining style picture features from the picture belonging to the second domain through a feature extractor of the second domain; Inputting the style picture features into the generator of the second generative adversarial network to change the style of the picture; The discriminator of the second generative adversarial network determines whether the picture output by the generator of the second generative adversarial network belongs to the second domain; The operation of obtaining the style picture features from the picture belonging to the second domain through the feature extractor of the second domain is repeatedly performed until the output of the discriminator reaches a predetermined second target.

5. The method according to claim 2, wherein: The aligning the latent space of the first domain with the latent space of the second domain comprises: Introducing hidden layers after the encoder layer of the network structure of the first domain and the network structure of the second domain, respectively, and binding data of the hidden layers introduced in the first generative adversarial network and the second generative adversarial network; Inputting a first picture belonging to a first domain into the network structure of the first domain, and obtaining a first vector after passing through a hidden layer; A decoding Decoder layer of the network structure of the second domain obtains a second picture according to the first vector; Input the second picture into the Encoder layer of the network structure of the second domain, and obtain a second vector after passing through the hidden layer; Obtaining the distance between the first vector and the second vector; The operation of inputting the same or different first images belonging to the first domain into the first generative adversarial network is repeatedly performed until the distance between the first vector and the second vector is less than a predetermined loss value.

6. The method according to claim 3, wherein: The algorithms based on the generation layer of the first generative adversarial network include recurrent neural network RNN, gated recurrent unit GRU or long short-term memory network LSTM; the algorithms based on the discriminator of the first generative adversarial network include MLP, SVM and naive Bayes.

7. The method according to claim 4, wherein: The algorithm based on which the generator of the second generative adversarial network is based includes a neural network; the algorithm based on which the discriminator of the second generative adversarial network is based includes a multi-layer perceptron MLP, a support vector machine SVM, and a naive Bayes.

8. A picture processing device, comprising: A completion unit is configured to input the image features extracted by the feature extractor of the first domain into a generator of a first generative adversarial network to obtain the completed features, wherein the generator of the first generative adversarial network is trained to complete the missing features of the image taken in the real environment; A style transfer unit is configured to input the padded features into a generator of a second generative adversarial network to obtain a style-transferred image, wherein the generator of the second generative adversarial network is trained to restore the input features to an image having a style consistent with that of the image in the second domain; Among them, the pictures taken in the real environment for training the first generative adversarial network and the pictures generated after the training belong to the first domain, the pictures used for training the second generative adversarial network and the pictures generated after the training belong to the second domain, and the latent spaces of the first domain and the second domain are aligned.

9. The apparatus according to claim 8, further comprising: A first training unit is configured to train a first generative adversarial network based on a picture with known features; A first domain acquisition unit is configured to acquire data completed by the generator of the first generative adversarial network and a first domain to which the picture of the known feature belongs; A second training unit is configured to train a second generative adversarial network based on images belonging to a second domain; The latent space alignment unit is configured to align the latent spaces of the first domain and the second domain.

10. A picture processing device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method according to any one of claims 1 to 7 based on instructions stored in the memory.

11. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images