Image generation method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image generation method, device, electronic device and storage medium, the method comprising obtaining a preset object code and a target style code of a target style; performing style fusion processing on the target style code and the preset object code based on a network fusion parameter corresponding to a preset number of network layers in a style fusion network to obtain a target style fusion code; the network fusion parameter is determined based on fusion data corresponding to a preset number of network layers and a target fusion weight, and the target fusion weight is obtained by learning the fusion weight based on the target style code and the preset object code; the target style fusion code is input into a target image generation network for image generation processing to obtain a preset object style image corresponding to the target style. The disclosed embodiment can quickly generate high-quality object style images and improve the efficiency of adaptive generation of multi-style object style images.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image generating method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of image processing technology, image style transfer has become a new and interesting way to play in the field of image applications. Image style transfer technology can generate an image with a target style based on a given object image such as a face, so that it has an artistic effect similar to the target style, thereby realizing the conversion of the object image into an object style image of different styles such as animation, oil painting, pencil drawing, etc.

[0003] In the related art, in the process of generating an image of a certain object, it is necessary to pre-train a style transfer network with a style image generation function. However, training a style transfer network often requires a large number of object style images as training images, and in order to ensure that the object style images generated by the style transfer network are consistent with the features of the original object and do not produce image distortion such as distortion, the quality of the training images needs to be guaranteed. Therefore, how to quickly generate high-quality object style images that can be used to train the style transfer network has become an urgent problem to be solved. Summary of the invention

[0004] The present disclosure provides an image generation method, device, electronic device and storage medium, which can quickly generate high-quality object style images and improve the efficiency of adaptive generation of multi-style object style images. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image generating method, comprising:

[0006] Get the preset object encoding and the target style encoding of the target style;

[0007] Based on the network fusion parameters corresponding to a preset number of network layers in the style fusion network, the target style code and the preset object code are subjected to style fusion processing to obtain a target style fusion code; the network fusion parameters are determined based on the fusion data corresponding to the preset number of network layers and the target fusion weight, and the target fusion weight is obtained by fusion weight learning based on the target style code and the preset object code;

[0008] The target style fusion code is input into the target image generation network for image generation processing to obtain a preset object style image corresponding to the target style.

[0009] Optionally, performing style fusion processing on the target style code and the preset object code based on network fusion parameters corresponding to a preset number of network layers in the style fusion network to obtain the target style fusion code includes:

[0010] Acquire target fusion control data, where the target fusion control data is used to control a fusion position of the target style code and the preset object code in the style fusion network;

[0011] Determining fusion data corresponding to the preset number of network layers according to the target fusion control data;

[0012] Performing splicing processing on the target style code and the preset object code to obtain a target splicing code;

[0013] Perform fusion weight learning based on the target concatenated code to obtain the target fusion weight;

[0014] Performing weighted processing on the fusion data and the target fusion weight to obtain the network fusion parameter;

[0015] Based on the network fusion parameter, style fusion processing is performed on the target style code and the preset object code in the preset number of network layers to obtain the target style fusion code.

[0016] Optionally, determining the fusion data corresponding to the preset number of network layers according to the target fusion control data includes:

[0017] Comparing the number of layers corresponding to the preset number of network layers with the target fusion control data to obtain a comparison result;

[0018] According to the comparison result, the fusion data corresponding to the preset number of network layers is determined.

[0019] Optionally, the step of acquiring the target style encoding of the target style includes:

[0020] Acquire a reference style image of the target style;

[0021] The reference style image is input into a style encoding network for style encoding processing to obtain the target style encoding.

[0022] Optionally, the method further includes:

[0023] Obtaining a positive sample style image pair and a negative sample style image pair of the target style;

[0024] Inputting the positive sample style image pair and the negative sample style image pair into the style encoding network to be trained for style encoding processing, to obtain sample style encodings corresponding to the positive sample style image pair and the negative sample style image pair respectively;

[0025] Inputting the sample style code into the perception network to be trained for perception processing, and obtaining the sample perception feature information corresponding to the positive sample style image pair and the negative sample style image pair;

[0026] Determining contrast loss information according to the sample perception feature information;

[0027] Based on the contrast loss information, training the style encoding network to be trained and the perception network to be trained;

[0028] The trained style encoding network to be trained is used as the style encoding network.

[0029] Optionally, the step of acquiring the target style encoding of the target style includes:

[0030] Based on a first preset distribution, randomly generate an initial style code;

[0031] The initial style code is input into a first multi-layer perception network for perception processing to obtain the target style code.

[0032] Optionally, the preset object code includes obtaining by the following steps:

[0033] Based on a second preset distribution, randomly generate an initial object code;

[0034] The initial object code is input into a second multi-layer perception network for perception processing to obtain the preset object code.

[0035] Optionally, the method further includes:

[0036] Acquire a first sample style code of the target style, a sample object code, a second sample style code of a non-target style, a preset style object image, and a preset object image;

[0037] Based on sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, performing style fusion processing on the first sample style code and the sample object code to obtain a sample style fusion code;

[0038] Inputting the sample style fusion code into the image generation network to be trained for image generation processing to obtain a sample object style image corresponding to the target style;

[0039] Inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information;

[0040] Determining target loss information according to the target discrimination information;

[0041] Based on the target loss information, training the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained;

[0042] The trained style fusion network to be trained is used as the style fusion network, and the trained image generation network to be trained is used as the target image generation network.

[0043] Optionally, the sample style fusion code includes a first style fusion code and a second style fusion code; the performing style fusion processing on the first sample style code and the sample object code based on the sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained to obtain the sample style fusion code includes:

[0044] Acquire first fusion control data and second fusion control data, wherein the first fusion control data is used to control the first sample style code and the sample object code, and the fusion starts from the first network layer in the style fusion network to be trained; the second fusion control data is used to control the first sample style code not to participate in the fusion in the style fusion network to be trained;

[0045] According to the first fused control data and the second fused control data, respectively determine the first sample fused data and the second sample fused data corresponding to the preset number of network layers to be trained;

[0046] Performing splicing processing on the first sample style code and the sample object code to obtain a sample splicing code;

[0047] Perform fusion weight learning based on the sample concatenation coding to obtain the sample fusion weight;

[0048] The first sample fusion data and the second sample fusion data are weighted with the sample fusion weights to obtain first sample network fusion parameters and second sample network fusion parameters;

[0049] Based on the first sample network fusion parameter and the second sample network fusion parameter, style fusion processing is performed on the first sample style code and the sample object code in the preset number of network layers to be trained, so as to obtain the first style fusion code and the second style fusion code.

[0050] Optionally, the sample object style image includes a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; the discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; the target discriminant information includes object discriminant information, style object discriminant information and style coding discriminant information;

[0051] The step of inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information comprises:

[0052] Inputting the second sample object style image and the preset object image into the object discrimination network for object discrimination processing to obtain object discrimination information;

[0053] Inputting the first sample object style image and the preset style object image into the style object discrimination network to perform style object discrimination processing to obtain style object discrimination information;

[0054] The second sample object style image, the first sample style code and the second sample style code are input into the style code discrimination network for style code discrimination processing to obtain style code discrimination information.

[0055] According to a second aspect of an embodiment of the present disclosure, there is provided an image generating method, comprising:

[0056] Acquire a first original object image of a first target object;

[0057] Inputting the first original object image into a first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object;

[0058] The first style conversion network is obtained by adversarially training a first preset image generation network based on a first sample object image and a preset object style image of a target style generated by any image generation method provided in the first aspect.

[0059] According to a third aspect of an embodiment of the present disclosure, there is provided an image generating method, including:

[0060] Acquire a second original object image and a target style label of a second target object;

[0061] Inputting the second original object image and the target style label into a second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object;

[0062] The second style transfer network is obtained by adversarially training a second preset target image generation network based on a second sample object image, multiple target style labels, and preset object style images of multiple target styles generated by any image generation method provided in the first aspect.

[0063] According to a fourth aspect of an embodiment of the present disclosure, there is provided an image generating device, including:

[0064] A coding acquisition module, configured to execute acquisition of a preset object coding and a target style coding of a target style;

[0065] A first style fusion processing module is configured to execute a style fusion process on the target style code and the preset object code based on network fusion parameters corresponding to a preset number of network layers in the style fusion network to obtain a target style fusion code; the network fusion parameters are determined based on fusion data corresponding to the preset number of network layers and a target fusion weight, and the target fusion weight is obtained by fusion weight learning based on the target style code and the preset object code;

[0066] The first image generation processing module is configured to execute the target style fusion encoding and input it into the target image generation network to perform image generation processing to obtain a preset object style image corresponding to the target style.

[0067] Optionally, the first style fusion processing module includes:

[0068] A target fusion control data acquisition unit is configured to acquire target fusion control data, where the target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network;

[0069] A fusion data determination unit is configured to determine the fusion data corresponding to the preset number of network layers according to the target fusion control data;

[0070] A first splicing processing unit is configured to perform splicing processing on the target style code and the preset object code to obtain a target splicing code;

[0071] A first fusion weight learning unit is configured to perform fusion weight learning based on the target concatenated code to obtain the target fusion weight;

[0072] A first weighted processing unit is configured to perform weighted processing on the fusion data and the target fusion weight to obtain the network fusion parameter;

[0073] The first style fusion processing unit is configured to perform style fusion processing on the target style code and the preset object code in the preset number of network layers based on the network fusion parameters to obtain the target style fusion code.

[0074] Optionally, the fusion data determining unit includes:

[0075] A comparison unit is configured to compare the number of layers corresponding to the preset number of network layers with the target fusion control data to obtain a comparison result;

[0076] The fusion data determination subunit is configured to determine the fusion data corresponding to the preset number of network layers according to the comparison result.

[0077] Optionally, the code acquisition module includes:

[0078] A reference style image acquisition unit, configured to acquire a reference style image of the target style;

[0079] The style encoding processing unit is configured to input the reference style image into a style encoding network for style encoding processing to obtain the target style encoding.

[0080] Optionally, the device further comprises:

[0081] A sample image acquisition module, configured to acquire a positive sample style image pair and a negative sample style image pair of the target style;

[0082] A style coding processing module is configured to perform style coding processing by inputting the positive sample style image pair and the negative sample style image pair into a style coding network to be trained, so as to obtain sample style codes corresponding to the positive sample style image pair and the negative sample style image pair respectively;

[0083] A perception processing module is configured to execute the sample style encoding and input it into the perception network to be trained for perception processing, so as to obtain the sample perception feature information corresponding to each of the positive sample style image pair and the negative sample style image pair;

[0084] A contrast loss information determination module, configured to determine contrast loss information according to the sample perception feature information;

[0085] A first network training module is configured to train the style encoding network to be trained and the perception network to be trained based on the contrast loss information;

[0086] The style encoding network determination module is configured to execute the trained style encoding network to be trained as the style encoding network.

[0087] Optionally, the code acquisition module includes:

[0088] An initial style code generating unit, configured to randomly generate an initial style code based on a first preset distribution;

[0089] The first perception processing unit is configured to input the initial style code into a first multi-layer perception network for perception processing to obtain the target style code.

[0090] Optionally, the code acquisition module includes:

[0091] An initial object code generating unit, configured to randomly generate an initial object code based on a second preset distribution;

[0092] The second perception processing unit is configured to execute perception processing by inputting the initial object code into a second multi-layer perception network to obtain the preset object code.

[0093] Optionally, the device further comprises:

[0094] A sample data acquisition module is configured to acquire a first sample style code of the target style, a sample object code, a second sample style code of a non-target style, a preset style object image, and a preset object image;

[0095] A second style fusion processing module is configured to execute style fusion processing on the first sample style code and the sample object code based on sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, so as to obtain a sample style fusion code;

[0096] A second image generation processing module is configured to execute image generation processing by inputting the sample style fusion encoding into the image generation network to be trained to obtain a sample object style image corresponding to the target style;

[0097] A style discrimination processing module is configured to perform style discrimination processing by inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into a discriminant network to be trained to obtain target discrimination information;

[0098] a target loss information determination module, configured to determine target loss information according to the target discrimination information;

[0099] A second network training module is configured to train the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained based on the target loss information;

[0100] The network determination module is configured to execute the trained style fusion network to be trained as the style fusion network, and to execute the trained image generation network to be trained as the target image generation network.

[0101] Optionally, the sample style fusion coding includes a first style fusion coding and a second style fusion coding; the second style fusion processing module includes:

[0102] The sample fusion control data acquisition unit is configured to execute acquisition of first fusion control data and second fusion control data, wherein the first fusion control data is used to control the first sample style code and the sample object code, and the fusion starts from the first network layer in the style fusion network to be trained; the second fusion control data is used to control the first sample style code not to participate in the fusion in the style fusion network to be trained;

[0103] A sample fusion data determination unit is configured to determine first sample fusion data and second sample fusion data corresponding to the preset number of network layers to be trained, respectively, according to the first fusion control data and the second fusion control data;

[0104] A second splicing processing unit is configured to perform splicing processing on the first sample style code and the sample object code to obtain a sample splicing code;

[0105] A second fusion weight learning unit is configured to perform fusion weight learning based on the sample concatenation coding to obtain the sample fusion weight;

[0106] A second weighted processing unit is configured to perform weighted processing on the first sample fusion data and the second sample fusion data and the sample fusion weight respectively to obtain a first sample network fusion parameter and a second sample network fusion parameter;

[0107] The second style fusion processing unit is configured to perform style fusion processing on the first sample style code and the sample object code in the preset number of network layers to be trained based on the first sample network fusion parameters and the second sample network fusion parameters, respectively, to obtain the first style fusion code and the second style fusion code.

[0108] Optionally, the sample object style image includes a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; the discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; the target discriminant information includes object discriminant information, style object discriminant information and style coding discriminant information;

[0109] The style discrimination processing module includes:

[0110] an object discrimination processing unit, configured to input the second sample object style image and the preset object image into the object discrimination network to perform object discrimination processing to obtain object discrimination information;

[0111] a style object discrimination processing unit, configured to input the first sample object style image and the preset style object image into the style object discrimination network to perform style object discrimination processing to obtain style object discrimination information;

[0112] The style coding discrimination processing unit is configured to input the second sample object style image, the first sample style code and the second sample style code into the style coding discrimination network to perform style coding discrimination processing to obtain style coding discrimination information.

[0113] According to a fifth aspect of an embodiment of the present disclosure, there is provided an image generating device, including:

[0114] An original object image acquisition module is configured to acquire a first original object image of a first target object;

[0115] A first style conversion processing module is configured to input the first original object image into a first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object;

[0116] The first style conversion network is obtained by adversarially training a first preset image generation network based on a first sample object image and a preset object style image of a target style generated by any image generation method provided in the first aspect.

[0117] According to a sixth aspect of an embodiment of the present disclosure, there is provided an image generating device, including:

[0118] A data acquisition module is configured to acquire a second original object image and a target style label of a second target object;

[0119] A second style conversion processing module is configured to input the second original object image and the target style label into a second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object;

[0120] The second style transfer network is obtained by adversarially training a second preset target image generation network based on a second sample object image, multiple target style labels, and preset object style images of multiple target styles generated by any image generation method provided in the first aspect.

[0121] According to the seventh aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method as described in any one of the first, second and third aspects above.

[0122] According to the eighth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any one of the first aspect, the second aspect and the third aspect of the embodiment of the present disclosure.

[0123] According to a ninth aspect of an embodiment of the present disclosure, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in any one of the first aspect, the second aspect and the third aspect of the embodiment of the present disclosure.

[0124] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0125] In the process of generating a stylized image of a certain type of object (preset object style image), the stylized image is decoupled into two parts: object code and style code. In the style fusion network, the target fusion weight is combined with the fusion data corresponding to the preset number of network layers to determine the network fusion parameters, and the target style code and the preset object code are style fused. By fusing the two, a target style fusion code that can characterize the object characteristics of a certain type of object and effectively integrate the target style can be obtained. The target fusion weight is based on the target style code and the preset object code for fusion weight learning, which can realize adaptive adjustment of the fusion weight of the object under different target styles, and thus better fuse the object style code under the target style. On the basis of greatly improving the stylization effect and the quality of the stylized image, the efficiency of adaptive generation of multi-style object style images can be greatly improved.

[0126] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0127] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0128] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0129] Figure 2is a flow chart of an image generating method according to an exemplary embodiment;

[0130] Figure 3 is a schematic diagram of a network structure of a style coding network provided according to an exemplary embodiment;

[0131] Figure 4 is a flowchart of pre-training a style encoding network according to an exemplary embodiment;

[0132] Figure 5 A flowchart of performing style fusion processing on a target style code and a preset object code based on network fusion parameters corresponding to a preset number of network layers in a style fusion network to obtain a target style fusion code is shown according to an exemplary embodiment;

[0133] Figure 6 is a flowchart showing a method of pre-training a target image generation network and a style fusion network according to an exemplary embodiment;

[0134] Figure 7 A flowchart of performing style fusion processing on a first sample style code and a sample object code based on sample network fusion parameters corresponding to a preset number of network layers to be trained in a style fusion network to be trained to obtain a sample style fusion code according to an exemplary embodiment;

[0135] Figure 8 is a schematic diagram of a training style fusion network and a target image generation network provided according to an exemplary embodiment;

[0136] Fig. 9 is a flow chart of an image generating method according to an exemplary embodiment;

[0137] Fig.10 is a flow chart of another image generating method according to an exemplary embodiment;

[0138] Fig.11 is a block diagram of an image generating device according to an exemplary embodiment;

[0139] Fig.12 is a block diagram of another image generating device according to an exemplary embodiment;

[0140] Fig.13 is a block diagram of another image generating device according to an exemplary embodiment;

[0141] Fig.14 is a block diagram of an electronic device for image generation according to an exemplary embodiment;

[0142] Fig.15is a block diagram of another electronic device for image generation according to an exemplary embodiment. DETAILED DESCRIPTION

[0143] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0144] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0146] See also Figure 1 , Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment. Figure 1 As shown, the application environment may include a terminal 100 and a server 200 .

[0147] The terminal 100 can be used to provide a stylized image (object style image) generation service for a target object to any user. Specifically, the terminal 100 may include but is not limited to a smart phone, a desktop computer, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, a smart wearable device and other electronic devices, and may also be software running on the above electronic devices, such as an application program. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0148] In an optional embodiment, the server 200 can provide background services for the terminal 100, pre-generate object style images for training the style conversion network, and train the object image to convert it into a stylized image (object style image of the target style). Specifically, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0149] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included, for example, more terminals may be included.

[0150] In the embodiments of this specification, the terminal 100 and the server 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0151] Figure 2 is a flowchart of an image generating method according to an exemplary embodiment. Figure 2 As shown, the image generation method is used in electronic devices such as terminals and servers, and includes the following steps.

[0152] In step S201, a preset object code and a target style code of a target style are obtained;

[0153] In a specific embodiment, the target style may be any image style. The image style may be divided in a variety of ways in combination with actual application requirements. Optionally, the target style may include but is not limited to image styles such as animation, oil painting, and pencil drawing.

[0154] In an optional embodiment, the target style code may be code information that can characterize the style features of the target style. The preset object code may be code information that can characterize the object features of a certain type of object. Optionally, the object may include but is not limited to objects that need to be style converted, such as human faces, cat faces, and dog faces.

[0155] In an optional embodiment, the target style may be an image style extracted from a certain reference style image. Accordingly, the target style encoding for obtaining the target style may include:

[0156] Get a reference style image of the target style;

[0157] The reference style image is input into the style encoding network for style encoding processing to obtain the target style encoding.

[0158] In a specific embodiment, the style image may be an image with a certain style. For example, if the object is a human face and the target style is an animation style, the style image is an animation face image.

[0159] In a specific embodiment, the style encoding network may be obtained by performing comparative training on a pair of positive sample style images and a pair of negative sample style images based on a target style to form a style encoding network to be trained.

[0160] In a specific embodiment, the network structure of the style encoding network can be preset in combination with actual applications. Figure 3 As shown, Figure 3 1 is a schematic diagram of a network structure of a style coding network provided according to an exemplary embodiment. Specifically, the style coding network may include a convolutional neural network, a style feature extraction network, a feature splicing network and a multi-layer perception network connected in sequence.

[0161] In a specific embodiment, a convolutional neural network can be used to extract image feature information of a reference style image; a style feature extraction network can be used to extract style feature information from image feature information; a feature concatenation network can be used to concatenate style feature information into a long vector of a preset dimension (the preset dimension is consistent with the input dimension of a multi-layer perceptual network); the above-mentioned multi-layer perceptual network can include two parts of multi-layer perceptual networks connected in sequence, the first part of the multi-layer perceptual network can be used to reduce the dimension, and convert the long vector of the preset dimension into a style code with denser information, which can make it easier to train the image generation network. The second part of the multi-layer perceptual network can be used to reduce the dimension of the encoded information output by the first part of the multi-layer perceptual network, and transform the encoded information from (encoding space) to the distribution space where the image generation network is located.

[0162] Specifically, the specific network structures of the convolutional neural network, style feature extraction network, feature splicing network and multi-layer perception network can also be set in combination with actual applications.

[0163] In an optional embodiment, the above method may further include: a step of pre-training a style encoding network, such as Figure 4 As shown, pre-training a style encoding network may include the following steps:

[0164] In step S401, a positive sample style image pair and a negative sample style image pair of a target style are obtained;

[0165] In step S403, the positive sample style image pair and the negative sample style image pair are input into the style encoding network to be trained for style encoding processing, so as to obtain the sample style encoding corresponding to the positive sample style image pair and the negative sample style image pair respectively;

[0166] In step S405, the sample style code is input into the perception network to be trained for perception processing to obtain the sample perception feature information corresponding to the positive sample style image pair and the negative sample style image pair;

[0167] In step S407, contrast loss information is determined according to the sample perception feature information;

[0168] In step S409, based on the contrast loss information, the style encoding network to be trained and the perception network to be trained are trained;

[0169] In step S411, the trained style encoding network to be trained is used as the style encoding network.

[0170] In a specific embodiment, the network structure of the style encoding network to be trained is the same as the network structure of the style encoding network, but the network parameters are different.

[0171] In a specific embodiment, a reference style image of a target style may be obtained, and multiple affine transformations may be performed on the reference style image. Accordingly, the style image after each affine transformation and the reference style image may form a pair of positive sample style images. Optionally, the translation amounts of the multiple affine transformations may be different.

[0172] In a specific embodiment, multiple non-target style reference style images may be obtained, and the multiple non-target style reference images are combined with the target style reference style image to form multiple pairs of target style negative sample style image pairs. Optionally, multiple affine transformations may be performed on a non-target style reference style image, and the style images after the multiple affine transformations are combined with the target style reference style image to form multiple pairs of negative sample style image pairs.

[0173] In a specific embodiment, in order to use a self-supervised learning strategy to train the above-mentioned style encoding network to be trained, the sample style encodings corresponding to the positive sample style image pairs and the negative sample style map pairs output by the style encoding network to be trained can be input into the perception network to be trained for perception processing to obtain the sample perception feature information corresponding to the positive sample style image pairs and the negative sample style map pairs, and the contrast loss information is determined based on the sample perception feature information.

[0174] In a specific embodiment, a preset contrast loss function can be combined in the process of determining contrast loss information based on sample perception feature information. In a specific embodiment, the preset contrast loss function can be any contrast loss function, specifically, for example, the NT-Xent contrast loss function (the normalized temperature-scaled cross entropy loss, normalized temperature-scaled cross entropy loss function).

[0175] In a specific embodiment, the training of the style encoding network to be trained and the perceptual network to be trained based on the contrast loss information may include: updating the network parameters in the style encoding network to be trained and the perceptual network to be trained based on the contrast loss information; then, based on the updated style encoding network to be trained and the perceptual network to be trained, repeatedly inputting the positive sample style image pair and the negative sample style image pair into the style encoding network to be trained for style encoding processing, obtaining the sample style encodings corresponding to the positive sample style image pair and the negative sample style image pair, and performing training iterative operations based on the contrast loss information, updating the network parameters in the style encoding network to be trained and the perceptual network to be trained, until a first preset convergence condition is reached.

[0176] In a specific embodiment, when the first preset convergence condition is met, the current style coding network to be trained (the trained style coding network to be trained) is used as the above style coding network.

[0177] In an optional embodiment, the first preset convergence condition may be achieved when the number of training iterations reaches the first preset number of training times. Optionally, the first preset convergence condition may also be achieved when the contrast loss information is less than a first preset threshold. In the embodiments of this specification, the first preset number of training times and the first preset threshold may be preset in combination with the training speed and accuracy of the network in actual applications.

[0178] In the above embodiment, the positive sample style image pair and the negative sample style image pair based on the target style are compared and trained to obtain the style coding network to be trained, which can realize the training of the self-supervised style coding network and effectively ensure the accuracy of the representation of style features by the trained style coding network. And based on the trained style coding network, the target style coding of the target style is extracted from the style image of the target style, which can effectively improve the accuracy of the target style coding in representing the target style.

[0179] In an optional embodiment, the target style may also be an image style obtained by random sampling. Accordingly, the target style encoding for obtaining the target style includes:

[0180] Based on a first preset distribution, randomly generate an initial style code;

[0181] The initial style code is input into the first multi-layer perception network for perception processing to obtain the target style code.

[0182] In a specific embodiment, the first preset distribution may be a preset coding distribution. Optionally, the first preset distribution may include but is not limited to a Gaussian distribution.

[0183] In a specific embodiment, the first multi-layer perception network can be used to reduce the dimension of the initial style coding and transform the initial style coding from (coding space, such as Gaussian distribution space) to the distribution space where the image generation network is located.

[0184] In the above embodiment, by randomly generating initial style codes, the generation efficiency and diversity of style codes can be improved, and the flexibility of generating object style images can be greatly improved; and after the initial style codes are randomly generated based on the first preset distribution, combined with the first multi-layer perception network for perception processing, it can be easier to train the image generation network.

[0185] In an optional embodiment, the preset object code includes obtaining by the following steps:

[0186] Based on a second preset distribution, randomly generate an initial object code;

[0187] The initial object code is input into the second multi-layer perception network for perception processing to obtain a preset object code.

[0188] In a specific embodiment, the second preset distribution may be a preset coding distribution. Optionally, the second preset distribution may include but is not limited to a Gaussian distribution.

[0189] In a specific embodiment, the second multi-layer perception network can be used to reduce the dimension of the initial object coding and transform the initial object coding from (coding space, such as Gaussian distribution space) to the distribution space where the image generation network is located.

[0190] In the above embodiment, by randomly generating the initial object code, the generation efficiency of the object code can be improved, and then a large number of object style images can be quickly generated for a certain style, which effectively improves the image generation efficiency of a certain style. After the initial object code is randomly generated based on the second preset distribution, it is combined with the second multi-layer perception network for perception processing, which can make it easier to train the image generation network.

[0191] In step S203, based on the network fusion parameters corresponding to a preset number of network layers in the style fusion network, style fusion processing is performed on the target style code and the preset object code to obtain the target style fusion code.

[0192] In an optional embodiment, the style fusion network can be used to perform style fusion processing on the target style code and the preset object code. Specifically, the style fusion network can include a preset number of network layers; specifically, the number of network layers (preset number) can be set in combination with actual applications.

[0193] In an optional embodiment, the style fusion network can be a style fusion network capable of regulating the degree of fusion and adaptively adjusting the fusion weight. Optionally, the degree of fusion can be controlled by regulating the fusion position of the target style code and the preset object code in the style fusion network (i.e., from which network layer the fusion starts). Optionally, the fusion weight of the object under different target styles can be adaptively adjusted by learning the fusion weight of the target style code and the preset object code.

[0194] In a specific embodiment, the above-mentioned network fusion parameters can be determined based on fusion data corresponding to a preset number of network layers and a target fusion weight, and the above-mentioned target fusion weight is obtained by learning a fusion weight based on a target style code and a preset object code.

[0195] In an optional embodiment, if Figure 5 As shown, the above-mentioned network fusion parameters corresponding to the preset number of network layers in the style fusion network are used to perform style fusion processing on the target style code and the preset object code to obtain the target style fusion code, which may include the following steps:

[0196] In step S501, target fusion regulation data is obtained.

[0197] In a specific embodiment, the target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network.

[0198] In step S503, according to the target fusion control data, fusion data corresponding to a preset number of network layers is determined;

[0199] In an optional embodiment, the determining of the fusion data corresponding to the preset number of network layers according to the target fusion control data may include:

[0200] Compare the number of layers corresponding to the preset number of network layers with the target fusion control data to obtain a comparison result;

[0201] According to the comparison result, the fused data corresponding to the preset number of network layers are determined.

[0202] In a specific embodiment, the fusion data corresponding to each network layer can represent whether the target style coding participates in the fusion at the network layer. Specifically, the preset number of network layers included in the style fusion network can be a preset number of network layers arranged in order, such as the 0th to nth layers; specifically, when the comparison result indicates that the number of network layers is less than the target fusion control data, the fusion data corresponding to the network layer is 0, that is, the target style coding does not participate in the fusion at the network layer; conversely, when the comparison result indicates that the number of network layers is greater than or equal to the target fusion control data, the fusion data corresponding to the network layer is 1, that is, the target style coding participates in the fusion at the network layer.

[0203] In the above embodiment, whether the target style encoding participates in the fusion in multiple network layers is controlled based on the fusion data determined by the target fusion control data, so that adjustable style feature fusion can be achieved, thereby ensuring the subsequent generation of object style images with high similarity to natural objects and different degrees of stylization, realizing flexible control of the strength of object stylization, better responding to the needs of different scenes, and greatly improving the image quality of the generated object style images and the generation efficiency of object style images.

[0204] In step S505, the target style code and the preset object code are concatenated to obtain a target concatenated code.

[0205] In step S507, fusion weight learning is performed based on the target concatenated code to obtain the target fusion weight.

[0206] In an optional embodiment, the target concatenated code can be input into the fully connected layer to perform fusion weight learning to obtain the above target fusion weight. Specifically, the target fusion weight can be used to control the fusion ratio of the target style code and the preset object code in the fusion song city.

[0207] In the above embodiment, the target concatenation coding including the target style coding and the preset object coding is combined to perform fusion weight learning, so that the learned target fusion weight is adaptive to different types of objects and styles, thereby better fusing the object image under the target style.

[0208] In step S509, the fusion data and the target fusion weight are weighted to obtain the network fusion parameters.

[0209] In a specific embodiment, the target fusion weight and the fusion data are both data in matrix form. Accordingly, the target fusion weight and the corresponding elements in the fusion data can be multiplied to obtain the above network fusion parameters.

[0210] In step S511, style fusion processing is performed on the target style code and the preset object code in a preset number of network layers based on the network fusion parameters to obtain the target style fusion code.

[0211] In a specific embodiment, the network fusion parameters corresponding to the above-mentioned preset number of network layers can be used as the weight of the target style coding, and the matrix difference after subtracting the network fusion parameters from the matrix whose elements are all ones can be used as the weight of the preset object coding, and based on the weight of the target style coding and the weight of the preset object coding, the target style coding and the preset object coding are weightedly summed to obtain the above-mentioned target fusion coding.

[0212] In a specific embodiment, it is assumed that the network fusion layer includes 18 network layers. Specifically, when the target fusion control data is 0, the target style code and the preset object code are fused from the first network layer, and accordingly, the subsequent image generation network outputs a fully stylized object style image; when the target fusion control data is 18, the target style code does not participate in the fusion; accordingly, the subsequent image generation network outputs an unstylized natural object image; when the target fusion control data is greater than 1 and less than 18, the target style code and the preset object code are fused from the i-th (target fusion control data) layer, and the low-resolution layer (i.e., the 1st to the i-1st) is not affected by the target style code; based on the target fusion control data, it can be ensured that the target style fusion code retains the same feature information as the natural object and has different degrees of style features, and then it can be ensured that the subsequent generation of object style images with high similarity to the natural object and different degrees of stylization can achieve flexible regulation of the strength of object stylization. Accordingly, the above-mentioned target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network, which can be used to control the starting fusion position of the target style code and the preset object code in the style fusion network.

[0213] In the above embodiment, in the style fusion network, the target style coding and the preset object coding are combined with the target fusion control data to control the fusion position of the target style coding and the preset object coding in multiple network layers, so as to achieve the regulation of the fusion degree. And in combination with the target splicing coding including the target style coding and the preset object coding, the fusion weight learning is performed, so that the learned target fusion weight is adaptive to different types of objects and styles, and the fusion weight of the object under different target styles is adaptively adjusted, so as to better fuse the object style coding under the target style, so as to ensure that the obtained target style fusion coding retains the same feature information as the natural object, and has adjustable style features and adaptive fusion weights, so as to ensure the subsequent generation of object style images with high similarity to natural objects and different degrees of stylization, so as to achieve flexible regulation of the strength of object stylization, and improve the adaptive generation efficiency of multi-style object style images, so as to better meet the needs of different scenes, and greatly improve the image quality of the generated object style images and the generation efficiency of multi-style object style images.

[0214] In step S205, the target style fusion code is input into the target image generation network for image generation processing to obtain a preset object style image corresponding to the target style.

[0215] In a specific embodiment, the target image generation network can be used to generate a preset object style image corresponding to the target style. In an optional embodiment, the above method also includes: pre-training the target image generation network and the style fusion network, specifically, Figure 6 As shown, the steps of pre-training the target image generation network and the style fusion network may include:

[0216] In step S601, a first sample style code of a target style, a sample object code, a second sample style code of a non-target style, a preset style object image, and a preset object image are obtained.

[0217] In a specific embodiment, the first sample style code and the second sample style code can be obtained by referring to the method for obtaining the target style code, which will not be described in detail here. The method for obtaining the sample object code can be obtained by referring to the method for obtaining the preset object code, which will not be described in detail here.

[0218] In a specific embodiment, the preset style object image can be obtained from a stylized object image training set corresponding to the target style. For example, if the object is a human face, the preset style face image can be obtained from a collected training set of face style images of the target style. Specifically, the preset object image can be an original object image. For example, if the object is a human face, it can be an image of a real face.

[0219] In step S603, based on sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, style fusion processing is performed on the first sample style code and the sample object code to obtain a sample style fusion code.

[0220] In an optional embodiment, the style fusion network to be trained may include a preset number of network layers to be trained; the above-mentioned sample network fusion parameters may be determined based on sample fusion data and sample fusion weights corresponding to the preset number of network layers to be trained, and the above-mentioned sample fusion weights are obtained by learning the fusion weights based on the first sample style code and the sample object code.

[0221] In an optional embodiment, the sample style fusion coding includes a first style fusion coding and a second style fusion coding; accordingly, Figure 7 As shown, the above-mentioned style fusion processing is performed on the first sample style code and the sample object code based on the sample network fusion parameters corresponding to the preset number of network layers to be trained in the style fusion network to be trained, and obtaining the sample style fusion code may include the following steps:

[0222] In step S701, first fused control data and second fused control data are obtained.

[0223] In a specific embodiment, the first fusion control data is used to control the first sample style encoding and the sample object encoding, and the fusion starts from the first network layer in the style fusion network to be trained. In combination with the above-mentioned target fusion control data embodiment, the first fusion control data is 1; specifically, the second fusion control data can be used to control the first sample style encoding not to participate in the fusion in the style fusion network to be trained. Taking the above-mentioned preset number of 18 as an example, the second fusion control data is 18.

[0224] In step S703, first sample fusion data and second sample fusion data corresponding to a preset number of network layers to be trained are respectively determined according to the first fusion control data and the second fusion control data;

[0225] In a specific embodiment, first sample fusion data corresponding to a preset number of network layers to be trained is determined according to the first fusion control data; and second sample fusion data corresponding to a preset number of network layers to be trained is determined according to the second fusion control data.

[0226] In step S705, the first sample style code and the sample object code are concatenated to obtain a sample concatenated code;

[0227] In step S707, fusion weight learning is performed based on sample concatenation coding to obtain sample fusion weight;

[0228] In step S709, the first sample fusion data and the second sample fusion data are weighted with the sample fusion weights to obtain the first sample network fusion parameters and the second sample network fusion parameters;

[0229] In step S711, based on the first sample network fusion parameter and the second sample network fusion parameter, style fusion processing is performed on the first sample style code and the sample object code in a preset number of network layers to be trained to obtain a first style fusion code and a second style fusion code.

[0230] In a specific embodiment, the specific details of the above steps S705 to 711 can refer to the specific details of the above steps S505 to 511, which will not be repeated here.

[0231] In the above embodiment, during the network training process, the first fusion control data and the second fusion control data are combined to perform style fusion processing on the sample object encoding and the first sample style encoding to varying degrees, and the strength of the object stylization can be flexibly controlled to improve the image quality of the subsequently generated object style image and the generation efficiency of the object style image.

[0232] In step S605, the sample style fusion code is input into the image generation network to be trained for image generation processing to obtain a sample object style image corresponding to the target style;

[0233] In an optional embodiment, the sample object style image may include a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; accordingly, the above-mentioned inputting the sample style fusion code into the image generation network to be trained for image generation processing to obtain the sample object style image corresponding to the target style includes: inputting the first style fusion code and the second style fusion code into the image generation network to be trained for image generation processing to obtain the first sample object style image and the second sample object style image.

[0234] In step S607, the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code are input into the discriminant network to be trained for style discrimination processing to obtain target discrimination information;

[0235] In a specific embodiment, the above-mentioned discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; correspondingly, the above-mentioned target discriminant information may include object discriminant information, style object discriminant information and style coding discriminant information;

[0236] In an optional embodiment, when the style fusion network to be trained is a network with a fixed fusion structure, the above-mentioned inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information includes: inputting the sample object style image and the preset object image into the object discrimination network for object discrimination processing to obtain object discrimination information (the object discrimination information may include feature information corresponding to the sample object style image and the preset object image output by the object discrimination network); inputting the sample object style image and the preset style object image into the style object discrimination network for style object discrimination processing to obtain style object discrimination information (the style object discrimination information may include feature information corresponding to the sample object style image and the preset style object image output by the style object discrimination network); inputting the sample object style image, the first sample style code and the second sample style code into the style coding discrimination network for style coding discrimination processing to obtain style coding discrimination information (the style coding discrimination information may include feature information corresponding to the first sample style code and the second sample style code output by the style coding discrimination network).

[0237] In another optional embodiment, when the style fusion network to be trained is a network capable of adjusting the degree of fusion, the above-mentioned inputting the sample object style image, the preset style object image, the preset object image, the first sample style code, and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information may include: inputting the second sample object style image and the preset object image into the object discrimination network for object discrimination processing to obtain object discrimination information (the object discrimination information may include feature information corresponding to the second sample object style image and the preset object image output by the object discrimination network); inputting the first sample object style image and the preset style object image into the style object discrimination network for style object discrimination processing to obtain style object discrimination information (the style object discrimination information may include feature information corresponding to the first sample object style image and the preset style object image output by the style object discrimination network); inputting the second sample object style image, the first sample style code, and the second sample style code into the style coding discrimination network for style coding discrimination processing to obtain style coding discrimination information (the style coding discrimination information may include feature information corresponding to the first sample style code and the second sample style code output by the style coding discrimination network).

[0238] In the above embodiment, adversarial training is performed on the image generation network to be trained for generating object style images from three dimensions: style discrimination, style object discrimination, and style encoding discrimination. This can greatly improve the representation ability of the trained image generation network for object stylized images and improve the quality of the generated object style images.

[0239] In step S609, target loss information is determined based on the target identification information;

[0240] In a specific embodiment, the target loss information may include generation loss information corresponding to the image generation network to be trained and discrimination loss information corresponding to the discrimination network to be trained.

[0241] In a specific embodiment, in the process of determining the target loss information according to the target discrimination information, an adversarial loss function can be combined. Specifically, the adversarial loss function can be combined to determine the object discrimination loss between the feature information corresponding to the second sample object style image output by the object discrimination network and the preset object image; the adversarial loss function can be combined to determine the style object discrimination loss between the feature information corresponding to the first sample object style image output by the style object discrimination network and the preset style object image; the adversarial loss function can be combined to determine the style coding discrimination loss between the feature information corresponding to the first sample style code output by the style coding discrimination network and the second sample style code.

[0242] Furthermore, the object discrimination loss, the style object discrimination loss and the style encoding discrimination loss may be added together to obtain the above-mentioned generation loss information, and the negative of the generation loss information may be used as the discrimination loss information.

[0243] In step S611, based on the target loss information, the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained are trained.

[0244] In a specific embodiment, the training of the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained based on the target loss information may include: updating the network parameters in the image generation network to be trained and the style fusion network to be trained based on the generation loss information, and updating the network parameters in the discriminant networks to be trained (object discriminant network, style object discriminant network, and style encoding discriminant network) based on the discriminant loss information; then, based on the updated style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained, repeating the style fusion processing of the first sample style code and the sample object code based on the sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained to obtain the sample style fusion code, and performing training iteration operations of updating the network parameters in the image generation network to be trained and the style fusion network to be trained based on the generation loss information, and updating the network parameters in the discriminant networks to be trained (object discriminant network, style object discriminant network, and style encoding discriminant network) based on the discriminant loss information, until a second preset convergence condition is reached.

[0245] In an optional embodiment, the second preset convergence condition may be achieved when the number of training iterations reaches a second preset number of training times. Optionally, the second preset convergence condition may also be achieved when the generated loss information is less than a second preset threshold. In the embodiment of this specification, the second preset number of training times and the second preset threshold may be preset in combination with the training speed and accuracy of the network in actual applications.

[0246] In step S613, the trained style fusion network to be trained is used as the style fusion network, and the trained image generation network to be trained is used as the target image generation network.

[0247] In a specific embodiment, when the second preset convergence condition is reached, the current style fusion network to be trained (the trained style fusion network to be trained) is used as the above-mentioned style fusion network, and the current image generation network to be trained (the trained image generation network to be trained) is used as the above-mentioned target image generation network.

[0248] In a specific embodiment, Figure 8 As shown, Figure 8 It is a schematic diagram of a training style fusion network and a target image generation network provided according to an exemplary embodiment.

[0249] In the above embodiment, the target image generation network and the style fusion network are jointly trained to achieve the fusion of style features and object features, and the target image generation network is trained based on the fused sample object style image, which can greatly improve the trained image generation network's ability to represent object style and object features, thereby effectively improving the quality of subsequently generated object style images.

[0250] In a specific embodiment, based on the above-mentioned image generation method provided in the embodiment of this specification, a large number of preset object style images of the target style can be generated. Accordingly, based on the first sample object image (a large number of object images of real objects) and the above-mentioned image generation method, a large number of preset object style images of the target style can be generated, and the first preset image generation network can be subjected to adversarial training to obtain a first style conversion network. The first style conversion network can be used to generate an object image of the target style of a certain target object. Specifically, in the process of generating a preset object style image of the target style based on the first sample object image and the above-mentioned image generation method, and conducting adversarial training on the first preset image generation network, the first sample object image can be input into the first preset image generation network for style conversion processing to obtain an object style image corresponding to the first sample object image; and the object style image and the corresponding preset object style image are input into the corresponding discriminant network for style discrimination processing to obtain the first style discrimination information; and based on the first style discrimination information, the corresponding discrimination loss information is determined; and then the first preset image generation network and the corresponding discriminant network can be trained based on the discrimination loss information, and the trained first preset image generation network is used as the first style conversion network.

[0251] In a specific embodiment, the above-mentioned image generation method provided in the embodiment of this specification can generate preset object style images of multiple target styles. Accordingly, preset object style images of multiple target styles can be generated based on the second sample object image (a large number of object images of real objects) multiple target style labels and the above-mentioned image generation method, and the second preset image generation network is adversarially trained to obtain a second style conversion network. The second style conversion network can be used to generate object style images of multiple target styles. Specifically, in the process of generating preset object style images of multiple target styles based on a second sample object image (a large number of object images of real objects) with multiple target style labels and the above-mentioned image generation method, and conducting adversarial training on the second preset image generation network, the second sample object image and the corresponding target style label can be input into the second preset image generation network for style conversion processing to obtain an object style image that matches the target style label corresponding to the second sample object image; and the object style image and the corresponding preset object style image are input into the corresponding discriminant network for style discrimination processing to obtain second style discrimination information; in addition, a discriminant network can be added to discriminate whether the object style image output by the second preset image generation network is the style corresponding to the target style label to obtain third style discrimination information; and based on the second style discrimination information and the third style discrimination information, the corresponding loss information is determined; and then the second preset image generation network and the corresponding discriminant network can be trained based on the loss information, and the trained second preset image generation network is used as the second style conversion network.

[0252] It can be seen from the technical solutions provided by the above embodiments of this specification that in the process of generating a stylized image of a certain type of object (preset object style image), this specification decouples the stylized image into two parts: object coding and style coding, and in the style fusion network, combines the target fusion weight with the fusion data corresponding to the preset number of network layers, determines the network fusion parameters, and performs style fusion processing on the target style coding and the preset object coding. By fusing the two, a target style fusion coding that can characterize the object characteristics of a certain type of object and effectively integrate the target style can be obtained, and the target fusion weight is based on the target style coding and the preset object coding for fusion weight learning, which can realize adaptive adjustment of the fusion weight of the object under different target styles, and thus better fuse the object style coding under the target style. On the basis of greatly improving the stylization effect and the quality of the stylized image, the efficiency of adaptive generation of multi-style object style images can be greatly improved.

[0253] Fig. 9 is a flowchart of another image generation method according to an exemplary embodiment. Fig. 9 As shown, the image generation method is used in terminal and server electronic devices and includes the following steps.

[0254] In step S901, a first original object image of a first target object is acquired;

[0255] In step S903, the first original object image is input into a first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object;

[0256] In a specific embodiment, the first original object image may be an object image of the first target object uploaded by the user through a terminal. Taking the first target object as a user's face as an example, the first original object image may be a real face image of the user.

[0257] In a specific embodiment, the first target object style image may be an object image of a target style of the first target object.

[0258] Optionally, the terminal may input the first original object image into the first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object. The terminal may also send the first original object image to the server, which generates the first target object style image based on the first style conversion network and transmits it to the terminal.

[0259] In the above embodiment, a first style conversion network trained by a preset object style image that retains the object features of natural objects and effectively integrates the style features of the target style is combined to perform style conversion on the original object image of the first target object. This can effectively improve the stylization effect while ensuring the consistency between the object features in the first target object style image after style conversion and the object features of the first target object, thereby greatly improving the quality of the stylized image.

[0260] Fig.10 is a flowchart of another image generation method according to an exemplary embodiment. Fig.10 As shown, the image generation method is used in terminal and server electronic devices and includes the following steps.

[0261] In step S1001, a second original object image and a target style label of a second target object are obtained;

[0262] In step S1003, the second original object image and the target style label are input into a second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object;

[0263] In a specific embodiment, the second original object image may be an object image of the second target object uploaded by the user through the terminal. Taking the second target object as a face of a certain user as an example, the second original object image may be a real face image of the user. The target style tag may be identification information of a certain style selected by the user.

[0264] In a specific embodiment, the second target object style image may be an object image of a style corresponding to the target style label of the second target object.

[0265] Optionally, the terminal may input the second original object image and the target style label into the second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object. The terminal may also send the second original object image and the target style label to the server, and the server generates the second target object style image based on the second style conversion network and transmits it to the terminal.

[0266] In the above embodiment, a second style conversion network trained by a preset object style image that retains the object features of natural objects and effectively integrates the style features of multiple target styles is combined to perform style conversion of the original object image of the second target object to a style corresponding to the target style label. This can effectively improve the stylization effect while ensuring the consistency between the object features in the second target object style image after style conversion and the object features of the second target object, thereby greatly improving the quality of the stylized image.

[0267] Fig.11 is a block diagram of an image generating device according to an exemplary embodiment. Fig.11 , the device comprises:

[0268] The encoding acquisition module 1110 is configured to execute the acquisition of the preset object encoding and the target style encoding of the target style;

[0269] The first style fusion processing module 1120 is configured to perform style fusion processing on the target style code and the preset object code based on the network fusion parameters corresponding to the preset number of network layers in the style fusion network to obtain the target style fusion code; the network fusion parameters are determined based on the fusion data corresponding to the preset number of network layers and the target fusion weight, and the target fusion weight is obtained by learning the fusion weight based on the target style code and the preset object code;

[0270] The first image generation processing module 1130 is configured to execute the target style fusion encoding input into the target image generation network for image generation processing to obtain a preset object style image corresponding to the target style.

[0271] Optionally, the first style fusion processing module 1120 includes:

[0272] A target fusion control data acquisition unit is configured to acquire target fusion control data, where the target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network;

[0273] A fusion data determination unit is configured to determine fusion data corresponding to a preset number of network layers according to target fusion control data;

[0274] A first splicing processing unit is configured to perform splicing processing on the target style code and the preset object code to obtain a target splicing code;

[0275] A first fusion weight learning unit is configured to perform fusion weight learning based on target concatenation coding to obtain a target fusion weight;

[0276] A first weighted processing unit is configured to perform weighted processing on the fusion data and the target fusion weight to obtain a network fusion parameter;

[0277] The first style fusion processing unit is configured to perform style fusion processing on the target style code and the preset object code in a preset number of network layers based on the network fusion parameters to obtain the target style fusion code.

[0278] Optionally, the fusion data determination unit includes:

[0279] A comparison unit is configured to compare the number of layers corresponding to a preset number of network layers with the target fusion control data to obtain a comparison result;

[0280] The fusion data determination subunit is configured to determine the fusion data corresponding to a preset number of network layers according to the comparison result.

[0281] Optionally, the code acquisition module 1110 includes:

[0282] A reference style image acquisition unit, configured to acquire a reference style image of a target style;

[0283] The style encoding processing unit is configured to perform style encoding processing by inputting the reference style image into the style encoding network to obtain the target style encoding.

[0284] Optionally, the above device further comprises:

[0285] A sample image acquisition module is configured to acquire a positive sample style image pair and a negative sample style image pair of a target style;

[0286] A style coding processing module is configured to perform style coding processing by inputting the positive sample style image pair and the negative sample style image pair into the style coding network to be trained, and obtain the sample style coding corresponding to the positive sample style image pair and the negative sample style image pair;

[0287] A perception processing module is configured to perform perception processing by inputting the sample style encoding into the perception network to be trained, and obtain the sample perception feature information corresponding to the positive sample style image pair and the negative sample style image pair;

[0288] A contrast loss information determination module is configured to determine contrast loss information based on sample perception feature information;

[0289] A first network training module is configured to train a style encoding network to be trained and a perception network to be trained based on contrast loss information;

[0290] The style encoding network determination module is configured to execute the trained style encoding network to be trained as the style encoding network.

[0291] Optionally, the code acquisition module 1110 includes:

[0292] An initial style code generating unit, configured to randomly generate an initial style code based on a first preset distribution;

[0293] The first perception processing unit is configured to input the initial style code into the first multi-layer perception network for perception processing to obtain the target style code.

[0294] Optionally, the code acquisition module 1110 includes:

[0295] An initial object code generating unit, configured to randomly generate an initial object code based on a second preset distribution;

[0296] The second perception processing unit is configured to execute perception processing by inputting the initial object code into the second multi-layer perception network to obtain a preset object code.

[0297] Optionally, the above device further comprises:

[0298] A sample data acquisition module, configured to execute acquisition of a first sample style encoding of a target style, a sample object encoding, a second sample style encoding of a non-target style, a preset style object image, and a preset object image;

[0299] The second style fusion processing module is configured to perform style fusion processing on the first sample style code and the sample object code based on the sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, so as to obtain a sample style fusion code;

[0300] The second image generation processing module is configured to execute the sample style fusion encoding and input it into the image generation network to be trained to perform image generation processing to obtain a sample object style image corresponding to the target style;

[0301] A style discrimination processing module is configured to perform style discrimination processing by inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained to obtain target discrimination information;

[0302] A target loss information determination module is configured to determine target loss information based on target discrimination information;

[0303] A second network training module is configured to execute training of a style fusion network to be trained, an image generation network to be trained, and a discriminant network to be trained based on target loss information;

[0304] The network determination module is configured to execute the trained style fusion network to be trained as the style fusion network, and to execute the trained image generation network to be trained as the target image generation network.

[0305] Optionally, the sample style fusion coding includes a first style fusion coding and a second style fusion coding; and the second style fusion processing module includes:

[0306] The sample fusion control data acquisition unit is configured to execute acquisition of first fusion control data and second fusion control data, wherein the first fusion control data is used to control the first sample style code and the sample object code, and the fusion starts from the first network layer in the style fusion network to be trained; the second fusion control data is used to control the first sample style code not to participate in the fusion in the style fusion network to be trained;

[0307] A sample fusion data determination unit is configured to determine first sample fusion data and second sample fusion data corresponding to a preset number of network layers to be trained, respectively, according to the first fusion control data and the second fusion control data;

[0308] A second splicing processing unit is configured to perform splicing processing on the first sample style code and the sample object code to obtain a sample splicing code;

[0309] The second fusion weight learning unit is configured to perform fusion weight learning based on sample concatenation coding to obtain sample fusion weight;

[0310] A second weighted processing unit is configured to perform weighted processing on the first sample fusion data and the second sample fusion data and the sample fusion weights respectively to obtain a first sample network fusion parameter and a second sample network fusion parameter;

[0311] The second style fusion processing unit is configured to perform style fusion processing on the first sample style code and the sample object code in a preset number of network layers to be trained based on the first sample network fusion parameters and the second sample network fusion parameters, respectively, to obtain the first style fusion code and the second style fusion code.

[0312] Optionally, the sample object style image includes a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; the discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; the target discriminant information includes object discriminant information, style object discriminant information and style coding discriminant information;

[0313] The style discrimination processing module includes:

[0314] An object discrimination processing unit is configured to input the second sample object style image and the preset object image into the object discrimination network to perform object discrimination processing to obtain object discrimination information;

[0315] A style object discrimination processing unit is configured to input the first sample object style image and the preset style object image into the style object discrimination network to perform style object discrimination processing to obtain style object discrimination information;

[0316] The style coding discrimination processing unit is configured to input the second sample object style image, the first sample style code and the second sample style code into the style coding discrimination network to perform style coding discrimination processing to obtain style coding discrimination information.

[0317] Fig.12 is a block diagram of another image generation device according to an exemplary embodiment. Fig.12 , the device comprises:

[0318] The original object image acquisition module 1210 is configured to acquire a first original object image of a first target object;

[0319] The first style conversion processing module 1220 is configured to input the first original object image into the first style conversion network to perform style conversion processing to obtain a first target object style image corresponding to the first target object;

[0320] The first style conversion network is obtained by adversarially training the first preset image generation network based on the first sample object image and the preset object style image of the target style generated by the above-mentioned image generation method.

[0321] Fig.13 is a block diagram of another image generation device according to an exemplary embodiment. Fig.13 , the device comprises:

[0322] The data acquisition module 1310 is configured to acquire a second original object image and a target style label of a second target object;

[0323] The second style conversion processing module 1320 is configured to input the second original object image and the target style label into the second style conversion network to perform style conversion processing, and obtain a second target object style image corresponding to the second target object;

[0324] The second style transfer network is obtained by adversarially training the second preset target image generation network based on the second sample object image, multiple target style labels and preset object style images of multiple target styles generated by the above-mentioned image generation method.

[0325] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0326] Fig.14 is a block diagram of an electronic device for image generation according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.14As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an image generation method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0327] Fig.15 is a block diagram of another electronic device for image generation according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Fig.15 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image generation method is implemented.

[0328] Those skilled in the art will understand that Fig.14 or Fig.15 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0329] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image generation method in the embodiment of the present disclosure.

[0330] In an exemplary embodiment, a computer-readable storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image generating method in the embodiment of the present disclosure.

[0331] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is caused to execute the image generating method in the embodiment of the present disclosure.

[0332] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0333] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0334] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image generation method, characterized in that: include: Get the preset object encoding and the target style encoding of the target style; Acquire target fusion control data, where the target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network; Determining fusion data corresponding to a preset number of network layers in the style fusion network according to the target fusion control data; Performing splicing processing on the target style code and the preset object code to obtain a target splicing code; Perform fusion weight learning based on the target concatenated code to obtain a target fusion weight; Performing weighted processing on the fusion data and the target fusion weight to obtain a network fusion parameter; Based on the network fusion parameter, style fusion processing is performed on the target style code and the preset object code in the preset number of network layers to obtain a target style fusion code; The target style fusion code is input into the target image generation network for image generation processing to obtain a preset object style image corresponding to the target style.

2. The image generation method according to claim 1, characterized in that: The determining, according to the target fusion control data, the fusion data corresponding to the preset number of network layers comprises: Comparing the number of layers corresponding to the preset number of network layers with the target fusion control data to obtain a comparison result; According to the comparison result, the fusion data corresponding to the preset number of network layers is determined.

3. The image generation method according to any one of claims 1 to 2, characterized in that: The target style encoding of obtaining the target style includes: Acquire a reference style image of the target style; The reference style image is input into a style encoding network for style encoding processing to obtain the target style encoding.

4. The image generation method according to claim 3, characterized in that: The method further comprises: Obtaining a positive sample style image pair and a negative sample style image pair of the target style; Inputting the positive sample style image pair and the negative sample style image pair into the to-be-trained style encoding network for style encoding processing, to obtain sample style encodings corresponding to the positive sample style image pair and the negative sample style image pair respectively; Inputting the sample style code into the perception network to be trained for perception processing, and obtaining the sample perception feature information corresponding to the positive sample style image pair and the negative sample style image pair; Determining contrast loss information according to the sample perception feature information; Based on the contrast loss information, training the style encoding network to be trained and the perception network to be trained; The trained style encoding network to be trained is used as the style encoding network.

5. The image generation method according to any one of claims 1 to 2, characterized in that: The target style encoding of obtaining the target style includes: Based on a first preset distribution, randomly generate an initial style code; The initial style code is input into a first multi-layer perception network for perception processing to obtain the target style code.

6. The image generation method according to any one of claims 1 to 2, characterized in that: The preset object code includes obtaining by the following steps: Based on a second preset distribution, randomly generate an initial object code; The initial object code is input into a second multi-layer perception network for perception processing to obtain the preset object code.

7. The image generation method according to any one of claims 1 to 2, characterized in that: The method further comprises: Acquire a first sample style code of the target style, a sample object code, a second sample style code of a non-target style, a preset style object image, and a preset object image; Based on sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, performing style fusion processing on the first sample style code and the sample object code to obtain a sample style fusion code; Inputting the sample style fusion code into the image generation network to be trained for image generation processing to obtain a sample object style image corresponding to the target style; Inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information; Determining target loss information according to the target discrimination information; Based on the target loss information, training the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained; The trained style fusion network to be trained is used as the style fusion network, and the trained image generation network to be trained is used as the target image generation network.

8. The image generation method according to claim 7, characterized in that: The sample style fusion code includes a first style fusion code and a second style fusion code; the sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, performing style fusion processing on the first sample style code and the sample object code to obtain the sample style fusion code includes: Acquire first fusion control data and second fusion control data, wherein the first fusion control data is used to control the first sample style code and the sample object code, and the fusion starts from the first network layer in the style fusion network to be trained; the second fusion control data is used to control the first sample style code not to participate in the fusion in the style fusion network to be trained; According to the first fused control data and the second fused control data, respectively determine the first sample fused data and the second sample fused data corresponding to the preset number of network layers to be trained; Performing splicing processing on the first sample style code and the sample object code to obtain a sample splicing code; Perform fusion weight learning based on the sample concatenation coding to obtain sample fusion weight; The first sample fusion data and the second sample fusion data are weighted with the sample fusion weights to obtain first sample network fusion parameters and second sample network fusion parameters; Based on the first sample network fusion parameter and the second sample network fusion parameter, style fusion processing is performed on the first sample style code and the sample object code in the preset number of network layers to be trained, so as to obtain the first style fusion code and the second style fusion code.

9. The image generation method according to claim 8, characterized in that: The sample object style image includes a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; the discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; the target discriminant information includes object discriminant information, style object discriminant information and style coding discriminant information; The step of inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into the discriminant network to be trained for style discrimination processing to obtain target discrimination information comprises: Inputting the second sample object style image and the preset object image into the object discrimination network for object discrimination processing to obtain object discrimination information; Inputting the first sample object style image and the preset style object image into the style object discrimination network to perform style object discrimination processing to obtain style object discrimination information; The second sample object style image, the first sample style code and the second sample style code are input into the style code discrimination network for style code discrimination processing to obtain style code discrimination information.

10. An image generation method, characterized in that: include: Acquire a first original object image of a first target object; Inputting the first original object image into a first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object; The first style conversion network is obtained by adversarially training a first preset image generation network based on a first sample object image and a preset object style image of a target style generated by any image generation method according to claims 1 to 9.

11. An image generation method, characterized in that: include: Acquire a second original object image and a target style label of a second target object; Inputting the second original object image and the target style label into a second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object; The second style transfer network is obtained by adversarially training a second preset target image generation network based on a second sample object image, a plurality of target style labels, and preset object style images of a plurality of target styles generated by any image generation method according to claims 1 to 9.

12. An image generating device, characterized in that: include: A coding acquisition module, configured to execute acquisition of a preset object coding and a target style coding of a target style; A first style fusion processing module, wherein the first style fusion processing module comprises: A target fusion control data acquisition unit is configured to acquire target fusion control data, wherein the target fusion control data is used to control the fusion position of the target style code and the preset object code in the style fusion network; a fusion data determination unit is configured to determine the fusion data corresponding to a preset number of network layers in the style fusion network according to the target fusion control data; a first splicing processing unit is configured to perform splicing processing on the target style code and the preset object code to obtain a target splicing code; a first fusion weight learning unit is configured to perform fusion weight learning based on the target splicing code to obtain a target fusion weight; a first weighted processing unit is configured to perform weighted processing on the fusion data and the target fusion weight to obtain a network fusion parameter; a first style fusion processing unit is configured to perform style fusion processing on the target style code and the preset object code in the preset number of network layers based on the network fusion parameter to obtain the target style fusion code; The first image generation processing module is configured to execute the target style fusion encoding and input it into the target image generation network to perform image generation processing to obtain a preset object style image corresponding to the target style.

13. The image generating device according to claim 12, characterized in that: The fusion data determination unit comprises: A comparison unit is configured to compare the number of layers corresponding to the preset number of network layers with the target fusion control data to obtain a comparison result; The fusion data determination subunit is configured to determine the fusion data corresponding to the preset number of network layers according to the comparison result.

14. The image generating device according to any one of claims 12 to 13, characterized in that: The code acquisition module comprises: A reference style image acquisition unit, configured to acquire a reference style image of the target style; The style encoding processing unit is configured to input the reference style image into a style encoding network for style encoding processing to obtain the target style encoding.

15. The image generating device according to claim 14, characterized in that: The device also includes: A sample image acquisition module, configured to acquire a positive sample style image pair and a negative sample style image pair of the target style; A style coding processing module is configured to perform style coding processing by inputting the positive sample style image pair and the negative sample style image pair into a style coding network to be trained, so as to obtain sample style codes corresponding to the positive sample style image pair and the negative sample style image pair respectively; A perception processing module is configured to execute the sample style encoding and input it into the perception network to be trained for perception processing, so as to obtain the sample perception feature information corresponding to each of the positive sample style image pair and the negative sample style image pair; A contrast loss information determination module, configured to determine contrast loss information according to the sample perception feature information; A first network training module is configured to train the style encoding network to be trained and the perception network to be trained based on the contrast loss information; The style encoding network determination module is configured to execute the trained style encoding network to be trained as the style encoding network.

16. The image generating device according to any one of claims 12 to 13, characterized in that: The code acquisition module comprises: An initial style code generating unit, configured to randomly generate an initial style code based on a first preset distribution; The first perception processing unit is configured to input the initial style code into a first multi-layer perception network for perception processing to obtain the target style code.

17. The image generating device according to any one of claims 12 to 13, characterized in that: The code acquisition module comprises: An initial object code generating unit, configured to randomly generate an initial object code based on a second preset distribution; The second perception processing unit is configured to execute perception processing by inputting the initial object code into a second multi-layer perception network to obtain the preset object code.

18. The image generating device according to any one of claims 12 to 13, characterized in that: The device also includes: A sample data acquisition module is configured to acquire a first sample style code of the target style, a sample object code, a second sample style code of a non-target style, a preset style object image, and a preset object image; A second style fusion processing module is configured to execute style fusion processing on the first sample style code and the sample object code based on sample network fusion parameters corresponding to a preset number of network layers to be trained in the style fusion network to be trained, so as to obtain a sample style fusion code; A second image generation processing module is configured to execute image generation processing by inputting the sample style fusion encoding into the image generation network to be trained to obtain a sample object style image corresponding to the target style; A style discrimination processing module is configured to perform style discrimination processing by inputting the sample object style image, the preset style object image, the preset object image, the first sample style code and the second sample style code into a discriminant network to be trained to obtain target discrimination information; a target loss information determination module, configured to determine target loss information according to the target discrimination information; A second network training module is configured to train the style fusion network to be trained, the image generation network to be trained, and the discriminant network to be trained based on the target loss information; The network determination module is configured to execute the trained style fusion network to be trained as the style fusion network, and to execute the trained image generation network to be trained as the target image generation network.

19. The image generating device according to claim 18, characterized in that: The sample style fusion coding includes a first style fusion coding and a second style fusion coding; The second style fusion processing module includes: The sample fusion control data acquisition unit is configured to execute acquisition of first fusion control data and second fusion control data, wherein the first fusion control data is used to control the first sample style code and the sample object code, and the fusion starts from the first network layer in the style fusion network to be trained; the second fusion control data is used to control the first sample style code not to participate in the fusion in the style fusion network to be trained; A sample fusion data determination unit is configured to determine first sample fusion data and second sample fusion data corresponding to the preset number of network layers to be trained, respectively, according to the first fusion control data and the second fusion control data; A second splicing processing unit is configured to perform splicing processing on the first sample style code and the sample object code to obtain a sample splicing code; A second fusion weight learning unit is configured to perform fusion weight learning based on the sample concatenation coding to obtain a sample fusion weight; A second weighted processing unit is configured to perform weighted processing on the first sample fusion data and the second sample fusion data and the sample fusion weight respectively to obtain a first sample network fusion parameter and a second sample network fusion parameter; The second style fusion processing unit is configured to perform style fusion processing on the first sample style code and the sample object code in the preset number of network layers to be trained based on the first sample network fusion parameters and the second sample network fusion parameters, respectively, to obtain the first style fusion code and the second style fusion code.

20. The image generating device according to claim 19, characterized in that: The sample object style image includes a first sample object style image corresponding to the first style fusion code and a second sample object style image corresponding to the second style fusion code; the discriminant network to be trained includes an object discriminant network, a style object discriminant network and a style coding discriminant network; The target discrimination information includes object discrimination information, style object discrimination information and style coding discrimination information; The style discrimination processing module includes: an object discrimination processing unit, configured to input the second sample object style image and the preset object image into the object discrimination network to perform object discrimination processing to obtain object discrimination information; a style object discrimination processing unit, configured to input the first sample object style image and the preset style object image into the style object discrimination network to perform style object discrimination processing to obtain style object discrimination information; The style coding discrimination processing unit is configured to input the second sample object style image, the first sample style code and the second sample style code into the style coding discrimination network to perform style coding discrimination processing to obtain style coding discrimination information.

21. An image generating device, characterized in that: include: An original object image acquisition module is configured to acquire a first original object image of a first target object; A first style conversion processing module is configured to input the first original object image into a first style conversion network for style conversion processing to obtain a first target object style image corresponding to the first target object; The first style conversion network is obtained by adversarially training a first preset image generation network based on a first sample object image and a preset object style image of a target style generated by any image generation method according to claims 1 to 9.

22. An image generating device, characterized in that: include: A data acquisition module is configured to acquire a second original object image and a target style label of a second target object; A second style conversion processing module is configured to input the second original object image and the target style label into a second style conversion network for style conversion processing to obtain a second target object style image corresponding to the second target object; The second style transfer network is obtained by adversarially training a second preset target image generation network based on a second sample object image, a plurality of target style labels, and preset object style images of a plurality of target styles generated by any image generation method according to claims 1 to 9.

23. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image generating method according to any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image generating method according to any one of claims 1 to 11.

25. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the image generating method according to any one of claims 1 to 11 is implemented.

Images