Image conversion method, model training method, device, equipment and storage medium

Abstract

Embodiments of the present application provide an image conversion method, a model training method, an apparatus, a device and a storage medium. The image conversion method comprises: obtaining a first image containing a target object, and inputting the image into an image encoding network. First image features extracted by a feature extraction layer in the image encoding network are input into a generation network as random noise, and second image features extracted by a feature mapping layer in the image encoding network are input into the generation network as random latent codes. The generation network can generate a second image containing the target object according to the two input image features, and the resolutions of the two images are different. Through the scheme, the generated second image can have high definition, and the target object contained in the second image has consistency with the target object in the first image, that is, the image conversion from low resolution to high resolution is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an image conversion method, a model training method, an apparatus, a device and a storage medium. Background Art

[0002] Due to limitations in the camera's capabilities, shooting conditions, data transmission capabilities, network environment, and other factors, users often only get low-quality images or videos. Furthermore, due to the long time since the shooting, images and videos may be incomplete.

[0003] To ensure a good viewing experience for users, there is a need to convert these low-quality images into high-quality images. The quality of an image can be reflected by its resolution. Of course, in some scenarios, there is also a need to convert high-quality images into low-quality images. Therefore, how to ensure the image conversion effect has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an image conversion method, a model training method, an apparatus, a device, and a storage medium to ensure the image conversion effect.

[0005] In a first aspect, an embodiment of the present invention provides an image conversion method, comprising:

[0006] Acquire a first image containing a target object;

[0007] Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0008] The first image features output by the feature extraction layer are input as random noise into a generation network, and the second image features output by the feature mapping layer are input as random latent codes into the generation network, so that the generation network generates a second image containing the target object, where the resolution of the first image is different from that of the second image.

[0009] In a second aspect, an embodiment of the present invention provides an image conversion device, comprising:

[0010] An acquisition module, configured to acquire a first image containing a target object;

[0011] A first input module, configured to input the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0012] The second input module is used to input the first image features output by the feature extraction layer as random noise into the generation network, and input the second image features output by the feature mapping layer as random latent codes into the generation network, so that the generation network generates a second image containing the target object, and the resolution of the first image is different from the resolution of the second image.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image conversion method described in the first aspect. The electronic device may also include a communication interface for communicating with other devices or a communication network.

[0014] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image conversion method described in the first aspect.

[0015] In a fifth aspect, an embodiment of the present invention provides an image conversion method, comprising:

[0016] In response to an input operation triggered by a user on an operation interface, receiving a first image input by the user, where the first image includes a target object;

[0017] Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0018] Inputting the first image feature output by the feature extraction layer as random noise into a generation network, and inputting the second image feature output by the feature mapping layer as a random latent code into the generation network, so that the generation network generates a second image containing the target object, where the resolution of the first image is different from the resolution of the second image;

[0019] The second image is displayed on the operation interface.

[0020] In a sixth aspect, an embodiment of the present invention provides an image conversion device, comprising:

[0021] a receiving module, configured to receive a first image input by the user in response to an input operation triggered by the user on the operation interface, wherein the first image includes a target object;

[0022] A first input module, configured to input the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0023] a second input module, configured to input the first image features output by the feature extraction layer as random noise into a generation network, and input the second image features output by the feature mapping layer as random latent codes into the generation network, so that the generation network generates a second image containing the target object, where the resolution of the first image is different from the resolution of the second image;

[0024] A display module is used to display the second image on the operation interface.

[0025] In a seventh aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image conversion method described in the fifth aspect. The electronic device may also include a communication interface for communicating with other devices or a communication network.

[0026] In an eighth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image conversion method described in the fifth aspect.

[0027] In a ninth aspect, an embodiment of the present invention provides a model training method, comprising:

[0028] Train the first generative network;

[0029] Inputting a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0030] Inputting the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein a resolution of the first image is different from a resolution of the second image;

[0031] Network training is performed according to the second image to obtain a second generation network and a second image encoding network.

[0032] In a tenth aspect, an embodiment of the present invention provides a model training device, comprising:

[0033] A first training module, used for training a first generation network;

[0034] A first input module is configured to input a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0035] a second input module, configured to input the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image;

[0036] The second training module is used to perform network training according to the second image to obtain a second generation network and a second image encoding network.

[0037] In an eleventh aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the model training method described in the ninth aspect. The electronic device may further comprise a communication interface for communicating with other devices or a communication network.

[0038] In the twelfth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the model training method described in the ninth aspect.

[0039] In a thirteenth aspect, an embodiment of the present invention provides a model training method, comprising:

[0040] In response to a user-triggered training start operation, training a first generative network;

[0041] Inputting the first image input by the user and containing the target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0042] Inputting the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein a resolution of the first image is different from a resolution of the second image;

[0043] Performing network training based on the second image to obtain a second generation network and a second image encoding network;

[0044] Output network parameters of the second generation network and the second image encoding network.

[0045] In a fourteenth aspect, an embodiment of the present invention provides a model training device, comprising:

[0046] A first training module, configured to train a first generation network in response to a training start operation triggered by a user;

[0047] a first input module, configured to input the first image input by the user and containing the target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0048] a second input module, configured to input the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image;

[0049] A second training module, configured to perform network training based on the second image to obtain a second generation network and a second image encoding network;

[0050] An output module is used to output network parameters of the second generation network and the second image encoding network.

[0051] In a fifteenth aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the model training method of the thirteenth aspect. The electronic device may further comprise a communication interface for communicating with other devices or a communication network.

[0052] In the sixteenth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the model training method described in the thirteenth aspect.

[0053] An image conversion method provided by an embodiment of the present invention first obtains a first image containing a target object and inputs this image into an image coding network. The first image features extracted by the feature extraction layer of the image coding network are then input into a generative network as random noise. Simultaneously, the second image features extracted by the feature mapping layer of the image coding network are input into the generative network as random latent codes. The generative network can generate a second image containing the same target object based on the input image features. The resolution of the first image and the resolution of the second image are different.

[0054] In the process of generating the second image, image features containing information about the target object are used, rather than random noise unrelated to the target object. Therefore, this approach allows the generated second image to contain both detailed and overall information about the target object. This means the generated second image has high clarity and the target object is highly consistent with the target object in the first image, ensuring a consistent image conversion from low resolution to high resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A flowchart of an image conversion method provided by an embodiment of the present invention;

[0057] Figure 2a for Figure 1 A schematic diagram of a model structure used in the image conversion method provided by the illustrated embodiment;

[0058] Figure 2b for Figure 1 A schematic diagram of a model structure used in the image conversion method provided by the illustrated embodiment;

[0059] Figure 3 for Figure 2b A schematic diagram of the structure of the generation module in the model structure shown;

[0060] Figure 4 A flowchart of another image conversion method provided by an embodiment of the present invention;

[0061] Figure 5 A schematic diagram of an image conversion method provided by an embodiment of the present invention applied in an image restoration scenario;

[0062] Figure 6 A flowchart of a model training method provided by an embodiment of the present invention;

[0063] Figure 7 A flowchart of a network training process is provided in an embodiment of the present invention;

[0064] Figure 8 A flowchart of another model training method provided by an embodiment of the present invention;

[0065] Figure 9 A schematic structural diagram of an image conversion device provided by an embodiment of the present invention;

[0066] Figure 10 For Figure 9 A schematic structural diagram of an electronic device corresponding to the image conversion device provided by the illustrated embodiment;

[0067] Figure 11 A schematic structural diagram of another image conversion device provided by an embodiment of the present invention;

[0068] Figure 12 For Figure 11 A schematic structural diagram of an electronic device corresponding to the image conversion device provided by the illustrated embodiment;

[0069] Figure 13 A schematic structural diagram of a model training device provided by an embodiment of the present invention;

[0070] Figure 14 For Figure 13 A schematic structural diagram of an electronic device corresponding to the model training device provided in the illustrated embodiment;

[0071] Figure 15 A schematic structural diagram of another model training device provided by an embodiment of the present invention;

[0072] Figure 16 For Figure 15 A schematic structural diagram of an electronic device corresponding to the model training device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0074] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. "A plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0075] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0076] As used herein, the words “if” and “if” may be interpreted as “at the time of” or “when” or “in response to determining” or “in response to identifying,” depending on the context. Similarly, the phrases “if it is determined” or “if (stated condition or event) is identified” may be interpreted as “when it is determined” or “in response to determining” or “when identifying (stated condition or event)” or “in response to identifying (stated condition or event),” depending on the context.

[0077] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0078] Before describing the image conversion method provided by the embodiment of the present invention, the practical significance of the image conversion can be described as follows:

[0079] As mentioned in the background, images captured by cameras are often of low quality due to insufficient camera capabilities or the long time since the capture. Furthermore, due to limitations in the network environment or the device's data transmission capacity, images captured by cameras often experience data loss during transmission. Regardless of the aforementioned reasons, users will only receive low-quality images. Image quality can be specifically reflected in factors such as resolution and color.

[0080] For these low-quality images, the method provided by the present invention can be used to convert low-resolution images into high-resolution images, that is, to restore the images. After restoration, a better viewing experience can be provided to users.

[0081] On the contrary, the method provided by the present invention can also be used to convert a high-resolution image into a low-resolution image.

[0082] In addition, for the video captured by the shooting device, each frame of the video can be considered as an image, and the resolution of each frame of the video can be adjusted according to the method provided by the present invention, thereby adjusting the resolution of the entire video.

[0083] Based on the above background, some embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the absence of conflicts between the embodiments, the following embodiments and the features in the embodiments may be combined with each other. In addition, the step timings in the following method embodiments are merely examples and are not strictly limiting.

[0084] Figure 1 This is a flow chart of an image conversion method provided by an embodiment of the present invention. The image conversion method provided by an embodiment of the present invention can be executed by a conversion device. It is understood that the conversion device can be implemented as software, or a combination of software and hardware. In this embodiment, the conversion device can specifically be a server. Figure 1 As shown, the method includes the following steps:

[0085] S101: Acquire a first image containing a target object.

[0086] S102: Input the first image into an image coding network, where the image coding network includes a feature extraction layer and a feature mapping layer.

[0087] S103, inputting the first image feature output by the feature extraction layer as random noise into the generation network, and inputting the second image feature output by the feature mapping layer as random latent code into the generation network, so that the generation network generates a second image containing the target object, and the resolution of the first image is different from the resolution of the second image.

[0088] The first image may be an image obtained by photographing an object; or an image obtained by photographing a physically damaged paper photo, such physical damage may be wrinkles, partial damage, etc.; or an image obtained through an Internet search; or a multi-frame image obtained by framing a video.

[0089] The object contained in the first image is the target object. Optionally, the target object can be a person, more specifically a face, or various types such as an animal or a plant. It should be noted that when the first image contains a single object, the image has a better conversion effect.

[0090] For the sake of simplicity in the following description, the resolution of the first image may be referred to as the first resolution, and the resolution of the second image obtained after the conversion may be referred to as the second resolution. In a common case, the first resolution is lower than the second resolution.

[0091] After obtaining the first image, the image conversion can be started. The image conversion can be realized by means of a conversion model. An optional conversion model can be as follows Figure 2a As shown, the conversion model includes an image coding network and a generative network. The image coding network specifically includes a feature extraction layer and a feature mapping layer. Optionally, the image coding network may further include a downsampling layer disposed after the feature extraction layer, and the number of downsampling layers and feature extraction layers may be the same. In practical applications, the feature extraction layer is a convolutional layer, and the feature mapping layer is a fully connected layer.

[0092] Based on the above model structure, a first image can be input into an image encoding network, where the feature extraction layer extracts first image features. The feature mapping layer then maps the image features extracted from the first image through multiple feature extractions to obtain second image features. Finally, the first and second image features are input into a generative network, which then generates a second image.

[0093] When the first resolution is lower than the second resolution, for example, the scale of the first image can be 1024*1024*3, and the image features input to the feature mapping layer, that is, the scale of the image features obtained after multiple feature extractions of the first image, can be 4*4*1024, and the mapping result output by the feature mapping layer, that is, the scale of the second image features can be 512*1.

[0094] The first and second image features described above can be understood as follows: The traditional approach involves inputting random noise and a random latent code into a generative network to generate an image. In this embodiment, the first image feature can be considered random noise, and the second image feature can be considered a random latent code. However, since the random noise in this embodiment is not actual noise, but rather an image feature extracted from the first image that contains information about the target object, the method provided in this embodiment can achieve better image conversion results.

[0095] Optionally, in Figure 2a Based on the model structure shown in the figure, the generation network can specifically include: mapping network and generation module. In this case, the structure of the conversion model can be as follows: Figure 2b Based on this structure, after the second image feature is input into the generation network, the following processing is usually required:

[0096] The second image features are input into the mapping network, which disentangles the features. A third image feature is then randomly generated based on the processing output of the mapping network. This third image feature, along with the first image feature, is then input into the generation module of the generation network, which then generates the second image. This process yields third image features that meet the input requirements of the generation module, further ensuring the image conversion effect.

[0097] In this embodiment, a first image containing a target object is obtained and input into an image coding network. The first image features extracted by the feature extraction layer of the image coding network are input as random noise into a generative network. Simultaneously, the second image features extracted by the feature mapping layer of the image coding network are input as random latent codes into the generative network. The generative network can generate a second image containing the target object based on the two input image features. The resolution of the first image is different from the resolution of the second image.

[0098] In the process of generating the second image, image features containing information about the target object are used, rather than random noise unrelated to the target object. This solution allows the generated second image to contain both rich details and overall information about the target object. This means the second image has high clarity, and the target object contained therein is highly consistent with the target object in the first image. This effectively achieves image conversion from low resolution to high resolution, ensuring the image conversion effect. Of course, this embodiment can also achieve image conversion from high resolution to low resolution.

[0099] Optionally, the generation network may specifically include multiple generation modules, and each generation module in the generation network may have the following Figure 3 The structure shown. Based on Figure 2b 、 Figure 3 In the structure shown, for any generation module A among the multiple generation modules, the image features output by the convolution layer are spliced ​​with the first image features after scaling transformation, and the splicing result and the third image features are input into the style transfer layer.

[0100] The above concatenation can be thought of as follows: the image feature output by the convolutional layer is [1, 2, 3], and the result of the scaling transformation of the first image feature is [4, 5, 6]. The concatenation of the two can be expressed as: [[1, 2, 3], [4, 5, 6]]. The third image feature is actually the result of the second image feature being mapped by the mapping network.

[0101] Optionally, according to Figure 3 In the illustrated structure, a single generation module can include multiple style transfer layers, and the stitching result input to each style transfer layer includes the scaling transformation results of the first image features. This allows for the reuse of scaling transformation results within the same generation module. This reuse avoids situations where the network's limited number of input channels prevents sufficient image features from being input, leading to poor image conversion results.

[0102] Figures 1 to 3 The embodiment shown provides an image conversion method, which can be deployed on a conversion device. The conversion device can be considered as a service platform for providing online image conversion services to users. Figure 4 This is a flow chart of another image conversion method provided by an embodiment of the present invention. This method is actually a conversion service providing method. Figure 4 As shown, the method may include the following steps:

[0103] S201 : In response to an input operation triggered by a user on an operation interface, receiving a first image input by the user, where the first image includes a target object.

[0104] S202: Input the first image into an image coding network, where the image coding network includes a feature extraction layer and a feature mapping layer.

[0105] S203, inputting the first image feature output by the feature extraction layer as random noise into the generation network, and inputting the second image feature output by the feature mapping layer as random latent code into the generation network, so that the generation network generates a second image containing the target object, and the resolution of the first image is different from the resolution of the second image.

[0106] S204: Display the second image on the operation interface.

[0107] Users can trigger an input operation on the interface provided by the online image conversion service platform. In response to this input operation, the service platform receives a first image at a first resolution input by the user and further converts the first image using a conversion model configured within the service platform to produce a second image at a second resolution, thus achieving image resolution conversion. A common scenario is that the first resolution of the first image is smaller than the second resolution of the second image.

[0108] Finally, the second image can optionally be displayed to the user via the service platform's user interface. Optionally, the service platform's user interface can also simultaneously display the user's input first image and the subsequently generated second image, allowing the user to understand the contrast between the images before and after conversion. The above process provides the user with an online image conversion service.

[0109] In addition, the second image displayed on the service platform can also be downloaded and saved by the user.

[0110] For parts not described in detail in this embodiment, please refer to Figures 1 to 3 The implementation process and technical effects of this technical solution can be found in Figures 1 to 3 The description in the illustrated embodiment will not be repeated here.

[0111] For ease of understanding, the specific implementation process of the image conversion method provided in the above embodiments is exemplified in combination with the image restoration scenario. Figure 5 understand.

[0112] A user may obtain a paper photograph taken long ago and may digitize the photograph using a photographing device. The digitized photograph is a first image having a first resolution. The first image is then input into a conversion device configured with a conversion model.

[0113] Next, the feature extraction layer of the image encoding network in the conversion model extracts the first image features. Simultaneously, the image encoding network outputs the second image features based on the features extracted by the feature extraction layer. The first image features are then used as random noise, and the second image features as random latent codes, which are then simultaneously input into the generator network in the conversion model. The generator network then generates a second image at a second resolution. Since the first resolution is smaller than the second resolution, this effectively restores a low-quality image.

[0114] According to the description of the above embodiments, image conversion can be achieved by means of a conversion model. The following description will take the example of a conversion model that can convert a low-resolution image into a high-resolution image. Figure 6 Flowchart of a model training method provided by an embodiment of the present invention. Figure 6 As shown, the method may include the following steps:

[0115] S301, training a first generative network.

[0116] The first generative network and the first discriminant network can form an anti-generative network, and the two networks in the anti-generative network can be obtained by alternating training. Optionally, the alternating training process can be to first train the discriminant network, then fix the network parameters of the discriminant network, and then train the generative network. After multiple rounds of training, the first generative network and the first discriminant network can be obtained. The specific process of alternating training of the two networks can be referred to as follows. Figure 7 Detailed description in the illustrated embodiment. Optionally, the generative network may be trained first, and then the discriminative network may be trained.

[0117] This step is equivalent to the first step of training the generative network, that is, training an initial generative network to obtain a first generative network. This first step of training enables the first generative network to learn detailed information, and the first generative network can generate high-resolution images that contain rich details of the target object. Optionally, the target object can be a human face, a person, an animal, a plant, etc.

[0118] S302: Input a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image.

[0119] S303: Input the extracted image features into a first generative network, so that the first generative network generates a second image containing the target object, wherein a resolution of the first image is different from a resolution of the second image.

[0120] Next, we collect training sample pairs from the internet or other sources. Each training sample pair consists of a high-resolution image and a low-resolution image. For simplicity, we will refer to the low-resolution image in the training data pair as the first image, and the high-resolution image as the third image. Both images contain the same target object.

[0121] The first image in the training sample pair is input into a first image coding network, so that the first image coding network performs feature extraction on the first image. The first image coding network can be considered as an initial coding network.

[0122] The extracted image features are then input into the first generative network, which generates a second image containing the target object. The second image has a different resolution than the first image, and the difference in resolution between the second and third images is less than a preset threshold. Of course, the preset threshold is typically quite small. A common approach is to set the preset threshold to zero, meaning that the second and third images have the same resolution.

[0123] S304: Perform network training according to the second image to obtain a second generation network and a second image encoding network.

[0124] Finally, the network is trained based on the generated second image, that is, the network parameters of the first generation network and the first image coding network are adjusted to obtain the second image coding network and the second generation network. Figure 1 The generation network and image coding network in the embodiment shown. The specific network structure can also be found in Figure 2a to Figure 3 .

[0125] It's easy to understand that the higher the consistency between the target object contained in the second image and the first image, the better the training effect of the second generation network and the second image encoding network. An optional parameter adjustment method can first calculate the similarity between the second image and the third image, and then adjust the network parameters based on the similarity to obtain the second generation network and the second image encoding network.

[0126] Steps 302-304 above are actually the second training step for the generative network. Based on the first generative network, a second generative network is trained through transfer learning. Furthermore, during this second training step, by utilizing the image features of the first image, the second generative network learns the overall information of the target object, ensuring that the target object in the second image is highly consistent with the target object in the first image.

[0127] In this embodiment, the generative network is actually obtained through a two-step training process. The first step enables the first generative network to learn detailed information about the target object. The second step enables the first generative network to further learn the overall information about the target object, thereby ensuring the training effect of the generative network.

[0128] The second generative network obtained through the above training not only generates a second image with improved resolution, that is, the second image contains rich detailed information of the target object, but also has a high consistency between the target objects contained in the first image and the second image, that is, the second image contains rich overall information of the target object, thereby ensuring the effect of image conversion.

[0129] It should be noted that the above two-step training method can also be used to train a conversion model for converting high-resolution images into low-resolution images, except that the first image used in the training process is correspondingly changed to the third image with high resolution in the training sample pair.

[0130] Furthermore, as described above, the second training step utilizes training data pairs consisting of a high-resolution third image and a low-resolution first image. However, in practice, high-resolution third images are often more readily available than low-resolution first images. Therefore, the third image can be processed in the following manner to obtain the first image.

[0131] Alternatively, the third image may be downsampled to obtain the first image.

[0132] Alternatively, the third image may be downsampled, and then data enhanced may be performed on the image obtained after the downsampling process, and the image obtained after the downsampling process and the image obtained after the data enhancement process may be jointly determined as the first image.

[0133] In practical applications, the third image may also include multiple target objects. In another optional manner, the third image is downsampled, at least one image region is extracted from the downsampled image, and the at least one extracted image region is determined as the first image. Each image region includes a complete target object.

[0134] above Figure 6 The embodiment shown is actually a two-step training method, in which the first step of training can be implemented in the following manner, which is also a specific optional implementation of step 301. Figure 7 As shown, the following steps may be included:

[0135] S401: Perform network training based on images generated by noise data to obtain a first discriminant network.

[0136] S402: Input the image generated by the noise data into the first discriminant network, so that the first discriminant network outputs a discrimination result.

[0137] S403: Perform network training according to the discrimination result output by the first discrimination network to obtain a first generation network.

[0138] Specifically, noise data is first fed into an initial generative network to generate an image. This image, generated from the noise data, is then fed into an initial discriminative network, which is then trained based on the discrimination results output by the initial discriminative network to obtain a first discriminative network. The initial generative network and the initial discriminative network can be considered untrained networks.

[0139] Next, the network parameters of the first discriminant network are fixed, and the image generated by the noise data is input into the first discriminant network again, so that the first discriminant network re-outputs the discrimination result. The initial generative network is trained based on the discrimination result re-output by the first discriminant network to obtain the first generative network.

[0140] In this embodiment, the first discriminant network and the first generative network are obtained by alternating training in the manner of "training the discriminant network first and then training the generative network", which means that the first step of training the generative network is completed. The trained first generative network can learn detailed information of the target object.

[0141] Of course, the method of “training the generative network first and then training the discriminative network” can also be adopted to train the first discriminative network and the first generative network.

[0142] As can be seen from the above description, the model composed of the first generative network, the first image coding network, and the first discriminative network is essentially a generative adversarial network. Therefore, alternatively, an alternating training approach can also be used to adjust the network parameters of the first generative network and the first image coding network to ultimately obtain the second generative network and the second image coding network.

[0143] Optionally, based on Figure 7 The embodiment shown has trained the first discriminant network, which can be further trained to obtain the second discriminant network. Then fix the second discriminant network and further train the first image encoding network and Figure 7 The illustrated embodiment trains the obtained first generation network to obtain a second generation network and a second image encoding network.

[0144] Specifically, the second image generated in step 303 is first input into the first discriminant network, so that the discriminant network outputs a discrimination result. The first discriminant network is then trained based on this discrimination result to obtain a second discriminant network. Then, the network parameters of the second discriminant network are fixed, and the second image generated in step 303 is input into the second discriminant network again to obtain another discrimination result. Based on this further discrimination result, the network parameters of the first generation network and the first image encoding network are simultaneously adjusted to obtain a second generation network and a second image encoding network.

[0145] Of course, in practical applications, the first generation network and the first image coding network can be trained first to obtain the second generation network and the second image coding network, and then the first discriminant network can be trained to obtain the second discriminant network.

[0146] It should be noted that the above-mentioned network parameter adjustment method can be used with Figure 6 Choose one of the methods provided in the article to adjust network parameters based on similarity and execute them all.

[0147] Figures 6 and 7 The embodiment shown provides a model training method that can be deployed on a training device. This training device can be considered as a service platform for providing users with training services for conversion models. Figure 8 This is a flow chart of another model training method provided by an embodiment of the present invention. This method is actually a method for providing model training services.

[0148] like Figure 8 As shown, the method may include the following steps:

[0149] S501 : In response to a training start operation triggered by a user, a first generation network is trained.

[0150] The user can trigger the training start operation on the operation interface provided by the training device, which can be a click operation or a voice command, etc. In response to this training start operation, the training device can train the first generated network, which means that the first step of training is completed. The training process of the first generated network can be found in Figure 7 Related description in the illustrated embodiment.

[0151] S502: Input a first image input by a user and containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image.

[0152] S503: Input the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image.

[0153] Next, the training device receives a first image containing a target object from a user and inputs it into a first image encoding network. The image features extracted by the first image encoding network are then input into a first generative network, which then generates a second image. The first and second images have different resolutions, with the first image typically having a higher resolution than the second.

[0154] S504: Perform network training based on the second image to obtain a second generation network and a second image encoding network.

[0155] S505: Output network parameters of the second generation network and the second image coding network.

[0156] Finally, according to the second image generated by the first generation network, the network parameters of the first generation network and the first image encoding network are adjusted. The specific adjustment process can be found in Figure 7 The following sections describe the embodiment shown. After training, a second generation network and a second image encoding network are obtained. The network parameters of the two networks are output to the user, which is a model training service provided to the user.

[0157] Optionally, according to Figure 6 As can be seen from the description of the illustrated embodiment, a user can obtain a training sample pair consisting of a first image and a third image. When the user inputs the first image, they also input the third image. After obtaining the second image, the network parameters of the first generative network and the first image encoding network can be adjusted based on the similarity of the target objects contained in the first and third images, ultimately resulting in a second generative network and a second image encoding network.

[0158] For parts not described in detail in this embodiment, please refer to Figures 1 to 7 The implementation process and technical effects of this technical solution can be found in Figures 1 to 7 The description in the illustrated embodiment will not be repeated here.

[0159] The image conversion device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will understand that these image conversion devices can be constructed by using commercially available hardware components and configuring them according to the steps taught in this solution.

[0160] Figure 9 A schematic diagram of the structure of an image conversion device provided by an embodiment of the present invention is shown in FIG. Figure 9 As shown, the device includes:

[0161] The acquisition module 11 is configured to acquire a first image containing a target object.

[0162] The first input module 12 is configured to input the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer.

[0163] The second input module 13 is used to input the first image features output by the feature extraction layer as random noise into the generation network, and input the second image features output by the feature mapping layer as random latent codes into the generation network, so that the generation network generates a second image containing the target object, and the resolution of the first image is different from the resolution of the second image.

[0164] Optionally, the generation network includes multiple generation modules, and any generation module among the multiple generation modules uses the processing result of the same first image feature after scaling transformation processing; the generation module includes a convolution layer and a style transfer layer.

[0165] The second input module 13 is specifically configured to stitch the image features output by the convolution layer and the processing results of the first image features after scaling transformation; and input the stitching results and the processing results of the second image features after mapping processing into the style transfer layer.

[0166] The target object includes a human face. The resolution of the first image is lower than that of the second image.

[0167] Figure 9 The device shown can perform Figures 1 to 3 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in this embodiment. Figures 1 to 3 The implementation process and technical effects of this technical solution can be found in Figures 1 to 3 The description in the illustrated embodiment will not be repeated here.

[0168] The above describes the internal functions and structure of the image conversion device. In a possible design, the structure of the image conversion device can be implemented as an electronic device, such as Figure 10 As shown, the electronic device may include: a processor 14 and a memory 15. The memory 15 is used to store the data that supports the electronic device to execute the above Figures 1 to 3 The program of the image conversion method provided in the illustrated embodiment is configured such that the processor 14 is configured to execute the program stored in the memory 15 .

[0169] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 14, can implement the following steps:

[0170] Acquire a first image containing a target object;

[0171] Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0172] The first image features output by the feature extraction layer are input as random noise into a generation network, and the second image features output by the feature mapping layer are input as random latent codes into the generation network, so that the generation network generates a second image containing the target object, where the resolution of the first image is different from that of the second image.

[0173] Optionally, the processor 14 is further configured to execute the aforementioned Figures 1 to 3 All or part of the steps in the illustrated embodiments.

[0174] The structure of the electronic device may further include a communication interface 16 for the electronic device to communicate with other devices or a communication network.

[0175] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figures 1 to 5 The procedures involved in the image conversion method in the method embodiment shown.

[0176] Figure 11 A structural diagram of another image conversion device provided by an embodiment of the present invention is shown in FIG. Figure 11 As shown, the device includes:

[0177] The receiving module 21 is configured to receive a first image input by a user in response to an input operation triggered by the user on the operation interface, where the first image includes a target object.

[0178] The first input module 22 is used to input the first image into an image coding network, where the image coding network includes a feature extraction layer and a feature mapping layer.

[0179] The second input module 23 is used to input the first image features output by the feature extraction layer as random noise into the generation network, and input the second image features output by the feature mapping layer as random latent codes into the generation network, so that the generation network generates a second image containing the target object, and the resolution of the first image is different from the resolution of the second image.

[0180] The display module 24 is configured to display the second image on the operation interface.

[0181] Figure 11 The device shown can perform Figure 4 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in this embodiment. Figure 4The implementation process and technical effects of this technical solution can be found in Figure 4 The description in the illustrated embodiment will not be repeated here.

[0182] The above describes the internal functions and structure of the image conversion device. In a possible design, the structure of the image conversion device can be implemented as an electronic device, such as Figure 12 As shown, the electronic device may include: a processor 25 and a memory 26. The memory 26 is used to store the data that supports the electronic device to execute the above Figure 4 The program of the image conversion method provided in the illustrated embodiment is configured such that the processor 25 is configured to execute the program stored in the memory 26 .

[0183] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 25, can implement the following steps:

[0184] In response to an input operation triggered by a user on an operation interface, receiving a first image input by the user, where the first image includes a target object;

[0185] Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer;

[0186] Inputting the first image feature output by the feature extraction layer as random noise into a generation network, and inputting the second image feature output by the feature mapping layer as a random latent code into the generation network, so that the generation network generates a second image containing the target object, where the resolution of the first image is different from the resolution of the second image;

[0187] The second image is displayed on the operation interface.

[0188] Optionally, the processor 25 is further configured to execute the aforementioned Figure 4 All or part of the steps in the illustrated embodiments.

[0189] The structure of the electronic device may further include a communication interface 27 for the electronic device to communicate with other devices or a communication network.

[0190] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figure 4 The procedures involved in the image conversion method in the method embodiment shown.

[0191] The following describes in detail the model training device of one or more embodiments of the present invention. Those skilled in the art will appreciate that these model training devices can be constructed by configuring commercially available hardware components through the steps taught in this solution.

[0192] Figure 13 A structural diagram of a model training device provided by an embodiment of the present invention is shown in FIG. Figure 13 As shown, the device includes:

[0193] The first training module 31 is used to train the first generation network.

[0194] The first input module 32 is configured to input a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image.

[0195] The second input module 33 is configured to input the extracted image features into the first generation network, so that the first generation network generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image.

[0196] The second training module 34 is configured to perform network training according to the second image to obtain a second generation network and a second image encoding network.

[0197] Optionally, the second training module 34 is specifically used to perform network training based on the similarity between the second image and a third image containing the target object, wherein the resolution difference between the second image and the third image is less than a preset threshold.

[0198] Wherein, the target object includes a human face.

[0199] Optionally, the device also includes: a third training module 35, used to input the second image into the first discriminant network so that the first discriminant network outputs a discrimination result; and train the first discriminant network according to the discrimination result output by the first discriminant network to obtain a second discriminant network.

[0200] The second training module 34 is configured to input the second image into the second discriminant network to obtain a discrimination result; and perform network training according to the discrimination result output by the second discriminant network.

[0201] Optionally, the first training module 31 is specifically used to: perform network training based on the image generated by the noise data to obtain the first discriminant network; input the image generated by the noise data into the first discriminant network so that the first discriminant network outputs a discrimination result; and perform network training based on the discrimination result output by the first discriminant network to obtain the first generation network.

[0202] Optionally, the apparatus further includes: a processing module 36, configured to downsample the third image to obtain the first image.

[0203] Optionally, the processing module 36 is further configured to determine an image obtained after downsampling processing and an image obtained after data enhancement processing on the image as the first image.

[0204] Optionally, the processing module 36 is further configured to: determine at least one image region cut out from the image obtained by downsampling processing as the first image, wherein different image regions contain different target objects.

[0205] Figure 13 The device shown can perform Figures 6 and 7 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in this embodiment. Figures 6 and 7 The implementation process and technical effects of this technical solution can be found in Figures 6 and 7 The description in the illustrated embodiment will not be repeated here.

[0206] The above describes the internal functions and structure of the model training device. In a possible design, the structure of the model training device can be implemented as an electronic device, such as Figure 14 As shown, the electronic device may include: a processor 41 and a memory 42. The memory 41 is used to store the data that supports the electronic device to execute the above Figures 6 and 7 The program of the model training method provided in the illustrated embodiment, the processor 41 is configured to execute the program stored in the memory 42.

[0207] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 41, can implement the following steps:

[0208] Train the first generative network;

[0209] Inputting a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0210] Inputting the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein a resolution of the first image is different from a resolution of the second image;

[0211] Network training is performed according to the second image to obtain a second generation network and a second image encoding network.

[0212] Optionally, the processor 41 is further configured to execute the aforementioned Figures 6 and 7 All or part of the steps in the illustrated embodiments.

[0213] The structure of the electronic device may further include a communication interface 43 for the electronic device to communicate with other devices or a communication network.

[0214] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figures 6 and 7 The procedures involved in the model training method in the method embodiment shown.

[0215] Figure 15 A structural diagram of another model training device provided by an embodiment of the present invention, such as Figure 15 As shown, the device includes:

[0216] The first training module 51 is configured to train the first generation network in response to a training start operation triggered by a user.

[0217] The first input module 52 is configured to input the first image input by the user and containing the target object into a first image coding network, so that the first image coding network performs feature extraction on the first image.

[0218] The second input module 53 is configured to input the extracted image features into the first generation network, so that the first generation network generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image.

[0219] The second training module 54 is configured to perform network training according to the second image to obtain a second generation network and a second image encoding network.

[0220] The output module 55 is configured to output network parameters of the second generation network and the second image encoding network.

[0221] Figure 15 The device shown can perform Figure 8 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in this embodiment. Figure 8 The implementation process and technical effects of this technical solution can be found in Figure 8 The description in the illustrated embodiment will not be repeated here.

[0222] The above describes the internal functions and structure of the model training device. In a possible design, the structure of the model training device can be implemented as an electronic device, such as Figure 16As shown, the electronic device may include: a processor 56 and a memory 57. The memory 57 is used to store the data that supports the electronic device to execute the above Figure 8 The program of the model training method provided in the illustrated embodiment, the processor 56 is configured to execute the program stored in the memory 57.

[0223] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 56, can implement the following steps:

[0224] In response to a user-triggered training start operation, training a first generative network;

[0225] Inputting the first image input by the user and containing the target object into a first image coding network, so that the first image coding network performs feature extraction on the first image;

[0226] Inputting the extracted image features into the first generative network, so that the first generative network generates a second image containing the target object, wherein a resolution of the first image is different from a resolution of the second image;

[0227] Performing network training based on the second image to obtain a second generation network and a second image encoding network;

[0228] Output network parameters of the second generation network and the second image encoding network.

[0229] Optionally, the processor 56 is further configured to execute the aforementioned Figure 8 All or part of the steps in the illustrated embodiments.

[0230] The structure of the electronic device may further include a communication interface 58 for the electronic device to communicate with other devices or a communication network.

[0231] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figure 8 The procedures involved in the model training method in the method embodiment shown.

[0232] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image conversion method, characterized in that: include: Acquire a first image containing a target object; Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer; The second image feature output by the feature mapping layer is input as a random latent code into a mapping network in a generation network, the first image feature output by the feature extraction layer is input as random noise and the third image feature output by the mapping network is input into a generation module in the generation network, so that the generation module generates a second image containing the target object, where the resolution of the first image is different from that of the second image, and the mapping network is used to deentangle the second image feature output by the feature mapping layer.

2. The method according to claim 1, characterized in that The generation network includes multiple generation modules, and any generation module among the multiple generation modules uses the processing result after the same first image feature is subjected to scaling transformation processing.

3. The method according to claim 2, characterized in that The generation module includes a convolution layer and a style transfer layer; The step of inputting the first image feature output by the feature extraction layer as random noise into a generation network and inputting the second image feature output by the feature mapping layer as random latent code into the generation network comprises: Splicing the image features output by the convolutional layer and the processing results of the first image features after scaling transformation; The stitching result and the result of mapping the second image feature are input into the style transfer layer.

4. The method according to any one of claims 1 to 3, characterized in that The target object includes a human face.

5. The method according to any one of claims 1 to 3, characterized in that The resolution of the first image is lower than the resolution of the second image.

6. An image conversion method, characterized in that: include: In response to an input operation triggered by a user on an operation interface, receiving a first image input by the user, where the first image includes a target object; Inputting the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer; inputting the second image feature output by the feature mapping layer as a random latent code into a mapping network in a generating network, inputting the first image feature output by the feature extraction layer as random noise and the third image feature output by the mapping network into a generating module in the generating network, so that the generating module generates a second image containing the target object, where the resolution of the first image is different from the resolution of the second image, and the mapping network is used to deentangle the second image feature output by the feature mapping layer; The second image is displayed on the operation interface.

7. A model training method, characterized in that: include: Train the first generative network; Inputting a first image containing a target object into a first image coding network so that the first image coding network performs feature extraction on the first image, wherein the first image coding network includes a feature extraction layer and a feature mapping layer; Inputting the image features extracted by the feature mapping layer into a mapping network in the first generation network, and inputting the image features extracted by the feature extraction layer and the image features output by the mapping network into a generation module in the first generation network, so that the generation module generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image, and the mapping network is used to deentangle the image features output by the feature mapping layer; Network training is performed according to the second image to obtain a second generation network and a second image encoding network.

8. The method according to claim 7, characterized in that The target object includes a human face.

9. The method according to claim 7, characterized in that The performing network training according to the second image generated by the first generation network includes: Network training is performed based on a similarity between the second image and a third image containing the target object, wherein a resolution difference between the second image and the third image is less than a preset threshold.

10. The method according to claim 7, characterized in that The method further comprises: Inputting the second image into the first discriminant network so that the first discriminant network outputs a discrimination result; Training the first discriminant network according to the discrimination result output by the first discriminant network to obtain a second discriminant network; The performing network training according to the second image includes: Inputting the second image into the second discriminant network to obtain a discrimination result; Network training is performed according to the discrimination result output by the second discriminant network.

11. The method according to claim 10, characterized in that The training of the first generation network includes: Performing network training based on images generated by noise data to obtain the first discriminant network; Inputting the image generated by the noise data into the first discriminant network so that the first discriminant network outputs a discrimination result; Network training is performed according to the discrimination result output by the first discriminant network to obtain the first generative network.

12. The method according to claim 9, characterized in that The method further comprises: Down-sampling the third image to obtain the first image.

13. The method according to claim 12, characterized in that The method further comprises: An image obtained after downsampling processing and an image obtained after data enhancement processing are determined as the first image.

14. The method according to claim 12, characterized in that The method further comprises: At least one image region cut out from the image obtained by the downsampling process is determined as the first image, wherein different image regions contain different target objects.

15. A model training method, characterized in that: include: In response to a user-triggered training start operation, training a first generative network; Inputting the first image input by the user and containing the target object into a first image coding network, so that the first image coding network extracts features from the first image, wherein the first image coding network includes a feature extraction layer and a feature mapping layer; Inputting the image features extracted by the feature mapping layer into a mapping network in the first generation network, and inputting the image features extracted by the feature extraction layer and the image features output by the mapping network into a generation module in the first generation network, so that the generation module generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image, and the mapping network is used to deentangle the image features output by the feature mapping layer; Performing network training based on the second image to obtain a second generation network and a second image encoding network; Output network parameters of the second generation network and the second image encoding network.

16. An image conversion device, characterized in that: include: An acquisition module, configured to acquire a first image containing a target object; A first input module, configured to input the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer; A second input module is configured to input the second image features output by the feature mapping layer as random latent codes into a mapping network in a generation network, and to input the first image features output by the feature extraction layer as random noise and the third image features output by the mapping network into a generation module in the generation network, so that the generation module generates a second image containing the target object, wherein the resolution of the first image is different from that of the second image, and the mapping network is configured to de-entangle the second image features output by the feature mapping layer.

17. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image conversion method according to any one of claims 1 to 5.

18. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the image conversion method according to any one of claims 1 to 5.

19. An image conversion device, characterized in that: include: a receiving module, configured to receive a first image input by the user in response to an input operation triggered by the user on the operation interface, wherein the first image includes a target object; A first input module, configured to input the first image into an image coding network, wherein the image coding network includes a feature extraction layer and a feature mapping layer; a second input module, configured to input the second image feature output by the feature mapping layer as a random latent code into a mapping network in the generation network, and input the first image feature output by the feature extraction layer as random noise and the third image feature output by the mapping network into a generation module in the generation network, so that the generation module generates a second image containing the target object, where the resolution of the first image is different from the resolution of the second image, and the mapping network is configured to deentangle the second image feature output by the feature mapping layer; A display module is used to display the second image on the operation interface.

20. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image conversion method according to claim 6.

21. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the image conversion method according to claim 6.

22. A model training device, characterized in that: include: A first training module, used for training a first generation network; A first input module is configured to input a first image containing a target object into a first image coding network, so that the first image coding network performs feature extraction on the first image, wherein the first image coding network includes a feature extraction layer and a feature mapping layer; a second input module, configured to input the image features extracted by the feature mapping layer into a mapping network in the first generation network, and input the image features extracted by the feature extraction layer and the image features output by the mapping network into a generation module in the first generation network, so that the generation module generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image, and the mapping network is configured to deentangle the image features output by the feature mapping layer; The second training module is used to perform network training according to the second image to obtain a second generation network and a second image encoding network.

23. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the model training method as described in any one of claims 7 to 14.

24. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the model training method as described in any one of claims 7 to 14.

25. A model training device, characterized in that: include: A first training module, configured to train a first generation network in response to a training start operation triggered by a user; a first input module, configured to input the first image input by the user and containing the target object into a first image coding network, so that the first image coding network performs feature extraction on the first image, wherein the first image coding network includes a feature extraction layer and a feature mapping layer; a second input module, configured to input the image features extracted by the feature mapping layer into a mapping network in the first generation network, and input the image features extracted by the feature extraction layer and the image features output by the mapping network into a generation module in the first generation network, so that the generation module generates a second image containing the target object, wherein the resolution of the first image is different from the resolution of the second image, and the mapping network is configured to deentangle the image features output by the feature mapping layer; A second training module, configured to perform network training based on the second image to obtain a second generation network and a second image encoding network; An output module is used to output network parameters of the second generation network and the second image encoding network.

26. An electronic device, characterized in that: include: Memory, processor; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor executes the model training method as described in claim 15.

27. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, which, when executed by a processor of an electronic device, causes the processor to execute the model training method as described in claim 15.

Images