Method and apparatus for generating images and for training image generation models

Abstract

The embodiments of the present disclosure disclose methods and devices for generating images and for training image generation models. A specific implementation of the method includes: inputting the acquired noise map of a first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator includes at least one convolution unit; based on texture migration technology, generating an image with a texture similar to the similar structure map as a quasi-image; based on super-resolution reconstruction technology, generating an image with a higher resolution than the quasi-image. This implementation achieves the effect of generating a large number of high-quality images using a small number of images or even a single image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to methods and devices for generating images and for training image generation models. Background Art

[0002] Image generation refers to the use of computers to generate new images based on existing datasets. With the rapid development of machine learning, artificial intelligence, and deep learning in recent years, image generation technology has become increasingly mature and widely used. For example, image generation is used as a data augmentation method to improve the performance of machine learning models and is used to implement applications such as image rendering, style transfer, and virtual reality.

[0003] Existing deep learning methods usually require a large number of training samples to train image generation models. When the number of samples is small, the quality of images generated by the trained models is usually not high. Summary of the Invention

[0004] Embodiments of the present disclosure provide methods and apparatus for generating images and for training image generation models.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating an image, the method comprising: inputting an acquired noise map of a first target size and a corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator comprises at least one convolution unit; based on texture migration technology, generating an image with a texture similar to that of the similar structure map as a quasi-image; based on super-resolution reconstruction technology, generating an image with a higher resolution than the quasi-image.

[0006] In some embodiments, the above position code is obtained by the following steps: generating an angular velocity variable according to a preset coding dimension; generating a sine and cosine position code according to the angular velocity variable and the width and height of the first target size.

[0007] In some embodiments, the above-mentioned texture migration technology generates an image with a texture similar to that of a similar structure image as a quasi-image, including: inputting the similar structure image into a pre-trained texture migration model, and generating an image with a texture similar to that of the similar structure image as a quasi-image, wherein the texture migration model is trained based on block matching using training images with a number less than a first preset threshold.

[0008] In some embodiments, the above-mentioned super-resolution reconstruction technology generates an image with a higher resolution than the quasi-image, including: inputting the quasi-image into a pre-trained super-resolution reconstruction model to generate an image with a higher resolution than the quasi-image, wherein the super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module includes a convolution layer, an upsampling layer and a downsampling layer.

[0009] In some embodiments, the above-mentioned image generator uses training images with a number less than a second preset threshold to train the target initial model, and the target initial model is obtained based on the knowledge distillation of a pre-trained image generation model obtained by training with a large-scale training set.

[0010] In some embodiments, the first target size is consistent with the size of the training image.

[0011] In a second aspect, an embodiment of the present disclosure provides a method for training an image generation model, the method comprising: downsampling a pre-acquired sample image to generate a low-resolution sample image having a lower resolution than the sample image; inputting the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having the same size as the sample image, wherein the initial texture reconstruction model comprises at least one of the following: an initial texture migration model, an initial super-resolution reconstruction model; generating a reconstruction loss value based on the difference between the sample reconstructed image and the sample image using a preset reconstruction loss function; generating an adversarial loss value based on the sample image and a pre-acquired initial generative adversarial network, wherein the generator in the initial generative adversarial network is used to capture the structural information of the image; adjusting the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value.

[0012] In some embodiments, the above-mentioned initial texture reconstruction model includes an initial super-resolution reconstruction model, the initial super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module includes a convolution layer, an upsampling layer and a downsampling layer.

[0013] In some embodiments, the above-mentioned generation of an adversarial loss value based on a sample image and a pre-acquired initial generative adversarial network includes: inputting the sample image into a pre-trained image generator to generate an image as a real sample, wherein the pre-trained image generator is trained through a large-scale training set; inputting the acquired noise map of the second target size and the corresponding position code into the generator in the initial generative adversarial network to generate an image of the second target size as a fake sample, wherein the second target size is smaller than the size of the sample image, and the generator in the initial generative adversarial network is obtained based on the knowledge distillation of the pre-trained image generator; based on the difference between the image of the second target size generated by downsampling the image of the real sample and the fake sample, the adversarial loss value is generated using a preset adversarial loss function.

[0014] In some embodiments, the above-mentioned adjustment of the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value includes: inputting a fake sample into the initial texture reconstruction model to generate a fake sample reconstructed image that is consistent with the size of the sample image; based on the difference between the fake sample reconstructed image and the sample image, generating a perceptual loss value using a preset perceptual loss function; based on the reconstruction loss value, the adversarial loss value and the perceptual loss value, adjusting the parameters of the initial texture reconstruction model and the initial generative adversarial network.

[0015] In some embodiments, the initial texture reconstruction model includes an initial texture migration model, and the initial texture migration model is obtained by block matching training using training images whose number is less than a first preset threshold.

[0016] In a third aspect, an embodiment of the present disclosure provides a device for generating an image, the device comprising: a first generation unit, configured to input the acquired noise map of a first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator comprises at least one convolution unit; a texture migration unit, configured to generate an image having a texture similar to that of the similar structure map as a quasi-image based on texture migration technology; and a super-resolution reconstruction unit, configured to generate an image with a higher resolution than the quasi-image based on super-resolution reconstruction technology.

[0017] In some embodiments, the above position code is obtained by the following steps: generating an angular velocity variable according to a preset coding dimension; generating a sine and cosine position code according to the angular velocity variable and the width and height of the first target size.

[0018] In some embodiments, the above-mentioned texture migration unit is further configured to: input the similar structure map into a pre-trained texture migration model, and generate an image with a texture similar to the similar structure map as a quasi-image, wherein the texture migration model is trained based on block matching using training images with a number less than a first preset threshold.

[0019] In some embodiments, the super-resolution reconstruction unit is further configured to: input the quasi-image into a pre-trained super-resolution reconstruction model to generate an image with higher resolution than the quasi-image, wherein the super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module includes a convolution layer, an upsampling layer and a downsampling layer.

[0020] In some embodiments, the image generator is trained to obtain a target initial model using training images whose number is less than a second preset threshold, and the target initial model is obtained based on knowledge distillation of a pre-trained image generation model obtained through training with a large-scale training set.

[0021] In some embodiments, the first target size is consistent with the size of the training image.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a device for training an image generation model, the device comprising: a downsampling unit configured to downsample a pre-acquired sample image to generate a low-resolution sample image having a lower resolution than the sample image; a second generation unit configured to input the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having the same size as the sample image, wherein the initial texture reconstruction model includes at least one of the following: an initial texture migration model, an initial super-resolution reconstruction model; a reconstruction loss generation unit configured to generate a reconstruction loss value based on the difference between the sample reconstructed image and the sample image using a preset reconstruction loss function; an adversarial loss generation unit configured to generate an adversarial loss value based on the sample image and a pre-acquired initial generative adversarial network, wherein the generator in the initial generative adversarial network is used to capture the structural information of the image; an adjustment unit configured to adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value.

[0023] In some embodiments, the above-mentioned initial texture reconstruction model includes an initial super-resolution reconstruction model, the initial super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module includes a convolution layer, an upsampling layer and a downsampling layer.

[0024] In some embodiments, the above-mentioned adversarial loss generation unit includes: a true sample generation module, configured to input the sample image into a pre-trained image generator to generate an image as a true sample, wherein the pre-trained image generator is obtained by training with a large-scale training set; a fake sample generation module, configured to input the acquired noise map of the second target size and the corresponding position code into the generator in the initial generative adversarial network to generate an image of the second target size as a fake sample, wherein the second target size is smaller than the size of the sample image, and the generator in the initial generative adversarial network is obtained based on the knowledge distillation of the pre-trained image generator; an adversarial loss generation module, configured to generate an adversarial loss value using a preset adversarial loss function based on the difference between the image of the second target size generated by downsampling the image of the real sample and the fake sample.

[0025] In some embodiments, the above-mentioned adjustment unit is further configured to input the fake sample into the initial texture reconstruction model to generate a fake sample reconstructed image that is consistent with the size of the sample image; based on the difference between the fake sample reconstructed image and the sample image, generate a perceptual loss value using a preset perceptual loss function; based on the reconstruction loss value, the adversarial loss value and the perceptual loss value, adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network.

[0026] In some embodiments, the initial texture reconstruction model includes an initial texture migration model, and the initial texture migration model is obtained by block matching training using training images whose number is less than a first preset threshold.

[0027] In a fifth aspect, an embodiment of the present disclosure provides an electronic device comprising: one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect or the second aspect.

[0028] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the implementation modes of the first aspect or the second aspect.

[0029] The methods and apparatus for generating images and training image generation models provided by the embodiments of the present disclosure capture the structural information of an image through a pre-trained image generator, generate a structurally similar image as an initial estimate of the image, and utilize position encoding to enhance the position perception of the convolutional unit. Texture transfer is then performed on the initial estimate of the image, followed by super-resolution reconstruction, to produce images with similar texture style and content, thereby achieving the effect of generating a large number of high-quality images using a small number of images, or even a single image. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings:

[0031] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0032] Figure 2 is a flow chart of one embodiment of a method for generating an image according to the present disclosure;

[0033] Figure 3 is a schematic diagram of an application scenario of the method for generating an image according to an embodiment of the present disclosure;

[0034] Figure 4 is a flowchart of another embodiment of a method for training an image generation model according to the present disclosure;

[0035] Figure 5 is a structural schematic diagram of an embodiment of a device for generating an image according to the present disclosure;

[0036] Figure 6 is a structural diagram of an embodiment of an apparatus for training an image generation model according to the present disclosure;

[0037] Figure 7 It is a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0039] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] Figure 1 An exemplary architecture 100 is shown to which the method for generating an image and for training an image generation model or the apparatus for generating an image and for training an image generation model of the present disclosure can be applied.

[0041] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0042] The terminal devices 101, 102, and 103 interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, image processing applications, etc.

[0043] The terminal devices 101, 102, and 103 can be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with display screens and supporting human-computer interaction, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When the terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules for providing distributed services), or they can be implemented as a single software or software module. No specific limitation is made here.

[0044] Server 105 can be a server that provides various services, such as a background server that supports image processing applications on terminal devices 101, 102, and 103. The background server can provide a foundation for texture transfer technology and super-resolution reconstruction technology, thereby enabling the terminal device to generate images based on texture transfer technology and super-resolution reconstruction technology.

[0045] It should be noted that the server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (e.g., software or software modules for providing distributed services), or as a single software or software module. No specific limitations are given here.

[0046] It should be noted that the method for generating images and for training image generation models provided in the embodiments of the present disclosure can be executed by the terminal devices 101, 102, 103, or by the server 105. Accordingly, the apparatus for generating images and for training image generation models can be set in the terminal devices 101, 102, 103, or in the server 105.

[0047] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0048] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for generating an image according to the present disclosure. The method for generating an image comprises the following steps:

[0049] Step 201 : Input the acquired noise map of the first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size.

[0050] In this embodiment, the execution subject of the method for generating an image (such as Figure 1 The server 105 shown in the figure can first obtain a noise map of the first target size through a wired connection or a wireless connection. As an example, the above-mentioned noise map can be obtained by random sampling from a Gaussian distribution. The above-mentioned first target size can be any size pre-specified according to actual application requirements. The above-mentioned first target size can also be a size determined according to a rule, such as a size that matches the size of the training sample used to train the above-mentioned image generator. Then, the above-mentioned execution entity can input the obtained noise map of the first target size and the corresponding position code into the pre-trained image generator in various ways to generate a similar structure map of the above-mentioned first target size. The above-mentioned image generator may include at least one convolution unit. The above-mentioned image generator can generate an image with a structure similar to the sample image used for its training. The above-mentioned structural similarity can generally refer to the objects contained in the image and the close relative position relationship between the objects.

[0051] In this embodiment, the dimension of the position code is generally matched with the first target size of the noise map, wherein the position code is used to represent the position of each pixel point on the entire image.

[0052] In some optional implementations of this embodiment, the above position code can be obtained by the following steps:

[0053] The first step is to generate angular velocity variables according to the preset encoding dimension.

[0054] In these implementations, the execution entity may generate the angular velocity variable in various ways based on the preset encoding dimension. For example, the angular velocity variable is generally positively correlated with the encoding dimension. Alternatively, the angular velocity variable is generally negatively correlated with the distance from the upper left corner pixel.

[0055] For example, Among them, the above ω kIt can be used to represent the angular velocity variable in the kth dimension. The value of k can be an integer in [0, h]. The value of h can be used to represent the width or height of the noise image. The value of d can be used to represent half of the encoding dimension, such as 8 or 16.

[0056] In the second step, the sine and cosine position codes are generated according to the angular velocity variable and the width and height of the first target size.

[0057] In these implementations, the execution entity may generate the sine and cosine position codes in various ways according to the angular velocity variable and the width and height of the first target size. The sine and cosine codes may be, for example, [sin(ω0·i), cos(ω0·i), sin(ω1·i), cos(ω1·i), …, sin(ω0·j), cos(ω0·j), …,] where ω0 may be the ω k The value when k=0 in the above-mentioned noise map. The above-mentioned i can be used to represent the height of the above-mentioned noise map. The above-mentioned j can be used to represent the width of the above-mentioned noise map.

[0058] In some optional implementations of this embodiment, the image generator may train the target initial model using training images having a number less than a second preset threshold. The target initial model may be obtained based on knowledge distillation of a pre-trained image generation model obtained through training with a large-scale training set.

[0059] In these implementations, as an example, the execution entity may first obtain a pretrained image generation model trained using a large-scale training set, thereby enabling the pretrained image model to better capture the structural information of the image. The execution entity may then use the pretrained image model as a teacher network and utilize knowledge distillation training to obtain a target initial model, which serves as a student network. The execution entity may then train the target initial model using a number of training images less than a second predetermined threshold, thereby obtaining the image generator.

[0060] Based on the above optional implementation methods, this solution can enrich the methods of obtaining image generators, and through the cooperation of pre-training and knowledge distillation, the generated image generator can have better effects without complicating the model structure.

[0061] Optionally, the second preset threshold may be 2. This allows an image generator to be trained using a single image.

[0062] Optionally, based on the above optional implementation, the first target size may be consistent with the size of the training image, thereby improving the quality of the generated image.

[0063] Step 202: Based on the texture migration technology, an image with a texture similar to that of the similar structure image is generated as a quasi-image.

[0064] In this embodiment, the execution entity may use various texture migration technologies to generate an image having a texture similar to that of the similar structure image generated in step 201 as a quasi-image.

[0065] In some optional implementations of this embodiment, the execution entity may input the similar structure image generated in step 201 into a pre-trained texture transfer model to generate an image with a texture similar to that of the similar structure image as the quasi-image. The texture transfer model may be trained based on patch matching using a number of training images less than a first preset threshold. The texture transfer model can effectively capture the texture information of the training images.

[0066] In these implementations, as an example, the texture transfer model may be a SinGAN model. Optionally, the first preset threshold may be the same as or different from the second preset threshold. The training images used to train the texture transfer model may also be the same as the training images used to train the image generator.

[0067] Based on the above optional implementation method, this scheme can obtain a texture migration model through a block matching-based training method, so that the generated quasi-image contains both the image structure information captured by the image generator and the texture information, thereby improving the content consistency between the generated quasi-image and the training image and improving the quality of the generated image.

[0068] Step 203: Generate an image with a higher resolution than the quasi-image based on super-resolution reconstruction technology.

[0069] In this embodiment, the execution entity may utilize various super-resolution reconstruction techniques to generate an image with a higher resolution than the quasi-image generated in step 202 .

[0070] In some optional implementations of this embodiment, the execution entity may input the quasi-image generated in step S202 into a pre-trained super-resolution reconstruction model to generate an image with a higher resolution than the quasi-image. The super-resolution reconstruction model may include at least one set of a serially connected equal-size smoothing module and an upsampling module. The equal-size smoothing module may include a convolutional layer, an upsampling layer, and a downsampling layer.

[0071] In these implementations, the execution entity may sequentially input the quasi-image generated in step S202 into the convolutional layer, upsampling layer, and downsampling layer of a pre-trained super-resolution reconstruction model, thereby performing filtering while ensuring that the size of the feature map remains unchanged. The execution entity may then input the filtered feature map into an upsampling module, thereby completing a resolution increase. Optionally, when the super-resolution reconstruction model includes multiple sets of equal-size smoothing modules and upsampling modules, the execution entity may repeat the resolution increase operation multiple times to generate an image with a higher resolution.

[0072] Optionally, the equal-size smoothing module may further include a convolution with a kernel size of 1. The convolution with a kernel size of 1 may be located between the convolution layer and the upsampling layer.

[0073] Based on the above optional implementation methods, this solution can achieve the goal of filtering the feature map by using the smoothness of convolution while ensuring that the size of the feature map does not change by upsampling first and then downsampling, thereby effectively removing noise from the image and improving the quality of the generated higher resolution image.

[0074] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of a method for generating an image according to an embodiment of the present disclosure. Figure 3 In an application scenario, the server can first input the acquired noise map of the first target size and the corresponding position code 301 into a pre-trained image generator 302 to generate a similar structure map 303 of the first target size. The image generator may include at least one convolution unit. The server can then use texture transfer technology to generate an image 304 with a texture similar to that of the similar structure map 303 as a quasi-image. Subsequently, using super-resolution reconstruction technology, the server can generate an image 305 with a higher resolution than the quasi-image 304.

[0075] At present, one of the existing technologies usually requires a large number of training samples to train the image generation model. When the number of samples is small, the quality of the images generated by the trained model is usually not high. The method provided by the above-mentioned embodiment of the present disclosure captures the structural information of the image through a pre-trained image generator, generates an image with a similar structure as the initial estimate of the image, and uses position encoding to enhance the convolution unit's perception of position. Texture migration is then performed on the initial estimate of the image, and then super-resolution reconstruction is performed to form an image with similar texture style and similar content, thereby achieving the effect of generating a large number of high-quality images using a small number of images or even a single image.

[0076] Further references Figure 4, which shows a process 400 of another embodiment of a method for training an image generation model. The process 400 of the method for training an image generation model includes the following steps:

[0077] Step 401 : down-sample the pre-acquired sample image to generate a low-resolution sample image having a lower resolution than the sample image.

[0078] In this embodiment, the execution subject of the method for training the image generation model (eg Figure 1 The server 105 shown in the figure may downsample the pre-acquired sample image in various ways (eg, 8-fold downsampling), thereby generating a low-resolution sample image having a lower resolution than the above-mentioned sample image.

[0079] Step 402: Input the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having the same size as the sample image.

[0080] In this embodiment, the execution entity may input the low-resolution sample image generated in step 401 into a pre-acquired initial texture reconstruction model through various methods to generate a sample reconstructed image having the same size as the sample image. The initial texture reconstruction model may include at least one of the following: an initial texture transfer model and an initial super-resolution reconstruction model. The initial texture transfer model is used for training to achieve image texture transfer. The initial super-resolution reconstruction model is used for training to achieve super-resolution reconstruction.

[0081] In some optional implementations of this embodiment, the initial texture reconstruction model may include the initial super-resolution reconstruction model. The initial super-resolution reconstruction model may include at least one set of equal-size smoothing modules and upsampling modules connected in series. The equal-size smoothing modules may include a convolution layer, an upsampling layer, and a downsampling layer.

[0082] In these implementations, the description of the initial super-resolution reconstruction model can be consistent with the corresponding description of the super-resolution reconstruction model in the optional implementation of step 203 in the aforementioned embodiment. The above description of the structure of the pre-trained super-resolution reconstruction model also applies to the initial super-resolution reconstruction model and will not be repeated here.

[0083] Based on the above optional implementation method, this solution can use the upsampling layer and downsampling layer of the above-mentioned initial super-resolution reconstruction model, and use the smoothing operation of the feature map obtained by convolution in the convolution layer to achieve filtering while ensuring that the size of the feature map does not change, thereby effectively removing noise in the image, and thus improving the quality of higher resolution images generated by the super-resolution reconstruction model after training is completed.

[0084] In some optional implementations of this embodiment, the initial texture reconstruction model may include an initial texture transfer model. The initial texture transfer model may be obtained by block matching training using training images whose number is less than the first preset threshold.

[0085] In these implementations, the description of the initial texture transfer model can be consistent with the corresponding description of the texture transfer model in the optional implementation of step 202 in the aforementioned embodiment. The above description of the pre-trained texture transfer model also applies to the initial texture transfer model and will not be repeated here.

[0086] Based on the above optional implementation, this solution can train the initial texture transfer model through a block matching-based training method to obtain a texture transfer model for capturing texture information.

[0087] Step 403 : Based on the difference between the sample reconstructed image and the sample image, a reconstruction loss value is generated using a preset reconstruction loss function.

[0088] In this embodiment, based on the difference between the sample reconstructed image and the sample image, the execution entity may generate a reconstruction loss value using a preset reconstruction loss function in various ways. The reconstruction loss function is used to determine the difference between the sample reconstructed image and the sample image, such as an L2 loss function.

[0089] Step 404: Generate an adversarial loss value based on the sample image and the pre-acquired initial generative adversarial network.

[0090] In this embodiment, based on the sample image and the pre-acquired initial generative adversarial network, the execution entity can generate the adversarial loss value in various ways. As an example, the execution entity can generate the adversarial loss value based on the sample image pre-acquired in step 401, the image generated by the generator in the initial generative adversarial network, and the discrimination result of the discriminator in the initial generative adversarial network. The generator in the initial generative adversarial network can be used to capture the structural information of the image. The structural information of the image can refer to the corresponding description in the previous embodiment and will not be repeated here.

[0091] In some optional implementations of this embodiment, based on the sample image and the pre-acquired initial generative adversarial network, the execution entity may generate the adversarial loss value according to the following steps:

[0092] In the first step, the sample image is input into the pre-trained image generator to generate an image as a real sample.

[0093] In these implementations, the execution entity may input the sample image pre-acquired in step 401 into a pre-trained image generator in various ways to generate an image serving as a real sample. The pre-trained image generator may be trained using a large-scale training set. The pre-trained image generator may be used to capture structural information about the image.

[0094] In these implementations, as an example, the pre-trained image generator can be represented by G_pre. For the pre-trained generator, the generative prior can be regarded as that the sample image (for example, represented by x) can be generated by a certain latent code through the pre-trained image generator G_pre. Based on this prior, we solve the optimization problem z * =arg min z ||G pre (z)-x|| to get the hidden code z of the generated sample image x * Then, the above pre-trained image generator can also be used to encode the hidden code z with random noise Δz * The result of the perturbation generates a new image y = G pre (z * +Δz). The execution subject may use the generated new image as a real sample.

[0095] In the second step, the obtained noise map of the second target size and the corresponding position code are input into the generator in the initial generative adversarial network to generate an image of the second target size as a fake sample.

[0096] In these implementations, the execution entity can input the acquired noise map of the second target size and the corresponding position code into the generator in the pre-acquired initial generative adversarial network in various ways to generate an image of the second target size as a fake sample. The second target size is smaller than the size of the sample image. The generator in the initial generative adversarial network is obtained based on the knowledge distillation of the pre-trained image generator. The method for obtaining the image generator in the initial generative adversarial network can refer to the corresponding description of the optional implementation method in step 201 in the aforementioned embodiment, and will not be repeated here.

[0097] It should be noted that the second target size may be the same as or different from the first target size, which is not limited here.

[0098] In the third step, based on the difference between the image of the second target size generated by downsampling the image of the real sample and the fake sample, the preset adversarial loss function is used to generate the adversarial loss value.

[0099] In these implementations, the execution entity may first downsample the real sample image generated in the first step to generate an image of the second target size. Then, the execution entity may generate an adversarial loss value using a preset adversarial loss function based on the difference between the generated image of the second target size and the fake sample generated in the second step.

[0100] In these implementations, as an example, since a low-resolution image is generated first and then super-resolution reconstruction is performed, the sample image after downsampling (for example, by x lr The texture quality of the image is often poor, but the overall structure is relatively reasonable. Downsampling can be used to retain the structural information of the image while ignoring the texture information of the image. lr The distribution of the structure is regarded as the generative prior p(y lr ). Therefore, the above adversarial loss function can be:

[0101]

[0102] The G and D above can be used to represent the generator and discriminator in the initial generative adversarial network, respectively. The meanings of other characters can be referred to above.

[0103] Based on the above optional implementation methods, this solution can enrich the way of training the image generator in the initial generative adversarial network, and through the mutual cooperation between pre-training, knowledge distillation and the generative adversarial network, the generated image generator can have better results without complicating the model structure.

[0104] Step 405: Adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value.

[0105] In this embodiment, based on the reconstruction loss and the adversarial loss, the execution entity may adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network in various ways. As an example, the execution entity may adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network using a gradient descent method and backpropagation according to the generated reconstruction loss and adversarial loss.

[0106] Optionally, the execution entity may also perform a weighted summation of the generated reconstruction loss and adversarial loss to generate a total loss. Optionally, the execution entity may also calculate a sparse loss that represents the complexity of the model structure (e.g., an attention matrix), and integrate the sparse loss into the total loss through a weighted summation.

[0107] In some optional implementations of this embodiment, based on the reconstruction loss value and the adversarial loss value, the execution entity may adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network through the following steps:

[0108] In the first step, the fake sample is input into the initial texture reconstruction model to generate a fake sample reconstructed image with the same size as the sample image.

[0109] In these implementations, the execution entity may input the fake sample generated in the second step into the initial texture reconstruction model pre-acquired in step 402 in various ways to generate a fake sample reconstructed image that is consistent with the size of the sample image. The fake sample reconstructed image may be an image obtained by using the super-resolution reconstruction technology of the fake sample described in the optional implementation of step 404. Alternatively, the execution entity may input the fake sample into the initial super-resolution reconstruction model included in the initial texture reconstruction model, and determine the generated image that is consistent with the size of the sample image (with a larger resolution than the fake sample) as the fake sample reconstructed image.

[0110] In the second step, based on the difference between the fake sample reconstructed image and the sample image, a preset perceptual loss function is used to generate a perceptual loss value.

[0111] In these implementations, based on the difference between the fake sample reconstructed image generated in the first step and the sample image pre-acquired in step 401, the execution entity may generate a perceptual loss value using a preset perceptual loss function in various ways. The preset perceptual loss function is used to measure the similarity between the fake sample reconstructed image and the sample image in a semantic space.

[0112] In the third step, based on the reconstruction loss value, adversarial loss value and perceptual loss value, the parameters of the initial texture reconstruction model and the initial generative adversarial network are adjusted.

[0113] In these implementations, the execution entity may adjust parameters of the initial texture reconstruction model and the initial generative adversarial network in various ways based on the reconstruction loss, adversarial loss, and perceptual loss. For example, the execution entity may adjust parameters of the initial texture reconstruction model and the initial generative adversarial network using backpropagation based on the generated reconstruction loss, adversarial loss, and perceptual loss using a gradient descent method.

[0114] Optionally, the execution entity may also perform a weighted summation of the generated reconstruction loss, adversarial loss, and perceptual loss to generate a total loss. Optionally, the execution entity may also calculate a sparse loss that represents the complexity of the model structure (e.g., an attention matrix), and integrate the sparse loss into the total loss through a weighted summation.

[0115] Based on the above optional implementation, this solution can refer to the perceptual loss value used to represent similarity in the semantic space when adjusting the parameters of the initial texture reconstruction model and the initial generative adversarial network, thereby preventing the generator in the initial generative adversarial network from losing important content information in the generated image. Furthermore, it can also ensure that the generated image has both a reasonable image structure and high-quality texture, thereby improving the quality of the generated image.

[0116] In some optional implementations of this embodiment, the execution entity may connect the generator in the initial generative adversarial network after parameter adjustment and the initial texture reconstruction model in series to form the image generation model.

[0117] In some optional implementations of this embodiment, the execution entity may further use the parameter-adjusted initial texture reconstruction model and initial generative adversarial network as new initial texture reconstruction model and initial generative adversarial network, and continue to iterate through steps 402-405. Furthermore, training may be stopped when a preset training stop condition is met. The execution entity may connect the generator in the trained initial generative adversarial network and the initial texture reconstruction model in series to form the image generation model.

[0118] from Figure 4 As can be seen, process 400 of the method for training an image generation model in this embodiment embodies the steps of inputting a lower-resolution sample image into a pre-acquired initial texture reconstruction model that includes texture reconstruction and super-resolution reconstruction functions, and adjusting the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the difference between the generated image and the sample image and the adversarial loss value obtained by the sample image passing through the initial generative adversarial network. Therefore, the solution described in this embodiment can form an image generation model by using the generator and texture reconstruction model in the generative adversarial network, thereby achieving the effect of generating a large number of images using fewer training images.

[0119] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for generating an image. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0120] like Figure 5As shown, the apparatus 500 for generating an image provided by this embodiment includes a first generating unit 501, a texture migration unit 502, and a super-resolution reconstruction unit 503. The first generating unit 501 is configured to input the acquired noise map of the first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator includes at least one convolution unit; the texture migration unit 502 is configured to generate an image with a texture similar to that of the similar structure map as a quasi-image based on a texture migration technique; and the super-resolution reconstruction unit 503 is configured to generate an image with a higher resolution than the quasi-image based on a super-resolution reconstruction technique.

[0121] In the apparatus 500 for generating an image in this embodiment, the specific processing of the first generating unit 501, the texture migration unit 502 and the super-resolution reconstruction unit 503 and the technical effects thereof can be referred to in the respective Figure 2 The relevant descriptions of step 201, step 202 and step 203 in the corresponding embodiment are not repeated here.

[0122] In some optional implementations of this embodiment, the above-mentioned position coding can be obtained through the following steps: generating an angular velocity variable according to a preset coding dimension; generating a sine and cosine position coding according to the angular velocity variable and the width and height of the first target size.

[0123] In some optional implementations of this embodiment, the above-mentioned texture migration unit 502 can be further configured to: input the similar structure image into a pre-trained texture migration model, and generate an image with a texture similar to the similar structure image as a quasi-image, wherein the texture migration model can be obtained based on block matching training using training images with a number less than a first preset threshold.

[0124] In some optional implementations of this embodiment, the super-resolution reconstruction unit 503 may be further configured to: input the quasi-image into a pre-trained super-resolution reconstruction model to generate an image with a higher resolution than the quasi-image, wherein the super-resolution reconstruction model may include at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module may include a convolution layer, an upsampling layer, and a downsampling layer.

[0125] In some optional implementations of this embodiment, the above-mentioned image generator can use a number of training images less than a second preset threshold to train the target initial model, and the above-mentioned target initial model can be obtained based on the knowledge distillation of a pre-trained image generation model obtained by training with a large-scale training set.

[0126] In some optional implementations of this embodiment, the first target size may be consistent with the size of the training image.

[0127] The apparatus provided by the above-described embodiments of the present disclosure uses a pre-trained image generator to capture the structural information of an image through a first generation unit 501, generates a structurally similar image as an initial estimate of the image, and uses position encoding to enhance the position perception of the convolution unit. The texture migration unit 502 then performs texture migration on the initial estimate of the image, and the super-resolution reconstruction unit 503 further performs super-resolution reconstruction to form images with similar texture styles and similar content, thereby achieving the effect of generating a large number of high-quality images using a small number of images, or even a single image.

[0128] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for training an image generation model. Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0129] like Figure 6 As shown, the device 600 for training an image generation model provided by this embodiment includes a downsampling unit 601, a second generation unit 602, a reconstruction loss generation unit 603, an adversarial loss generation unit 604 and an adjustment unit 605. Among them, the downsampling unit 601 is configured to downsample the pre-acquired sample image to generate a low-resolution sample image with a lower resolution than the sample image; the second generation unit 602 is configured to input the low-resolution sample image into the pre-acquired initial texture reconstruction model to generate a sample reconstructed image with the same size as the sample image, wherein the initial texture reconstruction model includes at least one of the following: an initial texture migration model, an initial super-resolution reconstruction model; the reconstruction loss generation unit 603 is configured to generate a reconstruction loss value based on the difference between the sample reconstructed image and the sample image using a preset reconstruction loss function; the adversarial loss generation unit 604 is configured to generate an adversarial loss value based on the sample image and the pre-acquired initial generative adversarial network, wherein the generator in the initial generative adversarial network is used to capture the structural information of the image; the adjustment unit 605 is configured to adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value.

[0130] In this embodiment, in the apparatus 600 for training an image generation model, the specific processing of the downsampling unit 601, the second generation unit 602, the reconstruction loss generation unit 603, the adversarial loss generation unit 604, and the adjustment unit 605 and the technical effects thereof can be referred to in detail. Figure 4 The relevant descriptions of step 401, step 402, step 403, step 404 and step 405 in the corresponding embodiment are not repeated here.

[0131] In some optional implementations of this embodiment, the above-mentioned initial texture reconstruction model may include an initial super-resolution reconstruction model, and the initial super-resolution reconstruction model may include at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing module includes a convolution layer, an upsampling layer and a downsampling layer.

[0132] In some optional implementations of this embodiment, the above-mentioned adversarial loss generation unit 604 may include: a true sample generation module (not shown in the figure), configured to input the sample image into a pre-trained image generator to generate an image as a true sample, wherein the pre-trained image generator is obtained by training with a large-scale training set; a fake sample generation module (not shown in the figure), configured to input the acquired noise map of the second target size and the corresponding position code into the generator in the initial generative adversarial network to generate an image of the second target size as a fake sample, wherein the second target size is smaller than the size of the sample image, and the generator in the initial generative adversarial network is obtained based on the knowledge distillation of the pre-trained image generator; an adversarial loss generation module (not shown in the figure), configured to generate an adversarial loss value using a preset adversarial loss function based on the difference between the image of the second target size generated by downsampling the image of the real sample and the fake sample.

[0133] In some optional implementations of this embodiment, the above-mentioned adjustment unit 605 can be further configured to input the fake sample into the initial texture reconstruction model to generate a fake sample reconstructed image that is consistent with the size of the sample image; based on the difference between the fake sample reconstructed image and the sample image, generate a perceptual loss value using a preset perceptual loss function; based on the reconstruction loss value, the adversarial loss value and the perceptual loss value, adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network.

[0134] In some optional implementations of this embodiment, the initial texture reconstruction model may include an initial texture transfer model. The initial texture transfer model may be obtained by block matching training using training images whose number is less than a first preset threshold.

[0135] The apparatus provided by the above-mentioned embodiment of the present disclosure inputs the lower-resolution sample image generated by the downsampling unit 601 into a pre-acquired initial texture reconstruction model including texture reconstruction and super-resolution reconstruction functions through the second generation unit 602. Based on the reconstruction loss value generated by the reconstruction loss generation unit 603 based on the difference between the image and the sample image and the adversarial loss value obtained by the adversarial loss generation unit 604 based on the sample image passing through the initial generative adversarial network, the adjustment unit 605 can adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network. Thus, the scheme described in this embodiment can form an image generation model by using the generator and texture reconstruction model in the generative adversarial network, thereby achieving the effect of generating a large number of images using fewer training images.

[0136] Reference below Figure 7 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of the server in (700). Figure 7 The server shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0137] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0138] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 7 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0139] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0140] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or convey a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer-readable medium may be conveyed using any suitable medium, including but not limited to wires, optical cables, RF (Radio Frequency), or any suitable combination thereof.

[0141] The computer-readable medium may be included in the electronic device, or may exist independently and not incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: inputs the acquired noise map of the first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator includes at least one convolution unit; generates an image with a texture similar to that of the similar structure map as a quasi-image based on texture transfer technology; and generates an image with a higher resolution than the quasi-image based on super-resolution reconstruction technology. Or the electronic device is configured to: downsample a pre-acquired sample image to generate a low-resolution sample image having a lower resolution than the sample image; input the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having the same size as the sample image, wherein the initial texture reconstruction model includes at least one of the following: an initial texture migration model, an initial super-resolution reconstruction model; based on the difference between the sample reconstructed image and the sample image, generate a reconstruction loss value using a preset reconstruction loss function; based on the sample image and a pre-acquired initial generative adversarial network, generate an adversarial loss value, wherein the generator in the initial generative adversarial network is used to capture the structural information of the image; and based on the reconstruction loss value and the adversarial loss value, adjust the parameters of the initial texture reconstruction model and the initial generative adversarial network.

[0142] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C", Python, or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to the various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising a first generation unit, a texture migration unit, and a super-resolution reconstruction unit. Alternatively, it may be described as: a processor comprising a downsampling unit, a second generation unit, a reconstruction loss generation unit, an adversarial loss generation unit, and an adjustment unit. In some cases, the names of these units do not constitute a limitation on the units themselves. For example, the first generation unit may also be described as a "unit that inputs the acquired noise map of the first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the image generator comprises at least one convolution unit".

[0145] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for generating an image, comprising: Inputting the acquired noise map of the first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the dimension of the position code matches the first target size, the position code is used to represent the position of each pixel on the noise map, and the image generator includes at least one convolution unit; Based on the texture migration technology, an image with a texture similar to that of the similar structure image is generated as a quasi-image; Based on the super-resolution reconstruction technology, an image with higher resolution than the quasi-image is generated.

2. The method according to claim 1, wherein The position code is obtained by the following steps: Generate angular velocity variables according to the preset encoding dimension; Generate sine and cosine position codes according to the angular velocity variable and the width and height of the first target size.

3. The method according to claim 1, wherein The method of generating an image having a texture similar to that of the similar structure image as a quasi-image based on the texture migration technology includes: The similar structure graph is input into a pre-trained texture transfer model to generate an image with a texture similar to that of the similar structure graph as the quasi-image, wherein the texture transfer model is trained based on block matching using training images whose number is less than a first preset threshold.

4. The method according to claim 1, wherein The step of generating an image with a higher resolution than the quasi-image based on a super-resolution reconstruction technique includes: The quasi-image is input into a pre-trained super-resolution reconstruction model to generate an image with a higher resolution than the quasi-image, wherein the super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing modules include convolution layers, upsampling layers, and downsampling layers.

5. The method according to any one of claims 1 to 4, wherein: The image generator trains a target initial model using training images whose number is less than a second preset threshold, and the target initial model is obtained based on knowledge distillation of a pre-trained image generation model obtained by training with a large-scale training set.

6. The method according to claim 5, wherein: The first target size is consistent with the size of the training image.

7. A method for training an image generation model, comprising: Downsampling the pre-acquired sample image to generate a low-resolution sample image having a lower resolution than that of the sample image; Inputting the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having the same size as the sample image, wherein the initial texture reconstruction model includes at least one of the following: an initial texture migration model and an initial super-resolution reconstruction model; generating a reconstruction loss value using a preset reconstruction loss function based on a difference between the sample reconstructed image and the sample image; Generate an adversarial loss value based on the sample image and a pre-acquired initial generative adversarial network, wherein the generator in the initial generative adversarial network is used to capture structural information of the image; Based on the reconstruction loss value and the adversarial loss value, parameters of the initial texture reconstruction model and the initial generative adversarial network are adjusted.

8. The method according to claim 7, wherein: The initial texture reconstruction model includes the initial super-resolution reconstruction model, and the initial super-resolution reconstruction model includes at least one group of equal-size smoothing modules and upsampling modules connected in series, and the equal-size smoothing modules include convolution layers, upsampling layers and downsampling layers.

9. The method according to claim 7, wherein: The generating an adversarial loss value based on the sample image and the pre-acquired initial generative adversarial network includes: Inputting the sample image into a pre-trained image generator to generate an image as a real sample, wherein the pre-trained image generator is trained by a large-scale training set; Inputting the obtained noise map of the second target size and the corresponding position code into the generator in the initial generative adversarial network to generate an image of the second target size as a fake sample, wherein the second target size is smaller than the size of the sample image, and the generator in the initial generative adversarial network is obtained based on knowledge distillation of the pre-trained image generator; Based on the difference between the image of the second target size generated by downsampling the image of the real sample and the fake sample, an adversarial loss value is generated using a preset adversarial loss function.

10. The method according to claim 9, wherein: The adjusting the parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value includes: Inputting the fake sample into the initial texture reconstruction model to generate a fake sample reconstructed image that is consistent with the size of the sample image; generating a perceptual loss value using a preset perceptual loss function based on a difference between the fake sample reconstructed image and the sample image; Based on the reconstruction loss value, the adversarial loss value, and the perceptual loss value, parameters of the initial texture reconstruction model and the initial generative adversarial network are adjusted.

11. The method according to any one of claims 7 to 10, wherein: The initial texture reconstruction model includes an initial texture migration model, and the initial texture migration model is obtained by block matching training using training images whose number is less than a first preset threshold.

12. An apparatus for generating an image, comprising: a first generating unit configured to input the acquired noise map of a first target size and the corresponding position code into a pre-trained image generator to generate a similar structure map of the first target size, wherein the dimension of the position code matches the first target size, and the position code is used to represent the position of each pixel on the noise map, and the image generator includes at least one convolution unit; a texture migration unit configured to generate an image having a texture similar to that of the similar structure image as a quasi-image based on a texture migration technology; The super-resolution reconstruction unit is configured to generate an image with a higher resolution than the quasi image based on a super-resolution reconstruction technology.

13. A device for training an image generation model, comprising: a downsampling unit configured to downsample the pre-acquired sample image to generate a low-resolution sample image having a lower resolution than that of the sample image; a second generating unit configured to input the low-resolution sample image into a pre-acquired initial texture reconstruction model to generate a sample reconstructed image having a size consistent with the sample image, wherein the initial texture reconstruction model includes at least one of the following: an initial texture migration model, an initial super-resolution reconstruction model; a reconstruction loss generating unit, configured to generate a reconstruction loss value using a preset reconstruction loss function based on a difference between the sample reconstructed image and the sample image; an adversarial loss generating unit, configured to generate an adversarial loss value based on the sample image and a pre-acquired initial generative adversarial network, wherein the generator in the initial generative adversarial network is used to capture structural information of the image; An adjustment unit is configured to adjust parameters of the initial texture reconstruction model and the initial generative adversarial network based on the reconstruction loss value and the adversarial loss value.

14. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

15. A computer-readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images