Image generation method and device, equipment, medium and program product

Abstract

The invention provides an image generation method and device, equipment, a medium and a program product. The method comprises the following steps: S101, carrying out autoregressive processing on a pre-filling token and a hidden space token with a plurality of resolutions through an autoregressive Transform model of an image generation network, and generating an input condition; wherein the hidden space token is obtained by encoding an original image by using an encoder; s102, carrying out up-sampling operation on the generated input condition; s103, inputting the up-sampled input condition into a diffusion model of the image generation network, performing diffusion denoising on the input condition through the diffusion model, and generating a hidden space token of the current resolution; s104, judging whether the current resolution reaches the target resolution or not; if yes, executing the step S105, and if not, returning to the step S101; and S105, restoring the hidden space token into an image through a decoder to obtain a target image.

Description

Technical Field

[0001] The present invention relates to the field of image generation, and in particular to an image generation method, device, equipment, medium and program product. Background Art

[0002] In recent years, autoregressive image generation technology has received widespread attention and application. Autoregressive models were initially successful in the field of natural language processing and were later applied to image generation tasks. This type of model predicts the image content pixel by pixel or token by token, using previously generated information to predict the next element. However, autoregressive models face challenges in image generation, such as low computational efficiency, difficulty in scaling to high resolution, and poorer generation results than diffusion models. In order to apply autoregressive image generation to practical applications, these challenges need to be addressed.

[0003] Resolution pyramids are widely used in traditional computer vision perception tasks. The core idea of ​​resolution pyramids is to represent an image as a series of images of different resolutions. Usually, the bottom layer of the pyramid is the original high-resolution image, and the resolution of the image is reduced by half with each layer. This multi-scale representation allows the algorithm to analyze images at different levels of detail, thereby capturing image features at different scales. This technology allows the model to explicitly process features at different resolutions separately, greatly improving the efficiency and effectiveness of various image processing and analysis algorithms.

[0004] VAR (Visual Autoregressive Modeling) is an autoregressive image generation algorithm based on resolution pyramid. It aims to solve some challenges of traditional autoregressive models in image generation, such as low computational efficiency and difficulty in scaling to high-resolution images. VAR decomposes the image into different resolutions and gradually generates image details from coarse to fine. At the same time, the autoregressive model can generate images by predicting the probability distribution of the next element in the sequence.

[0005] Compared with the traditional autoregressive model, the VAR method adopts the method of "next scale prediction" instead of "next token prediction", uses multi-scale VQ-VAE to encode images, and uses the transformer architecture to efficiently generate images of each scale. This method significantly improves image quality, generation speed and computational efficiency.

[0006] Although the VAR method has improved the quality of image generation, there is still a certain gap in the authenticity and detail of the generated images compared with the latest diffusion model and other methods. In addition, the training process of the autoregressive model is not stable enough, and it is prone to mode collapse or inconsistent generation quality. Summary of the invention

[0007] In view of this, the embodiments of the present disclosure provide an image generation method, apparatus, device, medium and program product, which at least partially solve the problems existing in the prior art.

[0008] In a first aspect, an embodiment of the present disclosure provides an image generation method, which includes:

[0009] S101, performing autoregressive processing on input pre-filled tokens and latent space tokens of multiple resolutions through an autoregressive Transformer model of an image generation network to generate corresponding input conditions; wherein the latent space tokens are obtained by encoding the original image using an encoder;

[0010] S102, performing an upsampling operation on the generated input condition;

[0011] S103, inputting the upsampled input condition into a diffusion model of an image generation network, performing diffusion denoising on the input condition through the diffusion model, and generating a latent space token of a current resolution;

[0012] S104, determining whether the current resolution reaches the target resolution; if so, executing S105, if not, returning to step S101;

[0013] S105, restoring the latent space token to an image through a decoder to obtain a target image.

[0014] According to a specific implementation of the embodiment of the present disclosure, in step S101:

[0015] Downsample the original image at different levels to obtain multiple images with different resolutions;

[0016] An encoder is used to convert images of different resolutions into corresponding multiple latent space tokens.

[0017] According to a specific implementation of the embodiment of the present disclosure, input conditions of different resolutions are subjected to diffusion denoising using different diffusion models.

[0018] According to a specific implementation of the embodiment of the present disclosure, a training process of an image generation network is also included, and the training process includes:

[0019] Downsampling the training images as training samples at different levels to obtain a series of images with different resolutions;

[0020] Use an encoder to convert images of different resolutions into corresponding target latent space tokens, thereby mapping the image information in the pixel space to the latent space;

[0021] Diffusion operation is performed on the target latent space tokens to add noise;

[0022] Calculate the loss between the latent space tokens output by the image generation network at each resolution and the corresponding target latent space tokens, and then sum the losses at different resolutions to get the overall loss;

[0023] By using the overall loss, the weight gradients of the autoregressive Transformer model and the diffusion model are calculated through the back propagation algorithm, and the network parameters of the image generation network are updated accordingly, thereby realizing end-to-end training of the image generation network.

[0024] According to a specific implementation manner of an embodiment of the present disclosure, the loss between the latent space token and the corresponding target latent space token is an L2 loss.

[0025] According to a specific implementation of an embodiment of the present disclosure, the encoder is a VAE encoder, and the decoder is a variational autoencoder VAE decoder.

[0026] In a second aspect, an embodiment of the present disclosure provides an image generating device, comprising:

[0027] An autoregressive processing unit, configured to perform autoregressive processing on input pre-filled tokens and latent space tokens of multiple resolutions through an autoregressive Transformer model of an image generation network to generate corresponding input conditions; wherein the latent space tokens are obtained by encoding the original image using an encoder;

[0028] An upsampling unit, used for performing an upsampling operation on the generated input condition;

[0029] A diffusion unit, used to input the upsampled input condition into a diffusion model of the image generation network, and perform diffusion denoising on the input condition through the diffusion model to generate a latent space token of the current resolution;

[0030] A resolution judgment unit, used to judge whether the current resolution reaches the target resolution; if so, the decoding unit is notified; if not, the autoregressive unit is notified;

[0031] A decoding unit is used to restore the latent space token to an image through a decoder to obtain a target image.

[0032] According to a specific implementation of the embodiment of the present disclosure, input conditions of different resolutions are subjected to diffusion denoising using different diffusion models.

[0033] In a third aspect, an embodiment of the present disclosure provides an image generating device, the device comprising:

[0034] at least one processor; and,

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned image generation method.

[0037] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer instructions for causing the computer to execute the image generation method as described above.

[0038] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes program instructions. When the program instructions are executed by a computer, the computer executes the image generation method as described above.

[0039] In summary, the high-resolution image generation method provided by the present invention realizes efficient and high-quality progressive image generation by organically combining the autoregressive Transformer model, the diffusion model and the resolution pyramid structure. Specifically, the autoregressive Transformer model is responsible for generating conditional information for each resolution, and the diffusion model is responsible for generating high-quality image content on this basis. Then, through the resolution pyramid structure, high-resolution images can be gradually generated from low resolutions, which not only improves computational efficiency but also ensures generation quality. In addition, the hybrid architecture of the present invention overcomes the limitations of a single model and provides a new solution for image generation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A schematic diagram of a flow chart of an image generating method provided by a first embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the working principle of an image generation network provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of training an image generation network provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of an image generating device provided by a second embodiment of the present invention;

[0045] Figure 5 An exemplary structural diagram of a device capable of implementing the method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0046] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0047] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0048] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0049] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0050] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0051] See also Figure 1 and Figure 2 As shown, the first embodiment of the present invention provides an image generating method, which can be executed by an image generating apparatus or an image generating device.

[0052] like Figure 1 As shown, at step S101, the input pre-filled tokens and latent space tokens of multiple resolutions are autoregressively processed through the autoregressive Transformer model of the image generation network to generate corresponding input conditions; wherein, the latent space tokens are obtained by encoding the original image using an encoder.

[0053] In this embodiment, the prefill tokens refer to a series of input tags (or tokens) that are pre-processed and filled in order to calculate the initial intermediate state (such as keys and values) before the model performs autoregressive generation. These tokens are the atomic parts of the language processed by the model and are used to start the decoding process of the model.

[0054] In this embodiment, for latent space tokens, the original image may be downsampled at different levels to obtain multiple images of different resolutions; and then an encoder is used to convert the images of different resolutions into corresponding multiple latent space tokens.

[0055] The encoder may use a variational auto-encoder (VAE) encoder, and of course other similar encoders may also be used, which is not specifically limited in the present invention. It should be understood that the internal structure of the VAE includes an encoder and a decoder, and the VAE encoder refers to the encoder inside the variational auto-encoder.

[0056] In this embodiment, the function of step S101 is to generate the input conditions of the diffusion model of each resolution, thereby realizing a progressive generation process from low resolution to high resolution. Figure 2 As shown in Figure 1, by inputting pre-filled tokens and latent space tokens z1-z3 of multiple resolutions into the autoregressive Transformer model, the input conditions c1-c4 can be obtained.

[0057] Next, go to step S102.

[0058] In step S102, an upsampling operation is performed on the generated input condition.

[0059] Among them, Figure 2 As shown, by performing an upsampling operation on the input conditions c1-c4, upsampled input conditions u1-u4 can be obtained.

[0060] The main purpose of upsampling is to enlarge the original image and adjust the low-resolution conditional information to a higher resolution to prepare for the subsequent diffusion process. There are three common methods of upsampling: bilinear interpolation, transposed convolution, and unpooling. The specific upsampling method can be set according to actual needs, and the present invention will not go into details here.

[0061] Next, go to step S103.

[0062] At step S103, the upsampled input condition is input into the diffusion model of the image generation network, and the input condition is subjected to diffusion denoising by the diffusion model to generate a latent space token of the current resolution.

[0063] In this embodiment, the image generation network includes an autoregressive Transformer model and a diffusion model, which works by gradually adding noise to the data and then learning to reverse the process to generate new samples. Unlike generative adversarial networks (GANs) or variational autoencoders (VAEs), diffusion models do not require careful adjustment of the model architecture or training procedures to generate high-quality images. A key advantage of diffusion models is that they can operate directly in continuous data space, avoiding information loss during discretization. This enables diffusion models to generate more realistic and diverse images.

[0064] In this embodiment, by incorporating the advantages of the diffusion model in step S103, the quality and detail expression of the generated image are improved.

[0065] It should be noted that, in some embodiments, different resolutions are subjected to diffusion denoising by different diffusion models, which can make the diffusion model have a smaller model size, thereby reducing the computational overhead. Figure 2 As shown, by inputting the input conditions u1-u4 into the diffusion models D1-D4, latent space tokens z1-z4 of the current resolution can be generated.

[0066] In step S104, it is determined whether the current resolution reaches the target resolution; if so, S105 is executed; if not, the process returns to step S101.

[0067] S105, restoring the latent space token to an image through a decoder to obtain a target image.

[0068] Specifically, in this embodiment, the resolution and details of the image can be gradually improved through continuous iteration. If the resolution of the image reaches the target resolution, the latent space token is restored to the image through the decoder to obtain the target image. If the target resolution has not been reached, the generated latent space token of the current resolution is fed back to the autoregressive Transformer model for the generation of the next higher resolution latent space token until the target resolution is reached.

[0069] In this embodiment, for example, a VAE decoder can be used to complete the conversion from latent space to pixel space, and the final latent space token can be restored to the original image.

[0070] In summary, the image generation method provided in this embodiment realizes efficient and high-quality progressive image generation by organically combining the autoregressive Transformer model, the diffusion model and the resolution pyramid structure. Specifically, the autoregressive Transformer model is responsible for generating conditional information for each resolution, and the diffusion model is responsible for generating high-quality image content on this basis. Then, through the resolution pyramid structure, high-resolution images can be gradually generated from low resolutions, which not only improves computational efficiency but also ensures generation quality. In addition, the hybrid architecture of this embodiment also overcomes the limitations of a single model and provides a new solution for image generation tasks.

[0071] To facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0072] Based on the above embodiment, before image generation, the image generation network needs to be trained. Figure 3 , the training method is as follows:

[0073] First, the training images used as training samples are downsampled at different levels to obtain a series of images with different resolutions.

[0074] The purpose of downsampling is to generate thumbnails of the training images. Specifically, for a training image, assuming its size is M*N, it is downsampled by s times, that is, an image with a resolution of (M / s)*(N / s) is obtained, where s should be a common divisor of M and N.

[0075] It can be seen that by setting different downsampling multiples s, a series of images with different resolutions can be obtained.

[0076] Then, an encoder is used to transform images of different resolutions into corresponding target latent space tokens, thereby mapping the image information in the pixel space to the latent space.

[0077] In particular, the encoder may be a VAE encoder.

[0078] Next, a diffusion operation is performed on the target latent space tokens to add noise.

[0079] Next, we calculate the latent space tokens ( Figure 3 z1-z4) and the corresponding target latent space tokens ( Figure 3 The loss between y1-y4) in the figure is then summed up at different resolutions to get the overall loss;

[0080] The loss between the latent space token and the corresponding target latent space token may be an L2 loss. Of course, it may also be other losses, which is not specifically limited in the present invention.

[0081] Finally, the overall loss is used to calculate the weight gradients of the autoregressive Transformer model and the diffusion model through the back-propagation algorithm, and the network parameters of the image generation network are updated accordingly, thereby realizing end-to-end training of the image generation network.

[0082] Figure 4 The image generating device 200 provided by the second embodiment of the present invention is shown. The image generating device 200 comprises: an autoregressive processing unit 210, an up-sampling unit 220, a diffusion unit 230, a resolution determination unit 240 and a decoding unit 250.

[0083] The autoregressive processing unit 210 is used to perform autoregressive processing on the input pre-filled tokens and latent space tokens of multiple resolutions through the autoregressive Transformer model of the image generation network to generate corresponding input conditions; wherein the latent space tokens are obtained by encoding the original image using an encoder.

[0084] The up-sampling unit 220 is used to perform an up-sampling operation on the generated input condition.

[0085] The diffusion unit 230 is used to input the upsampled input conditions into the diffusion model of the image generation network, and perform diffusion denoising on the input conditions through the diffusion model to generate latent space tokens of the current resolution.

[0086] The resolution determination unit 240 is used to determine whether the current resolution reaches the target resolution; if so, the decoding unit 250 is notified; if not, the autoregressive unit 210 is notified.

[0087] The decoding unit 250 is used to restore the latent space token to an image through a decoder to obtain a target image.

[0088] A third embodiment of the present invention provides an image generating device, the device comprising:

[0089] at least one processor; and,

[0090] a memory communicatively connected to the at least one processor; wherein,

[0091] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned image generation method.

[0092] Figure 5 A schematic diagram of a method or device 1000 that can implement an embodiment of the present invention is shown, which may include more or fewer devices than shown in the figure in some embodiments. In some embodiments, it can be implemented using a single or multiple devices. In some embodiments, it can be implemented using cloud or distributed devices.

[0093] like Figure 5 As shown, the device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) 1002 or the program and / or data loaded from the storage part 1008 to the random access memory (RAM) 1003. Processor 1001 can be a multi-core processor or can include multiple processors. In some embodiments, processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM 1003, various programs and data required for the operation of device 1000 are also stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0094] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0095] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed. Figure 5 Only some components are schematically shown in the figure, which does not mean that the device 1000 only includes Figure 5 Components shown.

[0096] A fourth embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image generation method as described above.

[0097] The fifth embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute the image generation method of any of the aforementioned embodiments.

[0098] It should be understood that although the above embodiments are described for application scenarios of image generation, those skilled in the art can easily expand and apply the technology described in the present disclosure to scenarios of video generation, which is not elaborated in the present disclosure.

[0099] The systems, devices, modules or units described in the above embodiments may be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server or a combination thereof.

[0100] Although not shown, in an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in Embodiment 1 is implemented.

[0101] Storage media in embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be used to store information by any method or technology. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0102] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which implements the method described in Embodiment 1 when executed by a processor.

[0103] The methods, programs, systems, devices, etc. of the embodiments of the present invention may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0104] Those skilled in the art should understand that the embodiments of the present specification may be provided as methods, systems or computer program products. Therefore, those skilled in the art may imagine that the functional modules / units or controllers and related method steps described in the above embodiments may be implemented in software, hardware or a combination of software / hardware.

[0105] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0106] In this article, multiple embodiments of the present invention are described, but for the sake of brevity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this article, "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" are meant to be applicable to at least one embodiment or example according to the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0107] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An image generation method, characterized in that: include: S101, performing autoregressive processing on input pre-filled tokens and latent space tokens of multiple resolutions through an autoregressive Transformer model of an image generation network to generate corresponding input conditions; wherein the latent space tokens are obtained by encoding the original image using an encoder; S102, performing an upsampling operation on the generated input condition; S103, inputting the upsampled input condition into a diffusion model of an image generation network, performing diffusion denoising on the input condition through the diffusion model, and generating a latent space token of a current resolution; S104, determining whether the current resolution reaches the target resolution; if so, executing S105, if not, returning to step S101; S105, restoring the latent space token to an image through a decoder to obtain a target image.

2. The image generation method according to claim 1, characterized in that: In step S101: Downsample the original image at different levels to obtain multiple images with different resolutions; An encoder is used to convert images of different resolutions into corresponding multiple latent space tokens.

3. The image generation method according to claim 1, characterized in that: In step S103, different diffusion models are used to perform diffusion denoising for different input conditions.

4. The image generation method according to claim 1, characterized in that: Also included is a training process of an image generation network, the training process comprising: Downsampling the training images as training samples at different levels to obtain a series of images with different resolutions; Use an encoder to convert images of different resolutions into corresponding target latent space tokens, thereby mapping the image information in the pixel space to the latent space; Diffusion operation is performed on the target latent space tokens to add noise; Calculate the loss between the latent space tokens output by the image generation network at each resolution and the corresponding target latent space tokens, and then sum the losses at different resolutions to get the overall loss; By using the overall loss, the weight gradients of the autoregressive Transformer model and the diffusion model are calculated through the back propagation algorithm, and the network parameters of the image generation network are updated accordingly, thereby realizing end-to-end training of the image generation network.

5. The image generation method according to claim 4, characterized in that: The loss between the latent space token and the corresponding target latent space token is the L2 loss.

6. The image generation method according to claim 1, characterized in that: The encoder is a variational autoencoder (VAE) encoder, and the decoder is a VAE decoder.

7. An image generating device, characterized in that: include: An autoregressive processing unit, configured to perform autoregressive processing on input pre-filled tokens and latent space tokens of multiple resolutions through an autoregressive Transformer model of an image generation network to generate corresponding input conditions; wherein the latent space tokens are obtained by encoding the original image using an encoder; An upsampling unit, used for performing an upsampling operation on the generated input condition; A diffusion unit, used to input the upsampled input condition into a diffusion model of the image generation network, and perform diffusion denoising on the input condition through the diffusion model to generate a latent space token of the current resolution; A resolution judgment unit, used to judge whether the current resolution reaches the target resolution; if so, the decoding unit is notified; if not, the autoregressive unit is notified; A decoding unit is used to restore the latent space token to an image through a decoder to obtain a target image.

8. An image generating device, characterized in that: The equipment includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image generation method described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which are used to enable the computer to execute the image generation method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the image generating method according to any one of claims 1 to 6.

Images