Image generation method and device

By selecting the processing mode according to the target resolution, images of different resolutions can be generated, solving the problem that existing technologies can only generate images of specific resolutions, and achieving flexible image generation and improved storage efficiency.

CN119068073APending Publication Date: 2024-12-03LENOVO (BEIJING) LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411215005.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing image generation models can only generate images at a specific resolution, which cannot meet users' needs to generate images of different resolutions in different scenarios.

Method used

By determining the target resolution, selecting the appropriate processing mode for the image generation model, and using the image generation model to process the input data, an image at the target resolution is generated.

Benefits of technology

It enables the generation of images with corresponding resolutions based on different resolution requirements, meeting the needs of different usage scenarios and saving device storage space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068073A_ABST
    Figure CN119068073A_ABST
Patent Text Reader

Abstract

The invention discloses an image generation method and device, and the method comprises the steps: obtaining input data used for generating an image and a target resolution, the target resolution being used for indicating the resolution of the generated image, and the target resolution being a first resolution or a second resolution; based on the target resolution, a processing mode of an image generation model corresponding to the target resolution is determined, according to the processing mode of the corresponding image generation model, the image generation model is utilized to process the input data, a generated image with the target resolution is obtained, and the processing modes of the image generation models corresponding to different resolutions are different.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to an image generation method and device. Background Art

[0002] Some electronic devices are equipped with an image generation model based on artificial intelligence technology. The image generation model can process the input data and output a number of images generated by the model. For example, the image generation model can generate another new frame of cat images based on an input frame of cat images.

[0003] Current image generation models can generally only generate images of a specific resolution, for example, only 512*512 pixel images, or only 1024*1024 pixel images, which cannot meet the needs of users to generate images of different resolutions in different scenarios. Summary of the invention

[0004] To this end, this application discloses the following technical solutions:

[0005] The first aspect of the present application provides an image generation method, comprising:

[0006] Obtaining input data and a target resolution for generating an image, wherein the target resolution is used to indicate a resolution of the generated image, and the target resolution is a first resolution or a second resolution;

[0007] Based on the target resolution, a processing mode of an image generation model corresponding to the target resolution is determined; according to the corresponding processing mode of the image generation model, the input data is processed using the image generation model to obtain a generated image with the target resolution, wherein different resolutions correspond to different processing modes of the image generation model.

[0008] Optionally, the image generation model includes a first image generation model corresponding to the first resolution, and a second image generation model corresponding to the second resolution, the first image generation model includes a first neural network and a first decoder, and the second image generation model includes a second neural network and a second decoder;

[0009] The step of processing the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution includes:

[0010] Determine a target image generation model corresponding to the target resolution, wherein the target image generation model is the first image generation model or the second image generation model;

[0011] The input data is processed according to the neural network and decoder of the target image generation model to obtain a generated image with the target resolution.

[0012] Optionally, the image generation model includes a target neural network and a target decoder, and the target neural network is a first neural network corresponding to the first resolution or a second neural network corresponding to the second resolution;

[0013] The step of processing the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution includes:

[0014] In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target features to be decoded corresponding to the target resolution. In different processing modes, the processing flow of processing the input data according to the target neural network is different;

[0015] The target to-be-decoded features are processed according to the target decoder to obtain a generated image with the target resolution.

[0016] Optionally, the target neural network is a second neural network corresponding to the second resolution, and the first resolution is smaller than the second resolution;

[0017] In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target to-be-decoded features corresponding to the target resolution, including:

[0018] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are processed to adjust the resolution to obtain target features to be decoded corresponding to the first resolution;

[0019] When the target resolution is the second resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are used as the target features to be decoded.

[0020] Optionally, the target neural network is a first neural network corresponding to the first resolution, and the first resolution is smaller than the second resolution;

[0021] In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target to-be-decoded features corresponding to the target resolution, including:

[0022] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the first resolution, and the features to be decoded are used as the target features to be decoded;

[0023] When the target resolution is the second resolution, multiple runs of the input data are processed by the target neural network to obtain multiple features to be decoded corresponding to the first resolution; and a target feature to be decoded corresponding to the second resolution composed of the multiple features to be decoded is obtained.

[0024] Optionally, the target neural network is loaded into a first processing unit of an embedded neural network processor of the electronic device, and the target decoder is loaded into a second processing unit of the embedded neural network processor, and the first processing unit and the second processing unit are different;

[0025] The first processing unit is used to run the target neural network multiple times when the target resolution is the second resolution.

[0026] Optionally, when the target resolution is a first resolution, the target decoder is a first decoder corresponding to the first resolution;

[0027] When the target resolution is a second resolution, the target decoder is a second decoder corresponding to the second resolution.

[0028] Optionally, the image processing model further includes an encoder, which is used to process input data to obtain input features, and the input features are used to input into the target neural network;

[0029] The encoder is loaded into the central processor of the electronic device.

[0030] Optionally, the input data includes input image data and input text data;

[0031] The encoder includes an image encoder and a text encoder;

[0032] The text encoder is used to process the input text data to obtain input text features;

[0033] The image encoder is used to process the input image data to obtain input image features;

[0034] The input image features and the input text features constitute the input features.

[0035] A second aspect of the present application provides an image generating device, comprising:

[0036] an obtaining unit, configured to obtain input data for generating an image and a target resolution, wherein the target resolution is used to indicate a resolution of the generated image, and the target resolution is a first resolution or a second resolution;

[0037] A processing unit is used to determine a processing mode of an image generation model corresponding to the target resolution based on the target resolution, and to process the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution, wherein different resolutions correspond to different processing modes of the image generation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 is a flow chart of an image generation method provided in an embodiment of the present application;

[0040] Figure 2 is a structural schematic diagram of an image generation model provided in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of obtaining a generated image according to different processing modes provided in an embodiment of the present application;

[0042] Figure 4 is another schematic diagram of obtaining a generated image according to different processing modes provided in an embodiment of the present application;

[0043] Figure 5 It is another schematic diagram of obtaining a generated image according to different processing modes provided in an embodiment of the present application;

[0044] Figure 6 It is a structural schematic diagram of an image generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] This application embodiment provides an image generation method, see Figure 1 , the method may include the following steps.

[0047] S101, obtaining input data and a target resolution for generating an image, where the target resolution is used to indicate the resolution of the generated image, and the target resolution is a first resolution or a second resolution.

[0048] The input data used to generate an image may include one or more frames of input image data. The input image data may include image data stored locally by the electronic device that executes the image generation method, or may include image data obtained by the electronic device from a server. The electronic device may determine one or more frames of image data from the locally stored image data and the image data on the server as input image data according to the user's operation.

[0049] The target resolution may be the resolution of the generated image. In S101, the electronic device may display two options corresponding to the first resolution and the second resolution respectively. If the user selects the option of the first resolution, it is determined that the generated image with the first resolution needs to be generated, and the target resolution is determined to be the first resolution. If the user selects the option of the second resolution, it is determined that the generated image with the second resolution needs to be generated, and the target resolution is determined to be the second resolution.

[0050] As some examples, the first resolution may be 512*512, the second resolution may be 1024*1024, and the device may determine the target resolution to be 512*512 or 1024*1024 according to the user's selection.

[0051] S102, based on the target resolution, determine the processing mode of the image generation model corresponding to the target resolution, and use the image generation model to process the input data according to the corresponding processing mode of the image generation model to obtain a generated image with the target resolution, wherein different resolutions correspond to different processing modes of the image generation model.

[0052] In step S102, if the target resolution is the first resolution, the device can process the input data according to the processing mode corresponding to the first resolution of the image generation model to obtain a generated image with the first resolution. If the target resolution is the second resolution, the device can process the input data according to the processing mode corresponding to the second resolution of the image generation model to obtain a generated image with the second resolution.

[0053] The above-mentioned image generation model can be pre-stored in the memory of the electronic device. When an image needs to be generated, the electronic device loads the image generation model from the memory to the processor for running the model, thereby processing the input data according to the image generation model.

[0054] See also Figure 2 In this embodiment, an image generation model may include at least one encoder, at least one neural network and at least one decoder. The encoder may be a text-image pre-training model (Constrastive Language-Image Pre-training, Clip) based on contrastive learning, or other model structures with encoding capabilities in the relevant technology; the neural network may be a U-type network structure (Unet), or other neural network structures in the relevant technology; the decoder may be a variational auto-encoder (VAE) decoder, or other decoders in the relevant technology.

[0055] When processing input data according to the image generation model, the input data is first processed by the encoder to obtain the corresponding input features, then the input features are processed according to the neural network to obtain the features to be decoded, and finally the features to be decoded are processed by the decoder to obtain the generated image.

[0056] In this embodiment, the processor used to run the model may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), and an embedded neural network processor (NPU) of the electronic device.

[0057] Optionally, when the device loads the image generation model, if the device has an NPU, the encoder, neural network, and decoder included in the image generation model can be loaded into the NPU. Alternatively, the encoder can be loaded into the CPU of the device, and the neural network and decoder can be loaded into the NPU of the device.

[0058] Compared with a solution of loading the encoder into the NPU, loading the encoder into the CPU can improve the efficiency of processing input data according to the encoder.

[0059] The beneficial effects of this embodiment are:

[0060] The image generation model can process input data according to different processing modes for different resolutions to obtain generated images of the first resolution or the second resolution, thereby meeting the requirements for obtaining images of different resolutions in different usage scenarios.

[0061] In some optional embodiments, the image generation model may include at least two image generation models, for example, may include a first image generation model corresponding to the first resolution, and a second image generation model corresponding to the second resolution.

[0062] The first image generation model may include a first neural network and a first decoder for generating an image with a first resolution, and the second image generation model may include a second neural network and a second decoder for generating an image with a second resolution.

[0063] In this case, see Figure 3 , according to the processing mode of the corresponding image generation model, the image generation model is used to process the input data to obtain a generated image with a target resolution in the following manner:

[0064] Determine a target image generation model corresponding to the target resolution, where the target image generation model is the first image generation model or the second image generation model;

[0065] The input data is processed according to the neural network and decoder of the target image generation model to obtain a generated image with the target resolution.

[0066] That is to say, if the target resolution is the first resolution, the target image generation model determined by the device may include a first neural network and a first decoder. At this time, the device can process the input data according to the first neural network and the first decoder to obtain a generated image with a first resolution.

[0067] If the target resolution is the second resolution, the target image generation model determined by the device may include a second neural network and a second decoder. At this time, the device can process the input data according to the second neural network and the second decoder to obtain a generated image with the second resolution.

[0068] Optionally, the first neural network and the first decoder, and the second neural network and the second decoder can all be stored in the memory of the device. If it is determined that the target image generation model is the first image generation model, the device loads the first neural network and the first decoder to the corresponding processor, thereby obtaining a generated image with a first resolution according to the first image generation model. If it is determined that the target image generation model is the second image generation model, the device loads the second neural network and the second decoder to the corresponding processor, thereby obtaining a generated image with a second resolution according to the second image generation model.

[0069] In the above embodiment, the first image generation model and the second image generation model may share a common encoder. Figure 3As shown, the device can process the input data according to the encoder to obtain corresponding input features. At this time, if the target resolution is the first resolution, the input features are input into the first neural network, and the generated image with the first resolution is obtained by sequentially processing through the first neural network and the first decoder. If the target resolution is the second resolution, the input features are input into the second neural network, and the generated image with the second resolution is obtained by sequentially processing through the second neural network and the second decoder.

[0070] The advantage of sharing the encoder is that when the device stores the first image generation model and the second image generation model, it only needs to store one encoder, thus saving the storage space of the device.

[0071] In some optional embodiments, the image generation model may include a target neural network and a target decoder.

[0072] Among them, the target neural network can be a first neural network corresponding to the first resolution and used to generate an image of the first resolution, or it can be a second neural network corresponding to the second resolution and used to generate an image of the second resolution.

[0073] The target decoder may include one or more decoders.

[0074] In this case, according to the processing mode of the corresponding image generation model, the image generation model is used to process the input data to obtain a generated image with a target resolution in the following manner:

[0075] In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target features to be decoded corresponding to the target resolution. In different processing modes, the processing flow of processing the input data according to the target neural network is different;

[0076] The target features to be decoded are processed according to the target decoder to obtain a generated image with a target resolution.

[0077] In this embodiment, in the processing modes corresponding to different resolutions, the target neural network may correspond to different processing flows. That is, if the target resolution is the first resolution, the device may process the input data according to the processing flow corresponding to the first resolution according to the target neural network to obtain the target features to be decoded corresponding to the first resolution, and then use the target decoder to process the target features to be decoded of the first resolution to obtain a generated image with the first resolution. If the target resolution is the second resolution, the device may process the input data according to the processing flow corresponding to the second resolution according to the target neural network to obtain the target features to be decoded corresponding to the second resolution, and then use the target decoder to process the target features to be decoded of the second resolution to obtain a generated image with the second resolution.

[0078] The beneficial effect of this embodiment is that the device only needs to use the target neural network to obtain generated images of different resolutions through different processing flows. Therefore, the memory of the device only needs to store one target neural network to meet the needs of generating images of multiple resolutions, without the need to store multiple neural networks corresponding to multiple resolutions, thereby saving the storage space of the device.

[0079] In some optional embodiments, the target neural network may be a second neural network corresponding to the second resolution, and the first resolution may be smaller than the second resolution. For example, the first resolution may be 512*512, and the second resolution may be 1024*1024.

[0080] In this case, in the processing mode corresponding to the target resolution, the method of processing the input data according to the target neural network to obtain the target to-be-decoded features corresponding to the target resolution may include:

[0081] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are processed to adjust the resolution to obtain target features to be decoded corresponding to the first resolution;

[0082] When the target resolution is the second resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are used as target features to be decoded.

[0083] like Figure 4 As shown, in the case where the target neural network is the second neural network, if the target resolution is the first resolution, then the processing flow of processing the input data according to the target neural network can be:

[0084] The input data is processed by the encoder to obtain input features, and the input features are processed according to the second neural network to obtain features to be decoded corresponding to the second resolution output by the second neural network. The resolution of the features to be decoded is then adjusted to convert the features to be decoded corresponding to the second resolution into target features to be decoded corresponding to the first resolution.

[0085] If the target resolution is the second resolution, then the processing flow for processing input data according to the target neural network can be:

[0086] The input data is processed by the encoder to obtain input features, and the input features are processed according to the second neural network to obtain features to be decoded corresponding to the second resolution output by the second neural network, and the features to be decoded output by the second neural network are determined as target features to be decoded.

[0087] Optionally, the device may implement the above-mentioned different processing flows by configuring different conversion parameters corresponding to different resolutions.

[0088] In this embodiment, the image generation model may include a conversion matrix between the target neural network and the target decoder, which is denoted as matrix D. The parameters contained in matrix D are called conversion parameters. After obtaining the features to be decoded output by the target neural network, the features to be decoded can be processed by the matrix D, and the results obtained are used as the target features to be decoded. The processing process can be expressed as C*D=E, where C represents the features to be decoded output by the target neural network, and E represents the target features to be decoded obtained after processing.

[0089] When the target resolution is the first resolution, the device may configure the conversion parameters of the matrix D to be the first conversion parameters capable of implementing resolution adjustment processing, so that the matrix D may be used to process the features to be decoded corresponding to the second resolution into the target features to be decoded corresponding to the first resolution;

[0090] When the target resolution is the second resolution, the device can configure the conversion parameters of the matrix D to be the second conversion parameters that make the data C and E before and after processing the same, so that processing the features to be decoded with the matrix D is equivalent to directly determining the features to be decoded as the target features to be decoded.

[0091] The form of the matrix D and the specific values ​​of the first conversion parameter and the second conversion parameter can be set according to actual conditions and are not limited.

[0092] As an example, when the first resolution is 512*512 and the second resolution is 1024*1024, the matrix D can be a 1024*1024 matrix. See the formula below. The conversion parameters contained in the matrix D can be four block matrices D1 to D4, each of which is a 512*512 matrix.

[0093]

[0094] If the target resolution is the second resolution, the second conversion parameter configured by the device may be that the four block matrices D1 to D4 are all unit matrices, that is, the elements on the diagonal of each block matrix are all equal to 1, and the elements on the off-diagonal are all equal to 0.

[0095] If the target resolution is the first resolution, the first conversion parameter configured by the device may be that at least one of the four block matrices D1 to D4 is not a unit matrix.

[0096] In some optional embodiments, the target neural network may be a first neural network corresponding to the first resolution. In this case, in a processing mode corresponding to the target resolution, processing the input data according to the target neural network to obtain the target features to be decoded corresponding to the target resolution may include:

[0097] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the first resolution, and the features to be decoded are used as target features to be decoded;

[0098] When the target resolution is the second resolution, multiple runs of the target neural network processing input data are performed to obtain multiple features to be decoded corresponding to the first resolution; and a target feature to be decoded corresponding to the second resolution composed of the multiple features to be decoded is obtained.

[0099] See also Figure 5 When the target resolution is the first resolution, the input features processed by the encoder can be input into the first neural network, and the input features are processed according to the first neural network to obtain features to be decoded corresponding to the first resolution. Then, the features to be decoded are input into the target decoder as target features to be decoded, and the target features to be decoded are processed according to the target decoder to obtain an image with the first resolution.

[0100] When the target resolution is the second resolution, the input features processed by the encoder can be input into the first neural network, and the first neural network is run N times to process the input features to obtain N features to be decoded corresponding to the first resolution, and then the N features to be decoded are combined into target features to be decoded corresponding to the second resolution, and finally the target features to be decoded are processed according to the target decoder to obtain an image with the second resolution.

[0101] Optionally, when the target resolution is the second resolution, the N features to be decoded can be directly input into the decoder in sequence, or the N features to be decoded can be spliced ​​into one feature in a specific manner, and the spliced ​​feature is input into the decoder as the target feature to be decoded.

[0102] Wherein, N can be determined according to the proportional relationship between the first resolution and the second resolution. As an example, when the first resolution is 512*512 and the second resolution is 1024*1024, the data volume of the second resolution is 4 times that of the image of the first resolution, so N can be set to 4.

[0103] Optionally, in order to implement the above-mentioned scheme of running the target neural network multiple times, the processor in the electronic device for running the target neural network and the target decoder can be divided into two processing units, and then the target neural network and the target decoder are loaded into the two processing units respectively, so that the processing unit that loads the target neural network can run the target neural network multiple times.

[0104] For example, in the case where an electronic device runs a target neural network and a target decoder through an embedded neural network processor (NPU), the NPU can be divided into a first processing unit and a second processing unit, the target neural network is loaded into the first processing unit of the embedded neural network processor of the electronic device, and the target decoder is loaded into the second processing unit of the embedded neural network processor, and the first processing unit and the second processing unit are different; thus, the first processing unit can be used to run the target neural network multiple times when the target resolution is the second resolution.

[0105] In some optional embodiments, depending on the target resolution, the target decoder may be a first decoder or a second decoder. For example, when the target resolution is the first resolution, the target decoder is the first decoder corresponding to the first resolution; when the target resolution is the second resolution, the target decoder is the second decoder corresponding to the second resolution.

[0106] The first decoder is used to process the input features to be decoded (or target features to be decoded) corresponding to the first resolution, and output a generated image with the first resolution.

[0107] The second decoder is used to process the input features to be decoded (or target features to be decoded) corresponding to the second resolution, and output a generated image with the second resolution.

[0108] In this case, the device's memory can store the first decoder and the second decoder at the same time. If the target resolution is determined to be the first resolution, the device determines the first decoder as the target decoder. If the target resolution is determined to be the second resolution, the device determines the second decoder as the target decoder, and then loads the target decoder into the corresponding processor, thereby processing the features to be decoded (or the target features to be decoded) according to the first decoder or the second decoder to obtain an output image of the corresponding resolution.

[0109] The advantage of applying different decoders according to different target resolutions is that generating images according to the target decoder corresponding to the target resolution is conducive to improving the quality of the obtained image and alleviating the problems of low clarity and excessive noise that may exist in the generated image.

[0110] Optionally, the input data may include input image data, and may also include input image data and input text data.

[0111] Correspondingly, the encoder of the image generation model may include an image encoder and a text encoder;

[0112] The text encoder is used to process the input text data and obtain the input text features;

[0113] The image encoder is used to process the input image data and obtain the input image features;

[0114] The input image features and the input text features constitute the input features.

[0115] The input text data may be used to describe the content that needs to be included in the generated image. For example, the input text data may be “draw a dog in the input image”, “add a house to the picture”, etc.

[0116] The text encoder may extract text features of the content indicated in the input text data as input text features. For example, the input text features may include the text features of “dog” and the text features of “house” in the above example.

[0117] Therefore, the neural network can add the image features of the corresponding content to the output features to be decoded according to the indication of the input text features, so that the decoder can obtain the generated image containing the corresponding content after decoding the features to be decoded.

[0118] In combination with the above examples, the neural network can add the image features of "dog" to the features to be decoded based on the text features of "dog", and add the image features of "house" to the features to be decoded based on the text features of "house"; the features to be decoded containing the image features of "dog" can be decoded by the decoder to obtain a generated image containing a dog, and the features to be decoded containing the image features of "house" can be decoded by the decoder to obtain a generated image containing a house.

[0119] The present application also provides an image generation device. Figure 6 , the device may include the following units.

[0120] An obtaining unit 601 is used to obtain input data for generating an image and a target resolution, where the target resolution is used to indicate the resolution of the generated image, and the target resolution is a first resolution or a second resolution;

[0121] The processing unit 602 is used to determine the processing mode of the image generation model corresponding to the target resolution based on the target resolution, and use the image generation model to process the input data according to the corresponding processing mode of the image generation model to obtain a generated image with the target resolution, wherein the processing modes of the image generation model corresponding to different resolutions are different.

[0122] Optionally, the image generation model includes a first image generation model corresponding to the first resolution, and a second image generation model corresponding to the second resolution, the first image generation model includes a first neural network and a first decoder, and the second image generation model includes a second neural network and a second decoder;

[0123] When the processing unit 602 processes the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with a target resolution, it can be used to:

[0124] Determine a target image generation model corresponding to the target resolution, where the target image generation model is the first image generation model or the second image generation model;

[0125] The input data is processed according to the neural network and decoder of the target image generation model to obtain a generated image with the target resolution.

[0126] Optionally, the image generation model includes a target neural network and a target decoder, and the target neural network is a first neural network corresponding to the first resolution or a second neural network corresponding to the second resolution;

[0127] When the processing unit 602 processes the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with a target resolution, it can be used to:

[0128] In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target features to be decoded corresponding to the target resolution. In different processing modes, the processing flow of processing the input data according to the target neural network is different;

[0129] The target features to be decoded are processed according to the target decoder to obtain a generated image with a target resolution.

[0130] Optionally, the target neural network is a second neural network corresponding to the second resolution, and the first resolution is smaller than the second resolution;

[0131] When the processing unit 602 processes the input data according to the target neural network in the processing mode corresponding to the target resolution to obtain the target to-be-decoded features corresponding to the target resolution, it can be used to:

[0132] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are processed to adjust the resolution to obtain target features to be decoded corresponding to the first resolution;

[0133] When the target resolution is the second resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are used as target features to be decoded.

[0134] Optionally, the target neural network is a first neural network corresponding to a first resolution, and the first resolution is smaller than the second resolution;

[0135] When the processing unit 602 processes the input data according to the target neural network in the processing mode corresponding to the target resolution to obtain the target to-be-decoded features corresponding to the target resolution, it can be used to:

[0136] When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the first resolution, and the features to be decoded are used as target features to be decoded;

[0137] When the target resolution is the second resolution, multiple runs of the target neural network processing input data are performed to obtain multiple features to be decoded corresponding to the first resolution; and a target feature to be decoded corresponding to the second resolution composed of the multiple features to be decoded is obtained.

[0138] Optionally, the target neural network is loaded into a first processing unit 602 of an embedded neural network processor of the electronic device, and the target decoder is loaded into a second processing unit 602 of the embedded neural network processor, and the first processing unit 602 and the second processing unit 602 are different;

[0139] The first processing unit 602 is used to run the target neural network multiple times when the target resolution is the second resolution.

[0140] Optionally, when the target resolution is a first resolution, the target decoder is a first decoder corresponding to the first resolution;

[0141] When the target resolution is the second resolution, the target decoder is a second decoder corresponding to the second resolution.

[0142] Optionally, the image processing model further includes an encoder, which is used to process input data to obtain input features, and the input features are used to input the target neural network;

[0143] The encoder is loaded into the central processor of the electronic device.

[0144] Optionally, the input data includes input image data and input text data;

[0145] The encoder includes an image encoder and a text encoder;

[0146] The text encoder is used to process the input text data and obtain the input text features;

[0147] The image encoder is used to process the input image data and obtain the input image features;

[0148] The input image features and the input text features constitute the input features.

[0149] The working principle of the image generating device of this embodiment can be found in the relevant steps of the image generating method provided in any embodiment of the present application, and will not be described in detail again.

[0150] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0151] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0152] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the relevant technology in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0153] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0154] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating an image, comprising: Obtaining input data and a target resolution for generating an image, wherein the target resolution is used to indicate a resolution of the generated image, and the target resolution is a first resolution or a second resolution; Based on the target resolution, a processing mode of an image generation model corresponding to the target resolution is determined; according to the corresponding processing mode of the image generation model, the input data is processed using the image generation model to obtain a generated image with the target resolution, wherein different processing modes of the image generation model correspond to different resolutions.

2. The method according to claim 1, wherein the image generation model comprises a first image generation model corresponding to the first resolution, and a second image generation model corresponding to the second resolution, the first image generation model comprises a first neural network and a first decoder, and the second image generation model comprises a second neural network and a second decoder; The step of processing the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution includes: Determine a target image generation model corresponding to the target resolution, wherein the target image generation model is the first image generation model or the second image generation model; The input data is processed according to the neural network and decoder of the target image generation model to obtain a generated image with the target resolution.

3. The method according to claim 1, wherein the image generation model comprises a target neural network and a target decoder, wherein the target neural network is a first neural network corresponding to the first resolution or a second neural network corresponding to the second resolution; The step of processing the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution includes: In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target features to be decoded corresponding to the target resolution. In different processing modes, the processing flow of processing the input data according to the target neural network is different; The target to-be-decoded features are processed according to the target decoder to obtain a generated image with the target resolution.

4. The method according to claim 3, wherein the target neural network is a second neural network corresponding to the second resolution, and the first resolution is smaller than the second resolution; In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target to-be-decoded features corresponding to the target resolution, including: When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are processed to adjust the resolution to obtain target features to be decoded corresponding to the first resolution; When the target resolution is the second resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the second resolution, and the features to be decoded are used as the target features to be decoded.

5. The method according to claim 3, wherein the target neural network is a first neural network corresponding to the first resolution, and the first resolution is smaller than the second resolution; In the processing mode corresponding to the target resolution, the input data is processed according to the target neural network to obtain the target to-be-decoded features corresponding to the target resolution, including: When the target resolution is the first resolution, the input data is processed according to the target neural network to obtain features to be decoded corresponding to the first resolution, and the features to be decoded are used as the target features to be decoded; When the target resolution is the second resolution, processing the input data multiple times according to the target neural network to obtain a plurality of features to be decoded corresponding to the first resolution; A target to-be-decoded feature corresponding to the second resolution is obtained, which is composed of a plurality of the to-be-decoded features.

6. The method according to claim 5, wherein the target neural network is loaded into a first processing unit of an embedded neural network processor of the electronic device, and the target decoder is loaded into a second processing unit of the embedded neural network processor, and the first processing unit and the second processing unit are different; The first processing unit is used to run the target neural network multiple times when the target resolution is the second resolution.

7. The method according to claim 3, when the target resolution is a first resolution, the target decoder is a first decoder corresponding to the first resolution; When the target resolution is a second resolution, the target decoder is a second decoder corresponding to the second resolution.

8. The method according to claim 3, wherein the image processing model further comprises an encoder, wherein the encoder is used to process input data to obtain input features, and the input features are used to input into the target neural network; The encoder is loaded into the central processor of the electronic device.

9. The method according to claim 8, wherein the input data comprises input image data and input text data; The encoder includes an image encoder and a text encoder; The text encoder is used to process the input text data to obtain input text features; The image encoder is used to process the input image data to obtain input image features; The input image features and the input text features constitute the input features.

10. An image generating device, comprising: an obtaining unit, configured to obtain input data for generating an image and a target resolution, wherein the target resolution is used to indicate a resolution of the generated image, and the target resolution is a first resolution or a second resolution; A processing unit is used to determine a processing mode of an image generation model corresponding to the target resolution based on the target resolution, and to process the input data using the image generation model according to the processing mode of the corresponding image generation model to obtain a generated image with the target resolution, wherein different resolutions correspond to different processing modes of the image generation model.