Image generation method and apparatus, electronic device and storage medium

Abstract

Provided are an image generation method and apparatus, an electronic device, and a storage medium, where a target generated image is obtained by gradually generating a plurality of image tokens from a low resolution to a high resolution and adding the image tokens, so that information included in the image is richer, and then the image tokens are continuously evaluated in a generation process, and an ineffective part is corrected in time, to improve quality of a specific generated image.SOLUTION: The method discretizes the target text to obtain a plurality of text tokens, and obtains a resolution sequence based on the initial resolution and the target resolution. The resolution sequence includes a plurality of resolutions, and a difference between two adjacent resolutions is a preset increment. The method also includes generating an image token corresponding to each resolution based on the plurality of text tokens and the resolution sequence, and fusing all image tokens to obtain a target image corresponding to the target text.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the field of computer technology, in particular to technical fields such as computer vision, deep learning, and large-scale models, and specifically to image generation methods, apparatuses, electronic devices, and storage media, which can be applied to scenarios such as artificial intelligence-based content generation. [Background technology]

[0002] Image generation methods based on the autoregressive paradigm have been very successful, using a single target image category or text description to generate images that meet the requirements through a diffusion or autoregressive model, lowering the creative barrier and accelerating content generation.

[0003] As people's demands for image resolution and detail increase, improving the quality of generated images has become a key challenge in the development of the image generation field. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to solve one of the technical problems in the related art to some extent. [Means for solving the problem]

[0005] Therefore, the objective of the present disclosure is to provide an image generation method, apparatus, electronic device and storage medium, which gradually generates multiple image tokens from low resolution to high resolution, adds the image tokens to obtain a target generated image, making the information contained in the image richer, and further, constantly evaluates the image tokens in the generation process, and corrects the poorly performing parts in a timely manner, thereby improving the quality of the specific image generated.

[0006] According to a first aspect of the present disclosure, there is provided an image generation method, the method comprising: discretizing the target text to obtain a plurality of text tokens; Obtaining a resolution sequence based on an initial resolution and a target resolution, wherein the resolution sequence includes a plurality of resolutions, and a difference between two adjacent resolutions is a preset increment; generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence; fusing all the image tokens to obtain a target image corresponding to the target text.

[0007] According to a second aspect of the present disclosure, there is provided an image generation apparatus, the apparatus comprising: a processing module for discretizing the target text to obtain a plurality of text tokens; a determining module for obtaining a resolution sequence based on an initial resolution and a target resolution, wherein the resolution sequence includes a plurality of resolutions, and a difference between two adjacent resolutions is a preset increment; a first generation module for generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence; a second generation module for fusing all the image tokens to obtain a target image corresponding to the target text.

[0008] According to a third aspect of the present disclosure, there is provided an electronic device, the electronic device comprising: at least one processor; a memory communicatively coupled to the at least one processor, wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image generation method described in the first aspect.

[0009] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform the image generation method according to the first aspect.

[0010] According to a fifth aspect of the present disclosure, there is provided a computer program comprising computer instructions which, when executed by a processor, implement the steps of the image generation method according to the first aspect. [Effects of the Invention]

[0011] The image generating method, apparatus, electronic device, and storage medium provided by the present disclosure have the following beneficial effects:

[0012] First, the target text is discretized to obtain multiple text tokens, a resolution sequence is obtained based on the initial resolution and the target resolution, and image tokens corresponding to each resolution are generated based on the multiple text tokens and the resolution sequence, and all the image tokens are fused to obtain a target image corresponding to the target text. By generating image tokens with multiple resolutions and fusing image tokens with different resolutions to obtain a target image, the generated image can capture different features and details, have richer image information, and improve the quality of the image generation result.

[0013] It should be noted that the contents described in this section do not identify the essential or important features that define the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will be easily understood from the following description. [Brief explanation of the drawings]

[0014] The above and / or additional aspects and advantages of the present disclosure will become apparent from the following description of the following embodiments in combination with the drawings, which are used to understand the present solution and are not intended to limit the present disclosure. [Figure 1]1 is a schematic flowchart of an image generation method provided by an embodiment of the present disclosure. [Figure 2] 1 is a schematic flowchart of an image generation method provided by another embodiment of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram of an image generation process provided by an embodiment of the present disclosure. [Figure 4] 1 is a structural schematic diagram of an image generating device provided by an embodiment of the present disclosure; [Figure 5] FIG. 1 is a block diagram of an exemplary electronic device that may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015]

[0023] The following describes exemplary embodiments of the present disclosure in conjunction with the drawings. For ease of understanding, various details of the embodiments of the present disclosure are included and should be considered as illustrative. Therefore, as can be understood by those skilled in the art, various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted.

[0016] Embodiments of the present disclosure relate to artificial intelligence technology fields such as computer vision, deep learning, and large-scale models.

[0017] Artificial Intelligence, abbreviated as AI in English, is a new technological science that researches and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence.

[0018] Deep learning learns the inherent rules and representation levels of sample data, and the information gained from this learning is very useful for interpreting data such as text, images, and voice. The ultimate goal of deep learning is to enable machines to have human-like analytical learning capabilities and to identify data such as text, images, and voice.

[0019] Computer vision refers to machine vision that uses cameras and computers to identify, track, and measure targets in place of the human eye, and then performs further graphics processing and computer processing to create images suitable for human observation and device detection.

[0020] A large-scale artificial intelligence model (abbreviated as "large-scale model") refers to an artificial intelligence model with a large number of parameters built using an artificial neural network. It is typically pre-trained with a large amount of data through self-supervised or semi-supervised learning, and then its performance and capabilities are further optimized through methods such as fine-tuning instructions and human alignment. Large-scale models are characterized by a large number of parameters, large training data, and large computing resources, and are capable of solving general-purpose tasks, following human commands, and performing complex reasoning.

[0021] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of relevant users' personal information shall all comply with the provisions of relevant laws and regulations and shall not violate public order and morals.

[0022] Hereinafter, an image generating method, an apparatus, an electronic device, and a storage medium according to embodiments of the present disclosure will be described in combination with the drawings.

[0023] Here, the entity that executes the image generation method of this embodiment is an image generation device, which can be realized by software and / or hardware, and which may be located within an electronic device, which may include, but is not limited to, a terminal, a client, etc.

[0024] FIG. 1 is a schematic flowchart of an image generation method provided by one embodiment of the present disclosure.

[0025] As shown in FIG. 1, the image generating method includes the following steps S101 to S104.

[0026] In S101, the target text is discretized to obtain a plurality of text tokens.

[0027] Here, the target text is a description of the content of the target generated image, for example, “blue sky.” A text token is the smallest semantic unit in the target text, such as a word, subword, or character.

[0028] In embodiments of the present disclosure, the target text can be obtained in various ways, for example, the target text can be obtained by user input, or the target text can be text within a document or web page, and the present disclosure is not limited thereto.

[0029] In embodiments of the present disclosure, the target text can be segmented using predefined rules or neural networks to discretize it into multiple text tokens, which can be processed more efficiently by the image generation model when generating images based on the target text.

[0030] In S102, a resolution sequence is obtained based on the initial resolution and the target resolution.

[0031] The initial resolution is the lowest resolution of the image that the model can generate, and for different types of models used in actual image generation, the initial resolution may be different due to model output limitations, for example, it may be 8x8.

[0032] The target resolution is the highest resolution at which an image is generated, and the target resolution can be set according to the resolution requirements for actually generating the image, the available resource space, the model capabilities, and so on.

[0033] The resolution sequence includes multiple resolutions, and the difference between two adjacent resolutions is a preset increment.

[0034] In the embodiment of the present disclosure, the image generation model does not directly generate a single image, but converts the process of generating a single image into a process of generating images of different resolutions by gradually quantizing the residual, gradually generating multiple image tokens from low resolution to high resolution, and fusing the image tokens to obtain a target image. The increasing speed from low resolution to high resolution can be customized based on the constraints of computing resources, the requirements for image generation speed, etc. Therefore, a predetermined increment can be set, and the resolution of the image token to be generated is determined for each predetermined increment from the initial resolution until the target resolution is reached and a resolution sequence is obtained. The model can determine the resolution of the image generated at each step based on the resolution sequence.

[0035] In S103, image tokens corresponding to each resolution are generated based on the plurality of text tokens and the resolution sequence.

[0036] In an embodiment of the present disclosure, the image generation model can generate image tokens corresponding to each resolution based on a unified generation framework of visual autoregression, in which the backbone network continues to use the Transformer structure commonly used in large-scale language models.

[0037] In an embodiment of the present disclosure, multiple text tokens can be input into an image generation model, and an image encoder in the image generation model uses a gradual residual quantization method to convert the image generation process into a generation process of images of different resolutions according to the resolution sequence, generating them gradually from lower resolutions to higher resolutions, to obtain image tokens corresponding to each resolution in the resolution sequence.

[0038] Note that each image token corresponds to one local area or feature of an image (e.g., color block, edge, texture, etc.), so the image generation model maps multiple text tokens to a high-dimensional semantic vector via an encoder (e.g., Transformer), and then generates multiple image tokens with the same resolution, each corresponding to a part of the target image, based on the semantic vector.

[0039] In the embodiments of the present disclosure, in order to improve the quality of the generated image, each time an image token corresponding to a resolution is generated, the image token is quality evaluated and modified, and then the modified image token is used to determine an image token with a higher resolution, thereby ensuring that the effect of the image token corresponding to each resolution is better, and furthermore, after the image tokens are fused, the quality of the finally generated image is further improved.

[0040] In S104, all image tokens are fused to obtain a target image corresponding to the target text.

[0041] In an embodiment of the present disclosure, all image tokens corresponding to all resolutions may be normalized to have similar pixel value ranges or distributions, to avoid fusion deviations due to scale differences, and then all image tokens are accumulated to obtain a target image corresponding to the target text.

[0042] In this embodiment, the target text is first discretized to obtain multiple text tokens, a resolution sequence is obtained based on the initial resolution and the target resolution, and image tokens corresponding to each resolution are generated based on the multiple text tokens and the resolution sequence, and all the image tokens are fused to obtain a target image corresponding to the target text. By generating image tokens with multiple resolutions and fusing image tokens with different resolutions to obtain a target image, the generated image can capture different features and details, have richer image information, and improve the quality of the image generation result.

[0043] FIG. 2 is a schematic flowchart of an image generation method provided by another embodiment of the present disclosure.

[0044] As shown in FIG. 2, the image generating method includes the following steps S201 to S206.

[0045] In S201, the target text is discretized to obtain a plurality of text tokens.

[0046] In S202, a resolution sequence is obtained based on the initial resolution and the target resolution.

[0047] For the explanation of S201 and S202, please refer to the above embodiment, and a detailed explanation will be omitted here.

[0048] In S203, a plurality of first image tokens are generated based on the plurality of text tokens and the initial resolution.

[0049] In an embodiment of the present disclosure, multiple text tokens can be input to an image generation model based on a visual autoregressive integrated generative framework, whereby an image encoder in the image generation model can generate multiple first image tokens with a resolution of an initial resolution for the multiple text tokens, and each first image token can correspond to one or more text dimensions.

[0050] In the present disclosure, in order to improve the quality of the generated image, an image token corresponding to one resolution can be generated, and then quality evaluation can be performed on the image token to modify it, and then the modified image token can be used to determine an image token with a higher resolution, thereby ensuring that the effect of the image token corresponding to each resolution is better, and therefore, after the image tokens are fused, the quality of the finally generated image is higher.

[0051] Optionally, a quality score for each first image token may be determined first, and then, if the quality score for any one of the first image tokens is below a score threshold, a new first image token is generated based on the one of the first image tokens, the initial resolution, and a text token corresponding to the one of the first image tokens.

[0052] The score threshold can be customized according to actual needs and is not limited in the present disclosure. The higher the requirement for the quality of the generated image, the lower the score threshold should be set; however, since it is necessary to consider whether the computational resources are sufficient, it is advisable to avoid setting the score threshold too small.

[0053] In an embodiment of the present disclosure, the first image tokens can be evaluated using an external tool to obtain a quality score for each first image token, and then each quality score can be compared with a score threshold. All first image tokens whose corresponding quality scores are lower than the score threshold can be determined to have poor mass evaluation results, and thus can be re-generated based on the generation results of the current resolution to improve the generation quality, i.e., the poor-quality first image tokens and corresponding text tokens are input into the model again to obtain new first image tokens whose resolution is the initial resolution, which are output by the model.

[0054] In the embodiments of the present disclosure, after generating an image token corresponding to one resolution, quality evaluation is performed on the image token, and image tokens with poor quality are corrected, thereby ensuring that the effect of the image tokens corresponding to each resolution is better, and therefore, after the image tokens are fused, the quality of the image finally generated is also higher.

[0055] In S204, each first image token is interpolated to obtain one second image token based on the next resolution adjacent to the initial resolution in the resolution sequence.

[0056] In the embodiment of the present disclosure, after obtaining a first image token, the next resolution adjacent to the initial resolution in the resolution sequence can be determined as the resolution of the next image token that needs to be generated. If necessary, any one of interpolation methods such as linear interpolation can be selected to interpolate one first image token to an image token having the next resolution adjacent to the initial resolution to obtain a second image token.

[0057] In the present disclosure, each first image token can be interpolated to obtain a corresponding second image token, and multiple interpolation methods can be used for the first image token to obtain multiple second image tokens, thereby improving the richness of graphic information after the image tokens are fused.

[0058] In the present disclosure, in order to improve the quality of the generated image, an image token corresponding to one resolution can be generated, and then quality evaluation can be performed on the image token to modify it, and then the modified image token can be used to determine an image token with a higher resolution, thereby ensuring that the effect of the image token corresponding to each resolution is better, and therefore, after the image tokens are fused, the quality of the finally generated image is higher.

[0059] Optionally, a quality score of the second image token may be determined, and if the quality score is below a score threshold, a new second image token is generated based on the first image token and the second image token.

[0060] It should be noted that the process for determining the quality scores of the second image tokens may be similar to the process for determining the quality scores of the first image tokens, and the score threshold settings may be similar or different.

[0061] In an embodiment of the present disclosure, a second image token having a quality score lower than a score threshold can be added to the corresponding first image token, and a Transformer network in the image generation model generates a new second image token, which is generated based on the first image token and has the same resolution as the conventional second image token but different content.

[0062] In the embodiments of the present disclosure, after generating an image token corresponding to one resolution, the quality of the image token is evaluated, and poor-quality image tokens are corrected, thereby ensuring that the effect of the image tokens corresponding to each resolution is better, and therefore, after the image tokens are fused, the quality of the final generated image is also higher.

[0063] In addition, in the embodiments of the present disclosure, in addition to using external tools to score the quality of image tokens, the token quality of each generated image can also be evaluated based on the change in pixel values ​​during the generation process, automatically identifying areas that may need optimization, and correcting the areas to be optimized once at the current resolution.

[0064] Optionally, first, the difference in pixel values ​​between the second image token and its corresponding first image token is calculated.

[0065] In an embodiment of the present disclosure, first, the second image token and its corresponding first image token are perfectly aligned in space by cutting or aligning, and then all pixel positions are traversed and the difference between the pixel value of the second image token and the pixel value of the first image token is calculated at each pixel position to obtain the pixel value difference between the second image token and its corresponding first image token, and the pixel value difference may be recorded in a matrix format.

[0066] A target area to be modified within the second image token is then determined based on the pixel value difference and a difference threshold.

[0067] Here, the difference threshold is used to determine the critical value for which the change in pixel value is small when changing from the first image token to the second image token, and can be set as needed, and the difference threshold may be different when evaluating images of different resolutions.

[0068] In an embodiment of the present disclosure, after obtaining the pixel value difference between the second image token and its corresponding first image token, the relationship between the magnitude of the pixel value difference at each pixel position and the difference threshold is traversed, and the pixel position where the pixel value difference is smaller than the difference threshold is determined as the target area to be corrected within the second image token, thereby ensuring that when generating image tokens with higher resolution at each step, their content is significantly more refined than that of the image tokens of the previous resolution.

[0069] The second image token can then be modified based on the first image token and the target area to obtain a new second image token.

[0070] In the embodiment of the present disclosure, after determining the target area to be modified in the second image token, the target area of ​​the second image token can be modified based on the first image token to obtain a new second image token, which can realize the need to modify multiple target areas in the image, improve the image quality more accurately, and ensure that the image token effect corresponding to each resolution is better, so that the quality of the final image generated after fusing the image tokens is also higher.

[0071] It should be noted that in the embodiments of the present disclosure, different difference thresholds can be set when evaluating images of different resolutions.

[0072] Optionally, if the resolution of the second image token is below a resolution threshold, areas in which the difference between corresponding pixel values ​​in the second image token is below a first difference threshold may be determined as target areas.

[0073] Alternatively, if the resolution of the second image token is equal to or greater than the resolution threshold, areas in which the difference between corresponding pixel values ​​in the second image token is less than a second difference threshold may be determined as target areas.

[0074] Here, the second difference threshold is lower than the first difference threshold.

[0075] In the embodiments of the present disclosure, in the early stage with a relatively low resolution, emphasis is placed on the contours of the image, and a large change in pixel values ​​between two generated results is required, so a larger difference threshold can be set to evaluate the quality of the generated image tokens. In contrast, in the later stage with a relatively high resolution, emphasis is placed on the details of the image, so a smaller change threshold can be set to evaluate the quality of the generated image tokens, thereby avoiding the problem of inability to fuse identical positions when fusing images with different resolutions later. Therefore, different evaluation thresholds are set when evaluating the quality of images with different resolutions, improving the reliability of image correction.

[0076] In S205, based on the resolution sequence and the second image token, an interpolation operation is repeatedly performed on the image token until a target image token corresponding to the target resolution is obtained.

[0077] In an embodiment of the present disclosure, after obtaining the second image token, the second image token may be interpolated again, so that the resolution of the new image token after interpolation is the next resolution in the resolution sequence, and this process is repeated multiple times until a target image token corresponding to the target resolution is obtained. Each time an image token of a new resolution is obtained, quality assessment and correction can be performed, and the quality assessment and correction method is the same as the quality assessment and correction method for the second image token.

[0078] In the embodiments of the present disclosure, if the quality of any one generated image token is good, the image token can be directly used for interpolation to obtain the next image token with a higher resolution. Conversely, if the quality of any generated image token is poor, the image token can be first modified and then the modified image token can be used for interpolation to obtain the next image token with a higher resolution.

[0079] In S206, all image tokens are fused to obtain a target image corresponding to the target text.

[0080] For a detailed explanation of S206, please refer to the above embodiment, and a detailed explanation will be omitted here.

[0081] In this embodiment, by gradually interpolating low-resolution image tokens to obtain high-resolution image tokens, high-frequency details can be restored and blurring can be reduced, and by gradually optimizing the image content, the rationality of the structure of each step can be ensured, further improving the quality of the generated image.

[0082] The image generation method based on the visual autoregressive paradigm provided by the present disclosure improves image quality without relying on additional image editing models, can directly integrate quality improvement functions into the image generation framework, and is faster and less computationally expensive due to the shorter length of the required generated sequences.

[0083] The image generation process is described using FIG. 3 as an example, which is a schematic diagram of the image generation process provided by an embodiment of the present disclosure.

[0084] In Figure 3, BOS (Begin Of Sentence) is the start symbol of a Text Token, which is represented by the letter T in Figure 3. BOI (Begin Of Image) is the start symbol of an Image Token, S1 represents an Image Token of the first resolution (Scale), S2 represents an Image Token of the second resolution (Scale), and so on.

[0085] First, multiple text tokens T are passed through a Transformer network to generate an image token S1 at the first resolution. Then, multiple image tokens S2 at the second resolution can be obtained by performing different interpolation processes on S1. The model then determines that the quality of S2 is poor and that the results for the current scale need to be corrected and regenerated. S1 and S2 are added (shown as S1' in Figure 3) to obtain a corrected result using the Transformer, denoted as S2'. Subsequent results can be used to generate S3 based on Scale2' until the target resolution is reached. After generating results for all resolutions, all results are accumulated and input into the image decoder to obtain the final image. If the image token corresponding to the target resolution is S3 and the quality of S3 is good, all S1, S2', and S3 can be added together to obtain the final generated image.

[0086] FIG. 4 is a structural schematic diagram of an image generating device provided by one embodiment of the present disclosure.

[0087] As shown in FIG. 4, the image generating device 40 includes: a processing module 401 for discretizing the target text to obtain a plurality of text tokens; A determination module 402 for obtaining a resolution sequence based on an initial resolution and a target resolution, where the resolution sequence includes multiple resolutions, and the difference between two adjacent resolutions is a preset increment; a first generation module 403 for generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence; a second generation module 404 for fusing all the image tokens to obtain a target image corresponding to the target text.

[0088] In some embodiments, the first generating module 403 specifically: generating a plurality of first image tokens based on the plurality of text tokens and the initial resolution; interpolating each first image token based on a next resolution adjacent to the initial resolution in the resolution sequence to obtain one second image token; Based on the resolution sequence and the second image token, an interpolation operation is repeatedly performed on the image token until a target image token corresponding to the target resolution is obtained.

[0089] In some embodiments, the first generating module 403 further determining a quality score for each first image token; If the quality score of any one of the first image tokens is lower than the score threshold, a new first image token is generated based on the one of the first image tokens, the initial resolution, and a text token corresponding to the one of the first image tokens.

[0090] In some embodiments, the first generating module 403 further comprises: determining a quality score for the second image token; If the quality score is lower than the score threshold, a new second image token is generated based on the first image token and the second image token.

[0091] In some embodiments, the first generating module 403 further comprises: calculating a difference in pixel values ​​between the second image token and its corresponding first image token; determining a target area to be modified within the second image token based on the pixel value difference and a difference threshold; The second image token is modified based on the first image token and the target area to obtain a new second image token.

[0092] In some embodiments, the first generating module 403 further comprises: if the resolution of the second image token is below the resolution threshold, determining as the target area an area in which the difference in corresponding pixel values ​​in the second image token is below the first difference threshold; or If the resolution of the second image token is equal to or greater than the resolution threshold, an area in which the difference between corresponding pixel values ​​in the second image token is lower than a second difference threshold is determined as the target area, where the second difference threshold is lower than the first difference threshold.

[0093] The above description of the image generating method is also applicable to the image generating apparatus of this embodiment, and a detailed description thereof will be omitted here.

[0094] In this embodiment, the target text is first discretized to obtain multiple text tokens, a resolution sequence is obtained based on the initial resolution and the target resolution, and image tokens corresponding to each resolution are generated based on the multiple text tokens and the resolution sequence, and all the image tokens are fused to obtain a target image corresponding to the target text. By generating image tokens with multiple resolutions and fusing image tokens with different resolutions to obtain a target image, the generated image can capture different features and details, have richer image information, and improve the quality of the image generation result.

[0095] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program.

[0096] 5 illustrates an example block diagram of an electronic device 500 in which example embodiments of the present disclosure can be implemented. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions illustrated herein are merely examples and are not intended to limit the practice of the present disclosure as described and / or claimed herein.

[0097] 5, device 500 includes a computing unit 501 that performs various appropriate operations and processes in accordance with a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may store various programs and data necessary for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are connected to one another via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] Multiple components of device 500 are connected to an I / O interface 505, including an input unit 506 such as a keyboard, a mouse, etc., an output unit 507 such as various types of displays, speakers, etc., a storage unit 508 such as a magnetic disk, an optical disk, etc., and a communication unit 509 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 enables device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various types of telecommunications networks.

[0099] The computing unit 501 may be various general-purpose and / or specialized processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 executes each of the methods and processes described above, e.g., the image generation method. For example, in some embodiments, the image generation method may be implemented as a computer software program tangibly embodied in a machine-readable medium such as the storage unit 508. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, it may perform one or more steps of the image generation method described above. Alternatively, in other embodiments, the computing unit 501 may be configured in any other suitable manner (eg, via firmware) to perform the image generation method.

[0100] Various embodiments of the systems and techniques herein described above may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being embodied in one or more computer programs that can be executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor, and that can receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0101] Program code for carrying out the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus such that, when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine, partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or can store a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a suitable combination of any of the above. More specific examples of machine-readable storage media include one or more line-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or a suitable combination of any of the above.

[0103] To provide for user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and can receive input from the user in any form (including acoustic, speech, and tactile input).

[0104] The systems and techniques described herein can be implemented on a computing system including a back-end component (e.g., a data server), or a computing system including a middleware component (e.g., an application server), or a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such back-end, middleware, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0105] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host. This is a host product in a cloud computing service system that overcomes the shortcomings of traditional physical hosts and VPS services (also known as "Virtual Private Servers," or "VPSs"), such as difficulty in management and poor business scalability. The server may be a server in a distributed system or a server incorporating blockchain.

[0106] It should be understood that steps can be rearranged, added, or deleted in the various processes described above. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, but this specification is not limited thereto, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved.

[0107] Additionally, terms such as "first," "second," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or the number of technical features being indicated. Accordingly, a feature qualified by "first" or "second" may explicitly or implicitly include at least one of the feature. In describing this disclosure, unless expressly and specifically limited, "plurality" means at least two, e.g., two, three, etc. In describing this disclosure, the terms "if" and "if-" are used to mean "when" or "when" or "responsive to determining" or "in the case of."

[0108] The above specific embodiments do not limit the scope of protection of the present disclosure. It should be understood that those skilled in the art can make various modifications, combinations, subcombinations, and substitutions according to design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. 1. A method for generating an image, comprising: discretizing the target text to obtain a plurality of text tokens; obtaining a resolution sequence based on an initial resolution and a target resolution, wherein the resolution sequence includes a plurality of resolutions, and a difference between two adjacent resolutions is a preset increment; generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence; fusing all the image tokens to obtain a target image corresponding to the target text; An image generating method comprising:

2. generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence, generating a plurality of first image tokens based on the plurality of text tokens and the initial resolution; interpolating each of the first image tokens based on a next resolution adjacent to the initial resolution in the resolution sequence to obtain one second image token; and repeatedly performing an interpolation operation on the image tokens based on the resolution sequence and the second image tokens until a target image token corresponding to the target resolution is obtained.

2. The method of claim 1 .

3. After the step of generating a plurality of first image tokens based on the plurality of text tokens and the initial resolution, determining a quality score for each of the first image tokens; If the quality score of any one of the first image tokens is lower than a score threshold, generating a new first image token based on the one of the first image tokens, the initial resolution, and a text token corresponding to the one of the first image tokens.

3. The method of claim 2.

4. After the step of interpolating each of the first image tokens to obtain one second image token, determining a quality score for the second image token; if the quality score is below a score threshold, generating a new second image token based on the first image token and the second image token; 3. The method of claim 2.

5. After the step of interpolating each of the first image tokens to obtain one second image token, calculating a difference in pixel values ​​between said second image token and its corresponding first image token; determining a target area to modify within the second image token based on the pixel value difference and a difference threshold; and modifying the second image token based on the first image token and the target area to obtain a new second image token.

3. The method of claim 2.

6. determining a target area to modify within the second image token based on the pixel value difference and a difference threshold, if the resolution of said second image token is below a resolution threshold, determining as target areas those areas in which the difference in corresponding pixel values ​​within said second image token is below a first difference threshold; or if the resolution of the second image token is equal to or greater than a resolution threshold, determining as target areas those areas in which the difference in corresponding pixel values ​​in the second image token is lower than a second difference threshold, the second difference threshold being lower than the first difference threshold; 6. The method of claim 5.

7. 1. An image generation device, comprising: a processing module for discretizing the target text to obtain a plurality of text tokens; a determination module for obtaining a resolution sequence based on an initial resolution and a target resolution, wherein the resolution sequence includes a plurality of resolutions, and a difference between two adjacent resolutions is a preset increment; a first generation module for generating image tokens corresponding to each resolution based on the plurality of text tokens and the resolution sequence; a second generation module for fusing all the image tokens to obtain a target image corresponding to the target text; An image generating device characterized by:

8. 1. An electronic device comprising: at least one processor; a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform the image generation method according to any one of claims 1 to 6; An electronic device characterized by:

9. A non-transitory computer readable storage medium having stored thereon computer instructions, the computer instructions causing a computer to perform the image generation method of any one of claims 1 to 6. A non-transitory computer-readable storage medium comprising:

10. A computer program, which when executed by a processor, implements the steps of the image generation method according to any one of claims 1 to 6. A computer program characterized by:

Images