Method and device for intelligent computing center cloud platform to cache and accelerate diffusion model to generate image through computing power calculation result

Abstract

The invention provides a method and a device for an intelligent computing center cloud platform to cache an accelerated diffusion model to generate an image through a computing power computing result, and relates to the technical field of intelligent computing centers, intelligent computing centers and computing power infrastructures, the method comprises the following steps: S1, receiving an image generation task sent by a client, the image generation task is used for indicating generation of a first image; s2, inputting the image generation task into a pre-trained diffusion model, wherein the diffusion model comprises a plurality of neural network layers connected in sequence; s3, the image generation task is executed based on each neural network layer in a plurality of sequentially connected neural network layers, the first image is obtained, and when the residual value between the jth neural network layer and the (j-1) th neural network layer is smaller than a preset threshold value, the first image is generated. And the jth neural network layer performs the image generation task based on a preset computing power calculation result cache. According to the invention, the image generation speed can be improved.

Description

Technical Field

[0001] This invention relates to the technical fields of intelligent computing centers, smart computing centers, smart computing clouds and computing infrastructure, specifically to a method and apparatus for accelerating the generation of images from a diffusion model through caching of computing results on an intelligent computing center cloud platform. Background Technology

[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "smart computing centers" have emerged.

[0003] An "intelligent computing center" refers to a facility that provides the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models) by utilizing large-scale heterogeneous computing resources, including general-purpose and intelligent computing power. Intelligent computing centers encompass facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0004] "Intelligent computing center" includes, but is not limited to, "intelligent computing center".

[0005] "Intelligent computing center" or artificial intelligence computing center is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting artificial intelligence computing architecture.

[0006] "Computing power" is the core of "intelligent computing center" and "smart computing center". It is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to perform a certain computing requirement. It is the computing power to achieve the target output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0007] Currently, AI-generated images are becoming increasingly popular, with a rapidly growing demand. However, the efficiency of AI-generated content (AIGC) images cannot keep up with this rapidly growing demand. Furthermore, as the resolution of generated images increases, inference time also increases, leading to continuously rising costs. It is evident that existing technologies suffer from both low efficiency and high cost in handling image generation tasks. Summary of the Invention

[0008] This invention provides a method and apparatus for accelerating image generation by a diffusion model through caching computing results on an intelligent computing center cloud platform, in order to solve the problems of low efficiency and low computing power utilization in image generation tasks in the prior art.

[0009] To solve the above problems, the present invention is implemented as follows: In a first aspect, embodiments of the present invention provide a method for accelerating image generation by a diffusion model through caching of computing power calculation results on an intelligent computing center cloud platform, comprising: Step S1: Receive an image generation task sent by the client, wherein the image generation task is used to instruct the generation of a first image; Step S2: Input the image generation task into a pre-trained diffusion model, the diffusion model comprising multiple sequentially connected neural network layers; Step S3: Execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is the computing power calculation result cache of the neural network layers before the j-th neural network layer, where j is an integer greater than 1.

[0010] In one embodiment, step S3 includes: Step S31: According to a preset diffusion step size, the denoising iteration process of the image generation task is executed for each of the plurality of sequentially connected neural network layers to obtain image rendering information. The preset diffusion step size is used to indicate the number of steps of the denoising iteration process of the image generation task executed by each neural network layer. The image rendering information includes: the output result of the last neural network layer among the plurality of sequentially connected neural network layers after the last step of the denoising iteration process in the preset diffusion step size is completed. Step S32: Based on the multiple sequentially connected neural network layers, the image rendering information is processed in a context-parallel manner to obtain the first image.

[0011] In one embodiment, during any one of the denoising iterations of the preset diffusion step size: The first neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task and obtain the first output result; The i-th neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, where i is an integer greater than 1.

[0012] In one embodiment, the i-th neural network layer among the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, including: If the residual value between the i-th neural network layer and the (i-1)-th neural network layer is less than a preset threshold, the calculation of the i-th neural network layer is skipped. Based on the calculation result cache of the preset neural network layer and the (i-1)-th output result, the image generation task is executed to obtain the i-th output result. Wherein, without skipping the calculation of the (i-1)th neural network layer, the preset neural network layer is the (i-1)th neural network layer; If the calculation of the (i-1)th neural network layer is skipped, the neural network layers before the i-th neural network layer are traversed forward until the next neural network layer whose calculation is not skipped is found. The preset neural network layer is the next neural network layer whose calculation is not skipped.

[0013] In one embodiment, step S32 includes: Step S321: Determine multiple sub-tasks based on the image rendering information. The first image includes multiple sub-layers. The multiple sub-tasks correspond one-to-one with the multiple sub-layers. The sub-tasks are used to indicate the generation of the corresponding sub-layers. Step S322: Execute the multiple sub-tasks in parallel using the multiple sequentially connected neural network layers to obtain the multiple sub-layers; Step S323: Merge the multiple sub-layers to obtain the first image.

[0014] In one embodiment, step S321 includes: Step S3211: Obtain the model parameters of the diffusion model, wherein the model parameters are used to indicate the pixel size of the image material, and the image material is used to train the image generation model; Step S3212: Determine multiple sub-tasks based on the model parameters to obtain the multiple sub-tasks. The pixel size of the multiple sub-layers corresponding to the multiple sub-tasks is the same as that of the model parameters.

[0015] Secondly, embodiments of the present invention also provide an apparatus for accelerating the generation of images from a diffusion model through caching of computing power calculation results on an intelligent computing center cloud platform, comprising: The receiving module is used to receive an image generation task sent by the client, wherein the image generation task is used to instruct the generation of a first image; An input module is used to input the image generation task into a pre-trained diffusion model, the diffusion model comprising multiple sequentially connected neural network layers; An execution module is configured to execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Specifically, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is a cache of computing power calculation results from the neural network layers preceding the j-th neural network layer, where j is an integer greater than 1.

[0016] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the method for accelerating the generation of images by a diffusion model through caching computing results by an intelligent computing center cloud platform as described in the first aspect above.

[0017] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for accelerating the generation of images by a diffusion model through caching computing results by an intelligent computing center cloud platform as described in the first aspect above.

[0018] Fifthly, the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the method for accelerating the generation of images by a diffusion model through a computing power-based cloud platform by caching computing results as described in the first aspect above.

[0019] In this embodiment of the invention, an image generation task sent by a client is received, the image generation task being used to instruct the generation of a first image; the image generation task is input into a pre-trained diffusion model, the diffusion model including multiple sequentially connected neural network layers; the image generation task is executed based on each of the multiple sequentially connected neural network layers to obtain the first image, wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache, the preset computing power calculation result cache being the computing power calculation result cache of the neural network layers preceding the j-th neural network layer, where j is an integer greater than 1. When the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer performs the image generation task based on the cached result of the preset computing power calculation. This allows the calculation process of the j-th neural network layer to be skipped, thereby improving the efficiency of image generation tasks and increasing the speed of image generation. While improving the efficiency of image generation to meet the needs, it can also ensure the quality of the generated images to meet user requirements. Furthermore, due to the improved efficiency of image generation, the time required for image generation can be reduced, thereby lowering the cost of image generation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a method for accelerating the generation of images by a diffusion model through caching computing results on an intelligent computing center cloud platform, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of image generation based on a diffusion model in related technologies; Figure 3 This is a schematic diagram of an image generation based on a diffusion model provided in an embodiment of the present invention; Figure 4 This is a structural diagram of a device for accelerating the generation of images by a diffusion model through caching computing results in an intelligent computing center cloud platform, as provided in an embodiment of the present invention. Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The “computing power” mentioned in this invention refers to: the ability of computer equipment or computing / data center to process information; the ability of computer hardware and software to work together to perform a certain computing requirement; the computing power to achieve the target result output by processing information data; and a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, mainly providing services to society through computing power infrastructure.

[0024] The "computational power" (CP) described in this invention refers to the ability of a data center server to process data and output results. It is a comprehensive indicator of a data center's computing power, encompassing general computing power, supercomputing power, and intelligent computing power. The commonly used unit of measurement is floating-point operations per second (FLOPS, 1 EFLOPS = 10^18 FLOPS), with higher values ​​indicating stronger overall computing power. It is estimated that 1 EFLOPS is approximately the computing power output of 5 Tianhe-2A supercomputers, 500,000 mainstream server CPUs, or 2 million mainstream laptops. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0025] The "Network Power" (NP) mentioned in this invention refers to the performance of data transmission capability of computing facilities, which includes comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, and involves network transmission within and between data centers. It is a comprehensive indicator for measuring network transmission scheduling capability.

[0026] The "Storage Power" (SP) described in this invention refers to the comprehensive capabilities of a data center in four aspects: data storage capacity, performance, security and reliability, and green and low-carbon operation. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and internal storage devices within servers. The commonly used unit of measurement for storage capacity is exabytes (EB, 1EB = 2^60 bytes), while the commonly used unit of measurement for performance is the number of read / write operations per second (IOPS / TB). Disaster recovery ratio is an important indicator of security and reliability.

[0027] The "computing infrastructure" mentioned in this invention refers to a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, enabling centralized computing, storage, transmission, and application of information.

[0028] The "new information infrastructure" mentioned in this invention refers to network infrastructure such as 5G networks, fiber optic broadband networks, backbone networks, international communication networks, and satellite internet; computing infrastructure such as data centers, general computing centers, intelligent computing centers, and supercomputing centers; and new technology facilities such as artificial intelligence, blockchain, and quantum computing.

[0029] The “computing power” mentioned in this invention includes: general computing power, intelligent computing power, and supercomputing power.

[0030] The "general computing power" mentioned in this invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0031] The "intelligent computing power" mentioned in this invention refers to: a computing platform deployed on a large scale based on dedicated chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and ASIC (Application Specific Integrated Circuit) for various artificial intelligence innovative applications, such as natural language processing and machine vision.

[0032] The “supercomputing power” mentioned in this invention refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for computing in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0033] The "intelligent computing center" described in this invention refers to a facility that, through the use of large-scale heterogeneous computing resources, including general-purpose computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.), primarily provides the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models). The intelligent computing center encompasses facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0034] The "intelligent computing center" mentioned in this invention includes, but is not limited to, "smart computing center".

[0035] The "intelligent computing center" mentioned in this invention, also known as an artificial intelligence computing center, is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting an artificial intelligence computing architecture.

[0036] The "computing center" mentioned in this invention refers to a facility that is mainly composed of infrastructure such as wind, thermal, hydro, and electricity, and IT hardware and software equipment, and has computing power, carrying capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.

[0037] The "supercomputing center" mentioned in this invention refers to a supercomputing data center, which is a data center based on supercomputers or large-scale computing clusters. It can provide large-scale computing, storage and network services and is widely used in aerospace, defense, oil exploration, climate modeling and genome sequencing and other application scenarios.

[0038] The “computing resources” mentioned in this invention refer to the technologies and facilities required for the development of the digital society that have the ability to compute, transmit, store and apply information, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guaranteeing resources such as wind, fire, water and electricity.

[0039] The "intelligent computing center cloud platform" described in this invention refers to a cloud computing platform that integrates hardware and software resources of an intelligent computing center.

[0040] The “computing power utilization rate” mentioned in this invention refers to the efficiency of actual utilization of computing power resources when executing tasks, and is an important indicator for evaluating system performance and resource optimization.

[0041] The "computing power calculation result cache" mentioned in this invention refers to the calculation result of floating-point operation corresponding to any neural network layer in the diffusion model, which is pre-cached in the video memory.

[0042] In related technologies, AI-generated images are gradually becoming popular. However, the efficiency of AIGC-generated images is generally low, and the inference time increases with the resolution, resulting in low efficiency and low utilization of computing power in image generation tasks. In this embodiment of the invention, when the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer performs the image generation task based on the cached result of the preset computing power calculation. This allows the calculation process of the j-th neural network layer to be skipped, thereby improving the efficiency of image generation tasks and increasing the speed of image generation. While improving the efficiency of image generation to meet the needs, it can also ensure the image quality meets user requirements. Furthermore, due to the improved efficiency, the time required for image generation can be reduced, thereby lowering the cost of image generation.

[0043] For details, please see Figure 1 , Figure 1 This is a flowchart illustrating a method for accelerating image generation from a diffusion model using a cloud platform with intelligent computing center capabilities through caching computing results, as provided in an embodiment of the present invention. Figure 1 As shown, it includes the following steps: Step S1: Receive an image generation task sent by the client, the image generation task being used to instruct the generation of a first image.

[0044] In this step, the image generation task can be an instruction submitted by the user through a client (such as a webpage, APP, etc.) to generate an image, which can include specific text descriptions or parameters, such as "generate an image of a snow-capped mountain under a starry sky".

[0045] The first image mentioned above can be the target image that the user ultimately needs, which may be an image corresponding to a high-resolution or complex scene.

[0046] Step S2: Input the image generation task into a pre-trained diffusion model, which includes multiple neural network layers connected in sequence.

[0047] In this step, the pre-trained diffusion model can be a pre-trained generative artificial intelligence model that can generate images based on input instructions by gradually adding and removing noise.

[0048] The aforementioned neural network layers can be multiple layers in a diffusion model. Each layer transforms the input data and can specifically be a convolutional layer, attention layer, fully connected layer, etc. These neural network layers are connected sequentially in a preset order.

[0049] Step S3: Execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is the computing power calculation result cache of the neural network layers before the j-th neural network layer, where j is an integer greater than 1.

[0050] In this step, the residual value mentioned above can be the difference or change between the computational results cache of adjacent neural network layers, used to measure the similarity of inter-layer computation.

[0051] The aforementioned preset threshold can be a dynamically adjusted value, which can be pre-stored in a cache to determine whether the residual value is small enough to skip the calculation of the next neural network layer.

[0052] It is understandable that if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the calculation of the j-th neural network layer can be skipped, and the j-th neural network layer can perform the image generation task based on the cached calculation result of the preset computing power.

[0053] In this embodiment of the invention, an image generation task sent by a client is received, the image generation task being used to instruct the generation of a first image; the image generation task is input into a pre-trained diffusion model, the diffusion model including multiple sequentially connected neural network layers; the image generation task is executed based on each of the multiple sequentially connected neural network layers to obtain the first image, wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache, the preset computing power calculation result cache being the computing power calculation result cache of the neural network layers preceding the j-th neural network layer, where j is an integer greater than 1. In this way, when the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer performs the image generation task based on the cached result of the preset computing power calculation. This allows the calculation process of the j-th neural network layer to be skipped, thereby improving the efficiency of image generation tasks and increasing the speed of image generation. While improving the efficiency of image generation to meet the needs, it can also take into account the image quality to meet user requirements. Furthermore, due to the improved efficiency of image generation, the time required for image generation can be reduced, thereby reducing the cost of image generation.

[0054] In one embodiment, step S3 includes: Step S31: According to a preset diffusion step size, the denoising iteration process of the image generation task is executed for each of the plurality of sequentially connected neural network layers to obtain image rendering information. The preset diffusion step size is used to indicate the number of steps of the denoising iteration process of the image generation task executed by each neural network layer. The image rendering information includes: the output result of the last neural network layer among the plurality of sequentially connected neural network layers after the last step of the denoising iteration process in the preset diffusion step size is completed. Step S32: Based on the multiple sequentially connected neural network layers, the image rendering information is processed in a context-parallel manner to obtain the first image.

[0055] Specifically, the preset diffusion step size can be the number of times each neural network layer in the plurality of sequentially connected neural network layers performs the image generation task. For example, if the preset diffusion step size is 3, then each neural network layer will perform the image generation task 3 times.

[0056] The aforementioned image rendering information can be intermediate data output by the image generation model, including key parameters such as the structure, color distribution, and texture of the first image. For example, the image rendering information may include at least one of the following: coordinate information of each vertex in the first image, color information, normal information, and texture coordinate information. The vertices can be points constituting the first image, and the color information can be the color value describing each vertex or pixel of the first image. The normal information can describe the orientation of the object's surface, used to calculate lighting and shadow effects. The texture coordinates can be coordinates that map a 2D texture image to the surface of a 3D model.

[0057] It is understandable that the above-mentioned image generation task can be performed by each neural network layer generating intermediate results based on input data, such as potentially noisy images or text prompts, and then inputting the intermediate results into the next neural network layer for further processing.

[0058] In the above embodiments, by performing multiple iterations of the pre-trained diffusion model according to a preset diffusion step size, multiple neural network layers can gradually optimize the generated results, reduce errors, and thus improve the image generation effect. Furthermore, the preset diffusion step size can be adjusted according to the complexity of the image generation task to avoid over-computation or resource waste, thereby improving the utilization rate of computing power in processing image generation tasks.

[0059] In one embodiment, during any one of the denoising iterations of the preset diffusion step size: The first neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task and obtain the first output result; The i-th neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, where i is an integer greater than 1.

[0060] Specifically, the first neural network layer mentioned above can be the first layer in the diffusion model, and can be responsible for initial processing, such as feature extraction. The first output result mentioned above can be the output of the first neural network layer, used as the input of the second neural network layer.

[0061] In the above embodiments, the image generation task is executed sequentially by each neural network layer, and the output results of the previous layer are processed to achieve hierarchical processing. This allows for the gradual optimization of the image generation process, thereby improving the effect and accuracy of image generation.

[0062] In one embodiment, the i-th neural network layer among the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, including: If the residual value between the i-th neural network layer and the (i-1)-th neural network layer is less than a preset threshold, the calculation of the i-th neural network layer is skipped. Based on the calculation result cache of the preset neural network layer and the (i-1)-th output result, the image generation task is executed to obtain the i-th output result. Wherein, without skipping the calculation of the (i-1)th neural network layer, the preset neural network layer is the (i-1)th neural network layer; If the calculation of the (i-1)th neural network layer is skipped, the neural network layers before the i-th neural network layer are traversed forward until the next neural network layer whose calculation is not skipped is found. The preset neural network layer is the next neural network layer whose calculation is not skipped.

[0063] It is understandable that skipping the calculation of the i-th neural network layer can mean not performing the floating-point operation of the i-th neural network layer, thereby saving time and resources.

[0064] The aforementioned forward traversal can start from the current neural network layer, i.e., the i-th layer, and search layer by layer in the direction of the (i-2), (i-3), (i-4), and so on.

[0065] The next neural network layer that does not skip computations can be the first neural network layer that is determined not to skip computations during the forward traversal.

[0066] In the above embodiments, by traversing forward to find neural network layers that have not been skipped in computation, and caching the computational results of the next neural network layer that has not been skipped in computation, the reliability of the output results can be improved, and multiple neural network layers can be skipped continuously, thereby further improving the efficiency of processing image generation tasks.

[0067] Image generation based on the diffusion model is an iterative process of step-by-step denoising. Each step (diffusion step) in image generation requires the complete execution of all Transformer layers of the diffusion model. Therefore, the diffusion model needs to calculate the output of all Transformer layers from beginning to end for each diffusion step, resulting in huge computational costs and low speed. For example, Figure 2 This is a schematic diagram of image generation based on a diffusion model in related technologies, such as... Figure 2 As shown, assuming the Transformer model has 5 layers and the preset diffusion step size is 5, that is, the image generation calculation is 5 steps, then each step requires the calculation of each Transformer once, for a total of 5*5=25 calculations.

[0068] In the method for accelerating image generation by a diffusion model through caching of computational results provided in the intelligent computing center cloud platform of this invention, the computational results cache of the previous Transformer layer can be used. During the entire denoising iteration process, if the residual matches, the calculation is skipped. For example, Figure 3 This is a schematic diagram of an image generation based on a diffusion model provided in an embodiment of the present invention, such as... Figure 3 As shown, a total of 11 calculations were performed, more than half the 25 calculations required by the traditional method, thus improving the speed of image generation by the diffusion model. It should be noted that... Figure 3 The dashed box in the diagram is used to indicate skipping the computation of that neural network layer.

[0069] For example, when generating images based on diffusion models using traditional methods, such as the Flux-Dev model, inferring a 1024*1024 resolution image takes 20 steps and 14 seconds. However, using the intelligent computing center cloud platform provided in this embodiment of the invention to accelerate image generation from diffusion models through computing power result caching, although still requiring 20 steps, the method only takes 2 seconds, saving significant time compared to traditional methods. In one embodiment, step S32 includes: Step S321: Determine multiple sub-tasks based on the image rendering information. The first image includes multiple sub-layers. The multiple sub-tasks correspond one-to-one with the multiple sub-layers. The sub-tasks are used to indicate the generation of the corresponding sub-layers. Step S322: Execute the multiple sub-tasks in parallel using the multiple sequentially connected neural network layers to obtain the multiple sub-layers; Step S323: Merge the multiple sub-layers to obtain the first image.

[0070] Specifically, the aforementioned subtask can be to break down the generation of the first image into multiple independent subtasks, each subtask being responsible for generating a part of the image, namely a sub-layer.

[0071] The aforementioned sublayers can be logical components or layers that constitute the final image, i.e., the first image, and each sublayer can be rendered independently.

[0072] It is understood that the above parallel execution means that the diffusion model uses the image rendering information to run multiple subtasks simultaneously, rather than processing them one by one. That is, each sub-layer is generated independently, which can reduce waiting time. Specifically, the GPU can process the subtasks through multi-threaded or distributed computing.

[0073] It should be noted that during the parallel execution of the aforementioned multiple subtasks, computing resources can be dynamically allocated according to the computational needs of the subtasks to avoid resource waste. For example, simple subtasks are allocated fewer computing resources, while complex subtasks are allocated more computing resources.

[0074] The above merging can be achieved by combining multiple sub-layers into a complete image according to logical rules or by splitting them into corresponding positions, thus obtaining the first image.

[0075] For example, the processing of each of the above subtasks may include vertex input, vertex shader, primitive assembly, rasterization, pixel shader, pixel operations, and blending operations.

[0076] Vertex input marks the beginning of GPU processing of graphics data; this is the process of inputting the geometric information of the 3D model, i.e., the image rendering information, into the GPU. Each 3D model consists of many vertices, including information such as position, normals, and texture coordinates. The GPU stores this information in a cache for later processing. In this step, the GPU receives the vertex data of the 3D model, including the coordinates, color, normals, and texture coordinates of each vertex. This data is typically stored in buffers for fast GPU access. These buffers can include vertex coordinate buffers, color buffers, normal buffers, and texture coordinate buffers.

[0077] Vertex rendering refers to the process where the GPU obtains the necessary vertex data and then processes each vertex using a vertex shader. A vertex shader is a piece of programmable code used to perform various operations, including coordinate transformation, lighting calculation, and animation. The goal of this stage is to transform the vertices from object space to camera space to enable primitive assembly, frustum clipping, and projection.

[0078] Primitive assembly refers to the process by which the GPU assembles all vertices into primitives, such as triangles, line segments, or points, after they have been transformed to the camera coordinate system. This process is used to represent 3D objects as 2D primitives on the screen for subsequent pixel processing.

[0079] Rasterization converts primitives into pixels. In this stage, the GPU determines which pixels are affected by the primitives and calculates their position and depth values—a process of mapping 3D information to 2D screen space. Each pixel is assigned a position and depth value for use in subsequent pixel rendering. Rasterization processes the position and relative depth of pixels for subsequent pixel rendering.

[0080] Pixel rendering is the stage where the GPU processes pixels, and in this step, the final color value of each pixel is calculated. This is a highly programmable stage, typically executed using pixel shaders. Pixel shaders can perform various operations, such as texture mapping, lighting models, shadow calculations, and effects processing. It can retrieve the color from the texture map based on texture coordinates, then calculate the final color based on lighting conditions, and finally output it to the framebuffer.

[0081] Pixel-by-pixel operations are the final step performed by the GPU, and this stage includes depth testing, stencil testing, and blending operations. Depth testing determines which pixels are in the foreground to ensure the correct ordering in the rendered output. Stencil testing allows defining the contents of the stencil buffer for specific pixel operations.

[0082] Blending operations combine the colors of multiple pixels to generate the final output color. In some cases, the final pixel color may require pixel transformations, such as gamma correction, color correction, or color space conversion. These transformations ensure the accuracy and quality of the image display, help determine the visibility of the final pixels, and how to combine multiple layers into the final image.

[0083] In the above embodiments, multiple sub-tasks are determined by image rendering information, and multiple sub-tasks are executed in parallel using a diffusion model to obtain multiple sub-layers, which are then merged to obtain the first image generated by the image generation task. This allows the image generation task to be processed in parallel, reducing the image generation time and thus improving the efficiency of image generation task processing.

[0084] In one embodiment, step S321 includes: Step S3211: Obtain the model parameters of the diffusion model, wherein the model parameters are used to indicate the pixel size of the image material, and the image material is used to train the image generation model; Step S3212: Determine multiple sub-tasks based on the model parameters to obtain the multiple sub-tasks. The pixel size of the multiple sub-layers corresponding to the multiple sub-tasks is the same as that of the model parameters.

[0085] Specifically, the pixel size of the aforementioned image materials can be the resolution of the image materials used to train the diffusion model. These image materials can be the raw data used to train the diffusion model, such as high-resolution images or text-image pairs.

[0086] In the above embodiments, since the pixel size of the sublayers is consistent with the model parameters, the waste of computing resources caused by resolution mismatch can be avoided, and the quality of each sublayer can be improved, thereby improving the quality of the generated first image. Furthermore, by adjusting the number of subtasks and the size of the sublayers, it is possible to adapt to image generation tasks of different resolutions and complexities, thereby improving the flexibility of image generation task processing.

[0087] Please see Figure 4 , Figure 4 This is a structural diagram of a device for accelerating image generation by a diffusion model through caching computing results in an intelligent computing center cloud platform, as provided in an embodiment of the present invention. Figure 4 As shown, the device 400 for accelerating image generation by a diffusion model through caching of computing power calculation results in an intelligent computing center cloud platform includes: The receiving module 401 is used to receive an image generation task sent by the client, wherein the image generation task is used to instruct the generation of a first image; The input module 402 is used to input the image generation task into a pre-trained diffusion model, the diffusion model comprising multiple sequentially connected neural network layers; The execution module 403 is used to execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is the computing power calculation result cache of the neural network layers before the j-th neural network layer, where j is an integer greater than 1.

[0088] In one embodiment, the execution module includes: The first execution unit is configured to execute the denoising iteration process of the image generation task according to a preset diffusion step size, based on each of the plurality of sequentially connected neural network layers, to obtain image rendering information. The preset diffusion step size is used to indicate the number of steps of the denoising iteration process of the image generation task executed by each neural network layer. The image rendering information includes the output result of the last neural network layer among the plurality of sequentially connected neural network layers after the last step of the denoising iteration process in the preset diffusion step size is completed. The processing unit is used to process the image rendering information in a context-parallel manner based on the plurality of sequentially connected neural network layers to obtain the first image.

[0089] In one embodiment, during any one of the denoising iterations of the preset diffusion step size: The first neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task and obtain the first output result; The i-th neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, where i is an integer greater than 1.

[0090] In one embodiment, the i-th neural network layer among the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, including: If the residual value between the i-th neural network layer and the (i-1)-th neural network layer is less than a preset threshold, the calculation of the i-th neural network layer is skipped. Based on the calculation result cache of the preset neural network layer and the (i-1)-th output result, the image generation task is executed to obtain the i-th output result. Wherein, without skipping the calculation of the (i-1)th neural network layer, the preset neural network layer is the (i-1)th neural network layer; If the calculation of the (i-1)th neural network layer is skipped, the neural network layers before the i-th neural network layer are traversed forward until the next neural network layer whose calculation is not skipped is found. The preset neural network layer is the next neural network layer whose calculation is not skipped.

[0091] In one embodiment, the input module includes: The determining unit is used to determine multiple sub-tasks based on the image rendering information. The first image includes multiple sub-layers, and the multiple sub-tasks correspond one-to-one with the multiple sub-layers. The sub-tasks are used to indicate the generation of the corresponding sub-layers. The second execution unit is used to execute the multiple sub-tasks in parallel using the multiple sequentially connected neural network layers to obtain the multiple sub-layers; The merging unit is used to merge the multiple sub-layers to obtain the first image.

[0092] In one embodiment, the determining unit is specifically used for: Obtain the model parameters of the diffusion model, wherein the model parameters are used to indicate the pixel size of the image material, and the image material is used to train the image generation model; Multiple sub-tasks are determined based on the model parameters, and the pixel size of the multiple sub-layers corresponding to the multiple sub-tasks is the same as that of the model parameters.

[0093] The device for generating images by caching computing results through a diffusion model in the intelligent computing center cloud platform provided in this embodiment of the invention can realize the various processes of the above-mentioned method for generating images by caching computing results through a diffusion model in the intelligent computing center cloud platform. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0094] It should be noted that the device for accelerating the generation of images by caching computing results through the intelligent computing center cloud platform in the embodiments of the present invention can be a device, or a component, integrated circuit, or chip in an electronic device.

[0095] This invention also provides an electronic device, see [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device includes a memory 501, a processor 502, and a program or instructions stored in the memory 501 that run on the processor 502. When the program or instructions are executed by the processor 502, they can achieve the following: Figure 1 The corresponding intelligent computing center cloud platform accelerates the generation of images by caching computing results through computing power, and achieves the same beneficial effect in any step of the method embodiment of the diffusion model. It will not be repeated here.

[0096] The processor 502 can be a CPU, ASIC, FPGA, or GPU.

[0097] Those skilled in the art will understand that all or part of the steps of the above-described method embodiment for accelerating the generation of images by a diffusion model through caching computing results on an intelligent computing center cloud platform can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0098] This invention also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 The corresponding intelligent computing center cloud platform accelerates the generation of images from the diffusion model by caching the computing results, and achieves the same technical effect. Therefore, to avoid repetition, it will not be described in detail here. The storage medium mentioned includes, for example, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The present invention also provides a computer program product, including computer instructions that, when executed by a processor, implement the above-described... Figure 1 The corresponding intelligent computing center cloud platform accelerates the generation of images by caching the computing results through computing power, and achieves the same technical effect. To avoid repetition, it will not be described in detail here.

[0100] In the embodiments of this invention, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: A alone, B alone, C alone, both A and B present, both B and C present, both A and C present, and A, B, and C present.

[0101] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or second terminal device, etc.) to execute the methods of the various embodiments of this application.

[0103] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for accelerating image generation from a diffusion model using a cloud platform with intelligent computing center computing power and caching computing results, characterized in that, include: Step S1: Receive an image generation task sent by the client, wherein the image generation task is used to instruct the generation of a first image; Step S2: Input the image generation task into a pre-trained diffusion model, the diffusion model comprising multiple sequentially connected neural network layers; Step S3: Execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Wherein, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is the computing power calculation result cache of the neural network layers before the j-th neural network layer, where j is an integer greater than 1.

2. The method as described in claim 1, characterized in that, Step S3 includes: Step S31: According to a preset diffusion step size, the denoising iteration process of the image generation task is executed for each of the plurality of sequentially connected neural network layers to obtain image rendering information. The preset diffusion step size is used to indicate the number of steps of the denoising iteration process of the image generation task executed by each neural network layer. The image rendering information includes: the output result of the last neural network layer among the plurality of sequentially connected neural network layers after the last step of the denoising iteration process in the preset diffusion step size is completed. Step S32: Based on the multiple sequentially connected neural network layers, the image rendering information is processed in a context-parallel manner to obtain the first image.

3. The method as described in claim 2, characterized in that, During any denoising iteration of the preset diffusion step size: The first neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task and obtain the first output result; The i-th neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, where i is an integer greater than 1.

4. The method as described in claim 3, characterized in that, The i-th neural network layer in the plurality of sequentially connected neural network layers is used to perform the image generation task based on the (i-1)-th output result to obtain the i-th output result, including: If the residual value between the i-th neural network layer and the (i-1)-th neural network layer is less than a preset threshold, the calculation of the i-th neural network layer is skipped. Based on the calculation result cache of the preset neural network layer and the (i-1)-th output result, the image generation task is executed to obtain the i-th output result. Wherein, without skipping the calculation of the (i-1)th neural network layer, the preset neural network layer is the (i-1)th neural network layer; If the calculation of the (i-1)th neural network layer is skipped, the neural network layers before the i-th neural network layer are traversed forward until the next neural network layer whose calculation is not skipped is found. The preset neural network layer is the next neural network layer whose calculation is not skipped.

5. The method as described in claim 2, characterized in that, Step S32 includes: Step S321: Determine multiple sub-tasks based on the image rendering information. The first image includes multiple sub-layers. The multiple sub-tasks correspond one-to-one with the multiple sub-layers. The sub-tasks are used to indicate the generation of the corresponding sub-layers. Step S322: Execute the multiple sub-tasks in parallel using the multiple sequentially connected neural network layers to obtain the multiple sub-layers; Step S323: Merge the multiple sub-layers to obtain the first image.

6. The method as described in claim 5, characterized in that, Step S321 includes: Step S3211: Obtain the model parameters of the diffusion model, wherein the model parameters are used to indicate the pixel size of the image material, and the image material is used to train the image generation model; Step S3212: Determine multiple sub-tasks based on the model parameters to obtain the multiple sub-tasks. The pixel size of the multiple sub-layers corresponding to the multiple sub-tasks is the same as that of the model parameters.

7. A device for accelerating image generation from a diffusion model through caching of computing results in an intelligent computing center cloud platform, characterized in that, include: The receiving module is used to receive an image generation task sent by the client, wherein the image generation task is used to instruct the generation of a first image; An input module is used to input the image generation task into a pre-trained diffusion model, the diffusion model comprising multiple sequentially connected neural network layers; An execution module is configured to execute the image generation task based on each of the plurality of sequentially connected neural network layers to obtain the first image. Specifically, if the residual value between the j-th neural network layer and the (j-1)-th neural network layer is less than a preset threshold, the j-th neural network layer executes the image generation task based on a preset computing power calculation result cache. The preset computing power calculation result cache is a cache of computing power calculation results from the neural network layers preceding the j-th neural network layer, where j is an integer greater than 1.

8. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the program implements the steps of the method for accelerating the generation of images by a diffusion model through caching computing results by a smart computing center cloud platform as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for accelerating the generation of images by a diffusion model through caching computing results by an intelligent computing center cloud platform as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, implement the steps of the method for generating images using a diffusion model by caching computing results through a cloud platform for intelligent computing centers as described in any one of claims 1 to 6.

Images