Diffusion model reasoning acceleration method based on feature caching and distributed parallelism

Abstract

The invention relates to a diffusion model reasoning acceleration method based on feature caching and distributed parallelism, and the method comprises the steps: exploring a caching method through a heuristic search algorithm, and confirming features corresponding to time steps needing to be cached; and performing distributed reasoning according to the caching scheme. The efficiency problem of an existing diffusion model in an image generation task in a high-resolution image generation scene is solved; the reasoning process of the diffusion model is efficiently processed based on the feature cache and the distributed reasoning architecture, and the resource utilization rate and the reasoning efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a diffusion model reasoning acceleration method based on feature caching and distributed parallelism. Background Art

[0002] Artificial intelligence technology is developing rapidly, particularly in generative modeling. Research on various generative models continues to deepen across application scenarios, from image, audio, video, and text generation, pushing AI beyond "understanding" to "creation." Representative models include generative adversarial networks (GANs), variational autoencoders (VAEs), and the recently garnered attention of diffusion models. These approaches demonstrate their unique technical strengths in multimodal generation, structure preservation, and detail control, continuously expanding the application boundaries of generative models in scientific research and engineering practice. Unlike traditional methods, diffusion models do not rely on adversarial optimization of a discriminator or impose assumptions on prior distributions of latent variables. Instead, they achieve step-by-step generation by establishing a stable Markov chain. This mechanism significantly improves sample quality and structural consistency while maintaining generative stability, achieving particularly strong performance in tasks such as high-resolution image generation and cross-modal transfer.

[0003] While diffusion models have made significant progress in generation quality and expression diversity, their inference processes are generally plagued by high resource consumption and slow response times. Current mainstream models typically require dozens to hundreds of denoising steps, each of which requires a full call to a large neural network for forward computation. This results in high overall inference overhead and places high demands on the computing platform. This issue is particularly prominent on edge devices or mobile terminals, limiting the widespread deployment of diffusion models.

[0004] Currently, efforts to accelerate diffusion model inference have focused on multiple aspects. Regarding sampling strategies, DDIM effectively reduces the number of required sampling steps by constructing deterministic inference paths. The DPM-Solver family of methods reparameterizes the diffusion process based on numerical integration techniques, reducing the number of inference steps to single digits. Regarding model compression and lightweighting, researchers have used techniques such as pruning and quantization to construct simplified models that can complete inference in a single or few steps, effectively accelerating the inference process and reducing graphics memory requirements. However, directly pruning or quantizing models significantly impacts performance, so distillation techniques are used to improve model performance. Specifically, the knowledge of a complex diffusion model (the teacher model) is transferred to a smaller, more efficient model (the student model), significantly improving inference speed while maintaining generation quality. During the reverse diffusion process of the diffusion model, intermediate features between the teacher and student models can be extracted. By minimizing the difference between the intermediate features of the teacher and student models, the student model can learn the teacher's feature representation. In addition to feature distillation, distillation can also be performed by minimizing the difference between the final outputs of the teacher and student models. This can be achieved by minimizing the mean squared error or perceptual loss of the generated data. Although this method can effectively speed up the reasoning process, its training cost cannot be ignored.

[0005] On the other hand, some recent work has focused on the U-Net module in the diffusion model. Since the reasoning of the diffusion model requires step-by-step denoising, and the changes between adjacent time steps are not much different, this provides an idea for caching the features of the previous step for use in subsequent steps. The main challenge of this method is: when can the cached output be reused to replace the output of the current time step, for example, when the current output is similar to the cached output, it can be reused. However, this difference cannot be predicted before calculating the current output. If a simple uniform partitioning method is used, that is, caching the features of a fixed time step, it may damage the model performance. Therefore, exploring an effective caching mechanism can effectively speed up the reasoning of the diffusion model without damaging the model.

[0006] In current image generation tasks, a single GPU often becomes a performance bottleneck in terms of computing power and video memory, especially for high-resolution image generation tasks. Therefore, rationally splitting the model or data and distributing it across multiple GPUs for parallel computing can effectively leverage the advantages of multiple machines and thus accelerate the inference process. Currently, there are multiple parallel computing strategies, including data parallelism, model parallelism, pipeline parallelism, and tensor parallelism, each with its own focus on applicable scenarios, communication costs, and implementation complexity. For the diffusion model architecture, combining the characteristics of multiple parallel strategies and selecting the appropriate parallel method can improve inference efficiency. In addition, due to the step-by-step iterative nature of the diffusion model, each machine must have complete input at each step, resulting in non-negligible communication overhead. Optimizing communication can further improve inference efficiency. Summary of the Invention

[0007] Based on this, it is necessary to provide a diffusion model inference acceleration method, system, computer device, storage medium and computer program product based on feature caching and distributed parallelism to address the above technical problems.

[0008] In a first aspect, the present application provides a diffusion model reasoning acceleration method based on feature caching and distributed parallelism, the method comprising:

[0009] Loading a data set and preprocessing the data set;

[0010] Based on the similarity of features in adjacent denoising steps, the features denoised in the previous step are cached and reused to optimize the calculation;

[0011] Explore caching methods through heuristic search algorithms and identify features corresponding to time steps that need to be cached;

[0012] Distributed reasoning based on caching schemes.

[0013] In one embodiment, the data set consists of pictures and description sentences;

[0014] The step of loading a data set and preprocessing the data set includes:

[0015] Load the dataset, preprocess the dataset, and separate the images from their corresponding descriptions.

[0016] In one embodiment, exploring a specific caching method by a heuristic search algorithm to determine features corresponding to time steps that need to be cached includes:

[0017] The specific caching method is explored through genetic algorithms to confirm the characteristics corresponding to the time steps that need to be cached.

[0018] In one embodiment, exploring a specific caching method using a genetic algorithm to determine the characteristics corresponding to the time steps that need to be cached includes:

[0019] Encode the cache strategy into chromosomes, where each chromosome serves as a set of cache strategies;

[0020] Randomly generate multiple sets of cache strategies as the initial population and iterate based on them;

[0021] Comprehensively consider the inference time and generated image quality to design a suitable fitness function;

[0022] Through selection, crossover and mutation operator operations, new chromosomes, i.e. new caching strategies, are generated to enhance population diversity;

[0023] After generating a new generation of population, the iteration is repeated and finally the caching strategy with the highest fitness is selected as the caching step of non-uniform caching.

[0024] In one embodiment, performing distributed reasoning according to the cache solution includes: analyzing the model structure and machine characteristics, dividing the parallel modules, distributing the computing load, and performing distributed reasoning according to the cache solution.

[0025] In one embodiment, analyzing the model structure and machine characteristics, dividing the parallel modules, distributing the computing load, and performing distributed reasoning according to the caching solution include:

[0026] Determine the parallel approach based on task requirements;

[0027] Determine the parallel modules and split the model according to the selected parallel method;

[0028] Evaluate computing node performance and divide computing load;

[0029] Perform distributed reasoning based on the solutions obtained by the search algorithm.

[0030] In a second aspect, the present application further provides a diffusion model inference acceleration system based on feature caching and distributed parallelism, the system comprising:

[0031] A preprocessing module, used for loading a data set and preprocessing the data set;

[0032] The optimization module is used to cache the denoising features of the previous step for reuse to optimize the calculation based on the similarity of features in adjacent denoising steps;

[0033] The exploration module is used to explore caching methods through heuristic search algorithms and identify the features corresponding to the time steps that need to be cached;

[0034] The inference module is used to perform distributed inference based on the caching scheme.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0036] Loading a data set and preprocessing the data set;

[0037] Based on the similarity of features in adjacent denoising steps, the features denoised in the previous step are cached and reused to optimize the calculation;

[0038] Explore caching methods through heuristic search algorithms and identify features corresponding to time steps that need to be cached;

[0039] Distributed reasoning based on caching schemes.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: loading a data set and preprocessing the data set;

[0041] Based on the similarity of features in adjacent denoising steps, the features denoised in the previous step are cached and reused to optimize the calculation;

[0042] Explore caching methods through heuristic search algorithms and identify features corresponding to time steps that need to be cached;

[0043] Distributed reasoning based on caching schemes.

[0044] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0045] Loading a data set and preprocessing the data set;

[0046] Based on the similarity of features in adjacent denoising steps, the features denoised in the previous step are cached and reused to optimize the calculation;

[0047] Explore caching methods through heuristic search algorithms and identify features corresponding to time steps that need to be cached;

[0048] Distributed reasoning based on caching schemes.

[0049] The aforementioned diffusion model inference acceleration method, system, computer device, storage medium, and computer program product based on feature caching and distributed parallelism load and preprocess a dataset; cache denoising features from the previous step based on feature similarity in adjacent denoising steps for reuse to optimize computation; explore caching methods through a heuristic search algorithm and identify features corresponding to time steps that require caching; and perform distributed inference based on the caching scheme. This method addresses the efficiency issues of existing diffusion models in image generation tasks in high-resolution image generation scenarios. Based on feature caching and a distributed inference architecture, the inference process of the diffusion model is efficiently processed, improving resource utilization and inference efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 1 is a flow chart of a diffusion model inference acceleration method based on feature caching and distributed parallelism in one embodiment;

[0051] Figure 2 A schematic diagram of operators in a genetic algorithm provided by one embodiment of the present invention;

[0052] Figure 3 A schematic diagram of a feature cache provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of tensor parallelism provided for one embodiment of the present invention;

[0054] Figure 5 A block parallel schematic diagram provided for one embodiment of the present invention;

[0055] Figure 6 1. A structural block diagram of a diffusion model inference acceleration system based on feature caching and distributed parallelism in one embodiment;

[0056] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] In diffusion model reasoning, on the one hand, diffusion models usually adopt complex network structures, such as U-Net, which contains multiple convolutional layers, attention layers and residual connections. Each step of the denoising process requires complex calculations, including feature extraction and prediction operations. The repetitiveness of these steps leads to an extension of the overall reasoning time. On the other hand, a large amount of image data and model parameters need to be processed during the reasoning process, which requires a large amount of video memory and computing resources. Therefore, the reasoning process of the diffusion model is relatively time-consuming. The present application provides a diffusion model reasoning acceleration method based on feature caching and distributed parallelism, which can effectively reduce the amount of calculation of the diffusion model and improve the reasoning efficiency of the diffusion model through handwritten code. Taking the computing cluster used as an example, the machine performance and model architecture are analyzed, the parallel modules and computing load are divided, and distributed reasoning is performed. At the same time, feature caching is used to reduce repeated calculations, a feature caching mechanism is designed, and a genetic algorithm is used to search for specific time steps that need to be cached and the corresponding features, thereby reducing the model loss caused by uniformly caching time steps.

[0059] In one embodiment of the present application, Figure 1 As shown, a diffusion model reasoning acceleration method based on feature caching and distributed parallelism is provided, the method comprising:

[0060] S100: Load a data set and preprocess the data set.

[0061] In this application, the dataset needs to be preprocessed to separate the images from their corresponding descriptions. Specifically, the dataset is first loaded from a specified path. The data consists of images and descriptions, with each image corresponding to a description describing the content, scene, and relationships between objects in the image. The images and corresponding descriptions are separated and saved separately to files to facilitate subsequent use of the search method.

[0062] S200, based on the similarity of features in adjacent denoising steps, cache the denoising features of the previous step for reuse to optimize calculation.

[0063] Specifically, during the diffusion model inference process, the denoising steps are performed step by step. Some features between adjacent denoising steps have a high degree of similarity. By caching the features of the previous step for subsequent denoising steps, the amount of computation can be reduced.

[0064] In this application, the core architecture of the diffusion model is the U-Net module, which predicts noise and then gradually performs denoising. The U-Net consists of an encoder and a decoder. Analysis of the encoder and decoder features at different time steps during the denoising process reveals that the encoder features do not vary much across time steps, while the decoder features vary significantly. Therefore, the encoder features are selected for caching.

[0065] In one embodiment, please refer to Figure 3 Because the denoising process of the diffusion model is performed step by step, each step denoises the image and produces a sharper version. Between adjacent denoising steps, features have not changed significantly, resulting in high similarity between corresponding feature maps. Feature caching methods calculate and store feature maps in the first step and directly use the cached data in the second step without recalculation, thus reducing computational effort. The figure shows two caching strategies: uniform caching and non-uniform caching. Uniform caching uses a fixed computational allocation pattern: a full network inference is performed every N steps to obtain a cached result. The result is then quickly computed in the next N-1 steps, and this process is repeated until inference completes. This is simple to implement, requiring only the replacement of the original model with the corresponding model. However, because it caches and reuses features at fixed intervals, it ignores differences in feature changes between steps. This can cause cached features to mismatch actual requirements in steps with significant feature changes, impacting generation quality. Non-uniform caching adjusts the caching frequency and number of updates based on the feature redundancy between steps in the network. It allocates fewer cache updates to steps with high feature redundancy and more cache updates to steps with low feature redundancy. It can more accurately utilize feature redundancy, avoid frequent cache updates in steps with small feature changes, and ensure timely cache updates in steps with large feature changes, thereby achieving more significant performance improvements while ensuring generation quality. However, its implementation is relatively complex.

[0066] S300, exploring caching methods through a heuristic search algorithm and confirming features corresponding to time steps that need to be cached.

[0067] A heuristic search algorithm is used to explore specific caching methods. Simply reusing the cache can impair model performance. For example, if the current output is similar to the cached output, it can be reused, but if it is not, it cannot be reused. However, this discrepancy cannot be predicted before the current output is calculated. A simple uniform partitioning method, caching features at fixed time steps, would ignore the impact of these discrepancies. Therefore, a search algorithm is used to identify features at specific time steps that need to be cached.

[0068] In this application, a specific caching method is usually explored through a genetic algorithm to confirm the features corresponding to the time steps that need to be cached. This is a random search algorithm based on the natural selection mechanism of Darwin's theory of evolution, which is used to solve optimization and search problems. It simulates the genetic and natural selection phenomena in the process of biological evolution, continuously and iteratively generates new solutions, and gradually approaches the optimal solution to the problem. The search target is a list of a set of time steps, which are called key time steps, representing the use of the features of these time steps as cache to provide input for subsequent time steps. A set of solutions is eventually found through iteration. Caching according to this set of solutions can accelerate the reasoning of the diffusion model, and effectively solve the model loss caused by the simple caching of fixed time steps.

[0069] In one embodiment, exploring a specific caching method through a genetic algorithm and determining features corresponding to time steps that need to be cached includes the following steps:

[0070] S310 , encoding the cache strategy into chromosomes, where each chromosome serves as a set of cache strategies.

[0071] Specifically, each gene in the chromosome corresponds to a time step of the diffusion model, and takes a value of 0 or 1, indicating whether to cache or not cache the feature at that step, respectively. For example, if the chromosome length is T (the total number of time steps in the diffusion model), a value of 1 at a certain position indicates that the feature at the corresponding step is cached.

[0072] S320: Randomly generate multiple cache strategies as the initial population, and iterate based on this. The population size is set according to the actual situation, such as 100, to ensure population diversity.

[0073] S330, comprehensively consider the inference time and the quality of the generated image, and design a suitable fitness function.

[0074] The inference time and image quality under each set of caching strategies are calculated, and the weighted sum of the inference time and the image quality index is used as the fitness value to balance the inference speed and image quality.

[0075] S340, through selection, crossover and mutation operator operations, generates new chromosomes, i.e., new caching strategies, to enhance population diversity.

[0076] The selection operator uses roulette wheel selection, determining the probability of an individual being selected based on its fitness. The total fitness of the population is calculated, and the probability of an individual being selected is the ratio of its fitness to the total. Individuals with higher fitness have a higher probability of advancing to the next generation. The crossover operator selects two parent individuals for a single-point crossover, randomly determining the crossover point and exchanging some genes from the parent chromosomes to generate the offspring. The mutation operator uses non-uniform mutation, reducing the magnitude of mutation as the number of generations increases. Introducing high randomness early on and reducing the magnitude later on prevents the destruction of high-quality solutions.

[0077] S350, after generating a new generation of population, repeat the iteration and finally select the caching strategy with the highest fitness as the caching step of non-uniform caching.

[0078] Specific reference Figure 2 In one embodiment, the present invention provides a schematic diagram of a genetic operator. The crossover operator simulates the phenomenon of gene recombination during biological reproduction, generating new offspring individuals by combining partial genes of two parent individuals. The single-point crossover principle in the figure is to randomly select a crossover point and exchange the genes of the two parent individuals at this point. Assuming that the chromosomes of parent individual 1 are 0, 1, 0, 1, 1, and the chromosomes of parent individual 2 are 1, 1, 0, 0, 0, and the crossover point is 3, then offspring individual 1 is 0, 1, 0, 0, 0, and offspring individual 2 is 1, 1, 0, 1, 1. The two-point crossover principle in the figure is to randomly select two crossover points and exchange the genes of the two parent individuals between these two points. Assuming that the chromosomes of parent individual 1 are 0, 1, 0, 1, 1, and the chromosomes of parent individual 2 are 1, 1, 0, 0, 0, and the crossover points are 1 and 4, then offspring individual 1 is 1, 1, 0, 0, 1, and offspring individual 2 is 0, 1, 0, 1, 0. The mutation operator simulates the genetic mutation phenomenon that occurs during biological reproduction, increasing population diversity by randomly altering certain genes within an individual. This works by randomly selecting certain gene bits within an individual and inverting or randomly changing their values. For example, if an individual's chromosomes are 0, 1, 0, 1, 1, and the mutation bit is 2, the mutated chromosomes will be 0, 0, 0, 1, 1.

[0079] S400: Perform distributed reasoning according to the cache solution.

[0080] In this application, the model structure and machine characteristics are analyzed, the parallel modules are divided, the computing load is distributed, and distributed reasoning is performed according to the caching scheme.

[0081] The core computational module in the Stable Diffusion model is the U-Net module, and only this module is parallelized. The U-Net module consists of a downsampling module (DownBlocks), a midblock module (MidBlocks), and an upsampling module (UpBlocks). Each module contains multiple ResNetBlocks, Self-Attention or Cross-Attention layers, as well as operators such as convolution, normalization, and activation. Tensor parallel versions of these structures are selectively designed to support multi-GPU parallel computing. For block-wise parallelism, the input feature map for each step is divided into multiple blocks, with the number of blocks determined by the number of GPUs used. Each GPU is responsible for its corresponding input block. Because simple block-wise computation results in a lack of information exchange between block edges and other blocks, resulting in a noticeable sense of fragmentation in the final generated image, the complete input feature map of the corresponding network layer from the previous time step is cached to provide context for the current time step, thereby generating a complete image.

[0082] In a distributed environment, multiple compute nodes are used for parallel inference. The entire model is split and the specific modules that require parallel execution are determined. The computational load is distributed based on the performance of the compute nodes used. The specific parallelization method is determined based on the inference task requirements. Specifically, based on the task requirements, if the generated image quality is high, tensor parallelization is used to reduce the greater model loss caused by error accumulation, as the feature caching mechanism is also used in block parallelization. Otherwise, block parallelization is used. After determining the parallelization method, the entire model is split and the specific modules that require parallel execution are determined. The computational load is distributed based on the performance of the compute nodes used. Finally, the inference task is executed in parallel according to the caching scheme generated in the previous step.

[0083] Specifically, analyzing the model structure and machine characteristics, dividing the parallel modules, distributing the computing load, and performing distributed reasoning based on the caching solution include the following steps:

[0084] S410: Confirm the parallel method according to task requirements.

[0085] To avoid a sense of disjointed final image generation, block parallelism caches the complete features of the current network layer from the previous step. This, combined with the previous caching strategy, can lead to cumulative errors and reduced generated image quality. Therefore, if the task requires high-quality generated images, use tensor parallelism; otherwise, use block parallelism.

[0086] S420: Determine the parallel modules and perform model segmentation based on the selected parallel mode.

[0087] The diffusion model consists of multiple modules. The core module, U-Net, is key to iterative denoising. Each step requires noise prediction through this module, so only this module is parallelized. After determining the parallelization method, the model is partitioned. For tensor parallelism, weights are partitioned by columns or rows. For multi-head self-attention layers, the weights of each head can be assigned to different devices. For block parallelism, the input features of each step are partitioned, and each compute node only calculates a subset of features.

[0088] S430: Evaluate computing node performance and divide computing load.

[0089] The performance of the compute nodes used is evaluated, such as memory and bandwidth. For non-parallel modules, compute nodes with better performance are used for computation. For parallel modules, data partitioning is performed to allocate more data to higher-performing compute nodes. For example, in block-based parallelism, the input features of each layer are partitioned, with higher-performing nodes receiving larger feature blocks.

[0090] S440: Execute distributed reasoning based on the solution obtained by the search algorithm.

[0091] In the preceding step, a set of solutions has been found through the search algorithm, i.e., the time steps that need to be cached, and then reasoning is performed in a determined parallel manner.

[0092] like Figure 4 The tensor parallel method shown in the figure. Among them, U-Net consists of a sampling module, an intermediate module, and an upsampling module. Each module contains multiple ResNetBlock, Self-Attention or Cross-Attention layers, as well as operator units such as convolution, normalization, and activation. Tensors are divided for these modules. Specifically, two different weight matrix splitting methods are shown in the figure. The upper part shows the column-by-column splitting strategy, in which the input data flows through multiple parameter matrices and is then merged to obtain the output. The lower part shows the row-by-row splitting strategy, in which the input is also distributed to different GPUs. Each GPU is responsible for calculating the product of a part of the parameter matrix and the input, and then merging these partial results through communication operations to obtain the final output.

[0093] like Figure 5The block-wise parallel approach shown in Figure 1 divides the input image into multiple small patches and assigns each patch to a different GPU for processing. GPUs with better performance are assigned larger patches. However, simply dividing the image into patches results in a lack of interaction between patches, resulting in obvious splicing artifacts. By leveraging the similarity between adjacent denoising steps in the diffusion process, the pre-computed feature map from the previous step is reused to provide contextual information for the current step, thereby achieving asynchronous communication and computational overlap.

[0094] In summary, this application loads and preprocesses a dataset; based on feature similarity between adjacent denoising steps, caches features from the previous denoising step for reuse to optimize computation; explores caching methods through a heuristic search algorithm and identifies features corresponding to time steps that require caching; and performs distributed inference based on the caching scheme. This approach addresses the efficiency issues of existing diffusion models in image generation tasks in high-resolution image generation scenarios. This approach efficiently handles the inference process of the diffusion model based on feature caching and a distributed inference architecture, improving resource utilization and inference efficiency.

[0095] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0096] Based on the same inventive concept, an embodiment of the present application also provides a system for implementing the aforementioned diffusion model inference acceleration system based on feature caching and distributed parallelism. The implementation solution provided by this system is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the diffusion model inference acceleration system based on feature caching and distributed parallelism provided below can be found in the limitations of the diffusion model inference acceleration method based on feature caching and distributed parallelism above, and will not be repeated here.

[0097] In one embodiment, Figure 6 As shown, a diffusion model inference acceleration system based on feature caching and distributed parallelism is provided, including:

[0098] The preprocessing module 100 is used to load a data set and preprocess the data set.

[0099] The optimization module 200 is used to cache the denoising features of the previous step for reuse to optimize calculations based on the similarity of features in adjacent denoising steps.

[0100] The exploration module 300 is used to explore the caching method through a heuristic search algorithm and confirm the features corresponding to the time steps that need to be cached.

[0101] The reasoning module 400 is configured to perform distributed reasoning according to the cache solution.

[0102] In one embodiment, the data set consists of pictures and description sentences; the preprocessing module 100 is further used to: load the data set, preprocess the data set, and separate the pictures and the corresponding description sentences.

[0103] In one embodiment, the exploration module 300 is further configured to: explore a specific caching method through a genetic algorithm to identify features corresponding to time steps that need to be cached.

[0104] In one embodiment, the exploration module 300 is further configured to:

[0105] Encode the cache strategy into chromosomes, where each chromosome serves as a set of cache strategies;

[0106] Randomly generate multiple sets of cache strategies as the initial population and iterate based on them;

[0107] Comprehensively consider the inference time and generated image quality to design a suitable fitness function;

[0108] Through selection, crossover and mutation operator operations, new chromosomes, i.e. new caching strategies, are generated to enhance population diversity;

[0109] After generating a new generation of population, the iteration is repeated and finally the caching strategy with the highest fitness is selected as the caching step of non-uniform caching.

[0110] In one embodiment, the reasoning module 400 is further used to: analyze the model structure and machine characteristics, divide the parallel modules, distribute the computing load, and perform distributed reasoning according to the cache solution.

[0111] In one embodiment, the reasoning module 400 is further configured to:

[0112] Determine the parallel approach based on task requirements;

[0113] Determine the parallel modules and split the model according to the selected parallel method;

[0114] Evaluate computing node performance and divide computing load;

[0115] Perform distributed reasoning based on the solutions obtained by the search algorithm.

[0116] Each module in the aforementioned feature caching and distributed parallel diffusion model inference acceleration system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0117] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a diffusion model inference acceleration method based on feature caching and distributed parallelism is implemented.

[0118] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned diffusion model reasoning acceleration method based on feature caching and distributed parallelism when executing the computer program.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the diffusion model reasoning acceleration method based on feature caching and distributed parallelism is implemented.

[0121] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned diffusion model reasoning acceleration method based on feature cache and distributed parallelism.

[0122] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to these.

[0123] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A diffusion model inference acceleration method based on feature caching and distributed parallelism, characterized in that: The method comprises: Loading a data set and preprocessing the data set; Based on the similarity of features in adjacent denoising steps, the features denoised in the previous step are cached and reused to optimize the calculation; Explore caching methods through heuristic search algorithms and identify features corresponding to time steps that need to be cached; Distributed reasoning based on caching schemes.

2. The method according to claim 1, characterized in that The dataset consists of pictures and description sentences; The step of loading a data set and preprocessing the data set includes: Load the dataset, preprocess the dataset, and separate the images from their corresponding descriptions.

3. The method according to claim 1, characterized in that The heuristic search algorithm is used to explore specific caching methods and confirm the characteristics corresponding to the time steps that need to be cached, including: The specific caching method is explored through genetic algorithms to confirm the characteristics corresponding to the time steps that need to be cached.

4. The method according to claim 3, characterized in that The genetic algorithm is used to explore specific caching methods and determine the characteristics corresponding to the time steps that need to be cached, including: Encode the cache strategy into chromosomes, where each chromosome serves as a set of cache strategies; Randomly generate multiple sets of cache strategies as the initial population and iterate based on them; Comprehensively consider the inference time and generated image quality to design a suitable fitness function; Through selection, crossover and mutation operator operations, new chromosomes, i.e. new caching strategies, are generated to enhance population diversity; After generating a new generation of population, the iteration is repeated and finally the caching strategy with the highest fitness is selected as the caching step of non-uniform caching.

5. The method according to claim 1, wherein The distributed reasoning according to the cache solution includes: analyzing the model structure and machine characteristics, dividing the parallel modules, distributing the computing load, and performing distributed reasoning according to the cache solution.

6. The method according to claim 5, characterized in that Analyzing the model structure and machine characteristics, dividing the parallel modules, distributing the computing load, and performing distributed reasoning based on the cache solution include: Determine the parallel approach based on task requirements; Determine the parallel modules and split the model according to the selected parallel method; Evaluate computing node performance and divide computing load; Perform distributed reasoning based on the solutions obtained by the search algorithm.

7. A diffusion model inference acceleration system based on feature caching and distributed parallelism, characterized in that: The system comprises: A preprocessing module, used for loading a data set and preprocessing the data set; The optimization module is used to cache the denoising features of the previous step for reuse to optimize the calculation based on the similarity of features in adjacent denoising steps; The exploration module is used to explore caching methods through heuristic search algorithms and identify the features corresponding to the time steps that need to be cached; The inference module is used to perform distributed inference based on the caching scheme.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images