A diffusion model denoising process block parallel acceleration method and system

Abstract

This invention provides a block-based parallel acceleration method and system for diffusion model denoising. The method includes: in a warm-up phase, uniformly segmenting the input image and distributing it to various computing devices; utilizing the local diffusion model on each computing device to perform noise prediction and image updates via synchronous communication; repeating this process a preset number of times and recording resource utilization time-series data; predicting the future resource utilization of each computing device based on a pre-trained resource utilization prediction model, generating resource utilization prediction time-series data; calculating a non-uniform image segmentation scheme for each round; in a non-uniform cyclic inference phase, segmenting the image according to the scheme; asynchronously acquiring edge data of adjacent sub-images and extracting global activation data through a sliding observation window; performing local noise prediction; and aggregating noise and updating the image through broadcast synchronous communication until denoising is complete. This invention achieves efficient parallel inference under nodes with unequal computing power, improving resource utilization and acceleration.

Description

Technical Field

[0001] This invention relates to the fields of generative artificial intelligence and high-performance computing technology, and in particular to a block-parallel acceleration method and system for the denoising process of diffusion models. Background Technology

[0002] In recent years, diffusion models have achieved revolutionary breakthroughs in the field of generative artificial intelligence. Large-scale text-to-image models such as Stable Diffusion, Flux, and Imagen can generate high-resolution images exceeding 1024×1024 pixels from text input. This technological advancement has spurred numerous applications such as image editing, digital content generation, and interactive design, while also placing higher demands on the real-time performance of model inference. Users often need multiple interactions and iterative optimizations to obtain satisfactory results. However, the enormous computational demands of generating high-resolution images have become a major bottleneck in reducing inference latency and improving user experience.

[0003] Faced with this computational bottleneck, a natural solution is to learn from the successful experience of other large models, such as large language models, and use multi-GPU parallel computing technology to distribute the computational load and shorten inference time.

[0004] To accelerate large-scale model inference, the industry has developed various parallel technologies, such as data parallelism, pipelined parallelism, tensor parallelism, and zero-redundancy parallelism. Among them, tensor parallelism, as a fine-grained partitioning scheme for the inner layers of a model, has attracted much attention due to its high efficiency. Its principle is to partition the weight matrix within a single layer of the model onto different GPUs. During computation, each GPU processes a part of the input tensor in parallel, and then merges the results through communication. This type of method has been proven to be effective in the training and inference of large language models. The reason is that large language models are huge in scale but relatively sparse in activation, and the communication overhead is negligible compared with the computational gains.

[0005] However, directly migrating tensor parallelism to diffusion models presents significant challenges due to the fundamental differences in structure and computational paradigm between diffusion models and large language models. First, the activation values ​​in diffusion models are generally dense during inference. Second, diffusion models involve an iterative denoising process. Directly introducing traditional parallel techniques such as tensor parallelism would lead to a sharp increase in communication costs due to the dense activation values, which cannot be ignored. More seriously, this would trigger a significant straggler problem, where inference speed is limited by the slowest computing node. When multiple computing devices have unequal computing power, faster devices must wait for slower devices to synchronize, greatly reducing the overall utilization of computing resources. In extreme cases, the latency of multi-device parallel inference may even be higher than that of single-device inference.

[0006] To improve the inference efficiency of diffusion models, existing technologies have proposed various solutions, but they still have significant limitations:

[0007] The first type of approach focuses on single-device optimization, such as DeepCache technology. It utilizes the feature redundancy between adjacent denoising steps in the diffusion model U-Net structure and adopts a "caching-reuse" mechanism to skip some computations, thereby achieving acceleration on a single computing node. However, this approach cannot inherently utilize multiple computing nodes for collaborative parallel computation, and its acceleration capability is limited by the upper limit of the computing performance of a single GPU. In addition, its acceleration effect is strictly controlled by the hyperparameter N. Increasing the value of N will improve the speedup ratio but will lead to a decrease in image quality. To maintain quality, the value of N must be limited, so there is a ceiling to the acceleration effect.

[0008] The second type of approach attempts to achieve multi-device parallelism, such as the Displaced Patch Parallelism technique. It utilizes the time redundancy characteristics of asynchronous communication and diffusion models to hide communication operations within the computation process, allowing multiple GPUs to perform parallel inference. However, this approach is constrained by communication primitives such as AllGather in the NVIDIA Collective Communications Library, which requires that the amount of data transmitted by each node in each communication must be strictly equal. Therefore, the input image must be evenly divided into subgraphs of the same size. This technical feature leads to inherent defects in computing environments where the computing power of multiple computing nodes is unequal: when the computing power of computing devices is unequal, even task allocation will cause a serious slacker problem, and the system performance will be constrained by the slowest node, resulting in low resource utilization.

[0009] It is evident that existing technologies cannot effectively address the core challenges of efficient parallel inference diffusion models on clusters with nodes of unequal computing power: one type of technology is limited by single-machine architecture and cannot utilize the resources of multiple devices; another type, while achieving multi-device parallelism, suffers from low resource utilization due to communication constraints. Therefore, there is an urgent need for a parallel acceleration method that can support non-uniform task partitioning, be aware of device computing capabilities, and dynamically schedule tasks, in order to fully leverage the potential of environments with unequal computing power across multiple nodes and achieve efficient and high-quality inference for diffusion models. Summary of the Invention

[0010] In view of this, embodiments of the present invention provide a block-based parallel acceleration method and system for the denoising process of diffusion models, which solves the problems of lagging behind and low utilization of computing resources caused by communication synchronization and static task allocation during parallel inference of diffusion models when the computing power of multiple computing nodes is not equal.

[0011] One aspect of the present invention provides a block-parallel acceleration method for a diffusion model denoising process, the method comprising the following steps:

[0012] During the warm-up phase, the input image is uniformly divided into a first number of sub-images and distributed to the first number of computing devices. Each computing device uses a local diffusion model to predict noise in its local sub-image. Layers containing convolution operations acquire edge data of adjacent sub-images based on fully aggregated synchronous communication, concatenate this data to the local sub-image, and then perform the convolution operation. Layers containing self-attention operations acquire activation data from all computing devices in the current self-attention layer based on the fully aggregated synchronous communication and concatenate this data into a global activation function. A key-value matrix is ​​calculated based on the global activation function, and a query matrix is ​​calculated based on the local activation function to perform attention calculation. Noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image. The noise prediction and denoising update of the input image are repeated a second predetermined number of times, and the resource utilization rate of each computing device is recorded to form resource utilization time-series data.

[0013] After preprocessing and format conversion of the resource utilization time series data, it is input into the pre-trained resource utilization prediction model, and outputs resource utilization prediction time series data representing the future resource utilization of each computing device; based on the resource utilization prediction time series data, a non-uniform segmentation scheme for the input image in each round is calculated.

[0014] In the non-uniform recurrent inference stage, the input image in each round is non-uniformly segmented into the first number of sub-images based on the non-uniform segmentation scheme, and distributed to the first number of computing devices one by one. The computing devices use a local diffusion model to predict noise in the local sub-images. The layer containing convolution operation obtains the edge data of the adjacent sub-images in the previous round based on fully aggregated asynchronous communication, and concatenates it to the local sub-image in the current round before performing the convolution operation. The layer containing self-attention operation introduces a sliding observation window based on fully aggregated asynchronous communication to extract data from the old global activation data prefetched in the previous round. The sliding observation window is smaller than or equal to the size of the old global activation data. The extracted data is supplemented with global activation data prefetched from the previous round, and the corresponding positions in the extracted data are replaced with the activation data calculated locally in the current round to obtain approximate global activation data. The key-value matrix is ​​calculated based on the approximate global activation data, and the query matrix is ​​calculated based on the activation data in the current round for attention calculation. The noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image. The non-uniform recurrent inference stage is repeated until denoising is completed.

[0015] In some embodiments, the resource utilization prediction model is pre-trained based on a Transformer model or a bidirectional long short-term memory model.

[0016] In some embodiments, the pre-training step of the resource utilization prediction model includes:

[0017] The sample resource utilization time series data is obtained and cut into initial blocks according to the first set length; the tail data of the initial blocks is repeatedly filled to ensure consistent length, and after linear mapping, they are concatenated with the original initial blocks to obtain target blocks; the target blocks are combined to obtain sample input data.

[0018] The initial resource utilization prediction model based on the Transformer model is trained using the sample input data and a prediction block is output. A loss function is constructed based on the deviation between the predicted value and the true value of the prediction block, and the parameters of the initial resource utilization prediction model are updated to obtain the resource utilization prediction model.

[0019] The formula for calculating the loss function is:

[0020]

[0021] in, Denotes the loss function, Indicates the first The predicted values ​​of each of the prediction blocks; Indicates the first The true value of each of the predicted blocks;

[0022] M represents the number of prediction blocks, which is an adjustable parameter.

[0023] In some embodiments, the resource utilization time-series data is preprocessed and format-converted before being input into the pre-trained resource utilization prediction model, and the model outputs resource utilization prediction time-series data characterizing the future resource utilization of each computing device, including:

[0024] The resource utilization time series data is cut into initial segments according to the first set length; the tail data of the initial segments is repeatedly filled to ensure consistent length, and after linear mapping, they are spliced ​​with the original initial segments to obtain target segments; the target segments are combined to obtain the input data to be processed.

[0025] The input data to be processed is imported into the resource utilization prediction model and the resource utilization prediction time series data is output.

[0026] In some embodiments, the step of calculating a non-uniform segmentation scheme for the input image in each round based on the resource utilization prediction time-series data includes:

[0027] The average value of the data from each computing device in the resource utilization time series data is calculated to obtain a first mean sequence, and the average value of the data from each computing device in the resource utilization prediction time series data is calculated to obtain a second mean sequence. These are then weighted and fused to obtain an aggregated sequence.

[0028] A recursive loop is executed, defining the initial group size as half the total number of computing devices, and defining the first weight and the second weight as both 1; the aggregation sequence is divided into a first group and a second group according to the initial group size, and the mean of the first group and the mean of the second group are calculated respectively; the initial subgraph partitioning ratio of the first group is determined based on the magnitude of the mean of the first group and the mean of the second group; the initial subgraph partitioning ratio of the first group is corrected by setting a preset upper dead zone threshold and a lower dead zone threshold to obtain the subgraph partitioning ratio of the first group, including:

[0029] When the absolute value of the difference between the initial subgraph partition ratio and the median value is less than the lower limit threshold of the dead zone, the initial subgraph partition ratio is corrected to the median value.

[0030] When the absolute value of the difference between the initial subgraph partition ratio and the median value is greater than the upper limit threshold of the dead zone, the initial subgraph partition ratio is corrected to the median value plus the product of the upper limit threshold of the dead zone and the sign of the difference.

[0031] When the absolute value of the difference between the initial subgraph partition ratio and the intermediate value is between the lower dead zone threshold and the upper dead zone threshold, a sine correction term is introduced, which adds the product of the sine correction term and the sign of the difference to the initial subgraph partition ratio.

[0032] The difference between the value 1 and the first group subgraph division ratio is used as the second group subgraph division ratio.

[0033] The first weight is the product of the first weight and the first group subgraph division ratio; the second weight is the product of the second weight and the second group subgraph division ratio, and the group size is halved.

[0034] Based on the first weight, the second weight, and the group size that are updated step by step, the first group and the second group are grouped step by step, and the recursive loop is repeated until each group contains only one computing device. The weight set corresponding to each computing device at this time is taken as the non-uniform partitioning scheme.

[0035] In some embodiments, when the absolute value of the difference between the initial subgraph partitioning ratio and the median value is less than the lower dead zone threshold, the initial subgraph partitioning ratio is corrected to the median value, as expressed by:

[0036] ;

[0037] ;

[0038] in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This refers to the intermediate value.

[0039] When the absolute value of the difference between the initial subgraph partitioning ratio and the median value is greater than the upper limit threshold of the dead zone, the initial subgraph partitioning ratio is corrected to the product of the median value, the upper limit threshold of the dead zone, and the sign of the difference, as expressed in the following expression:

[0040] ;

[0041] ;

[0042] in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This represents the upper limit threshold of the dead zone; This refers to the intermediate value.

[0043] When the absolute value of the difference between the initial subgraph partition ratio and the median value is between the lower dead zone threshold and the upper dead zone threshold, a sine correction term is introduced. This involves adding the product of the sine correction term and the sign of the difference to the initial subgraph partition ratio. The expression is as follows:

[0044] ;

[0045] ;

[0046] in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This represents the upper limit threshold of the dead zone; This refers to the intermediate value; This represents the sine correction term.

[0047] In some embodiments, the sine correction term is calculated as follows:

[0048]

[0049] in, This represents the sine correction term; This indicates the strength of the adjustment of the result by the correction term, and is an adjustable hyperparameter; This represents the upper limit threshold of the dead zone; This represents the lower limit threshold of the dead zone; This indicates the initial subgraph division ratio of the first group.

[0050] On the other hand, the present invention also provides a block-parallel acceleration system for diffusion model denoising process, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device / equipment / system implements the steps of the above method.

[0051] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0052] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0053] The diffusion model denoising process block parallel acceleration method and system described in this invention uniformly divides the input image into multiple sub-images during the preheating stage. It utilizes full aggregation synchronous communication to achieve the acquisition and stitching of edge data of adjacent sub-images by the convolutional layer, and the synchronous aggregation and attention calculation of global activation data by the self-attention layer. It also aggregates the predicted noise from each device through broadcast synchronous communication to update the image, repeating this process a set number of times and recording the time-series data of resource utilization of each computing device. Subsequently, after preprocessing and format conversion of this time-series data, it is input into a pre-trained resource utilization prediction model, outputting future resource utilization prediction time-series data. Based on this, the non-uniform segmentation scheme for the input image in each round is calculated.

[0054] In the non-uniform recurrent inference stage, the input image is non-uniformly divided according to the non-uniform segmentation scheme. Old edge data from the previous round is obtained using fully aggregated asynchronous communication for use by the convolutional layers. A sliding observation window is introduced to partially extract old global activation data from the self-attention layer. The extracted data is then supplemented with global activation data prefetched from an even earlier round, and the corresponding positions are replaced with activation data calculated locally in the current round to obtain approximate global activation data for attention calculation. Finally, noise is aggregated and the image is updated via broadcast synchronous communication. This process is repeated until denoising is complete. This method effectively solves the load balancing problem between nodes with unequal computing power, significantly improving computing resource utilization and distributed inference efficiency.

[0055] Furthermore, this invention constructs a dynamic partitioning strategy for subgraph computation tasks, mapping the prediction results to the actual segmentation process based on iterative scheduling. This strategy introduces dead-zone constraints and sine correction in the resource utilization prediction and image segmentation scheme generation stages, enabling dynamic adjustment of the prediction output. This effectively avoids the risk of overly aggressive or conservative prediction results, causing them to tend towards a stable intermediate value. This effectively overcomes the limitation of static partitioning in adapting to dynamic resource fluctuations, significantly improving computational resource utilization efficiency.

[0056] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention; the objects and other advantages of the invention may be realized and obtained by means of the structures specifically pointed out in the description and the drawings.

[0057] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0058] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0059] Figure 1 This refers to the steps of the block-parallel acceleration method for the diffusion model denoising process described in an embodiment of the present invention;

[0060] Figure 2 This is a flowchart of a sliding window-based data transmission method and a non-uniform block parallel inference process utilizing the feature redundancy characteristics between adjacent denoising steps of a diffusion model, as described in an embodiment of the present invention.

[0061] Figure 3 This is a diagram illustrating the movement rules of the sliding observation window according to an embodiment of the present invention; Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings; here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention;

[0063] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0064] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0065] In recent years, diffusion models have achieved significant breakthroughs in generative artificial intelligence, particularly in generating high-resolution images. However, the computational cost required to generate high-quality images has become a bottleneck for inference latency. Existing parallel techniques, such as tensor parallelism, suffer from high communication costs and severe lag issues when directly transferred due to the dense activation values ​​and iterative denoising characteristics of diffusion models. Single-device optimization schemes cannot utilize multi-node resources, while multi-device schemes are limited by uniform partitioning, resulting in low resource utilization in environments with unequal computing power across multiple nodes.

[0066] To address the problems existing in the prior art, this invention provides a block-parallel acceleration method for the diffusion model denoising process, such as... Figure 1 As shown, the method includes the following steps S101~S103:

[0067] S101: In the warm-up phase, the input image is uniformly divided into a first number of sub-images and distributed to a first number of computing devices. The computing devices use a local diffusion model to predict noise in their local sub-images. Layers containing convolution operations acquire edge data of adjacent sub-images based on full aggregation synchronous communication, and then concatenate them to the local sub-image before performing convolution operations. Layers containing self-attention operations acquire activation data of all computing devices in the current self-attention layer based on full aggregation synchronous communication and concatenate them into a global activation function. A key-value matrix is ​​calculated based on the global activation function, and a query matrix is ​​calculated based on the local activation function to perform attention calculation. The noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image. The noise prediction and denoising update of the input image are repeated a second set number of times, and the resource utilization of each computing device is recorded to form resource utilization time-series data.

[0068] S102: After preprocessing and format conversion of the resource utilization time series data, input it into the pre-trained resource utilization prediction model, and output resource utilization prediction time series data representing the future resource utilization of each computing device; calculate the non-uniform segmentation scheme of the input image for each round based on the resource utilization prediction time series data.

[0069] S103: In the non-uniform looping stage, the input image in each round is non-uniformly segmented into a first number of sub-images based on the non-uniform segmentation scheme, and distributed to a first number of computing devices one by one; the computing devices use a local diffusion model to predict noise in the local sub-images. The layers containing convolution operations obtain the edge data of the adjacent sub-images in the previous round based on fully aggregated asynchronous communication, and then concatenate them to the local sub-image in the current round before performing convolution operations; the layers containing self-attention operations introduce a sliding observation window based on fully aggregated asynchronous communication to extract data from the old global activation data prefetched in the previous round. The sliding observation window is smaller than or equal to the size of the old global activation data; the extracted data is supplemented with the global activation data prefetched from the previous round, and the corresponding positions in the extracted data are replaced with the activation data calculated in the current round to obtain approximate global activation data; the key-value matrix is ​​calculated based on the approximate global activation data, and the query matrix is ​​calculated based on the activation data in the current round to perform attention calculation; the noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image; the non-uniform looping stage is repeated until denoising is completed.

[0070] In step S101, the diffusion model is a generative model based on deep learning. Its core idea is to learn complex data distributions, such as image distributions, through a forward noise injection process and a backward denoising process. Let the original input data be... The total number of noise reduction steps is In the forward propagation process, the diffusion model follows a fixed Markov chain at each step. Gradually towards data Adding Gaussian noise results in data with a higher level of noise. ,in, .go through After one iteration, the data Ultimately, it is transformed into pure noise that perfectly conforms to a Gaussian distribution. In the reverse denoising process, the diffusion model uses a pre-trained denoising network. Learning from pure noise Initially, through an iterative denoising process, noise is predicted and removed step by step, ultimately restoring and generating high-quality original input data. The aforementioned denoising network is also called an inference model. A diffusion model is a pre-trained neural network, typically a U-Net structure, and is one of the core components of the diffusion model. The method proposed in this invention operates on the inverse denoising process of the diffusion model, aiming to improve the parallel efficiency of the diffusion model in performing inverse denoising inference on multiple computing devices.

[0071] To achieve efficient parallel inference, this invention employs an optimization strategy in the preheating stage of the diffusion model. In this stage, the denoising step is first... Input image Divide evenly into Sub-image, The number of computing devices is determined, and these subgraphs are assigned one by one to each computing device. Each computing device uses an inference model stored locally. Each assigned sub-image independently performs noise prediction, gradually restoring image details. In this way, each computing device can participate in the inverse denoising process in parallel, initially accelerating image generation.

[0072] The above reasoning model It includes various types of neural network layers, such as linear layers, cross-attention layers, convolutional layers, and self-attention layers. Each type of layer has different computational characteristics and data dependencies, which directly determines whether inter-device data interaction is required in a parallel computing environment. Based on the computational characteristics of different layers, this invention employs differentiated communication strategies during the warm-up phase to ensure computational correctness while optimizing communication overhead.

[0073] Specifically, during the parallel inference process in the warm-up phase, the following strategies are adopted for different execution layers and their operations:

[0074] Assume the diffusion model includes Each layer has a sequence that can be represented as follows: For any execution layer ,in The computational operations it performs are defined as follows: The equipment number is ,in The subgraph to which it is assigned and processed .

[0075] If the execution layer Operation If it is a linear operation or a cross-attention operation, then the computing device Local subgraphs can be directly accessed. The calculations can be performed without exchanging data with other computing devices. Therefore, the calculation process can be carried out independently, reducing communication overhead and improving the efficiency of parallel computing.

[0076] If the operation in the execution layer is a convolution operation, then inter-device communication is necessary to ensure the correctness of the computation. All computing devices first use synchronous aggregation communication to obtain the edge data of adjacent subgraphs.

[0077] In some embodiments, if subgraph To perform a horizontal partition, swap the left and right adjacent subgraphs. Column data; if subgraph If it's a vertical partition, then swap the adjacent subgraphs. Row data, in which It is an adjustable hyperparameter, typically determined by the kernel size, to ensure that the convolution operation covers the entire receptive field. (Device) In obtaining subgraphs Hezi Diagram After obtaining the edge data, these data are stitched together to the local subgraph. The corresponding edges are then used to form a complete extended region containing contextual information, and then a convolution operation is performed on the concatenated data. This step is to compensate for the convolution kernel's position on the subgraph. The calculation error may be caused by the lack of adjacent pixels at the boundary.

[0078] In some embodiments, the above-mentioned synchronous aggregation communication can be implemented using the AllGather communication primitive from the NVIDIA Collections Communication Library.

[0079] If the operation at the execution layer is a self-attention operation, all computing devices first need to exchange activation data at this layer using full-aggregate synchronous communication and concatenate it into a global activation function. Subsequently, each device uses the global activation function to calculate the key and value matrices in the self-attention mechanism, while the query matrix is ​​calculated using its local activation function. Finally, standard attention calculation is performed. This communication process ensures that each device obtains complete global context information, thereby guaranteeing the accuracy of the self-attention calculation, but it also introduces significant communication overhead. Complete the current denoising step. After all layers have been computed, each computing device uses broadcast synchronous communication to aggregate the noise predicted by each computing device, and stitches together all received noise prediction results into a complete noise map, thereby updating the input image to obtain... Repeat the above process. Second-rate This is an adjustable hyperparameter, typically set to 4, to complete the preheating phase. During this process, the resource utilization sequence of each computing device is continuously monitored and recorded to form resource utilization time-series data, expressed as:

[0080] ;

[0081] in, Indicates the equipment number, D represents ; Indicates equipment The resource utilization rate sequence.

[0082] In step S102, the resource utilization time-series data obtained in the preheating stage of step S101 is preprocessed and converted in format to adapt to the input requirements of the prediction model. The preprocessing and format conversion process includes steps S1021 to S1024:

[0083] Step S1021: Divide the resource utilization sequence monitored by each computing device into fixed block lengths. The length of the non-overlapping region between two consecutive data blocks is denoted as the sliding step size. Divide into multiple data blocks, the expression is:

[0084] ;

[0085] ;

[0086] in, Indicates the equipment number; Indicates the block length; Represents a block sequence; The number of data blocks is represented; L represents the length of the resource utilization sequence monitored by each computing device. The length of the non-overlapping region between two consecutive data blocks is denoted as the sliding step.

[0087] Step S1022: To maintain data integrity and adapt to the input requirements of the subsequent resource utilization prediction model, a data filling operation is performed at the end of the block sequence. The last data in the block sequence is filled repeatedly, and the number of fillings is the value corresponding to the sliding step size.

[0088] Step S1023: To enrich the input features, a linear mapping layer is used to divide each block. Mapped to a new linear space In the middle. Then the mapping result is compared with the block. The sequences are spliced ​​together to form the enhanced sequence. .

[0089] Step S1024: Enhance the sequence of all computing devices By concatenating the data, we obtain a unified tensor that fully contains the historical resource utilization of all computing devices during the warm-up phase. The expression is:

[0090] ;

[0091] in, Represents the integrated tensor; Indicates the number of computing devices; This indicates the number of data blocks; P represents the block length.

[0092] This completes the preprocessing and format conversion of the resource utilization time series data obtained during the preheating phase.

[0093] By segmenting the resource utilization sequence, we can better capture the temporal correlation within and between the resource utilization sequence segments. On the other hand, by converting long sequences into fewer fixed-length sequence segments, we can significantly reduce the memory usage and computation time of the subsequent resource utilization prediction model during the inference process, thereby meeting the stringent real-time requirements of parallel inference.

[0094] Based on this, the preprocessed tensor Input resource utilization prediction model for autoregressive prediction, and generate prediction blocks step by step. The generated prediction block By piecing them together in chronological order, a complete time series of future resource utilization rate forecasts can be formed. ,in, , , This represents the number of prediction blocks and is an adjustable hyperparameter.

[0095] In some embodiments, the resource utilization prediction model described above can be pre-trained using a Transformer model or a bidirectional long short-term memory model.

[0096] In some embodiments, the pre-training step of the resource utilization prediction model based on the Transformer model includes:

[0097] The integrated tensor obtained from the resource utilization time series data obtained in the above preheating stage after preprocessing and format conversion. The initial resource utilization prediction model based on the Transformer model is input, and autoregressive prediction training is performed to output prediction blocks. A loss function is constructed based on the deviation between the predicted values ​​and the true values ​​of the prediction blocks, and the parameters of the initial resource utilization prediction model are updated to complete the pre-training process. The loss function is calculated as follows:

[0098] ;

[0099] in, Represents the loss function. Indicates the first The predicted values ​​for each prediction block; Indicates the first The true value of each predicted block; This indicates the number of prediction blocks and is an adjustable parameter.

[0100] The resource utilization rate prediction time series data output by the above resource utilization rate prediction model. The mapping is to a specific non-uniform segmentation scheme for the input image, including steps S1025~S1029:

[0101] Step S1025: Calculate the original input sequences of the resource utilization prediction model respectively. and the predicted time series output The mean of the two values ​​is combined with the dependence of short-term observations and the confidence level of long-term observations. A weighted fusion of these two means yields the aggregated sequence, calculated as follows:

[0102] ;

[0103] in, Represents an aggregated sequence. ; is a weighted hyperparameter, representing the dependence on short-term observations; is a weight hyperparameter, representing the confidence level for long-term observations; This represents the mean of the original input sequence for the resource utilization prediction model. ; This represents the output of the resource utilization prediction model, predicting time series. The mean, ;

[0104] Step S1026: Set initial recursion parameters, define the initial group size as half the total number of computing devices, and define the first weight and the second weight as both 1; divide the aggregation sequence into a first group and a second group according to the initial group size, and calculate the mean of the first group and the mean of the second group respectively. Determine the initial subgraph partitioning ratio of the first group based on the magnitude of the mean of the first group and the mean of the second group. The calculation formula is:

[0105] ;

[0106] in, Indicates the first group; This indicates the initial subgraph partitioning ratio for the first group; This represents the aggregated resource capability index of the first group; This indicates the aggregation resource capability index for the second group; This represents the average aggregated resource capacity index of all computing devices in the first group; This represents the average aggregated resource capability index of all computing devices in the second group.

[0107] Step S1027: To prevent the partitioning ratio from deviating excessively from the effective interval and to avoid extreme ratios during task partitioning, while ensuring that the partitioning ratio converges towards the effective interval, thereby improving partitioning accuracy and resource utilization efficiency, this invention introduces a dead zone constraint and sine correction mechanism. When the initial subgraph partitioning ratio deviates excessively from the median value, it will lead to uneven resource allocation, such as some computing devices being overloaded while others are idle, thus reducing inference efficiency. Therefore, this invention can effectively limit the problem of unreasonable partitioning ratios by setting upper and lower limits for the dead zone threshold. At the same time, by introducing a sine correction term to adjust the partitioning ratio, the partitioning ratio is driven to move towards the effective interval to improve prediction accuracy.

[0108] In some embodiments, when the absolute value of the difference between the initial subgraph partitioning ratio and the median value is less than the lower dead zone threshold, the initial subgraph partitioning ratio is adjusted to the median value, calculated as follows:

[0109] ;

[0110] ;

[0111] in, This indicates the initial subgraph partitioning ratio for the first group; This indicates the division ratio of the first group of subgraphs; This represents the lower limit threshold of the dead zone; This represents the intermediate value.

[0112] In some embodiments, when the absolute value of the difference between the initial subgraph partitioning ratio and the median value is greater than the upper dead zone threshold, the initial subgraph partitioning ratio is corrected to the product of the median value, the upper dead zone threshold, and the sign of the aforementioned difference. The calculation formula is as follows:

[0113] ;

[0114] ;

[0115] in, This indicates the initial subgraph partitioning ratio for the first group; This indicates the division ratio of the first group of subgraphs; This represents the lower limit threshold of the dead zone; Indicates the upper limit threshold of the dead zone; This represents the intermediate value.

[0116] In some embodiments, when the absolute value of the difference between the initial subgraph partition ratio and the median value is between the lower dead zone threshold and the upper dead zone threshold, a sine correction term is introduced. The initial subgraph partition ratio is then added to the product of the sine correction term and the sign of the difference, calculated as follows:

[0117] ;

[0118] ;

[0119] in, This indicates the initial subgraph partitioning ratio for the first group; This indicates the division ratio of the first group of subgraphs; This represents the lower limit threshold of the dead zone; Indicates the upper limit threshold of the dead zone; Indicates the intermediate value; This represents the sine correction term.

[0120] The formula for calculating the sine correction term is:

[0121]

[0122] in, Indicates the sine correction term; This indicates the strength of the adjustment of the result by the correction term, and is an adjustable hyperparameter; Indicates the upper limit threshold of the dead zone; This represents the lower limit threshold of the dead zone; This indicates the initial subgraph partitioning ratio for the first group.

[0123] Step S1028: The initial subgraph partitioning ratio of the first group is obtained by dead zone limitation and sine correction. The difference between the value 1 and the subgraph partitioning ratio of the first group is used as the subgraph partitioning ratio of the second group. The calculation formula is:

[0124] ;

[0125] in, This indicates the division ratio of the first group of subgraphs; This indicates the division ratio of the second group of subgraphs.

[0126] Step S1029: Use the product of the first weight and the first group subgraph partitioning ratio as the first weight; use the product of the second weight and the second group subgraph partitioning ratio as the second weight, and halve the group size. The calculation formula is:

[0127] ;

[0128] ;

[0129] ;

[0130] in, Indicates the first weight; This indicates the division ratio of the first group of subgraphs; This indicates the updated first weight; Indicates the second weight; Indicates the division ratio of the second group of subgraphs; This indicates the updated second weight; This indicates the updated group size.

[0131] By updating the first weight, the second weight, and the group size step by step, the first group and the second group are grouped step by step, and steps S1026 to S1029 are repeated until each group contains only one computing device. The weight set corresponding to each computing device at this time is taken as a non-uniform partitioning scheme.

[0132] In step S103, after the initialization loop of the preheating stage is completed, based on the non-uniform segmentation scheme obtained in step S102, this invention adopts a parallel inference scheme based on a sliding window data transmission method and utilizing the feature redundancy characteristics between adjacent denoising steps of the diffusion model, such as... Figure 2 As shown, efficient parallel denoising inference of the diffusion model is completed collaboratively on multiple computing devices with varying computing power. This stage introduces asynchronous communication and a sliding observation window to hide communication overhead within the computation process and effectively solves the data dependency problem between non-uniform subgraphs, thereby significantly improving parallel efficiency while ensuring the quality of the generated image.

[0133] Specifically, the non-uniform cyclic reasoning process includes steps S1031~S1034:

[0134] Step S1031: Based on the above non-uniform segmentation scheme, the current denoising step... Input image Divided into A subgraph of different sizes, denoted as Each subgraph Area and target computing device The computing power is positively correlated; subsequently, each subgraph is assigned to the corresponding computing device.

[0135] Step S1032: Each computing device uses the locally stored inference model Each assigned sub-image independently performs noise prediction, gradually restoring image details. In this way, each computing device can participate in the inverse denoising process in parallel, accelerating image generation. During this process, the inference model... Different communication strategies are adopted for data communication between different execution layers in order to ensure computational accuracy while optimizing communication overhead.

[0136] Specifically, let the diffusion model include Each layer has a sequence that can be represented as follows: For any execution layer ,in The computational operations it performs are defined as follows: Let the equipment number be... ,in The subgraph to which it is assigned and processed .

[0137] If the execution layer Operation If it is a linear operation or a cross-attention operation, then the computing device Local subgraphs can be directly accessed. It can perform calculations without exchanging data with other computing devices.

[0138] If the execution layer Operation If it is a convolution operation, then inter-device communication is necessary to ensure the correctness of the calculation. Start calculating the current denoising step When the convolutional layer is applied, it immediately uses asynchronous aggregation communication to request the edge data of adjacent subgraphs at the corresponding positions. Simultaneously, the computing device does not wait for the completion of the aforementioned communication request, but immediately reads the data from the previous denoising step from the local cache. In step ( ), edge data from neighboring subgraphs is pre-acquired and stored in the same way. The pre-fetched edge data is then compared with the current local subgraph. The data is concatenated to form an expanded tensor containing the necessary contextual information. Then, using this expanded tensor as input, a convolution operation is performed to obtain the output activation values ​​of the convolutional layer. The fully aggregated asynchronous communication operation initiated in this round will be performed in the background, and the new data exchanged will be cached for the next denoising step. This mechanism ensures that communication always precedes computation, thus completely hiding communication overhead within the computation time.

[0139] In some embodiments, if subgraph To perform a horizontal partition, swap the left and right adjacent subgraphs. Column data; if subgraph If it's a vertical partition, then swap the adjacent subgraphs. Row data, in which It is an adjustable hyperparameter, usually determined by the kernel size, to ensure that the convolution operation can cover the entire receptive field.

[0140] In some embodiments, the above-mentioned fully aggregated asynchronous communication can be implemented using the AllGather communication primitive from the NVIDIA Collections Communication Library.

[0141] If the execution layer Operation For self-attention operations, an asynchronous communication mechanism based on an adjustable sliding observation window is used for processing. The specific process includes:

[0142] For self-attention layer Define an observation window whose size is equal to the size of the fully aggregated asynchronous communication buffer of the self-attention layer. The size of the fully aggregated asynchronous communication buffer is the size of the complete image activation values ​​of the self-attention layer divided by the number of computing devices. Noise reduction steps The position of the observation window in the steps Time to determine. For computing devices The subgraph activation data is only observed in the window covered portion and the full aggregate asynchronous communication buffer is filled: if the window size is greater than or equal to the subgraph activation data size, all data is filled into the buffer; if the window size is less than the subgraph activation data size, only the window covered portion is filled, and the rest of the data is discarded.

[0143] After completing the sliding window-based data filling, each computing device initiates a fully aggregated asynchronous communication operation to exchange buffer data. Due to the use of asynchronous communication, the computing devices perform better in the noise reduction step... During calculation, the actual steps used are as follows: Prefetched and cached outdated global activation data, composed of data from various device windows. .

[0144] Due to sliding window limitations, outdated global activation data Some data may be missing. To construct a complete global tensor, this invention employs a two-step approximation method, specifically:

[0145] Used in the noise reduction step Prefetched global activation data, compared with outdated global activation data Fill in the missing areas due to window size limitations; replace the data in the corresponding area of ​​the local subgraph in the filled-in stale tensor with the data from the current step. Calculated local activation data Ultimately, we obtain approximately globally activated data.

[0146] The key and value matrices in the self-attention mechanism are calculated based on the aforementioned approximate global activation data, while the query matrix is ​​calculated from the aforementioned local activation data. The calculation is performed, and then standard attention calculation is executed. This method ensures the global nature of attention calculation while avoiding costly synchronous global communication.

[0147] After the current step's calculation is completed, the observation window position is updated based on the relationship between the subgraph and the sliding window size, such as... Figure 3 As shown: If the size of the active data in the subgraph is smaller than the window size, the window position is fixed; if the size of the active data in the subgraph is larger than the window size, the window is translated along the specified direction by a distance equal to the length of the window side. If the window extends beyond the subgraph boundary after translation, it is aligned with the boundary; if the window has already moved to the end, it is moved back to the initial position and aligned with the starting boundary. This sliding mechanism ensures that during the denoising step in the iteration, data from different parts of each subgraph can gradually participate in global communication, converging complete semantic information.

[0148] Step S1033: Complete the current noise reduction step. After all layers have been computed, each computing device uses broadcast synchronous communication to aggregate the predicted noise outputs of each computing device, and stitches together all the received noise prediction results into a complete noise map, thereby updating the input image. ,get .

[0149] In some embodiments, the above-mentioned broadcast synchronization communication can be implemented using the Broadcast communication primitives of the NVIDIA Collective Communication Library.

[0150] Step S1034: The above As input for the next round of non-uniform cyclic inference, the non-uniform cyclic inference process described in steps S1031 to S1033 is repeated until the diffusion model completes all denoising iterations, ultimately generating a high-quality image. .

[0151] In non-uniform cyclic inference, this invention achieves non-uniform block-based parallel inference of the diffusion model by utilizing the feature redundancy between adjacent denoising steps and the sliding window data exchange method. The preheating initialization in S101 is necessary because the feature redundancy between adjacent steps is low in the initial stage of the denoising process. If an asynchronous data exchange strategy based on a sliding window is directly adopted at this time, the quality of the generated image will significantly degrade due to incomplete context information. The preheating process effectively initializes the model's activation data, laying the foundation for subsequent steps. In non-uniform parallel inference, the observation window moves its position after each denoising step. This design ensures that each computing device can gradually acquire the comprehensive semantic information required to generate the image over time, thereby significantly improving parallel efficiency while ensuring that the quality of the final generated image is not compromised.

[0152] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0153] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0154] In summary, the block-parallel acceleration method and system for the diffusion model denoising process described in this invention uniformly divides the input image into multiple sub-images during the warm-up phase. It utilizes full-aggregation synchronous communication to achieve the acquisition and stitching of edge data of adjacent sub-images by the convolutional layer, and the synchronous aggregation and attention calculation of global activation data by the self-attention layer. Furthermore, it aggregates the predicted noise from each device through broadcast synchronous communication to update the image, repeating this process a set number of times and recording the time-series data of resource utilization of each computing device. Subsequently, after preprocessing and format conversion of this time-series data, it is input into a pre-trained resource utilization prediction model, outputting future resource utilization prediction time-series data. Based on this, the non-uniform segmentation scheme for the input image in each round is calculated.

[0155] In the non-uniform recurrent inference stage, the input image is non-uniformly divided according to the non-uniform segmentation scheme. Old edge data from the previous round is obtained using fully aggregated asynchronous communication for use by the convolutional layers. A sliding observation window is introduced to partially extract old global activation data from the self-attention layer. The extracted data is then supplemented with global activation data prefetched from an even earlier round, and the corresponding positions are replaced with activation data calculated locally in the current round to obtain approximate global activation data for attention calculation. Finally, noise is aggregated and the image is updated via broadcast synchronous communication. This process is repeated until denoising is complete. This method effectively solves the load balancing problem between nodes with unequal computing power, significantly improving computing resource utilization and distributed inference efficiency.

[0156] Furthermore, this invention constructs a dynamic partitioning strategy for subgraph computation tasks, mapping the prediction results to the actual segmentation process based on iterative scheduling. This strategy introduces dead-zone constraints and sine correction in the resource utilization prediction and image segmentation scheme generation stages, enabling dynamic adjustment of the prediction output. This effectively avoids the risk of overly aggressive or conservative prediction results, causing them to tend towards a stable intermediate value. This effectively overcomes the limitation of static partitioning in adapting to dynamic resource fluctuations, significantly improving computational resource utilization efficiency.

[0157] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both; whether implemented in hardware or software depends on the specific application and design constraints of the technical solution; those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention; when implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc.; when implemented in software, the elements of the invention are programs or code segments used to perform the required tasks; the programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave;

[0158] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures; for the sake of brevity, detailed descriptions of known methods are omitted here; in the above embodiments, several specific steps are described and shown as examples; however, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps after understanding the spirit of the present invention.

[0159] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments;

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A block-parallel acceleration method for denoising a diffusion model, characterized in that, The method includes the following steps: During the warm-up phase, the input image is uniformly divided into a first number of sub-images and distributed to the first number of computing devices. Each computing device uses a local diffusion model to predict noise in its local sub-image. Layers containing convolution operations acquire edge data of adjacent sub-images based on full aggregation synchronous communication, concatenate them to the local sub-image, and then perform the convolution operation. Layers containing self-attention operations acquire activation data of all computing devices in the current self-attention layer based on the full aggregation synchronous communication and concatenate them into a global activation function. A key-value matrix is ​​calculated based on the global activation function, and a query matrix is ​​calculated based on the local activation function to perform attention calculation. Noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image. The noise prediction and denoising update of the input image are repeated a second predetermined number of times, and the resource utilization rate of each computing device is recorded to form resource utilization time-series data. After preprocessing and format conversion of the resource utilization time series data, it is input into the pre-trained resource utilization prediction model, and outputs resource utilization prediction time series data representing the future resource utilization of each computing device; based on the resource utilization prediction time series data, a non-uniform segmentation scheme for the input image in each round is calculated. In the non-uniform recurrent inference stage, the input image in each round is non-uniformly segmented into the first number of sub-images based on the non-uniform segmentation scheme, and distributed to the first number of computing devices one by one. The computing devices use a local diffusion model to predict noise in the local sub-images. The layer containing convolution operation obtains the edge data of the adjacent sub-images in the previous round based on fully aggregated asynchronous communication, and concatenates it to the local sub-image in the current round before performing the convolution operation. The layer containing self-attention operation introduces a sliding observation window based on fully aggregated asynchronous communication to extract data from the old global activation data prefetched in the previous round. The sliding observation window is smaller than or equal to the size of the old global activation data. The extracted data is supplemented with global activation data prefetched from the previous round, and the corresponding positions in the extracted data are replaced with the activation data calculated locally in the current round to obtain approximate global activation data. The key-value matrix is ​​calculated based on the approximate global activation data, and the query matrix is ​​calculated based on the activation data in the current round for attention calculation. The noise predicted by each computing device is aggregated through broadcast synchronous communication to update the input image. The non-uniform recurrent inference stage is repeated until denoising is completed.

2. The block-parallel acceleration method for the diffusion model denoising process according to claim 1, characterized in that, The resource utilization prediction model is obtained by pre-training based on the Transformer model or the bidirectional long short-term memory model.

3. The block-parallel acceleration method for the diffusion model denoising process according to claim 2, characterized in that, The pre-training steps of the resource utilization prediction model include: The sample resource utilization time series data is obtained and cut into initial blocks according to the first set length; the tail data of the initial blocks is repeatedly filled to ensure consistent length, and after linear mapping, they are concatenated with the original initial blocks to obtain target blocks; the target blocks are combined to obtain sample input data. The initial resource utilization prediction model based on the Transformer model is trained using the sample input data and a prediction block is output. A loss function is constructed based on the deviation between the predicted value and the true value of the prediction block, and the parameters of the initial resource utilization prediction model are updated to obtain the resource utilization prediction model. The formula for calculating the loss function is: in, Denotes the loss function, Indicates the first The predicted values ​​of each of the prediction blocks; Indicates the first The true value of the predicted block; M represents the number of predicted blocks, which is an adjustable parameter.

4. The block-parallel acceleration method for the diffusion model denoising process according to claim 3, characterized in that, After preprocessing and format conversion, the resource utilization time-series data is input into the pre-trained resource utilization prediction model, which outputs resource utilization prediction time-series data characterizing the future resource utilization of each computing device, including: The resource utilization time series data is cut into initial segments according to the first set length; the tail data of the initial segments is repeatedly padded to ensure consistent length, and after linear mapping, they are spliced ​​with the original initial segments to obtain target segments; the target segments are combined to obtain the input data to be processed. The input data to be processed is imported into the resource utilization prediction model and the resource utilization prediction time series data is output.

5. The block-parallel acceleration method for diffusion model denoising process according to claim 1, characterized in that, The step of calculating the non-uniform segmentation scheme for the input image in each round based on the resource utilization prediction time-series data includes: The average value of the data from each computing device in the resource utilization time series data is calculated to obtain a first mean sequence, and the average value of the data from each computing device in the resource utilization prediction time series data is calculated to obtain a second mean sequence. These are then weighted and fused to obtain an aggregated sequence. A recursive loop is executed, defining the initial group size as half the total number of computing devices, and defining the first weight and the second weight as both 1; the aggregation sequence is divided into a first group and a second group according to the initial group size, and the mean of the first group and the mean of the second group are calculated respectively; the initial subgraph partitioning ratio of the first group is determined based on the magnitude of the mean of the first group and the mean of the second group; the initial subgraph partitioning ratio of the first group is corrected by setting a preset upper dead zone threshold and a lower dead zone threshold to obtain the subgraph partitioning ratio of the first group, including: When the absolute value of the difference between the initial subgraph partition ratio and the median value is less than the lower limit threshold of the dead zone, the initial subgraph partition ratio is corrected to the median value. When the absolute value of the difference between the initial subgraph partition ratio and the median value is greater than the upper limit threshold of the dead zone, the initial subgraph partition ratio is corrected to the median value plus the product of the upper limit threshold of the dead zone and the sign of the difference. When the absolute value of the difference between the initial subgraph partition ratio and the median value is between the lower dead zone threshold and the upper dead zone threshold, a sine correction term is introduced, and the initial subgraph partition ratio is added to the product of the sine correction term and the sign of the difference. The difference between the value 1 and the first group sub-graph division ratio is used as the second group sub-graph division ratio; The first weight is the product of the first weight and the first group subgraph partitioning ratio; the second weight is the product of the second weight and the second group subgraph partitioning ratio, and the group size is halved. Based on the first weight, the second weight, and the group size that are updated step by step, the first group and the second group are grouped step by step, and the recursive loop is repeated until each group contains only one computing device. The weight set corresponding to each computing device at this time is taken as the non-uniform partitioning scheme.

6. The block-parallel acceleration method for the diffusion model denoising process according to claim 5, characterized in that, When the absolute value of the difference between the initial subgraph partitioning ratio and the median value is less than the lower dead zone threshold, the initial subgraph partitioning ratio is corrected to the median value, as expressed by: ； ； in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This refers to the intermediate value; When the absolute value of the difference between the initial subgraph partitioning ratio and the median value is greater than the upper limit threshold of the dead zone, the initial subgraph partitioning ratio is corrected to the product of the median value, the upper limit threshold of the dead zone, and the sign of the difference, as expressed in the following expression: ； ； in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This represents the upper limit threshold of the dead zone; This refers to the intermediate value; When the absolute value of the difference between the initial subgraph partition ratio and the median value is between the lower dead zone threshold and the upper dead zone threshold, a sine correction term is introduced. This involves adding the product of the sine correction term and the sign of the difference to the initial subgraph partition ratio. The expression is as follows: ； ； in, This indicates the initial subgraph division ratio of the first group; This indicates the division ratio of the first group of sub-graphs; This represents the lower limit threshold of the dead zone; This represents the upper limit threshold of the dead zone; This refers to the intermediate value; This represents the sine correction term.

7. The block-parallel acceleration method for the diffusion model denoising process according to claim 6, characterized in that, The formula for calculating the sine correction term is: in, This represents the sine correction term; This indicates the strength of the adjustment of the result by the correction term, and is an adjustable hyperparameter; This represents the upper limit threshold of the dead zone; This represents the lower limit threshold of the dead zone; This indicates the initial subgraph division ratio of the first group.

8. A block-parallel acceleration system for diffusion model denoising, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images