Diffusion model CFG and SP mixed parallel reasoning method based on orthogonal process group topology

Abstract

The present application belongs to the technical field of artificial intelligence, and particularly relates to a diffusion model CFG and SP mixed parallel reasoning method based on orthogonal process group topology, which solves the problem of how to realize more efficient diffusion model reasoning acceleration on a multi-NPU cluster. The NPU cluster processes are arranged into a two-dimensional grid topology of CFG dimension x SP dimension, two groups of orthogonal communication process groups are constructed, the SP group is used for all_to_all communication of distributed attention, and the CFG group is used for all_gather synchronization of conditional / conditional prediction. In each denoising step, each process holds a fragment along the sequence dimension and independently performs forward propagation, and the results are synchronized through the CFG group at the end of the step. The method unifies CFG parallel and SP parallel in one framework, and the total number of processes is 2N, which can be flexibly expanded to 2 / 4 / 8 card configurations.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a hybrid parallel inference method based on orthogonal process group topology diffusion model CFG and SP. Background Technology

[0002] The diffusion model is currently the mainstream generative AI model architecture. During the inference phase, the diffusion model generates target content through iterative denoising. To improve generation quality, classifier-free guidance (CFG) is typically employed. This involves simultaneously performing conditional prediction (using positive cues) and unconditional prediction (using negative cues) in each denoising step, and then combining them with weighted values ​​to obtain the final prediction. CFG requires two transformer forward propagations in each denoising step, doubling the computational cost.

[0003] To accelerate inference in diffusion models, existing technologies have proposed two parallel strategies: CFG Parallelism: Conditional and unconditional predictions are distributed across two computing devices and executed in parallel, with results synchronized via aggregated communication operations. This strategy reduces the computational overhead of CFG from two serial forward propagations to one parallel forward propagation (plus communication overhead), theoretically achieving approximately a 2x speedup. Existing parallel CFG implementations typically assume only two processes, one responsible for conditional prediction and the other for unconditional prediction.

[0004] Sequence Parallelism (SP): The attention computation sequence of the transformer is divided into multiple segments along the sequence dimension and distributed to multiple computing devices for parallel execution. This strategy can reduce the computation time of a single forward propagation to about 1 / N (where N is the degree of sequence parallelism), but requires additional communication overhead.

[0005] However, in existing technologies, CFG parallelism and sequential parallelism are two independent parallel strategies, applied to different scenarios. When the number of computing devices is large (e.g., 4 cards, 8 cards), using CFG parallelism alone (only 2-way) cannot fully utilize all devices; while using sequential parallelism alone can scale to more devices, it does not include CFG parallelization, and each device still needs to perform conditional prediction and unconditional prediction serially. Therefore, there is an urgent need for a hybrid parallelism method that can unify CFG parallelism and sequential parallelism within a single framework, thereby achieving more efficient parallel inference on multi-device clusters. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention aims to provide a classifier-independent guided and sequential hybrid parallel inference method for diffusion models based on orthogonal process group topology. This method unifies classifier-independent guided parallelism and sequential parallelism within a single parallel framework, allowing the total number of processes to be flexibly expanded to the product of the CFG dimension and the SP dimension. This enables more efficient acceleration of diffusion model inference across multiple NPU clusters.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a hybrid parallel inference method based on the diffusion model CFG and SP using orthogonal process group topology, comprising the following steps: Step S1: Arrange all processes in the NPU cluster into a two-dimensional grid topology, defining two orthogonal dimensions: CFG Dimensions: The dimensions are 2 in size, and they are responsible for conditional prediction and unconditional prediction respectively. SP Dimension: The dimension size is N, N≥1, and each holds 1 / N segments of the attention calculation sequence; Step S2: Assign two-dimensional coordinates (cfg_rank) to each process. (, sp_rank), and establish a mapping relationship between the global ranking and the two-dimensional coordinates: rank= cfg_rank×sp_degree + sp_rank Where rank represents the global ranking; cfg_rank represents the CFG dimension coordinate, cfg_rank∈{0,1}; sp_rank represents the SP dimension coordinate, sp_rank∈{0,1,...,N-1}; sp_degree represents the SP parallelism. Step S3: Based on the two-dimensional mesh topology, construct two sets of orthogonal communication process groups: Step S3.1, SP Group: Divide processes with consecutive SP rankings within the same CFG dimension into a group. For a process with cfg_rank=g, its SP group contains processes with rankings of {g×N,g×N+1,...,g×N+N-1}. The SP group is used for all_to_all communication and sequence gather operations in attention calculation. Step S3.2, CFG group: Divide processes with the same SP ranking in different CFG dimensions into a group. For a process with sp_rank=s, its CFG group includes processes with rankings of {s,N+s}, that is, two processes with the same sp_rank in cfg_rank=0 and cfg_rank=1. The CFG group is used for all_gather communication at the end of each denoising step to synchronize conditional prediction and unconditional prediction results. Step S4: In each denoising step, each process performs the following parallel computations: Step S4.1, Sequence Segmentation: Divide the input sequence into N segments along the sequence dimension, with each process holding the sp_rank segment; Step S4.2, Conditional / Unconditional Assignment: Processes with cfg_rank=0 use positive prompts as encoder input, and processes with cfg_rank=1 use negative prompts as encoder input; Step S4.3, Transformer forward propagation: Each process independently executes the transformer forward propagation calculation. In the attention calculation, the processes in the SP group exchange query, key, and value tensors through the all_to_all operation to achieve distributed attention calculation. Step S4.4, Sequence gather: After the transformer calculation is completed, the processes in the SP group use the all_gather operation to splice the sequence fragments they hold into a complete sequence; Step S5: At the end of each noise reduction step, perform CFG synchronization: Step S5.1: The two processes within the CFG group exchange their prediction results through the all_gather operation; Step S5.2: Each process calculates the final prediction result using the following formula: noise_pred=uncond_pred+guidance_scale×(cond_pred-uncond_pred) Where noise_pred represents the final prediction result for each process; uncond_pred represents the unconditional prediction result; cond_pred represents the conditional prediction result; and guidance_scale represents the guidance strength coefficient. Step S6: Perform post-processing of the denoising step using the final prediction result, and proceed to the next denoising step or output the final result.

[0008] Preferably, when constructing the communication process group in step S3, all processes must call dist.new_group() to create the process group in the same order to ensure global consistency, as follows: First, create all SP groups: Iterate through cfg_rank from 0 to 1, and create an SP group containing N consecutive ranking processes for each cfg_rank; Then create all CFG groups: Iterate through sp_rank from 0 to N-1, and create a CFG group containing 2 processes across CFG dimensions for each sp_rank; Each process selectively saves references to the corresponding SP group and CFG group based on its own cfg_rank and sp_rank.

[0009] Preferably, in step S4.3, the all-to-all communication process for distributed attention computation includes: (1) Convert the shape [batch, heads, seq_local, head_dim] of the query, key, and value tensors to a scatter ready state; (2) Execute the all_to_all operation within the SP group, with the scatter dimension being the head dimension (dim=2) and the gather dimension being the sequence dimension (dim=1), so that each process obtains the query, key, and value of the complete head but a partial sequence; (3) Perform standard attention calculation or Flash Attention calculation; (4) Perform the all_to_all operation again to restore the original sequence distribution.

[0010] Preferably, the sequence parallelism N is an integer, and its value ranges from [1, 1 / 2Q]. NPU ], Q NPU This indicates the number of NPUs; when N=1, it is pure CFG parallelism; when N>1, it is a hybrid parallelism of CFG and SP.

[0011] Preferably, in the hybrid parallel mode, each process independently maintains its own TeaCache state, and the branch state of TeaCache is always 0, so there is no need to synchronize the TeaCache state across processes.

[0012] Preferably, when the sequence parallelism N>1 is detected, the AttentionCache function is automatically disabled.

[0013] Preferably, during the video decoding stage, only processes with cfg_rank=0 and sp_rank=0 perform VAE decoding and video output, while other processes skip this step.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. By using a two-dimensional orthogonal process group topology, CFG parallelism and sequence parallelism are unified in a single framework. Parallelism in the CFG dimension allows conditional prediction and unconditional prediction to be executed simultaneously, eliminating the twice the computational overhead of serial CFG. Parallelism in the SP dimension reduces the sequence length of attention computation to 1 / N, reducing the computation time of a single forward propagation. The communication operations of the two orthogonal process groups are independent of each other, without communication conflicts, and can be executed efficiently. The total number of processes = 2 × N, which can be flexibly expanded to various configurations such as 2 cards, 4 cards, and 8 cards, making full use of the computing resources of multi-NPU clusters.

[0015] 2. Actual tests show that 2 CFG cards in parallel achieve a speedup of 1.80 times (close to the theoretical 2 times), 4 CFG+SP cards in parallel achieve a speedup of 3.17 times, and 8 CFG+SP cards in parallel achieve a speedup of 5.60 times, demonstrating good scalability.

[0016] 3. Compared with pure sequence parallelism, this invention achieves CFG parallelism simultaneously with the same number of devices, and gains approximately 2 times the CFG acceleration. Compared with pure CFG parallelism, this invention can expand the speedup ratio through the SP dimension when the number of devices exceeds 2, thus avoiding device idleness. Attached Figure Description

[0017] Figure 1 This is a flowchart of the hybrid parallel inference method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0019] The NPU is based on the Ascend 910A processor, and the aggregated communication operations are implemented on the backend of the Huawei HCCL aggregated communication library.

[0020] like Figure 1 As shown, this invention discloses a hybrid parallel inference method for diffusion models CFG and SP based on orthogonal process group topology, comprising the following steps: Step S1: Arrange all processes in the NPU cluster into a two-dimensional grid topology, defining two orthogonal dimensions: CFG Dimensions: The dimensions are 2 in size, and they are responsible for conditional prediction and unconditional prediction respectively. SP Dimension: The dimension size is N, N≥1, and each holds 1 / N segments of the attention calculation sequence; Total number of processes = CFG dimension × SP dimension = 2N; Step S2: Assign two-dimensional coordinates (cfg_rank) to each process. (, sp_rank), and establish a mapping relationship between the global ranking and the two-dimensional coordinates: rank= cfg_rank×sp_degree + sp_rank Where rank represents the global ranking; cfg_rank represents the CFG dimension coordinate, cfg_rank∈{0,1}; sp_rank represents the SP dimension coordinate, sp_rank∈{0,1,...,N-1}; sp_degree represents the SP parallelism, i.e., the SP dimension N; Step S3: Based on the two-dimensional mesh topology, construct two sets of orthogonal communication process groups: Step S3.1, Sequence Parallel Group (SP Group): Processes with consecutive SP rankings within the same CFG dimension are grouped together. For a process with cfg_rank=g, its SP group contains processes with rankings of {g×N,g×N+1,...,g×N+N-1}. The SP group is used for all_to_all communication and sequence gather operations in attention calculation. Step S3.2, CFG group: Divide processes with the same SP ranking in different CFG dimensions into a group. For a process with sp_rank=s, its CFG group includes processes with rankings of {s,N+s}, that is, two processes with the same sp_rank in cfg_rank=0 and cfg_rank=1. The CFG group is used for all_gather communication at the end of each denoising step to synchronize conditional prediction and unconditional prediction results. Step S4: In each denoising step, each process performs the following parallel computations: Step S4.1, Sequence Segmentation: Divide the input sequence into N segments along the sequence dimension, with each process holding the sp_rank segment; Step S4.2, Conditional / Unconditional Assignment: Processes with cfg_rank=0 use positive prompts (conditional prompts) as encoder input, and processes with cfg_rank=1 use negative prompts (unconditional prompts) as encoder input; Step S4.3, Transformer forward propagation: Each process independently executes the transformer forward propagation calculation. In the attention calculation, the processes in the SP group exchange query, key, and value tensors through the all_to_all operation to achieve distributed attention calculation. Step S4.4, Sequence gather: After the transformer calculation is completed, the processes in the SP group use the all_gather operation to splice the sequence fragments they hold into a complete sequence; Step S5: At the end of each noise reduction step, perform CFG synchronization: Step S5.1: The two processes within the CFG group exchange their prediction results (conditional prediction and unconditional prediction) through the all_gather operation. Step S5.2: Each process calculates the final prediction result using the following formula: noise_pred=uncond_pred+guidance_scale×(cond_pred-uncond_pred) Wherein, noise_pred represents the final prediction result of each process; uncond_pred represents the unconditional prediction result; cond_pred represents the conditional prediction result; guidance_scale represents the guidance strength coefficient, which is used to control the guidance strength of conditional prediction on the final prediction result. Its value range is [3,12], and it can be adjusted accordingly within this range in actual operation according to the generation effect.

[0021] Step S6: Use the final prediction result to perform post-processing of the denoising step (such as scheduler stepping, conditional frame mask application, etc.), and proceed to the next denoising step or output the final result.

[0022] In a further implementation, when constructing the communication process group in step S3, all processes must call dist.new_group() to create the process group in the same order to ensure global consistency, as follows: First, create all SP groups: Iterate through cfg_rank from 0 to 1, and create an SP group containing N consecutive ranking processes for each cfg_rank; Then create all CFG groups: Iterate through sp_rank from 0 to N-1, and create a CFG group containing 2 processes across CFG dimensions for each sp_rank; Each process selectively saves references to the corresponding SP group and CFG group based on its own cfg_rank and sp_rank.

[0023] In a further implementation, step S4.3, the all-to-all communication process for distributed attention computation includes: (1) Convert the shape [batch, heads, seq_local, head_dim] of the query, key, and value tensors to a scatter ready state; (2) Execute the all_to_all operation within the SP group, with the scatter dimension being the head dimension (dim=2) and the gather dimension being the sequence dimension (dim=1), so that each process obtains the query, key, and value of the complete head but a partial sequence; (3) Perform standard attention calculation or Flash Attention calculation; (4) Perform the all_to_all operation again to restore the original sequence distribution.

[0024] In a further implementation, the sequence parallelism N is an integer, and its value ranges from [1, 1 / 2Q]. NPU ], Q NPUThis indicates the number of NPUs; when N=1, it is pure CFG parallelism; when N>1, it is a hybrid parallelism of CFG and SP.

[0025] In a further implementation, the hybrid parallel inference method also includes a TeaCache compatible step: in hybrid parallel mode, each process independently maintains its own TeaCache state (the residual cache of the previous step), and the branch state of TeaCache is always 0 (each process performs forward propagation once per step), without the need to synchronize TeaCache state across processes.

[0026] In a further implementation, the hybrid parallel inference method also includes an AttentionCache disabling step: when the sequence parallelism N>1 is detected, the AttentionCache function is automatically disabled because the block-level caching mechanism of AttentionCache is incompatible with sequence segmentation.

[0027] In a further embodiment, the hybrid parallel inference method also includes a VAE decoding parallel step: during the video decoding stage, only the process with cfg_rank=0 and sp_rank=0 (i.e., rank=0) performs VAE decoding and video output, while other processes skip this step to avoid redundant calculations.

[0028] Example 1: 2-card CFG parallel operation (N=1) This embodiment provides a 2-card CFG parallel inference method based on the Ascend 910A NPU, applied to the Wan2.2-TI2V-5B video diffusion model.

[0029] Hardware environment: 2×Ascend 910A NPUs, 32GB HBM video memory; Software environment: CANN 9.0.0-beta.2, PyTorch 2.9.0, HCCL collection communication library.

[0030] The process topology is as follows: Total number of processes: 2; CFG dimensions: 2, cfg_rank=0 is responsible for conditional prediction, cfg_rank=1 is responsible for unconditional prediction; SP dimension: 1, N=1, no sequence partitioning; Rank mapping: rank0=(cfg=0, sp=0), rank1=(cfg=1, sp=0).

[0031] The process group is constructed as follows: SP Group: Each CFG dimension has only 1 process, and the SP group is a single-process group (no communication required). CFG group: contains rank 0 and rank 1, used for all_gather synchronization.

[0032] The execution flow of each noise reduction step is as follows: (1) Rank0 uses positive prompts to perform transformer forward propagation to obtain conditional prediction cond_pred; (2) Rank1 uses negative cue words to perform transformer forward propagation to obtain unconditional prediction uncond_pred; (3) The two ranks exchange prediction results using all_gather; (4) Calculate for each rank: noise_pred=uncond_pred+6.0×(cond_pred-uncond_pred); In this embodiment, the guidance strength coefficient guidance_scale is set to 6.

[0033] (5) Execute scheduler stepping and post-processing.

[0034] The performance benchmark (720P 121fps, 20-step denoising, TeaCache enabled) test results are as follows: Single-card noise reduction: 279.9 seconds; Parallel denoising using 2 CFG cards: 155.4 seconds; Speedup ratio: 1.80x.

[0035] Example 2: 4-card CFG+SP hybrid parallel (N=2) This embodiment provides a 4-card CFG+SP hybrid parallel inference method based on the Ascend 910A NPU.

[0036] Hardware environment: 4 Ascend 910A NPUs.

[0037] The process topology is as follows: Total number of processes: 4; CFG dimension: 2; SP dimension: 2; Rank mapping: - rank0 = (cfg = 0, sp = 0), conditional prediction, first half of the sequence; - rank1 = (cfg = 0, sp = 1), conditional prediction, second half of the sequence; - rank2 = (cfg = 1, sp = 0), unconditional prediction, first half of the sequence; - rank3 = (cfg=1, sp=1), unconditional prediction, second half of the sequence.

[0038] The process group is constructed as follows: - SP group 0: contains rank0 and rank1 (two SP processes with cfg=0); - SP group 1: contains rank2 and rank3 (two SP processes with cfg=1); - CFG group 0: contains rank0 and rank2 (two CFG processes with sp=0); - CFG group 1: contains rank1 and rank3 (two CFG processes with sp=1).

[0039] The execution flow of each noise reduction step is as follows: (1) Sequence segmentation: Each process holds 1 / 2 of the complete sequence; (2) Forward propagation of the converter: a. SP group all_to_all: Swap the header dimension and sequence dimension of query, key, and value; b. Each process performs attention calculations; c. SP group all_to_all: Restores the sequence distribution; d. Execute subsequent layers such as FFN; (3) sequence gather: all_gather within the SP group, splicing together the complete sequence; (4) CFG synchronization: all_gather within the CFG group, exchange conditional / unconditional prediction; (5) Calculate the final prediction result.

[0040] The performance benchmark (720P 121fps, 20-step noise reduction) test results are as follows: Single-card noise reduction: 280 seconds; 4-card CFG+SP noise reduction: 88.3 seconds; Speedup ratio: 3.17x (vs single card), 1.76x (vs 2-card CFG).

[0041] Example 3: 8-card CFG+SP hybrid parallel operation (N=4) This embodiment provides an 8-card CFG+SP hybrid parallel inference method based on the Ascend 910A NPU.

[0042] The process topology is as follows: Total number of processes: 8; CFG dimension: 2; SP dimension: 4; Rank mapping: - rank0-3: cfg=0, sp=0-3 (conditional prediction, sequence 1 / 4 segment × 4); - rank4-7: cfg=1, sp=0-3 (unconditional prediction, sequence 1 / 4 segment × 4); The process group is constructed as follows: SP group 0: rank 0, 1, 2, 3 (4 SP processes with cfg=0); SP Group 1: rank 4, 5, 6, 7 (4 SP processes with cfg=1). CFG groups 0-3: contain ranks {0,4}, {1,5}, {2,6}, and {3,7} respectively; The execution process for each noise reduction step is the same as in Example 2.

[0043] The performance benchmark (720P 121fps, 20-step noise reduction) test results are as follows: 8-card CFG+SP noise reduction: 50.0 seconds; Speedup ratio: 5.60x (vs single card), 1.77x (vs 4 cards).

[0044] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, alterations, substitutions, or variations made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. A hybrid parallel inference method based on the diffusion model CFG and SP using orthogonal process group topology, characterized by: Includes the following steps: Step S1: Arrange all processes in the NPU cluster into a two-dimensional grid topology, defining two orthogonal dimensions, CFG and SP, with a total number of processes of 2N, where N is the size of the SP dimension; Step S2: Assign two-dimensional coordinates (cfg_rank, sp_rank) to each process and establish a mapping relationship between the global ranking and the two-dimensional coordinates; cfg_rank is the CFG dimension coordinate and sp_rank is the SP dimension coordinate. Step S3: Construct two orthogonal communication process groups, CFG and SP, based on the two-dimensional mesh topology; Step S4: In each denoising step, each process holds 1 / N segments along the sequence dimension. cfg_rank determines whether to use conditional or unconditional cue words. Distributed attention computation is achieved within the SP group through all_to_all communication. Step S5: At the end of each denoising step, the CFG group synchronizes conditional prediction and unconditional prediction through all_gather communication and calculates the final prediction result.

2. The hybrid parallel inference method for diffusion model CFG and SP based on orthogonal process group topology as described in claim 1, characterized in that: The mapping relationship in step S2 is as follows: rank= cfg_rank×sp_degree + sp_rank Where rank represents the global ranking; sp_degree represents the parallelism of SP; cfg_rank∈{0,1}; sp_rank∈{0,1,...,N-1}.

3. The hybrid parallel inference method for diffusion model CFG and SP based on orthogonal process group topology as described in claim 1, characterized in that: In step S3, when constructing the communication process group, all processes call dist.new_group() to create the process group in the same order.

4. The hybrid parallel inference method for diffusion model CFG and SP based on orthogonal process group topology as described in claim 1, characterized in that: The distributed attention computation in step S4 includes: within the SP group, the head dimension and sequence dimension of the query, key, and value tensors are exchanged through the all_to_all operation, so that each process obtains the complete head number but a partial sequence of query, key, and value. After performing attention computation, the sequence distribution is restored through all_to_all.

5. The hybrid parallel inference method for diffusion model CFG and SP based on orthogonal process group topology according to claim 1, characterized in that: The formula for calculating the final prediction result in step S5 is as follows: noise_pred=uncond_pred+guidance_scale×(cond_pred-uncond_pred) Wherein, noise_pred represents the final prediction result for each process; uncond_pred represents the unconditional prediction result; cond_pred represents the conditional prediction result; and guidance_scale represents the guidance strength coefficient.

6. The hybrid parallel inference method for diffusion model CFG and SP based on orthogonal process group topology according to claim 1, characterized in that: Each process independently maintains its own TeaCache state, and the branch state is always 0.

7. The hybrid parallel inference method for diffusion models CFG and SP based on orthogonal process group topology as described in claim 1, characterized in that: During the video decoding stage, only the process with rank=0 performs VAE decoding and video output; other processes skip this step.

Images