Npu-oriented diffusion model attention calculation type perception dynamic scheduling method

Abstract

The present application belongs to the technical field of artificial intelligence reasoning optimization, and particularly relates to a diffusion model attention calculation type perception dynamic scheduling method for NPU. The specific technical scheme is as follows: when the reasoning service is started, the scaled_dot_product_attention function of PyTorch is globally replaced by the monkey-patch method, the attention type is identified by three-level judgment according to the shape parameters of the input tensor, the three-level judgment includes self-attention / cross-attention / sequential attention, the Transformer main network / other modules, and large sequence / small sequence, and the optimal calculation backend is dynamically selected according to the attention type identification result, the long sequence Transformer self-attention uses the NPU Flash Attention backend, and the short sequence and other types use the standard matrix multiplication+softmax backend. The problem that the performance of the Flash Attention backend on the NPU is significantly different under different sequence lengths is solved, and the method can be applied to various diffusion models without modifying the model source code.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence inference optimization technology, specifically involving a type-aware dynamic scheduling method for attention computation in diffusion models for NPU. Background Technology

[0002] Diffusion models are currently the mainstream generative artificial intelligence model architecture, widely used in tasks such as image generation and video generation. The core network structure of diffusion models is usually based on the Transformer architecture, which includes various types of attention computation, such as spatial self-attention, cross-attention, and temporal attention.

[0003] On NPU (Neural Processing Unit) hardware platforms, Flash Attention is a commonly used operator to accelerate attention computation. Taking the Ascend 910A NPU as an example, its `torch_npu._npu_flash_attention_unpad` operator can significantly accelerate the attention computation of long sequences by optimizing memory access patterns and reducing intermediate result storage. However, the performance of this operator varies significantly across different types of attention computation.

[0004] Currently, the vLLM-Ascend framework uses a global configuration approach (such as setting the environment variable `DIFFUSERS_ATTN_BACKEND=_chunked_npu`), unifying all attention computations to the same backend, which fails to account for the performance differences between different types of attention. The SGLang framework selects the backend at the attention layer level, based on static configuration, and does not support dynamic determination based on tensor shape at runtime. torch_npu provides Flash Attention API documentation but does not describe a type-aware scheduling strategy. None of these techniques address the issue of using different computation backends for different types of attention computations within the same model to optimize overall inference performance.

[0005] Furthermore, in models with Grouped Query Attention (GQA) architectures (such as ERNIE-Image), the number of attention heads for the key / value pairs is inconsistent with the number of attention heads for the query, requiring head number expansion and adaptation before computation. This adaptation requirement is independent of the attention backend selection, but existing technologies do not unify the two within a single scheduling framework. Summary of the Invention

[0006] To address the issue of adapting different types of attention computations to different computation backends within the same diffusion model, this invention aims to provide a type-aware dynamic scheduling method for attention computations in diffusion models that is designed for NPUs. When running a diffusion model on an NPU hardware platform, this method automatically identifies the attention type and dynamically selects the optimal computation backend based on the characteristics of different attention computation types (including sequence length, number of heads, and head dimension) to optimize overall inference performance.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a type-aware dynamic scheduling method for attention computation in a diffusion model for NPU, comprising the following steps: Step S1: When the inference service starts, globally replace the scaled_dot_product_attention function in PyTorch using Monkey-Patch to set the attention calculation interception point; Step S2: When a specific scheduling function is called, detect the device type of the input tensor query: if the device type is NPU, proceed to step S3; otherwise, call PyTorch's scaled_dot_product_attention function again and return the result directly. Step S3: Based on the shape parameters of the input tensor, a three-level judgment is used to identify the attention type. The three-level judgment includes self-attention judgment, Transformer backbone network judgment, and sequence size judgment. Step S4: Select the computation backend based on the type recognition result of step S3: If the attention computation is determined to be self-attention, Transformer backbone network, or large sequence, then select the NPU's Flash Attention backend and call the Flash Attention operator to perform the computation; otherwise, select the standard matrix multiplication + softmax backend and call the PyTorch standard scaled_dot_product_attention operator to perform the computation. Step S5: If the Flash Attention backend is selected but an exception occurs during the call, the exception is captured and a warning log is recorded. The system is then automatically switched to the standard matrix multiplication + softmax backend and the calculation is re-executed.

[0008] Preferably, the implementation process of the Monkey-Patch method is as follows: (1.1) Save the original torch.nn.functional.scaled_dot_product_attention function reference; (1.2) Define a specific scheduling function that includes the logic of steps S2-S5; (1.3) In a specific scheduling function, the original function saved in step (1.1) is called for non-NPU devices; (1.4) Assign a specific scheduling function to torch.nn.functional.scaled_dot_product_attention; (1.5) The above replacement is executed once when the inference service starts and takes effect globally for all subsequent inference requests.

[0009] Preferably, the three-level determination in step S3 includes: Step S3.1, Self-attention determination: Compare whether the shapes of the three input tensors, query, key, and value, are completely identical. If the shapes of the three are completely identical, it is determined to be self-attention; otherwise, it is determined to be other types of attention. Step S3.2, Transformer backbone network determination: Determine whether the number of attention heads is equal to the preset threshold for the number of heads, and determine whether the head dimension is equal to the preset threshold for the head dimension. If the number of attention heads is equal to the preset threshold for the number of heads and the head dimension is equal to the preset threshold for the head dimension, then it is determined to belong to the spatial self-attention layer of the Transformer backbone network; otherwise, it is determined to be the attention calculation of other modules. The preset thresholds for the number of heads and head dimensions are determined by the feature parameters of the diffusion model. For example, when using the Wan2.2-TI2V-5B Transformer, the preset threshold for the number of heads is 24, and the preset threshold for head dimensions is 128. Step S3.3, Sequence Size Determination: Calculate the total number of elements in the input tensor. If the total number of elements exceeds a preset threshold, it is determined to be a large sequence; otherwise, it is determined to be a small sequence. The preset threshold for the total number of elements is determined based on the performance intersection of FlashAttention on the NPU and standard computation. The formula for calculating the total number of elements is as follows: N = B × H × S × D h Where N represents the total number of elements in the input tensor; B represents the batch size; H represents the number of heads; S represents the sequence length; D h This represents the vector dimension of each token.

[0010] Preferably, in step S3.3, the method for determining the preset threshold for the total number of elements is as follows: (3.1) On the target NPU hardware, the time consumption of FlashAttention and standard computation was tested for self-attention computation of various sequence lengths; (3.2) Plot the time consumption-sequence length curves for the Flash Attention backend and the standard matrix multiplication + softmax backend, respectively; (3.3) Determine the sequence length corresponding to the intersection point of the two curves in step (3.2); (3.4) The total number of elements corresponding to the intersection points determined in step (3.3) is used as the preset threshold for the total number of elements.

[0011] Preferably, before performing step S3, it is checked whether the number of attention heads of the key / value input tensor is consistent with the number of attention heads of the query input tensor: If they match, proceed directly to step S3; If they are inconsistent, calculate the repetition multiple, and use the torch.repeat_interleave operation to expand the key / value pairs in the head dimension (dim=1) to match the number of query heads, and then proceed to step S3; The formula for calculating the repetition multiple is as follows: R=Q h / KV h Where R represents the repetition multiple; Q h Key-value pairs (KV) represent the number of attention heads for a query. h This indicates the number of attention heads for the key / value pair.

[0012] Preferably, the NPU uses the Ascend 910A processor, the Flash Attention operator is the torch_npu._npu_flash_attention_unpad operator, and the preset threshold for the total number of elements is 10M.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. By identifying attention types at runtime and selecting dynamic backends, the most suitable computational backend is used for all types of attention calculations in the diffusion model, significantly improving overall inference performance. Experimental results show that spatial self-attention achieves approximately 66% speedup using the FlashAttention backend, while cross-attention and temporal attention achieve approximately 23% and 17% speedup respectively using the standard backend.

[0014] 2. By using a three-level type determination (self-attention determination, Transformer backbone network determination, and sequence size determination), the type of attention calculation is accurately identified, avoiding the performance loss of a globally unified backend.

[0015] 3. By using an abnormal rollback mechanism, the system can still operate normally even when the Flash Attention operator is not supported under certain input shapes, thus improving the robustness of the scheduling method.

[0016] 4. Interception points are implemented using the monkey-patch method, without modifying the source code of the diffusion model. It has good versatility and portability and can be applied to any diffusion model based on PyTorch.

[0017] 5. Through the GQA adaptation step, it is compatible with grouped query attention architecture models (such as ERNIE-Image), thus expanding the applicability of the method.

[0018] 6. By using the performance crossover analysis method of sequence size threshold, the threshold can be optimized according to different NPU hardware platforms, thus having hardware adaptability. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0021] like Figure 1 As shown, the diffusion model attention computation type-aware dynamic scheduling method for NPU includes the following steps: Step S1: When the inference service starts, globally replace the scaled_dot_product_attention function in PyTorch using Monkey-Patch to set an attention calculation interception point. Specifically, define a specific scheduling function and assign it to torch.nn.functional.scaled_dot_product_attention so that all calls to attention calculation go through this specific scheduling function without modifying the source code of the diffusion model. Step S2: When a specific scheduling function is called (attention calculation interception point triggered), the device type of the input tensor query is detected: if the device type is NPU, the attention type identification logic in step S3 is executed; if the device type is not NPU, such as CPU or GPU, the original scaled_dot_product_attention function of PyTorch is called again and the result is returned directly. Step S3: Based on the shape parameters of the input tensor, a three-level judgment is used to identify the attention type. The three-level judgment includes self-attention judgment, Transformer backbone network judgment, and sequence size judgment. Step S4: Select the computation backend based on the type identification result of step S3: If all conditions are met, i.e., the attention computation is determined to be self-attention, Transformer backbone network, or large sequence, then select the NPU's Flash Attention backend and call the Flash Attention operator to perform the computation. For example, the NPU uses the Ascend 910A processor, and the Flash Attention operator is the torch_npu._npu_flash_attention_unpad operator; if none of the above conditions are met, then select the standard matrix multiplication + softmax backend and call the PyTorch standard scaled_dot_product_attention operator to perform the computation. Step S5: If an exception occurs during the Flash Attention backend call, such as an unsupported input shape or insufficient memory, the exception is captured and a warning log is recorded. The system then automatically switches to the standard matrix multiplication + softmax backend and re-executes the calculation. This exception fallback mechanism ensures that the system can still function normally when the Flash Attention operator does not support a specific input shape.

[0022] In a further implementation, the Monkey-Patch method is implemented in step S1 as follows: (1.1) Save the original torch.nn.functional.scaled_dot_product_attention function reference; (1.2) Define a specific scheduling function that includes the logic of steps S2-S5; (1.3) In a specific scheduling function, the original function saved in step (1.1) is called for non-NPU devices; (1.4) Assign a specific scheduling function to torch.nn.functional.scaled_dot_product_attention; (1.5) The above replacement is executed once when the inference service starts and takes effect globally for all subsequent inference requests.

[0023] In a further embodiment, the dynamic scheduling method disclosed in this application also includes a Group Query Attention (GQA) adaptation step: before step S3 is executed, the number of attention heads KV of the key / value input tensor is detected. hIs the number of attention heads Q related to the query input tensor? h Consistency: If they match, proceed directly to step S3; If there is a discrepancy (GQA architecture), calculate the repetition multiple R, and expand the key / value pairs in the head dimension (dim=1) to match the number of query heads using the torch.repeat_interleave operation, before proceeding to step S3.

[0024] The formula for calculating the repetition multiple is as follows: R=Q h / KV h Where R represents the repetition multiple; Q h Key-value pairs (KV) represent the number of attention heads for a query. h This indicates the number of attention heads for the key / value pair.

[0025] This GQA adaptation step makes the method applicable to models that employ grouped query attention architectures, such as ERNIE-Image-Turbo, thus expanding the method's applicability.

[0026] In a further implementation, the three-level determination in step S3 includes: Step S3.1, Self-attention determination: Compare whether the shapes of the three input tensors, query, key, and value, are completely identical. If the shapes of the three are completely identical, it is determined to be self-attention; if the shapes of the three are not completely identical, it is determined to be cross-attention or temporal attention. Step S3.2, Transformer backbone network determination: Determine whether the number of attention heads is equal to the preset threshold for the number of heads, and whether the head dimension is equal to the preset threshold for the head dimension. If the number of attention heads is equal to the preset threshold for the number of heads and the head dimension is equal to the preset threshold for the head dimension, then the attention calculation is determined to belong to the spatial self-attention layer of the Transformer backbone network; otherwise, it is determined to be the attention calculation of other modules, including VAE, conditional encoder, etc. For example, the preset threshold for the number of heads is 24, and the preset threshold for the head dimension is 128; Step S3.3, Sequence Size Determination: Calculate the total number of elements in the input tensor. If the total number of elements exceeds a preset threshold, it is determined to be a large sequence; otherwise, it is determined to be a small sequence. The preset threshold for the total number of elements is determined based on the performance intersection of FlashAttention on the NPU and standard computation. The formula for calculating the total number of elements is as follows: N = B × H × S × D h Where N represents the total number of elements in the input tensor; B represents the batch size; H represents the number of heads; S represents the sequence length; D h This represents the vector dimension of each token.

[0027] In a further implementation, the dynamic scheduling method disclosed in this application also includes a VAE attention-enforcing standard backend step: for the attention calculation in the variational autoencoder VAE module, by detecting tensor shape features (such as the number of heads not equal to 24 or the head dimension not equal to 128), it is automatically determined by step S3.2 to be a non-Transformer backbone network, thereby forcing the use of the standard matrix multiplication + softmax backend and not using the Flash Attention backend.

[0028] In a further implementation, the method for determining the preset threshold for the total number of elements in step S3.3 is as follows: (3.1) On the target NPU hardware, the time consumption of FlashAttention and standard computation was tested for self-attention computation of various sequence lengths; (3.2) Plot the time consumption-sequence length curves for the Flash Attention backend and the standard matrix multiplication + softmax backend; (3.3) Determine the sequence length corresponding to the intersection point of the two curves in step (3.2); (3.4) The total number of elements corresponding to the intersection points determined in step (3.3) is used as the preset threshold for the total number of elements.

[0029] For example, the NPU uses the Ascend 910A processor, the crossover point is located between 1000 and 1500 in sequence length, and the total number of elements is preset to a threshold of 10M (10,000,000).

[0030] It should be noted that the decision-making framework of this invention (steps S1 to S5) is independent of the specific NPU model and has universality: the Monkey-Patch interception mechanism in step S1 is based on the PyTorch framework and is applicable to any hardware platform; the device type detection in step S2 can automatically distinguish between NPU and non-NPU devices, with non-NPU devices directly reverting to standard computation; the three-level decision-making logic in step S3 is based on tensor shape features and does not depend on specific hardware; the backend selection in step S4 only requires the target hardware to provide equivalent Flash Attention operators and standard matrix multiplication operators for adaptation; the anomaly fallback mechanism in step S5 further enhances cross-hardware compatibility. The only hardware-specific aspect is the preset threshold for the total number of elements in step S3.3, which can be re-determined using the general cross-point measurement method described in step S3.3 (testing the time consumption curves of Flash Attention and standard computation under different sequence lengths on the target hardware), without modifying the decision-making framework. Similarly, the Flash Attention operator name in step S4 can also be replaced with the corresponding implementation based on the operator library of the target NPU. Example

[0031] This embodiment provides a type-aware dynamic scheduling method for attention computation in diffusion models for Ascend 910A NPU, applied to the inference service system of Wan2.2-TI2V-5B video diffusion model.

[0032] Hardware environment: Ascend 910A NPU, 32GB HBM video memory.

[0033] Software environment: CANN 9.0.0-beta.2, PyTorch 2.9.0, torch_npu 2.9.0.post1.

[0034] When the inference service starts, the diffusion model attention computation type-aware dynamic scheduling method disclosed in this invention for NPU is executed, and the performance benchmark test data obtained is shown in Table 1.

[0035] Table 1 Performance Benchmark Test Data

[0037] At this point, an example of a level-three judgment result is as follows: (a) Spatial self-attention at 720P resolution: The query shape is [1,24,3680,128], N=1×24×3680×128=11304960>10M, which is determined to be a large sequence. The Flash Attention backend is selected. The actual time taken is 6.11ms, which is 66% faster than the 17.76ms calculated by the standard backend.

[0038] (b) Cross attention: The query shape is [1,24,512,128], N=1×24×512×128=1572864<10M, which is determined to be a small sequence. The standard calculation backend is selected, and the actual time taken is 0.46ms, which is 23% faster than the 0.57ms calculation of the Flash Attention backend.

[0039] (c) Temporal attention: The query shape is [1,24,31,128], N=1×24×31×128=95232<10M, which is determined to be a small sequence. The standard calculation backend is selected, and the actual time taken is 0.37ms, which is 17% faster than the 0.44ms calculation of the Flash Attention backend.

[0040] Further analysis using the msprof hardware performance analysis tool revealed that the total forward propagation time for a single 720P transformer was 11.61 seconds, of which: -UnpadFlashAttention cumulative percentage: 38.4% (spatial self-attention path only); -ViewCopy (data migration) cumulative percentage: 30.7% (FA padding / unpadding data migration); -MatMul's cumulative share: 7.9%; -TransdataNdToNzKernel cumulative percentage: 0.39% (Nd Nz format conversion, not a bottleneck.

[0041] The fundamental reason for the aforementioned performance differences lies in the fact that the NPU's Flash Attention operator (`_npu_flash_attention_unpad`) requires padding / unpadding of variable-length sequence data. This data transfer operation (ViewCopy) accompanies each Flash Attention call (each FA call is accompanied by 4 ViewCopy operations, each ViewCopy taking approximately 30ms), while the actual Nd↔Nz format conversion (TransdataNdToNzKernel) only accounts for 0.39% of the cumulative time. In small sequence scenarios, the fixed overhead of padding / unpadding (a constant time independent of sequence length) is not negligible. When the computational acceleration brought by increasing sequence length is insufficient to offset this fixed overhead, the total time consumption of Flash Attention actually exceeds that of standard matrix multiplication + softmax computation. Experiments have determined that this performance crossover point is located between approximately 1000-1500 elements in the sequence. The preset threshold for the total number of elements in the sequence is 10M. Example

[0042] This embodiment provides the application of the method disclosed in this invention in the ERNIE-Image model, extending support for Grouped Query Attention (GQA). The specific implementation is as follows: (1) Model architecture features: The ERNIE-Image-Turbo model adopts the GQA (Grouped Query Attention) architecture, where the number of attention heads for the query is 32, and the number of attention heads for the key and value is 8, with a repetition factor of R=32÷8=4. This means that every 4 query heads share 1 key / value head, and the number of key / value heads needs to be expanded to match the number of query heads before attention calculation.

[0043] (2) GQA adaptation logic: When it is detected that the number of key / value heads is inconsistent with the number of query heads (GQA architecture), the number of key / value heads needs to be expanded to be consistent with the query. The specific judgment condition is: if key.shape[1] != query.shape[1], it is determined to be a GQA architecture, and the number of heads needs to be expanded for adaptation. The expansion method is to use the torch.repeat_interleave function to repeat the key / value tensor on the head dimension (dim=1) so that its number of heads is consistent with the query.

[0044] (3) Backend selection logic: After GQA adaptation is completed, the three-level type determination process is entered. For the ERNIE-Image model, since the number of heads of its Transformer structure is 32 (not equal to 24), the Transformer backbone network determination result in step S3.2 is negative. Therefore, all attention calculations select the standard matrix multiplication + softmax backend and do not use the FlashAttention backend.

[0045] (4) Compatibility: The GQA adaptation step ensures that the scheduling method is compatible with models of different attention architectures. For the Wan2.2 model (head_num=24, no GQA), it directly enters the third-level decision; for the ERNIE-Image model (head_num=32, GQA architecture), it first performs head number expansion and then enters the decision. Both models use the same scheduling framework and do not require modification of the model source code.

[0046] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, alterations, substitutions, or variations made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. A type-aware dynamic scheduling method for attention computation in a diffusion model for NPU, characterized by: The process includes the following: Step S1: When the inference service starts, globally replace the scaled_dot_product_attention function in PyTorch using Monkey-Patch to set the attention calculation interception point; Step S2: When the attention calculation interception point is triggered, check whether the input tensor is located on the NPU device. If so, proceed to step S3. Otherwise, re-call PyTorch's scaled_dot_product_attention function; Step S3: Based on the shape parameters of the input tensor, perform a three-level judgment to identify the attention type: Determine if it is self-attention by checking the shape consistency of the query / key / value pairs; Whether a network belongs to the Transformer backbone is determined by the number of attention heads and the head dimension. Determine whether the sequence size exceeds a preset threshold for the total number of elements by checking the total number of tensor elements. Step S4: Select the calculation backend based on the attention type recognition result: If it is a self-attention mechanism of the Transformer backbone network and the sequence size exceeds the preset threshold of the total number of elements, then select the NPUFlash Attention backend. Otherwise, choose standard matrix multiplication with a softmax backend; Step S5: If the Flash Attention backend call fails, it will automatically fall back to the standard matrix multiplication + softmax backend.

2. The diffusion model attention computation type-aware dynamic scheduling method for NPU as described in claim 1, characterized in that: The Monkey-Patch method in step S1 includes: saving the original scaled_dot_product_attention function reference, defining a specific scheduling function containing the scheduling logic of steps S2-S5, calling the original function for non-NPU devices in the specific scheduling function, and globally replacing the original function with the specific scheduling function.

3. The diffusion model attention computation type-aware dynamic scheduling method for NPU as described in claim 1, characterized in that: The three-level determination in step S3 includes: Step S3.1, Self-attention determination: Compare whether the shapes of the three input tensors, query, key, and value, are completely identical. If the shapes of the three are completely identical, it is determined to be self-attention; otherwise, it is determined to be other types of attention. Step S3.2, Transformer backbone network determination: Determine whether the number of attention heads is equal to the preset threshold for the number of heads, and determine whether the head dimension is equal to the preset threshold for the head dimension. If the number of attention heads is equal to the preset threshold for the number of heads and the head dimension is equal to the preset threshold for the head dimension, then it is determined to belong to the spatial self-attention layer of the Transformer backbone network; otherwise, it is determined to be the attention calculation of other modules. Step S3.3, Sequence Size Determination: Calculate the total number of elements in the input tensor. If the total number of elements exceeds a preset threshold, it is determined to be a large sequence; otherwise, it is determined to be a small sequence. The preset threshold for the total number of elements is determined based on the performance intersection of FlashAttention on the NPU and standard computation.

4. The diffusion model attention computation type-aware dynamic scheduling method for NPU as described in claim 1, characterized in that: In step S3, the preset threshold for the total number of elements is determined based on the performance intersection of Flash Attention on the NPU and standard computation. By testing the time consumption of the two backends for various sequence lengths on the target NPU hardware and plotting the time consumption-sequence length curves respectively, the total number of elements corresponding to the intersection of the two curves is taken as the preset threshold for the total number of elements.

5. The type-aware dynamic scheduling method for attention computation in a diffusion model for NPU as described in claim 1, characterized in that: Before step S3 is executed, it is checked whether the number of attention heads of the key / value input tensor is consistent with the number of attention heads of the query input tensor: If they match, proceed directly to step S3; If they are inconsistent, calculate the repetition multiple, and use the torch.repeat_interleave operation to expand the key / value pairs in the head dimension (dim=1) to match the number of query heads, and then proceed to step S3; The formula for calculating the repetition multiple is as follows: R=Q h / KV h Where R represents the repetition multiple; Q h Key-value pairs (KV) represent the number of attention heads for a query. h This indicates the number of attention heads for the key / value pair.

6. The type-aware dynamic scheduling method for attention computation in a diffusion model for NPU as described in claim 1, characterized in that: The NPU uses an Ascend 910A processor, and the Flash Attention operator is the torch_npu._npu_flash_attention_unpad operator. The preset threshold for the total number of elements is 10M.

7. The diffusion model attention computation type-aware dynamic scheduling method for NPU as described in claim 6, characterized in that: The preset threshold for the number of heads is 24; the preset threshold for head dimension is 128.

8. The type-aware dynamic scheduling method for attention computation in a diffusion model for NPU as described in claim 1, characterized in that: The attention calculation in the VAE module is forced to use standard matrix multiplication with a softmax backend.

Images