Diffusion model calculation method and device based on least square fitting and hardware unit

Abstract

The invention provides a diffusion model calculation method and device based on least square fitting and a hardware unit, and the method comprises the steps: generating differential calculation reference data through a least square fitting method based on cached previous frame input data, a previous frame calculation result and original input data of a current time step; performing structured sparse calculation according to the original input data of the current time step, the differential calculation reference data and the previous frame input data to generate an average error correction amount; on the basis of a linear layer of a diffusion model, the average error correction and differential calculation reference data are calculated to generate current frame data, and a high-similarity differential calculation reference is constructed through least square fitting, so that the quality and the stability of the reference data are improved; according to the method, the dynamic threshold value and the structured sparse calculation are combined, the calculation complexity is remarkably reduced on the premise that the model precision is guaranteed, and efficient and rapid calculation with low delay and low energy consumption can be continuously achieved in a modern diffusion model with the inter-frame difference increased.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus and hardware unit for calculating a diffusion model based on least squares fitting. Background Technology

[0002] In recent years, generative neural networks have been widely used in image generation, video generation, and multimodal modeling. Among them, diffusion models have become one of the mainstream generation schemes due to their outstanding performance in terms of generation quality and diversity. With the development of deep learning technology, the basic architecture of diffusion models has gradually evolved from U-Net-based convolutional neural networks to the Diffusion Transformer model (DiT) based on Transformers. However, the DiT model needs to repeatedly perform forward computation at multiple time steps during inference, resulting in a significant increase in inference latency and energy consumption, making it difficult to meet the application requirements of real-time or low-power scenarios.

[0003] To alleviate the aforementioned problems, an acceleration method based on differential computation has been proposed in the prior art. This method utilizes the similarity of input data between adjacent time steps, using the input of the previous time step and its corresponding layer output as a benchmark, and approximates the current output by calculating the difference between the current input and the previous input. Since the data changes little between adjacent time steps in traditional multi-step diffusion, and most elements in the difference are close to zero, this method can reduce the computational load by using low-precision computation or sparse computation, thereby improving computational efficiency.

[0004] However, the aforementioned difference calculation methods heavily rely on the high similarity between input data at adjacent time steps. With the development of diffusion model techniques, methods such as few-step iteration and model distillation have significantly reduced the number of sampling steps from over a hundred to less than twenty, leading to a substantial increase in data differences between adjacent time steps. In this situation, difference calculation methods directly based on the previous time step lose their significant sparsity or low-value characteristics due to the large difference between the baseline and the current input, resulting in a sharp decline in their speedup performance. They may even frequently degenerate into full-scale computation, failing to maintain an effective speedup in modern diffusion models with reduced iteration steps.

[0005] Therefore, existing technologies still lack an efficient acceleration solution that can effectively mine data correlations between iterations, maintain computational efficiency, and have hardware scalability even when time step differences increase significantly. Summary of the Invention

[0006] One objective of this invention is to provide a diffusion model calculation method based on least squares fitting. By constructing a high-similarity difference calculation benchmark through least squares fitting, the quality and stability of the benchmark data are effectively improved. Combined with dynamic thresholding and structured sparse computation, a significant reduction in computational complexity is achieved while ensuring model accuracy. This enables consistently low-latency, low-power, and efficient computation in modern diffusion models with increasing inter-frame differences. Another objective of this invention is to provide a hardware unit. A further objective is to provide a diffusion model calculation device based on least squares fitting. A further objective is to provide a computer-readable medium. A final objective is to provide a computer device.

[0007] To achieve the above objectives, this invention discloses a diffusion model calculation method based on least squares fitting, comprising:

[0008] Based on the cached previous frame input data, previous frame calculation results, and the original input data of the current time step, the difference calculation benchmark data is generated by the least squares fitting method.

[0009] Structured sparse computation is performed based on the original input data of the current time step, the differential calculation baseline data, and the input data of the previous frame to generate the average error correction amount;

[0010] The linear layer based on the diffusion model calculates the average error correction and the differential calculation baseline data to generate the current frame data, which includes the current frame input data and the current frame calculation result.

[0011] Preferably, based on the cached previous frame input data, the previous frame calculation results, and the original input data of the current time step, the difference calculation benchmark data is generated using the least squares fitting method, including:

[0012] By constructing an optimization objective function, least squares fitting coefficients are generated based on the input data of the previous frame and the original input data of the current time step.

[0013] Based on the least squares fitting coefficients, the previous frame input data, and the previous frame calculation results, differential calculation benchmark data is generated.

[0014] Preferably, structured sparse computation is performed based on the original input data of the current time step, the differential calculation baseline data, and the input data of the previous frame to generate an average error correction amount, including:

[0015] A data difference matrix is ​​generated based on the baseline data calculated using the difference calculation and the original input data at the current time step.

[0016] The data difference matrix is ​​grouped and column-by-column structured sparse computation is performed to generate sparse discrimination index;

[0017] Based on the sparse discrimination index, the original input data at the current time step, and the input data of the previous frame, the data difference matrix is ​​grouped and row-by-row structured sparse computation is performed to generate the average error correction amount.

[0018] Preferably, based on the sparse discrimination index, the original input data at the current time step, and the input data of the previous frame, the data difference matrix is ​​grouped and processed row-by-row in a structured sparse manner to generate the average error correction, including:

[0019] A dynamic threshold is generated based on the original input data at the current time step and the input data of the previous frame.

[0020] Based on sparse discrimination index and dynamic threshold, structured sparse discrimination is performed on the data difference matrix to generate a sparse mask;

[0021] Based on sparse masking, the data difference matrix is ​​grouped and structured sparsely computed row by row to generate the average error correction amount.

[0022] Preferably, the differential calculation benchmark data includes differential calculation benchmark input data and differential calculation benchmark similarity results;

[0023] Based on the linear layer of the diffusion model, the average error correction and differential calculation baseline data are calculated to generate the current frame data, including:

[0024] The current frame input data is generated based on the average error correction and the differential calculation reference input data.

[0025] Based on the linear layer of the diffusion model, the benchmark similarity results and average error correction are calculated to generate the calculation results for the current frame.

[0026] Preferably, the method further includes:

[0027] The process involves determining the current frame input data as the previous frame input data, determining the current frame calculation result as the previous frame calculation result, determining the input data for the next time step as the original input data for the current time step, and continuing to execute the steps of generating differential calculation benchmark data using the least squares fitting method based on the cached previous frame input data, previous frame calculation result, and original input data for the current time step.

[0028] This invention also discloses a hardware unit applied to the above-mentioned least-squares fitting-based diffusion model calculation method, the hardware unit comprising:

[0029] Multiple vector subtractors, multiple vector multipliers, vector absolute value generators, multiple addition trees, maximum value trees, multiple adders, multiple multipliers, comparators, and dividers.

[0030] This invention also discloses a diffusion model computation device based on least squares fitting, applied to the aforementioned hardware unit, comprising:

[0031] The benchmark data calculation module is used to generate differential benchmark data based on the cached previous frame input data, previous frame calculation results and the original input data of the current time step through the least squares fitting method.

[0032] The structured sparse computation module is used to perform structured sparse computation based on the original input data of the current time step, the differential calculation reference data, and the input data of the previous frame, and generate the average error correction amount.

[0033] The linear calculation module is used for linear layers based on the diffusion model to calculate the average error correction and differential calculation baseline data, and generate the current frame data, which includes the current frame input data and the current frame calculation results.

[0034] The present invention also discloses a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0035] The present invention also discloses a computer device, including a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, wherein the processor executes the program to implement the method described above.

[0036] The present invention also discloses a computer program product, including a computer program / instructions, which, when executed by a processor, implement the method described above.

[0037] This invention generates differential calculation benchmark data based on cached previous frame input data, previous frame calculation results, and the original input data of the current time step using a least-squares fitting method. It then performs structured sparse computation based on the original input data of the current time step, the differential calculation benchmark data, and the previous frame input data to generate an average error correction. Using a linear layer of the diffusion model, it calculates the average error correction and the differential calculation benchmark data to generate the current frame data, which includes the current frame input data and the current frame calculation results. A high-similarity differential calculation benchmark is constructed using least-squares fitting, effectively improving the quality and stability of the benchmark data. By combining dynamic thresholding and structured sparse computation, it significantly reduces computational complexity while ensuring model accuracy, enabling consistently low-latency, low-energy, and efficient computation in modern diffusion models where inter-frame differences are increasing. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram of the hardware architecture of a diffusion model calculation system based on least squares fitting provided for an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the hardware circuit of a multi-mode least squares fitting and sparse preprocessing unit provided in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a least squares fitting coefficient calculation path provided in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of a computational pathway for generating difference calculation benchmark data based on least squares fitting coefficients, provided by an embodiment of the present invention.

[0043] Figure 5 A schematic diagram of a calculation path for the sum of the absolute values ​​of the differences between the current input data frame and the previous data frame, provided for an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of a computational path for generating grouped column-wise sparse matrices and grouped row-wise sparse matrices provided in an embodiment of the present invention;

[0045] Figure 7 A flowchart illustrating a diffusion model calculation method based on least squares fitting, provided in an embodiment of the present invention;

[0046] Figure 8 A flowchart illustrating another diffusion model calculation method based on least squares fitting provided in an embodiment of the present invention;

[0047] Figure 9 A schematic diagram of the structure of a diffusion model calculation device based on least squares fitting provided in an embodiment of the present invention;

[0048] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] To facilitate understanding of the technical solutions provided in this application, the relevant content of the technical solutions in this application will be explained below. This invention proposes a data similarity mining method based on least squares fitting, which can construct a high-similarity benchmark even when adjacent time steps have large differences; a structured sparse computation method based on dynamic thresholds, which utilizes the tolerance of neural networks to residual noise in the residuals constructed based on least squares fitting and the actual input, only performing correction calculations on elements exceeding the threshold; this invention proposes a hardware architecture that supports the above computational process, which can efficiently execute loads such as least squares fitting and sparse matrix multiplication, ultimately accelerating the computation of the diffusion model.

[0051] Figure 1 A schematic diagram of the hardware architecture of a diffusion model calculation system based on least squares fitting is provided for an embodiment of the present invention, as shown below. Figure 1 As shown, the diffusion model computation system based on least squares fitting includes: a pulsating array, a vector computation unit, a nonlinear function unit, on-chip storage, a controller, and a multi-mode least squares fitting and sparse preprocessing unit.

[0052] The systolic array consists of multiple computation subarrays interconnected in a 2D mesh topology. Adjacent computation subarrays are connected from left to right and from top to bottom via registers. Each computation subarray receives two types of input data horizontally: data from the storage units of the input data buffer and data passed from the computation subarray to its left. These two types of data are processed by a 2-to-1 multiplexer before being input to the computation subarray. Vertically, each computation subarray receives two types of input data: data from the storage units of the weight buffer and data passed from the computation subarray above it. These two types of data are processed by a 2-to-1 multiplexer before being input to the computation subarray. Each computation subarray consists of 16×16 computation units interconnected in a 2D mesh topology. Each computation unit contains one multiply-accumulate (MAC) unit, one output resident register, and two pipeline registers. Each computation unit receives output data from the computation units to its left and above, and outputs the data through the pipeline registers to the computation units to its right and bottom, respectively, thus achieving systolic computation. During each cycle, the computation unit multiplies the data from the computation units on the left and top, adds it to the data from the output resident register, and writes it back to the output resident register.

[0053] The controller controls the functions and operating status of each module: the controller issues the 2-to-1 multiplexer control signals between computational subarrays, enabling flexible adaptation to various matrix multiplication scales. The broadcast unit in the input data buffer receives control signals from the controller, enabling data copying between input data storage units; the nonlinear function unit receives control signals from the controller, enabling the processing of various nonlinear functions; the multi-mode least squares fitting and sparse preprocessing unit receives control signals from the controller, enabling switching between various operating modes; and the output data accumulator module receives control signals from the controller, enabling the accumulation of output data from the computational array.

[0054] The metadata cache unit is used to store key data that supports the computational model proposed in this invention, including least squares fitting coefficients, sparse mask thresholds, etc.

[0055] The weight cache is used to store the weight data of the linear layer. The weight cache internally consists of multiple weight storage units. Each weight storage unit corresponds to a column of computation subarrays.

[0056] The input data buffer stores the input data for the linear layer. Internally, the input data buffer consists of multiple input data storage units and one broadcast unit. Each input data storage unit corresponds to a row of computation subarrays. The broadcast unit is connected to all input data storage units and can write data read from off-chip memory to multiple input data storage units in copy mode according to control signals.

[0057] Nonlinear function units are used to execute nonlinear activation functions in diffusion models, including Softmax, SiLU, GeLU, etc.

[0058] The output data accumulator consists of a storage unit and an adder. It receives resident register data from each computational unit within each computational subarray and accumulates the results of multiple batches of matrix multiplication operations using the adder.

[0059] The multi-mode least squares fitting and sparse preprocessing unit is used to perform the least squares fitting calculation, benchmark input data generation, grouped column-by-column sparse, and grouped row-by-row sparse matrix generation disclosed in this invention. By reusing the data paths of each calculation process, hardware overhead can be reduced and system energy efficiency can be improved.

[0060] In the technical solution provided by the embodiments of the present invention, the multi-mode least squares fitting and sparse preprocessing unit can efficiently support the least squares fitting calculation process, grouped column-by-column sparse matrix generation, and grouped row-by-row sparse matrix generation processes proposed in the present invention, which is more efficient than general hardware calculation. The systolic array of the present invention can realize efficient calculation of grouped column-by-column sparse matrix and grouped row-by-row sparse matrix by controlling the selection state of the two-to-one multiplexer, which is more efficient than general systolic array.

[0061] Figure 2 This is a schematic diagram of a hardware unit for a multi-mode least squares fitting and sparse preprocessing unit provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the hardware unit is specifically a multi-mode least squares fitting and sparse preprocessing unit. The hardware unit includes: three vector subtractors (vector subtractor 0, vector subtractor 1, and vector subtractor 2), two vector multipliers (vector multiplier 0 and vector multiplier 1), one vector absolute value generator (for calculating the absolute value of vectors), two addition trees (addition tree 0 and addition tree 2), one addition tree / maximum tree 1, three adders (adder 0, adder 1, and adder 2), two multipliers (multiplier 0 and multiplier 1), one comparator, one divider, and several registers and multiplexers. This hardware unit supports multiple computation modes, and by constructing reusable computation paths, hardware overhead can be reduced. The hardware unit can receive previous frame input data, current input data, least squares fitting coefficient α, and grouped column-by-column sparse threshold; it can output the baseline data of least squares fitting, grouped column-by-column sparse matrix, grouped row-by-row sparse matrix, least squares fitting coefficient α, and the sum of the absolute values ​​of the differences between the current input data and the previous frame input data.

[0062] The vector subtractor takes two vectors and performs a subtraction operation; each vector contains 16 data points. The vector multiplier takes two vectors and performs a multiplication operation; each vector contains 16 data points. The vector absolute value calculator takes one vector and performs an absolute value operation; each vector contains 16 data points. The addition tree takes one vector (16 elements), performs a merge addition operation, and outputs a scalar data point. The addition / maximum tree 1 takes one vector (16 elements), performs a merge addition or merge maximum operation, and outputs a scalar data point. This unit contains both an adder and a maximum value generator at each merge node, allowing for both operations.

[0063] Figure 3 A schematic diagram of a least squares fitting coefficient calculation path provided in an embodiment of the present invention is shown below. Figure 3 As shown, when calculating the least squares fitting coefficients, both the previous frame input data and the current input data are divided into multiple groups, each containing 16 rows of data. Each group of data is input column by column into the calculation unit. First, the previous frame data x... t+2 or f(x) t+2 ), x t+1 or f(x) t+1 The result is calculated by vector subtractor 0 and then transmitted to vector multiplier 0 for multiplication; simultaneously, x t+1 and x tAfter vector subtractor 2 performs the calculation, the result is transmitted to vector multiplier 1. Vector multiplier 1 simultaneously receives the output of vector subtractor 0 and performs vector multiplication. Addition / maximum tree 1 then operates in addition tree logic (addition tree 1). Addition trees 1 and 2 merge vectors into scalars, and the data accumulation operation is performed by adder 0, adder 2, and the resident register, respectively. After all the data in the set is input to the calculation unit, the divider performs the division operation and outputs the least squares fitting coefficient α corresponding to the set of data.

[0064] Figure 4 This invention provides a schematic diagram of a computational pathway for generating difference calculation benchmark data based on least squares fitting coefficients, as shown in the embodiment of the invention. Figure 4 As shown, the preceding frame data x t+2 or f(x) t+2 ), x t+1 or f(x) t+1 The least squares fitting coefficient α and the output of vector subtractor 0 are then transferred to vector multiplier 0 for calculation. Simultaneously, the output of vector subtractor 0 is transferred to vector multiplier 1 for multiplication. Finally, the output of vector multiplier 0 and vector subtractor 1 are transferred to vector subtractor 1 for calculation, and the difference calculation benchmark data based on the least squares fitting coefficient is output.

[0065] Figure 5 This invention provides a schematic diagram of a calculation path for the sum of the absolute values ​​of the differences between the current input data frame and the previous data frame, as shown in the embodiment of the invention. Figure 5 As shown, the preceding frame data x t+1 or f(x) t+1 ) and the current input data x t The data is transmitted to vector subtractor 2 for subtraction, then processed by vector absolute value generator; the result is input to adder tree 1 for merge addition; the result is input to adder 0 and resident register to complete data accumulation, obtaining the sum of the absolute values ​​of the differences between the current input data frame and the previous data frame (sum|x). t - x t+1 |).

[0066] Figure 6 This is a schematic diagram of a computational path for generating grouped column-wise sparse matrices and grouped row-wise sparse matrices, provided by an embodiment of the present invention. Figure 6 As shown, the preceding frame data x t+2 or f(x) t+2 ), x t+1 or f(x) t+1The vector subtractor 0 performs the calculation, and the result is then passed to the vector multiplier 0 for multiplication. Simultaneously, the result from the vector subtractor 0 is input to the vector multiplier 1 for multiplication, and the result is also input to the vector subtractor 1. The result from the vector subtractor 1 is then compared with x. t After calculation by vector subtractor 2, a grouped column-wise sparse matrix is ​​output. The calculation result is transmitted to the vector absolute value unit and adder tree 0. The calculation result is input into multiplier 1 to perform multiplication operation. The vector absolute value unit transmits the result to the maximum value tree 1 and adder tree 2 respectively. The calculation is performed by multiplier 0 and adder 1. The calculation result is transmitted to the comparator to be compared with the dynamic threshold and output as a grouped row-wise sparse matrix.

[0067] This invention constructs a reusable computational pathway, which can simultaneously support the solution of least squares fitting optimization problems and the input data for difference calculation. The calculation and difference calculation benchmark similarity results It has functions such as calculation, generation of column-by-column sparse mask, and calculation of row-by-row sparse vector.

[0068] In the technical solution provided by the embodiments of the present invention, the data selection function of the multiplexer can realize the reuse of data paths between multiple computing modes, thereby realizing the various computing operations proposed in the present invention with lower hardware overhead.

[0069] It is worth noting that, Figure 1 The diffusion model calculation system based on least squares fitting shown is as follows: Figures 2 to 6 The hardware units and computing pathways shown are also applicable to Figure 7 or Figure 8 The calculation method for the diffusion model based on least squares fitting will not be elaborated here.

[0070] The following example uses a hardware unit as the execution subject to illustrate the implementation process of the diffusion model calculation method based on least squares fitting provided in this embodiment of the invention. It is understood that the execution subject of the diffusion model calculation method based on least squares fitting provided in this embodiment of the invention includes, but is not limited to, a hardware unit.

[0071] Figure 7 A flowchart illustrating a diffusion model calculation method based on least squares fitting, as provided in this embodiment of the invention, is shown below. Figure 7 As shown, the method includes:

[0072] Step 101: Based on the cached previous frame input data, previous frame calculation results, and the original input data of the current time step, generate differential calculation benchmark data using the least squares fitting method.

[0073] In this embodiment of the invention, the calculation process of a certain linear layer in a neural network model can be expressed as f(x) = w×x+b. This invention caches the input data x from two preceding frames of this linear layer. t+1 x t+2 And the corresponding output of the preceding frame calculation result f(x) t+1 f(x) t+2 This involves constructing an optimization objective function to optimize the original input data x at the current time step. t Fitting.

[0074] In this embodiment of the invention, the differential calculation benchmark data includes differential calculation benchmark input data and differential calculation benchmark similarity results. Since the optimization objective function is linear computation, calculating the differential calculation benchmark similarity results does not require actual matrix multiplication. Instead, it is obtained through linear computation based on the preceding frame data, allowing for the construction of a higher similarity differential calculation benchmark with lower computational overhead.

[0075] Step 102: Perform structured sparse computation based on the original input data of the current time step, the differential calculation reference data, and the input data of the previous frame to generate the average error correction amount.

[0076] In this embodiment of the invention, the difference calculation benchmark input data is constructed based on the least squares method. Based on this, differential calculations are performed. Through preliminary experiments, this invention found that the diffusion model has strong tolerance for numerical approximation calculations. Based on this, this invention further introduces sparse calculations on the basis of differential calculations to reduce the amount of computation.

[0077] In this embodiment of the invention, a dynamic threshold for sparse computation is constructed and used in conjunction with a sparse discrimination index calculated by a group-based column-wise structured sparse method to determine ignorable elements in the data difference matrix. Based on the column-wise sparse computation, a group-based row-wise structured sparse method is applied to the data difference matrix based on the original input data of the current time step, the differential computation benchmark data, and the input data of the previous frame. This further reduces the error introduced by the sparse approximation computation with a lower computational load and generates an average error correction amount.

[0078] Step 103: Based on the linear layer of the diffusion model, calculate the average error correction amount and the differential calculation reference data to generate the current frame data.

[0079] In this embodiment of the invention, the current frame data includes the current frame input data and the current frame calculation result.

[0080] Specifically, the current frame input data is generated based on the average error correction amount and the differential calculation benchmark input data; the linear layer based on the diffusion model generates the current frame calculation result based on the average error correction amount and the differential calculation benchmark data.

[0081] In the technical solution provided by this invention, based on the cached previous frame input data, previous frame calculation results, and the original input data of the current time step, differential calculation benchmark data is generated using the least squares fitting method. Structured sparse calculation is performed based on the original input data of the current time step, the differential calculation benchmark data, and the previous frame input data to generate an average error correction. Based on the linear layer of the diffusion model, the average error correction and the differential calculation benchmark data are calculated to generate the current frame data, which includes the current frame input data and the current frame calculation results. A high-similarity differential calculation benchmark is constructed using least squares fitting, effectively improving the quality and stability of the benchmark data. Combining dynamic thresholding and structured sparse calculation, a significant reduction in computational complexity is achieved while ensuring model accuracy. This enables consistently low-latency, low-energy, and efficient fast computation in modern diffusion models where inter-frame differences are increasing.

[0082] Figure 8 A flowchart of another diffusion model calculation method based on least squares fitting provided in an embodiment of the present invention is shown below. Figure 8 As shown, the method includes:

[0083] Step 201: By constructing the optimization objective function, generate least squares fitting coefficients based on the input data of the previous frame and the original input data of the current time step.

[0084] In this embodiment of the invention, each step is executed by a hardware unit.

[0085] In this embodiment of the invention, two preceding frame input data are used to solve for the least squares fitting coefficients through an optimization objective function. The two preceding frame input data are the first preceding frame input data x from the previous time step. t+1 The second predecessor frame input data x from the previous two time steps t+2 .

[0086] Specifically, the objective function for optimization is:

[0087]

[0088] Where, x t+1 Input data for the first preceding frame; x t+2 Input data for the second preceding frame; These are the least squares fitting coefficients, and the least squares fitting coefficients are rational numbers; This is the raw input data for the current time step.

[0089] This invention utilizes the data from the first two frames to predict a reference direction that is closest to the current input through least-squares fitting, rather than passively relying on a single previous frame. This fundamentally improves the quality and stability of the constructed reference when there are large differences between adjacent time steps, laying a reliable foundation for subsequent efficient difference calculations.

[0090] Step 202: Generate differential calculation baseline data based on the least squares fitting coefficients, the previous frame input data, and the previous frame calculation results.

[0091] In this embodiment of the invention, the differential calculation benchmark data includes differential calculation benchmark input data and differential calculation benchmark similarity results; the differential calculation benchmark similarity results are solved using the calculation results of two preceding frames, where the two preceding frame calculation results are respectively the calculation results of the first preceding frame. Calculation results of the second preceding frame .

[0092] Specifically, based on the least squares fitting coefficient of this parameter, a high-similarity difference calculation benchmark and the result of this benchmark passing through a linear layer can be constructed:

[0093]

[0094]

[0095] in, Input data as the baseline for differential calculation; x t+1 Input data for the first preceding frame; x t+2 Input data for the second preceding frame; These are the least squares fitting coefficients; The results are based on the benchmark similarity calculated using the difference method. The result is calculated for the first preceding frame; This is the result calculated for the second preceding frame.

[0096] This invention obtains differential calculation baseline data with extremely low computational cost (only two scalar multiplications and one vector addition), completely avoiding redundant matrix operations to obtain baseline output, which is the key to achieving computational acceleration.

[0097] Step 203: Generate a data difference matrix based on the differential calculation baseline data and the original input data of the current time step.

[0098] In this embodiment of the invention, a vector subtraction operation is performed based on the differential calculation baseline input data and the original input data at the current time step:

[0099] △x = x t - x base

[0100] Where △x is the data difference matrix; x t x represents the raw input data for the current time step. base Input data as the baseline for differential calculation.

[0101] In this embodiment of the invention, due to x base The high-quality baseline data is obtained by fitting with the least squares method. Therefore, the overall value of Δx is expected to be smaller than that of the traditional method (directly subtracting the previous frame), implying higher sparsity.

[0102] Step 204: Perform grouped, column-by-column structured sparse computation on the data difference matrix to generate sparse discrimination index.

[0103] In this embodiment of the invention, the token dimension (rows) of the data difference matrix △x is divided into multiple groups based on a specified number of tokens, resulting in multiple groups of difference numerator matrices. As an optional approach, the groups are divided into sets of 16 tokens each. Assuming △x has T tokens, it is divided into G = T / 16 groups.

[0104] In this embodiment of the invention, a sparsity discrimination index is calculated for each channel dimension of each set of difference numerator matrices:

[0105]

[0106] in, Let be the sparsity index of the i-th channel in the difference numerator matrix of the g-th group (g=1,2,...,G); Let be the data corresponding to the i-th channel of the g-th group difference numerator matrix in the data difference matrix Δx. If the specified number of tokens is 16, then... It is a 16×1 vector.

[0107] In this embodiment of the invention, the sparsity discrimination index comprehensively reflects the maximum and average deviation of the channel in the set of difference numerator matrices.

[0108] This invention transforms global sparsity judgment into local (grouping, channeling) judgment. The sparsity discrimination index, as a comprehensive indicator, can effectively characterize the degree of deviation of the local data block from the benchmark. Grouping processing creates conditions for the subsequent generation of structured sparse masks (at the group and channel granularity), which is beneficial for efficient and well-organized data loading and computation scheduling in hardware.

[0109] Step 205: Based on the sparse discrimination index, the original input data of the current time step and the input data of the previous frame, perform grouped row-by-row structured sparse calculation on the data difference matrix to generate the average error correction amount.

[0110] In this embodiment of the invention, step 205 specifically includes:

[0111] Step 2051: Generate a dynamic threshold based on the original input data of the current time step and the input data of the previous frame.

[0112] In this embodiment of the invention, an adaptive dynamic threshold is calculated based on the change magnitude of the preceding frame data:

[0113]

[0114] in, For dynamic thresholds; x t x represents the raw input data for the current time step. t+1 The input data is the first preceding frame; N is x. t The number of elements; This is a hyperparameter that applies to all linear layers in the DiT model that use difference computation.

[0115] It's worth noting that as the computation of input data progresses at different time steps, the input data from preceding frames also iterates continuously, and the threshold is dynamically updated. This allows the sparsity assessment to be automatically adjusted based on the characteristics of the input data itself (the magnitude of the difference between adjacent frames). When the differences between frames are generally small, the threshold is lowered, allowing more subtle changes to be ignored; when the differences increase, the threshold is raised, retaining more key difference information for computation. This automatically maintains a balance between accuracy and efficiency in changing scenarios, achieving unified support for the wide range of data differences between different time steps.

[0116] Step 2052: Based on the sparse discrimination index and dynamic threshold, perform structured sparse discrimination on the data difference matrix to generate a sparse mask.

[0117] In this embodiment of the invention, the sparsity discrimination index and dynamic threshold of each channel in each group of difference numerator matrices are compared. If the sparsity discrimination index of the i-th channel in the g-th group of difference numerator matrices is... Less than the dynamic threshold This indicates that the data in the ith channel of the difference numerator matrix of the g-th group is not significantly different from the baseline data. Ignoring the calculation of this column will not affect the model results, so a sparse mask of 0 is generated for this position; if the sparse discrimination index of the ith channel of the difference numerator matrix of the g-th group is... Greater than or equal to the dynamic threshold This indicates that the data in the i-th channel of the difference numerator matrix of the g-th group is significantly different from the baseline data, affecting the calculation accuracy. It is necessary to perform calculations to generate a sparse mask of 1 for this position.

[0118] Furthermore, the sparse masks at all positions are combined to obtain the sparse mask matrix corresponding to the data difference matrix.

[0119] The sparse mask generated by this invention, with groups and channels as granularity, clearly identifies which parts of the data difference matrix Δx can be safely skipped in computation. This structured mask is highly beneficial for hardware (such as systolic arrays) to perform regular zero-value skipping and data flow control, and is a direct basis for reducing computational load. By constructing a sparse mask with column-level granularity, a trade-off can be achieved between structuring matrix multiplication load and computational efficiency.

[0120] Step 2053: Based on the sparse mask, perform grouped row-by-row structured sparse computation on the data difference matrix to generate the average error correction amount.

[0121] In this embodiment of the invention, the vector data corresponding to the positions where the sparse mask is 0 is filtered out, and then... The selected vector data is then calculated to generate the corresponding average error correction. Let be the average error correction amount corresponding to the vector data of the i-th channel in the g-th group difference numerator matrix, and be a scalar; This represents the vector data of the i-th channel in the g-th group difference numerator matrix.

[0122] In this embodiment of the invention, the average error correction amount corresponding to the vector data at the position where the sparse mask is 0 is used for error compensation of the corresponding data in subsequent data. Although the differences of a certain channel within a certain token group are ignored as a whole, its average deviation... It still exists. By recording and subsequently compensating for the average error correction, the error introduced by the sparse approximation is further reduced with lower computational cost, while simultaneously achieving computational simplification and accuracy assurance.

[0123] Step 206: Generate the current frame input data based on the average error correction amount and the differential calculation reference input data.

[0124] In this embodiment of the invention, by The average error correction and the differential calculation baseline input data are added together to generate the current frame input data. Input data for the current frame; Let be the sparse mask for the i-th channel in the g-th group of difference numerator matrices; x is the average error correction amount corresponding to the vector data of the i-th channel in the g-th difference numerator matrix; base Input data as the baseline for differential calculation.

[0125] In this embodiment of the invention, the current frame input data is the input data of the current time step after an approximate correction. It is closer to the original input data of the current time step than the original differential calculation benchmark input data, but avoids performing a complete calculation for all subtle differences. This corrected data is stored in a cache and can be used as higher-quality preceding frame data as a benchmark for future time steps, forming a virtuous cycle.

[0126] Step 207: Based on the linear layer of the diffusion model, calculate the benchmark similarity results and average error correction amount of the differential calculation to generate the calculation results of the current frame.

[0127] Specifically, through The calculation of the differential calculation benchmark similarity result and the average error correction amount is performed to generate the calculation result for the current frame. The result is calculated for the current frame; The difference calculation represents the baseline similarity result; w is the weight matrix of the linear layer of the diffusion model; M is the sparse mask matrix; Δx represents the average error correction for the vector data of the i-th channel in the g-th difference matrix; Δx is the data difference matrix.

[0128] In this embodiment of the invention, The sparse mask matrix M represents element-wise multiplication, ensuring that all positions marked as 0 in the data difference matrix Δx are zeroed before computation. Therefore, the systolic array actually only performs matrix multiplication on the non-zero blocks with a sparse mask of 1, significantly reducing computational cost. For data with a sparse mask of 0, multiplying its corresponding weight matrix by the corresponding average error correction amount corrects the baseline similarity result of the difference calculation. This is a scalar multiplication of a vector operation, with computational overhead far less than matrix multiplication.

[0129] This invention replaces the traditional full matrix multiplication with a three-stage computation approach: high-quality benchmark, sparsity correction, and low-overhead compensation. Even with increasing inter-frame differences, it maintains high sparsity through the high-quality benchmark, thereby significantly reducing the computational complexity and hardware power consumption of general matrix multiplication while ensuring generation quality.

[0130] Step 208: Determine the current frame input data as the previous frame input data, determine the current frame calculation result as the previous frame calculation result, determine the input data of the next time step as the original input data of the current time step, and continue to execute step 201.

[0131] In this embodiment of the invention, the current frame input data is written into the buffer as the previous frame input data, and the current frame calculation result is written into the buffer as the previous frame calculation result; the input data of the next time step is received, and the input data of the next time step is used as the original input data of the current time step, and step 201 is continued to be executed to enter the next calculation loop.

[0132] As an optional approach, the first preceding frame data x t+1 As the second preceding frame data x t+2 The current frame input data is used as the first preceding frame data x. t+1 ;Calculate the result of the first preceding frame As the calculation result of the second preceding frame The calculation result of the current frame is used as the calculation result of the first preceding frame. .

[0133] This invention enables the pipelined and continuous operation of the algorithm. By continuously updating the cache, the system always retains the valid information of the two most recent frames, allowing the least squares fitting to be dynamically adjusted based on the latest data. This allows the entire acceleration method to adapt to the continuous changes in data distribution during the diffusion model inference process, ensuring long-term stability and acceleration performance.

[0134] It is worth noting that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. The user information in the embodiments of this application was obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been authorized and agreed upon by the client.

[0135] It is worth noting that the information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0136] It is worth noting that the technical solution provided in this application provides users with a corresponding operation entry point, allowing users to choose to agree to or reject the automated decision-making result; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0137] The technical solution of the diffusion model calculation method based on least squares fitting provided in this invention involves generating differential calculation benchmark data based on cached previous frame input data, previous frame calculation results, and the original input data of the current time step using the least squares fitting method. Structured sparse calculation is then performed on the original input data of the current time step, the differential calculation benchmark data, and the previous frame input data to generate an average error correction. Based on the linear layer of the diffusion model, the average error correction and the differential calculation benchmark data are calculated to generate the current frame data, which includes the current frame input data and the current frame calculation results. A high-similarity differential calculation benchmark is constructed through least squares fitting, effectively improving the quality and stability of the benchmark data. Combining dynamic thresholding and structured sparse calculation, a significant reduction in computational complexity is achieved while ensuring model accuracy. This enables consistently low-latency, low-energy, and efficient fast calculations in modern diffusion models where inter-frame differences are increasing.

[0138] Figure 9 This is a schematic diagram of a domain-specific new word recognition device provided in an embodiment of the present invention. This device is used to perform the aforementioned domain-specific new word recognition method, such as... Figure 9 As shown, the device includes: a reference data calculation module 11, a structured sparse calculation module 12, and a linear calculation module 13.

[0139] The benchmark data calculation module 11 is used to generate differential benchmark data based on the cached previous frame input data, the previous frame calculation results and the original input data of the current time step through the least squares fitting method.

[0140] The structured sparse computation module 12 is used to perform structured sparse computation based on the original input data of the current time step, the differential computation reference data, and the input data of the previous frame, and generate the average error correction amount.

[0141] The linear calculation module 13 is used for the linear layer based on the diffusion model to calculate the average error correction and the differential calculation benchmark data to generate the current frame data, which includes the current frame input data and the current frame calculation result.

[0142] In this embodiment of the invention, the benchmark data calculation module 11 is specifically used to generate least squares fitting coefficients based on the previous frame input data and the original input data of the current time step by constructing an optimization objective function; and to generate differential calculation benchmark data based on the least squares fitting coefficients, the previous frame input data and the previous frame calculation results.

[0143] In this embodiment of the invention, the structured sparse computation module 12 is specifically used to generate a data difference matrix based on the differential computation benchmark data and the original input data of the current time step; perform grouped column-by-column structured sparse computation on the data difference matrix to generate a sparse discrimination index; and perform grouped row-by-row structured sparse computation on the data difference matrix based on the sparse discrimination index, the original input data of the current time step and the input data of the previous frame to generate an average error correction amount.

[0144] In this embodiment of the invention, the structured sparse computation module 12 is specifically used to generate a dynamic threshold based on the original input data of the current time step and the input data of the previous frame; perform structured sparse discrimination on the data difference matrix based on the sparse discrimination index and the dynamic threshold to generate a sparse mask; and perform grouped row-by-row structured sparse computation on the data difference matrix based on the sparse mask to generate an average error correction amount.

[0145] In this embodiment of the invention, the differential calculation benchmark data includes differential calculation benchmark input data and differential calculation benchmark similarity results; the linear calculation module 13 is specifically used to generate the current frame input data based on the average error correction amount and the differential calculation benchmark input data; and the linear layer based on the diffusion model calculates the differential calculation benchmark similarity results and the average error correction amount to generate the current frame calculation results.

[0146] In this embodiment of the invention, the device further includes an iteration unit 14.

[0147] The iteration unit 14 is used to determine the current frame input data as the previous frame input data, determine the current frame calculation result as the previous frame calculation result, determine the input data of the next time step as the original input data of the current time step, and trigger the benchmark data calculation module 11 to continue to execute the steps of generating differential benchmark data based on the cached previous frame input data, previous frame calculation result and the original input data of the current time step through the least squares fitting method.

[0148] In the scheme of this invention embodiment, based on the cached previous frame input data, previous frame calculation results, and the original input data of the current time step, differential calculation benchmark data is generated by least squares fitting method; structured sparse calculation is performed based on the original input data of the current time step, differential calculation benchmark data, and previous frame input data to generate average error correction amount; based on the linear layer of the diffusion model, the average error correction amount and differential calculation benchmark data are calculated to generate current frame data, which includes current frame input data and current frame calculation results. A high similarity differential calculation benchmark is constructed by least squares fitting, which effectively improves the quality and stability of the benchmark data; by combining dynamic threshold and structured sparse calculation, a significant reduction in computational complexity is achieved while ensuring model accuracy, enabling high-efficiency and fast calculation with low latency and low energy consumption in modern diffusion models where inter-frame differences increase.

[0149] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer device, specifically, a computer device can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0150] This invention provides a computer device, including a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the steps of the above-described embodiment of the diffusion model calculation method based on least squares fitting. For a detailed description, please refer to the above-described embodiment of the diffusion model calculation method based on least squares fitting.

[0151] The following is for reference. Figure 10 It shows a schematic diagram of the structure of a computer device 600 suitable for implementing the embodiments of this application.

[0152] like Figure 10 As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer device 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0153] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal feedback (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed in storage section 608 as needed.

[0154] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611.

[0155] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0156] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0162] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0163] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0165] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0166] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for calculating a diffusion model based on least squares fitting, characterized in that, The method includes: Based on the cached previous frame input data, previous frame calculation results, and the original input data of the current time step, the difference calculation benchmark data is generated by the least squares fitting method. Structured sparse computation is performed based on the original input data of the current time step, the differential calculation baseline data, and the input data of the previous frame to generate the average error correction amount; Based on the linear layer of the diffusion model, the average error correction amount and the differential calculation benchmark data are calculated to generate the current frame data, which includes the current frame input data and the current frame calculation result.

2. The diffusion model calculation method based on least squares fitting according to claim 1, characterized in that, The cached previous frame input data, previous frame calculation results, and the original input data of the current time step are used to generate differential calculation baseline data through a least squares fitting method, including: By constructing an optimization objective function, least squares fitting coefficients are generated based on the previous frame input data and the original input data at the current time step. Based on the least squares fitting coefficients, the previous frame input data, and the previous frame calculation results, differential calculation benchmark data is generated.

3. The diffusion model calculation method based on least squares fitting according to claim 1, characterized in that, The step of performing structured sparse computation based on the original input data of the current time step, the differential calculation reference data, and the previous frame input data to generate the average error correction includes: A data difference matrix is ​​generated based on the differential calculation baseline data and the original input data of the current time step; The data difference matrix is ​​grouped and column-by-column structured sparse computation is performed to generate a sparsity discrimination index. Based on the sparse discrimination index, the original input data of the current time step, and the input data of the previous frame, the data difference matrix is ​​subjected to grouped row-by-row structured sparse computation to generate the average error correction amount.

4. The diffusion model calculation method based on least squares fitting according to claim 3, characterized in that, The step of performing grouped, row-by-row structured sparse computation on the data difference matrix based on the sparse discrimination index, the original input data at the current time step, and the input data of the previous frame to generate the average error correction amount includes: A dynamic threshold is generated based on the original input data at the current time step and the input data of the previous frame. Based on the sparse discrimination index and dynamic threshold, the data difference matrix is ​​subjected to structured sparse discrimination to generate a sparse mask. Based on the sparse mask, the data difference matrix is ​​subjected to grouped row-by-row structured sparse computation to generate the average error correction amount.

5. The diffusion model calculation method based on least squares fitting according to claim 1, characterized in that, The differential calculation benchmark data includes differential calculation benchmark input data and differential calculation benchmark similarity results; The linear layer based on the diffusion model calculates the average error correction and the differential calculation baseline data to generate the current frame data, including: The current frame input data is generated based on the average error correction amount and the differential calculation reference input data. Based on the linear layer of the diffusion model, the similarity result of the differential calculation benchmark and the average error correction amount are calculated to generate the calculation result of the current frame.

6. The diffusion model calculation method based on least squares fitting according to claim 1, characterized in that, The method further includes: The current frame input data is determined as the previous frame input data, the current frame calculation result is determined as the previous frame calculation result, the input data of the next time step is determined as the original input data of the current time step, and the steps of generating differential calculation benchmark data by using the cached previous frame input data, previous frame calculation result and original input data of the current time step are continued.

7. A hardware unit, characterized in that, The hardware unit, applied to the least squares fitting-based diffusion model calculation method according to any one of claims 1 to 6, comprises: multiple vector subtractors, multiple vector multipliers, vector absolute value generators, multiple addition trees, maximum value trees, multiple adders, multiple multipliers, comparators, and dividers.

8. A computational device for a diffusion model based on least squares fitting, characterized in that, Applied to the hardware unit of claim 7, the device comprises: The benchmark data calculation module is used to generate differential benchmark data based on the cached previous frame input data, previous frame calculation results and the original input data of the current time step through the least squares fitting method. The structured sparse computation module is used to perform structured sparse computation based on the original input data of the current time step, the differential calculation reference data, and the input data of the previous frame, and generate the average error correction amount. The linear calculation module is used to calculate the average error correction amount and the differential calculation benchmark data based on the linear layer of the diffusion model, and generate the current frame data, which includes the current frame input data and the current frame calculation result.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the diffusion model calculation method based on least squares fitting as described in any one of claims 1 to 6.

10. A computer device comprising a memory and a processor, the memory for storing information including program instructions, and the processor for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by the processor, they implement the diffusion model calculation method based on least squares fitting as described in any one of claims 1 to 6.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the diffusion model calculation method based on least squares fitting as described in any one of claims 1 to 6.

Images