An acceleration method, device, equipment, medium and product of a flow matching model

Abstract

The application provides an acceleration method and device of a flow matching model, equipment, a medium and a product, and relates to the technical field of computer vision. The method comprises the following steps: obtaining historical data of a flow matching model to be accelerated, performing random screening processing on the historical data to obtain sample data; constructing a stability error graph set according to the sample data and a cache set; performing fusion processing on the stability error graph set according to a fusion algorithm to obtain a multiple graph; performing calculation processing according to the multiple graph, a preset cache condition and a peak suppression shortest path algorithm to obtain an optimal scheduling scheme; and performing acceleration processing on the flow matching model to be accelerated according to the optimal scheduling scheme to generate a model result. The application solves the technical problem that the prior art relies on instantaneous speed information, is prone to cause trajectory deviation and error accumulation, causes generated content to be blurred or distorted, and thus causes the flow matching model to be low in efficiency.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, device, medium and product for accelerating flow matching models. Background Technology

[0002] Flow matching models have demonstrated outstanding performance in image, video, and multimodal generation tasks. However, in commercial-grade large-scale models, the huge memory consumption, heavy computational load, and long inference latency severely hinder their application in real-time interactive or resource-constrained scenarios.

[0003] Current solutions mainly fall into two categories: retraining methods, such as distillation, pruning, and quantization, which involve modifying the model architecture and retraining on large-scale datasets; and training-free caching methods, which reduce redundant computation by reusing intermediate features or outputs at specific time steps.

[0004] However, existing technologies rely on instantaneous velocity information, which can easily lead to trajectory deviations and error accumulation, resulting in blurry or distorted generated content, thus causing low efficiency of the flow matching model. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for accelerating a flow matching model, in order to solve the problem that the prior art relies on instantaneous speed information, which easily leads to trajectory deviation and error accumulation, resulting in blurry or distorted generated content, thus causing low efficiency of the flow matching model.

[0006] Firstly, this application provides a method for accelerating a flow matching model, comprising:

[0007] Obtain historical data of the flow matching model to be accelerated, and perform random filtering on the historical data to obtain sample data;

[0008] Based on the sample data and the cache set, a set of stability error maps is constructed;

[0009] Based on the fusion algorithm, the stability error map set is fused to obtain a multimap;

[0010] The optimal scheduling scheme is obtained by performing calculations based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm.

[0011] The model to be accelerated is processed according to the optimal scheduling scheme to generate the model result.

[0012] In one possible design, multiple diagrams include:

[0013] Multiple time nodes are used to represent time steps;

[0014] There are multiple operation edges; each operation edge corresponds to a cache operation; and there are multiple operation edges between a pair of time nodes.

[0015] Multiple operation edge weights; where each operation edge corresponds to an operation edge weight, which is used to represent the stability error of the corresponding cache operation; where the stability error is calculated based on the actual average speed and the estimated average speed.

[0016] In one possible design, the stability error is calculated based on the actual average speed and the estimated average speed, including:

[0017] The true average velocity is calculated based on the sample data and the starting identity; where the starting identity is used to represent the average flow identity at the starting time node.

[0018] The estimated average velocity is calculated based on the sample data, the approximate Jacobian vector product, and the approximate identity.

[0019] The absolute value of the difference between the actual average speed and the estimated average speed is taken to obtain the stable error.

[0020] In one possible design, the approximate Jacobian vector product is calculated based on the sample data and the preceding reference point; where the preceding reference point is located before the start time node;

[0021] The approximate identity is determined based on the starting identity and the approximate Jacobian vector product.

[0022] In one possible design, the preset caching conditions include a preset speedup parameter and a preset cache step size parameter set; wherein the cache step size parameter set includes multiple cache steps.

[0023] In one possible design, calculations are performed based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm to obtain the optimal scheduling scheme, including:

[0024] Use the preset acceleration ratio parameter as a constraint.

[0025] The minimized path equation is obtained based on the constraints and the peak suppression shortest path algorithm;

[0026] In a multigraph, the optimal scheduling scheme is obtained by solving the path equation minimizing the path equation using a dynamic programming algorithm.

[0027] In one possible design, the optimal scheduling scheme includes the full computation set, the cache operation application set, and the cache step size application set;

[0028] The set of applications for caching operations includes at least one application time step; where the application time step is the time step in which caching operations need to be performed.

[0029] Correspondingly, the accelerated flow matching model is processed according to the optimal scheduling scheme to generate model results, including:

[0030] Based on the application time step, the corrected average velocity is calculated using historical data, the corrected Jacobian vector product, and the corrected identity.

[0031] The acceleration process is applied to the flow matching model, and the model results are generated based on the corrected average velocity.

[0032] Secondly, this application provides an acceleration device for a flow matching model, comprising:

[0033] The acquisition module is used to acquire historical data of the flow matching model to be accelerated, and to randomly filter the historical data to obtain sample data.

[0034] The building module is used to construct a set of stability error maps based on sample data and cache set;

[0035] The fusion module is used to fuse the set of stability error maps according to the fusion algorithm to obtain multiple maps;

[0036] The calculation module is used to perform calculations based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm to obtain the optimal scheduling scheme.

[0037] The acceleration module is used to accelerate the matching model of the stream to be accelerated according to the optimal scheduling scheme and generate the model result.

[0038] Thirdly, this application provides an acceleration device for a stream matching model, including: a memory and a processor;

[0039] The memory stores the instructions that the computer executes;

[0040] The processor executes computer execution instructions stored in memory, causing the processor to execute the flow matching model as described in the first aspect of the invention, thus accelerating the process.

[0041] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement an accelerated method for a flow matching model as described in the first aspect of the invention.

[0042] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements an acceleration method for the flow matching model of the first aspect of the invention.

[0043] This application provides a method, apparatus, device, medium, and product for accelerating a stream matching model, comprising: acquiring historical data of the stream matching model to be accelerated; randomly filtering the historical data to obtain sample data; constructing a set of stability error maps based on the sample data and a cache set; fusing the set of stability error maps according to a fusion algorithm to obtain a multimap; performing calculations based on the multimap, preset caching conditions, and a peak suppression shortest path algorithm to obtain an optimal scheduling scheme; and accelerating the stream matching model to be accelerated according to the optimal scheduling scheme to generate model results. Compared to existing technologies that rely on instantaneous speed information, which are prone to trajectory deviation and error accumulation, resulting in blurred or distorted generated content and thus low efficiency of the stream matching model, this application improves the acceleration efficiency of the stream matching model by changing the caching perspective from "instantaneous speed" to "average speed" and introducing a caching scheduling method based on "stability maps." This alleviates the accumulated error during the acceleration of the diffusion model and optimizes the caching timing, achieving high-quality image or video generation while maintaining rapid generation, thereby improving the acceleration efficiency of the stream matching model. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic diagram of the system architecture for an acceleration method of a flow matching model provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 1 ;

[0047] Figure 3 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 2 ;

[0048] Figure 4 A schematic diagram illustrating the average speed estimation based on cached JVP provided in this application embodiment;

[0049] Figure 5 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 3 ;

[0050] Figure 6 A schematic diagram of the structure of the acceleration device for the flow matching model provided in the embodiments of this application;

[0051] Figure 7 This is a schematic diagram of the structure of an acceleration device for a flow matching model provided in an embodiment of this application. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, nor do they necessarily imply difference. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more.

[0054] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation; the embodiments of this application do not specifically limit this. Furthermore, the method for accelerating a flow matching model provided in the embodiments of this application is merely an example; the method for accelerating a flow matching model may include more or fewer elements.

[0055] Flow matching models have demonstrated outstanding performance in image, video, and multimodal generation tasks. However, in commercial-grade large-scale models, the huge memory consumption, heavy computational load, long inference latency, and users' extremely high requirements for the real-time performance and quality of generated content severely hinder their application in real-time interactive or resource-constrained scenarios.

[0056] To address the aforementioned efficiency issues, existing solutions mainly fall into two categories:

[0057] Retraining methods, such as distillation, pruning, and quantization, involve retraining the model on large-scale datasets to reduce computational complexity by modifying the model architecture or simplifying the computational process. However, these methods require additional training costs and may lead to a decrease in the quality of generated models due to changes in model structure, making them difficult to adapt to the deployment requirements of existing commercial-grade large-scale models.

[0058] Training-free caching methods reduce redundant computation by directly reusing intermediate features or extrapolating instantaneous velocity information. For example, instantaneous velocity-based methods assume that feature changes are small between adjacent time steps and cache the computation results of the current time step or extrapolate to predict subsequent steps. However, such methods have two major drawbacks:

[0059] Optional, error accumulation problem: The instantaneous speed fluctuates drastically during the denoising process, and direct caching can easily lead to trajectory deviations. At high speedup ratios, the error is amplified exponentially, resulting in blurry or distorted content.

[0060] Optionally, rigid cache scheduling: caching strategies that rely on fixed intervals or a single threshold (such as caching once every N steps) cannot adapt to the dynamic needs of complex generation processes, resulting in unstable acceleration effects and generation quality.

[0061] Existing caching methods are typically based on the perspective of instantaneous velocity or instantaneous features. They assume that the features change little between adjacent time steps, directly reusing the calculation results of step t or simply extrapolating them to subsequent steps. Although cache-based methods can accelerate inference, the quality of the generated images / videos often degrades significantly at high speedup ratios. We observe that this is because existing methods rely on instantaneous velocity information, and the instantaneous velocity in the diffusion / stream matching process often exhibits sharp fluctuations. Directly reusing such fluctuating instantaneous velocity leads to severe trajectory deviations and error accumulation, causing the final generated trajectory to deviate from the true path, resulting in blurry or distorted generated content.

[0062] Furthermore, existing caching strategies (when to cache, how many steps to cache) typically rely on fixed heuristic rules or a single threshold, making it difficult to adapt to complex generation processes.

[0063] In summary, existing technologies have not effectively resolved the contradiction between error accumulation and dynamic cache scheduling in the field of training-free acceleration, and it is difficult to balance acceleration efficiency and generation quality.

[0064] The existing technical solutions have the following limitations:

[0065] Optionally, instantaneous velocity can easily lead to error accumulation: existing methods cache in the instantaneous-Velocity domain. Since the instantaneous velocity fluctuates drastically during the denoising process, direct reuse can lead to severe trajectory drift, especially at high speedup ratios, where the error will be amplified exponentially.

[0066] Optionally, the caching scheduling strategy is rigid: existing methods mostly rely on fixed intervals or manually set thresholds to decide whether to cache, lacking a dynamic programming mechanism that can globally balance computational budget and trajectory stability, resulting in unstable generation quality after acceleration.

[0067] To address the aforementioned problems, the inventors, during their research on the low acceleration efficiency of flow matching models, discovered that existing technologies rely on instantaneous velocity information, which easily leads to trajectory deviation and error accumulation, resulting in blurred or distorted generated content, thus causing low efficiency in flow matching models. Accordingly, the inventors considered constructing a stability graph and introducing a peak suppression shortest path search strategy to dynamically optimize caching timing and step size. Simultaneously, they utilized the Jacobian-vector product (JVP) to correct the conversion relationship between instantaneous velocity and average velocity, fundamentally alleviating error accumulation and improving the stability of the acceleration effect. Based on this, embodiments of this application provide a method, apparatus, device, medium, and product for accelerating flow matching models, applicable in the field of computer vision technology, aiming to solve the problem of low acceleration efficiency in existing flow matching models.

[0068] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0069] Figure 1 This is a schematic diagram of the system architecture for an acceleration method of a flow matching model provided in an embodiment of this application. The acceleration system for the flow matching model is a computer device. Figure 1 In the above architecture, at least one of data acquisition device 101, processing device 102 and display device 103 is included.

[0070] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the processing system architecture of the acceleration method for the convection matching model. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or divide some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0071] In the specific implementation process, the data acquisition device 101 may include an input / output interface or a communication interface. The data acquisition device 101 can connect to the processing device through the input / output interface or the communication interface to acquire relevant data.

[0072] The processing device 102 can accelerate the flow matching model to be accelerated based on relevant data and generate model results.

[0073] The display device 103 can also be a touch screen or the screen of a terminal device, used to receive user commands while displaying the above-mentioned content, so as to realize interaction with the user.

[0074] It should be understood that the aforementioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0075] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0076] The technical solution of this application will be described in detail below with reference to specific embodiments:

[0077] Figure 2 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 1 ,like Figure 2 As shown, the method includes:

[0078] S201. Obtain historical data of the flow matching model to be accelerated, and perform random filtering on the historical data to obtain sample data.

[0079] In this embodiment, historical data of the flow matching model to be accelerated is obtained, and the historical data is randomly filtered to randomly select a small number of sample data (20~50).

[0080] S202. Based on the sample data and the cache set, construct a set of stability error maps.

[0081] For example, select a small number of samples (20~50), calculate the JVP under different cache time steps (t>s) and different cache step sizes k as edge weights, and use the time steps as nodes to construct a static stability error graph (StabilityMap).

[0082] S203. According to the fusion algorithm, the stability error map set is fused to obtain a multimap.

[0083] In this embodiment, the stability graph is formally fused into a multigraph, which serves as the basis for subsequent short path search.

[0084] Among them, multigraphs include:

[0085] Optional, multiple time nodes are used to represent time steps.

[0086] Specifically, nodes represent time steps in the denoising process.

[0087] Optional, multiple operation edges.

[0088] Each operation edge corresponds to a cache operation.

[0089] There are multiple operation edges between a pair of time nodes.

[0090] Specifically, edges represent potential caching operations, with edges from time step t to s indicating that caching is used within this interval.

[0091] Optional, multiple operation edge weights.

[0092] Each operation edge corresponds to an operation edge weight, which is used to represent the stability error of the corresponding cache operation.

[0093] The stability error is calculated based on the actual average speed and the estimated average speed.

[0094] Specifically, edge weights are defined as the stability error introduced by the caching operation, i.e., the difference between the actual average speed and the estimated average speed (estimated by JVP):

[0095]

[0096] Since there are different cache step sizes k (e.g., k=2,3,4,5), there may be multiple edges between the same pair of nodes, thus forming a multigraph.

[0097] Specifically, multiple stability error graphs are fused using a fusion algorithm to combine (average) node and edge weights, ultimately transforming them into a multigraph structure.

[0098] In one possible implementation, a multi-level hierarchical stability graph is constructed, dividing the caching strategy into a global layer (long step size) and a local layer (short step size). The global layer is used for overall path planning, while the local layer is used to refine caching decisions for key regions. Through the hierarchical graph structure, the algorithm prioritizes matching the stability error of the global layer when searching for the shortest path, and then optimizes the details at the local layer.

[0099] Among these, the multi-level hierarchical scheduling strategy significantly improves the caching accuracy of complex local areas (such as image edges and video keyframes) while ensuring global stability. Through hierarchical optimization, conflicts between global and local requirements of a single-granularity map are avoided, further enhancing the stability of the generated quality.

[0100] S204. Based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm, perform calculations to obtain the optimal scheduling scheme.

[0101] The preset caching conditions include a preset set of speedup parameters and cache step size parameters.

[0102] The preset speedup parameter corresponds to the number of forward steps in the actual full computation of the model, and can be set according to user expectations (e.g., B=15, B=17).

[0103] The cache step size parameter set includes multiple cache steps.

[0104] Specifically, the cache step size parameter set is k=1, 2, 3…

[0105] S205. Accelerate the flow matching model to be accelerated according to the optimal scheduling scheme and generate the model result.

[0106] This embodiment provides a method for accelerating a stream matching model, comprising: acquiring historical data of the stream matching model to be accelerated; randomly filtering the historical data to obtain sample data; constructing a set of stability error maps based on the sample data and a cache set; fusing the set of stability error maps according to a fusion algorithm to obtain a multimap; performing calculations based on the multimap, preset caching conditions, and a peak suppression shortest path algorithm to obtain an optimal scheduling scheme; and accelerating the stream matching model to be accelerated according to the optimal scheduling scheme to generate the model result. Compared with existing technologies that rely on instantaneous speed information, which are prone to trajectory deviation and error accumulation, resulting in blurred or distorted generated content, thus leading to low efficiency of the stream matching model, this application improves the acceleration efficiency of the stream matching model by changing the caching perspective from "instantaneous speed" to "average speed" and introducing a caching scheduling method based on "stability maps". This alleviates the accumulated error during the acceleration of the diffusion model and optimizes the caching timing, achieving high-quality image or video generation while maintaining rapid generation, thereby improving the acceleration efficiency of the stream matching model.

[0107] Figure 3 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 2 As shown in the figure, the steady-state error is calculated based on the actual average velocity and the estimated average velocity, specifically including:

[0108] S301. Calculate the true average speed based on the sample data and the starting point identity.

[0109] The starting point identity is used to represent the average flow identity at the starting time node.

[0110] According to the identity, there is a derivative relationship between instantaneous velocity and average velocity.

[0111] For example, the average flow identity at the initial time node is:

[0112]

[0113] in, The instantaneous velocity at the starting point t; Let be the average velocity at the starting point t.

[0114] Among them, the derivative term It can be represented as the Jacobian-vector product (JVP).

[0115] S302. Based on the sample data, the approximate Jacobian vector product, and the approximate identity, the estimated average velocity is calculated.

[0116] The approximate Jacobian vector product is calculated based on the sample data and the preceding reference point.

[0117] The preceding reference point is located before the start time node.

[0118] The approximate identity is determined based on the starting identity and the approximate Jacobian vector product.

[0119] For example, a preceding reference point is introduced. Using the preceding reference point To the starting point Use known information to approximate the starting point To the prediction point JVP is defined as .

[0120] Where k represents the discrete time step distance between r and t.

[0121] Specifically, estimating the average speed The structure is as follows:

[0122]

[0123] in, It is an approximate Jacobian vector product calculated using the historical state of the cache.

[0124] Specifically, in this way, the present invention uses cached JVP to correct the instantaneous speed, resulting in a smoother and more accurate average speed, thereby effectively suppressing error accumulation locally.

[0125] In one possible embodiment, Figure 4 A schematic diagram of average speed estimation based on cached JVP provided in this application embodiment is shown below. Figure 4 As shown, the execution flow of this diagram begins at the initial time point r, and reuses the JVP (Jacobi-variable derivative, labeled as) from the previous time step r to the current time step t through the cache region. (Arrow), combined with instantaneous velocity at time t Using the time difference (st), the estimated average velocity from t to s is calculated. .

[0126] Specifically, the calculation path is as follows: the x1 data stored in the cache area is reused via a loop arrow, and then processed by the formula... After completing the velocity estimation, the final driving trajectory travels from point t along the gray curve to the target time point s, forming a complete trajectory correction path.

[0127] Among them, the direct cache range Instantaneous velocity within This can easily lead to trajectory drift and error accumulation; while average speed This allows for more accurate arrival at the target time. However, during forward reasoning, we do not know... Therefore, this invention introduces a previous time step. and reuse from arrive of To calculate an estimated average speed This corrects the trajectory and effectively reduces error accumulation.

[0128] In this method, converting instantaneous velocity into interval average velocity for caching and reuse effectively alleviates the problems of trajectory drift and error accumulation caused by instantaneous velocity fluctuations.

[0129] S303. Take the absolute value of the difference between the actual average speed and the estimated average speed to obtain the stable error.

[0130] In one possible embodiment, a dynamic JVP sampling frequency adjustment mechanism is further introduced to dynamically adjust the sampling interval based on the instantaneous velocity fluctuations of the current time step (e.g., by calculating the gradient change rate of adjacent time steps). For example, the sampling interval is shortened (k=2) in high-fluctuation regions (e.g., the initial stage of denoising) and extended (k=4) in low-fluctuation regions (e.g., the final stage of denoising) to more accurately capture key velocity changes.

[0131] By dynamically adjusting the JVP sampling frequency, this scheme provides denser correction points in regions with drastic instantaneous velocity fluctuations, significantly reducing the accumulation of local errors. Simultaneously, sparse sampling in low-fluctuation regions reduces computational overhead, further optimizing the balance between accuracy and efficiency in average velocity estimation.

[0132] In this embodiment, the true average velocity is calculated by combining the starting point identity with sample data, and an approximate Jacobian vector product is constructed using preceding reference point data. The estimated average velocity is then calculated using the approximate identity derived from the starting point identity and the approximate Jacobian vector product. Finally, the stability error is quantified by the absolute difference between the true and estimated average velocities. This achieves accurate estimation and error stability assessment of the average velocity of the dynamic system, effectively improving the reliability and robustness of velocity calculation in time series analysis. This, in turn, enhances the acceleration efficiency of the flow matching model.

[0133] Figure 5 A flowchart illustrating an acceleration method for a flow matching model provided in this application embodiment. Figure 3 ,like Figure 5 As shown, the specific implementation steps of S204 above include:

[0134] S501, Use the preset acceleration ratio parameter as a constraint condition.

[0135] For example, a preset speedup ratio parameter (Budget, B) can be used as a constraint.

[0136] S502. Obtain the minimized path equation based on the constraints and the peak suppression shortest path algorithm.

[0137] Specifically, a peak suppression parameter is introduced. Under the constraint of (Budget, B), the path equation that is minimized is obtained:

[0138]

[0139] In one possible implementation, an adaptive peak suppression parameter is introduced, which dynamically adjusts the suppression threshold based on the cumulative error of the current path. For example, when the cumulative error approaches a preset upper limit, the suppression threshold is automatically lowered to avoid high-error nodes; when the cumulative error is low, the threshold is relaxed to increase the cache step size.

[0140] Among them, the adaptive peak suppression mechanism enables the algorithm to flexibly adjust its strategy at different generation stages (such as high error risk in the early stage and low error risk in the later stage). This avoids both excessive suppression leading to a small cache step size (efficiency reduction) and excessively loose thresholds leading to error accumulation, thus significantly improving the dynamic balance between speedup and generation quality.

[0141] S503. Solve the path equation minimizing algorithm in the multigraph to obtain the optimal scheduling scheme.

[0142] The optimal scheduling scheme includes the full computation set, the cache operation application set, and the cache step size application set.

[0143] Specifically, the full computation set includes the time steps in which full computation is performed.

[0144] Specifically, the cache operation application set includes the time steps in which cache operations are used.

[0145] Specifically, the cache stride application set includes the cache stride used when performing cache operations.

[0146] The cache operation application set includes at least one application time step.

[0147] The application time step is the time step where caching operations are required.

[0148] Furthermore, the specific implementation steps of step S205 include:

[0149] Specifically, based on the application time step, the corrected average velocity is calculated using historical data, the corrected Jacobian vector product, and the corrected identity.

[0150] Specifically, the acceleration process is applied to the flow matching model to be accelerated, and the model results are generated based on the corrected average velocity.

[0151] For example, the model for matching the stream to be accelerated is generated according to the optimal scheduling scheme described above. At the time step requiring caching, the formula is called:

[0152]

[0153] Specifically, based on average speed Direct calculation Latent variables at time It skips redundant time steps in the flow matching model, accelerates model inference, and enables the model to generate text-to-image data without training.

[0154] In particular, when accelerating the flow matching model, the historical data used is real-time data that is updated in real time.

[0155] In one possible implementation, a model prediction error feedback mechanism is introduced. The weighting of the average velocity is dynamically adjusted by comparing the prediction result with the actual result at the current time step in real time. For example, if the prediction error is large, the weight of the JVP of the preceding reference point is increased; conversely, the weight is decreased to avoid overcorrection.

[0156] This mechanism makes average velocity estimation more adaptive by providing real-time feedback on model prediction errors. In complex generation tasks (such as high dynamic range video scenes), it can quickly respond to model prediction biases, significantly reduce the risk of trajectory drift caused by model uncertainty, and further improve the accuracy of generated content.

[0157] In this embodiment, a minimum path equation is constructed by using a preset speedup parameter as a constraint and combining it with a peak suppression shortest path algorithm. The equation is then solved in a multi-graph using a dynamic programming algorithm to obtain an optimal scheduling scheme that includes the full computation set, the cache operation application set (containing at least one application time step), and the cache step size application set. Furthermore, based on the application time step, a corrected average speed is calculated using historical data, a corrected Jacobian vector product, and a corrected identity. This speedup is then used to accelerate the flow matching model and generate model results. This achieves efficient scheduling optimization and accurate acceleration of complex dynamic systems, significantly improving computational efficiency and resource utilization. At the same time, the coordinated application of the corrected Jacobian vector product and the identity ensures the stability and accuracy of the model acceleration process.

[0158] Figure 6 This is a schematic diagram of the structure of the acceleration device for the flow matching model provided in the embodiments of this application, as shown below. Figure 6 As shown, the device includes: an acquisition module 61, a construction module 62, a fusion module 63, a calculation module 64, and an acceleration module 65.

[0159] The acquisition module 61 is used to acquire historical data of the flow matching model to be accelerated, and to perform random filtering on the historical data to obtain sample data.

[0160] Module 62 is used to construct a set of stability error maps based on sample data and cache set;

[0161] The fusion module 63 is used to perform fusion processing on the set of stability error maps according to the fusion algorithm to obtain multiple maps;

[0162] The calculation module 64 is used to perform calculations based on the multigraph, preset caching conditions and peak suppression shortest path algorithm to obtain the optimal scheduling scheme.

[0163] The acceleration module 65 is used to accelerate the matching model of the flow to be accelerated according to the optimal scheduling scheme and generate the model result.

[0164] In one possible design, multiple diagrams include:

[0165] Multiple time nodes are used to represent time steps;

[0166] There are multiple operation edges; each operation edge corresponds to a cache operation; and there are multiple operation edges between a pair of time nodes.

[0167] Multiple operation edge weights; where each operation edge corresponds to an operation edge weight, which is used to represent the stability error of the corresponding cache operation; where the stability error is calculated based on the actual average speed and the estimated average speed.

[0168] In one possible design, the stability error is calculated based on the actual average speed and the estimated average speed, including:

[0169] The true average velocity is calculated based on the sample data and the starting identity; where the starting identity is used to represent the average flow identity at the starting time node.

[0170] The estimated average velocity is calculated based on the sample data, the approximate Jacobian vector product, and the approximate identity.

[0171] The absolute value of the difference between the actual average speed and the estimated average speed is taken to obtain the stable error.

[0172] In one possible design, the approximate Jacobian vector product is calculated based on the sample data and the preceding reference point; where the preceding reference point is located before the start time node;

[0173] The approximate identity is determined based on the starting identity and the approximate Jacobian vector product.

[0174] In one possible design, the preset caching conditions include a preset speedup parameter and a preset cache step size parameter set; wherein the cache step size parameter set includes multiple cache steps.

[0175] In one possible design, calculations are performed based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm to obtain the optimal scheduling scheme, including:

[0176] The calculation module 64 is also used to use the preset speedup ratio parameter as a constraint condition;

[0177] The minimized path equation is obtained based on the constraints and the peak suppression shortest path algorithm;

[0178] In a multigraph, the optimal scheduling scheme is obtained by solving the path equation minimizing the path equation using a dynamic programming algorithm.

[0179] In one possible design, the optimal scheduling scheme includes the full computation set, the cache operation application set, and the cache step size application set;

[0180] The set of applications for caching operations includes at least one application time step; where the application time step is the time step in which caching operations need to be performed.

[0181] Correspondingly, the accelerated flow matching model is processed according to the optimal scheduling scheme to generate model results, including:

[0182] The acceleration module 65 is also used to calculate the corrected average speed based on the application time step, through historical data, the corrected Jacobian vector product, and the corrected identity.

[0183] The acceleration process is applied to the flow matching model, and the model results are generated based on the corrected average velocity.

[0184] This embodiment provides an acceleration device for a flow matching model, which can execute an acceleration method for a flow matching model in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0185] In a specific implementation of the aforementioned method for accelerating a flow matching model, each module can be implemented as a processor. The processor can execute computer execution instructions stored in memory, thereby enabling the processor to execute the aforementioned method for accelerating a flow matching model.

[0186] Figure 7 This is a schematic diagram of the structure of an acceleration device for a flow matching model provided in an embodiment of this application. Figure 7As shown, the acceleration device 70 for the flow matching model includes at least one processor 71 and a memory 72. The acceleration device 70 also includes a communication component 73. The processor 71, memory 72, and communication component 73 are connected via a bus 74.

[0187] In a specific implementation, at least one processor 71 executes computer execution instructions stored in memory 72, causing at least one processor 71 to execute a method in the field of computer vision technology, such as that executed on the acceleration device side of the above-mentioned flow matching model.

[0188] The specific implementation process of processor 71 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0189] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0190] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0191] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0192] The above description of the functions implemented by the acceleration device and main control device for the flow matching model illustrates the solution provided by the embodiments of the present invention. It is understood that, in order to achieve the above functions, the acceleration device or main control device for the flow matching model includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solution of the embodiments of the present invention.

[0193] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described above in the field of computer vision technology.

[0194] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0195] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the acceleration device or master device of the flow matching model.

[0196] This application also provides a computer program product comprising: a computer program stored in a readable storage medium, at least one processor of the stream matching model acceleration device being able to read the computer program from the readable storage medium, and at least one processor executing the computer program causing the stream matching model acceleration device to perform the scheme provided in any of the above embodiments.

[0197] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0198] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for accelerating a flow matching model, characterized in that, include: Obtain historical data of the flow matching model to be accelerated, and perform random filtering on the historical data to obtain sample data; Based on the sample data and the cache set, a set of stability error maps is constructed; According to the fusion algorithm, the set of stability error maps is fused to obtain multiple maps; Based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm, calculations are performed to obtain the optimal scheduling scheme; The matching model of the flow to be accelerated is accelerated according to the optimal scheduling scheme to generate model results.

2. The method according to claim 1, characterized in that, The multiple graphs include: Multiple time nodes are used to represent time steps; Multiple operation edges; wherein each operation edge corresponds to a cache operation; wherein multiple operation edges exist between a pair of time nodes; Multiple operation edge weights; wherein each operation edge corresponds to an operation edge weight, and the operation edge weight is used to represent the stability error of the corresponding cache operation; wherein the stability error is calculated based on the actual average speed and the estimated average speed.

3. The method according to claim 2, characterized in that, The steady-state error calculated based on the true average speed and the estimated average speed includes: The true average velocity is calculated based on the sample data and the starting identity; wherein the starting identity is used to represent the average flow identity at the starting time node; The estimated average velocity is calculated based on the sample data, the approximate Jacobian vector product, and the approximate identity. The absolute value of the difference between the true average speed and the estimated average speed is taken to obtain the stability error.

4. The method according to claim 3, characterized in that, The approximate Jacobian vector product is calculated based on the sample data and the preceding reference point; wherein the preceding reference point is located before the starting time node; The approximate identity is determined based on the product of the starting point identity and the approximate Jacobian vector.

5. The method according to claim 4, characterized in that, The preset caching conditions include a preset speedup parameter and a preset set of cache step size parameters; wherein, the set of cache step size parameters includes multiple cache steps.

6. The method according to claim 5, characterized in that, The step of calculating and processing based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm to obtain the optimal scheduling scheme includes: The preset acceleration ratio parameter is used as a constraint condition; The minimized path equation is obtained based on the constraints and the peak suppression shortest path algorithm. The optimal scheduling scheme is obtained by solving the minimized path equation using a dynamic programming algorithm in the multigraph.

7. The method according to claim 6, characterized in that, The optimal scheduling scheme includes a full computation set, a cache operation application set, and a cache step size application set; The cache operation application set includes at least one application time step; wherein, the application time step is the time step in which the cache operation needs to be performed; Correspondingly, the step of accelerating the matching model of the flow to be accelerated according to the optimal scheduling scheme and generating model results includes: Based on the application time step, the corrected average velocity is calculated using the historical data, the corrected Jacobian vector product, and the corrected identity. The flow matching model to be accelerated is accelerated, and the model result is generated based on the corrected average velocity.

8. An acceleration device for a flow matching model, characterized in that, include: The acquisition module is used to acquire historical data of the flow matching model to be accelerated, and to perform random filtering on the historical data to obtain sample data. A construction module is used to construct a set of stability error maps based on the sample data and the cache set; The fusion module is used to perform fusion processing on the set of stability error maps according to the fusion algorithm to obtain multiple maps; The calculation module is used to perform calculations based on the multigraph, preset caching conditions, and peak suppression shortest path algorithm to obtain the optimal scheduling scheme. The acceleration module is used to accelerate the matching model of the flow to be accelerated according to the optimal scheduling scheme and generate model results.

9. An acceleration device for a flow matching model, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images