Sampling method, device, equipment, medium and program product of diffusion model

Abstract

The application provides a diffusion model sampling method and device, equipment, medium and program product, relating to the generative artificial intelligence technical field, including: obtaining a pre-constructed directed acyclic graph containing a virtual starting node and a virtual terminating node; determining a shortest path from the virtual starting node to the virtual terminating node in the directed acyclic graph according to a preset dynamic programming algorithm; obtaining an optimal sampling trajectory of a corresponding single training sample according to the shortest path; generating a general sampling scheduling sequence according to the optimal sampling trajectories corresponding to the plurality of training samples respectively; and calling the diffusion model according to the general sampling scheduling sequence to execute a corresponding sampling process. The method provided by the application achieves the effect of improving the sampling efficiency and accuracy of the diffusion model.

Description

Technical Field

[0001] This application relates to the field of generative artificial intelligence technology, and in particular to a sampling method, apparatus, device, medium and program product for a diffusion model. Background Technology

[0002] In the field of generative artificial intelligence, diffusion models and flow matching models have become core technologies for generating multimodal content such as images, audio, and video. These models generate high-quality data through a progressive denoising process and are widely used in scenarios such as digital content creation, virtual reality, medical image reconstruction, and autonomous driving scenario simulation. Therefore, to ensure the generation of high-quality data, developing an efficient sampling method for diffusion models has become a promising direction.

[0003] In existing technologies, sampling schemes for diffusion models mainly fall into three categories: fixed scheduling methods, such as linear and cosine scheduling in Denoising Diffusion Probabilistic Models (DDPM), which use a preset deterministic time step sequence to control denoising; heuristic adjustment methods, which dynamically optimize step size or sampling trajectory based on empirical rules; and knowledge distillation methods, such as Consistency Models, which achieve one-step generation by training a dedicated network.

[0004] However, existing technologies suffer from low sampling efficiency and poor accuracy due to the decoupling of scheduling strategies from generated content features, lack of theoretical optimality guarantees, high computational complexity, and insufficient versatility, as well as limitations imposed by fixed rules and empirical design. Summary of the Invention

[0005] The sampling method, apparatus, equipment, medium, and program products of the diffusion model provided in this application are intended to achieve the technical effect of improving sampling efficiency and accuracy.

[0006] Firstly, this application provides a sampling method for a diffusion model, comprising:

[0007] Obtain a pre-constructed directed acyclic graph (DAG); wherein, the nodes of the DAG correspond to the sampling time steps of the diffusion model, the edge weights of the DAG are the sampling trajectory correction costs calculated based on the training samples, and the DAG also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state.

[0008] Based on the preset dynamic programming algorithm, determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph.

[0009] Based on the shortest path, the optimal sampling trajectory for a single training sample is obtained;

[0010] Generate a general sampling schedule sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples;

[0011] Based on the general sampling schedule sequence, the diffusion model is invoked to execute the corresponding sampling process.

[0012] In one possible implementation, before obtaining the pre-constructed directed acyclic graph, the method further includes:

[0013] Obtain a pre-established mathematical model for the propagation of the upper bound of the error;

[0014] Based on the mathematical model, the reconstruction error at each sampling time step in the diffusion model sampling process is determined, and multiple corresponding inverse quantization values ​​are determined based on the reconstruction error at each sampling time step.

[0015] Based on the mathematical model and the reconstruction error at each sampling time step, the cost of correcting the sampling trajectory between different sampling time steps during the diffusion model sampling process is determined.

[0016] Each sampling time step of the diffusion model is mapped to a corresponding graph node, where the number of sampling time steps is a preset maximum number of time steps;

[0017] Each reconstruction error is set as the first edge weight between the virtual starting node and each sampling time step node;

[0018] Set each inverse quantization value as the second side weight between each sampling time step node and the virtual termination node;

[0019] Set the cost of correcting the sampling trajectory to the third side weight between the previous sampling time step and the next sampling time step;

[0020] Construct a directed acyclic graph based on each graph node, the first edge weight, the second edge weight, and the third edge weight.

[0021] In one possible implementation, a general sampling scheduling sequence is generated based on the optimal sampling trajectories corresponding to multiple training samples, including:

[0022] Obtain the optimal sampling trajectory corresponding to multiple training samples;

[0023] The time step transition combinations of each optimal sampling trajectory are analyzed and processed to obtain the transition frequency, repetition pattern and data characteristics corresponding to each optimal sampling trajectory; the analysis and processing includes frequency statistics, repetition pattern recognition and feature extraction.

[0024] Based on the transfer frequency, repetition pattern, and data characteristics corresponding to each optimal sampling trajectory, the common characteristics of each optimal sampling trajectory are determined.

[0025] Based on common characteristics, a general sampling scheduling sequence is generated.

[0026] In one possible implementation, the reconstruction error at each sampling time step during the diffusion model sampling process is determined based on the mathematical model, including:

[0027] Obtain the raw data from the training samples;

[0028] Based on the forward noise addition process of the mathematical model and the diffusion model, the ideal noise state corresponding to each sampling time step is calculated.

[0029] The ideal noise state at each sampling time step is processed by a diffusion model to obtain the original data estimate corresponding to each sampling time step;

[0030] Based on the original data and the estimated values ​​of the original data, the corresponding difference quantization results are determined, and the difference quantization results are used as the reconstruction error for each sampling time step.

[0031] In one possible implementation, the sampling trajectory correction cost between different sampling time steps during the diffusion model sampling process is determined based on the mathematical model and the reconstruction error at each sampling time step, including:

[0032] Based on the mathematical model, the error propagation relationship between different sampling time steps is determined, wherein the error propagation relationship is that the upper bound of the error of the next sampling time step is equal to the minimum value of the sum of the upper bound of the error of the previous sampling time step and the corresponding sampling trajectory correction cost.

[0033] Based on the Euler sampling method of the preset flow model, the noise scheduling correlation coefficient between different sampling time steps is determined. The noise scheduling correlation coefficient is the ratio of the difference between the previous and subsequent sampling time steps to the subsequent sampling time step.

[0034] Based on the error propagation relationship, the noise scheduling correlation coefficient, and the reconstruction error of each sampling time step, the sampling trajectory correction cost between different sampling time steps in the diffusion model sampling process is determined. The sampling trajectory correction cost is the product of the noise scheduling correlation coefficient and the reconstruction error of the corresponding sampling time step.

[0035] In one possible implementation, the optimal sampling trajectory for a single sample is obtained based on the shortest path, including:

[0036] Based on the directed acyclic graph, the shortest path is determined to be the first path from the sampling time step node with the smaller noise value to the sampling time step node with the larger noise value;

[0037] According to the diffusion model, the shortest path is reversed to obtain a second path from the sampling time step node with high noise value to the sampling time step node with low noise value.

[0038] The second path is determined as the optimal sampling trajectory for a single sample.

[0039] In one possible implementation, the general sampling scheduling sequence is generated offline;

[0040] Accordingly, based on the general sampling schedule sequence, the diffusion model is invoked to execute the corresponding sampling process, including:

[0041] During online sampling, the original fixed schedule of the diffusion model is replaced by a general sampling schedule sequence generated offline, so that the diffusion model can execute the corresponding sampling process.

[0042] In one possible implementation, the dynamic programming algorithm includes a first algorithm and a second algorithm;

[0043] Accordingly, based on the preset dynamic programming algorithm, the shortest path from the virtual start node to the virtual end node in the directed acyclic graph is determined, including:

[0044] Determine whether a fixed number of sampling steps is needed;

[0045] If a fixed number of sampling steps is not required, a preset first algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph; wherein, the first algorithm includes Dijkstra's algorithm or Floyd-Vorchard's algorithm;

[0046] If a fixed number of sampling steps is required, a pre-defined second algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph, passing through a preset number of intermediate sampling time step nodes; wherein, the first algorithm is a dynamic programming algorithm with step constraints.

[0047] Secondly, this application provides a sampling device for a diffusion model, comprising:

[0048] The acquisition module is used to acquire a pre-constructed directed acyclic graph (DAG); wherein, the nodes of the DAG correspond to the sampling time steps of the diffusion model, the edge weights of the DAG are the sampling trajectory correction costs calculated based on the training samples, and the DAG also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state.

[0049] The first processing module is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph according to a preset dynamic programming algorithm.

[0050] The second processing module is used to obtain the optimal sampling trajectory for a single sample based on the shortest path.

[0051] The third processing module is used to generate a general sampling scheduling sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples.

[0052] The calling module is used to invoke the diffusion model according to the general sampling scheduling sequence to execute the corresponding sampling process.

[0053] In one possible implementation, the acquisition module is further configured to:

[0054] Obtain a pre-established mathematical model for the propagation of the upper bound of the error;

[0055] Based on the mathematical model, the reconstruction error at each sampling time step in the diffusion model sampling process is determined, and multiple corresponding inverse quantization values ​​are determined based on the reconstruction error at each sampling time step.

[0056] Based on the mathematical model and the reconstruction error at each sampling time step, the cost of correcting the sampling trajectory between different sampling time steps during the diffusion model sampling process is determined.

[0057] Each sampling time step of the diffusion model is mapped to a corresponding graph node, where the number of sampling time steps is a preset maximum number of time steps;

[0058] Each reconstruction error is set as the first edge weight between the virtual starting node and each sampling time step node;

[0059] Set each inverse quantization value as the second side weight between each sampling time step node and the virtual termination node;

[0060] Set the cost of correcting the sampling trajectory to the third side weight between the previous sampling time step and the next sampling time step;

[0061] Construct a directed acyclic graph based on each graph node, the first edge weight, the second edge weight, and the third edge weight.

[0062] In one possible implementation, the third processing module is further configured to:

[0063] Obtain the optimal sampling trajectory corresponding to multiple training samples;

[0064] The time step transition combinations of each optimal sampling trajectory are analyzed and processed to obtain the transition frequency, repetition pattern and data characteristics corresponding to each optimal sampling trajectory; the analysis and processing includes frequency statistics, repetition pattern recognition and feature extraction.

[0065] Based on the transfer frequency, repetition pattern, and data characteristics corresponding to each optimal sampling trajectory, the common characteristics of each optimal sampling trajectory are determined.

[0066] Based on common characteristics, a general sampling scheduling sequence is generated.

[0067] In one possible implementation, the acquisition module is further configured to:

[0068] Obtain the raw data from the training samples;

[0069] Based on the forward noise addition process of the mathematical model and the diffusion model, the ideal noise state corresponding to each sampling time step is calculated.

[0070] The ideal noise state at each sampling time step is processed by a diffusion model to obtain the original data estimate corresponding to each sampling time step;

[0071] Based on the original data and the estimated values ​​of the original data, the corresponding difference quantization results are determined, and the difference quantization results are used as the reconstruction error for each sampling time step.

[0072] In one possible implementation, the acquisition module is further configured to:

[0073] Based on the mathematical model, the error propagation relationship between different sampling time steps is determined, wherein the error propagation relationship is that the upper bound of the error of the next sampling time step is equal to the minimum value of the sum of the upper bound of the error of the previous sampling time step and the corresponding sampling trajectory correction cost.

[0074] Based on the Euler sampling method of the preset flow model, the noise scheduling correlation coefficient between different sampling time steps is determined. The noise scheduling correlation coefficient is the ratio of the difference between the previous and subsequent sampling time steps to the subsequent sampling time step.

[0075] Based on the error propagation relationship, the noise scheduling correlation coefficient, and the reconstruction error of each sampling time step, the sampling trajectory correction cost between different sampling time steps in the diffusion model sampling process is determined. The sampling trajectory correction cost is the product of the noise scheduling correlation coefficient and the reconstruction error of the corresponding sampling time step.

[0076] In one possible implementation, the second processing module is further configured to:

[0077] Based on the directed acyclic graph, the shortest path is determined to be the first path from the sampling time step node with the smaller noise value to the sampling time step node with the larger noise value;

[0078] According to the diffusion model, the shortest path is reversed to obtain a second path from the sampling time step node with high noise value to the sampling time step node with low noise value.

[0079] The second path is determined as the optimal sampling trajectory for a single sample.

[0080] In one possible implementation, the general sampling scheduling sequence is generated offline;

[0081] Accordingly, the calling module is also used for:

[0082] During online sampling, the original fixed schedule of the diffusion model is replaced by a general sampling schedule sequence generated offline, so that the diffusion model can execute the corresponding sampling process.

[0083] In one possible implementation, the dynamic programming algorithm includes a first algorithm and a second algorithm;

[0084] Accordingly, the first processing module is also used for:

[0085] Determine whether a fixed number of sampling steps is needed;

[0086] If a fixed number of sampling steps is not required, a preset first algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph; wherein, the first algorithm includes Dijkstra's algorithm or Floyd-Vorchard's algorithm;

[0087] If a fixed number of sampling steps is required, a pre-defined second algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph, passing through a preset number of intermediate sampling time step nodes; wherein, the first algorithm is a dynamic programming algorithm with step constraints.

[0088] Thirdly, this application provides a sampling device for a diffusion model, including: a memory and a processor;

[0089] The memory stores the instructions that the computer executes;

[0090] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0091] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect.

[0092] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0093] This application provides a sampling method, apparatus, device, medium, and program product for a diffusion model. It acquires a pre-constructed directed acyclic graph, whose edge weights are based on the cost of correcting the sampling trajectory of training samples, accurately quantifying the time-step adjustment cost. Through dynamic programming, it determines the shortest path from the virtual start to the end node, obtaining the optimal sampling trajectory for a single training sample and reducing invalid sampling steps. It generates a general sampling scheduling sequence based on the optimal trajectories of multiple training samples, avoiding repeated calculations of single-sample trajectories and improving overall efficiency. The general sequence is used to call the diffusion model to perform sampling, making the sampling process more closely match the optimal path of the training samples, thereby achieving the overall technical effect of improving the sampling efficiency and accuracy of the diffusion model. Attached Figure Description

[0094] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0095] Figure 1 This application provides a schematic diagram of an application data processing system architecture.

[0096] Figure 2 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 1 ;

[0097] Figure 3 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 2 ;

[0098] Figure 4 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 3 ;

[0099] Figure 5 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 4 ;

[0100] Figure 6 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 5 ;

[0101] Figure 7 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 6 ;

[0102] Figure 8 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 7 ;

[0103] Figure 9 This is a schematic diagram of the sampling device for the diffusion model provided in the embodiments of this application;

[0104] Figure 10 This is a schematic diagram of the sampling device for the diffusion model provided in this application embodiment.

[0105] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0106] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0107] Because existing technologies suffer from low sampling efficiency and poor accuracy due to the decoupling of scheduling strategies and generated content features, lack of theoretical optimality guarantees, high computational complexity, and insufficient versatility, and are limited by fixed rules and empirical design.

[0108] To address the aforementioned issues, this application provides a sampling method, apparatus, device, medium, and program product for a diffusion model. It acquires a pre-constructed directed acyclic graph, whose edge weights are based on the cost of correcting the sampling trajectory of training samples, accurately quantifying the time-step adjustment cost. Through dynamic programming, it determines the shortest path from the virtual start to the end node, obtaining the optimal sampling trajectory for a single training sample and reducing invalid sampling steps. A general sampling scheduling sequence is generated based on the optimal trajectories of multiple training samples, avoiding repeated calculations of single-sample trajectories and improving overall efficiency. The general sequence is used to call the diffusion model to perform sampling, making the sampling process more closely match the optimal path of the training samples, thereby achieving the overall technical effect of improving the sampling efficiency and accuracy of the diffusion model.

[0109] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0110] Figure 1 This is a schematic diagram of an application data processing system architecture provided in an embodiment of this application. The application data processing system is a computer device. Figure 1As shown, the above architecture includes at least one of a data acquisition device 101, a processing device 102, and a display device 103.

[0111] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the architecture of the application data processing system. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0112] In the specific implementation process, the data acquisition device 101 may include an input / output interface or a communication interface, and the data acquisition device 101 can be connected to the processing device through the input / output interface or the communication interface.

[0113] The processing device 102 can first construct a directed acyclic graph containing virtual start and end nodes and edge weights equal to the cost of sampling trajectory correction; use dynamic programming to find the shortest path from the virtual start to the end node to obtain the optimal sampling trajectory of a single training sample; generate a general sampling scheduling sequence from the optimal trajectories of multiple training samples; and finally use this sequence to call the diffusion model to perform sampling.

[0114] The display device 103 can also be a touch screen or the screen of a terminal device, used to receive user commands while displaying the above-mentioned content, so as to realize interaction with the user.

[0115] It should be understood that the aforementioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0116] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0117] Figure 2 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 1 ,like Figure 2 As shown, the sampling method for the diffusion model provided in this embodiment includes:

[0118] S201. Obtain the pre-constructed directed acyclic graph.

[0119] In this embodiment, the nodes of the directed acyclic graph correspond to the sampling time steps of the diffusion model, the edge weights of the directed acyclic graph are the sampling trajectory correction costs calculated based on the training samples, and the directed acyclic graph also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state.

[0120] Obtain a pre-constructed Directed Acyclic Graph (DAG). For example, if the training samples are high-quality face images, the sampling trajectory correction cost at each time step needs to be calculated as the edge weights. The nodes in the graph correspond to the sampling time steps of the diffusion model, and also include virtual start nodes corresponding to the initial noise state and virtual end nodes corresponding to the target image state. This graph quantifies the correction cost of transitions at each time step during the process of generating images from noise.

[0121] S202. Based on the preset dynamic programming algorithm, determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph.

[0122] The shortest path is determined based on a pre-defined dynamic programming algorithm. An algorithm such as Dijkstra's is used to calculate the shortest path from a virtual start node to a virtual end node in a directed acyclic graph. This path represents the sequence of sampling trajectories that minimizes the cost of correction from the initial noisy state to the target image state, providing a mathematical basis for finding the optimal sampling trajectory.

[0123] S203. Based on the shortest path, obtain the optimal sampling trajectory for the corresponding single training sample.

[0124] The optimal sampling trajectory is obtained based on the shortest path. This trajectory corresponds to a single training sample and represents the optimal denoising process from high-noise time steps to low-noise time steps. It clarifies at which specific time steps the diffusion model should perform denoising operations to minimize error for this sample.

[0125] S204. Generate a general sampling scheduling sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples.

[0126] A general sampling scheduling sequence is generated based on the optimal trajectories of multiple training samples. Statistical analysis is performed on the optimal sampling trajectories of a large number of different training samples to extract common patterns. The generated sequence is no longer targeted at a single sample, but rather is a general time-step scheduling scheme applicable to various image generation methods.

[0127] S205. Based on the general sampling schedule sequence, invoke the diffusion model to execute the corresponding sampling process.

[0128] In this embodiment, the general sampling scheduling sequence is generated offline.

[0129] In one possible implementation, the diffusion model is invoked according to a general sampling schedule sequence to execute the corresponding sampling process, including: during online sampling, invoking a general sampling schedule sequence generated offline to replace the original fixed schedule of the diffusion model so that the diffusion model executes the corresponding sampling process.

[0130] The diffusion model is invoked to perform sampling based on a general sampling schedule sequence. This general sequence is typically generated offline and is directly called during online sampling to replace the model's original fixed schedule. This allows the diffusion model to follow a more optimized time-step path when generating images, improving sampling efficiency and accuracy.

[0131] This application provides a sampling method for a diffusion model. It obtains a pre-constructed directed acyclic graph (DAG), whose edge weights are adjusted based on the sampling trajectory correction cost of training samples, accurately quantifying the time-step adjustment cost. Dynamic programming is used to determine the shortest path from the virtual start to the end node, obtaining the optimal sampling trajectory for a single training sample and reducing invalid sampling steps. A general sampling scheduling sequence is generated based on the optimal trajectories of multiple training samples, avoiding repeated calculations of single-sample trajectories and improving overall efficiency. The general sequence is used to call the diffusion model to perform sampling, making the sampling process more closely match the optimal path of the training samples, thereby achieving the overall technical effect of improving the sampling efficiency and accuracy of the diffusion model.

[0132] Figure 3 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 2 ,like Figure 3 As shown, this embodiment, based on the above embodiments, provides a detailed explanation of the specific construction process of a directed acyclic graph, including:

[0133] S301. Obtain the pre-established mathematical model of error upper bound propagation.

[0134] Obtain a mathematical model for the propagation of a pre-established upper bound on the error. This model is based on the concept of generation as error correction, and defines... For time steps The reconstruction error of single-step denoising, where The model is from Direct prediction The estimated value; this mathematical model follows the following rules:

[0135]

[0136] in, This is the cost of correcting the sampling trajectory.

[0137] Specifically, for a given clean image The ideal noise state is calculated through a forward noise addition process, and the single-step denoising estimation and reconstruction error are calculated using a diffusion model, thereby establishing the mathematical basis for error propagation.

[0138] S302. Based on the mathematical model, determine the reconstruction error of each sampling time step in the diffusion model sampling process, and determine the corresponding multiple inverse quantization values ​​based on the reconstruction error of each sampling time step.

[0139] The reconstruction error and multiple inverse quantization values ​​for each sampling time step are determined based on a mathematical model. This is done for each training image in the training set. The ideal noise state at each time step t is calculated according to the forward diffusion formula. Predict using diffusion models Calculate reconstruction error The inverse quantization value can be expressed as: , representing the error credit with time step k as the starting point for denoising. For example, for time step k=50, the reconstruction error is calculated and then the negative value is taken to obtain the inverse quantization value, which is used for the definition of edge weights in subsequent graph theory modeling.

[0140] S303. Based on the mathematical model and the reconstruction error of each sampling time step, determine the cost of correcting the sampling trajectory between different sampling time steps during the diffusion model sampling process.

[0141] The sampling trajectory correction cost between different sampling time steps is determined based on the mathematical model and reconstruction error. The cost required to transition from time step t to k (t>k) is expressed by the following formula:

[0142]

[0143] in, It is a coefficient related to noise scheduling.

[0144] Specifically, for each pair of time steps, the calculated reconstruction error is used. Sum of coefficients Algebraic operations are performed to obtain the specific cost value required to pull a non-ideal trajectory point back to the ideal trajectory. This value serves as the basis for the edge weights connecting nodes at different time steps in the graph.

[0145] S304. Map each sampling time step of the diffusion model to a corresponding graph node.

[0146] In this embodiment, the number of sampling time steps is the preset maximum number of time steps.

[0147] Each sampling time step of the diffusion model is mapped to a corresponding graph node. A maximum time step T is preset, such as T=1000, and time steps 1, 2, ..., T are mapped to independent nodes in the graph. These nodes represent all possible accessed noisy time steps, forming the main structure of the directed acyclic graph. In the embodiment, when constructing the reverse directed acyclic graph, these nodes carry state information from different denoising stages, providing the basic topology for subsequently connecting virtual start nodes, virtual end nodes, and defining edge weights.

[0148] S305. Set each reconstruction error as the first edge weight between the virtual starting node and each sampling time step node.

[0149] Each reconstruction error is set as the first edge weight between the virtual start node and each sampling time step node. The virtual start node (Start) represents the starting point of generation, and the first edge weight... This represents the remaining error when time step k is used as the denoising endpoint.

[0150] Specifically, for each time step node k, calculate its corresponding reconstruction error. This value is used as the weight (first edge weight) of the directed edge from the virtual starting node to node k, quantifying the initial error cost when entering this time step from the generation starting point.

[0151] S306. Set each inverse quantization value as the second side weight between each sampling time step node and the virtual termination node.

[0152] Each inverse quantization value is set as the second edge weight between each sampling time step node and the virtual termination node. The virtual termination node (End) represents the generated endpoint, and the second edge weight... This represents the error credit with time step k as the starting point for denoising.

[0153] Specifically, for each time step node k, the negative value of its reconstruction error is taken as the weight (second edge weight) of the directed edge from node k to the virtual termination node. This negative weight reflects the error correction gain that can be obtained from denoising starting from this time step and is used to balance the total weight of the path.

[0154] S307. Set the sampling trajectory correction cost as the third side weight between the previous sampling time step and the next sampling time step.

[0155] The correction cost is set as the weight of the third edge between the previous and next sampling time steps. Here, the previous sampling time step refers to the time step t with higher noise, and the next sampling time step refers to the time step k with lower noise (t>k). Third edge weight That is, the cost of sampling trajectory correction. , representing the sampling trajectory correction cost required to transition from time step t to k. In graph construction, for all time step pairs satisfying t>k, the calculated cost is... The weight of the directed edge from node t to node k (the third edge weight) defines the cost of time step jumps during the sampling process.

[0156] S308. Construct a directed acyclic graph based on each graph node, the first edge weight, the second edge weight, and the third edge weight.

[0157] Construct a directed acyclic graph (DAG) based on each graph node, its first edge weight, second edge weight, and third edge weight. Integrate the T sampling time step nodes, virtual start node, and virtual end node obtained from the mapping. Connect the virtual start node to each time step node according to the first edge weight, connect each time step node to the virtual end node according to the second edge weight, and connect each time step pair (t>k) according to the third edge weight. In the final graph structure, any path from Start to End corresponds to a sampling trajectory, and the total path weight is: This provides a complete graph theory model for subsequent dynamic programming solutions to find the shortest path.

[0158] The sampling method of the diffusion model provided in this application establishes a mathematical model for the propagation of error upper bounds, accurately calculates the reconstruction error and correction cost at each time step, and transforms the sampling trajectory optimization problem into a shortest path problem in graph theory. By constructing a directed acyclic graph using virtual start and end nodes and three types of edge weights, finding the optimal sampling trajectory becomes solving for a deterministic path, effectively reducing invalid sampling steps. A general sampling scheduling sequence is generated based on statistical analysis of multiple training samples, enabling the diffusion model to directly call the pre-calculated optimal path during online sampling, significantly improving sampling efficiency and reducing generation error, thus achieving high-quality image generation.

[0159] Figure 4 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 3 ,like Figure 4 As shown, this embodiment, based on the above embodiments, elaborates on the generation process of the general sampling scheduling sequence, including:

[0160] S401. Obtain the optimal sampling trajectory corresponding to multiple training samples.

[0161] Obtain the optimal sampling trajectory corresponding to multiple training samples. Training samples are high-quality images; for each image... Calculate the ideal noise state at each time step t using the forward diffusion formula. The diffusion model is used to calculate the single-step denoising estimate. and reconstruction error Construct a directed acyclic graph containing virtual start and end nodes, use dynamic programming to solve for the shortest path from the virtual start to the end, and then reverse the path to obtain the optimal sampling trajectory of the sample.

[0162] S402. Analyze and process the time step transfer combination of each optimal sampling trajectory to obtain the transfer frequency, repetition pattern and data characteristics corresponding to each optimal sampling trajectory.

[0163] In this embodiment, the analysis and processing include frequency statistics, repetition pattern recognition, and feature extraction.

[0164] For each optimal sampled trajectory, the time step transition combination analysis is performed to obtain the transition frequency, repetition pattern, and data characteristics. The analysis includes frequency statistics, repetition pattern identification, and feature extraction. It records the transition relationship between adjacent time steps in the trajectory, counts the occurrence frequency of specific time step pairs (t,k), identifies repetition patterns such as high-frequency jump intervals, and extracts data characteristics such as step length distribution.

[0165] S403. Based on the transfer frequency, repetition pattern and data characteristics corresponding to each optimal sampling trajectory, determine the common characteristics of each optimal sampling trajectory.

[0166] Common features are determined based on transfer frequency, repetition patterns, and data characteristics. By comparing the analysis results of different samples, general patterns are identified, such as the optimal trajectory of most images concentrating on transfers within a specific time step interval, or certain time step pairs appearing frequently in multiple samples. These stable patterns across samples are the common features.

[0167] S404. Generate a general sampling scheduling sequence based on common characteristics.

[0168] Generate a general sampling scheduling sequence based on common features. Transform the common features into a specific time step sequence. For example, if the common features show that the transition frequency of time steps 1000, 500, 200, and 0 is the highest, then generate a sequence containing these time steps. This sequence is suitable for the general scheduling of online sampling of the diffusion model.

[0169] The sampling method of the diffusion model provided in this application extracts the stable time step transition pattern across samples by statistically analyzing the optimal sampling trajectory of multiple training samples. The generated general sequence enables the diffusion model to directly call the pre-calculated optimal path pattern during online sampling, avoiding real-time pathfinding overhead, reducing the number of sampling steps, improving efficiency while maintaining generation quality.

[0170] Figure 5 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 4 ,like Figure 5 As shown, this embodiment, based on the above embodiment, elaborates on the process of determining the reconstruction error at each sampling time step, including:

[0171] S501. Obtain the original data from the training samples.

[0172] Obtain the raw data from the training samples. The raw data consists of clean images, such as high-quality face images, which serve as the foundational data for the training samples and are used to subsequently calculate the ideal noise state and reconstruction error at each sampling time step.

[0173] S502. Based on the forward noise addition process of the mathematical model and the diffusion model, calculate the ideal noise state corresponding to each sampling time step.

[0174] The ideal noise state is calculated based on the forward noise addition process using mathematical and diffusion models. The forward noise addition process starts from the raw data. Initially, standard Gaussian noise is added step by step according to the diffusion formula to obtain the ideal noise state corresponding to each sampling time step t. For example, when time step t=500, the ideal noise state at that time step can be calculated using the forward formula.

[0175] S503. The ideal noise state at each sampling time step is processed by the diffusion model to obtain the original data estimate corresponding to each sampling time step.

[0176] The raw data estimates are obtained by processing the ideal noise state using a diffusion model. The ideal noise state at each time step is then calculated. Input a diffusion model, and the model predicts the estimated value of the raw data corresponding to the current state. That is, from estimated value.

[0177] S504. Based on the original data and the estimated value of the original data, determine the corresponding difference quantization result, and use the difference quantization result as the reconstruction error for each sampling time step.

[0178] The difference quantification result is determined as the reconstruction error based on the raw data and the estimated values. The raw data is calculated. Compared with the estimated value Differences, such as the square of the Euclidean distance This difference is the reconstruction error at time step t. .

[0179] The sampling method of the diffusion model provided in this application obtains the original data, calculates the ideal noise state, obtains the estimated value of the original data and determines the difference quantization result, accurately calculates the reconstruction error at each time step, provides a basis for constructing a directed acyclic graph and searching for the optimal sampling trajectory, makes the sampling process more in line with the actual error distribution, and improves sampling efficiency and accuracy.

[0180] Figure 6Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 5 ,like Figure 6 As shown, this embodiment, based on the above embodiments, provides a detailed explanation of the process for determining the sampling trajectory correction cost, including:

[0181] S601. Based on the mathematical model, determine the error propagation relationship between different sampling time steps.

[0182] In this embodiment, the error propagation relationship is that the upper bound of the error in the next sampling time step is equal to the minimum value of the sum of the upper bound of the error in the previous sampling time step and the corresponding sampling trajectory correction cost.

[0183] The error propagation relationship between different sampling time steps is determined based on a mathematical model. This relationship is expressed as the upper bound of the error at the next sampling time step being equal to the minimum sum of the upper bound of the error at the previous sampling time step and the correction cost of the corresponding sampling trajectory. The specific formula is as follows:

[0184]

[0185] Where t>k, The reconstruction error is the result of single-step denoising at time step t. This relationship forms the theoretical basis for error upper bound propagation, quantifying the upper limit of error accumulation caused by time step jumps.

[0186] S602. Based on the preset Euler sampling method of the flow model, determine the noise scheduling correlation coefficient between different sampling time steps.

[0187] In this embodiment, the noise scheduling correlation coefficient is the ratio of the difference between the previous and subsequent sampling time steps to the value of the next sampling time step.

[0188] The noise scheduling correlation coefficient is determined based on the Eulerian sampling method of the pre-defined flow model. (Noise scheduling correlation coefficient) The value is the ratio of the difference between the next sampling time step t and the previous sampling time step k to the next sampling time step t. The calculation formula is: This coefficient reflects the noise scheduling weight between different time steps. It is calculated in the flow model through Euler sampling and is used to measure the cost weight of transitioning from time step t to k.

[0189] S603. Based on the error propagation relationship, noise scheduling correlation coefficient, and reconstruction error at each sampling time step, determine the cost of correcting the sampling trajectory between different sampling time steps during the diffusion model sampling process.

[0190] In this embodiment, the cost of sampling trajectory correction is the product of the noise scheduling correlation coefficient and the reconstruction error of the corresponding sampling time step.

[0191] The cost of sampling trajectory correction is determined based on error propagation, noise scheduling correlation coefficient, and reconstruction error. Noise dispatch correlation coefficient Reconstruction error at corresponding sampling time step t The product of, i.e. The sampling trajectory correction cost represents the cost required to pull a non-ideal trajectory point back to the ideal trajectory from time step t.

[0192] The sampling method for the diffusion model provided in this application accurately quantifies the correction cost between different sampling time steps by establishing a mathematical model of error propagation and combining it with Euler sampling to calculate noise scheduling coefficients. This cost serves as the edge weight of a directed acyclic graph, transforming the search for the optimal sampling trajectory into solving the shortest path problem. This process accurately reflects the propagation law of error between time steps, allowing the diffusion model to select the path with the minimum correction cost, effectively reducing invalid sampling steps, and significantly improving sampling efficiency and generation accuracy.

[0193] Figure 7 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 6 ,like Figure 7 As shown, this embodiment, based on the above embodiments, provides a detailed explanation of the specific process for obtaining the optimal sampling trajectory for a single sample, including:

[0194] S701. Based on the directed acyclic graph, determine the shortest path as the first path from the sampling time step node with the smaller noise value to the sampling time step node with the larger noise value.

[0195] The shortest path is determined using a directed acyclic graph (DAG), where nodes represent sampling time steps. Edge weights are determined by the difference in noise values ​​between adjacent time steps. Using a shortest path algorithm, the first path from a sampling time step node with a smaller noise value to a sampling time step node with a larger noise value is found; this path represents the optimal sequence for noise growth.

[0196] Specifically, in the directed acyclic graph, each node represents a sampling time step. The difference in noise values ​​between adjacent time steps is calculated as the edge weight. The smaller the noise value difference, the smaller the weight. Then, Dijkstra's algorithm is applied to find paths to sampling time step nodes with large noise values, starting from all nodes with small noise values. The path with the smallest total weight is selected as the first path. This path ensures that the cumulative difference during the noise growth process is minimized, providing a basis for subsequent reversal.

[0197] S702. Based on the diffusion model, the shortest path is reversed to obtain a second path from the sampling time step node with a large noise value to the sampling time step node with a small noise value.

[0198] The shortest path is inverted based on the diffusion model. The nodes of the first path are reversed to obtain the second path. This path points from the sampling time step node with the highest noise value to the sampling time step node with the lowest noise value. This inversion operation corresponds to the reverse generation process of the diffusion model.

[0199] Specifically, after obtaining the first path, the node sequence is directly reversed. For example, the original path is t0-t1-t2, and after reversal it becomes t2-t1-t0. Here, t2 has a higher noise value, and t0 has a lower noise value. This reversed path corresponds to the inverse denoising process of the diffusion model. At each time step, the model predicts noise and subtracts it from the current state. This process utilizes the model's learned denoising capabilities to gradually restore high-noise states to low-noise states, forming a continuous generated trajectory.

[0200] S703. Determine the second path as the optimal sampling trajectory for the corresponding single sample.

[0201] The second path is determined as the optimal sampling trajectory for a single sample. This trajectory starts from a high-noise state and gradually transitions to a low-noise state. It guides the generation process of a single sample, ensuring that the sampling path is optimal.

[0202] Specifically, the second path is used as the generation instruction, starting from the sampling time step node with the highest noise value, with pure noise as input. The process proceeds sequentially along the path, passing through each time step node, applying a diffusion model for denoising. The output of each step serves as the input for the next step, until the sampling time step node with the lowest noise value is reached. The final output is the generated single sample. This trajectory, based on the shortest path reversal, ensures the efficiency and quality of the generation process, reducing unnecessary computational steps.

[0203] The sampling method for the diffusion model provided in this application plans the noise growth path through a graph structure and obtains the generation trajectory using model inversion. This method determines the optimal sequence from high noise to low noise. It avoids the uncertainty of random sampling and improves generation efficiency. At the same time, it reduces computational costs and ensures the generation quality of individual samples. The entire process achieves efficient and stable sample generation.

[0204] Figure 8 Flowchart of the sampling method for the diffusion model provided in the embodiments of this application Figure 7 ,like Figure 8 As shown, this embodiment, based on the above embodiment, provides a supplementary explanation of the process for obtaining the shortest path. The dynamic programming algorithm includes a first algorithm and a second algorithm, and the process includes:

[0205] S801. Determine whether a fixed number of sampling steps is required.

[0206] Determine whether a fixed number of sampling steps is needed. This determination depends on the specific requirements of the generated task.

[0207] S802. If a fixed number of sampling steps is not required, the first preset algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph.

[0208] In this embodiment, the first algorithm includes the Dijkstra algorithm or the Floyd-Warshall algorithm.

[0209] If a fixed number of sampling steps is not required, a pre-defined first algorithm is used to determine the shortest path. This first algorithm includes either the Dijkstra algorithm or the Floyd-Worchard algorithm. The first algorithm calculates the shortest path from a virtual start node to a virtual end node in a directed acyclic graph. This path represents the sequence with the smallest cumulative difference in noise value changes and is not limited by the number of steps.

[0210] Specifically, if Dijkstra's algorithm is used, it starts from the virtual starting node and greedily expands using a priority queue to calculate the shortest path to the virtual ending node. If Floyd-Worchard's algorithm is used, it calculates the shortest path matrix between all node pairs through dynamic programming and then extracts the required path. For example, when the number of nodes is small, Floyd-Worchard's algorithm is used for preprocessing, while when the number of nodes is large, Dijkstra's algorithm is used for a single query. This process ensures that a globally optimal path is found, providing the best foundation for subsequent reversal.

[0211] S803. If a fixed number of sampling steps is required, the preset second algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph through a preset number of intermediate sampling time step nodes.

[0212] In this embodiment, the first algorithm is a dynamic programming algorithm with step constraints.

[0213] If a fixed number of sampling steps is required, a pre-defined second algorithm is used. This second algorithm is a dynamic programming algorithm with step constraints. It searches for the shortest path from the virtual start node to the virtual end node in a directed acyclic graph. This path must pass through a pre-defined number of intermediate sampling time step nodes to meet the step constraint requirement.

[0214] The sampling method for the diffusion model provided in this application flexibly switches algorithms based on the required number of sampling steps. When a fixed number of steps is not required, Dijkstra's or Floyd-Worchard's algorithms are used to find the globally optimal path. When a fixed number of steps is required, a constrained dynamic programming algorithm is used to ensure accurate path length. This method balances path optimality and step controllability, adapting to different generation scenarios. It improves the flexibility of sampling trajectory planning, reduces invalid computation, and ensures the stability and resource utilization of the generation process.

[0215] Figure 9 This is a schematic diagram of the sampling device for the diffusion model provided in an embodiment of this application. The device in this embodiment can be in the form of software and / or hardware. Figure 9 As shown in the embodiment of this application, the sampling device 900 for the diffusion model includes: an acquisition module 901, a first processing module 902, a second processing module 903, a third processing module 904, and a calling module 905.

[0216] The acquisition module 901 is used to acquire a pre-constructed directed acyclic graph; wherein, the nodes of the directed acyclic graph correspond to the sampling time steps of the diffusion model, the edge weights of the directed acyclic graph are the sampling trajectory correction costs calculated based on the training samples, and the directed acyclic graph also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state.

[0217] The first processing module 902 is used to determine the shortest path from the virtual start node to the virtual end node in a directed acyclic graph according to a preset dynamic programming algorithm.

[0218] The second processing module 903 is used to obtain the optimal sampling trajectory for a single sample based on the shortest path.

[0219] The third processing module 904 is used to generate a general sampling scheduling sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples.

[0220] Module 905 is used to invoke the diffusion model according to the general sampling schedule sequence to execute the corresponding sampling process.

[0221] In one possible implementation, the acquisition module 901 is further configured to:

[0222] Obtain a pre-established mathematical model for the propagation of the upper bound of the error;

[0223] Based on the mathematical model, the reconstruction error at each sampling time step in the diffusion model sampling process is determined, and multiple corresponding inverse quantization values ​​are determined based on the reconstruction error at each sampling time step.

[0224] Based on the mathematical model and the reconstruction error at each sampling time step, the cost of correcting the sampling trajectory between different sampling time steps during the diffusion model sampling process is determined.

[0225] Each sampling time step of the diffusion model is mapped to a corresponding graph node, where the number of sampling time steps is a preset maximum number of time steps;

[0226] Each reconstruction error is set as the first edge weight between the virtual starting node and each sampling time step node;

[0227] Set each inverse quantization value as the second side weight between each sampling time step node and the virtual termination node;

[0228] Set the cost of correcting the sampling trajectory to the third side weight between the previous sampling time step and the next sampling time step;

[0229] Construct a directed acyclic graph based on each graph node, the first edge weight, the second edge weight, and the third edge weight.

[0230] In one possible implementation, the third processing module 904 is further configured to:

[0231] Obtain the optimal sampling trajectory corresponding to multiple training samples;

[0232] The time step transition combinations of each optimal sampling trajectory are analyzed and processed to obtain the transition frequency, repetition pattern and data characteristics corresponding to each optimal sampling trajectory; the analysis and processing includes frequency statistics, repetition pattern recognition and feature extraction.

[0233] Based on the transfer frequency, repetition pattern, and data characteristics corresponding to each optimal sampling trajectory, the common characteristics of each optimal sampling trajectory are determined.

[0234] Based on common characteristics, a general sampling scheduling sequence is generated.

[0235] In one possible implementation, the acquisition module 901 is further configured to:

[0236] Obtain the raw data from the training samples;

[0237] Based on the forward noise addition process of the mathematical model and the diffusion model, the ideal noise state corresponding to each sampling time step is calculated.

[0238] The ideal noise state at each sampling time step is processed by a diffusion model to obtain the original data estimate corresponding to each sampling time step;

[0239] Based on the original data and the estimated values ​​of the original data, the corresponding difference quantization results are determined, and the difference quantization results are used as the reconstruction error for each sampling time step.

[0240] In one possible implementation, the acquisition module 901 is further configured to:

[0241] Based on the mathematical model, the error propagation relationship between different sampling time steps is determined, wherein the error propagation relationship is that the upper bound of the error of the next sampling time step is equal to the minimum value of the sum of the upper bound of the error of the previous sampling time step and the corresponding sampling trajectory correction cost.

[0242] Based on the Euler sampling method of the preset flow model, the noise scheduling correlation coefficient between different sampling time steps is determined. The noise scheduling correlation coefficient is the ratio of the difference between the previous and subsequent sampling time steps to the subsequent sampling time step.

[0243] Based on the error propagation relationship, the noise scheduling correlation coefficient, and the reconstruction error of each sampling time step, the sampling trajectory correction cost between different sampling time steps in the diffusion model sampling process is determined. The sampling trajectory correction cost is the product of the noise scheduling correlation coefficient and the reconstruction error of the corresponding sampling time step.

[0244] In one possible implementation, the second processing module 903 is further configured to:

[0245] Based on the directed acyclic graph, the shortest path is determined to be the first path from the sampling time step node with the smaller noise value to the sampling time step node with the larger noise value;

[0246] According to the diffusion model, the shortest path is reversed to obtain a second path from the sampling time step node with high noise value to the sampling time step node with low noise value.

[0247] The second path is determined as the optimal sampling trajectory for a single sample.

[0248] In one possible implementation, the general sampling scheduling sequence is generated offline;

[0249] Accordingly, module 905 is also used for:

[0250] During online sampling, the original fixed schedule of the diffusion model is replaced by a general sampling schedule sequence generated offline, so that the diffusion model can execute the corresponding sampling process.

[0251] In one possible implementation, the dynamic programming algorithm includes a first algorithm and a second algorithm;

[0252] Accordingly, the first processing module 902 is also used for:

[0253] Determine whether a fixed number of sampling steps is needed;

[0254] If a fixed number of sampling steps is not required, a preset first algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph; wherein, the first algorithm includes Dijkstra's algorithm or Floyd-Vorchard's algorithm;

[0255] If a fixed number of sampling steps is required, a pre-defined second algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph, passing through a preset number of intermediate sampling time step nodes; wherein, the first algorithm is a dynamic programming algorithm with step constraints.

[0256] The sampling device for the diffusion model provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0257] Figure 10 This is a schematic diagram of the sampling device for the diffusion model provided in an embodiment of this application. Figure 10 As shown, the sampling device 1000 for the diffusion model provided in this embodiment includes at least one processor 1001 and a memory 1002. Optionally, the device 1000 further includes a communication component 1003. The processor 1001, memory 1002, and communication component 1003 are connected via a bus.

[0258] In a specific implementation, at least one processor 1001 executes computer execution instructions stored in memory 1002, causing at least one processor 1001 to perform the above-described method.

[0259] The specific implementation process of processor 1001 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0260] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0261] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0262] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0263] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0264] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0265] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0266] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0267] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0268] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0269] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0270] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0271] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0272] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A sampling method for a diffusion model, characterized in that, include: Obtain a pre-constructed directed acyclic graph; wherein, the nodes of the directed acyclic graph correspond to the sampling time steps of the diffusion model, the edge weights of the directed acyclic graph are the sampling trajectory correction costs calculated based on the training samples, and the directed acyclic graph also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state. Based on a preset dynamic programming algorithm, determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph; Based on the shortest path, the optimal sampling trajectory corresponding to a single training sample is obtained; A general sampling schedule sequence is generated based on the optimal sampling trajectory corresponding to each of the multiple training samples; According to the general sampling scheduling sequence, the diffusion model is invoked to execute the corresponding sampling process.

2. The method according to claim 1, characterized in that, Before obtaining the pre-constructed directed acyclic graph, the method further includes: Obtain a pre-established mathematical model for the propagation of the upper bound of the error; Based on the mathematical model, the reconstruction error of each sampling time step in the diffusion model sampling process is determined, and based on the reconstruction error of each sampling time step, multiple corresponding inverse quantization values ​​are determined. Based on the mathematical model and the reconstruction error at each sampling time step, determine the sampling trajectory correction cost between different sampling time steps during the diffusion model sampling process; Each sampling time step of the diffusion model is mapped to a corresponding graph node, wherein the number of sampling time steps is a preset maximum number of time steps; Each of the reconstruction errors is set as the first edge weight between the virtual starting node and each of the sampling time step nodes; Each of the inverse quantization values ​​is set as the second side weight between each of the sampling time step nodes and the virtual termination node; The cost of correcting the sampling trajectory is set as the third side weight between the previous sampling time step and the next sampling time step; The directed acyclic graph is constructed based on each graph node, the first edge weight, the second edge weight, and the third edge weight.

3. The method according to claim 2, characterized in that, The step of generating a general sampling scheduling sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples includes: Obtain the optimal sampling trajectory corresponding to multiple training samples; The time step transition combinations of each optimal sampling trajectory are analyzed and processed to obtain the transition frequency, repetition pattern, and data features corresponding to each optimal sampling trajectory; wherein, the analysis and processing includes frequency statistics, repetition pattern recognition, and feature extraction; Based on the transfer frequency, repetition pattern, and data characteristics corresponding to each optimal sampling trajectory, the common characteristics of each optimal sampling trajectory are determined; Based on the common characteristics, the general sampling scheduling sequence is generated.

4. The method according to claim 3, characterized in that, The step of determining the reconstruction error at each sampling time step in the diffusion model sampling process based on the mathematical model includes: Obtain the raw data from the training samples; Based on the mathematical model and the forward noise addition process of the diffusion model, calculate the ideal noise state corresponding to each sampling time step; The ideal noise state at each sampling time step is processed by the diffusion model to obtain the original data estimate corresponding to each sampling time step; Based on the original data and the estimated value of the original data, the corresponding difference quantization result is determined, and the difference quantization result is used as the reconstruction error for each sampling time step.

5. The method according to claim 4, characterized in that, The step of determining the sampling trajectory correction cost between different sampling time steps during the diffusion model sampling process, based on the mathematical model and the reconstruction error of each sampling time step, includes: Based on the mathematical model, the error propagation relationship between different sampling time steps is determined, wherein the error propagation relationship is that the upper bound of the error of the later sampling time step is equal to the minimum value of the sum of the upper bound of the error of the previous sampling time step and the corresponding sampling trajectory correction cost. Based on the Euler sampling method of the preset flow model, the noise scheduling correlation coefficient between different sampling time steps is determined, wherein the noise scheduling correlation coefficient is the ratio of the difference between the next sampling time step and the previous sampling time step to the next sampling time step. Based on the error propagation relationship, the noise scheduling correlation coefficient, and the reconstruction error of each sampling time step, the sampling trajectory correction cost between different sampling time steps during the diffusion model sampling process is determined, wherein the sampling trajectory correction cost is the product of the noise scheduling correlation coefficient and the reconstruction error of the corresponding sampling time step.

6. The method according to claim 5, characterized in that, The step of obtaining the optimal sampling trajectory for a single sample based on the shortest path includes: Based on the directed acyclic graph, the shortest path is determined to be the first path from the sampling time step node with a small noise value to the sampling time step node with a large noise value; According to the diffusion model, the shortest path is reversed to obtain a second path from the sampling time step node with the larger noise value to the sampling time step node with the smaller noise value; The second path is determined as the optimal sampling trajectory for the corresponding single sample.

7. The method according to claim 6, characterized in that, The general sampling scheduling sequence is generated offline. Accordingly, the step of invoking the diffusion model according to the general sampling scheduling sequence to execute the corresponding sampling process includes: During online sampling, the general sampling schedule sequence generated in the offline method is called to replace the original fixed schedule of the diffusion model, so that the diffusion model can perform the corresponding sampling process.

8. The method according to any one of claims 1-7, characterized in that, The dynamic programming algorithm includes a first algorithm and a second algorithm; Accordingly, determining the shortest path from the virtual start node to the virtual end node in the directed acyclic graph according to a preset dynamic programming algorithm includes: Determine whether a fixed number of sampling steps is needed; If the fixed number of sampling steps is not required, the first algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph; wherein, the first algorithm includes Dijkstra's algorithm or Floyd-Worchard's algorithm. If the fixed number of sampling steps is required, the preset second algorithm is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph through a preset number of intermediate sampling time step nodes; wherein, the first algorithm is a dynamic programming algorithm with step constraints.

9. A sampling device for a diffusion model, characterized in that, include: An acquisition module is used to acquire a pre-constructed directed acyclic graph (DAG); wherein the nodes of the DAG correspond to the sampling time steps of the diffusion model, the edge weights of the DAG are the sampling trajectory correction costs calculated based on the training samples, and the DAG also includes a virtual start node corresponding to the initial noise state and a virtual end node corresponding to the target image state. The first processing module is used to determine the shortest path from the virtual start node to the virtual end node in the directed acyclic graph according to a preset dynamic programming algorithm. The second processing module is used to obtain the optimal sampling trajectory for a single sample based on the shortest path. The third processing module is used to generate a general sampling scheduling sequence based on the optimal sampling trajectory corresponding to each of the multiple training samples. The calling module is used to invoke the diffusion model according to the general sampling scheduling sequence to execute the corresponding sampling process.

10. A sampling device for a diffusion model, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images