Method for accelerating diffusion probabilistic model based on upper bound of global discretization error and apparatus for performing the same

By optimizing the temporal positions of sampling steps using an error evaluation function and non-uniform time schedule, the method addresses computational inefficiencies and quality degradation in diffusion probabilistic models, achieving reduced computation and maintained quality.

KR103046907B1Active Publication Date: 2026-11-10CHUNG ANG UNIV IND ACADEMIC COOP FOUND
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
KR1020260025647
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2026-02-09
Publication Date
2026-11-10
Estimated Expiration
2046-02-09

AI Technical Summary

Technical Problem

Diffusion probabilistic models require numerous sampling steps, leading to increased computational costs and longer inference times, and existing heuristic-based scheduling methods lack optimality across varying conditions, resulting in potential quality degradation and error accumulation.

Method used

A method to optimize the temporal position of sampling steps by setting an error evaluation function based on the upper bound of global discretization error, calculating a non-uniform time schedule, and converting it into a discrete sampling step sequence to minimize error, thereby reducing computational load without retraining the model.

Benefits of technology

This approach effectively reduces the number of function evaluations and total computation time while maintaining generation quality by strategically distributing sampling steps based on error sensitivity, avoiding uniform step distribution pitfalls.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112026016891928-PAT00173_ABST
    Figure 112026016891928-PAT00173_ABST
Patent Text Reader

Abstract

The present invention relates to a method for accelerating a Diffusion Probabilistic Model and a computing device for performing the same. The present invention relates to a method for generating the second data from the first data by driving the Diffusion Probabilistic Model according to the generated sampling step sequence, wherein the method comprises setting an error evaluation function based on an upper bound of the Global Discretization Error that occurs when a pre-trained Diffusion Probabilistic Model approximates a sampling process for generating second data from first data with a finite number of sampling steps, optimizing the temporal positions of the sampling steps to minimize the value of the error evaluation function to calculate a non-uniform time schedule, mapping the calculated non-uniform time schedule to generate an executable discrete sampling step sequence according to a preset operating environment of the Diffusion Probabilistic Model, and driving the Diffusion Probabilistic Model according to the generated sampling step sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention belongs to the field of artificial intelligence and deep learning technology, and in particular, relates to a method for accelerating a diffusion probabilistic model and an apparatus for performing the same, which optimizes the temporal position of a sampling step in the sampling process of a generative model, such as a diffusion probabilistic model, based on an upper bound of error, thereby reducing the amount of sampling computation without retraining the model while maintaining the quality of generation or restoration. Background Technology

[0002] Diffusion Probabilistic Models are generative models that provide excellent performance in various application fields, including image generation and restoration (denoising, deblurring, removal of weather effects, etc.), and have recently been utilized as an important technology in the fields of artificial intelligence and deep learning. Generally, diffusion probabilistic models generate target data by repeatedly performing a reverse diffusion process, in which noise is progressively removed, through multiple sampling steps. However, due to this iterative sampling structure, a large number of steps is required to obtain high-quality results, leading to increased computational costs and longer inference times.

[0003] To mitigate this, methods such as reducing the number of sampling steps or deploying steps at uniform intervals have been widely used. However, since a uniform schedule places steps at equal intervals across the entire sampling period, generation quality may degrade because insufficient steps are allocated in specific sections where restoration difficulty or error sensitivity is high; conversely, computational waste may occur as an unnecessarily large number of steps are placed in sections with low error sensitivity.

[0004] Furthermore, while various heuristic-based scheduling techniques have been proposed to improve step allocation, these methods tend to rely on empirically determined rules for specific datasets or model configurations, presenting a limitation in that it is difficult to guarantee optimal performance under different conditions or application environments. Moreover, although the sampling process of a diffusion probability model can be interpreted as a continuous-time or continuous-parameter-based process, actual implementations approximate it by discretizing it into a finite number of steps; consequently, a Global Discretization Error can accumulate during this approximation process, and the accumulated error and final quality can vary significantly depending on the step allocation.

[0005] Therefore, there is a continuous demand for technology that can stably maintain and improve generation or restoration quality while reducing the number of samples and computational load without model retraining, by quantifying the global discretization error occurring in the sampling process of a diffusion probability model into a theoretically evaluable form and systematically optimizing the temporal position of the sampling step based on its upper bound. Prior art literature

[0006] Republic of Korea Published Patent No. 10-2023-0165686 The problem to be solved

[0007] The present invention was developed to solve the problems of the aforementioned background technology, and aims to provide an acceleration method and an apparatus for performing the same that can maintain or improve generation / recovery quality while reducing the amount of computation required during the sampling process of a Diffusion Probabilistic Model.

[0008] Specifically, since the diffusion probability model generates second data from first data by repeatedly performing the reverse diffusion process with multiple sampling steps, there is a problem in that inference time increases and computational resources are excessively consumed as the number of sampling steps increases. Conversely, if the number of steps is simply reduced or steps are arranged using a uniform schedule, quality may degrade rapidly because sufficient steps are not placed in sections of the sampling interval that are highly sensitive to error. Furthermore, existing heuristic-based scheduling has limitations, such as significant performance variability depending on the application environment or data characteristics, and the difficulty of theoretically guaranteeing optimality.

[0009] Accordingly, the present invention aims to ensure stable quality while reducing the number of samples (e.g., NFE) without retraining the model by (i) theoretically quantifying the Global Discretization Error that occurs during the process of approximating a sampling process interpretable based on continuous time or continuous parameters by discretizing it into finite steps, (ii) setting an error index based on the upper bound of the Global Discretization Error, (iii) calculating a Non-uniform Time Schedule by optimizing the temporal positions of the sampling steps so that the error index is minimized, and (iv) converting the calculated Non-uniform Time Schedule into a discrete sampling step sequence executable in the model's preset operating environment and applying it.

[0010] In other words, the core objective of the present invention is to systematically achieve a balance between computational efficiency and generation quality by automatically determining which intervals require more dense placement of steps through a theoretical error index based on a global discretization error upper limit, instead of empirically placing sampling steps. means of solving the problem

[0011] As a means for achieving the technical problem described above, according to one aspect of the present invention, the means for solving the problem described above is merely illustrative and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments described in the drawings and the detailed description of the invention may exist.

[0012] According to one aspect of the present invention, as a means for achieving the technical problem described above, a method for accelerating a diffusion probabilistic model performed in a computing device having one or more processors and memory is provided, comprising: a step of setting an error evaluation function based on an upper bound of a global discretization error that occurs when a pre-trained diffusion probabilistic model approximates a sampling process for generating second data from first data with a finite number of sampling steps; a step of calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so that the value of the error evaluation function is minimized; a step of generating a discrete sampling step sequence executable by mapping the calculated non-uniform time schedule to a preset operating environment of the diffusion probabilistic model; and a step of driving the diffusion probabilistic model according to the generated sampling step sequence to generate the second data from the first data.

[0013] According to one aspect of the present invention, a method is provided in which the sampling process performs a reverse diffusion process to generate noise-reduced second data from noise-added first data.

[0014] According to one aspect of the present invention, a method is provided characterized in that the step of setting the error evaluation function further comprises the step of reparameterizing the time variable of the sampling process into a log-signal-to-noise ratio (log-SNR) and formulating the reparameterized sampling process into a probability flow ordinary differential equation (Probability Flow ODE).

[0015] According to one aspect of the present invention, a method is provided in which the error evaluation function is set to an upper bound value derived from an error bounding expression indicating that the global discretization error occurring when the sampling process is approximated by discretizing it into a finite number of sampling steps, or a function value based on said upper bound, based on the difference between a continuous-time-based modeling result and an approximation result according to said discretization, wherein the global discretization error is less than or equal to a specific value.

[0016] According to one aspect of the present invention, a method is provided characterized in that the upper limit of the global discretization error is composed of a fixed component determined by the start and end times of the sampling interval and independent of the temporal positions of the sampling steps, and a variable error component whose value changes according to the temporal positions of the sampling steps.

[0017] According to one aspect of the present invention, a method is provided in which the step of calculating the non-uniform time schedule is characterized by setting only the variable error component, excluding the fixed component, as the objective function, and optimizing the temporal positions of the sampling steps so that the value of the objective function is minimized.

[0018] According to one aspect of the present invention, a method is provided in which the non-uniform time schedule is calculated such that the value of the error evaluation function is minimized by densely arranging sampling steps in sections with high contribution to the variable error component and sparsely arranging sampling steps in sections with low contribution.

[0019] According to one aspect of the present invention, a method is provided characterized in that the step of optimizing the temporal positions of the sampling steps is performed by applying a boundary constraint that fixes the start step and end step of the sampling interval and a monotonic constraint that prevents the order between adjacent sampling steps from being reversed.

[0020] According to one aspect of the present invention, a method is provided comprising the step of generating a non-uniform time schedule by mapping it into a discrete sampling step sequence, the step of performing scaling and rounding to convert continuous values ​​included in the non-uniform time schedule into discrete values ​​operable by the diffusion probability model; the step of performing clamping so that the rounded values ​​exist within a preset valid range; and the step of removing duplicates if duplicates exist among the converted values ​​and correcting the sequence to satisfy monotonicity constraints.

[0021] According to one aspect of the present invention, a method is provided in which the diffusion probability model is applied to at least one image restoration task among image denoising, deblurring, or weather removal, and the non-uniform time schedule is calculated to satisfy a preset quality criterion with fewer Number of Function Evaluations (NFE) compared to a uniform schedule.

[0022] According to another aspect of the present invention, a computing device comprising: a processor; and a memory for storing one or more programs executed by the processor, wherein the processor performs the following operations: setting an error evaluation function based on an upper bound of a Global Discretization Error that occurs when a pre-trained diffusion probability model approximates a sampling process for generating second data from first data with a finite number of sampling steps; calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so as to minimize the value of the error evaluation function; generating a discrete sampling step sequence executable by mapping the calculated non-uniform time schedule to a preset operating environment of the diffusion probability model; and driving the diffusion probability model according to the generated sampling step sequence to generate the second data from the first data.

[0023] According to another aspect of the present invention, a computer program stored in a non-transitory computer-readable storage medium is provided, wherein the computer program comprises one or more instructions, and when the instructions are executed by a computing device having one or more processors, the computing device performs the steps of: setting an error evaluation function based on an upper bound of a Global Discretization Error that occurs when a pre-learned diffusion probability model approximates a sampling process for generating second data from first data in a finite number of sampling steps; calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so that the value of the error evaluation function is minimized; generating a discrete sampling step sequence executable by mapping the calculated non-uniform time schedule to a preset operating environment of the diffusion probability model; and driving the diffusion probability model according to the generated sampling step sequence to generate the second data from the first data. Effects of the invention

[0024] According to one disclosed embodiment, by non-uniformly distributing the sampling steps of a diffusion probability model, the number of function evaluations (NFE) and the total computation time can be effectively reduced while maintaining the same generation quality. Conventional methods often distribute sampling steps uniformly or set schedules based on empirical rules, which has limitations such as the accumulation of errors in quality-sensitive sections or the excessive input of computations into unnecessary sections. However, the present invention sets an error index based on the upper bound of the global discretization error and calculates a non-uniform time schedule by optimizing the temporal positions of the sampling steps to minimize the error index. Consequently, steps can be densely distributed in sections with high error contribution and sparsely distributed in sections with low error contribution. Furthermore, the calculated schedule can be converted into a discrete sampling step sequence executable in the model's operating environment and applied, thereby achieving acceleration effects without retraining the model. Brief explanation of the drawing

[0025] The drawings attached below for use in describing embodiments of the present invention are merely some of the embodiments of the present invention, and other drawings can be obtained based on these drawings without effort to arrive at a novel invention by a person skilled in the art to which the present invention pertains (hereinafter referred to as "person skilled in the art"). FIG. 1 is a flowchart illustrating the procedure of an acceleration method for a diffusion probabilistic model according to one embodiment of the present invention. FIG. 2 is a diagram illustrating a procedure for calculating a non-uniform time schedule and generating an optimized discrete sampling step sequence according to an embodiment of the present invention. FIG. 3 is a conceptual diagram for visually explaining the effect of a non-uniform time schedule according to one embodiment of the present invention. FIG. 4 is a diagram illustrating a comparison of the effect of a non-uniform time schedule according to one embodiment of the present invention on actual image denoising performance. FIG. 5 is a diagram illustrating a comparison of the effect of a non-uniform time schedule according to one embodiment of the present invention on actual image denoising performance. FIG. 6 is a block diagram showing the configuration of a computing device for accelerating a diffusion probability model based on a global discretization error upper bound, according to one embodiment. Specific details for implementing the invention

[0026] The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, this is merely illustrative and the invention is not limited thereto.

[0027] In describing the embodiments of the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions may unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments of the present invention and should not be limiting in any way. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0028] Additionally, terms such as "first," "second," etc., may be used to describe various components, but the components should not be limited by these terms. The terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0030] FIG. 1 is a flowchart illustrating the procedure of an acceleration method for a diffusion probabilistic model according to an embodiment of the present invention. In an embodiment of the present invention, the diffusion probabilistic model may include a generation model that performs a sampling process to progressively update data over a plurality of steps to generate noise-reduced or restored data (e.g., denoising / deblurring / weather effect removal results, etc.) from noise-containing data (e.g., an image with added noise, a degraded image, etc.).

[0031] The sampling process is performed by approximating the inverse dynamics defined by continuous time or continuous parameters (e.g., noise level) in a finite number of sampling steps, wherein at each step, a function evaluation of a neural network-based predictor (e.g., score network or data prediction network) may be performed at least once. Accordingly, in this specification, the 'number of sampling steps' and the 'number of function evaluations (NFE)' may be equal to or proportional to each other depending on the model implementation.

[0032] Additionally, in one embodiment, a single sampling step may correspond to a unit of operation that updates an intermediate sample of the next state by calling a predictor (e.g., a score network or a data prediction network) once, and in another embodiment, the predictor may be called multiple times within a single step for calculating a correction term or a polynomial approximation order (e.g., a second-order / third-order approximation). In this case, the NFE may not be the same as the “number of steps,” but the present invention can provide the same technical effect in that it optimizes the step placement (temporal position) under a limited NFE budget.

[0033] Here, pre-training means that before the schedule calculation and application process of the present invention is performed, the corresponding diffusion probability model is trained in advance based on training data, so that the parameters of the neural network predictor (prediction model) to be used in the sampling step are already determined; and the present invention is configured in principle to accelerate sampling without changing or retraining these model parameters.

[0034] In one embodiment, "not changing model parameters" includes not changing weights, network architecture, or the form of conditional inputs used during inference, and the present invention is characterized by controlling an inference path by calculating and applying only a time schedule (arrangement of sampling steps) and its discrete sequence.

[0035] Furthermore, "not retraining" may include not performing additional parameter updates on the training data, knowledge distillation or student model training, or fine-tuning using loss functions / learning rates. However, during the schedule optimization process, inference of the predictor may be performed to obtain function evaluation values ​​necessary for calculating the upper error boundary equation, and such inference can be distinguished from the training / updating of model parameters.

[0036] Additionally, the first data may be input data containing noise, and the second data may be target output data with reduced noise, restored data, or generated data.

[0037] First, in step S110 of FIG. 1, the computing device sets an error evaluation function based on the upper bound of the Global Discretization Error.

[0038] Here, 'global discretization error' refers not to a local approximation error at a single step, but to the accumulated error that is reflected in the generated result (second data) at the end point, resulting from the accumulation of errors occurring during the discrete approximation process across the entire sampling interval.

[0039] In the present invention, approximating the sampling process with a finite number of sampling steps means that, even though the original sampling process may have a continuously changing flow (time or continuous parameter change), the number of steps is reduced or placed at specific locations to perform the approximation so that only a limited number of steps are performed during actual execution.

[0040] In this case, if fewer steps are performed or step positions are selected inappropriately, quality degradation may occur as the difference between the continuous ideal execution result and the actual discrete execution result accumulates; this accumulated quality degradation can be understood as global discrete error.

[0041] However, since ideal continuous results or correct data are not always available in actual applications, instead of directly calculating the error, the present invention utilizes an error bounding expression that guarantees or explains that the global discretization error is below a specific value based on the difference between the continuous-time (or continuous parameter) based modeling result and the discretized approximation result, and sets the upper bound value derived therefrom or the function value based on the upper bound as the error evaluation function. Here, the 'error evaluation function' may refer to an example of an error index defined using the upper bound of the global discretization error or the function value based on the said upper bound.

[0042] The upper error equation is (i) the pre-trained predictor output value (e.g., at each step It can be calculated based on (ii) interval boundaries determined by the selected schedule, and (iii) preset constants / coefficients (e.g., upper limit coefficients determined by modeling assumptions), and does not require separate ground truth data.

[0043] In one embodiment, the error upper bound formula may be defined to take as inputs the predictor output at least at each sampling step, the step boundary, and upper bound coefficients (e.g., prediction error upper bound coefficient, scale coefficient) based on modeling assumptions, and output an upper bound value of the global discretization error or an upper bound-based function value. Accordingly, the computing device can compare the superiority of candidate schedules by recalculating the upper bound value from the inputs whenever the schedule candidate changes.

[0044] The error evaluation function can be used as an indicator to evaluate the upper limit of the error that varies according to the step arrangement, rather than the actual error itself, and thereby the computing device can quantitatively compare and determine how the steps should be arranged to reduce the possibility of accumulated error in the entire sampling process.

[0045] In addition, depending on the embodiment, the error evaluation function may use the upper limit value as is, or use it in the form of a function transformed to suit computational stability or optimization convenience. In this case, specific mathematical examples of the upper limit value derived from the continuous-time (or continuous-parameter) based modeling result and the error upper bound equation may be set as follows.

[0046] First, to formulate the backsampling of the diffusion probability model as an ordinary differential equation in continuous time, the computing device can set the data-prediction-based ODE as follows.

[0047]

[0048] [Mathematical Equation 1] expresses the sampling process as a differential equation in continuous time, and can then theoretically analyze the global discretization error that occurs when approximating it with a finite step.

[0049] [Mathematical Formula 1] is time Intermediate sample at The rate of change of (i) A term proportional to itself and (ii) a neural network-based predictor It indicates that it is determined by a combination of terms proportional to the output of, , This indicates that the time change of is reflected in the sample update. Therefore, by providing a basis for comparing a continuous solution (ideal execution result) and a discrete step approximation solution (actual execution result), it is possible to implement the step (S110) of deriving an upper limit of the global discretization error and setting the upper limit as an error evaluation function.

[0050] Here is time Intermediate sample at, is continuous time (or continuous parameter), is the signal component scaling factor, is the noise component scaling factor, is the output of the neural network-based predictor, and , represents the time rate of change of the corresponding scale.

[0051] Next, the computing device, in order to reparameterize the sampling time axis into the log-signal-to-noise ratio (log-SNR) axis, time Parameters corresponding to It can be defined as follows.

[0052]

[0053] [Mathematical Equation 2] represents the sampling process as simple time Instead, reflecting signal-noise relative dominance The purpose is to express it along the axis so that the subsequent integral expression and the derivation of the upper error boundary formula are stable, and the sensitivity (error contribution) for each interval is clearly revealed. [Equation 2] is go and It represents the degree of signal / noise dominance at the corresponding stage as the logarithm of the ratio, and Is Time again at It indicates that it is a transformation function mapping to. [Equation 2] subsequently sets the upper bound of the global discretization error. - By organizing within the domain, it facilitates the separation and evaluation of the parts of the terms constituting the upper limit that vary due to step arrangement.

[0054] Here , is the log-SNR corresponding to the start / end boundary, The number of steps, silver It refers to the temporal position of the nth step.

[0055] In particular, on the log-SNR axis, the change in sensitivity in the later part of the sampling interval (e.g., the interval corresponding to t→0) is relatively well separated, so the difference in error contribution due to step placement can be reliably evaluated.

[0056] Next, the computing device can express the solution of the continuous model in integral form (continuous solution) and set up a basis equation as follows to offset the global discretization error that occurs when approximating with finite steps.

[0057]

[0058] [Equation 3] is intended to decompose the continuous sampling result into boundary condition terms + integral terms so that the discrete approximation error eventually results in the approximation error of the integral term.

[0059] [Mathematical Formula 3] is the result at the end point The state at the start point and Indicates that it is determined by the cumulative effect of interval integration, and is the integrand of the integral It indicates that is defined as the output of the neural network predictor. Therefore, the difference between the continuous solution (integral) and the discrete approximation (weighted sum) is defined as the global discretization error, and by constructing an error upper bound equation that uppers this, the upper bound can be set as the error evaluation function.

[0060] Next, the computing device, in order to numerically approximate the integral, interval by interval We can approximate it with a local polynomial and construct the entire approximate solution as a weighted sum of the function evaluation values.

[0061]

[0062]

[0063]

[0064] [Mathematical Equation 4] is in each subinterval It is possible to implement integral calculations by approximating with only a limited number of function evaluation values, and basis functions As a linear combination of local approximation of It is possible to constitute.

[0065] [Equation 5] is an approximation of the continuous integral solution of [Equation 3] that can be actually implemented. It involves converting to, and interval weights The final approximation result is formed by the accumulation of the function evaluation values ​​multiplied by .

[0066] [Equation 6] has weights on the interval boundaries and to specify that it is determined by integration based on the form of the basis function, and if the temporal position of the sampling step (i.e., the interval boundary) changes It indicates that since they change together, the upper limit of the error can consequently also vary depending on the step arrangement.

[0067] Therefore, even with the same pre-trained model, the set of weights changes if the interval boundary (i.e., the temporal position of the sampling step) changes, which implies that the upper bound of the error for the final result may vary depending on the schedule. The technical feature of the present invention is that it calculates a schedule that minimizes this schedule-dependent upper bound.

[0068] Finally, the computing device can derive an upper bound indicating that the global discretization error is less than or equal to a specific value from the error upper bound formula without directly calculating the global discretization error, and set that upper bound or a function value based on the upper bound as the error evaluation function.

[0069]

[0070] [Equation 7] is the global discretization error caused by the finite step approximation. By offsetting, the actual answer The purpose is to establish an error evaluation function that can manage or compare errors even in situations where direct use is not possible.

[0071] [Equation 7] indicates that the upper limit of the global discretization error can be expressed as the sum of a fixed component, which is independent of the step batch and determined by the sampling start / end boundary conditions, and a variable error component, whose value changes as the weighted combination structure changes according to the temporal position of the step.

[0072] Therefore, instead of the actual error, the upper limit of the error (or the upper limit-based function value) is used as the error evaluation function to quantitatively determine the superiority of the sampling step arrangement, and in the subsequent step (S120), the temporal position of the step is optimized to provide a criterion for calculating a non-uniform time schedule.

[0073] Here is the discrete step-based approximation result, is the target data (ideal restoration result), is the noise scale at the end point, is a constant related to the upper bound of the prediction error, is time time-dependent error term at, is the interval weight calculated by [Equation 6], and is a specific index It represents the combination of weights contributing to.

[0074] Accordingly, in step S110, the computing device may use the global discretization error upper limit value in [Equation 7] as is as an error evaluation function, or, according to the embodiment, set a function value based on the variable error component among the upper limits as an error evaluation function, thereby enabling optimization centered on the error possibility that varies by step arrangement.

[0075] Next, in step S120 of FIG. 1, the computing device calculates a non-uniform time schedule by optimizing the temporal positions of the sampling steps so that the value of the error evaluation function set in step S110 is minimized.

[0076] Here, the temporal positions of the sampling steps are such that each sampling step is within the entire sampling interval (e.g., start time). End point at It may refer to a position placed on or a coordinate on a continuous parameter axis (e.g., log-SNR axis) up to), and a non-uniform time schedule refers to a schedule configured such that the intervals between adjacent steps are not equal and have different intervals for each segment.

[0077] An important aspect of the present invention is that, instead of uniformly distributing steps across the entire interval, steps are distributed relatively densely in intervals with a high contribution to the error evaluation function (global discretization error upper limit or function value based thereon) (i.e., intervals where widening the step interval significantly increases the error upper limit), and steps are distributed relatively sparsely in intervals with a low contribution (i.e., intervals where reducing the steps relatively limits the increase in the error upper limit).

[0078] Such non-uniform placement allows for lower error evaluation function values ​​under the same total number of steps or the same number of function evaluations (NFE), or conversely, reduces the required NFE under conditions that satisfy the same quality criteria.

[0079] In addition, according to the embodiment, in the optimization process of step S120, boundary conditions may be applied to fix the start and end steps of the sampling interval so that both ends of the schedule are maintained consistently, or monotonic conditions may be applied so that the order of adjacent steps is not reversed so that the schedule has an executable sequence structure.

[0080] Consequently, the non-uniform time schedule produced in step S120 may be a continuous (or continuous parameter) schedule in which the temporal position of the step is determined in the direction in which the error evaluation function is minimized.

[0081] In addition, the upper limit of the global discretization error (error evaluation function) set in step S110 can be interpreted by separating it into a fixed component and a variable error component according to the embodiment, wherein the fixed component is a term determined independently of the step arrangement by the start / end boundary conditions of the sampling interval, etc., and the variable error component is a term whose value changes depending on how the temporal positions of the steps are selected and is subject to optimization. For example, the upper limit of the global discretization error can be set in the form of [Equation 7] to distinguish between the fixed component and the variable error component.

[0082] Here, [Equation 7] is intended to provide an example of an error upper bound formula to ensure / explain that the cumulative error (global discretization error) between a continuously defined ideal result and a result approximated by a finite step is below a specific upper bound, and to indicate that the upper bound of the global discretization error can be expressed as the sum of (i) a fixed component determined by boundary conditions, etc. and (ii) a variable error component that depends on the step arrangement, thereby enabling the design of a schedule in a direction that reduces the value of only the term (variable error component) that can be controlled by optimization.

[0083] In [Equation 7] is the total number of sampling steps (or a value directly associated with NFE), and Is It is the temporal position of the nth step, and is the target (correct answer) data, can mean the result at the end point obtained by the finite step approximation, and is the point in time It is a time-dependent error term (a coefficient reflecting the possibility of cumulative prediction error at that point in time), and is a weight calculated by interval integration (e.g., a weight defined in another formula) as the step boundary and the approximation order , index It can be determined according to, and It can be understood as a scale / constant term included in the upper bound expression.

[0084] Accordingly, in step S120, the computing device determines the temporal position of the step to minimize the variable error component that is directly controllable by the step arrangement among the entire upper limit (error evaluation function), and to this end, it can determine a non-uniform time schedule by setting up an optimization problem such as the following [Equation 8].

[0085]

[0086] Here, [Equation 8] is intended to find a step sequence that minimizes the variable error component (or the corresponding objective function) excluding the fixed component, and consequently lower the error evaluation function (the upper bound of the global discretized error or its function value), and at each time point time-dependent error term in and the combination of weights contributing at that time So that the sum of the products over the entire interval becomes smaller It means choosing.

[0087] Therefore, the purpose is to enable achieving a lower upper bound under the same NFE conditions or reducing the NFE under the same quality conditions by placing steps more densely in error-sensitive sections (sections with large error terms or weighted combinations) and placing fewer steps in less sensitive sections. In addition, in [Equation 8] is time It is a continuous parameter corresponding to (e.g., coordinates on the log-SNR axis), and , is a value that fixes the start / end boundary, and This corresponds to an example of a monotonicity constraint that causes the parameter to change monotonically as the back diffusion proceeds, and and represents the start / end step position.

[0088] Consequently, step S120 corresponds to a step of specifically implementing the step of calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so as to minimize the error evaluation function, and the calculated non-uniform time schedule can be converted into a discrete sampling step sequence in which the actual diffusion probability model can be executed and applied in step S130.

[0089] Next, in step S130 of FIG. 1, the computing device generates a non-uniform time schedule calculated in step S120 by mapping it into a discrete sampling step sequence that is actually acceptable to a pre-trained diffusion probability model.

[0090] The pre-configured operating environment here is such that the time step representation received as input by the diffusion probability model when performing sampling is not a continuous value but an integer index within a specific range (e.g., 0 to This means that it may include execution constraints limited to ).

[0091] Therefore, the continuous value-based schedule obtained in step S120 (e.g., continuous time values ​​for each step) ) cannot be applied as is, and the model must be converted into an executable sequence of integer indices.

[0092] In one embodiment, the mapping / generation can be performed by the following discretization transformation equations.

[0093]

[0094]

[0095]

[0096]

[0097] [Equation 9] is the value of each consecutive step of a non-uniform time schedule The maximum number of steps in the model It is intended for scaling along the integer time step index axis in accordance with, continuous step It is to linearly map to an integer index range while maintaining the relative position it possesses.

[0098] Therefore, the goal is to transfer the continuous schedule to the model's internal timestep index system to enable subsequent integer operations (rounding / clamping).

[0099] [Mathematical Equation 10] is a scaled real value As for converting into an integer candidate value, Round to the nearest integer to obtain discrete time step candidates It is to obtain. Therefore, it becomes a key step of integerization to satisfy the integer time step input format required by the diffusion probability model.

[0100] [Mathematical Formula 11] is the rounded value The valid range allowed by this model (e.g., 0 to This is intended to prevent cases where it goes beyond ), minimum 0, maximum Final integer time step by limiting (clamping) It is to confirm.

[0101] Therefore, the purpose is to ensure suitability for the operating environment by preventing sampling errors or abnormal behavior caused by time step inputs outside the valid range.

[0102] [Mathematical Formula 12] is each Since duplicate time steps may occur during the process of converting to integers, where different consecutive step values ​​are mapped to the same integer time step, temporary sequence The final discrete sequence by removing duplicates from As for generating, The step sequence is determined by removing identical values ​​from the accumulated integer candidates to form a sequence of unique elements.

[0103] Therefore, the purpose is to prevent inefficiency caused by the repeated execution of the same time step and, consequently, to reliably secure an executable discrete sampling step sequence.

[0104] Here, represents a continuous value-based step position included in the non-uniform time schedule calculated in step S120, and represents the maximum integer timestep index used internally by the diffusion probability model, and is a scaled real value, is a rounded integer candidate value, represents a fixed integer time step clamped to a valid range, and is a temporary sequence in which integer timestep candidates generated for each step are stored sequentially, and represents the final discrete sampling step sequence determined after duplicate removal.

[0105] In addition, if there is a possibility that the order of adjacent steps may be reversed or monotonicity may be degraded during the deduplication or transformation process, the computing device can generate a discrete sequence in a form that is sequentially executable by the diffusion probability model over time (or the progression of reverse diffusion) by correcting to satisfy sequence alignment or monotonic conditions.

[0106] Finally, in step S140 of FIG. 1, the computing device generates second data from first data by driving a diffusion probability model according to the generated sampling step sequence. At this time, the sampling may include a reverse diffusion process to generate second data with reduced noise from first data with added noise, depending on the embodiment, and in each sampling step, the model performs gradual restoration by receiving data in the current state (intermediate generated data) as input and calculating data in the next state. Based on the error evaluation function and non-uniform time schedule designed in steps S110 to S130, the present invention can be configured to reduce the number of sampling steps (i.e., NFE) required to satisfy the same quality criteria, or to obtain second data of superior quality under the same NFE conditions. In particular, the present invention has the advantage of being applicable retrospectively to already distributed pre-trained models, as it achieves an acceleration effect by optimizing and applying only the schedule (the temporal position of the step and its discrete sequence) without retraining the model itself or changing its structure.

[0108] FIG. 2 is a diagram illustrating a procedure for calculating a non-uniform time schedule according to an embodiment of the present invention, converting it into a discrete sampling step sequence, and finally generating an optimized discrete time step schedule.

[0109] The algorithm of Fig. 2 corresponds to an example that specifically implements the step of calculating a non-uniform time schedule and the step of generating the non-uniform time schedule by mapping it to a discrete sampling step sequence.

[0110] First, the algorithm of Fig. 2 takes the number of objective function evaluations (NFE) as input. , maximum integer time step , start time and end time Requires and an optimized discrete timestep schedule as output It guarantees. Here represents the number of sampling steps (or an equivalent constraint on the number of function evaluations), and is the maximum value of the timestep index accepted by the pre-trained diffusion probability model at runtime (e.g., 0 to It means ), and It defines the start / end boundaries of the interval where sampling takes place.

[0111] Next, as part of performing continuous time step optimization (StepOptim), first, to set boundary conditions - Fix the start and end points in the domain (e.g., log-SNR-based parameter axis), and this , Place it as follows.

[0112] At this time and represents the parameter values ​​of the start step and end step, respectively, and and is a value that fixes the boundary and serves to prevent schedule optimization from going outside the boundary.

[0113] Next, the computing device uses weights for numerical approximation in each interval In calculating [Equation 6], in one embodiment, the weight can be defined as the following interval integral form [Equation 6].

[0114] [Mathematical Equation 6] represents each sub-interval when approximating an entire continuous interval by dividing it into sub-intervals Since this is to calculate weights that numerically reflect the contribution of the corresponding section, weighted basis functions The value obtained by integrating over an interval is the evaluation value of a specific function (index This indicates the extent to which ) contributes to the approximate solution.

[0115] Therefore, [Equation 6] has the same number of steps Even if maintaining the step boundary , The purpose is to ensure that step placement optimization is actually reflected in the upper error bound (error evaluation function) by making the weights vary depending on how it is arranged. Here is the index of the sub-interval, is the approximation order used in the interval (or the number of function evaluation values ​​used for approximation), is an index designating a specific function evaluation value among them, and is a basis function (e.g., Lagrange basis polynomial, etc.) in the corresponding interval It is a function for.

[0116] Subsequently, the computing device solves the optimization problem by [Equation 8] above to determine the temporal positions of the steps so that the error evaluation function (global discretization error upper limit or function value based thereon) is minimized.

[0117] [Equation 8] specifies the step position such that the part of the terms constituting the global discretization error upper limit that varies particularly by the step placement (variable error component) is minimized. Since it is for the purpose of selecting, at each step point time-dependent error term in and the weighted combination for that point in time Since the intervals that grow together dominate the overall objective function value, optimization is induced so that steps are placed more densely in such intervals.

[0118] Therefore, [Equation 8] is the same In order to lower the error evaluation function value compared to a uniform schedule, or to maintain the same quality, the necessary It is to enable the reduction of (or NFE). Also, constraint formula and corresponds to a boundary condition that fixes the start / end boundary, and This corresponds to an example of a monotonic condition that prevents the step order from being reversed as the reverse diffusion proceeds, thereby ensuring that the resulting schedule has an actual executable order.

[0119] In this way, through StepOptim, the computing device obtains an optimized step position in the continuous parameter domain After obtaining it, use it as a schedule in the form of an actual continuous-time sequence. Convert to (line 5 of Fig. 2). Here is a non-uniform time schedule expressed as continuous values, and since this schedule itself is not yet an integer time step sequence that the model can execute immediately, the next step of discretization transformation is performed.

[0120] Next, as a discretization step, the computing device first [takes] an empty integer sequence After initializing, Each consecutive time step included in For this, it is mapped to an integer index. In one embodiment, this mapping can be implemented as “scaling → rounding → clamping → deduplication,” and specifically, scaling is intended to match the continuous schedule value to the maximum integer range of the model and can be expressed as [Equation 9].

[0121] [Mathematical Equation 9] is a continuous value cast It involves performing a scale transformation to map to an integer index domain, which means linearly mapping continuous time (or continuous parameter position) to the model's internal timestep resolution, and the effect is to shift the non-uniform schedule obtained from optimization into a step range that the model can understand, thereby enabling subsequent integer conversion. Subsequently, rounding is performed to obtain an integer timestep and can be set as [Equation 10].

[0122] [Equation 10] approximates real values ​​to integers to match the integer index format required by the model, maps scaled continuous values ​​to the nearest integer time step, and generates candidate steps that can be directly used in the iterative sampling loop during execution. Additionally, to prevent the rounding result from exceeding the valid range allowed by the model, clamping can be performed as in [Equation 11].

[0123] [Mathematical Equation 11] is the integerization result It is to prevent execution errors or abnormal behavior by restricting it so that it does not go out of range, and the lower limit is 0 and the upper limit It involves forcibly projecting values ​​through the inter-channel. Therefore, its purpose is to ensure that only valid time steps suitable for the model's operating environment are included in the final sequence, regardless of the input or optimization result. Subsequently, the computing device each obtained about cast Add to sequentially.

[0124] Finally, different continuous values Since it can be mapped to the same integer value It may contain duplicate time steps, which are removed to obtain the final discrete sequence ...is determined. In one embodiment, duplicate removal can be expressed as [Equation 12].

[0125] [Equation 12] is intended to create a valid discrete sampling step sequence by removing duplicate time steps, which combines identical values ​​among the integer indices included in the temporary sequence into one to form the final schedule. Therefore, it aims to prevent the repeated execution of the same step, thereby reducing the problem of unnecessary increase in actual NFE, and to ensure an executable sequence while maintaining consistency in the step order. Additionally, if there is a possibility that the order of adjacent steps may be reversed or monotonicity may be compromised during the deduplication or integerization process, the computing device may correct to satisfy the alignment or monotonic conditions, so that the diffusion probability model can generate a discrete sampling step sequence that is sequentially executable according to the progression of time (or the progression of reverse diffusion).

[0126] As a result, Algorithm 1 of FIG. 2 provides a specific implementation example that (i) performs step batch optimization (producing a continuous schedule) to minimize an error evaluation function based on a global discretization error upper bound, and (ii) maps this to an integer index-based discrete sampling step sequence to match the pre-set operating environment of a pre-trained diffusion probability model, thereby enabling a reduction in the number of required function evaluations (NFE) or achieving a lower error evaluation function value at the same NFE while maintaining the same quality.

[0127] FIG. 3 is a conceptual diagram for visually explaining the effect of a non-uniform time schedule according to an embodiment of the present invention. More specifically, FIG. 3 exemplarily shows how the quality of a restored image differs under the same number of function evaluations (NFE) conditions by comparing the case where a uniform schedule is used with the case where an optimized non-uniform time schedule calculated according to the method of the present invention (e.g., StepOptim scheduler) is used while the same diffusion probability model is fixed.

[0128] In addition, FIG. 3 is a conceptual diagram intended to intuitively explain that output quality can differ under the same computational budget (NFE) when (i) only the “temporal position (schedule) of the sampling step” is changed while (ii) the parameters of the pre-trained diffusion denoising model are fixed. In other words, FIG. 3 is intended to visually convey the principles and effects of schedule optimization, and quantitative performance comparisons are explained in FIGs. 4 and 5.

[0129] The upper area of ​​Fig. 3 illustrates an example using a conventional uniform schedule. In the upper left, a first image (input data) severely degraded by snow, rain, noise, etc. is placed, and an arrow illustrates the flow in which this image passes through a scheduler labeled "Uniform Scheduler" and is input into a pre-trained Diffusion Denoising Model. In the upper center, along with phrases such as "Total Timesteps = 1000" and "NFE = 20," a bar-shaped diagram is shown in which 20 sampling steps out of a total of 1000 steps are arranged at nearly equal intervals; this represents a uniform time schedule that selects sampling steps by dividing the entire time axis into uniform intervals, as in conventional technology. In the upper right, a second image restored using the uniform schedule is displayed, and an example is shown where noise has been removed but some detailed contours or textures remain blurry, or the edges are somewhat blurred.

[0130] Here, “Total Timesteps = 1000” refers to the total length of the reference time axis (or reference discrete timestep index) defined by the diffusion model, and “NFE = 20” may refer to the total number of times a predictor (e.g., score network or data prediction network) is called during the sampling process (or a number of calls proportional to this). Therefore, the comparison between the left and right sides in Figure 3 can be understood as a comparison under substantially equivalent computational cost conditions, in that only the schedule arrangement differs under the same NFE (same predictor call budget).

[0131] The lower area of ​​FIG. 3 illustrates the case where an optimized non-uniform time schedule calculated according to the method of the present invention is applied. A degraded first image identical to the one at the top is placed on the lower left, and a configuration is illustrated in which this image passes through a scheduler labeled "StepOptim Scheduler" and is input into the same pre-trained diffusion denoising model. At this time, the bar diagram displayed in the center of the lower section has the same total timestep and function evaluation count conditions as the top, with Total Timesteps = 1000 and NFE = 20, but is characterized by the vertical lines (sampling step positions) inside the bar being arranged densely or sparsely depending on the section. This visually indicates that, according to the error evaluation function of the present invention, a non-uniform time schedule is applied in which steps are arranged more densely in sections with a large contribution to the global discretization error and steps are reduced in sections with a small contribution.

[0132] In diffusion-based denoising, the later section where sampling proceeds and the noise level decreases (e.g., in terms of time variables) Small time step changes in the corresponding interval (or specific interval in terms of log-SNR) can have a significant impact on output quality. The non-uniform time schedule of the present invention can be designed to maintain a lower global discretization error upper limit even under the same NFE conditions by placing sampling steps relatively densely in these later sensitive intervals.

[0133] In the bottom right, a second image restored using an optimized non-uniform time schedule is displayed. When compared to the uniform schedule result shown above, it intuitively demonstrates that noise reduction performance and sharpness are improved in the background, object contours, and fine texture areas, even under the same NFE conditions. For example, it illustrates that boundaries are sharper in human silhouettes, building corners, and detailed road structures, while residual noise and blurring are reduced.

[0134] Furthermore, the restoration results may show a reduction in (i) blurring or ringing near edges, (ii) loss of fine textures, and (iii) patterned artifacts from residual noise components, while object contours and textures remain relatively sharp. This can be understood as a result of concentrating the computational budget on critical sections of step placement to more efficiently track the restoration path under the same NFE conditions.

[0135] In summary, Figure 3 is an example diagram showing that when only the sampling step sequence is optimized based on the global discretization error upper limit without retraining or changing the structure of a pre-trained diffusion probability model, superior image restoration quality can be achieved compared to a conventional uniform schedule under the same number of function evaluations, intuitively illustrating the substantial performance improvement effect of the non-uniform time schedule according to the present invention.

[0137] Figure 4 illustrates the effect of the non-uniform time schedule of the present invention on actual image denoising performance by comparing it with example images from a public dataset (e.g., ImageNet). Each column of the figure represents the results according to different sampling schedules and the Number of Function Evaluations (NFE), and at the bottom of each result image, the region of interest (indicated by a red dotted box) is magnified to more clearly show the differences in detailed quality.

[0138] First, the first column (410) on the left side of FIG. 4 represents a noise image (Noisy) at the input stage. This image is a degraded image with strong Gaussian noise added, and corresponds to the first data used as input for a pre-trained diffusion probability model. In the enlarged bottom area, the texture and boundaries are completely collapsed, showing that the outline or character information of the original object is almost indistinguishable.

[0139] The second column (420) shows the restoration result (Original / NFE=20) when using a conventional uniform schedule and setting the number of function evaluations NFE to 20. In this case, denoising is performed through a sequence of 20 sampling steps arranged at regular intervals throughout the entire sampling interval, and in the full image at the top, most of the noise is removed and the surface texture of the envelope and the printed logo are restored relatively naturally. In the magnified area at the bottom, the boundary lines and character outlines are rendered somewhat smoothly, but it can be seen that there is still some blur and loss of detail.

[0140] The third column (430) shows the restoration result (Original / NFE=12) when the same uniform schedule is maintained but the NFE is reduced to 12. Although the amount of computation decreases as the number of sampling steps is reduced, as can be seen from the full image at the top and the magnified area at the bottom, quality degradation occurs, such as increased blurring and blurred outlines near the corners, borders, and printed characters of the envelope. This visually demonstrates that if the number of steps is simply reduced in a uniform schedule, the global discretization error increases, and the restoration quality may deteriorate.

[0141] The fourth column (440) illustrates the restoration result (Optimized / NFE=12) when applying the non-uniform time schedule (Optimized Schedule) according to the present invention while maintaining the NFE at 12, as with the uniform schedule. In this case, the pre-trained diffusion probability model is the same, but the sampling step sequence is optimized to minimize the error evaluation function based on the global discretization error upper bound as in claims 1 to 9. As can be seen in the figure, even under the same NFE=12 condition, the result of applying the non-uniform schedule shows that the contours near the edges are sharper, the gradients and shadow boundaries of the envelope surface are expressed more naturally, and the character strokes and boundaries appear less blurry in the enlarged area compared to the uniform schedule (NFE=12) in the third column. This means that the non-uniform time schedule of the present invention effectively reduces the global discretization error under the same amount of computation, thereby improving the restoration quality.

[0142] The last column, the fifth column (450), shows the Ground Truth (GT), that is, a noise-free reference ground truth image. This image corresponds to the target output data that the diffusion probability model ideally intends to restore, and the relative quality level can be assessed by visually comparing it with the result images of each column. When the optimization schedule result of the fourth column is compared with the uniform schedule (NFE=20) result of the second column and the GT, it can be seen that a similar level of quality is achieved in terms of boundary sharpness and structure preservation despite the low NFE.

[0143] Accordingly, FIG. 4 is an example diagram that experimentally demonstrates that the global discretization error upper limit-based non-uniform time scheduling technique according to the present invention can provide uniform or improved image restoration quality with fewer function evaluations compared to a uniform schedule by adjusting only the schedule without retraining the model.

[0145] Figure 5 illustrates the effect provided by the non-uniform time schedule of the present invention in image restoration tasks, such as removing weather effects (e.g., snow removal), through a comparison using example images included in the public dataset Snow100K. Each column of the figure represents the restoration results according to different sampling schedules and the Number of Function Evaluations (NFE), and the region of interest indicated by the red dotted box at the bottom of each result image is magnified to more clearly show the differences in detailed structure and texture.

[0146] First, the first column on the left (510) illustrates a noisy image with a strong snow effect superimposed. In the full screen at the top, roads, buildings, pedestrians, and vehicles are heavily obscured by snow particles, and in the magnified area at the bottom, severe texture distortion and structural collapse appear to the extent that people, motorcycles, and road surfaces are almost indistinguishable. This image corresponds to the first data input to a pre-trained diffusion probability model.

[0147] The second column (520) shows the result (Original / NFE=10) when using a conventional uniform schedule and setting the number of function evaluations NFE=10. Snow removal is performed using a sequence of 10 sampling steps arranged at regular intervals, and in the upper full image, the building exterior, sky, and road shape are restored relatively naturally, and most of the snow particles can be seen removed. In the lower magnified area, the outline of a person and the shape of a motorcycle are also somewhat identifiable, and severe noise caused by snow particles is greatly reduced.

[0148] The third column (530) shows the result when the same uniform schedule is maintained but the NFE is reduced to 4 (Original / NFE=4). Although the amount of computation is significantly reduced, as can be seen in the full image at the top and the magnified area at the bottom, there are still residual snow particles and smudges around the road, people, and vehicles, and the quality degradation is clearly evident, such as blurring of object boundaries and distortion of detailed structures. This demonstrates that if the number of steps is simply reduced in a uniform schedule, the global discretization error increases, which can significantly reduce the performance of removing weather effects.

[0149] The fourth column (540) illustrates the result (Optimized / NFE=4) when applying the non-uniform time schedule (Optimized Schedule) according to the present invention while maintaining NFE=4, identical to the uniform schedule. Here, the diffusion probability model itself is fixed identically, and only the non-uniform time schedule is applied in which the temporal positions of the sampling steps are optimized to minimize an error evaluation function defined based on the upper bound of the Global Discretization Error. As a result, it can be seen that, even under the same NFE=4 condition, the textures of the road surface and wall surface are restored more smoothly compared to the third column, the shapes of people and motorcycles are clearly revealed, and residual snow particles are also significantly reduced. In other words, the non-uniform schedule of the present invention provides superior weather effect removal performance compared to the uniform schedule by concentrating steps in sections where error accumulation is large, even with a small number of steps.

[0150] Column 5 (550) shows the ground truth image (GT) with snow removed. This image corresponds to ideal second data where no weather effects exist, and its relative quality can be visually evaluated by comparing it with the restoration results of columns 2 through 4. In particular, the optimization schedule result of column 4 shows that it achieves a visual quality similar to the uniform schedule result (NFE=10) of column 2 and the ground truth image, despite having a much lower NFE (4).

[0151] Accordingly, FIG. 5 is a figure demonstrating, through experimental examples, that the global discretization error upper limit-based non-uniform time scheduling technique according to the present invention can produce results of equivalent or improved quality with fewer function evaluations compared to a uniform schedule by adjusting only the sampling step sequence without retraining the model in actual restoration scenarios such as the removal of weather effects.

[0153] FIG. 6 is a block diagram illustrating the configuration of a computing device for accelerating a diffusion probability model based on a global discretization error upper bound, according to one embodiment. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

[0154] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) can perform a diffusion probability model acceleration method based on the global discretization error upper bound described in FIGS. 1 to 5.

[0155] The computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) can enable the computing device (12) to operate according to the exemplary embodiments described above. For example, the processor (14) can execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to enable the computing device (12) to perform operations according to the exemplary embodiments when executed by the processor (14).

[0156] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by a processor (14). In one embodiment, the computer-readable storage medium (16) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by a computing device (12) and capable of storing desired information, or a suitable combination thereof.

[0157] The communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and the computer-readable storage medium (16).

[0158] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and network communication interfaces (26) are connected to a communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) through the input / output interfaces (22). An exemplary input / output device (24) may include an input device such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or an output device such as a display device, a printer, a speaker and / or a network card. An exemplary input / output device (24) may be included inside the computing device (12) as a component constituting the computing device (12), or it may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0160] Through FIGS. 1 to 6, a system for accelerating a diffusion probability model based on a global discretization error upper bound can also be implemented in the form of a computer program stored on a computer-readable recording medium executed by a computer, or in the form of a recording medium containing instructions executable by a computer. Additionally, the method for accelerating a diffusion probability model based on a global discretization error upper bound described through FIGS. 1 to 6 can also be implemented in the form of a computer program stored on a computer-readable recording medium executed by a computer.

[0161] A computer-readable recording medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and inremovable media. Additionally, a computer-readable recording medium may include a computer storage medium. A computer storage medium includes both volatile and non-volatile, removable and inremovable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0162] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0163] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0164] Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.

Claims

Claim 1 A method for accelerating a Diffusion Probabilistic Model performed on a computing device equipped with one or more processors and memory, comprising: a step of setting an error evaluation function based on an upper bound of a Global Discretization Error that occurs when a pre-trained Diffusion Probabilistic Model approximates a sampling process for generating second data from first data with a finite number of sampling steps; a step of calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so as to minimize the value of the error evaluation function; a step of generating a discrete sampling step sequence executable by mapping the calculated non-uniform time schedule to a preset operating environment of the Diffusion Probabilistic Model; and a step of driving the Diffusion Probabilistic Model according to the generated sampling step sequence to generate the second data from the first data.The method includes the step of setting the error evaluation function, wherein the time variable of the sampling process is reparameterized into a log-signal-to-noise ratio (log-SNR), and the reparameterized sampling process is formulated as a Probability Flow Ordinary Differential Equation (Probability Flow ODE); the step of calculating the non-uniform time schedule is to calculate the non-uniform time schedule by optimizing the temporal positions of the sampling steps in a continuous parameter domain based on the log-SNR; the spreading probability model accepts integer time steps from 0 to a preset maximum integer time step index as input; and the step of generating by mapping is to convert the step positions of the non-uniform time schedule calculated in the continuous parameter domain based on the log-SNR into continuous time values, scale and round the continuous time values ​​based on the maximum integer time step index, and such that the rounded value exists within a valid range greater than or equal to 0 and less than or equal to the maximum integer time step index. A method for clamping, removing duplicate integer time steps when different consecutive time values ​​are converted to the same integer time step, and correcting the discrete sampling step sequence so that the order between adjacent integer time steps is not reversed. Claim 2 A method according to claim 1, wherein the sampling process performs a reverse diffusion process to generate the second data with reduced noise from the first data with added noise. Claim 3 delete Claim 4 A method according to claim 1, wherein the error evaluation function is set as an upper bound value derived from an error bounding expression indicating that the global discretization error occurring when the sampling process is approximated by discretizing it into a finite number of sampling steps, based on the difference between the continuous-time-based modeling result and the approximation result according to the discretization, or as a function value based on said upper bound. Claim 5 A method according to claim 4, wherein the upper limit of the global discretization error is composed of the sum of a fixed component, which is determined by the start and end times of the sampling interval and is independent of the temporal positions of the sampling steps, and a variable error component, the value of which varies according to the temporal positions of the sampling steps. Claim 6 A method according to claim 5, wherein the step of calculating the non-uniform time schedule is characterized by setting only the variable error component, excluding the fixed component, as the objective function, and optimizing the temporal positions of the sampling steps so that the value of the objective function is minimized. Claim 7 A method according to claim 6, wherein the non-uniform time schedule is calculated such that the value of the error evaluation function is minimized by densely arranging sampling steps in sections with high contribution to the variable error component and sparsely arranging sampling steps in sections with low contribution. Claim 8 A method according to claim 6, wherein the step of optimizing the temporal positions of the sampling steps is performed by applying a boundary constraint that fixes the start and end steps of the sampling interval and a monotonic constraint that prevents the order between adjacent sampling steps from being reversed. Claim 9 delete Claim 10 A method according to claim 1, wherein the diffusion probability model is applied to at least one image restoration task among image denoising, deblurring, or weather removal, and the non-uniform time schedule is calculated to satisfy a preset quality standard with fewer Number of Function Evaluations (NFE) compared to a uniform schedule. Claim 11 A computing device comprising: a processor; and a memory for storing one or more programs executed by the processor, wherein the processor performs the following operations: setting an error evaluation function based on an upper bound of a Global Discretization Error that occurs when a pre-trained diffusion probability model approximates a sampling process for generating second data from first data with a finite number of sampling steps; optimizing the temporal positions of the sampling steps to calculate a non-uniform time schedule such that the value of the error evaluation function is minimized; mapping the calculated non-uniform time schedule into an executable discrete sampling step sequence in accordance with a preset operating environment of the diffusion probability model to generate the second data from the first data; and driving the diffusion probability model according to the generated sampling step sequence to generate the second data from the first data.The processor performs the operation of setting the error evaluation function, wherein the processor reparameterizes the time variable of the sampling process into a log-signal-to-noise ratio (log-SNR), formulates the reparameterized sampling process into a Probability Flow Ordinary Differential Equation (Probability Flow ODE), wherein the processor calculates the non-uniform time schedule by optimizing the temporal positions of the sampling steps in a continuous parameter domain based on the log-signal-to-noise ratio in the operation of calculating the non-uniform time schedule, wherein the diffusion probability model accepts integer time steps from 0 to a preset maximum integer time step index as input, wherein the processor converts the step positions of the non-uniform time schedule calculated in the continuous parameter domain based on the log-signal-to-noise ratio into continuous time values, scales and rounds the continuous time values ​​based on the maximum integer time step index, and wherein the rounded value is greater than or equal to 0 and less than or equal to the maximum integer time step index. A computing device that clamps the discrete sampling step sequence to exist within a valid range, removes duplicate integer time steps when different consecutive time values ​​are converted to the same integer time step, and corrects the discrete sampling step sequence so that the order between adjacent integer time steps is not reversed. Claim 12 A computing device according to claim 11, wherein the sampling process performs a reverse diffusion process to generate the second data with reduced noise from the first data with added noise. Claim 13 delete Claim 14 A computing device according to claim 11, wherein the error evaluation function is set to an upper limit value derived from an error upper bound equation indicating that the global discretization error occurring when the sampling process is discretized into a finite number of sampling steps is less than or equal to a specific value, or a function value based on said upper limit, based on the difference between the continuous-time-based modeling result and the approximation result according to said discretization, with respect to the global discretization error. Claim 15 A computing device according to claim 14, wherein the upper limit of the global discretization error is determined by the start and end times of the sampling interval and is composed of a fixed component independent of the temporal position of the sampling steps and a variable error component whose value varies according to the temporal position of the sampling steps. Claim 16 In claim 15, the processor sets only the variable error component, excluding the fixed component, as the objective function, and optimizes the temporal positions of the sampling steps so that the value of the objective function is minimized, a computing device. Claim 17 A computing device according to claim 16, wherein the above non-uniform time schedule is calculated such that the value of the error evaluation function is minimized by densely arranging sampling steps in sections with high contribution to the variable error component and sparsely arranging sampling steps in sections with low contribution. Claim 18 In claim 16, the processor optimizes the temporal positions of the sampling steps by applying a boundary constraint that fixes the start and end steps of the sampling interval and a monotonic constraint that prevents the order between adjacent sampling steps from being reversed. Claim 19 A computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises one or more instructions, and when the instructions are executed by a computing device having one or more processors, the computing device is configured to set an error evaluation function based on an upper bound of the Global Discretization Error that occurs when a pre-trained diffusion probability model approximates a sampling process for generating second data from first data to a finite number of sampling steps; a step of calculating a non-uniform time schedule by optimizing the temporal positions of the sampling steps so that the value of the error evaluation function is minimized; a step of generating a discrete sampling step sequence executable by mapping the calculated non-uniform time schedule to a preset operating environment of the diffusion probability model; and a step of driving the diffusion probability model according to the generated sampling step sequence to generate the second data from the first data.The computer program causes the computing device to perform the following: in the step of setting the error evaluation function, reparameterize the time variable of the sampling process into a log-signal-to-noise ratio (log-SNR), and formulate the reparameterized sampling process into a probability flow ordinary differential equation (Probability Flow ODE); the computer program causes the computing device to calculate the non-uniform time schedule in the step of calculating the non-uniform time schedule by optimizing the temporal positions of the sampling steps in a continuous parameter domain based on the log-signal-to-noise ratio; the spreading probability model accepts integer time steps from 0 to a preset maximum integer time step index as input; and the computer program causes the computing device to convert the step positions of the non-uniform time schedule calculated in the continuous parameter domain based on the log-signal-to-noise ratio into continuous time values ​​in the step of generating by mapping, and scales and A computer program that rounds, clamps the rounded value so that it exists within a valid range of 0 or more and less than or equal to the maximum integer time step index, removes duplicate integer time steps when different consecutive time values ​​are converted to the same integer time step, and corrects the discrete sampling step sequence so that the order between adjacent integer time steps is not reversed.