A method and apparatus for diffusion model feature prediction

Abstract

The embodiment of the present application discloses a kind of diffusion model feature prediction method and device.In the embodiment of the present application, a predetermined target coefficient combination is obtained, wherein the target coefficient combination is generated using a multi-fidelity adaptive search strategy in an offline state, and the target coefficient combination includes N target coefficients.A target residual feature at a current time step is calculated based on the target coefficient combination and N historical cached residual features, wherein the N historical cached residual features are cached residual features corresponding to the previous N time steps of the current time step.An output feature of the diffusion model at the current time step is determined based on the target residual feature.Through the above method, the target coefficient combination generated using the multi-fidelity adaptive search strategy is used as the coefficient of the historical cached residual feature, which can improve the calculation efficiency of the diffusion model while maintaining good generation quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and apparatus for feature prediction using a diffusion model. Background Technology

[0002] With the development of artificial intelligence, the application of diffusion models is becoming more and more widespread. The diffusion model includes a forward process of gradually adding noise and a reverse process of gradually denoising to generate a clean image. The diffusion model has evolved from the traditional U-Net to the Transformer architecture. The surge in the number of parameters in the diffusion model has led to extremely slow inference speed and consumed a lot of computing resources, becoming a key bottleneck in actual deployment.

[0003] In existing technologies, a method of caching before prediction is used to predict future features, which improves the computational efficiency of the above diffusion model. However, at high speedup ratios, serious error accumulation is likely to occur, resulting in poor quality of the generated images and videos.

[0004] In summary, how to improve the computational efficiency of the above diffusion model while maintaining good generation quality is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for feature prediction of a diffusion model, which can improve the computational efficiency of the diffusion model while maintaining good generation quality.

[0006] In a first aspect, embodiments of the present invention provide a method for feature prediction of a diffusion model, the method comprising: obtaining a predetermined target coefficient combination, wherein the target coefficient combination is generated offline using a multi-fidelity adaptive search strategy, the target coefficient combination including N target coefficients, wherein N is a positive integer greater than or equal to 2; calculating target residual features for the current time step based on the target coefficient combination and N historical cached residual features, wherein the N historical cached residual features are cached residual features corresponding to the previous N time steps of the current time step; and determining the output features of the diffusion model at the current time step based on the target residual features.

[0007] Optionally, obtaining the predetermined combination of target coefficients specifically includes: generating N-1 target coefficients according to the multi-fidelity adaptive search strategy; determining the Nth target coefficient according to the affine constraints between the N-1 target coefficients and the N target coefficients; and determining the combination of target coefficients according to the N-1 target coefficients and the Nth target coefficient.

[0008] Optionally, generating N-1 target coefficients according to the multi-fidelity adaptive search strategy specifically includes: obtaining multiple sets of initial candidate coefficients, wherein each set of initial candidate coefficients includes N-1 candidate coefficients; determining a set of initial candidate coefficients from the multiple sets of initial candidate coefficients using a tree-structured Parsons estimator; progressively filtering each set of initial candidate coefficients to generate an evaluation score; and determining the N-1 target coefficients from the multiple sets of initial candidate coefficients based on the evaluation score.

[0009] Optionally, the step of progressively filtering each group of initial candidate coefficients to generate an evaluation score specifically includes: evaluating each group of initial candidate coefficients on a subset of prompt words corresponding to low fidelity to generate a first evaluation score; using a continuous halving algorithm pruning device to determine the first evaluation score based on a first intermediate threshold score; in response to the first evaluation score being greater than or equal to the first intermediate threshold score, evaluating each group of initial candidate coefficients on a subset of prompt words corresponding to medium fidelity to generate a second evaluation score; using a continuous halving algorithm pruning device to determine the second evaluation score based on a second intermediate threshold score; in response to the second evaluation score being greater than or equal to the second intermediate threshold score, evaluating each group of initial candidate coefficients on a subset of prompt words corresponding to high fidelity to generate the evaluation score.

[0010] Optionally, the subset of prompt words corresponding to low fidelity, the subset of prompt words corresponding to medium fidelity, and the subset of prompt words corresponding to high fidelity are nested subsets.

[0011] Optionally, the step of calculating the target residual feature of the current time step based on the target coefficient combination and N historical cache residual features specifically includes: determining the product of each target coefficient in the target coefficient combination and the corresponding historical cache residual feature; and determining the first sum of the N products as the target residual feature of the current time step.

[0012] Optionally, determining the output feature of the diffusion model at the current time step based on the target residual feature specifically includes: determining a second sum of the input feature at the current time step and the target residual feature; and determining the second sum as the output feature at the current time step.

[0013] Secondly, embodiments of the present invention provide an apparatus for feature prediction using a diffusion model, the apparatus comprising: An acquisition unit is used to acquire a predetermined target coefficient combination, wherein the target coefficient combination is generated offline using a multi-fidelity adaptive search strategy, and the target coefficient combination includes N coefficients, where N is a positive integer greater than or equal to 2; a calculation unit is used to calculate the target residual features of the current time step based on the target coefficient combination and N historical cache residual features, wherein the N historical cache residual features are the cache residual features corresponding to the previous N time steps of the current time step; a determination unit is used to determine the output features of the diffusion model at the current time step based on the target residual features.

[0014] Optionally, the acquisition unit is specifically used to: generate N-1 target coefficients according to the multi-fidelity adaptive search strategy; determine the Nth target coefficient according to the affine constraints between the N-1 target coefficients and the N target coefficients; and determine the target coefficient combination according to the N-1 target coefficients and the Nth target coefficient.

[0015] Optionally, the acquisition unit is further configured to: acquire multiple sets of initial candidate coefficients, wherein each set of initial candidate coefficients includes N-1 candidate coefficients; determine a set of initial candidate coefficients from the multiple sets of initial candidate coefficients using a tree-structured Parsons estimator; perform progressive filtering on each set of initial candidate coefficients to generate an evaluation score; and determine the N-1 target coefficients from the multiple sets of initial candidate coefficients based on the evaluation score.

[0016] Optionally, the acquisition unit is further configured to: evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to low fidelity to generate a first evaluation score; determine the first evaluation score based on a first intermediate threshold score using a continuous halving algorithm pruning device; in response to the first evaluation score being greater than or equal to the first intermediate threshold score, evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to medium fidelity to generate a second evaluation score; determine the second evaluation score based on a second intermediate threshold score using a continuous halving algorithm pruning device; in response to the second evaluation score being greater than or equal to the second intermediate threshold score, evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to high fidelity to generate the evaluation score.

[0017] Optionally, the subset of prompt words corresponding to low fidelity, the subset of prompt words corresponding to medium fidelity, and the subset of prompt words corresponding to high fidelity are nested subsets.

[0018] Optionally, the calculation unit is specifically used to: determine the product of each target coefficient in the target coefficient combination and the corresponding historical cache residual feature; and determine the first sum of the N products as the target residual feature of the current time step.

[0019] Optionally, the determining unit is specifically used to: determine a second sum of the input features of the current time step and the target residual features; and determine the second sum as the output features of the current time step.

[0020] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect or any one of the possible methods of the first aspect.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any one of the possibilities of the first aspect.

[0022] In this embodiment of the invention, a predetermined combination of target coefficients is obtained. This combination is generated offline using a multi-fidelity adaptive search strategy and includes N target coefficients, where N is a positive integer greater than or equal to 2. The target residual features for the current time step are calculated based on the target coefficient combination and N historical cache residual features, where the N historical cache residual features are the cache residual features corresponding to the previous N time steps. The output features of the diffusion model at the current time step are determined based on the target residual features. By using the target coefficient combination generated by the multi-fidelity adaptive search strategy as the coefficients of the historical cache residual features, the computational efficiency of the diffusion model can be improved while maintaining good generation quality. Attached Figure Description

[0023] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of a diffusion model feature prediction method according to an embodiment of the present invention; Figure 2 This is a flowchart of a method for generating a target coefficient combination according to an embodiment of the present invention; Figure 3 This is a flowchart of a method for generating N-1 target coefficients according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a device for feature prediction of a diffusion model in an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0025] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0026] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0027] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0028] In existing technologies, the method of caching before prediction can essentially be regarded as a linear combination based on historical cache features. However, it is limited by the fact that the solution of prediction coefficients is highly dependent on complex specific mathematical derivations or empirical rules such as the cumulative error of shallow probes. For example, complex explicit mathematical derivations such as Taylor expansion or empirical threshold rules are used to determine the prediction coefficients. The above methods are not only cumbersome in theoretical derivation, but also the prediction error will be amplified exponentially under a large cache interval, resulting in serious visual distortion, color deviation and noise artifacts in the generated images or videos. This cannot meet the visual quality requirements of high-magnification acceleration scenes. Therefore, how to determine more accurate prediction coefficients so as to improve the computational efficiency of diffusion models while maintaining good generation quality is a problem that needs to be solved.

[0029] In this embodiment of the invention, the diffusion model is a mainstream high-fidelity visual generation framework that includes a forward process of progressively adding noise and a reverse process of progressively denoising to generate a clean image.

[0030] In this embodiment of the invention, to solve the above problems, a method for feature prediction using a diffusion model is proposed, specifically as follows: Figure 1 As shown, the method includes: Step S101: Obtain a predetermined combination of target coefficients.

[0031] Specifically, the target coefficient combination is generated offline using a multi-fidelity adaptive search strategy. The target coefficient combination includes N target coefficients, where N is a positive integer greater than or equal to 2.

[0032] In one possible implementation, obtaining the predetermined target coefficient combination specifically includes, for example: Figure 2 As shown, it includes the following: Step S201: Generate N-1 target coefficients according to the multi-fidelity adaptive search strategy.

[0033] Specifically, in the denoising process of the diffusion model, the complete Diffusion Transformer (DiT) module is calculated at the time step that needs to be calculated according to the predetermined caching strategy, and the residual feature r_{t_j} is cached as the historical cached residual feature. The DiT is a diffusion model architecture with stronger scalability and expressive power. The cached residual feature and the historical cached residual feature can both be called residual features, defined as r_t = F(x_t) - x_t, where x_t is the input of the DiT module and F(x_t) is the output.

[0034] In one possible implementation, a method of caching before prediction is used to predict features. This involves using historical cached residual features as input variables and estimating future target residual features through different forms of linear combinations. The specific formula is as follows: r^{pred}_{t^*} = \sum_{j} c_j r_{t_j} Wherein, t^* is the time step to be predicted, t_j is the time step of the historical cache, c_j is the scalar coefficient, i.e. the target coefficient, and j represents the number of historical cache residual features to be used, i.e. the number of target coefficients. Among the target coefficients, there is an affine constraint, which means that the sum of the linear combination coefficients is 1, i.e., ∑ c_j = 1. The affine constraint ensures that the prediction is unbiased and can be accurately recovered when the features remain constant in a steady state.

[0035] In one possible implementation, the Multi-Fidelity Adaptive Search is an optimization strategy that efficiently searches for optimal parameters by evaluating candidates on subsets of data of different sizes and fidelity levels, combined with an early stopping mechanism. The Multi-Fidelity Adaptive Search is based on the Optuna framework.

[0036] For example, assuming the use of three historical cache residual features, the prediction formula for the target residual feature is: r^{pred}_{t^*} = c_0 r_{t_0} + c_1 r_{t_1} + c_2 r_{t_2} Wherein, t_0, t_1, and t_2 are the three cache time steps closest to t^* in descending order, and c_0, c_1, and c_2 are the target coefficients. Since the affine constraint simplifies the search space to a two-dimensional feasible region, the target coefficients c_0 and c_1 are generated according to the multi-fidelity adaptive search strategy.

[0037] Step S202: Determine the Nth target coefficient based on the N-1 target coefficients and the affine constraint between the N target coefficients.

[0038] In this embodiment of the invention, due to the affine constraint between the N-1 target coefficients and the N target coefficients, for example, N=3, c_0 + c_1 + c_2 = 1, and c_2 = 1 - c_0 - c_1.

[0039] Step S203: Determine the target coefficient combination based on the N-1 target coefficients and the Nth target coefficient.

[0040] In one possible implementation, the generation of N-1 target coefficients according to the multi-fidelity adaptive search strategy is specifically as follows: Figure 3 As shown, it includes the following: Step S301: Obtain multiple sets of initial candidate coefficients.

[0041] Specifically, each initial set of candidate coefficients includes N-1 candidate coefficients.

[0042] In one possible implementation, prior points are first derived based on a known predictor, for example, the known predictor could be a DiCache predictor; then, a large number of uniformly distributed initial candidate coefficients are generated in the prior point search space through Latin Hypercube Sampling (LHS). The multiple initial candidate coefficients uniformly cover the entire parameter search interval, making up for the defect that the prior points are only concentrated in the local optimal region and preventing the omission of the global optimum.

[0043] For example, assuming N=3, each initial candidate coefficient group includes 2 candidate coefficients (c_0, c_1).

[0044] Step S302: Determine a set of initial candidate coefficients from the multiple sets of initial candidate coefficients using the tree-structured Parsons estimator.

[0045] Specifically, the Tree-structured Parzen Estimator (TPE) is a sampling algorithm for hyperparameter optimization.

[0046] Step S303: Perform progressive filtering on each group of initial candidate coefficients to generate an evaluation score.

[0047] Specifically, each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to low fidelity to generate a first evaluation score; the first evaluation score is determined by a Successive Halving Algorithm (SHA) pruning algorithm based on a first intermediate threshold score; if the first evaluation score is greater than or equal to the first intermediate threshold score, each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to medium fidelity to generate a second evaluation score; the second evaluation score is determined by a Successive Halving Algorithm pruning algorithm based on a second intermediate threshold score; if the second evaluation score is greater than or equal to the second intermediate threshold score, each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to high fidelity to generate the evaluation score; this method can reduce the number of evaluations; wherein, the subset of prompt words corresponding to low fidelity, the subset of prompt words corresponding to medium fidelity, and the subset of prompt words corresponding to high fidelity are nested subsets, with increasing size.

[0048] For example, the low fidelity can be represented as L1, and the corresponding subset of prompt words is D_{L1}; the low fidelity can be represented as L2, and the corresponding subset of prompt words is D_{L2}; the high fidelity can be represented as L3, and the corresponding subset of prompt words is D_{L3}, where D_{L1} is the subset of prompt words. D_{L2} D_{L3} That is, D_{L2} is a subset of D_{L3}, and D_{L1} is a subset of D_{L2}; this is only an illustrative example.

[0049] Step S304: Determine the N-1 target coefficients based on the evaluation scores from the plurality of initial candidate coefficients.

[0050] Specifically, the N-1 target coefficients are determined by selecting the initial candidate coefficient with the highest evaluation score.

[0051] In this embodiment of the invention, the optimization objective is to determine the N-1 target coefficients, for example, (c_0, c_1) such that, on a given dataset, the peak signal-to-noise ratio (PSNR) between the accelerated generation result G(p; c_0, c_1) and the unaccelerated reference output G_{ref}(p) is maximized, where G represents the generation function and p represents the prompt word.

[0052] In this embodiment of the invention, the specific values ​​of c_0, c_1 and c_2 are determined by the above method.

[0053] Step S102: Calculate the target residual features for the current time step based on the target coefficient combination and N historical cache residual features.

[0054] Specifically, the N historical cache residual features are the cache residual features corresponding to the previous N time steps of the current time step.

[0055] In one possible implementation, the step of calculating the target residual feature for the current time step based on the target coefficient combination and N historical cache residual features specifically includes: determining the product of each target coefficient in the target coefficient combination with the corresponding historical cache residual feature; determining the first sum of the N products as the target residual feature for the current time step, that is, calculating the target residual feature according to the target residual feature prediction formula.

[0056] Step S103: Determine the output features of the diffusion model at the current time step based on the target residual features.

[0057] Specifically, a second sum of the input features at the current time step and the target residual features is determined; the second sum is then determined as the output features at the current time step.

[0058] The following is a detailed explanation of the process of predicting the output features of a diffusion model through a complete embodiment: First, the optimal target coefficient combination (c^*_0, c^*_1, c^*_2) is obtained through offline search. Then, during the denoising process of the diffusion model, the residual features r_{t_j} are cached at the time steps requiring computation, following a predetermined caching strategy, by performing a complete DiT module computation at the required time step. Next, at the time step t^* where computation needs to be skipped (i.e., the time step requiring prediction), the target coefficient combination is directly predicted using the three most recently cached historical residual features through the linear formula r^{pred}_{t^*} = c^*_0 r_{t_0} + c^*_1 r_{t_1} + c^*_2 r_{t_2}. Finally, the predicted target residual features are added back to the input of the diffusion model to obtain the output estimate F_{pred}(x_{t^*}) = x_{t^*} + ... r^{pred}_{t^*}, thus skipping the complete calculation process and accelerating the calculation of predicted features.

[0059] In one possible implementation, the linear formula for calculating the target residual features uses three historical cache residual features, but it can also use two, four or more historical cache residual features. This embodiment of the invention does not impose any limitations.

[0060] In one possible implementation, during the search process of the multi-fidelity adaptive search strategy, the objective function can be PSNR, or it can be the Structural Similarity Index Measure (SSIM), the Learned Perceptual Image Patch Similarity (LPIPS), or a weighted combination of PSNR, SSIM, and LPIPS, as the target to be maximized or minimized to adapt to different visual quality requirements. In addition, during the search process of the multi-fidelity adaptive search strategy, black-box optimization algorithms can also be used to search for target coefficients, such as genetic algorithms, pure random search, or Bayesian optimization without multi-fidelity mechanisms, depending on the actual situation.

[0061] In one possible implementation, in addition to the residual features, the cached data can also cache the output residual features of the diffusion model. The target coefficient combination is determined by the method of this embodiment of the invention, and the output features of the non-computation time step are predicted by the formula F_{pred}(x_{t^*}) = \sum c_j F(x_{t_j}), wherein \sum c_j = 1 satisfies the affine constraint.

[0062] In this embodiment of the invention, the determination of target coefficients is transformed into an offline search problem under affine constraints, overcoming the limitations of existing solutions that rely on complex theoretical derivations or single empirical rules, and obtaining more accurate prediction results. By combining TPE sampling and SHA pruning multi-fidelity adaptive search algorithms, the target coefficients of linear combinations are efficiently optimized on nested subsets of prompt words, significantly reducing search costs and obtaining target coefficient combinations that are more adapted to diffusion models and data distributions, thus avoiding errors caused by discrepancies between theoretical assumptions and actual conditions.

[0063] In this embodiment of the invention, an apparatus for feature prediction using a diffusion model is provided, such as... Figure 4 As shown, it specifically includes: an acquisition unit 401, a calculation unit 402, and a determination unit 403; The acquisition unit 401 is used to acquire a predetermined target coefficient combination, wherein the target coefficient combination is generated offline using a multi-fidelity adaptive search strategy, and the target coefficient combination includes N coefficients, where N is a positive integer greater than or equal to 2; the calculation unit 402 is used to calculate the target residual features of the current time step based on the target coefficient combination and N historical cache residual features, wherein the N historical cache residual features are the cache residual features corresponding to the previous N time steps of the current time step; and the determination unit 403 is used to determine the output features of the diffusion model at the current time step based on the target residual features.

[0064] Further, the acquisition unit is specifically used to: generate N-1 target coefficients according to the multi-fidelity adaptive search strategy; determine the Nth target coefficient according to the affine constraints between the N-1 target coefficients and the N target coefficients; and determine the target coefficient combination according to the N-1 target coefficients and the Nth target coefficient.

[0065] Furthermore, the acquisition unit is specifically used to: acquire multiple sets of initial candidate coefficients, wherein each set of initial candidate coefficients includes N-1 candidate coefficients; determine a set of initial candidate coefficients from the multiple sets of initial candidate coefficients according to a tree-structured Parsons estimator; perform progressive filtering on each set of initial candidate coefficients to generate an evaluation score; and determine the N-1 target coefficients from the multiple sets of initial candidate coefficients according to the evaluation score.

[0066] Furthermore, the acquisition unit is specifically configured to: evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to low fidelity to generate a first evaluation score; determine the first evaluation score based on a first intermediate threshold score using a continuous halving algorithm pruning device; in response to the first evaluation score being greater than or equal to the first intermediate threshold score, evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to medium fidelity to generate a second evaluation score; determine the second evaluation score based on a second intermediate threshold score using a continuous halving algorithm pruning device; in response to the second evaluation score being greater than or equal to the second intermediate threshold score, evaluate each set of initial candidate coefficients on the subset of prompt words corresponding to high fidelity to generate the evaluation score.

[0067] Furthermore, the subset of prompt words corresponding to low fidelity, the subset of prompt words corresponding to medium fidelity, and the subset of prompt words corresponding to high fidelity are nested subsets.

[0068] Furthermore, the calculation unit is specifically used to: determine the product of each target coefficient in the target coefficient combination and the corresponding historical cache residual feature; and determine the first sum of the N products as the target residual feature of the current time step.

[0069] Further, the determining unit is specifically used to: determine a second sum of the input features of the current time step and the target residual features; and determine the second sum as the output features of the current time step.

[0070] Figure 5 This is a schematic diagram of the structure of the electronic device described in an embodiment of the present invention. Figure 5 As shown, it includes a general computer hardware architecture, which includes at least a processor 501 and a memory 502. The processor 501 and the memory 502 are connected via a bus 503. The memory 502 is adapted to store instructions or programs executable by the processor 501. The processor 501 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 501 executes the instructions stored in the memory 502 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 503 connects the above-mentioned components together, and also connects the above-mentioned components to the display controller 504, the display device, and the input / output (I / O) device 505. The input / output (I / O) device 505 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 505 is connected to the system via an input / output (I / O) controller 506.

[0071] The instructions stored in memory 502 are executed by at least one processor 501 to achieve: obtaining a predetermined target coefficient combination, calculating the target residual features of the current time step based on the target coefficient combination and N historical cache residual features, and determining the output features of the diffusion model at the current time step based on the target residual features.

[0072] Specifically, the electronic device includes: one or more processors 501 and a memory 502. Figure 5 Take processor 501 as an example. Processor 501 and memory 502 can be connected via a bus or other means. Figure 5 Taking a bus connection as an example, memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 501 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 502, thereby realizing the aforementioned method for predicting diffusion model characteristics.

[0073] Memory 502 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 502 may optionally include memory remotely located relative to processor 501, and these remote memories can be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0074] One or more modules are stored in memory 502 and, when executed by one or more processors 501, perform the diffusion model feature prediction method in any of the above method embodiments.

[0075] As those skilled in the art will recognize, various aspects of the embodiments of the present invention can be implemented as a system, method, or computer program product. Therefore, various aspects of the embodiments of the present invention can take the form of a completely hardware implementation, a completely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." Furthermore, various aspects of the embodiments of the present invention can take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0076] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, (but not limited to) an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the context of embodiments of the present invention, a computer-readable storage medium can be any tangible medium capable of containing or storing a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0077] Computer-readable signal media may include propagated digital signals having computer-readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and can communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0078] Program code implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0079] Computer program code for performing operations relating to various aspects of embodiments of the present invention can be written in any combination of one or more programming languages, including: object-oriented programming languages ​​such as Java, Smalltalk, C++, etc.; and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0080] The flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present invention describe various aspects of the embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions (executed via the processor of the computer or other programmable data processing apparatus) create means for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0081] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus or other means to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing that includes instructions that implement the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0082] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operable steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0083] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse processing. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A method for feature prediction in a diffusion model, characterized in that, The method includes: Obtain a predetermined combination of target coefficients, wherein the combination of target coefficients is generated offline using a multi-fidelity adaptive search strategy, and the combination of target coefficients includes N target coefficients, where N is a positive integer greater than or equal to 2; The target residual features for the current time step are calculated based on the target coefficient combination and N historical cache residual features, wherein the N historical cache residual features are the cache residual features corresponding to the previous N time steps for the current time step; The output features of the diffusion model at the current time step are determined based on the target residual features.

2. The method according to claim 1, characterized in that, The acquisition of the predetermined target coefficient combination specifically includes: N-1 target coefficients are generated based on the multi-fidelity adaptive search strategy; The Nth target coefficient is determined based on the affine constraints between the N-1 target coefficients and the N target coefficients. The target coefficient combination is determined based on the N-1 target coefficients and the Nth target coefficient.

3. The method according to claim 2, characterized in that, The step of generating N-1 target coefficients according to the multi-fidelity adaptive search strategy specifically includes: Obtain multiple sets of initial candidate coefficients, where each set of initial candidate coefficients includes N-1 candidate coefficients; A set of initial candidate coefficients is determined from the multiple sets of initial candidate coefficients using a tree-structured Parsons estimator. Each set of initial candidate coefficients is progressively filtered to generate an evaluation score; The N-1 target coefficients are determined from the plurality of initial candidate coefficients based on the evaluation scores.

4. The method according to claim 3, characterized in that, The step of progressively filtering each group of initial candidate coefficients to generate an evaluation score specifically includes: Each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to low fidelity to generate a first evaluation score; The first evaluation score is determined by the continuous halving algorithm pruning tool based on the first intermediate threshold score. In response to the first evaluation score being greater than or equal to the first intermediate threshold score, each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to the medium fidelity, and a second evaluation score is generated. The second evaluation score is determined by a continuous halving algorithm pruning tool based on a second intermediate threshold score. In response to the second evaluation score being greater than or equal to the second intermediate threshold score, each set of initial candidate coefficients is evaluated on the subset of prompt words corresponding to high fidelity, thereby generating the evaluation score.

5. The method according to claim 4, characterized in that, The subset of prompt words corresponding to low fidelity, the subset of prompt words corresponding to medium fidelity, and the subset of prompt words corresponding to high fidelity are nested subsets.

6. The method according to claim 1, characterized in that, The step of calculating the target residual features for the current time step based on the target coefficient combination and N historical cache residual features specifically includes: Determine the product of each target coefficient in the target coefficient combination with the corresponding historical cache residual feature; The first sum of the N products is determined as the target residual feature for the current time step.

7. The method according to claim 1, characterized in that, Determining the output features of the diffusion model at the current time step based on the target residual features specifically includes: Determine a second sum of the input features at the current time step and the target residual features; The second sum is determined as the output feature of the current time step.

8. An apparatus for predicting features of a diffusion model, characterized in that, The device includes: The acquisition unit is used to acquire a predetermined target coefficient combination, wherein the target coefficient combination is generated in an offline state using a multi-fidelity adaptive search strategy, and the target coefficient combination includes N coefficients, where N is a positive integer greater than or equal to 2; The calculation unit is used to calculate the target residual features of the current time step based on the target coefficient combination and N historical cache residual features, wherein the N historical cache residual features are the cache residual features corresponding to the previous N time steps of the current time step; A determining unit is used to determine the output features of the diffusion model at the current time step based on the target residual features.

9. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images