A robust optimization method for conditional diffusion model

Abstract

The present application discloses a method for optimizing the robustness of a conditional diffusion model. The method inputs a noise-added information data set and preset conditions into a conditional neural network for training, and inputs a noise-added information data set into an unconditional neural network for training; obtains the fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtains the quantization errors generated by the trained conditional neural network when the input conditions are wrong in the reverse process of the conditional diffusion model; inputs the fitting errors and quantization errors into a constructed filter to calculate the target parameters; inputs the trained conditional neural network, the trained unconditional neural network and the target parameters into a constructed controller to generate a control signal; inputs the Gaussian distribution and the control signal into a constructed generation model to obtain the generation data output by the generation model. The present application can improve the reliability and effectiveness of the model.

Description

Technical Field

[0001] The present application relates to the technical field of image generation, and in particular to a method for optimizing the robustness of a conditional diffusion model. Background Art

[0002] Diffusion models are a type of generative model that gradually applies noise to an image in the forward phase until the image is corrupted and becomes completely Gaussian noise, and then learns the process of restoring the original image from Gaussian noise in the reverse phase. Diffusion models are becoming increasingly popular in computer vision applications due to their excellent generative performance and variability. In particular, conditional diffusion models can guide the model to generate results of a certain category.

[0003] However, inaccurate conditional inputs in the reverse process can cause the neural network to generate fixed errors, thereby reducing the adaptability of the trained conditional diffusion model. The traditional solution is to clean the data before inference, but this method is time-consuming and cannot solve the problem of low robustness of the conditional diffusion model. Summary of the invention

[0004] This application aims to propose a robustness optimization method for conditional diffusion model, which can reduce model error and thus improve the reliability and effectiveness of the model.

[0005] In a first aspect, an embodiment of the present application provides a method for optimizing robustness of a conditional diffusion model, the method comprising:

[0006] Noise the input signal to obtain a noisy information data set;

[0007] Construct conditional and unconditional neural networks;

[0008] Inputting the noise-added information data set and the preset conditions into the conditional neural network for training to obtain a trained conditional neural network, and inputting the noise-added information data set into the unconditional neural network for training to obtain a trained unconditional neural network;

[0009] Obtaining fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtaining quantization errors generated by the trained conditional neural network when an input condition is wrong during the reverse process of the conditional diffusion model;

[0010] Inputting the fitting error and the quantization error into the constructed filter to calculate the target parameter;

[0011] Inputting the trained conditional neural network, the trained unconditional neural network and the target parameter into a constructed controller to generate a control signal;

[0012] The Gaussian distribution and the control signal are input into the constructed generation model to obtain the generation data output by the generation model.

[0013] Compared with the prior art, this application has the following beneficial effects:

[0014] This method obtains the fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtains the quantization errors generated by the trained conditional neural network when the input conditions are wrong in the reverse process of the conditional diffusion model, inputs the fitting errors and quantization errors into the constructed filter, calculates the target parameters, and minimizes the error influence of the calculated target parameters by comprehensively considering the fitting errors generated during the training process and the quantization errors generated when the input conditions are wrong, thereby effectively reducing the model error and improving the robustness of the model to the system error. The trained conditional neural network, the trained unconditional neural network and the target parameters are input into the constructed controller to generate a control signal, and then the Gaussian distribution and the control signal are input into the constructed generation model to obtain the generated data output by the generation model, which can improve the reliability and effectiveness of the model while minimizing the error influence.

[0015] In some embodiments, the constructed filter is obtained by:

[0016] According to the first training function corresponding to the trained conditional neural network, a first fitting error generated by the trained conditional neural network during the training process is obtained; and according to the second training function corresponding to the trained unconditional neural network, a second fitting error generated by the trained unconditional neural network during the training process is obtained;

[0017] Converting the constructed controller into a distribution form including the quantization error, the first fitting error and the second fitting error;

[0018] The distribution form is divided into a distribution of expected output and a distribution actually generated;

[0019] Calculating the intersection area between the distribution of the expected output and the distribution actually generated;

[0020] Constructing a first function corresponding to half of the intersection area;

[0021] According to the first function, determining a function monotonicity constraint;

[0022] Constructing a second function corresponding to all the intersection areas;

[0023] Determining a function value constraint according to the second function;

[0024] By balancing the function monotonicity constraint and the function value constraint, a constructed filter is obtained.

[0025] In some embodiments, obtaining a first fitting error generated by the trained conditional neural network during the training process according to a first training function corresponding to the trained conditional neural network includes:

[0026] Construct the first training function corresponding to the trained conditional neural network:

[0027]

[0028] The trained conditional neural network is trained according to the first training function to obtain a first fitting error generated by the trained conditional neural network during the training process:

[0029]

[0030] in, represents the input signal, Indicates the input conditions. express and A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained conditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the conditional neural network in the forward process, represents the first fitting error.

[0031] In some embodiments, obtaining a second fitting error generated by the trained unconditional neural network during the training process according to a second training function corresponding to the trained unconditional neural network includes:

[0032] Construct a second training function corresponding to the trained unconditional neural network:

[0033]

[0034] The trained unconditional neural network is trained according to the second training function to obtain a second fitting error generated by the trained unconditional neural network during the training process:

[0035]

[0036] in, represents the input signal, express A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained unconditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the unconditional neural network in the forward process, represents the second fitting error.

[0037] In some implementations, the constructed controller is obtained by:

[0038]

[0039] in, represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates the input conditions. Indicates different moments The signal after adding Gaussian noise, Represents a controller.

[0040] In some implementations, converting the constructed controller into a distribution form including the quantization error, the first fitting error, and the second fitting error includes:

[0041] Let the controller be , converting the constructed controller into a distribution form including the quantization error, the first fitting error and the second fitting error in the following manner:

[0042]

[0043] in, represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, Indicates that the covariance matrix is ​​the identity matrix.

[0044] In some implementations, constructing a first function corresponding to half of the intersection area includes:

[0045] According to the properties of Gaussian distribution, the first function in different situations is constructed as follows:

[0046]

[0047] The integral probability density function solution method of Gaussian distribution is used to solve the first function in different situations to obtain the first function corresponding to half of the intersection area:

[0048]

[0049] in, represents the intersection of the distribution of the expected output and the distribution of the actual generation value, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, represents the standard deviation, A first function corresponding to half of the intersection area is represented.

[0050] In some implementations, constructing a second function corresponding to all of the intersection areas includes:

[0051]

[0052] in, represents the second function, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, represents the intersection of the distribution of the expected output and the distribution of the actual generation value, Represents standard deviation.

[0053] In some embodiments, the constructed filter is obtained by:

[0054]

[0055] in, represents the target parameter, represents the balance factor, represents the quantization error, represents the fitting error.

[0056] In some implementations, the constructed generative model is obtained by:

[0057]

[0058] in, represents the output result of the constructed generative model, represents the hyperparameter, express At different times The cumulative sum of represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates different moments The signal after adding Gaussian noise, Indicates the input conditions. represents a noise parameter, Represents a Gaussian distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0060] Figure 1 It is a flowchart of an embodiment of a method for optimizing robustness of a conditional diffusion model provided by the present application;

[0061] Figure 2 It is a schematic diagram of the framework of robustness optimization of the conditional diffusion model provided by this application;

[0062] Figure 3 It is a schematic diagram of the smile theory provided by this application;

[0063] Figure 4 This is a schematic diagram of the analysis of the smile theory conclusions provided by this application;

[0064] Figure 5 It is a schematic diagram of the experimental results of the MNIST handwritten digit set when the system error provided by this application is 0.3;

[0065] Figure 6 It is a schematic diagram of the experimental results of the MNIST handwritten digit set when the system error provided by this application is -0.3;

[0066] Figure 7 It is a schematic diagram of the experimental results of the MNIST handwritten digit set when the system error provided by this application is 3;

[0067] Figure 8 It is a schematic diagram of the experimental results of the MNIST handwritten digit set when the system error provided by this application is -3;

[0068] Fig. 9 This is a schematic diagram of the experimental results of the CIFAR-10 data set when the system error provided in this application is 0.03. DETAILED DESCRIPTION

[0069] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0070] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0071] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0072] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0073] The popularity of data-driven decision-making makes the accuracy and reliability of data analysis models particularly important. As a data analysis model that is simple to train, stable and controllable, the diffusion model has been widely used in many fields, including computer vision, natural language processing, and time series data prediction. However, this type of data analysis model based on the diffusion model may show certain fragility in the face of systematic errors. These errors can come from the data itself, such as measurement errors or missing information, or they can be caused by human factors, such as inaccurate text input in large language models. If effective preprocessing measures are not taken, these errors will seriously affect the generalization ability and prediction accuracy of the model, reducing the reliability and effectiveness of the model.

[0074] Although existing data cleaning technologies can improve the quality of single-source or even multi-source data fusion to a certain extent, it is usually very challenging to completely eliminate systematic errors, and new errors may be introduced in some processes that rely on human judgment. Therefore, it is very urgent to develop a conditional diffusion model that can inherently resist systematic errors.

[0075] In order to solve the problems of poor generalization ability and low prediction accuracy of the model, as well as low reliability and low effectiveness of the model, this application proposes a conditional diffusion model robustness optimization method.

[0076] Reference Figure 1 The embodiment of the present application provides a method for optimizing the robustness of a conditional diffusion model, the method comprising the following steps:

[0077] Step S100, adding noise to the input signal to obtain a noise information data set;

[0078] Step S200, constructing a conditional neural network and an unconditional neural network;

[0079] Step S300, inputting the noise-added information data set and the preset conditions into a conditional neural network for training to obtain a trained conditional neural network, and inputting the noise-added information data set into an unconditional neural network for training to obtain a trained unconditional neural network;

[0080] Step S400, obtaining fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtaining quantization errors generated by the trained conditional neural network when the input conditions are wrong during the reverse process of the conditional diffusion model;

[0081] Step S500, input the fitting error and the quantization error into the constructed filter, and calculate the target parameters;

[0082] Step S600, inputting the trained conditional neural network, the trained unconditional neural network and the target parameters into the constructed controller to generate a control signal;

[0083] Step S700: input the Gaussian distribution and the control signal into the constructed generation model to obtain the generation data output by the generation model.

[0084] In this embodiment, by obtaining the fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtaining the quantization error generated by the trained conditional neural network when the input condition is wrong in the reverse process of the conditional diffusion model, the fitting error and the quantization error are input into the constructed filter, and the target parameter is calculated. By comprehensively considering the fitting error generated during the training process and the quantization error generated when the input condition is wrong, the calculated target parameter achieves the minimum error effect, thereby effectively reducing the model error and improving the robustness of the model to the system error. The trained conditional neural network, the trained unconditional neural network and the target parameter are input into the constructed controller to generate a control signal, and then the Gaussian distribution and the control signal are input into the constructed generation model to obtain the generated data output by the generation model, which can improve the reliability and effectiveness of the model while minimizing the error effect.

[0085] It should be noted that the input signal of this embodiment may be image data, or data corresponding to a relevant actual scene may be selected, and this embodiment does not make any specific limitation.

[0086] In some embodiments, the constructed filter is obtained by:

[0087] According to the first training function corresponding to the trained conditional neural network, a first fitting error generated by the trained conditional neural network during the training process is obtained, and according to the second training function corresponding to the trained unconditional neural network, a second fitting error generated by the trained unconditional neural network during the training process is obtained;

[0088] Converting the constructed controller into a distribution form including a quantization error, a first fitting error, and a second fitting error;

[0089] The distribution form is divided into the distribution of expected output and the distribution actually generated;

[0090] Calculate the intersection area between the expected output distribution and the actual generated distribution;

[0091] Construct the first function corresponding to half of the intersection area;

[0092] According to the first function, determining the function monotonicity constraint;

[0093] Construct a second function corresponding to all intersection areas;

[0094] Determining a function value constraint based on the second function;

[0095] By balancing the function monotonicity constraint and the function value constraint, a constructed filter is obtained.

[0096] In this embodiment, by converting the constructed controller into a distribution form including quantization error, first fitting error and second fitting error, a variety of errors are comprehensively considered to train the model, so that the model reduces the influence of system errors. By balancing the function monotonicity constraint and the function value constraint, a constructed filter is obtained, and the optimal parameters can be obtained through the constructed filter to train the conditional diffusion model, thereby minimizing the influence of errors.

[0097] In some implementations, obtaining a first fitting error generated by the trained conditional neural network during the training process according to a first training function corresponding to the trained conditional neural network includes:

[0098] Construct the first training function corresponding to the trained conditional neural network:

[0099]

[0100] The trained conditional neural network is trained according to the first training function to obtain the first fitting error generated by the trained conditional neural network during the training process:

[0101]

[0102] in, represents the input signal, Indicates the input conditions. express and A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained conditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the conditional neural network in the forward process, represents the first fitting error.

[0103] In this embodiment, since there may be an error between the predicted value and the true value in the training stage, there is a fitting error when the conditional neural network is fitted, and this fitting error can be obtained by the conditional neural network during the training process. By considering the fitting error of the conditional neural network for training, the conditional neural network trained later can reduce the influence of the error.

[0104] In some embodiments, obtaining a second fitting error generated by the trained unconditional neural network during the training process according to a second training function corresponding to the trained unconditional neural network includes:

[0105] Construct the second training function corresponding to the trained unconditional neural network:

[0106]

[0107] The trained unconditional neural network is trained according to the second training function to obtain the second fitting error generated by the trained unconditional neural network during the training process:

[0108]

[0109] in, represents the input signal, express A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained unconditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the unconditional neural network in the forward process, represents the second fitting error.

[0110] In this embodiment, since there may be an error between the predicted value and the true value in the training stage, there is a fitting error when the unconditional neural network is fitted, and this fitting error can be obtained by the unconditional neural network during the training process. By considering the fitting error of the unconditional neural network for training, the error influence of the unconditional neural network trained later can be reduced.

[0111] In some implementations, the constructed controller is obtained by:

[0112]

[0113] in, represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates the input conditions. Indicates different moments The signal after adding Gaussian noise, Represents a controller.

[0114] In some implementations, converting the constructed controller into a distribution form including a quantization error, a first fitting error, and a second fitting error includes:

[0115] Let the controller be , the constructed controller is converted into a distribution form including quantization error, first fitting error and second fitting error in the following way:

[0116]

[0117] in, represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, Indicates that the covariance matrix is ​​the identity matrix.

[0118] In this embodiment, the constructed controller is converted into a distribution form that includes quantization error, first fitting error and second fitting error. This can comprehensively consider the fitting error generated during the training process and the quantization error generated when the input conditions are incorrect, so as to minimize the error impact of the target parameters obtained by the subsequent calculation, thereby effectively reducing the model error and improving the robustness of the model to system errors.

[0119] In some implementations, constructing a first function corresponding to half the intersection area includes:

[0120] According to the properties of Gaussian distribution, the first function in different situations is constructed as follows:

[0121]

[0122] The integral probability density function solution method of Gaussian distribution is used to solve the first function in different cases, and the first function corresponding to half of the intersection area is obtained:

[0123]

[0124] in, The intersection of the expected output distribution and the actual generated distribution value, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, represents the standard deviation, Represents the first function corresponding to half the intersection area.

[0125] In some implementations, constructing a second function corresponding to all intersection areas includes:

[0126]

[0127] in, represents the second function, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, The intersection of the expected output distribution and the actual generated distribution value, Represents standard deviation.

[0128] In some embodiments, the constructed filter is obtained by:

[0129]

[0130] in, represents the target parameter, represents the balance factor, represents the quantization error, represents the fitting error.

[0131] In this embodiment, since the constructed filter can balance the function monotonicity constraint and the function value constraint, the optimal parameters (ie, target parameters) can be obtained through the constructed filter to minimize the impact of the error.

[0132] In some implementations, the constructed generative model is obtained by:

[0133]

[0134] in, represents the output result of the constructed generative model, represents the hyperparameter, express At different times The cumulative sum of represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates different moments The signal after adding Gaussian noise, Indicates the input conditions. represents a noise parameter, Represents a Gaussian distribution.

[0135] In this embodiment, the constructed generative model uses both the trained unconditional neural network and the trained conditional neural network, and also uses the target parameters. Therefore, the generative model can improve the reliability and effectiveness of the model while minimizing the influence of errors.

[0136] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:

[0137] This embodiment proposes a novel robustness optimization method for the conditional diffusion model. The core of this method is to introduce an innovative "smile theory", and through theoretical proof, it can improve the robustness of the conditional diffusion model to system errors to a certain extent. Compared with traditional data cleaning technology, the "smile theory" provides a more efficient and objective method, which can reduce the impact of system errors without significantly increasing costs, thereby improving the application potential of the conditional diffusion model in complex data environments. The method of this embodiment specifically includes the following contents:

[0138] 1. Framework for robust optimization of conditional diffusion models.

[0139] Reference Figure 2 , input signal After adding noise, the set , and thus train the unconditional Neural Networks and Conditional Neural Network, Condition It can be assumed that the input data is wrong. Systematic error (i.e. quantization error) caused by inherent inaccuracy in the input data The neural network fitting error c enters the filter, and the filter outputs the parameter w used in the controller to adjust the weights of the two neural networks. Then the Gaussian distribution z and the control signal output by the controller work together to generate .

[0140] In the architecture of robustness optimization of the conditional diffusion model, the design of the filter is one of the key innovations to achieve model robustness improvement. The role of the filter is to optimize the data reconstruction quality in the inverse process by reducing the impact of errors. In this embodiment, special attention is paid to how to obtain the optimal parameter (i.e., target parameter) w of the filter through theoretical analysis and proof to minimize the impact of errors.

[0141] 2. The specific process of the method in this embodiment.

[0142] The conditional diffusion model is the result of the exploration of the diffusion model at the application level. It can control the output results according to the will and is used to solve various challenging real-world tasks. The scaling mechanism of the parameter w added to the conditional diffusion model can balance relevance and diversity. The mathematical mechanism of the conditional diffusion model generation stage becomes:

[0143] (1)

[0144] in, represents the output result of the constructed generative model, It indicates the hyperparameters set before model training, which determines the intensity of noise addition at each step; express At different times The cumulative sum of Indicates that it has parameters The trained conditional neural network can be used in a given , , predict the noise vector under condition y; Indicates that it has parameters The trained unconditional neural network, Indicates different moments The signal after adding Gaussian noise, represents a time series, , Indicates the input conditions. represents a noise parameter that determines the new signal The Gaussian noise contained in size; represents a Gaussian distribution, represents the target parameter, , was used to balance relevance and diversity.

[0145] 1. Analysis of the necessity of the study.

[0146] Before studying the robustness of the conditional diffusion model, it is necessary to verify the systematic error Will the accumulation of T steps lead to the generation effect of the diffusion model being uncontrollable in the end? The mathematical induction method is used to study the final error accumulation:

[0147] (2)

[0148] in, , . Focus on the largest item , when T is very large, If the input data is inaccurate, the neural network will have systematic errors. ,So It’s going to be very big.

[0149] Therefore, the errors produced by the neural network due to the inaccuracy of the input data will be significantly amplified due to the multiplication of coefficients, making the final generation effect out of control.

[0150] 2. Filter design.

[0151] The training functions of the two neural networks in the conditional diffusion model are:

[0152] (3)

[0153] make:

[0154] (4)

[0155] in, and is the true value, that is, the Gaussian noise added in a certain step in the forward process, then , , Indicates that the random vector follows a Gaussian distribution with a mean of zero and a covariance matrix of the identity matrix; and is the difference, represents the fitting error of the conditional neural network during training (i.e., the first fitting error), It represents the fitting error (i.e., the second fitting error) of the unconditional neural network during training. Obviously It is the Gaussian noise added to a forward process at a certain step under the condition of y; is the Gaussian noise added to a forward process in the case of unconditional y. The two neural network distributions are independent and unrelated, then Recorded as , can be converted into a distributed form as follows:

[0156] (5)

[0157] in, represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, Indicates that the covariance matrix is ​​the identity matrix.

[0158] Ideally, the neural network will not produce errors, that is, , the expected output distribution is recorded as However, due to the inaccuracy of the input data and the errors in the training process of the neural network itself, the actual generated distribution is recorded as , which contains the error Next, we need to pay attention to how the change of parameter w affects the generation effect, that is, we need to pay attention to each step and This analysis can help evaluate the performance of the neural network model and adjust the parameter w to minimize the error in the generated effect.

[0159] 2.1. Function monotonicity constraints.

[0160] Compared to KL divergence, the distribution and The area of ​​intersection It can more intuitively show the similarities between them and the impact of the change of parameter w on the cross area, such as Figure 3 Shown is a schematic diagram of the smile theory. Is the distribution and Intersection Value, according to the probability density function of Gaussian distribution, we can get .

[0161] because and The area of ​​the intersection is symmetrical, half the size of It can reflect the change of the intersection area with the parameter w. According to the properties of Gaussian distribution, this area function (i.e. the first function) The expression is as follows:

[0162] (6)

[0163] in, ,Right now The positive and negative The positive and negative of is consistent. Referring to the solution method of the integral probability density function of Gaussian distribution, the solution is:

[0164] (7)

[0165] From this we can see that regardless of The positive and negative function There is no change, which conforms to the symmetry of Gaussian distribution. Next, we will explore the range of parameter w and the function This means that the increase of parameter w within this range will make and The intersection area increases, which is closer to the ideal, and the ability to suppress interference increases with the increase of parameter w.

[0166] when When :

[0167] (8)

[0168] In the analysis function When the monotonicity of The positive or negative sign of The monotonicity range is affected.

[0169] (9)

[0170] From this we can get:

[0171] like ,function exist monotonically increasing;

[0172] like When, function exist monotonically increasing;

[0173] like When, function It is always monotonically increasing.

[0174] when When :

[0175] Similarly, we can get:

[0176] like ,function exist monotonically increasing;

[0177] like When, function exist monotonically increasing;

[0178] like When, function It is always monotonically increasing.

[0179] Next follow ,make ,in It is a constant whose positive or negative value cannot be determined.

[0180] (10)

[0181] Neural network during training Using the same network structure and hyperparameters, it can be basically determined that Very small. Although The positive or negative value cannot be determined, but is a very small value, so Will not affect the overall The positive and negative conditions of This means that the condition is and When The increase of can improve the robustness of the conditional diffusion model. Similarly, when the condition is and When The increase of can improve the robustness of the conditional diffusion model, as shown in the following examples: Figure 4 shown.

[0182] In short, when Big enough or small enough, The positive and negative Therefore, this embodiment draws the following conclusions: When , add parameters The anti-interference ability of the diffusion model will be enhanced; if , as long as you ensure , add parameters It will also enhance the anti-interference ability of the model. Therefore, this embodiment only needs to consider different Case variables The positive and negative situation.

[0183] Therefore, we will focus on situation, in which and is a range of variables.

[0184] by For example, in this case, we need to consider the value of parameter w to ensure .

[0185] (11)

[0186] exist And if it is large enough, the parameter w is In the range, function Monotonically increasing.

[0187] exist In case of:

[0188] (12)

[0189] Then, in And when it is small enough, the parameter w is In the range, function is monotonically increasing. So the function Monotonous Monotonically increasing in range.

[0190] 2.2. Function value constraints.

[0191] For functions (i.e., the second function), it may be monotonically increasing within a certain range, but whether the maximum value within the range satisfies the condition that the intersection area should exceed a certain threshold is a question worth considering. According to the properties of the Gaussian distribution, the distribution and The area of ​​intersection for:

[0192] (13)

[0193] Obviously, the expected function in this embodiment is is always equal to 1, which corresponds to is always 0. However, the monotonicity constraint of the function requires , the union of the two constraints is .because ,in and is not a fixed constant, which means that although there may be a parameter w that makes d equal to 0 at a certain moment, due to and changes, cannot be guaranteed is always 0. So the function The value constraint and the monotonicity constraint constrain each other, forming a contradictory situation. Therefore, this embodiment needs to find a balance point between these two constraints.

[0194] 2.3. Cross constraints.

[0195] Let the random variable ; , the two are independent of each other, then The probability density function of is solved as:

[0196] (14)

[0197] From this, we can see that from the perspective of probability, the larger the parameter w is, The greater the probability of being 0. However, for the monotonicity constraint, the parameter w should be To balance the two constraints, this embodiment adds a balancing factor ,Right now:

[0198] (15)

[0199] According to formula (15), from a probability perspective, parameter w can make most cases satisfy both monotonicity constraints and value constraints. Regardless of whether the positive or negative value of Why is the value of , which can satisfy the monotonicity constraint and value constraint. Therefore, there is no need to assume If it is large enough, the monotonicity constraint and the value constraint can be satisfied at the same time as long as the parameter w satisfies formula (15).

[0200] In this embodiment, the "smile theory" is introduced to improve the resistance of the conditional diffusion model to systematic errors, effectively reduce the introduction of human errors in data preprocessing, and optimize the data processing process without significantly increasing costs. This framework has a solid theoretical foundation and certain applicability. It provides reliable technical support for high-precision analysis in multiple fields such as computer vision, natural language processing, and time series data prediction, thereby promoting the reliability and efficiency of data-driven decision-making, and has important practical value and broad application prospects.

[0201] For better explanation, the following experiment is performed in this embodiment:

[0202] 1. Verification on the MNIST handwritten digit set.

[0203] When discussing the deterministic problem of the balance factor n in the "Smile Theory", there is a challenge: the exact value of this factor cannot be directly derived in theory. This is because the interaction between the value constraint and the monotonicity constraint constitutes a complex balance point, and its precise position is difficult to grasp. In order to overcome this problem, this embodiment uses the MNIST dataset to conduct an exploratory analysis of the possible value range of the balance factor n through experimental methods. There are two reasons for choosing the MNIST dataset: first, the relative simplicity of the dataset helps to focus on the impact of the balance factor n; second, by conducting experiments under different system error conditions, the generalization ability of the discovered balance factor n can be enhanced, and its applicability can be comprehensively evaluated.

[0204] This embodiment further introduces the concept of systematic error, which simulates the error that may occur in the input data during the neural network training process. The systematic error is introduced in the generation stage of the back propagation process to simulate its impact on image generation. In order to verify the generalization of the balance factor n, this embodiment tests the performance of different parameters w at four different system error values ​​(0.3, -0.3, +3 and -3). Through these experiments, it is intended to explore whether the value range of the balance factor n is consistent under different error levels, thereby indirectly verifying the rationality of the "smile theory". The experimental results are obtained through Figures 5 to 8 was demonstrated. Figures 5 to 8 This provides some initial insights into the range of values ​​for the balance factor n. Figure 5 to Figure 6 The parameters w involved are 0.0, 0.5, 5.0, 50.0 and 500.0, and Figures 7 and 8 The parameter w involved is 5, 50, 500 and 5000. Figures 5 to 8 The 'first round' and 'tenth round' in the figure represent the results generated by the reverse process of the neural network after completing a single training iteration and ten training iterations, respectively.

[0205] In a series of experiments using the MNIST handwriting dataset, the systematic error was explored. The influence of the selection of parameter w. The experimental results show that under different system error levels, in order to achieve the optimal model performance, the parameter w needs to be adjusted accordingly. Specifically, when the system error When the value is plus or minus 0.3, the optimal value of parameter w should be set to 50; When the absolute value of increases to 3, the parameter w needs to be adjusted to 5000 to adapt to a higher error level.

[0206] Further analysis shows that the systematic error When it is -3, the image quality generated by the model is not as good as The results are shown in Figure 2 when the value of the balance factor n is 3. This phenomenon may be related to the numerical range of the image pixels after normalization, and it also suggests that when the system error is too large, the conditional diffusion model may exceed the range that its robustness can adapt to. Despite this difference, by comparing the experimental results under different system error conditions, it can be reasonably inferred that the appropriate setting value of the balance factor n is about 10.

[0207] Taking all factors into consideration, in order to maintain the robustness and generation quality of the model under different system error conditions, the parameter w can be adjusted according to the specific value of the system error, while the value of the balance factor n is maintained at around 10. This parameter selection strategy is designed to provide the conditional diffusion model with the necessary flexibility to adapt to various system error conditions and ensure high-quality image generation in a wide range of application scenarios.

[0208] 2. Application of CIFAR-10 dataset.

[0209] In order to verify the conclusion of setting the balance factor n to 10 and evaluate its universality on more complex datasets, this embodiment takes further experimental steps. First, it is recognized that the simplicity of the MNIST handwriting dataset may limit the wide applicability of the conclusion. In addition, the previous verification method based on sensory judgment lacks scientific quantitative indicators. To overcome these limitations, the CIFAR-10 dataset is introduced for experimentation, which is a more complex and diverse image dataset than MNIST.

[0210] In the new experimental setting, Fréchet Inception Distance (FID) is used as a quantitative indicator to scientifically evaluate the quality of generated images. FID is a distance metric based on feature space, which evaluates the performance of the generative model by comparing the statistical differences in feature representation between generated images and real images. The lower the FID value, the closer the distribution of the generated image is to the real image in the feature space, and thus the higher the generation quality.

[0211] The calculation of FID is based on the mean and covariance matrix of the generated image and the real image in the feature space. Specifically, the calculation formula of FID is as follows:

[0212] (16)

[0213] in, represents the trace of the matrix, which is the sum of all diagonal elements. and represents the real image and the generated image, and Respectively and The covariance matrix of .

[0214] The FID evaluation on the CIFAR-10 dataset aims to verify and expand the conclusions drawn on the MNIST dataset, thereby providing a more rigorous and universal basis for setting the balance factor n. This cross-dataset quantitative verification method not only enhances the scientific nature of the conclusions, but also helps to more comprehensively understand the performance and applicability of the model on different types of datasets. The experimental results are shown in Fig. 9 As shown in Table 1, Fig. 9 (a) where w=0, Fig. 9 (b) w=5, Fig. 9 (c) where w=10, Fig. 9 In (d), w=20.

[0215] Table 1 Quantitative indicators of CIFAR-10 dataset when the system error is 0.03

[0216]

[0217] exist Fig. 9 In the paper, the effects of different parameter w settings on the generated image set in the reverse process after a neural network model has been trained for 2000 rounds are shown. The image set contains 400 images of 10 different categories. The experimental results show that the increase of parameter w within a certain range can significantly improve the authenticity of the generated images. However, when the parameter w exceeds a certain threshold, the so-called "overload" phenomenon occurs, that is, the model's robustness is excessive, causing the denoising process to turn into a denoising process, which ultimately affects the quality of the images generated by the conditional diffusion model. In addition, Table 1 provides quantitative data, which further confirms the optimal range of parameter w and the negative impact on the authenticity of the generated images after exceeding this range. It is worth noting that the experimental results also indicate that the appropriate setting of the balance factor n is around 10, which is consistent with the previous conclusion based on the MNIST dataset.

[0218] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A robustness optimization method for a conditional diffusion model, characterized in that: The method comprises: Adding noise to an input signal to obtain a noise-added information data set, wherein the input signal is image data; Construct a conditional neural network and an unconditional neural network, wherein the conditional neural network has input conditions The unconditional neural network is a neural network without input conditions. Neural network; Inputting the noise-added information data set and the preset conditions into the conditional neural network for training to obtain a trained conditional neural network, and inputting the noise-added information data set into the unconditional neural network for training to obtain a trained unconditional neural network; Obtaining fitting errors generated by the trained conditional neural network and the trained unconditional neural network during the training process, and obtaining quantization errors generated by the trained conditional neural network when an input condition is wrong during the reverse process of the conditional diffusion model; Inputting the fitting error and the quantization error into the constructed filter to calculate the target parameter; Inputting the trained conditional neural network, the trained unconditional neural network and the target parameter into a constructed controller to generate a control signal; The Gaussian distribution and the control signal are input into the constructed generation model to obtain the generation data output by the generation model.

2. The robustness optimization method of the conditional diffusion model according to claim 1, characterized in that: The constructed filter is obtained as follows: According to the first training function corresponding to the trained conditional neural network, a first fitting error generated by the trained conditional neural network during the training process is obtained; and according to the second training function corresponding to the trained unconditional neural network, a second fitting error generated by the trained unconditional neural network during the training process is obtained; Converting the constructed controller into a distribution form including the quantization error, the first fitting error and the second fitting error; The distribution form is divided into a distribution of expected output and a distribution actually generated; Calculating the intersection area between the distribution of the expected output and the distribution actually generated; Constructing a first function corresponding to half of the intersection area; According to the first function, determining a function monotonicity constraint; Constructing a second function corresponding to all the intersection areas; Determining a function value constraint according to the second function; By balancing the function monotonicity constraint and the function value constraint, a constructed filter is obtained.

3. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The step of obtaining a first fitting error generated by the trained conditional neural network during the training process according to a first training function corresponding to the trained conditional neural network includes: Construct the first training function corresponding to the trained conditional neural network: ； The trained conditional neural network is trained according to the first training function to obtain a first fitting error generated by the trained conditional neural network during the training process: ； in, represents the input signal, Indicates the input conditions. express and A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained conditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the conditional neural network in the forward process, represents the first fitting error.

4. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The step of obtaining a second fitting error generated by the trained unconditional neural network during the training process according to a second training function corresponding to the trained unconditional neural network includes: Construct a second training function corresponding to the trained unconditional neural network: ； The trained unconditional neural network is trained according to the second training function to obtain a second fitting error generated by the trained unconditional neural network during the training process: ； in, represents the input signal, express A random sampling of indicates that the random vector follows a Gaussian distribution with a zero mean and an identity covariance matrix. represents Gaussian noise, Indicates that it has parameters The trained unconditional neural network, and represents the hyperparameter, and Respectively At different times The cumulative sum and At different times The cumulative sum of Indicates different moments The signal after adding Gaussian noise, represents the Gaussian noise added by the unconditional neural network in the forward process, represents the second fitting error.

5. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: Get the constructed controller as follows: ； in, represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates the input conditions, Indicates different moments The signal after adding Gaussian noise, Represents a controller.

6. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The step of converting the constructed controller into a distribution form including the quantization error, the first fitting error and the second fitting error comprises: Let the controller be , converting the constructed controller into a distribution form including the quantization error, the first fitting error and the second fitting error in the following manner: ； in, represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, Indicates that the covariance matrix is ​​the identity matrix.

7. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The constructing of a first function corresponding to half of the intersection area comprises: According to the properties of Gaussian distribution, the first function in different situations is constructed as follows: ； The integral probability density function solution method of Gaussian distribution is used to solve the first function in different situations to obtain the first function corresponding to half of the intersection area: ； in, represents the intersection of the distribution of the expected output and the distribution of the actual generation value, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, represents the standard deviation, A first function corresponding to half of the intersection area is represented.

8. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The constructing of the second function corresponding to all the intersection areas comprises: ； in, represents the second function, , represents the target parameter, represents the first fitting error, represents the second fitting error, represents the quantization error, represents the intersection of the distribution of the expected output and the distribution of the actual generation value, Represents standard deviation.

9. The robustness optimization method of the conditional diffusion model according to claim 2, characterized in that: The constructed filter is obtained as follows: ； in, represents the target parameter, represents the balance factor, represents the quantization error, represents the fitting error.

10. The robustness optimization method of the conditional diffusion model according to claim 1, characterized in that: The constructed generative model is obtained in the following way: ； in, represents the output result of the constructed generative model, represents the hyperparameter, express At different times The cumulative sum of represents the target parameter, Indicates that it has parameters The trained conditional neural network, Indicates that it has parameters The trained unconditional neural network, Indicates different moments The signal after adding Gaussian noise, Indicates the input conditions. represents a noise parameter, Represents a Gaussian distribution.

Images