Method for continuously sampling image
By introducing conventional distillation and semi-process noise distillation in diffusion model training, the problem that the diffusion model is prone to forget old task knowledge when learning new tasks, and the model's knowledge memory and training time are shortened.
Patent Information
- Application Number
- CN202411434797.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-11
AI Technical Summary
When diffusion models learn domain distribution from new task datasets, they tend to forget the previous knowledge of the old task dataset, resulting in catastrophic forgetting problems.
By setting up routine distillation and semi-process noise distillation, the diffusion model is trained multiple times, and the memory information of all tasks before semi-process noise distillation is used to prevent the model from being forgotten.
Effectively solve the problem of catastrophic forgetting, retain the integrity of the learning distribution, and significantly reduce training time.
Smart Images
Figure CN119418147A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for continuously sampling images, and belongs to the technical field of picture processing. Background Art
[0002] Generative diffusion models have achieved remarkable success in the fields of image, audio, and video generation. However, they face a key challenge: when learning domain distributions from new task datasets, they tend to forget prior knowledge of old task datasets. For example, after adapting to anime-style image datasets, the model will have difficulty generating images in a normal style. Similarly, if the model is trained on 512×512 images and then trained on 1024×1024 images, it will only be able to maintain image generation of 1024×1024 sampling, and it will be difficult to maintain the high fidelity of 512×512 sampling images. The above phenomenon is named catastrophic forgetting of diffusion models.
[0003] Therefore, it is necessary to conduct more in-depth research to solve the catastrophic forgetting problem of diffusion models. Summary of the invention
[0004] In order to overcome the above problems, the inventors conducted in-depth research and designed a method for continuous sampling of images, including:
[0005] S1. Set up the diffusion model;
[0006] S2. The diffusion model is trained multiple times, and conventional distillation and semi-process noise distillation are set during the training process, where the semi-process distillation is used to learn the memory information of all previous tasks;
[0007] S3. The trained diffusion model outputs a forgetting-free image.
[0008] In a preferred embodiment, in the semi-process noise distillation, the following sub-steps are included:
[0009] S21, obtain the diffusion model of the previous task;
[0010] S22, obtaining the noise image at the previous moment according to the diffusion model of the previous task;
[0011] S23, based on the noise image at the previous moment, using the diffusion model of the previous task to output a first predicted noise;
[0012] Based on the noise image at the previous moment, a diffusion model of the current task is used to output a second predicted noise;
[0013] S24, training multiple times to make the first prediction noise and the second prediction noise loss as small as possible, so as to update the diffusion model parameters of the current task.
[0014] In a preferred embodiment, in S22, by t Inference is performed multiple times to obtain the noise image at the previous moment.
[0015] In a preferred embodiment, the reasoning is gradient-free reasoning, expressed as:
[0016]
[0017] Among them, ∈ n-1 () represents the diffusion model of the previous task, x t+1 represents the noise image generated by the diffusion model of the previous task at time t+1, x t represents the noise image generated by the diffusion model of the previous task at time t, α t Represents the settable parameters of the diffusion model.
[0018] In a preferred embodiment, in S24, the training loss function L SND It is expressed as:
[0019] L SND =ω(t)||∈ n (x t , c k , t)-∈ n-1 (x t , c k , t)|| 2
[0020] Wherein, ω(t) represents the factor of the semi-process noise distillation loss, which is a function of the variable T. The specific form can be freely set by those skilled in the art based on experience, c k Represents a random category in the previous task.
[0021] In a preferred embodiment, in S2, conventional distillation and semi-process noise distillation are performed alternately during the training process.
[0022] In a preferred embodiment, during conventional distillation, the loss function L FT Set to:
[0023] L FT =ω(t)||∈ n (α t x+σ t ∈, c n , t)-∈|| 2
[0024] Among them, ∈ represents standard Gaussian noise, x is the image in the current task training set, c n Indicates the category of the current task.
[0025] In a preferred embodiment, in S1, the number of parameters of the diffusion model is further expanded, including the following steps:
[0026] Expand the feedforward network of the extended model in parallel and add multiple feedforward networks;
[0027] Set up task guidance to assign image patches of different tasks to different feed-forward networks.
[0028] The present invention also provides an electronic device, comprising:
[0029] at least one processor; and
[0030] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.
[0031] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any of the above-mentioned methods.
[0032] The beneficial effects of the present invention include:
[0033] (1) Effectively solve the problem of catastrophic forgetting;
[0034] (2) It uniquely addresses the dual needs of class-conditional and non-class-conditional diffusion models;
[0035] (3) It not only preserves the integrity of the learned distribution, but also significantly reduces the training time. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A schematic flow chart of a method for continuously sampling images according to a preferred embodiment of the present invention is shown;
[0037] Figure 2 The comparison results of Example 1 and Comparative Examples 1-4 on the TinyImageNet dataset are shown;
[0038] Figure 3 The comparison results of Example 1 and Comparative Examples 1-4 on the CIFAR-10 dataset are shown. DETAILED DESCRIPTION
[0039] The present invention will be further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.
[0040] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.
[0041] According to a method for continuously sampling images provided by the present invention, Figure 1 As shown, including:
[0042] S1. Set up the diffusion model;
[0043] S2. The diffusion model is trained multiple times, and conventional distillation and semi-process noise distillation are set during the training process, where the semi-process distillation is used to learn the memory information of all previous tasks;
[0044] S3. The trained diffusion model outputs a forgetting-free image.
[0045] In the present invention, a semi-process noise distillation is set to prevent catastrophic forgetting of the diffusion model.
[0046] In S1, the specific structure of the diffusion model is not limited, and technicians in this field can freely choose from existing diffusion models.
[0047] In S2, both the conventional distillation and the semi-process noise distillation are for training the diffusion model and updating the parameters of the diffusion model.
[0048] When training traditional diffusion models, only conventional distillation methods are used, such as progressive distillation and fine-tuning methods.
[0049] However, conventional distillation methods will cause the diffusion model to forget how to generate images in the previous dataset after training on the new dataset, which is catastrophic forgetting. In the present invention, a semi-process noise distillation is set to solve the problem of catastrophic forgetting.
[0050] According to the present invention, in the semi-process noise distillation, the following sub-steps are included:
[0051] S21, obtain the diffusion model of the previous task;
[0052] S22, obtaining the noise image at the previous moment according to the diffusion model of the previous task;
[0053] S23, based on the noise image at the previous moment, using the diffusion model of the previous task to output a first predicted noise;
[0054] Based on the noise image at the previous moment, a diffusion model of the current task is used to output a second predicted noise;
[0055] S24, training multiple times to make the first prediction noise and the second prediction noise loss as small as possible, so as to update the diffusion model parameters of the current task.
[0056] During the training process, the traditional diffusion model does not retain the diffusion model of the previous task, but directly updates the parameters based on the diffusion model of the previous task. In step S21 of the present invention, the diffusion model of the previous task is retained and the noise image is generated using the model.
[0057] In the traditional diffusion model training process, the diffusion model is trained with complete images, such as directly using sample images, or using images generated by the diffusion model of the previous task. However, this training cannot guide the latent space trajectory of the diffusion model well, causing the diffusion model to forget. In the present invention, in S22-S24, the diffusion model is trained with noise images. These noise images simulate the intermediate steps in the data generation process, rather than the final clear images. In the present invention, they can be called semi-process images. These noise images are input into the diffusion model of the current task, so that the model learns how to restore clear images from these noise images. In the present invention, it can be called semi-process denoising, thereby better guiding the latent space trajectory of the diffusion model.
[0058] According to the present invention, in S22, by T Inference is performed multiple times to obtain the noise image at the previous moment.
[0059] The reasoning is gradient-free reasoning, expressed as:
[0060]
[0061] Among them, ∈ n-1 () represents the diffusion model of the previous task, x t+1 represents the noise image generated by the diffusion model of the previous task at time t+1, x t represents the noise image generated by the diffusion model of the previous task at time t, α t The settable parameter representing the diffusion model is a function of the variable t, and the specific form can be freely set by those skilled in the art based on experience.
[0062] Through multiple gradient-free inferences, with Gaussian noise x T is the noise image at the last moment, and forward infers the noise image x generated by the diffusion model of the previous task at the previous time t t .
[0063] The above reasoning process does not require the gradient information of the calculation process, which greatly reduces the consumption of video memory.
[0064] Furthermore, the value of the specific previous moment t can be freely selected by those skilled in the art based on experience. For example, when the last moment is the moment of 999 training steps, the previous moment t can be selected as the moment of 500 training steps.
[0065] In S23, the noise image at the previous time t is input into the diffusion model ∈ of the previous task n-1 In (), the first predicted noise is output by the diffusion model of the previous task.
[0066] Furthermore, using the noise image x at the same time t , input to the diffusion model of the current task ∈ n In (), the second prediction noise is output by the diffusion model of the current task.
[0067] In S24, the training loss function L SND It can be expressed as:
[0068] L SND =ω(t)||∈ n (x t , c k , t)-∈ n-1 (x t , c k , t)|| 2
[0069] Wherein, ω(t) represents the factor of the semi-process noise distillation loss, which is a function of the variable T. The specific form can be freely set by those skilled in the art based on experience, c k Represents a random category in the previous task.
[0070] Furthermore, in S2, conventional distillation and semi-process noise distillation are performed alternately during training.
[0071] Preferably, a half-process noise distillation is performed after multiple conventional distillations during the training process, for example, a half-process noise distillation is performed after every 10 conventional distillations, so as to save training costs.
[0072] In a preferred embodiment, during conventional distillation, the loss function L FT Set to:
[0073] L FT =ω(t)||∈ n (α t x+σ t ∈, c n , t)-∈|| 2
[0074] Among them, ∈ represents standard Gaussian noise, x is the image in the current task training set, c nIndicates the category of the current task.
[0075] According to the present invention, during the training process, the semi-process noise distillation can support both conditional guidance and unconditional guidance. Conditional guidance and unconditional guidance are two commonly used training types in diffusion model training.
[0076] Under conditional guided semi-process noise distillation, we randomly sample category information from the previous task and perform gradient-independent semi-process denoising on the diffusion model of the previous task.
[0077] Under the unconditional guided semi-process noise distillation, the training loss function L SND，uncond It can be expressed as:
[0078] L SND，uncond =ω(t)||∈ n (x t , t)-∈ n-1 (x t , t)|| 2
[0079] This is done by simply omitting the category information.
[0080] In a preferred embodiment, the number of parameters of the diffusion model is also expanded to increase the memory capacity of the diffusion model, reduce training time, and improve recognition accuracy.
[0081] The extended models all have a feed-forward network (FFN). In the present invention, the expansion of the number of parameters of the diffusion model includes the following steps:
[0082] Expand the feedforward network of the extended model in parallel and add multiple feedforward networks;
[0083] Set up task guidance to assign image patches of different tasks to different feed-forward networks.
[0084] By expanding the feedforward network of the extended model, the memory capacity of the extended model can be significantly increased;
[0085] Through parallel expansion, it can be ensured that the training cost and inference cost will not change. This is because when the extended model processes an image, it only selects one feedforward network to participate in the calculation, so the training cost and inference cost will not change.
[0086] Furthermore, although the number of feedforward networks is increased, in actual training and inference, the extended model may only use one of the feedforward networks. For example, the extended model only uses the first feedforward network in each recognition, that is, the extended model converges to one feedforward network. Then, although the other extended feedforward networks have parameters, they have no practical effect.
[0087] In the present invention, the above problem is solved by setting task guidance. Preferably, the image blocks of different tasks are assigned to different feedforward networks by setting a task guidance routing network.
[0088] More preferably, a task-guided routing network is provided, wherein the task-guided routing network is a fully connected layer provided in each block of the diffusion model, which receives task information and determines a feedforward network to be used according to image information in the block.
[0089] More preferably, the task number is encoded into the embedding so that the task directed routing network can receive the task information.
[0090] In a preferred embodiment, the output of the fully connected layer of the task-guided routing network can be expressed as:
[0091] h(x,n)=W r (x+e(n))
[0092] Where n represents the task number, e(n) represents embedding, x represents the image block, h(x, n) represents the task-guided routing network output with the image block and task number as variables, and W r Represents the task-guided routing network parameters.
[0093] Furthermore, the usage probability of different feedforward networks is obtained according to the output of the fully connected layer of the task-guided routing network, which is expressed as:
[0094]
[0095] Among them, p i (x, n) represents the probability of the i-th feedforward network being used, h(x, n) i represents the usage probability of the feedforward network corresponding to the block where the fully connected layer of the task-guided routing network is located, h(x, n) j represents the usage probability of other feedforward networks except the i-th feedforward network, and M represents the total number of feedforward networks.
[0096] The feedforward network used in the final extended model is a linear weighted combination of image patches and routing network probabilities, expressed as:
[0097]
[0098] Among them, E i (x) represents the result after the i-th feedforward network encoding, and y represents the final weighted sum.
[0099] Example
[0100] Example 1
[0101] Sampling image experiments are conducted in the training datasets TinyImageNet and CIFAR-10. Both TinyImageNet and CIFAR-10 are public datasets, where TinyImageNet contains 200 categories for 10 tasks and CIFAR-10 contains 10 categories for 5 tasks.
[0102] The sampled image includes:
[0103] S1. Set up the diffusion model;
[0104] S2. The diffusion model is trained multiple times, and conventional distillation and semi-process noise distillation are set during the training process, where the semi-process distillation is used to learn the memory information of all previous tasks;
[0105] S3. The trained diffusion model outputs a forgetting-free image.
[0106] In the semi-process noise distillation, the following sub-steps are included:
[0107] S21, obtain the diffusion model of the previous task;
[0108] S22, obtaining the noise image at the previous moment according to the diffusion model of the previous task;
[0109] S23, based on the noise image at the previous moment, using the diffusion model of the previous task to output a first predicted noise;
[0110] Based on the noise image at the previous moment, a diffusion model of the current task is used to output a second predicted noise;
[0111] S24, training multiple times to make the first prediction noise and the second prediction noise loss as small as possible, so as to update the diffusion model parameters of the current task.
[0112] In S22, by using Gaussian noise x T The noise image at the previous moment is obtained by performing reasoning multiple times. The reasoning is gradient-free reasoning, which is expressed as:
[0113]
[0114] In S24, the training loss function is:
[0115] L SND =ω(t)||∈ n (x t , c k , t)-∈ n-1 (x t , c k , t)|| 2
[0116] During the training process, regular distillation and semi-process noise distillation are performed alternately, and semi-process noise distillation is performed once after every 10 regular distillations.
[0117] In the conventional distillation process, the loss function L FT Set to:
[0118] L FT =ω(t)||∈ n (α t x+σ t ∈, c n , t)-∈|| 2 .
[0119] In S1, the number of parameters of the diffusion model is also expanded, including the following steps:
[0120] Expand the feedforward network of the extended model in parallel and add multiple feedforward networks;
[0121] Set up task guidance to assign image patches of different tasks to different feed-forward networks.
[0122] Among them, the image blocks of different tasks are assigned to different feedforward networks by setting a task-guided routing network, and the task-guided routing network is a fully connected layer set in each block of the diffusion model. The output of the task-guided routing network fully connected layer is expressed as:
[0123] h(x,n)=W r (x+e(n))
[0124] The usage probability of different feedforward networks is obtained according to the output of the fully connected layer of the task-guided routing network, which is expressed as:
[0125]
[0126] The feedforward network used in the final extended model is a linear weighted combination of image patches and routing network probabilities, expressed as:
[0127]
[0128] During the experiment, the extended model was trained on TinyImageNet, which contains category information, and the training on CIFAR-10 was performed without category information. For the TinyImageNet dataset, four categories “22, bald eagle”, “119 rock crab”, “247 saint bernard” and “397 globefish” were selected, where globefish corresponds to the last task and the rest of the categories belong to the previous tasks.
[0129] Comparative Example 1
[0130] The same experiment as in Example 1 was performed, except that the LWF method was used. The specific process of the LWF method can be found in the literature Z. Li and D. Hoiem, "Learning without forgetting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2935-2947, 2017.
[0131] Comparative Example 2
[0132] The same experiment as in Example 1 was performed, except that the generative replay method was used. For the specific process of the generative replay method, see S. Masip, P. Rodriguez, T. Tuytelaars, and GM van de Ven, "Continual learning of diffusion models with generative distillation," arXiv preprint arXiv: 2311.14028, 2023.
[0133] Comparative Example 3
[0134] The same experiment as in Example 1 is performed, except that the traditional fine-tuning method is used, that is, the new task training expansion model is used sequentially.
[0135] Comparative Example 4
[0136] The same experiment as in Example 1 is performed, except that joint training is performed, that is, all task data are used to train the diffusion model simultaneously.
[0137] Comparing the results of Example 1 with Comparative Examples 1-4, Fréchet Inception Distance (FID) and Kullback-Leibler Divergence (KLD) were used as evaluation indicators, where Fréchet Inception Distance (FID) is a statistical measure that quantifies the difference between two image distributions by considering the distance between their respective feature representations captured by the Inception model; Kullback-Leibler Divergence (KLD) is a measure that quantifies the difference between two probability distributions from information theory. The results are shown in FIG. Figure 2 , 3 As shown in Table 1, Figure 2This is the comparison result on the TinyImageNet dataset. Figure 3 Table 1 shows the comparison results on the CIFAR-10 dataset. Table 2 shows the comparison results after all tasks are trained.
[0138] Table 1
[0139]
[0140] from Figure 2 , 3 It can be seen that as the tasks change, the average FID of Example 1 is better than that of Comparative Examples 1 to 3. It is worth noting that under the same computing resources, the quality of the replayed image produced in Comparative Example 2 is poor, resulting in its average FID being significantly lower than that of the other comparative examples and Example 1.
[0141] Comparative Example 4 adopts a joint training method. Since all task data are used to train the diffusion model at the same time, there is no forgetting problem, but the training time is long. Example 1 significantly reduces the training time. After all task training is completed, compared with Comparative Examples 1-3, the results of Example 1 in FID and KLD indicators are close to joint training, as shown in Table 1.
[0142] Various embodiments of the method described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0143] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0144] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0145] To provide interaction with a user, the methods and apparatus described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0146] The methods and apparatus described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0147] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.
Claims
1. A method for continuously sampling images, characterized in that: include: S1. Set up the diffusion model; S2. The diffusion model is trained multiple times, and conventional distillation and semi-process noise distillation are set during the training process, where the semi-process distillation is used to learn the memory information of all previous tasks; S3. The trained diffusion model outputs a forgetting-free image.
2. The method for continuously sampling images according to claim 1, characterized in that: In the semi-process noise distillation, the following sub-steps are included: S21, obtain the diffusion model of the previous task; S22, obtaining the noise image at the previous moment according to the diffusion model of the previous task; S23, based on the noise image at the previous moment, using the diffusion model of the previous task to output a first predicted noise; Based on the noise image at the previous moment, a diffusion model of the current task is used to output a second predicted noise; S24, training multiple times to make the first prediction noise and the second prediction noise loss as small as possible, so as to update the diffusion model parameters of the current task.
3. The method for continuously sampling images according to claim 2, characterized in that: In S22, by using Gaussian noise x T Inference is performed multiple times to obtain the noise image at the previous moment.
4. The method for continuously sampling images according to claim 3, characterized in that: The reasoning is gradient-free reasoning, expressed as: Among them, ∈ n-1 () represents the diffusion model of the previous task, x t+1 represents the noise image generated by the diffusion model of the previous task at time t+1, x t represents the noise image generated by the diffusion model of the previous task at time t, α t Represents the settable parameters of the diffusion model.
5. The method for continuously sampling images according to claim 2, characterized in that: In S24, the training loss function L SND It is expressed as: L SND =ω(t)||∈ n (x t ,c k ,t)-∈ n-1 (x t ,c k ,t)|| 2 Wherein, ω(t) represents the factor of the semi-process noise distillation loss, which is a function of the variable t. The specific form can be freely set by those skilled in the art based on experience. c k Represents a random category in the previous task.
6. The method for continuously sampling images according to claim 1, characterized in that: In S2, regular distillation and semi-process noise distillation are performed alternately during training.
7. The method for continuously sampling images according to claim 1, characterized in that: In the conventional distillation process, the loss function L FT Set to: L FT =ω(t)||∈ n (a t x+s t ∈,c n ,t)-∈|| 2 Among them, ∈ represents standard Gaussian noise, x is the image in the current task training set, c n Indicates the category of the current task.
8. The method for continuously sampling images according to claim 1, characterized in that: In S1, the number of parameters of the diffusion model is also expanded, including the following steps: Expand the feedforward network of the extended model in parallel and add multiple feedforward networks; Set up task guidance to assign image patches of different tasks to different feed-forward networks.
9. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-8.
10. A computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.