Method, resource generation method and apparatus for training a generative model

Abstract

Embodiments of the present application disclose a method and device for training a generative model and a resource generation method. When the generative model is trained by using a denoising diffusion mechanism, a fixed linear function is no longer used to determine the noise process of each time step, but a brand-new idea is adopted, that is, the noise process corresponding to each subsequent diffusion step is determined according to the value of a first loss function corresponding to the inverse diffusion process of each time step, wherein the first loss function represents the difference between the sample distribution generated by the noise diffusion process and the sample distribution generated by the inverse diffusion process. This adaptive determination of the noise process according to the value of the first loss function can effectively improve the resource generation capability of the generative model, thereby improving the model effect.

Description

Technical Field

[0001] This application relates to the fields of natural language processing and artificial intelligence, and in particular to a method for training a generative model, a resource generation method, and an apparatus. Background Technology

[0002] Denoising diffusion probabilistic models have been widely applied to image generation, where a denoising probability model is used to train a generative model for image generation. The denoising diffusion probability model adds noise to the real sample at each time step by constructing a forward diffusion process, and removes noise at each time step by learning a backward diffusion process to obtain the real sample. In the denoising diffusion process, the proportion of noise added to the real sample at each time step is called the noise progression. Different noise progressions have different impacts on the sample quality and image generation capability of the denoising diffusion model. Currently, most models use a fixed linear function to determine the noise progression at each time step, which results in poor performance of the generative model. Summary of the Invention

[0003] In view of this, this application provides a method for training a generative model, a resource generation method, and an apparatus to improve the effect of the generative model in generating output resources.

[0004] This application provides the following solution:

[0005] Firstly, a method for training a generative model is provided, the method comprising:

[0006] Acquire training data containing multiple training samples, wherein the training samples include sample pairs consisting of input samples and output samples, and the input samples and output samples are images or text;

[0007] The output samples in the sample pair are subjected to noise-adding and diffusion processing to obtain the noisy samples;

[0008] The input sample and the noisy sample are input into the generative model. During the training process, the generative model simulates the inverse diffusion process of the noisy diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output.

[0009] In the noise-adding diffusion process, each time step of the diffusion process adds noise to the sample obtained from the diffusion process of the previous time step. The noise progression corresponding to each time step of the diffusion process is related to the value of the first loss function corresponding to each time step of the reverse diffusion process. The noise progression corresponding to each time step of the diffusion process is the proportion of noise added to the output sample by the diffusion process of each time step. The first loss function characterizes the difference between the sample distribution generated by the noise-adding diffusion process and the sample distribution generated by the reverse diffusion process.

[0010] According to one achievable method in an embodiment of this application, the method further includes:

[0011] By using the values ​​of the first loss function corresponding to the inverse diffusion processing at each time step that have been sampled and the noise process at each time step, the mapping relationship between the values ​​of the first loss function and the noise process is determined.

[0012] Based on the mapping relationship, the noise processes corresponding to the first loss function values ​​that are uniformly distributed at each time step are determined as the noise processes corresponding to the diffusion processing at each subsequent time step.

[0013] According to one achievable method in the embodiments of this application, the noise process corresponding to the diffusion process at each time step is determined for each element Token in the output sample. The noise process corresponding to the diffusion process of the i-th Token at each time step is related to the value of the first loss function corresponding to the reverse diffusion process of the i-th Token at each time step.

[0014] According to one achievable method in an embodiment of this application, in the reverse diffusion process, the samples obtained from the reverse diffusion process at each time step are obtained by upsampling the posterior distribution of the samples obtained from the reverse diffusion process at the previous time step; or...

[0015] The samples obtained from the backdiffusion processing at each time step are obtained by sampling using a first method according to a first preset probability and a second method according to a second preset probability. The first method involves sampling from the posterior distribution of the samples obtained from the backdiffusion processing at the previous time step, and the second method involves sampling from the posterior distribution of the samples obtained from the prediction. Upsampling of the prior distribution, the This is the prediction result for the sample obtained from the first step of diffusion processing.

[0016] According to one achievable method in an embodiment of this application, the generative model includes an encoder and a decoder;

[0017] The encoder acquires the feature representation of the input sample of the input generation model, and the decoder uses the feature representation of the input sample and the noisy sample to perform the inverse diffusion process to obtain the output sample;

[0018] The training objective includes minimizing the value of the first loss function.

[0019] According to one achievable method in the embodiments of this application, if both the input sample and the output sample are text, then the output sample in the sample pair is subjected to noise-adding and diffusion processing to obtain the noisy sample, including:

[0020] The feature representation of the output sample is obtained, and the feature representation of the output sample is subjected to noise diffusion processing to obtain the noise-added feature representation as the noise-added sample.

[0021] According to one achievable method in the embodiments of this application, the training objective further includes: minimizing the value of the second loss function and / or the value of the third loss function;

[0022] The second loss function characterizes the difference between the sample distribution obtained by diffusion processing in the last time step and the normal distribution; the third loss function characterizes the difference between the sample obtained by reverse diffusion in the last time step and the output sample.

[0023] Secondly, a resource generation method is provided, the method comprising:

[0024] Obtain input resources;

[0025] The input resources and random noise are input into the generation model, and the generation model performs inverse diffusion processing based on the input resources and random noise to obtain the output resources.

[0026] The input and output resources are images or text; the generation model is pre-trained using the method described in the first aspect above.

[0027] Thirdly, a method for generating summaries is provided, the method comprising:

[0028] Get the input text;

[0029] The input text and random noise are input into the generation model, which then performs inverse diffusion processing based on the input text and random noise to obtain a summary of the input text.

[0030] The generative model is pre-trained using the method described in the first aspect above.

[0031] Fourthly, a machine translation method is provided, the method comprising:

[0032] Retrieve text written in the first language;

[0033] The text in the first language and random noise are input into the generation model, and the generation model performs inverse diffusion processing based on the text in the first language and the random noise to obtain the text in the second language.

[0034] The generative model is pre-trained using the method described in the first aspect above.

[0035] Fifthly, an apparatus for training a generative model is provided, the apparatus comprising:

[0036] The sample acquisition unit is configured to acquire training data containing multiple training samples, wherein the training samples include sample pairs consisting of input samples and output samples, and the input samples and output samples are images or text.

[0037] The noise-adding and diffusion unit is configured to perform noise-adding and diffusion processing on the output samples in the sample pair to obtain the noise-added samples.

[0038] The model training unit is configured to input the input sample and the noisy sample into the sample pair to generate a model. During the training process, the generating model simulates the inverse diffusion process of the noisy diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output.

[0039] In the noise-adding diffusion process, the noise-adding diffusion unit adds noise to the sample obtained from the diffusion process of the previous time step at each time step. The noise process corresponding to the diffusion process at each time step is related to the value of the first loss function corresponding to the inverse diffusion process at each time step. The noise process corresponding to the diffusion process at each time step is the proportion of noise added to the output sample by the diffusion process at each time step. The first loss function characterizes the difference between the sample distribution generated by the noise-adding diffusion process and the sample distribution generated by the inverse diffusion process.

[0040] Sixthly, a resource generation apparatus is provided, the apparatus comprising:

[0041] The resource acquisition unit is configured to acquire input resources;

[0042] The resource generation unit is configured to input the input resources and random noise into the generation model, and the generation model performs inverse diffusion processing based on the input resources and random noise to obtain the output resources.

[0043] The input and output resources are images or text, and the generation model is pre-trained by the apparatus described in the fifth aspect above.

[0044] According to a seventh aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0045] According to the eighth aspect, an electronic device is provided, comprising:

[0046] One or more processors; and

[0047] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first aspects above.

[0048] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0049] 1) In training the generative model using a denoising diffusion mechanism, this application no longer uses a fixed linear function to determine the noise process at each time step. Instead, it adopts a novel approach: determining the noise process for each diffusion step based on the value of the first loss function corresponding to the reverse diffusion processing at each time step. This effectively controls the amount of noise added to the output samples during the denoising diffusion process, resulting in higher quality generated resources. Therefore, this adaptive method of determining the noise process based on the value of the first loss function can effectively improve the resource generation capability of the generative model, thereby improving the model's performance.

[0050] 2) This application introduces the diffusion probability generation mechanism into the field of text generation, and simulates the text generation process as a reverse diffusion process of noise diffusion, thereby eliminating the impact of information loss caused by noise on text generation and thus obtaining better text generation results.

[0051] 3) In the actual prediction process, the input and processing of the encoder remain unchanged. That is, the encoder still only needs to perform one neural network feedforward calculation, without having to participate in the inverse diffusion process, which may require hundreds or thousands of processing steps. Therefore, it can greatly save computing resources.

[0052] 4) In the reverse diffusion process of this application, a novel sampling method with a certain probability is incorporated based on the upsampling of the posterior distribution of the samples obtained from the diffusion process in the previous time step. This novel sampling method is based on the upsampling of the prior distribution of the predicted results of the samples obtained from the first diffusion process. This sampling method can effectively improve the diversity of samples generated during the reverse diffusion process under the aforementioned noise process determination method.

[0053] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0056] Figure 2 A flowchart illustrating the method for training and generating models provided in this application embodiment;

[0057] Figure 3 A schematic diagram illustrating the training principle of the generative model provided in this application embodiment;

[0058] Figure 4 A flowchart of the resource generation method provided in the embodiments of this application;

[0059] Figure 5 A schematic diagram illustrating the prediction principle of the generative model provided in this application embodiment;

[0060] Figure 6 A schematic block diagram of an apparatus for training and generating models provided in an embodiment of this application;

[0061] Figure 7 A schematic block diagram of a resource generation apparatus provided in the embodiments of this application;

[0062] Figure 8 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0064] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0065] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0066] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0067] To facilitate understanding of this application, a brief description of the system architecture to which this application applies will be given first. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture includes a model training device and a generation device.

[0068] In this embodiment, after acquiring training data in the offline stage, the model training device can use the method provided in this application to train the model and obtain the generated model.

[0069] The resource generation device utilizes an established generation model online to generate output resources based on input resources. In this embodiment, the input and output resources can be images or text. For example, if the input is text, the output is text. Similarly, if the input is an image, the output is an image. The generation model can be either an image generation model or a text generation model. Specific application scenarios will be described with examples in subsequent embodiments.

[0070] The model training device and resource generation device can be configured as independent servers, or they can be configured on the same server or server group, or they can be configured on a separate or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPs) services, such as high management difficulty and weak service scalability. The model training device and resource generation device can also be configured on computer terminals with strong computing capabilities.

[0071] It should be noted that, in addition to generating output resources online for input resources, the above-mentioned resource generation device can also generate output resources offline for input resources, such as generating output resources for batches of input resources.

[0072] It should be understood that Figure 1 The model training device, resource generation device, and number of generated models shown are merely illustrative. Depending on implementation needs, any number of model training devices, resource generation devices, and generated models can be included.

[0073] Figure 2 This is a flowchart of a method for training and generating a model provided in an embodiment of this application. This method can be implemented by... Figure 1 The model training device in the system shown is executed. For example... Figure 2 As shown, the method may include:

[0074] Step 202: Obtain training data containing multiple training samples. The training samples include sample pairs consisting of input samples and output samples, which can be images or text.

[0075] Step 204: Perform noise-adding and diffusion processing on the output samples of the sample pair to obtain the noise-added samples.

[0076] Step 206: Input the input sample and the noisy sample into the generative model. During training, the generative model simulates the inverse diffusion process of noise addition based on the input sample and the noisy sample, with the output sample as the target output. In the noise addition diffusion process, each time step adds noise to the sample obtained from the previous time step. The noise progression corresponding to each time step is related to the value of the first loss function corresponding to the inverse diffusion process at each time step. The noise progression corresponding to each time step is the proportion of noise added to the output sample by the diffusion process at each time step.

[0077] As can be seen from the above process, when training the generative model using the denoising diffusion mechanism, this application no longer uses a fixed linear function to determine the noise process at each time step. Instead, it adopts a novel approach: determining the noise process for each diffusion step based on the value of the first loss function corresponding to the inverse diffusion processing at each time step. This adaptive method of determining the noise process based on the value of the first loss function effectively improves the resource generation capability of the generative model, thereby enhancing the model's performance.

[0078] The following is a detailed description of each step in the above process. First, step 202, namely "obtaining training data containing multiple training samples", will be described in detail with reference to the embodiment.

[0079] The training data involved in this embodiment includes multiple training samples, each of which is an input sample W. x and output sample W y The sample pairs formed.

[0080] Depending on the application scenario, the input and output samples mentioned above will differ.

[0081] In image generation scenarios, for example, the input sample described above could be text, which could be a description of an image, and the output sample could be an image containing content consistent with the description in the input text. This type of sample pair can be obtained from things like illustrations in an article and descriptive statements about those illustrations. As another example, the input sample could be an image, and the output sample could be an image obtained after restoring the input image. And so on.

[0082] In text generation scenarios, for example, the input sample mentioned above can be text, and the output sample can also be text. The input sample could be an article, and the output sample could be a summary of that article. As another example, the input sample could be text in a first language, and the output text could be text in a second language corresponding to the input sample. And so on.

[0083] The following describes step 204, namely, "to perform noise-adding and diffusion processing on the output sample of the sample pair to obtain the noise-added sample," in conjunction with an embodiment.

[0084] The denoising diffusion probabilistic model has been applied in image generation, achieving results that surpass traditional generative adversarial models, but it remains a blank in the field of natural language processing. The denoising diffusion probabilistic model mainly includes two processes: forward noise diffusion and backward denoising. This step is the forward noise diffusion process, which is the process of gradually adding noise to the output sample.

[0085] In the field of image processing, we can first analyze the output sample W. y A multi-step diffusion process is applied. The first time step of the diffusion process is performed on the output sample W. y Adding noise yields z0. Subsequent diffusion processes at each time step add noise to the sample obtained from the previous time step's diffusion process. The samples obtained from the diffusion processes at each time step conform to a normal distribution; that is, the forward noise-adding diffusion process can be viewed as adding an additional Markov transformation distribution.

[0086] like Figure 3 As shown, the Markov transformation distribution in the diffusion process at the first time step can be defined as q(z0|W y ),For example:

[0087] q(z0|W y )=N(z0;W y,β0I) (1)

[0088] In subsequent diffusion steps, taking step t as an example, the Markov transition distribution can be defined as q(z). t |z t-1 ):

[0089] q(z t |z t-1 )=N(z t ;z t-1 ,β t I) (2)

[0090] Wherein, N(z0; W) y Let ,β0I) be a distribution with respect to z0, and this distribution is expressed as W y The distribution is a normal distribution with mean β0I and variance β0I. The diffusion processing at each time step described above uses β0,β... t These are preset parameters, where I is the identity matrix. t These are the samples obtained from the diffusion process at time step t+1 and time step t. After diffusion for a preset number of time steps (e.g., T+1 time steps), z is obtained. T , make z as much as possible T It approximates a normal distribution. The more diffusion steps there are, the better z... T The closer the distribution is to a normal distribution, the better the effect, but the more computing resources are consumed and the longer it takes. Therefore, it is necessary to choose a relatively balanced value based on experience or experiments, such as 2000 steps.

[0091] In the field of natural language processing, due to the discretized nature of natural language, denoising diffusion probability models cannot be directly applied to natural language generation tasks. In this embodiment, the output sample W can first be... y Encode the sequence containing each token (element) into W. y Mapped to continuous feature representation g φ (W y This refers to the feature representation composed of word vectors of each token. Here, each token in the text refers to an element that constitutes the text. For text, if it is segmented into a sequence of characters or words, then the characters or words in the text sequence, as well as the start character and separator, are all tokens. φ (W y )∈R n×d n and d are W y The length and dimension of the word vector.

[0092] The noise diffusion process is a multi-step diffusion process applied to the feature representation of the output sample. The first time step of the diffusion process applies the diffusion to the feature representation g of the output sample. φ (Wy Noise is added to obtain z0. Subsequent diffusion processes at each time step add noise to the feature representation obtained from the previous time step. In other words, each sample obtained at each time step is actually a feature representation. The noise added at each time step can be sampled from a normal distribution; that is, the forward noise-adding diffusion process can be viewed as adding an additional Markov transformation distribution.

[0093] In this case, the Markov transformation distribution in the diffusion process at the first time step can be defined as q. φ (z0|W y ),For example:

[0094] q φ (z0|W y )=N(z0;g φ (W y ),β0I) (3)

[0095] Wherein, N(z0;g) φ (W y Let ),β0I) be a distribution with respect to z0, and this distribution is expressed in terms of g φ (W y ) is a normal distribution with mean β0I and variance β0I.

[0096] In subsequent diffusion steps, taking step t as an example, the Markov transition distribution can be defined as q. φ (z t |z t-1 ):

[0097] q φ (z t |z t-1 )=N(z t ;z t-1 ,β t I) (4)

[0098] Through the aforementioned forward noise diffusion process, discrete text-type output samples are integrated into a continuous denoising diffusion probability model, gradually reaching g. φ (W y To obtain samples z that conform to the prior distribution described above, noise is added to the sample. T The prior distribution used in this application embodiment is a normal distribution.

[0099] The following describes in detail step 206, namely, "inputting the input sample and the noisy sample into the sample pair into the generation model, and during the training process, simulating the inverse diffusion process of noise diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output," with reference to the embodiments.

[0100] The training of the generative model is actually a process of simulating (i.e., learning) the inverse diffusion process based on the aforementioned forward denoising diffusion. The generative model architecture used in this application is an encoder-decoder structure. Figure 3 As shown, the input sample W x Input encoder, encoder to W x Encode the input sample W. x The characteristics are represented.

[0101] If the input sample is an image, then the encoder described above is an image encoding model. It is used to encode the input image sample to obtain the feature representation of the input image sample.

[0102] If the input sample is text, the encoder described above can be implemented based on a pre-trained language model. This can be achieved by using pre-trained language models such as BERT (Bidirectional Encoder Representation from Transformers), XLNet (an autoregressive model that uses permutations of language models to achieve bidirectional contextual information), and GPT (Generative Pre-Training) as the initial encoder, and then performing further time-step training on top of it. BERT is a bidirectional pre-trained language model that uses a Transformer Encoder as its model structure, and it effectively utilizes contextual information for feature learning. XLNet is a model similar to BERT, but it is a more generalized autoregressive pre-trained model. GPT uses a Transformer Decoder structure, and only retains masked multi-head attention within the Transformer Decoder.

[0103] The Transformer network is a model that uses a self-attention mechanism to encode each token in the input to transform it into a feature representation. In addition to using an encoder-decoder architecture based on Transformer, encoder-decoder structures based on other networks can also be employed, such as those based on RNNs (Recurrent Neural Networks).

[0104] The decoder uses the feature representation of the input sample and the above-mentioned noisy sample to perform inverse diffusion processing to obtain the output sample.

[0105] For the generation task, each time step can be viewed as removing noise from the sample obtained by backdiffusion in the previous time step, given the input sample. Specifically, for the first time step of backdiffusion, this involves removing noise from the noisy sample z.T Noise removal is performed. The noise removal process is the reverse of the noise addition process.

[0106] Denoising at each time step (i.e., inverse diffusion processing) can be considered as the inverse process of simulating noise addition, that is, simulating the posterior distribution of the forward noise addition and diffusion process, denoted as p. θ (z t-1 |z t W x It follows the form of a Gaussian distribution family. θ (z t-1 |z t W x This can be represented as:

[0107]

[0108] It's about z t-1 The distribution of , the mean of which is variance is

[0109] in,

[0110]

[0111] β t =1-α t (8)

[0112]

[0113] Where, α t The proportion of noise increasing at time step t is then... In fact, it represents the ratio of the amount of noise added to the output sample at time step t to the total amount of noise added in the end, which is the noise process involved in the embodiments of this application. It refers to the processing function that the generation model needs to simulate, or the denoising function learned by the model.

[0114] Since the denoising process is the inverse of the noise diffusion process, ideally the generative model should completely learn the inverse of the noise diffusion process to predict W. yTherefore, the training objective adopted in this embodiment for training the generative model mainly includes minimizing the difference between the sample distribution generated by the noise diffusion process and the sample distribution generated by the reverse diffusion process. A first loss function can be constructed based on this training objective. In each iteration, the model parameters are updated using methods such as gradient descent based on the value of the loss function until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0115] At each time step, the first loss function L1 can be defined as follows:

[0116]

[0117] In the above formula, E represents the expected value. This refers to the expected value of taking the content within [] under constraints. q(z) t ,z0,W x W y ) refers to W x W y ,z0 of z t It conforms to the distribution of q.

[0118] Furthermore, in the noise diffusion process, the ideal situation is that the samples obtained by the diffusion process in the last time step are the same as random noise. Therefore, the above training objective may also include: minimizing the value of the second loss function, which characterizes the difference between the distribution of the samples obtained by the diffusion process in the last time step and the normal distribution.

[0119] Furthermore, in the reverse diffusion (i.e., denoising) process, the ideal situation is to completely simulate the reverse process of the noise-adding diffusion, so that the sample obtained by reverse diffusion in the last time step is completely consistent with the output sample. Therefore, the above training objective can also include minimizing the value of the third loss function, which characterizes the difference between the sample obtained by reverse diffusion in the last time step and the output sample.

[0120] In the embodiments of this specification, a total loss function can be constructed based on the above-mentioned training objective. In each iteration, the model parameters are updated using methods such as gradient descent, based on the value of the total loss function, until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0121] One possible approach is to construct the total loss function L as follows:

[0122]

[0123] If the output sample in this embodiment is text, then the total loss function can be as follows:

[0124]

[0125] Among them, in the formula It refers to q φ (z t ,z0,W x W y The expected value of taking the contents within [] under the constraint q. φ (z t ,z0,W x W y ) refers to W x W y ,z0 of z t Conforms to q φ The distribution of .

[0126] as well as This reflects the difference between the distribution generated by the reverse diffusion process and the distribution generated by the noisy diffusion process. This represents the expected value of the sample obtained from the diffusion process at the last time step, which reflects the difference between the sample obtained from the diffusion process at the last time step and the normal distribution. This indicates that W is predicted based on z0 obtained through reverse diffusion. y The probability, therefore What it actually reflects is the difference between the sample obtained by the retrodiffusion at the last time step and the output sample.

[0127] Based on the understanding of the above basic principles and training structure, the noise process control method provided in the embodiments of this application will be described in detail below. The noise process involved in the embodiments of this application refers to the proportion of noise added to the output sample at each time step of the noise diffusion process, i.e., the proportion involved in the formula above. Correspondingly, the noise process also characterizes the denoising difficulty at each time step in the reverse diffusion (i.e., denoising) process.

[0128] In this application Instead of using a fixed linear function to determine the method, a different approach is adopted. Theoretically, predicting the samples at the first time step (assuming the use of...) The difficulty of characterizing the sample at the first predicted time step should increase linearly with the time step size. Therefore, the degree of noise removal at each inverse diffusion time step should be evenly distributed across different time steps to achieve better results. Thus, the first loss function corresponding to time step t can be used, for example... This is used to measure the difficulty of denoising at time step t. Therefore, a method can be designed... The value of is related to the value of the first loss function corresponding to each time step of the reverse diffusion process.

[0129] One feasible approach is to utilize the sampled values ​​of the first loss function corresponding to the reverse diffusion processing at each time step and the noise process at each time step to determine the mapping relationship between the first loss function values ​​and the noise process. Then, based on this mapping relationship, the noise processes corresponding to the first loss function values ​​that are uniformly distributed at each time step are determined as the noise processes corresponding to the diffusion processing at each time step.

[0130] The above mapping relationship can be obtained through fitting, for example, assuming that the first loss function corresponding to time step t is L. t This indicates that the noise process can be recorded at each time step. With the L already obtained t Then, the noise process at each time step is recorded. With the L already obtained t Fitting a process from noise L to the first loss function t The mapping function M():

[0131]

[0132] One feasible approach is to fit the mapping function using linear interpolation. Other methods can also be used to determine the mapping function.

[0133] Furthermore, the mapping function described above can be smoothed, that is, the L function can be smoothed for each time step t. t Downsampling is performed to obtain multiple discretized first loss functions, and the noise process is then processed using the same downsampling step size. Multiple discretized noise processes are also obtained through downsampling. The discretized noise processes and the discretized first loss function are then used to fit the above mapping function.

[0134] After obtaining the fitted mapping function, the inverse mapping function M can be obtained. -1 ():

[0135]

[0136] Then it can be done in L t To determine the corresponding value by uniformly selecting values ​​above Therefore, the noise process at each time step is determined during the subsequent forward diffusion process, i.e. For example, the L values ​​obtained at each time step can be... t After performing linear interpolation, a curve is fitted, and then values ​​are taken on this curve for each time step, so that the values ​​corresponding to each time step are evenly distributed.

[0137] As training progresses, the noise process can be redefined using the first loss function that has already been sampled after a certain training duration or number of rounds, and the newly defined noise process can be used in subsequent processes to achieve continuous calibration of the noise process.

[0138] Furthermore, the more uniform the distribution of the first loss function values, the better the effect. As a preferred implementation, equally spaced values ​​can be used. Besides the noise process corresponding to the uniformly distributed first loss function values ​​at each time step used in the above embodiment, other types of distributions can also be used, as long as the interval between the first loss function values ​​at each time step is small.

[0139] Furthermore, regardless of whether the output sample is an image or text, it can be viewed as consisting of multiple elements (Tokens). In an image, each Token can be a tile, and in text, each Token can be a character or word. Considering that Tokens at different locations may carry different amounts of information and have varying generation difficulties during the back-diffusion process, a different noise process for each Token will yield better results. Therefore, we can determine the noise process corresponding to the diffusion process at each time step for each Token in the output sample. The noise process corresponding to the diffusion process of the i-th Token at each time step is related to the value of the first loss function corresponding to the back-diffusion process of the i-th Token at each time step.

[0140] In this case, it actually records the noise process of the i-th token at each time step. The loss function corresponding to the i-th token at each time step. Then recorded With what has already been obtained Fitting a process from noise To the first loss function The mapping function M():

[0141]

[0142] One possible approach is to use linear interpolation to fit the mapping function described above.

[0143] Furthermore, the mapping function described above can be smoothed, that is, the smoothing effect can be applied to each time step t. Downsampling is performed to obtain multiple discretized first loss functions, and the noise process is then processed using the same downsampling step size. Multiple discretized noise processes are also obtained through downsampling. The discretized noise processes and the discretized first loss function are then used to fit the above mapping function.

[0144] After obtaining the fitted mapping function, the inverse mapping function M can be obtained. -1 ():

[0145]

[0146] Then it can be done in To determine the corresponding value by uniformly selecting values ​​above Therefore, in the subsequent forward diffusion process, the determined noise process at each time step is adopted for the i-th token, i.e. As training progresses, the noise process can be redefined using the first loss function obtained after a certain training duration or number of rounds, and the newly defined noise process can be used in subsequent processes to achieve continuous calibration of the noise process.

[0147] Based on the above description, it can be seen that, as one feasible approach, in reverse diffusion processing, the samples obtained from reverse diffusion at each time step are sampled from the posterior distribution of the samples obtained from reverse diffusion at the previous time step. That is, z t It is in p θ (z t-1 |z t W x (Originally obtained from upsampling)

[0148] The noise process determined in the above manner can improve the quality of the generated output samples, but it will reduce diversity. To compensate for the loss of diversity, this application provides a new sampling method. The samples obtained by the back-diffusion processing at each time step are obtained by sampling according to a first preset probability using a first method and according to a second preset probability using a second method. The first method involves sampling on the posterior distribution of the samples obtained by the back-diffusion processing at the previous time step, and the second method involves sampling on the predicted... Upsampling from the prior distribution, This represents the prediction result for the samples obtained from the first step of the diffusion process. In other words, based on the aforementioned posterior distribution, a sampling method based on the prior distribution is randomly inserted with a certain probability.

[0149] The second method described below, namely the sampling method based on the prior distribution, is discussed below. At each time step in the noise diffusion process, a prior distribution based on z0 can be obtained, expressed as:

[0150]

[0151] Therefore, in the reverse diffusion process, the sample z obtained from the reverse diffusion process at each time step... t It can be based on prediction. The prior distribution was obtained by upsampling. This represents the prediction result for the sample obtained from the first step of the diffusion process. In other words, each step of the reverse diffusion process can predict one... Then based on the prior distribution q(z) t |z0) sampled to obtain z t Initial It is inaccurate, but as time progresses, the accuracy of the data will improve. The predictions are becoming increasingly accurate, and the final step of reverse diffusion processing yields... At that time, the goal was to make Consistent with z0.

[0152] Since for sample z t-1 The prediction, based on the sampling method corresponding to the prior distribution, has a variance of 1- The variance corresponding to the posterior distribution is: According to formulas (7) and (8), we can obtain:

[0153]

[0154] therefore, Clearly, this sampling method can produce a larger variance, thus generating more diverse samples at each time step.

[0155] Based on the generative model obtained from the above training, specific resource generation tasks can be performed using the generative model. Figure 4 This is a flowchart of a resource generation method provided in an embodiment of this application. This method can be generated by... Figure 1 The resource generation device in the system shown is executed. For example... Figure 4 As shown, the method may include the following steps:

[0156] Step 402: Obtain input resources.

[0157] Step 404: Input the input resources and random noise into the generation model. The generation model then performs inverse diffusion processing based on the input resources and random noise to obtain the output resources. The generation model employs methods such as... Figure 2 The method shown is used for pre-training.

[0158] The structure of the pre-trained generative model obtained in this application embodiment is as follows: Figure 5 As shown, it includes an encoder and a decoder.

[0159] The encoder acquires the feature representation of the input resource.

[0160] The decoder uses the feature representation of the input resources and random noise to perform inverse diffusion processing to predict the output resources.

[0161] In other words, in the actual prediction process, the encoder's input and processing remain unchanged. The encoder only needs to perform one feedforward calculation of the neural network, without having to participate in the backdiffusion process, which may require hundreds or thousands of processing steps. Therefore, it can greatly save computing resources.

[0162] In addition to the encoder's output, the decoder also receives random noise as input. The decoder performs denoising step by step based on the feature representation of the input resources, obtaining the feature representation at the last time step. Then The output resource is obtained by mapping.

[0163] The input and output resources involved in the above resource generation process can be images or text. For example, if the input resource is text and the output resource is an image, the above generation model generates an image based on the text. As another example, if the input resource is an image and the output resource is an image, the above generation model generates an output image based on the input image. Similarly, if the input resource is an image and the output resource is text, the above generation model generates text based on the input image. Finally, if the input resource is text and the output resource is text, the above generation model generates output text based on the input text.

[0164] As one possible approach, the resource generation method described above can be executed by a cloud server, meaning that the resource generation function is integrated into the cloud. A cloud server, also known as a cloud computing server or cloud host, is a host product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0165] When a user wants to generate output resources from input resources, they can upload the input resources to the cloud server via the user terminal.

[0166] The aforementioned user terminals may be, but are not limited to, mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), wearable devices, PCs (Personal Computers), etc.

[0167] The cloud server obtains input resources from the user terminal; then, based on a pre-trained generative model, it performs inverse diffusion processing on the input resources and random noise to obtain output resources, and returns the output resources to the user terminal.

[0168] The methods provided in this application embodiment can be applied to various application scenarios, and only a few of them are described here:

[0169] Application Scenario 1: Summary Generation Scenario

[0170] In this scenario, when training the generative model, we can use articles as input samples and their summaries as output samples to form sample pairs. For example, we can use news texts as input samples and their summaries as output samples. Similarly, we can use academic papers as input samples and their summaries as output samples. News texts and their summaries, as well as academic papers and their summaries, are all readily available online, thus providing a large number of training samples for use as training data.

[0171] Then, the feature representation of the output sample in the sample pair is obtained, and noise diffusion processing is applied to the feature representation of the output sample to obtain the noisy feature representation. The input sample of the sample pair and the noisy feature representation are then input into the generation model. During the training process, the generation model simulates the inverse diffusion processing of noise diffusion based on the input sample and the noisy feature representation to obtain the output sample. For the specific training process, please refer to the method embodiment for details. Figure 2 and Figure 3 The relevant records will not be elaborated here.

[0172] In actual summary generation, the input text is obtained, and the input text and random noise are fed into the pre-trained generation model. The generation model performs inverse diffusion processing based on the input text and random noise to obtain a summary of the input text.

[0173] This method can automatically generate accurate summaries from input text, enabling the automatic generation and publication of summaries along with news articles and academic papers when they are published online. Alternatively, the text generation device can be provided to users as a tool, allowing them to upload their own documents as input and use the tool to obtain automatically generated summaries.

[0174] Application Scenario 2: Machine Translation

[0175] In this scenario, when training the generative model, bilingual corpora can be used as sample pairs. These bilingual corpora include text in the first language as input samples and text in the second language as output samples. For example, a sample pair consisting of Chinese text and its corresponding English text can be used as training samples.

[0176] Then, the feature representation of the output sample in the sample pair is obtained, and noise diffusion processing is applied to the feature representation of the output sample to obtain the noisy feature representation. The input sample of the sample pair and the noisy feature representation are then input into the generation model. During the training process, the generation model simulates the inverse diffusion processing of noise diffusion based on the input sample and the noisy feature representation to obtain the output sample. For the specific training process, please refer to the method embodiment for details. Figure 2 and Figure 3 The relevant records will not be elaborated here.

[0177] In actual machine translation, text in the first language is obtained, and the text in the first language and random noise are input into the pre-trained generative model. The generative model performs inverse diffusion processing based on the text in the first language and random noise to obtain text in the second language.

[0178] This method allows for the automatic translation of text in a first language into text in a second language. For example, when text is published online, it can be automatically translated into another language so that users in different countries or regions can view it. Another example is providing a text generation device as a tool to users, who can upload the document to be translated as input text and use the tool to obtain automatically translated text in the specified language.

[0179] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0180] According to another embodiment, an apparatus for training a generative model is provided. Figure 6 A schematic block diagram of an apparatus for training a generative model according to one embodiment is shown. Figure 1 The model training device in the architecture shown. Figure 6 As shown, the device 600 may include: a sample acquisition unit 601, a noise addition and diffusion unit 602, and a model training unit 603. The main functions of each component are as follows:

[0181] The sample acquisition unit 601 is configured to acquire training data containing multiple training samples, wherein the training samples include sample pairs consisting of input samples and output samples, and the input samples and output samples are images or text.

[0182] The noise-adding and diffusion unit 602 is configured to perform noise-adding and diffusion processing on the output samples of the sample pair to obtain the noise-added samples.

[0183] The model training unit 603 is configured to input the input sample and the noisy sample into the sample pair to generate the model. During the training process, the model simulates the inverse diffusion process of the noise diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output.

[0184] In the noise diffusion process, the noise-adding diffusion unit 602 adds noise to the sample obtained by the diffusion process of the previous time step in each time step. The noise process corresponding to the diffusion process of each time step is related to the value of the first loss function corresponding to the inverse diffusion process of each time step. The noise process corresponding to the diffusion process of each time step is the proportion of noise added to the output sample by the diffusion process of each time step. The first loss function characterizes the difference between the sample distribution generated by the noise-adding diffusion process and the sample distribution generated by the inverse diffusion process.

[0185] As one possible approach, the noise diffusion unit 602 can use the first loss function values ​​corresponding to the inverse diffusion processing of each time step that have been sampled and the noise process of each time step to determine the mapping relationship between the first loss function values ​​and the noise process; based on the mapping relationship, the noise processes corresponding to the first loss function values ​​that are uniformly distributed at each time step are determined as the noise processes corresponding to the diffusion processing of each subsequent time step.

[0186] As one possible approach, the noise diffusion unit 602 can determine the noise process corresponding to the diffusion process at each time step for each element Token in the output sample. The noise process corresponding to the diffusion process of the i-th Token at each time step is related to the value of the first loss function corresponding to the reverse diffusion process of the i-th Token at each time step.

[0187] As one possible approach, in the generative model, the samples obtained from the backdiffusion process at each time step are obtained by upsampling the posterior distribution of the samples obtained from the backdiffusion process at the previous time step.

[0188] As another feasible approach, in the backdiffusion processing of the generative model, the samples obtained from the backdiffusion processing at each time step are obtained by sampling according to a first preset probability using a first method and according to a second preset probability using a second method. The first method involves sampling from the posterior distribution of the samples obtained from the backdiffusion processing of the previous time step, and the second method involves sampling from the posterior distribution of the samples obtained from the prediction. Upsampling from the prior distribution, This is the prediction result for the sample obtained from the first step of diffusion processing.

[0189] As one possible approach, the generative model described above includes an encoder and a decoder.

[0190] The encoder acquires the feature representation of the input samples to the model, and the decoder uses the feature representation of the input samples and the noisy samples to perform inverse diffusion processing to obtain the output samples. The training objectives include minimizing the value of the first loss function.

[0191] In one typical scenario, if both the input and output samples are text, the noise-adding and diffusion unit 602 can be specifically configured to: obtain the feature representation of the output sample, perform noise-adding and diffusion processing on the feature representation of the output sample, and obtain the noise-adding feature representation as the noise-adding sample.

[0192] Furthermore, the training objective can also include minimizing the value of the second loss function and / or the value of the third loss function. The second loss function characterizes the difference between the sample distribution obtained by the diffusion process in the last time step and the normal distribution; the third loss function characterizes the difference between the samples obtained by the reverse diffusion in the last time step and the output samples.

[0193] According to another embodiment, a resource generation apparatus is provided. Figure 7 A schematic block diagram of a resource generation apparatus according to one embodiment is shown. Figure 7 As shown, the device 700 may include a resource acquisition unit 701 and a resource generation unit 702. The main functions of each component are as follows:

[0194] The text acquisition unit 701 is configured to acquire input resources.

[0195] The text generation unit 702 is configured to input input resources and random noise into the generation model, and the generation model performs back-diffusion processing based on the input resources and random noise to obtain the output resources.

[0196] The input and output resources mentioned above are images or text, and the generation model is... Figure 6 The device shown was pre-trained.

[0197] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0198] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0199] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0200] And an electronic device, comprising:

[0201] One or more processors; and

[0202] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0203] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0204] in, Figure 8An exemplary architecture of an electronic device is shown, which may include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820 can communicate with each other via a communication bus 830.

[0205] The processor 810 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0206] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system 821 for controlling the operation of the electronic device 800, and the basic input / output system (BIOS) 822 for controlling the low-level operations of the electronic device 800. Additionally, it can store a web browser 823, a data storage management system 824, and a model training device / resource generation device 825, etc. The aforementioned model training device / resource generation device 825 can be the application program that specifically implements the aforementioned steps in the embodiments of this application. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0207] The input / output interface 813 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0208] Network interface 814 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0209] Bus 830 includes a pathway for transmitting information between various components of the device, such as processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820.

[0210] It should be noted that although the above-described device only shows the processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, memory 820, bus 830, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0211] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0212] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training a generative model, characterized in that, The method includes: Acquire training data containing multiple training samples, wherein the training samples include sample pairs consisting of input samples and output samples, and the input samples and output samples are images or text; The output samples in the sample pair are subjected to noise-adding and diffusion processing to obtain the noisy samples; The input sample and the noisy sample are input into the generative model. During the training process, the generative model simulates the inverse diffusion process of the noisy diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output. In the noise-adding diffusion process, each time step of the diffusion process adds noise to the sample obtained from the diffusion process of the previous time step. The noise progression corresponding to each time step of the diffusion process is related to the value of the first loss function corresponding to each time step of the reverse diffusion process. The noise progression corresponding to each time step of the diffusion process is the proportion of noise added to the output sample by the diffusion process of each time step. The first loss function characterizes the difference between the sample distribution generated by the noise-adding diffusion process and the sample distribution generated by the reverse diffusion process.

2. The method according to claim 1, characterized in that, The method further includes: By using the values ​​of the first loss function corresponding to the inverse diffusion processing at each time step that have been sampled and the noise process at each time step, the mapping relationship between the values ​​of the first loss function and the noise process is determined. Based on the mapping relationship, the noise processes corresponding to the first loss function values ​​that are uniformly distributed at each time step are determined as the noise processes corresponding to the diffusion processing at each subsequent time step.

3. The method according to claim 1, characterized in that, For each element Token in the output sample, the noise process corresponding to the diffusion process at each time step is determined. The noise process corresponding to the diffusion process of the i-th Token at each time step is related to the value of the first loss function corresponding to the reverse diffusion process of the i-th Token at each time step.

4. The method according to claim 1, characterized in that, In the reverse diffusion process, the samples obtained from the reverse diffusion process at each time step are sampled from the posterior distribution of the samples obtained from the reverse diffusion process at the previous time step; or, The samples obtained from the backdiffusion processing at each time step are obtained by sampling using a first method according to a first preset probability and a second method according to a second preset probability. The first method involves sampling from the posterior distribution of the samples obtained from the backdiffusion processing at the previous time step, and the second method involves sampling from the posterior distribution of the samples obtained from the prediction. Upsampling of the prior distribution, the This is the prediction result for the sample obtained from the first step of diffusion processing.

5. The method according to any one of claims 1 to 4, characterized in that, The generative model includes an encoder and a decoder; The encoder acquires the feature representation of the input sample of the input generation model, and the decoder uses the feature representation of the input sample and the noisy sample to perform the inverse diffusion process to obtain the output sample; The training objective includes minimizing the value of the first loss function.

6. The method according to any one of claims 1 to 4, characterized in that, If both the input sample and the output sample are text, then the output sample in the sample pair is subjected to noise-adding and diffusion processing to obtain the noisy sample, including: The feature representation of the output sample is obtained, and the feature representation of the output sample is subjected to noise diffusion processing to obtain the noise-added feature representation as the noise-added sample.

7. The method according to claim 5, characterized in that, The training objective also includes minimizing the value of the second loss function and / or the value of the third loss function; The second loss function characterizes the difference between the sample distribution obtained by diffusion processing in the last time step and the normal distribution; the third loss function characterizes the difference between the sample obtained by reverse diffusion in the last time step and the output sample.

8. A resource generation method, characterized in that, The method includes: Obtain input resources; The input resources and random noise are input into the generation model, and the generation model performs inverse diffusion processing based on the input resources and random noise to obtain the output resources. The input and output resources are images or text; the generation model is pre-trained using the method described in any one of claims 1 to 7.

9. A method for generating abstracts, characterized in that, The method includes: Get the input text; The input text and random noise are input into the generation model, which then performs inverse diffusion processing based on the input text and random noise to obtain a summary of the input text. The generative model is pre-trained using the method described in any one of claims 1 to 7.

10. A machine translation method, characterized in that, The method includes: Retrieve text written in the first language; The text in the first language and random noise are input into the generation model, and the generation model performs inverse diffusion processing based on the text in the first language and the random noise to obtain the text in the second language. The generative model is pre-trained using the method described in any one of claims 1 to 7.

11. An apparatus for training a generative model, characterized in that, The device includes: The sample acquisition unit is configured to acquire training data containing multiple training samples, wherein the training samples include sample pairs consisting of input samples and output samples, and the input samples and output samples are images or text. The noise-adding and diffusion unit is configured to perform noise-adding and diffusion processing on the output samples in the sample pair to obtain the noise-added samples. The model training unit is configured to input the input sample and the noisy sample into the sample pair to generate a model. During the training process, the generating model simulates the inverse diffusion process of the noisy diffusion based on the input sample and the noisy sample, so as to use the output sample as the target output. In the noise-adding diffusion process, the noise-adding diffusion unit adds noise to the sample obtained from the diffusion process of the previous time step at each time step. The noise process corresponding to the diffusion process at each time step is related to the value of the first loss function corresponding to the inverse diffusion process at each time step. The noise process corresponding to the diffusion process at each time step is the proportion of noise added to the output sample by the diffusion process at each time step. The first loss function characterizes the difference between the sample distribution generated by the noise-adding diffusion process and the sample distribution generated by the inverse diffusion process.

12. A resource generation device, characterized in that, The device includes: The resource acquisition unit is configured to acquire input resources; The resource generation unit is configured to input the input resources and random noise into the generation model, and the generation model performs inverse diffusion processing based on the input resources and random noise to obtain the output resources. The input and output resources are images or text, and the generation model is pre-trained by the device described in claim 11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 10.

14. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 10.

Images