Model processing method, system, electronic device and storage medium

Abstract

The present application discloses a model processing method, system, electronic device and storage medium, which relate to the fields of artificial intelligence and machine learning. The method includes: obtaining condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, determining the pre-trained model corresponding to the model to be generated, and determining the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determining the parameters of the initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain the target solver; using the target solver to train the pre-trained model to obtain the generated model in the application scenario. The present application solves the technical problem of poor generation effect of the generated model.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and machine learning, and specifically, to a model processing method, system, electronic device and storage medium. Background Art

[0002] Currently, in the research of generative models, pre-trained models have attracted widespread attention due to their powerful generative capabilities. Pre-trained models usually learn data distribution through an iterative denoising process of noisy data and generate new, seemingly real data samples.

[0003] However, when the pre-trained model uses low-step sampling, the error of the generation result of the pre-trained generative model will increase significantly, seriously reducing the generation performance, resulting in the problem of poor generation effect of the generative model.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a model processing method, system, electronic device and storage medium to at least solve the technical problem of poor generation effect of the generated model.

[0006] According to one aspect of an embodiment of the present application, a method for processing a model is provided. The method may include: obtaining condition information of a model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; under the condition information, determining a pre-trained model corresponding to the model to be generated, and determining a noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determining the parameters of the initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain a target solver; using the target solver to train the pre-trained model to obtain a generated model in the application scenario.

[0007] According to another aspect of the embodiment of the present application, a model processing method is provided. The method may include: obtaining condition information of the denoising generation model to be generated, wherein the condition information is used to indicate the conditions that the denoising generation model to be generated needs to meet in the media application scenario; under the condition information, determining the pre-trained denoising model corresponding to the denoising generation model to be generated, and determining the noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model; determining the parameters of the initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained denoising model, and adjusting the initial solver using the obtained sampling results to obtain a target solver; using the target solver to train the pre-trained denoising model to obtain a denoising generation model in the media application scenario.

[0008] According to another aspect of the embodiment of the present application, a method for processing a model is provided. The method may include: obtaining condition information of the model to be generated by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes condition information, and the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determine the parameters of the initial solver under the noise scheduling strategy; use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained model to obtain a generated model in the application scenario; output the generated model by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the generated model.

[0009] According to another aspect of the embodiment of the present application, a model processing system is provided. The system may include: a client, used to send condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; a server, connected to the client, used to determine the pre-trained model corresponding to the model to be generated under the condition information, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determine the parameters of the initial solver under the noise scheduling strategy; use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained model to obtain a generated model in the application scenario; send the generated model to the client.

[0010] According to another aspect of an embodiment of the present application, an electronic device is also provided. The electronic device may include a memory and a processor: the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions. When the above-mentioned computer-executable instructions are executed by the processor, a processing method of a model of any one of the above-mentioned items is implemented.

[0011] According to another aspect of an embodiment of the present application, a processor is further provided, and the processor is used to run a program, wherein any one of the above-mentioned model processing methods is executed when the program is running.

[0012] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute any one of the above-mentioned model processing methods.

[0013] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and the computer program implements any one of the above-mentioned model processing methods when executed by a processor.

[0014] In an embodiment of the present application, condition information of a model to be generated is obtained, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; under the condition information, a pre-trained model corresponding to the model to be generated is determined, and a noise scheduling strategy adopted by the pre-trained model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; the pre-trained model is trained using the target solver to obtain a generated model in an application scenario. In other words, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, and can determine the target solver under the noise scheduling strategy adopted by the pre-trained model, so that the target solver can be used to accelerate the generation performance of the generated model sampling, thereby achieving the improvement of the generation effect of the generated model and solving the technical problem of poor generation effect of the generated model.

[0015] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a schematic diagram of an application scenario of a model processing method according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a model processing method according to an embodiment of the present application;

[0019] Figure 3 is a flowchart of another model processing method according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of another model processing method according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of a processing system of a model according to an embodiment of the present application;

[0022] Figure 6 It is a structural block diagram of a computing environment of a model processing method according to an embodiment of the present application;

[0023] Figure 7 is a schematic diagram of a processing device of a model according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of a processing device of another model according to an embodiment of the present application;

[0025] Fig. 9 is a schematic diagram of a processing device of another model according to an embodiment of the present application;

[0026] Fig.10 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0027] Fig.11 is a block diagram of an electronic device according to a model processing method of an embodiment of the present application;

[0028] Fig.12 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a model processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application, for example, the data for verification, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0033] Noise scheduler, which defines the strategy of adding or removing noise at different time points in the process of generating data, so as to gradually recover clear and structured data samples from random noise;

[0034] The denoising generative model is a model that gradually transforms noisy data into clear data;

[0035] Parameterization means expressing the solver as a function of solver parameters so that the solver can be adjusted and optimized by numerical optimization algorithms;

[0036] Generative model sampling refers to the process of generating new data samples from a generative model. This process usually starts with a random noise and gradually generates samples similar to the distribution of training data through the forward propagation of the generative model.

[0037] Low-step sampling uses fewer iteration steps to generate samples in the generative model. In the diffusion model, it usually takes more steps to recover from the noisy state to the clear state, but low-step sampling reduces the required steps by optimizing the solution strategy, thereby increasing the generation speed;

[0038] The linear multi-step method, a numerical integration method for solving differential equations, provides more accurate sampling results by leveraging multiple previous time-step information to predict the next state during the sampling process of the generative model.

[0039] The processing method of the above model provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario of a model processing method according to an embodiment of the present application, in which Figure 1 In the application scenario shown, the server 10 can be a cloud. The server 10 can be connected to one or more client devices 20 via a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client devices 20 here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client devices together constitute the client relative to the server. An operating interface for obtaining conditional information of the model to be generated can be deployed on the graphical user interface on the client device. The client device 20 can interact with the user through the graphical user interface to implement the processing method of the model provided in the embodiment of the present application.

[0040] In an embodiment of the present application, the system composed of the client device 20 and the server 10 can perform the following steps: perform corresponding operations in the operation interface on the client device 20 to obtain the condition information of the model to be generated. The client device can obtain the condition information of the model to be generated and send it to the server through the network. After receiving the condition information of the model to be generated, the server can perform the following steps: Step S102, obtain the condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; Step S104, under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; Step S106, determine the parameters of the initial solver under the noise scheduling strategy; Step S108, use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain the target solver; Step S110, use the target solver to train the pre-trained model to obtain the generated model in the application scenario.

[0041] Under the above operating environment, this application provides Figure 2 The processing method of the model shown. Figure 2 is a flow chart of a model processing method according to an embodiment of the present application, such as Figure 2 As shown, the method may include the following steps:

[0042] Step S202, obtaining condition information of the model to be generated.

[0043] In the technical solution provided in the above step S202 of the present application, condition information of the model to be generated can be obtained. Among them, the model to be generated can be a denoising model to be generated, and the denoising model to be generated can also be called a denoising generation model to be generated. The condition information can be used to represent the conditions that the model to be generated needs to meet in the application scenario, and the condition information can be called conditional coding information, which can be obtained by converting the category label or text prompt information input by the user. The application scenario can be a media application scenario, which is only used as an example here, and the type of application scenario is not specifically limited.

[0044] Optionally, the category label or text prompt information input by the user is obtained, and the category label or text prompt information is converted into a vector form that can be understood by the model to be generated through the encoder of the model to be generated, thereby obtaining conditional coding information.

[0045] For example, the user inputs a category label, such as "cat", or a text prompt information, such as "a black cat napping in the sun", which can be used to define the conditional information of the above-mentioned model to be generated. When the user inputs a category label, the category (label) encoder of the model to be generated can convert the category label into a vector form that can be understood by the model to be generated, thereby obtaining conditional encoding information. When the user inputs text prompt information, the text encoder of the model to be generated can convert the text into one or more vectors to obtain conditional encoding information.

[0046] Optionally, the conditional encoding information can be used to convert the category label or text prompt information provided by the user into a conditional input of the model to be generated, so as to ensure that the user's expected conditions for the generation result of the model to be generated are met.

[0047] Step S204: Under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model.

[0048] In the technical solution provided in the above step S204 of the present application, after obtaining the condition information of the model to be generated, under the condition information, the pre-trained model corresponding to the model to be generated can be determined, and the noise scheduling strategy adopted by the pre-trained model can be determined. Among them, the pre-trained model can be a pre-trained generation model, and the pre-trained model can also be called a pre-trained denoising model, a pre-trained denoising generation model, and a pre-trained proxy model. The noise scheduling strategy can be used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model.

[0049] In this embodiment, after obtaining the condition information of the model to be generated, a pre-trained model matching the model to be generated under the condition information can be queried from the pre-trained model library. It should be noted that each pre-trained model in the pre-trained model library has a specific training background, a data type that it is good at, and a noise scheduling strategy.

[0050] For example, if a user requests to generate an image of a "cat", an image generation model can be searched from a pre-trained model library as a pre-trained model, and the image generation model can be a model specially pre-trained for animal images.

[0051] Optionally, after determining the pre-trained model corresponding to the model to be generated, since each pre-trained model in the pre-trained model library has a specific noise scheduling strategy, the noise scheduling strategy of the pre-trained model determined above can be automatically identified or read to determine the noise scheduling strategy adopted by the pre-trained model.

[0052] It should be noted that the noise scheduling strategy is usually defined during the training of the pre-trained model. The noise scheduling strategy describes how to gradually increase or decrease the noise during the generation process, as well as the nature of the noise (for example, Gaussian noise, uniform noise, etc.). Different pre-trained models can adopt different noise scheduling strategies.

[0053] Step S206, determining the parameters of the initial solver under the noise scheduling strategy.

[0054] In the technical solution provided in the above step S206 of the present application, after determining the noise scheduling strategy adopted by the pre-trained model, the parameters of the initial solver under the noise scheduling strategy can be determined. The initial solver can be a differentiable solver, which can be used to optimize the parameters in the solution process.

[0055] It should be noted that the parameters of the initial solver may include at least the coefficient matrix and time step of the initial solver. The coefficient matrix can be the parameters for numerical integration to solve differential equations. In the numerical solution of differential equations, the coefficient matrix can be used to determine which numerical integration method to use, such as the Euler method, to approximate the differential equations of the model. The coefficient matrix can be represented by M. The time step can be a discretization unit on the time axis. The time step can be used to control the addition and removal of noise. Each time step corresponds to a noise level. By iterating these time steps, the model gradually generates clear and structured output samples from an initial high-noise state. The time step can be expressed by To express.

[0056] Optionally, determine the search space of the initial solver, for example, the search space can be the coefficient matrix M and the sampling time step , then the coefficient matrix M and the sampling time step can be Determine the parameters of the initial solver under the noisy scheduling strategy.

[0057] Step S208, using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain the target solver.

[0058] In the technical solution provided in the above step S208 of the present application, after determining the parameters of the initial solver under the noise scheduling strategy, the pre-trained model can be sampled using the determined parameters of the initial solver to obtain a sampling result. The initial solver can be further adjusted using the obtained sampling result to obtain a target solver. Among them, the sampling result can be a vector of the latent space obtained after sampling the pre-trained model. The performance index of the target solver is greater than the performance index threshold, and the performance index of the target solver can be used to indicate the performance quality of the target solver. For example, the performance index can be a sampling performance, and the performance index threshold can be an index threshold pre-set according to actual conditions.

[0059] It should be noted that the latent obtained by the above sampling needs to be converted into an understandable or visual form through the decoder of the pre-trained model corresponding to the model to be generated determined in step S204. The decoding process can map the vector of the latent space back to the data space to generate the final output.

[0060] Optionally, use the coefficient matrix M of the initial solver and the sampling time step , the pre-trained model can be sampled linearly in multiple steps to obtain the sampling result (i.e. latent). The coefficient matrix M of the initial solver and the sampling time step can be further calculated based on the sampling result. Adjust the target solver. For example, the coefficient matrix M of the initial solver and the sampling time step can be adjusted by sampling results. Optimize until the optimized coefficient matrix M and the sampling time step are used , the sampling error obtained after linear multi-step sampling of the pre-trained model is minimized, thereby obtaining the target solver, that is, the coefficient matrix M of the target solver and the sampling time step This can minimize the sampling error.

[0061] Step S210, using the target solver to train the pre-trained model to obtain a generation model in the application scenario.

[0062] In the technical solution provided in the above step S210 of the present application, after the initial solver is adjusted using the obtained sampling results to obtain the target solver, the pre-trained model can be trained using the obtained target solver to obtain a generative model in the application scenario. The generative model can be a generated denoising model, which can also be called a denoising generative model.

[0063] Optionally, after adjusting (optimizing) the parameters of the initial solver to obtain the target solver, the pre-trained model is trained using the obtained target solver to obtain the generated model. That is, this embodiment accelerates the generation process by optimizing the parameters of the initial solver, thereby achieving high-quality data generation using the pre-trained model under low step sampling.

[0064] Through the above steps S202 to S210 of the present application, the condition information of the model to be generated is obtained, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, the pre-trained model corresponding to the model to be generated is determined, and the noise scheduling strategy adopted by the pre-trained model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain the target solver; the pre-trained model is trained using the target solver to obtain the generated model in the application scenario. In other words, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, and can determine the target solver under the noise scheduling strategy adopted by the pre-trained model, so that the target solver can be used to accelerate the generation performance of the generated model sampling, thereby achieving the improvement of the generation effect of the generated model and solving the technical problem of poor generation effect of the generated model.

[0065] The above method of this embodiment is further introduced below.

[0066] As an optional implementation, step S206, determining the parameters of the initial solver under the noise scheduling strategy, includes: determining at least one variable to be adjusted in the initial solver; and converting the variable to be adjusted into the parameters of the initial solver.

[0067] In this embodiment, after determining the noise scheduling strategy adopted by the pre-trained model, at least one variable to be adjusted in the initial solver under the noise scheduling strategy can be determined. After determining at least one variable to be adjusted in the initial solver, the variable to be adjusted can be converted into a parameter of the initial solver. Among them, the variable to be adjusted can at least include a time variable and a coefficient variable, the time variable can be represented by r, and the time variable can be called an r time vector or an r vector. The coefficient variable can be represented by c, and the coefficient variable can be called a c coefficient vector or a c vector.

[0068] Optionally, an optimization space of the initial solver is determined, for example, the optimization space may be a time variable r and a coefficient variable c. The time variable r may be converted into a time step of the initial solver: , and convert the coefficient variable c into the coefficient matrix M of the initial solver, thereby determining the parameters of the initial solver under the noise scheduling strategy.

[0069] This embodiment continuously optimizes the time variable r and the coefficient variable c in the optimization space to find the coefficient matrix M and the time step length that minimize the sampling error. , the optimized solver can produce similar generation results as high-step sampling with fewer sampling steps (i.e., low-step sampling), thereby significantly reducing computing time and resource consumption.

[0070] As an optional implementation, the variables to be adjusted include time variables and coefficient variables, the parameters of the initial solver include the time step of the initial solver, and the coefficient matrix of the initial solver, and the variables to be adjusted are converted into the parameters of the initial solver, including: initializing the time variables and converting the initialized time variables into the time step of the initial solver; initializing the coefficient variables and converting the initialized coefficient variables into the coefficient matrix of the initial solver.

[0071] In this embodiment, after determining the time variable r of the initial solver, the time variable r can be initialized, and the r vector can be initialized to a uniform distribution. The dimension of the uniform distribution can be represented by N, and N can be the number of sampling steps searched by the initial solver. The initialized r vector can be further converted to a time step by a normalized exponential (Softmax) function. , that is, = Softmax(r).

[0072] Optionally, after determining the coefficient variable c of the initial solver, the coefficient variable c may be initialized by initializing the c vector to the following matrix:

[0073]

[0074] Among them, the above matrix has a total of (N*(N-1)) / 2 numbers. The matrix is ​​accessed by indexing through superscripts and subscripts. The superscript can be represented by i and the subscript can be represented by j. The superscript i is always smaller than the subscript j. The subscript j ranges from 0 to N-1, and the diagonal part of the matrix is ​​determined by implicit constraints. The initialized c vector can be further converted into a coefficient matrix M, that is, the above matrix is ​​determined as the coefficient matrix M.

[0075] This embodiment initializes the r vector and converts it into a time step through the Softmax function, which can achieve more accurate sampling time point selection, thereby ensuring that the time allocation in the entire sampling process is more reasonable, helping to improve the efficiency of the generation model within a limited number of steps and reducing unnecessary calculations. By optimizing the c vector, the sampling strategy of the linear multi-step method can be optimized, so that the model can recover clear images or data from noise with fewer steps. The optimization of the coefficient matrix helps to enhance the model's memory and utilization of past sampling steps, thereby more accurately predicting and adjusting the state vector in each subsequent step.

[0076] As an optional implementation, the parameters of the initial solver include the time step of the initial solver and the coefficient matrix of the initial solver. Step S208, using the parameters of the initial solver, samples the pre-trained model, including: using the time step and the coefficient matrix to linearly sample the pre-trained model to obtain a sampling result.

[0077] In this embodiment, the coefficient matrix M and the sampling time step of the initial solver under the determined noise scheduling strategy are Afterwards, using the coefficient matrix M and the sampling time step , the pre-trained model can be sampled linearly in multiple steps to obtain the sampling results.

[0078] Optionally, using the coefficient matrix M and the sampling time step , the sampling parameters (such as the number of sampling times and the number of sampling steps, etc.) of the linear multi-step sampling of the pre-trained model can be updated, and further based on the updated sampling parameters, the linear multi-step sampling can be performed on the pre-trained model to obtain the sampling results.

[0079] This embodiment uses the time step and the coefficient matrix to perform linear sampling on the pre-trained model, which can reduce the number of sampling steps required to generate high-quality samples, thereby significantly speeding up the generation process.

[0080] As an alternative implementation, linear sampling is performed on the pre-trained model using the time step and the coefficient matrix to obtain a sampling result, including: a first determination step of determining the current sampling number for linear sampling of the pre-trained model and determining the current time step corresponding to the current sampling number; a sampling step of, in response to the current sampling number being less than the target sampling number, performing linear sampling on the pre-trained model using the current time step and the input data of the pre-trained model at the current time step to obtain the current sampling result, where the target sampling number is less than the number threshold; the method further includes: a second determination step of determining the row coefficient corresponding to the current sampling number in the coefficient matrix and determining the difference between the row coefficient and the current sampling result; an update step of incrementing the current sampling number, updating the current time step using the time step, and updating the input data using the difference, the time step, and the input data, and returning to execute from the sampling step until the current sampling number is greater than or equal to the target sampling number.

[0081] In this embodiment, the current sampling number can be represented by i, and the current time step can be represented by The target sampling number can be the sampling number searched by the initial solver and can be represented by N. The input data can be used to represent the current generation state at a certain time point during the generation process and can be represented by The number threshold can be a threshold set in advance according to the actual situation. The row coefficient can be the i-th row coefficient in the coefficient matrix M.

[0082] It should be noted that when performing linear multi-step sampling on the pre-trained model using the coefficient matrix M and the sampling time step , an empty buffer can be initialized first, and the sampling number i = 0 can be initialized, where the buffer can be represented by Q.

[0083] Optionally, determine the current sampling number i and the current time step for linear multi-step sampling of the pre-trained model . If the current sampling number i is less than the target sampling number N (i.e., i < N), then use the current time step and the input data of the pre-trained model at the current time step to perform linear multi-step sampling on the pre-trained model to obtain the current sampling result (latent), and the obtained current sampling result can be input into the buffer Q.

[0084] Optionally, after obtaining the current sampling result and inputting the obtained current sampling result into buffer Q, the i-th row coefficient in the coefficient matrix M can be determined according to the current sampling count i. Further, the difference between the i-th row coefficient and the current sampling result stored in the buffer can be determined, and this difference is used as the estimation result (estimated velocity field).

[0085] Optionally, after determining the difference between the i-th row coefficient and the current sampling result stored in the buffer, an increment operation can be performed on the current sampling count (i = i + 1), that is, update the sampling count. The time step can also be updated by the following formula:

[0086]

[0087] where, can be used to represent the updated time step, can be used to represent the time step. In addition, the estimated velocity field can be used to update the input data :

[0088]

[0089] where, can be used to represent the updated input data, and v can be used to represent the estimated velocity field. If i < N, return to the step of performing linear multi-step sampling on the pre-trained model until the current sampling count is greater than or equal to the target sampling count, that is .

[0090] This embodiment accelerates the generation process by reducing the sampling count (the target sampling count is less than the count threshold), so that the model can generate high-quality outputs in a shorter time, and reducing the sampling count means reducing the consumption of computing resources.

[0091] As an alternative implementation, the method further includes: storing the current sampling result in a buffer; determining the difference between the row coefficient and the current sampling result in the buffer, including: determining the difference between the row coefficient and the current sampling result in the buffer.

[0092] In this embodiment, after obtaining the current sampling result, the obtained current sampling result can be input into buffer Q:

[0093]

[0094] where, can be used to represent the pre-trained model.

[0095] Optionally, after obtaining the current sampling result and inputting the obtained current sampling result into the buffer Q, the coefficient of the i-th row in the coefficient matrix M can be determined according to the current sampling number i. Further, the difference between the coefficient of the i-th row and the current sampling result stored in the buffer can be determined, and the difference is used as the estimated result (estimated velocity field).

[0096] This embodiment optimizes the efficiency of linear sampling by iteratively updating input data and time steps and using row coefficients in a coefficient matrix to perform difference calculations, thereby achieving higher sampling accuracy within a limited number of steps.

[0097] As an optional implementation, step S208, using the obtained sampling results to adjust the initial solver to obtain the target solver, includes: in response to the current sampling number being greater than or equal to the target sampling number, using the obtained current sampling results to adjust the initial solver to obtain the target solver.

[0098] In this embodiment, if the current sampling number is greater than or equal to the target sampling number, that is, , then use the current sampling result (latent) stored in the buffer to calculate the coefficient matrix M of the initial solver and the sampling time step Perform optimization and obtain the target solver.

[0099] Optionally, based on the current sampling results stored in the buffer, the sampling error of the linear multi-step sampling of the pre-trained model can be determined, and further based on the sampling error, the time variable r and the coefficient variable c of the initial solver can be optimized, and the optimized time variable r can be converted into a time step , and convert the optimized coefficient variable c into a coefficient matrix M, thereby obtaining the target solver, that is, the coefficient matrix M of the target solver and the sampling time step This can minimize the sampling error.

[0100] This embodiment optimizes the solver parameters through an iterative sampling process, so that the generated samples are closer to the ideal trajectory under the target sampling number, thereby improving the performance and efficiency of the solver. After each sampling step, the error between the current sampling result and the preset ideal trajectory is determined. When the sampling number reaches the target sampling number, the optimization process will ensure that the error is minimized, thereby improving the accuracy and quality of the generated samples.

[0101] As an optional implementation, in response to the current sampling number being greater than or equal to the target sampling number, the initial solver is adjusted using the current sampling result to obtain the target solver, including: in response to the current sampling number being greater than or equal to the target sampling number, calling the decoder in the model to be generated, decoding the current sampling result to obtain a decoding result; using the decoding result, adjusting the initial solver to obtain the target solver.

[0102] In this embodiment, the decoding result can be used to represent the generated data after decoding. If the current sampling number is greater than or equal to the target sampling number, that is, , then the decoder in the model to be generated is called to decode the current sampling result (latent) stored in the buffer to obtain the decoding result. The decoding result can be further used to calculate the coefficient matrix M of the initial solver and the sampling time step Perform optimization and obtain the target solver.

[0103] Optionally, after decoding the current sampling result (latent), the decoded generated data can be obtained. Further based on the decoded generated data and the real data, the sampling error of the linear multi-step sampling of the pre-trained model can be determined. Based on the sampling error, the time variable r and coefficient variable c of the initial solver can be optimized to find the coefficient matrix M and time step that minimize the sampling error. , thus obtaining the target solver.

[0104] This embodiment adjusts the parameters of the solver by determining the difference between the decoding result and the preset ideal trajectory until the parameter configuration of the target solver is obtained.

[0105] As an optional implementation, the method also includes: determining the error between the target solver and the standard solver corresponding to the pre-trained model; using the error to adjust the time variable and coefficient variable in the initial solver; converting the adjusted time variable into the time step of the initial solver, and converting the adjusted coefficient variable into the coefficient matrix of the initial solver, returning to execute from the first determination step until the target solver is in a converged state.

[0106] In this embodiment, the standard solver can be used as a reference standard for the target solver. The generated trajectory of the target solver and the ground truth trajectory of the standard solver corresponding to the pre-trained model can be determined. The error between the generated trajectory of the target solver and the ground truth trajectory of the standard solver can be determined. Further using this error, the time variable r and coefficient variable c of the initial solver can be optimized, and the optimized time variable r can be converted into a time step , and convert the optimized coefficient variables c into the coefficient matrix M until the target solver is in a converged state.

[0107] Optionally, after determining the generated trajectory of the target solver and the ground truth trajectory of the standard solver, the mean squared error (MSE) loss function (i.e., MSE Loss function) and the Huber Loss function (i.e., Huber Loss function) can be used to calculate the error of the target trajectory at the corresponding point of the ground truth trajectory.

[0108] Optionally, after obtaining the error of the target trajectory at the corresponding point of the ground truth trajectory, the time variable r and coefficient variable c of the initial solver can be optimized using the EvoLved Sign Momentum Optimizer (lion optimizer) until the target solver is in a converged state.

[0109] This embodiment can reduce the discrete error in the sampling process by optimizing the time step and the coefficient matrix, so that the sampling result is closer to the ideal or ground truth trajectory, which is particularly critical when generating complex, high-dimensional data, such as high-resolution images or long sequence texts, and helps to ensure the quality and authenticity of the generated content.

[0110] As an optional implementation, step S202, obtaining condition information of the model to be generated, includes: label encoding the category label of the model to be generated to obtain condition information, wherein the category label is used to identify the category to which the model to be generated belongs; and / or, text encoding the text prompt information of the model to be generated to obtain condition information, wherein the text prompt information is used to describe the conditions of the model to be generated.

[0111] In this embodiment, the category label can be used to identify the category to which the model to be generated belongs. The text prompt information can be used to describe the conditions of the model to be generated, and the text prompt information can also be called text description information.

[0112] Optionally, the category label and text prompt information input by the user are obtained. The category label can be encoded by a label encoder to obtain the condition information, and / or the text prompt information can be encoded by a text encoder to obtain the condition information.

[0113] This embodiment uses category labels or text prompt information to enable the generation model to better understand the generation conditions, thereby generating outputs that are more in line with specific categories or descriptions. This is crucial for generation tasks with clear requirements, such as generating images of specific categories, audio or video that matches specific text descriptions, etc.

[0114] In an embodiment of the present application, condition information of a model to be generated is obtained, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; under the condition information, a pre-trained model corresponding to the model to be generated is determined, and a noise scheduling strategy adopted by the pre-trained model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; the pre-trained model is trained using the target solver to obtain a generated model in an application scenario. In other words, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, and can determine the target solver under the noise scheduling strategy adopted by the pre-trained model, so that the target solver can be used to accelerate the generation performance of the generated model sampling, thereby achieving the improvement of the generation effect of the generated model and solving the technical problem of poor generation effect of the generated model.

[0115] The present application embodiment also provides another model processing method, Figure 3 is a flowchart of another model processing method according to an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps:

[0116] Step S302, obtaining condition information of the denoising generation model to be generated.

[0117] In the technical solution provided in the above step S302 of the present application, the condition information of the denoising generation model to be generated can be obtained. The condition information can be used to indicate the conditions that the denoising generation model to be generated needs to meet in the media application scenario, and the condition information can be called conditional coding information, which can be obtained by converting the category label or text prompt information input by the user.

[0118] Optionally, the category label or text prompt information input by the user is obtained, and the category label or text prompt information is converted into a vector form that can be understood by the denoising generation model to be generated through the encoder of the denoising generation model to be generated, thereby obtaining conditional coding information.

[0119] For example, the user inputs a category label, such as "cat", or a text prompt information, such as "a black cat napping in the sun", which can be used to define the conditional information of the denoising generative model to be generated. When the user inputs a category label, the label encoder of the denoising generative model to be generated can convert the category label into a vector form that can be understood by the denoising generative model to be generated, thereby obtaining conditional encoding information. When the user inputs text prompt information, the text encoder of the denoising generative model to be generated can convert the text into one or more vectors to obtain conditional encoding information.

[0120] Optionally, the conditional encoding information can be used to convert the category label or text prompt information provided by the user into a conditional input of the denoising generation model to be generated, so as to ensure that the user's expected conditions for the generation result of the generation model to be generated are met.

[0121] Step S304, under the condition information, determining a pre-trained denoising model corresponding to the denoising generation model to be generated, and determining a noise scheduling strategy adopted by the pre-trained denoising model.

[0122] In the technical solution provided in the above step S304 of the present application, after obtaining the condition information of the denoising generation model to be generated, the pre-trained denoising model corresponding to the denoising generation model to be generated can be determined, and the noise scheduling strategy adopted by the pre-trained denoising model can be determined. Among them, the noise scheduling strategy can be used to represent the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model.

[0123] In this embodiment, after obtaining the condition information of the denoising generation model to be generated, a pre-trained denoising model matching the denoising generation model to be generated under the condition information can be queried from the pre-trained denoising model library. It should be noted that each pre-trained denoising model in the pre-trained denoising model library has a specific training background, a data type that it is good at, and a noise scheduling strategy.

[0124] For example, if a user requests to generate an image of a "cat", an image generation model can be searched from a pre-trained denoising model library as a pre-trained denoising model, and the image generation model can be a model specially pre-trained for animal images.

[0125] Optionally, after determining the pre-trained denoising model corresponding to the denoising generation model to be generated, since each pre-trained denoising model in the pre-trained denoising model library has a specific noise scheduling strategy, the noise scheduling strategy of the pre-trained denoising model determined above can be automatically identified or read to determine the noise scheduling strategy adopted by the pre-trained denoising model.

[0126] It should be noted that the noise scheduling strategy is usually defined during the training of the pre-trained denoising model. The noise scheduling strategy describes how to gradually increase or decrease the noise during the generation process, as well as the nature of the noise (for example, Gaussian noise, uniform noise, etc.). Different pre-trained denoising models can adopt different noise scheduling strategies.

[0127] Step S306, determining the parameters of the initial solver under the noise scheduling strategy.

[0128] In the technical solution provided in the above step S306 of the present application, after determining the noise scheduling strategy adopted by the pre-trained denoising model, the parameters of the initial solver under the noise scheduling strategy can be determined. The initial solver can be a differentiable solver that can be used to optimize the parameters in the solution process.

[0129] It should be noted that the parameters of the initial solver may include at least the coefficient matrix and time step of the initial solver. The coefficient matrix can be the parameters for numerical integration to solve differential equations. In the numerical solution of differential equations, the coefficient matrix can be used to determine which numerical integration method to use, such as the Euler method, to approximate the differential equations of the model. The coefficient matrix can be represented by M. The time step can be a discretization unit on the time axis. The time step can be used to control the addition and removal of noise. Each time step corresponds to a noise level. By iterating these time steps, the model gradually generates clear and structured output samples from an initial high-noise state. The time step can be expressed by To express.

[0130] Optionally, determine the search space of the initial solver, for example, the search space can be the coefficient matrix M and the sampling time step , then the coefficient matrix M and the sampling time step can be Determine the parameters of the initial solver under the noisy scheduling strategy.

[0131] Step S308, using the parameters of the initial solver, sampling the pre-trained denoising model, and adjusting the initial solver using the obtained sampling results to obtain a target solver.

[0132] In the technical solution provided in the above step S308 of the present application, after determining the parameters of the initial solver under the noise scheduling strategy, the pre-trained denoising model can be sampled using the determined parameters of the initial solver to obtain a sampling result. The initial solver can be further adjusted using the obtained sampling result to obtain a target solver. Among them, the sampling result can be a vector in the latent space obtained after sampling the pre-trained denoising model. The performance index of the target solver is greater than the performance index threshold, and the performance index of the target solver can be used to indicate the performance quality of the target solver. For example, the performance index can be a sampling performance, and the performance index threshold can be an index threshold pre-set according to actual conditions.

[0133] It should be noted that the latent obtained by the above sampling needs to be converted into an understandable or visual form through the decoder of the pre-trained denoising model corresponding to the denoising generation model to be generated determined in the above steps. The decoding process can map the vector of the latent space back to the data space to generate the final output.

[0134] Optionally, use the coefficient matrix M of the initial solver and the sampling time step , the pre-trained denoising model can be sampled linearly in multiple steps to obtain the sampling result (i.e. latent). The coefficient matrix M of the initial solver and the sampling time step can be further calculated based on the sampling result. Adjust the target solver. For example, the coefficient matrix M of the initial solver and the sampling time step can be adjusted by sampling results. Optimize until the optimized coefficient matrix M and the sampling time step are used , the sampling error obtained after linear multi-step sampling of the pre-trained denoising model is minimized, thereby obtaining the target solver, that is, the coefficient matrix M of the target solver and the sampling time step This can minimize the sampling error.

[0135] Step S310: Use the target solver to train the pre-trained denoising model to obtain a denoising generation model in a media application scenario.

[0136] In the technical solution provided in the above step S310 of the present application, after adjusting the initial solver using the obtained sampling results to obtain the target solver, the obtained target solver can be used to train the pre-trained denoising model to obtain a denoising generation model for the media application scenario.

[0137] Optionally, after adjusting (optimizing) the parameters of the initial solver to obtain the target solver, the pre-trained denoising model is trained using the obtained target solver to obtain a denoising generation model. That is, this embodiment accelerates the generation process by optimizing the parameters of the initial solver, thereby achieving high-quality data generation using the pre-trained denoising model under low step sampling.

[0138] Through the above steps S302 to S310 of the present application, the condition information of the denoising generation model to be generated is obtained, wherein the condition information is used to indicate the conditions that the denoising generation model to be generated needs to meet in the media application scenario; under the condition information, the pre-trained denoising model corresponding to the denoising generation model to be generated is determined, and the noise scheduling strategy adopted by the pre-trained denoising model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained denoising model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; the pre-trained denoising model is trained using the target solver to obtain the denoising generation model in the media application scenario. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained denoising model to adjust the parameters of the initial solver, so as to determine the target solver under the noise scheduling strategy adopted by the pre-trained denoising model. Thus, by using the target solver, the generation performance of the generation model sampling can be accelerated, thereby improving the generation effect of the generation model and solving the technical problem of poor generation effect of the generation model.

[0139] The present application embodiment also provides another model processing method, Figure 4 is a flowchart of another model processing method according to an embodiment of the present application, such as Figure 4 As shown, the method may include the following steps:

[0140] Step S402: Acquire condition information of the model to be generated by calling the first interface.

[0141] In the technical solution provided in the above step S402 of the present application, the condition information of the model to be generated can be obtained by calling the first interface. Among them, the first interface may include a first parameter, and the parameter value of the first parameter may include condition information. The model to be generated may be a denoising model to be generated, and the denoising model to be generated may also be referred to as a denoising generation model to be generated. The condition information can be used to represent the conditions that the model to be generated needs to meet in the application scenario, and the condition information can be referred to as conditional coding information, which can be obtained by converting the category label or text prompt information input by the user. The application scenario can be a media application scenario, which is only used as an example here, and no specific restriction is made on the type of application scenario.

[0142] Optionally, the category label or text prompt information input by the user is obtained, and the category label or text prompt information is converted into a vector form that can be understood by the model to be generated through the encoder of the model to be generated, thereby obtaining conditional coding information.

[0143] For example, the user inputs a category label, such as "cat", or a text prompt information, such as "a black cat napping in the sun", which can be used to define the conditional information of the above-mentioned model to be generated. When the user inputs a category label, the label encoder of the model to be generated can convert the category label into a vector form that can be understood by the model to be generated, thereby obtaining conditional encoding information. When the user inputs text prompt information, the text encoder of the model to be generated can convert the text into one or more vectors to obtain conditional encoding information.

[0144] Optionally, conditional encoding can be used to convert category labels or text prompt information provided by the user into conditional inputs of the model to be generated, so as to ensure that the user's expected conditions for the generation result of the model to be generated are met.

[0145] Step S404: Under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model.

[0146] In the technical solution provided in the above step S404 of the present application, after obtaining the condition information of the model to be generated, under the condition information, the pre-trained model corresponding to the model to be generated can be determined, and the noise scheduling strategy adopted by the pre-trained model can be determined. Among them, the pre-trained model can be a pre-trained generation model, and the pre-trained model can also be called a pre-trained denoising model, a pre-trained denoising generation model, and a pre-trained proxy model. The noise scheduling strategy can be used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model.

[0147] In this embodiment, after obtaining the condition information of the model to be generated, a pre-trained model matching the model to be generated under the condition information can be queried from the pre-trained model library. It should be noted that each pre-trained model in the pre-trained model library has a specific training background, a data type that it is good at, and a noise scheduling strategy.

[0148] For example, if a user requests to generate an image of a "cat", an image generation model can be searched from a pre-trained model library as a pre-trained model, and the image generation model can be a model specially pre-trained for animal images.

[0149] Optionally, after determining the pre-trained model corresponding to the model to be generated, since each pre-trained model in the pre-trained model library has a specific noise scheduling strategy, the noise scheduling strategy of the pre-trained model determined above can be automatically identified or read to determine the noise scheduling strategy adopted by the pre-trained model.

[0150] It should be noted that the noise scheduling strategy is usually defined during the training of the pre-trained model. The noise scheduling strategy describes how to gradually increase or decrease the noise during the generation process, as well as the nature of the noise (for example, Gaussian noise, uniform noise, etc.). Different pre-trained models can adopt different noise scheduling strategies.

[0151] Step S406, determining the parameters of the initial solver under the noise scheduling strategy.

[0152] In the technical solution provided in the above step S406 of the present application, after determining the noise scheduling strategy adopted by the pre-trained model, the parameters of the initial solver under the noise scheduling strategy can be determined. The initial solver can be a differentiable solver, which can be used to optimize the parameters in the solution process.

[0153] It should be noted that the parameters of the initial solver may include at least the coefficient matrix and time step of the initial solver. The coefficient matrix can be the parameters for numerical integration to solve differential equations. In the numerical solution of differential equations, the coefficient matrix can be used to determine which numerical integration method to use, such as the Euler method, to approximate the differential equations of the model. The coefficient matrix can be represented by M. The time step can be a discretization unit on the time axis. The time step can be used to control the addition and removal of noise. Each time step corresponds to a noise level. By iterating these time steps, the model gradually generates clear and structured output samples from an initial high-noise state. The time step can be expressed by To express.

[0154] Optionally, determine the search space of the initial solver, for example, the search space can be the coefficient matrix M and the sampling time step , then the coefficient matrix M and the sampling time step can be Determine the parameters of the initial solver under the noisy scheduling strategy.

[0155] Step S408, using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain the target solver.

[0156] In the technical solution provided in the above step S408 of the present application, after determining the parameters of the initial solver under the noise scheduling strategy, the pre-trained model can be sampled using the determined parameters of the initial solver to obtain a sampling result. The initial solver can be further adjusted using the obtained sampling result to obtain a target solver. Among them, the sampling result can be a vector in the latent space obtained after sampling the pre-trained model. The performance index of the target solver is greater than the performance index threshold, and the performance index of the target solver can be used to indicate the performance quality of the target solver. For example, the performance index can be a sampling performance, and the performance index threshold can be an index threshold pre-set according to actual conditions.

[0157] It should be noted that the latent obtained by the above sampling needs to be converted into an understandable or visual form through the decoder of the pre-trained model corresponding to the model to be generated determined in the above steps. The decoding process can map the vector of the latent space back to the data space to generate the final output.

[0158] Optionally, use the coefficient matrix M of the initial solver and the sampling time step , the pre-trained model can be sampled linearly in multiple steps to obtain the sampling result (i.e. latent). The coefficient matrix M of the initial solver and the sampling time step can be further calculated based on the sampling result. Adjust the target solver. For example, the coefficient matrix M of the initial solver and the sampling time step can be adjusted by sampling results. Optimize until the optimized coefficient matrix M and the sampling time step are used , the sampling error obtained after linear multi-step sampling of the pre-trained model is minimized, thereby obtaining the target solver, that is, the coefficient matrix M of the target solver and the sampling time step This can minimize the sampling error.

[0159] Step S410, using the target solver to train the pre-trained model to obtain a generation model in the application scenario.

[0160] In the technical solution provided in the above step S410 of the present application, after the initial solver is adjusted using the obtained sampling results to obtain the target solver, the pre-trained model can be trained using the obtained target solver to obtain a generative model in the application scenario. The generative model can be a generated denoising model, which can also be called a denoising generative model.

[0161] Optionally, after adjusting (optimizing) the parameters of the initial solver to obtain the target solver, the pre-trained model is trained using the obtained target solver to obtain the generated model. That is, this embodiment accelerates the generation process by optimizing the parameters of the initial solver, thereby achieving the generation of high-quality data using the pre-trained model under low-step sampling without changing the structure or parameters of the model itself.

[0162] Step S412, outputting the generation model by calling the second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the generation model.

[0163] In the technical solution provided in the above step S412 of the present application, after the pre-trained model is trained using the target solver to obtain the generated model in the application scenario, the generated model can be output by calling the second interface. The second interface may include a second parameter, and the parameter value of the second parameter may include the generated model.

[0164] Through the above steps S402 to S412 of the present application, the condition information of the model to be generated is obtained by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the condition information, and the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, the pre-trained model corresponding to the model to be generated is determined, and the noise scheduling strategy adopted by the pre-trained model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the sampling results to obtain a target solver; the pre-trained model is trained using the target solver to obtain a generated model in the application scenario; the generated model is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the generated model. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, so as to determine the target solver under the noise scheduling strategy adopted by the pre-trained model. Then, by using the target solver, the generation performance of the generation model sampling can be accelerated, thereby improving the generation effect of the generation model and solving the technical problem of poor generation effect of the generation model.

[0165] Figure 5 is a schematic diagram of a processing system of a model according to an embodiment of the present application, such as Figure 5 As shown, the processing system 500 of the model may include: a client 502 and a server 504 .

[0166] The client 502 is used to send condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario.

[0167] Optionally, the model to be generated may be a denoising model to be generated, and the denoising model to be generated may also be referred to as a denoising generation model to be generated. The conditional information may be used to indicate the conditions that the model to be generated needs to meet in an application scenario, and the conditional information may be referred to as conditional coding information, which may be obtained by converting a category label or text prompt information input by a user. The application scenario may be a media application scenario, which is only used as an example here, and the type of application scenario is not specifically limited.

[0168] Optionally, the category label or text prompt information input by the user is obtained, and the category label or text prompt information is converted into a vector form that can be understood by the model to be generated through the encoder of the model to be generated, thereby obtaining conditional coding information.

[0169] For example, the user inputs a category label, such as "cat", or a text prompt information, such as "a black cat napping in the sun", which can be used to define the conditional information of the above-mentioned model to be generated. When the user inputs a category label, the label encoder of the model to be generated can convert the category label into a vector form that can be understood by the model to be generated, thereby obtaining conditional encoding information. When the user inputs text prompt information, the text encoder of the model to be generated can convert the text into one or more vectors to obtain conditional encoding information.

[0170] Optionally, the conditional encoding information can be used to convert the category label or text prompt information provided by the user into a conditional input of the model to be generated, so as to ensure that the user's expected conditions for the generation result of the model to be generated are met.

[0171] The server 504 is connected to the client and is used to determine the pre-trained model corresponding to the model to be generated under the condition information, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determine the parameters of the initial solver under the noise scheduling strategy; use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained model to obtain a generated model in the application scenario; and send the generated model to the client.

[0172] Optionally, the condition information of the model to be generated is obtained at the client 502, and the obtained condition information of the model to be generated can be sent to the server 504. Among them, the above-mentioned pre-trained model can be a pre-trained generation model, and the pre-trained model can also be called a pre-trained denoising model, a pre-trained denoising generation model, and a pre-trained proxy model. The noise scheduling strategy can be used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model. The initial solver can be a differentiable solver, which can be used to optimize the parameters in the solution process. The sampling result can be a vector in the latent space obtained after sampling the pre-trained model. The performance index of the target solver is greater than the performance index threshold, and the performance index of the target solver can be used to represent the performance quality of the target solver. For example, the performance index can be the sampling performance, and the performance index threshold can be the index threshold pre-set according to the actual situation. The generation model can be a generated denoising model, which can also be called a denoising generation model.

[0173] Optionally, after obtaining the condition information of the model to be generated, a pre-trained model matching the model to be generated under the condition information can be queried from the pre-trained model library. It should be noted that each pre-trained model in the pre-trained model library has a specific training background, a data type that it is good at, and a noise scheduling strategy.

[0174] For example, if a user requests to generate an image of a "cat", an image generation model can be searched from a pre-trained model library as a pre-trained model, and the image generation model can be a model specially pre-trained for animal images.

[0175] Optionally, after determining the pre-trained model corresponding to the model to be generated, since each pre-trained model in the pre-trained model library has a specific noise scheduling strategy, the noise scheduling strategy of the pre-trained model determined above can be automatically identified or read to determine the noise scheduling strategy adopted by the pre-trained model.

[0176] It should be noted that the noise scheduling strategy is usually defined during the training of the pre-trained model. The noise scheduling strategy describes how to gradually increase or decrease the noise during the generation process, as well as the nature of the noise (for example, Gaussian noise, uniform noise, etc.). Different pre-trained models can adopt different noise scheduling strategies.

[0177] Optionally, determine the search space of the initial solver, for example, the search space can be the coefficient matrix M and the sampling time step , then the coefficient matrix M and the sampling time step can be Determine the parameters of the initial solver under the noisy scheduling strategy.

[0178] Optionally, use the coefficient matrix M of the initial solver and the sampling time step , the pre-trained model can be sampled linearly in multiple steps to obtain the sampling result (i.e. latent). The coefficient matrix M of the initial solver and the sampling time step can be further calculated based on the sampling result. Adjust the target solver. For example, the coefficient matrix M of the initial solver and the sampling time step can be adjusted by sampling results. Optimize until the optimized coefficient matrix M and the sampling time step are used , the sampling error obtained after linear multi-step sampling of the pre-trained model is minimized, thereby obtaining the target solver, that is, the coefficient matrix M of the target solver and the sampling time step This can minimize the sampling error.

[0179] Optionally, after adjusting (optimizing) the parameters of the initial solver to obtain the target solver, the pre-trained model is trained using the obtained target solver to obtain the generated model. That is, this embodiment accelerates the generation process by optimizing the parameters of the initial solver, thereby achieving high-quality data generation using the pre-trained model under low step sampling.

[0180] In this system, the condition information of the model to be generated is sent through the client 502, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; the server 504 is connected to the client, and the server determines the pre-trained model corresponding to the model to be generated under the condition information, and determines the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determines the parameters of the initial solver under the noise scheduling strategy; samples the pre-trained model using the parameters of the initial solver, and adjusts the initial solver using the obtained sampling results to obtain the target solver; trains the pre-trained model using the target solver to obtain the generated model in the application scenario; and sends the generated model to the client. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, and can determine the target solver under the noise scheduling strategy adopted by the pre-trained model, so that the target solver can be used to accelerate the generation performance of the generated model sampling, thereby achieving the improvement of the generation effect of the generated model and solving the technical problem of poor generation effect of the generated model.

[0181] The technical solution of the embodiment of the present disclosure is further introduced below with examples in combination with preferred implementation modes.

[0182] Currently, in the research of generative models, pre-trained models have attracted widespread attention due to their powerful generative capabilities. Pre-trained models usually learn data distribution through an iterative denoising process of noisy data and generate new, seemingly real data samples. However, when the pre-trained model uses low-step sampling, the error of the generated results of the pre-trained generative model will increase significantly, seriously reducing the generation performance, resulting in the problem of poor generation effect of the generative model.

[0183] In order to solve the above problems, the present application provides a generative model sampling acceleration algorithm based on a differentiable solver search, which searches for a target solver under a given noise schedule based on a pre-trained denoising generative model to accelerate the generation performance of the denoising generative model under low-step sampling. The target solver (solver) found by the differentiable solver search algorithm can generate high-quality samples in the denoising generative model with fewer steps (i.e., low-step sampling). Compared with the manually designed solver, the solver searched in this embodiment can more accurately approximate the ideal integral trajectory of the pre-trained model, thereby reducing errors in the generation process and improving the quality of the generated samples.

[0184] It should be noted that, given the pre-trained noise schedule and pre-trained model, the integral sampling expression of the diffusion model framework based on linear mixture in the ideal case is:

[0185]

[0186] in, Can be used to represent the current time step for linear sampling of the pre-trained model. can be used to represent the updated time step, It can be used to represent the discrete intermediate values ​​of the integral of the ordinary differential equation (ODE). can be used to represent the discrete update value of the ODE integral after the next time step, Can be used to represent continuous intermediate values, It can be used to represent the velocity field estimation function trained based on the diffusion model framework. It can be a neural network, and dt can be used to represent the time interval. In the actual generation process, it is inevitable to introduce discrete errors. In the case of low step numbers, the error will be further amplified, resulting in poor sampling performance at low step numbers. Therefore, this embodiment searches for the target solver at low step numbers so that the error between the generated result and the ideal result is small enough:

[0187]

[0188] in, Can be used to represent the sampler (i.e., target solver) obtained by the search. Can be used to represent random noise and is the starting point for integration.

[0189] The above method of this embodiment is further introduced below.

[0190] First, the solver is parameterized, assuming that the number of sampling steps for the search is N. The search space of the solver is the coefficient matrix M and the sampling time step , the solver's optimization space is the time variable r and the coefficient variable c. The time variable r and the coefficient variable c can be converted into time steps respectively And coefficient matrix M. By continuously optimizing the time variable r and coefficient variable c in the optimization space, the time step that minimizes the sampling error can be found and coefficient matrix M.

[0191] Optionally, the time variable r is initialized, and the r vector is initialized to be uniformly distributed, and the dimension of the uniform distribution may be N. The initialized r vector may be further converted to a time step by a Softmax function , that is, = Softmax(r). Initialize the coefficient variable c and initialize the c vector to the following form:

[0192]

[0193] Among them, the above matrix has a total of (N*(N-1)) / 2 numbers. The matrix is ​​accessed by indexing through superscripts and subscripts. The superscript can be represented by i and the subscript can be represented by j. The superscript i is always smaller than the subscript j. The subscript j ranges from 0 to N-1, and the diagonal part of the matrix is ​​determined by implicit constraints. The initialized c vector can be further converted into a coefficient matrix M, that is, the above matrix is ​​determined as the coefficient matrix M.

[0194] Secondly, the solver is searched, using any pre-trained denoising model (pre-trained proxy model) under the given noise scheduler, and the sampler (i.e., Euler ODE sampler) for numerical integration of ordinary differential equations (ODE) using the first-order Euler method (Euler) for L-step sampling, where L is much larger than the target solver The number of steps N is usually selected as L = 100 as the ground truth trajectory.

[0195] Optionally, the coefficient matrix M obtained by transformation and the sampling time step are used , you can perform linear multi-step sampling on the pre-trained model. You can initialize an empty buffer Q and initialize the number of sampling times i=0. Using the current time step and the input data of the pre-trained model at the current time step to perform linear multi-step sampling on the pre-trained model to obtain the current sampling result (latent), and input the obtained current sampling result into buffer Q:

[0196]

[0197] Furthermore, according to the current sampling times i, the coefficients of the i-th row in the coefficient matrix M can be determined. The difference between the coefficients of the i-th row and the current sampling result stored in the buffer can be determined, and this difference is used as the estimation result (estimated velocity field). The current sampling times can be incremented (i = i + 1), that is, the sampling times are updated. The time step can also be updated through the following formula:

[0198]

[0199] where can be used to represent the updated time step, can be used to represent the time step size. In addition, the estimated velocity field can be used to update the input data :

[0200]

[0201] where can be used to represent the updated input data, and v can be used to represent the estimated velocity field. If i < N, return to the step of performing linear multi-step sampling on the pre-trained model until the current sampling times is greater than or equal to the target sampling times, that is to obtain the generation trajectory of the target solver.

[0202] Align the generation trajectory of the target solver with the ground truth trajectory, that is, use the MSE Loss function and the Huber Loss function to calculate the error of the target trajectory at the corresponding points of the ground truth trajectory. Further, the lion optimizer can be used to optimize the time variable r and the coefficient variable c of the initial solver until the target solver is in a convergent state.

[0203] Finally, based on the searched differential solver, perform the generation process of sampling, input the user-defined class label or text prompt, convert the class label through label encoding, or convert the text prompt through the text encoder to obtain the conditional encoding, select the corresponding generation model target generation steps, and the solver parameters searched under the noise scheduler adopted for this generation model. The transformed coefficient matrix M and the sampling time step size ​, execute the above sampling process, and decode the sampled latent through the decoder in the selected generation model.

[0204] The solver obtained by the search is verified on the pre-trained denoising generation model, that is, the sampling performance of the solver obtained by the search is tested in different pre-trained denoising generation models, which can ensure that the performance of the solver is verified in different types of pre-trained denoising generation models, thereby improving the versatility and reliability of the solver. It should be noted that Fréchet Inception Distance (FID) is a metric used to evaluate the quality of samples generated by the generation model. The lower the FID value, the closer the generated samples are to the samples of the real data set in distribution, that is, the higher the quality of the generated samples. The FID effect of the solver obtained by the search in this embodiment is better than that of the previously manually designed solver, that is, the quality of the samples generated by the searched solver is higher than the sample quality when the previously manually designed solver is used in the case of low-step sampling.

[0205] On the image dataset, the 10-step FID performance of the pre-trained denoising generative model was improved by using the solver under noise scheduling. That is, under the above noise scheduling strategy, the searched solver can significantly improve the performance of the generative model with fewer sampling steps.

[0206] In an embodiment of the present application, condition information of a model to be generated is obtained, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; under the condition information, a pre-trained model corresponding to the model to be generated is determined, and a noise scheduling strategy adopted by the pre-trained model is determined, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; the parameters of the initial solver under the noise scheduling strategy are determined; the pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; the pre-trained model is trained using the target solver to obtain a generated model in an application scenario. In other words, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, and can determine the target solver under the noise scheduling strategy adopted by the pre-trained model, so that the target solver can be used to accelerate the generation performance of the generated model sampling, thereby achieving the improvement of the generation effect of the generated model and solving the technical problem of poor generation effect of the generated model.

[0207] Figure 6 is a structural block diagram of a computing environment of a model processing method according to an embodiment of the present application, such as Figure 6As shown, computing environment 601 includes multiple computing nodes (such as servers) running on a distributed network (shown as 610-1, 610-2, ..., in the figure). The computing nodes all contain local processing and memory resources, and end users 602 can remotely run applications or store data in computing environment 601. Applications can be provided as multiple services 620-1, 620-2, 620-3 and 620-4 in computing environment 601, representing services "A", "D", "E" and "H" respectively.

[0208] The end user 602 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 602's provision and / or request can be provided to the entry gateway 630. The entry gateway 630 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 601).

[0209] Services are provided or deployed based on various virtualization technologies supported by the computing environment 601. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (Operating System, referred to as OS) so that multiple workloads can run on a single operating system instance.

[0210] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 6 As shown, service 620-2 can be equipped with one or more Pods 640-1, 640-2, ..., 640-N (collectively referred to as Pods). Pods can include a proxy 645 and one or more containers 642-1, 642-2, ..., 642-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 645 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with Pods similar to Pods.

[0211] During operation, executing a user request from the end user 602 may require calling one or more services in the computing environment 601, and executing one or more functions of a service may require calling one or more functions of another service. Figure 6As shown, service "A" 620-1 receives a user request from end user 602 from ingress gateway 630, service "A" 620-1 may call service "D" 620-2, and service "D" 620-2 may request service "E" 620-3 to perform one or more functions.

[0212] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0213] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 2 The model processing method shown is a model processing device.

[0214] Figure 7 is a schematic diagram of a processing device of a model according to an embodiment of the present application, such as Figure 7 As shown, the processing device 700 of the model may include: a first acquisition unit 702 , a first determination unit 704 , a second determination unit 706 , a first sampling unit 708 and a first training unit 710 .

[0215] The first acquisition unit 702 is used to acquire condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario.

[0216] The first determination unit 704 is used to determine the pre-trained model corresponding to the model to be generated under the condition information, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model.

[0217] The second determining unit 706 is used to determine the parameters of the initial solver under the noise scheduling strategy.

[0218] The first sampling unit 708 is used to sample the pre-trained model using the parameters of the initial solver, and adjust the initial solver using the obtained sampling results to obtain a target solver.

[0219] The first training unit 710 is used to train the pre-trained model using the target solver to obtain a generated model in an application scenario.

[0220] It should be noted that the first acquisition unit 702, the first determination unit 704, the second determination unit 706, the first sampling unit 708 and the first training unit 710 correspond to steps S202 to S210, and the five units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above disclosed contents. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b..., 1002n), and the above units can also be run in a computer terminal A as part of the device.

[0221] In the processing device of the model, the condition information of the model to be generated is obtained by the first acquisition unit 702, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario. The first determination unit 704 determines the pre-trained model corresponding to the model to be generated under the condition information, and determines the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model. The parameters of the initial solver under the noise scheduling strategy are determined by the second determination unit 706. The pre-trained model is sampled by the first sampling unit 708 using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain the target solver. The pre-trained model is trained by the first training unit 710 using the target solver to obtain the generated model in the application scenario. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, so as to determine the target solver under the noise scheduling strategy adopted by the pre-trained model. Then, by using the target solver, the generation performance of the generation model sampling can be accelerated, thereby improving the generation effect of the generation model and solving the technical problem of poor generation effect of the generation model.

[0222] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 3 The model processing method shown is a model processing device.

[0223] Figure 8 is a schematic diagram of a processing device of another model according to an embodiment of the present application, such as Figure 8 As shown, the processing device 800 of the model may include: a second acquisition unit 802 , a third determination unit 804 , a fourth determination unit 806 , a second sampling unit 808 and a second training unit 810 .

[0224] The second acquisition unit 802 is used to acquire condition information of the denoising generation model to be generated, wherein the condition information is used to indicate the conditions that the denoising generation model to be generated needs to meet in a media application scenario.

[0225] The third determination unit 804 is used to determine the pre-trained denoising model corresponding to the denoising generation model to be generated under the conditional information, and determine the noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model.

[0226] The fourth determining unit 806 is used to determine the parameters of the initial solver under the noise scheduling strategy.

[0227] The second sampling unit 808 is used to sample the pre-trained denoising model using the parameters of the initial solver, and adjust the initial solver using the obtained sampling results to obtain a target solver.

[0228] The second training unit 810 is used to train the pre-trained denoising model using the target solver to obtain a denoising generation model in a media application scenario.

[0229] Here, the second acquisition unit 802, the third determination unit 804, the fourth determination unit 806, the second sampling unit 808, and the second training unit 810 correspond to steps S302 to S310, and the five units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above disclosed contents. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b..., 1002n), and the above units can also be run in a computer terminal A as part of the device.

[0230] In the processing device of the model, the condition information of the denoising generation model to be generated is obtained by the second acquisition unit 802, wherein the condition information is used to indicate the conditions that the denoising generation model to be generated needs to meet in the media application scenario. The third determination unit 804 determines the pre-trained denoising model corresponding to the denoising generation model to be generated under the condition information, and determines the noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model. The fourth determination unit 806 determines the parameters of the initial solver under the noise scheduling strategy. The second sampling unit 808 samples the pre-trained denoising model using the parameters of the initial solver, and adjusts the initial solver using the obtained sampling results to obtain the target solver. The second training unit 810 trains the pre-trained denoising model using the target solver to obtain the denoising generation model in the media application scenario. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, so as to determine the target solver under the noise scheduling strategy adopted by the pre-trained model. Then, by using the target solver, the generation performance of the generation model sampling can be accelerated, thereby improving the generation effect of the generation model and solving the technical problem of poor generation effect of the generation model.

[0231] According to an embodiment of the present application, there is also provided a method for implementing the above Figure 4 The model processing method shown is a model processing device.

[0232] Fig. 9 is a schematic diagram of a processing device according to another model of an embodiment of the present application, such as Fig. 9 As shown, the processing device 900 of the model may include: a third acquisition unit 902 , a fifth determination unit 904 , a sixth processing unit 906 , a third sampling unit 908 , a third training unit 910 and an output unit 912 .

[0233] The third acquisition unit 902 is used to obtain condition information of the model to be generated by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter includes condition information, and the condition information is used to represent the conditions that the model to be generated needs to meet in the application scenario.

[0234] The fifth determination unit 904 is used to determine the pre-trained model corresponding to the model to be generated under the condition information, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model.

[0235] The sixth processing unit 906 is used to determine the parameters of the initial solver under the noise scheduling strategy.

[0236] The third sampling unit 908 is used to sample the pre-trained model using the parameters of the initial solver, and adjust the initial solver using the obtained sampling results to obtain a target solver.

[0237] The third training unit 910 is used to train the pre-trained model using the target solver to obtain a generated model in an application scenario.

[0238] The output unit 912 is used to output the generated model by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the generated model.

[0239] It should be noted that the third acquisition unit 902, the fifth determination unit 904, the sixth processing unit 906, the third sampling unit 908, the third training unit 910 and the output unit 912 correspond to steps S402 to S412, and the six units are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the above disclosed contents. It should be noted that the above units can be hardware components or software components stored in a memory (e.g., memory 1004) and processed by one or more processors (e.g., processors 1002a, 1002b..., 1002n), and the above units can also be run in a computer terminal A as part of the device.

[0240] In the processing device of the model, the third acquisition unit 902 is used to obtain the condition information of the model to be generated by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the condition information, and the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario. The fifth determination unit 904 determines the pre-trained model corresponding to the model to be generated under the condition information, and determines the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rule for adjusting the noise of the pre-trained model during the training of the pre-trained model. The sixth processing unit 906 determines the parameters of the initial solver under the noise scheduling strategy. The third sampling unit 908 samples the pre-trained model using the parameters of the initial solver, and adjusts the initial solver using the obtained sampling results to obtain the target solver. The pre-trained model is trained by the third training unit 910 using the target solver to obtain the generated model in the application scenario. The generated model is output by calling the second interface through the output unit 912, wherein the second interface includes the second parameter, and the parameter value of the second parameter includes the generated model. That is to say, the embodiment of the present application uses the sampling results obtained by sampling the pre-trained model to adjust the parameters of the initial solver, so as to determine the target solver under the noise scheduling strategy adopted by the pre-trained model. Then, by using the target solver, the generation performance of the generation model sampling can be accelerated, thereby improving the generation effect of the generation model and solving the technical problem of poor generation effect of the generation model.

[0241] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0242] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0243] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the model processing method: obtaining condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, determining the pre-trained model corresponding to the model to be generated, and determining the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determining the parameters of the initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain a target solver; training the pre-trained model using the target solver to obtain a generated model in the application scenario.

[0244] Optionally, Fig.10 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.10 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1002 , a memory 1004 and a transmission device 1006 .

[0245] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the processing method and device of the model in the embodiment of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the processing method of the above-mentioned model. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0246] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain condition information of the denoising generation model to be generated, wherein the condition information is used to indicate the conditions that the denoising generation model to be generated needs to meet in the media application scenario; under the condition information, determine the pre-trained denoising model corresponding to the denoising generation model to be generated, and determine the noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model; determine the parameters of the initial solver under the noise scheduling strategy; use the parameters of the initial solver to sample the pre-trained denoising model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained denoising model to obtain a denoising generation model in the media application scenario.

[0247] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the condition information of the model to be generated by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the condition information, and the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determine the parameters of the initial solver under the noise scheduling strategy; use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained model to obtain a generated model in the application scenario; output the generated model by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the generated model.

[0248] It can be understood by those skilled in the art that Fig.10 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, abbreviated as MID), a personal access display (Personal Access Display, abbreviated as PAD), and other terminal devices. Fig.10 It does not limit the structure of the above-mentioned computer terminal A. For example, the computer terminal A may also include Fig.10 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Fig.10 Different configurations are shown.

[0249] A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0250] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the model processing method provided in the first embodiment.

[0251] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0252] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining condition information of the model to be generated, wherein the condition information is used to represent the conditions that the model to be generated needs to meet in the application scenario; under the condition information, determining a pre-trained model corresponding to the model to be generated, and determining a noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determining the parameters of the initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain a target solver; training the pre-trained model using the target solver to obtain a generated model in the application scenario.

[0253] As an optional example, a computer-readable storage medium is configured to store program code for executing the following steps: obtaining condition information of a denoising generation model to be generated, wherein the condition information is used to indicate conditions that the denoising generation model to be generated needs to meet in a media application scenario; under the condition information, determining a pre-trained denoising model corresponding to the denoising generation model to be generated, and determining a noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to indicate rules for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model; determining parameters of an initial solver under the noise scheduling strategy; using the parameters of the initial solver to sample the pre-trained denoising model, and using the obtained sampling results to adjust the initial solver to obtain a target solver; using the target solver to train the pre-trained denoising model to obtain a denoising generation model in a media application scenario.

[0254] As an optional example, a computer-readable storage medium is configured to store program code for executing the following steps: obtaining condition information of a model to be generated by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes condition information, and the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; under the condition information, determining a pre-trained model corresponding to the model to be generated, and determining a noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to indicate the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determining parameters of an initial solver under the noise scheduling strategy; using the parameters of the initial solver, sampling the pre-trained model, and adjusting the initial solver using the obtained sampling results to obtain a target solver; using the target solver to train the pre-trained model to obtain a generated model in the application scenario; outputting the generated model by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the generated model.

[0255] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0256] Optionally, the computer program product may include a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium may be used to store a computer program. When the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0257] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0258] Fig.11 It is a block diagram of an electronic device according to a processing method of a model of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0259] like Fig.11As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0260] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0261] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the processing method of the model. For example, in some embodiments, the processing method of the model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the processing method of the model described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the processing method of the model in any other appropriate manner (eg, by means of firmware).

[0262] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0263] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0264] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Fig.12 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a model processing method according to an embodiment of the present application, such as Fig.12As shown, the computer terminal 120 (or mobile device) may include one or more (1202a, 1202b, ..., 1202n are used to illustrate in the figure) processors 1202 (the processor 1202 may include but is not limited to a processing device such as a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA)), a memory 1204 for storing data, and a transmission device 1206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Fig.12 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.12 More or fewer components as shown, or with Fig.12 Different configurations are shown.

[0265] Fig.12 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned computer terminal 120 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Fig.12 The block diagram shows the use of the above Fig.12 The computer terminal 120 (or mobile device) is shown as an embodiment of a computing node in the computing environment 701 .

[0266] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or for use in combination with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory for short), an optical fiber, a portable compact disc read-only memory (CD-ROM for short), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0267] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD), a monitor for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a path ball), through which the user may provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0268] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0269] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0270] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0271] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0272] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0273] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0274] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0275] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory, random access memory, mobile hard disk, disk or optical disk, etc., which can store program code.

[0276] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model processing method, characterized in that: include: Acquire condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in an application scenario; Under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent a rule for adjusting the noise of the pre-trained model during the training of the pre-trained model; Determining parameters of an initial solver under the noise scheduling strategy, wherein the initial solver is a differentiable solver, and the parameters include a time step of the initial solver and a coefficient matrix of the initial solver; The pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; Using the target solver to train the pre-trained model to obtain a generation model in the application scenario, wherein the pre-trained model is at least used to process text prompt information in a media application scenario; The pre-trained model is sampled using the parameters of the initial solver, including: linearly sampling the pre-trained model using the time step and the coefficient matrix to obtain the sampling result.

2. The method according to claim 1, characterized in that Determining parameters of an initial solver under the noise scheduling strategy includes: determining at least one variable to be adjusted in the initial solver; The variables to be adjusted are converted into parameters of the initial solver.

3. The method according to claim 2, characterized in that The variables to be adjusted include time variables and coefficient variables, and converting the variables to be adjusted into parameters of the initial solver includes: Initializing the time variable and converting the initialized time variable into the time step of the initial solver; The coefficient variables are initialized, and the initialized coefficient variables are converted into the coefficient matrix of the initial solver.

4. The method according to claim 1, characterized in that: Using the time step and the coefficient matrix, linear sampling is performed on the pre-trained model to obtain the sampling result, including: A first determination step is to determine a current sampling number for linear sampling of the pre-trained model, and determine a current time step corresponding to the current sampling number; A sampling step, in response to the current sampling number being less than the target sampling number, linearly sampling the pre-trained model using the current time step and the input data of the pre-trained model at the current time step to obtain a current sampling result, wherein the target sampling number is less than a number threshold; The method further comprises: a second determination step of determining the row coefficient corresponding to the current sampling number in the coefficient matrix, and determining the difference between the row coefficient and the current sampling result; The updating step increments the current sampling number, updates the current time step using the time step, and updates the input data using the difference, the time step, and the input data, and returns to execute from the sampling step until the current sampling number is greater than or equal to the target sampling number.

5. The method according to claim 4, characterized in that The method further comprises: Storing the current sampling result in a buffer; Determining the difference between the row coefficient and the current sampling result in the buffer includes: determining the difference between the row coefficient and the current sampling result in the buffer.

6. The method according to claim 4, characterized in that The initial solver is adjusted using the obtained sampling results to obtain a target solver, including: In response to the current sampling number being greater than or equal to the target sampling number, the initial solver is adjusted using the current sampling result to obtain the target solver.

7. The method according to claim 6, characterized in that In response to the current sampling number being greater than or equal to the target sampling number, adjusting the initial solver using the obtained current sampling result to obtain the target solver, including: In response to the current sampling number being greater than or equal to the target sampling number, calling a decoder in the to-be-generated model to decode the current sampling result to obtain a decoding result; The initial solver is adjusted using the decoding result to obtain the target solver.

8. The method according to claim 6, characterized in that The method further comprises: Determining an error between the target solver and a standard solver corresponding to the pre-trained model; Using the error, adjusting the time variable and coefficient variable in the initial solver; The adjusted time variable is converted into the time step of the initial solver, and the adjusted coefficient variable is converted into the coefficient matrix of the initial solver, and the execution is returned to start from the first determination step until the target solver is in a converged state.

9. The method according to any one of claims 1 to 8, characterized in that Get the condition information of the model to be generated, including: Performing label encoding on the category label of the model to be generated to obtain the condition information, wherein the category label is used to identify the category to which the model to be generated belongs; and / or, The text prompt information of the model to be generated is text-encoded to obtain the condition information, wherein the text prompt information is used to describe the condition of the model to be generated.

10. A model processing method, characterized in that: include: Acquire condition information of a denoising generation model to be generated, wherein the condition information is used to indicate conditions that the denoising generation model to be generated needs to satisfy in a media application scenario; Under the condition information, determine a pre-trained denoising model corresponding to the denoising generation model to be generated, and determine a noise scheduling strategy adopted by the pre-trained denoising model, wherein the noise scheduling strategy is used to represent a rule for adjusting the noise of the pre-trained denoising model during the training of the pre-trained denoising model; Determining parameters of an initial solver under the noise scheduling strategy, wherein the initial solver is a differentiable solver, and the parameters include a time step of the initial solver and a coefficient matrix of the initial solver; The pre-trained denoising model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; The pre-trained denoising model is trained by using the target solver to obtain a denoising generation model in the media application scenario, wherein the pre-trained denoising model is at least used to process text prompt information in the media application scenario; The pre-trained denoising model is sampled using the parameters of the initial solver, including: linearly sampling the pre-trained denoising model using the time step and the coefficient matrix to obtain the sampling result.

11. A method for processing a model, characterized in that: include: Acquire condition information of the model to be generated by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes the condition information, and the condition information is used to indicate a condition that the model to be generated needs to satisfy in an application scenario; Under the condition information, determine the pre-trained model corresponding to the model to be generated, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent a rule for adjusting the noise of the pre-trained model during the training of the pre-trained model; Determining parameters of an initial solver under the noise scheduling strategy, wherein the initial solver is a differentiable solver, and the parameters include a time step of the initial solver and a coefficient matrix of the initial solver; The pre-trained model is sampled using the parameters of the initial solver, and the initial solver is adjusted using the obtained sampling results to obtain a target solver; Using the target solver to train the pre-trained model to obtain a generation model in the application scenario, wherein the pre-trained model is at least used to process text prompt information in a media application scenario; Outputting the generated model by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the generated model; The pre-trained model is sampled using the parameters of the initial solver, including: linearly sampling the pre-trained model using the time step and the coefficient matrix to obtain the sampling result.

12. A model processing system, characterized in that: include: The client is used to send condition information of the model to be generated, wherein the condition information is used to indicate the conditions that the model to be generated needs to meet in the application scenario; The server is connected to the client and is used to determine the pre-trained model corresponding to the model to be generated under the condition information, and determine the noise scheduling strategy adopted by the pre-trained model, wherein the noise scheduling strategy is used to represent the rules for adjusting the noise of the pre-trained model during the training of the pre-trained model; determine the parameters of the initial solver under the noise scheduling strategy, wherein the initial solver is a differentiable solver, and the parameters include the time step of the initial solver and the coefficient matrix of the initial solver; use the parameters of the initial solver to sample the pre-trained model, and use the obtained sampling results to adjust the initial solver to obtain a target solver; use the target solver to train the pre-trained model to obtain a generation model in the application scenario, wherein the pre-trained model is at least used to process text prompt information in a media application scenario; send the generation model to the client; The pre-trained model is sampled using the parameters of the initial solver, including: linearly sampling the pre-trained model using the time step and the coefficient matrix to obtain the sampling result.

13. A computing device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 11 when running.

14. An electronic device, characterized in that: include: A memory storing an executable program; A processor, connected to the memory via a bus, and configured to run the program, wherein the program executes the method described in any one of claims 1 to 11 when running.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 11.

16. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images