Image generation model training method, image generation method, device and electronic equipment

Abstract

Embodiments of the present application provide an image generation model training method, an image generation method, an image generation device and an electronic device, and relate to the technical field of image generation. The image generation model training method is applied to a center server in an image generation model training system. The image generation model training system further includes a plurality of training servers. The method includes: obtaining a plurality of first training tasks; detecting the running states of the plurality of training servers; for each first training task, selecting a first training server for executing the first training task from the plurality of training servers in an idle state; sending the first training task to the first training server, so that the first training server executes the received first training task, obtains a trained second image generation model, and stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the center server, thereby improving the training efficiency and meeting the user demand.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and in particular to an image generation model training method, an image generation method, an apparatus, and an electronic device. Background Technology

[0002] In existing technologies, image generation models are trained on an initial model structure using training samples of sample images containing various sample types. Therefore, these models can process images of the sample types. However, when the image to be processed is not a sample type, it cannot be processed using the image generation model. To obtain an image generation model capable of processing multiple types of images, it is necessary to train the initial model structure using training samples containing various sample images.

[0003] However, if there are a large number of training samples containing various types of sample images, the training time for the initial structure model based on these training samples will be too long, resulting in low efficiency in generating the trained image generation model. Summary of the Invention

[0004] The purpose of this invention is to provide an image generation model training method, an image generation method, an apparatus, and an electronic device to improve training efficiency and meet user needs. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, an image generation model training method is provided. The method is applied to a central server in an image generation model training system, which further includes multiple training servers. The method includes:

[0006] Multiple first training tasks are acquired; each first training task represents training a first image generation model corresponding to that first training task; the multiple first training tasks correspond to different image types; the running status of multiple training servers is detected; the running status of the training servers includes: working status and idle status; working status indicates that the training server is executing a first training task; idle status indicates that the training server is not executing a first training task; for each first training task, a first training server is selected from the multiple training servers in the idle state to execute the first training task; the first training task is sent to the first training server so that the first training server executes the received first training task, obtains a trained second image generation model, stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the central server.

[0007] Optionally, detecting the operating status of multiple training servers includes: determining which training servers are in operation among the multiple training servers; for each determined training server, sending operating status detection information to that training server; and when receiving response information carrying a status identifier from that training server, determining the operating status of that training server based on the status identifier; wherein, when the status identifier is a first status identifier, it indicates that the training server is in a working state; and when the status identifier is a second status identifier, it indicates that the training server is in an idle state.

[0008] Optionally, after sending running status detection information to each identified training server and, upon receiving a response carrying a status identifier from the training server, determining the running status of the training server based on the status identifier, the method further includes: determining whether the number of training servers in an idle state is less than the number of first training tasks; if the number of training servers in an idle state is less than the number of first training tasks, calculating a first difference between the number of first training tasks and the number of training servers in an idle state, and calculating the sum of the number of training servers in a running state and the first difference to obtain a first sum; calculating the ratio of the first sum to a preset load index to obtain a first ratio; calculating the difference between the first ratio and the number of training servers in a running state to obtain a first number; and starting the first number of training servers in a closed state.

[0009] Optionally, after determining whether the number of training servers in an idle state is less than the number of first training tasks acquired, the method further includes: if the number of training servers in an idle state is equal to the number of first training tasks, calculating the ratio of the number of training servers in a running state to a preset load index to obtain a second ratio; calculating the difference between the second ratio and the number of training servers in a running state to obtain a second number; and starting the second number of training servers in a closed state.

[0010] Optionally, after determining whether the number of training servers in an idle state is less than the number of acquired first training tasks, the method further includes: if the number of servers in an idle state is greater than the number of first training tasks, calculating the sum of the number of training servers in a working state and the number of first training tasks to obtain a second sum; calculating the ratio of the second sum to the number of training servers in a running state to obtain a third ratio; calculating the ratio of the second sum to a preset load index to obtain a fourth ratio; if the third ratio is greater than the preset load index, calculating the difference between the fourth ratio and the number of training servers in a working state to obtain a third number; starting the third number of training servers in a closed state; if the third ratio is less than the preset load index, calculating the difference between the number of training servers in a working state and the fourth ratio to obtain a fourth number; and shutting down the fourth number of training servers in an idle state.

[0011] Optionally, the step of selecting a first training server from a plurality of idle training servers for each first training task includes: for each first training task, selecting a training server corresponding to the training priority of the first training task from a plurality of idle training servers according to a preset correspondence between training priority and training servers, and using it as the first training server for executing the first training task.

[0012] Optionally, after sending the first training task to the first training server, the method further includes: obtaining a second training task; wherein the second training task represents training the second image generation model corresponding to a specified image type using images from a specified user; selecting a second training server from the idle training servers according to the running status of multiple training servers to execute the second training task; sending the second training task to the second training server so that the second training server executes the received second training task, obtains the trained third image generation model, stores the third image generation model in a preset database, and returns the storage address of the third image generation model to the central server.

[0013] Optionally, the second training task carries the storage address and specified image type of the specified user's image;

[0014] The step of sending the second training task to the second training server to enable the second training server to execute the received second training task and obtain a trained third image generation model includes: sending the second training task to the second training server so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from a preset database according to the specified image type carried in the second training task, and train the second image generation model corresponding to the specified image type using the image of the specified user to obtain a trained third image generation model.

[0015] Optionally, the central server includes a control server and a management server;

[0016] Before selecting a second training server from idle training servers according to the running status of multiple training servers to execute the second training task, the method further includes: the management server calculating the similarity between the specified image type and the image type corresponding to the already trained image generation model; determining the image generation model with the highest similarity between the corresponding image type and the specified image type from the already trained image generation models, as a candidate image generation model; and when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, determining the candidate image generation model as the third image generation model corresponding to the second training task.

[0017] The step of selecting a second training server to perform the second training task from the training servers in an idle state according to the operating status of multiple training servers includes: when the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, selecting a second training server to perform the second training task from the training servers in an idle state according to the operating status of multiple training servers.

[0018] Optionally, after determining the candidate image generation model as the third image generation model corresponding to the second training task when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, the method further includes: recording the correspondence between the third image generation model corresponding to the second training task and the specified user.

[0019] In a second aspect of the present invention, an image generation method is provided, the method comprising:

[0020] The process involves: acquiring the target image type input by the target user; determining the image generation model corresponding to the target image type in a pre-recorded database as the first image generation model based on the pre-recorded correspondence between image types and image generation models; wherein the image generation model stored in the pre-recorded database is obtained based on any of the image generation model training methods described in the first aspect above; adjusting the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model; and inputting the image to be processed by the target user into the target image generation model to obtain the target image.

[0021] Optionally, before inputting the target user's image to be processed into the target image generation model to obtain the target image, the method further includes:

[0022] Obtain the image to be processed input by the target user; or, obtain the image to be processed from the target user stored locally.

[0023] Optionally, before adjusting the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model, the method further includes:

[0024] Based on the pre-recorded correspondence between user identifiers and image generation models, an image generation model corresponding to the user identifier of the target user in a preset database is determined as the second image generation model.

[0025] The step of adjusting the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model includes:

[0026] Calculate the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model to obtain the fusion model parameters; adjust the model parameters of the deployed image generation model to the fusion model parameters to obtain the target image generation model.

[0027] In a third aspect of the present invention, an image generation model training apparatus is provided. The apparatus is applied to a central server in an image generation model training system, the image generation model training system further comprising multiple training servers. The apparatus includes:

[0028] The first acquisition module is used to acquire multiple first training tasks; wherein, a first training task represents training a first image generation model corresponding to the first training task; and the multiple first training tasks correspond to different image types.

[0029] The detection module is used to detect the running status of multiple training servers. The running status of the training servers includes: working status and idle status. The working status indicates that the training server is executing the first training task; the idle status indicates that the training server is not executing the first training task.

[0030] The first selection module is used to select a first training server from a plurality of training servers that are in an idle state for each first training task.

[0031] The first sending module is used to send the first training task to the first training server, so that the first training server executes the received first training task, obtains the trained second image generation model, stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the central server.

[0032] Optionally, the detection module is specifically used for: identifying training servers that are in a running state among multiple training servers; sending running status detection information to each identified training server; and determining the running status of the training server based on the status identifier when receiving a response information carrying a status identifier from the training server; wherein, when the status identifier is a first status identifier, it indicates that the training server is in a working state; and when the status identifier is a second status identifier, it indicates that the training server is in an idle state.

[0033] Optionally, the device further includes:

[0034] The judgment module is used to send running status detection information to each identified training server when the detection module performs the operation. When the response information carrying the status identifier is received from the training server, the module determines the running status of the training server based on the status identifier and then determines whether the number of training servers in the idle state is less than the number of the first training tasks.

[0035] The first sum calculation module is used to calculate the first difference between the number of first training tasks and the number of training servers in the idle state if the number of training servers in the idle state is less than the number of first training tasks, and to calculate the sum of the number of training servers in the running state and the first difference to obtain the first sum.

[0036] The first ratio calculation module is used to calculate the ratio of the first sum to a preset load index to obtain the first ratio.

[0037] The first number calculation module is used to calculate the difference between the first ratio and the number of training servers in operation to obtain the first number;

[0038] The first startup module is used to start the first number of training servers that are in a closed state.

[0039] Optionally, the device further includes:

[0040] The second ratio calculation module is used to calculate the ratio of the number of training servers in the running state to a preset load index after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of training servers in the idle state is equal to the number of the first training tasks, the module obtains the second ratio.

[0041] The second number calculation module is used to calculate the difference between the second ratio and the number of training servers in operation to obtain the second number;

[0042] The second startup module is used to start the second number of training servers that are in a closed state.

[0043] Optionally, the device further includes:

[0044] The second sum calculation module is used to calculate the sum of the number of training servers in the working state and the number of the first training tasks after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of servers in the idle state is greater than the number of the first training tasks, the module calculates the sum of the number of training servers in the working state and the number of the first training tasks to obtain the second sum.

[0045] The third ratio calculation module is used to calculate the ratio of the second sum to the number of training servers in operation, and obtain the third ratio.

[0046] The fourth ratio calculation module is used to calculate the ratio of the second sum to the preset load index to obtain the fourth ratio.

[0047] The third number calculation module is used to calculate the difference between the fourth ratio and the number of training servers in working state if the third ratio is greater than the preset load index, so as to obtain the third number.

[0048] The third startup module is used to start the third number of training servers that are in a closed state;

[0049] The fourth number calculation module is used to calculate the difference between the number of training servers in working state and the fourth ratio if the third ratio is less than the preset load index, and obtain the fourth number.

[0050] The first shutdown module is used to shut down the fourth number of training servers that are in an idle state.

[0051] Optionally, the first selection module is specifically used to: for each first training task, according to a preset correspondence between training priorities and training servers, select a training server from multiple training servers in an idle state that corresponds to the training priority of the first training task, and use it as the first training server for executing the first training task.

[0052] Optionally, the device further includes:

[0053] The second acquisition module is used to acquire a second training task after the first sending module executes the sending of the first training task to the first training server; wherein the second training task represents training the second image generation model corresponding to a specified image type using images from a specified user;

[0054] The second selection module is used to select a second training server from the training servers that are in an idle state to perform the second training task, according to the running status of multiple training servers.

[0055] The second sending module is used to send the second training task to the second training server, so that the second training server executes the received second training task, obtains the trained third image generation model, stores the third image generation model in a preset database, and returns the storage address of the third image generation model to the central server.

[0056] Optionally, the second training task carries the storage address and specified image type of the specified user's image;

[0057] The second sending module is specifically used to: send the second training task to the second training server, so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from a preset database according to the specified image type carried in the second training task, and use the image of the specified user to train the second image generation model corresponding to the specified image type to obtain a trained third image generation model.

[0058] Optionally, the central server includes a control server and a management server;

[0059] The device further includes:

[0060] The similarity calculation module is used to calculate the similarity between the specified image type and the image type corresponding to the already trained image generation model before the second selection module performs the step of selecting a second training server from the training servers in an idle state according to the running status of multiple training servers to perform the second training task.

[0061] The first determining module is used to determine, from the already trained image generation models, the image generation model with the highest similarity to the specified image type, as a candidate image generation model;

[0062] The second determining module is used to determine the candidate image generation model as the third image generation model corresponding to the second training task when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold.

[0063] The second selection module is specifically used to: when the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, select a second training server from the training servers in an idle state to perform the second training task according to the running status of multiple training servers.

[0064] Optionally, the device further includes:

[0065] The recording module is used to record the correspondence between the third image generation model and the specified user after the second determining module determines the candidate image generation model as the third image generation model when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold.

[0066] In a fourth aspect of the present invention, an image generation apparatus is provided, the apparatus comprising:

[0067] The first acquisition module is used to acquire the target image type input by the target user;

[0068] The first image generation model determination module is used to determine, based on the pre-recorded correspondence between image types and image generation models, an image generation model corresponding to the target image type in a preset database, and use it as the first image generation model; wherein, the image generation model stored in the preset database is obtained based on any of the image generation model training methods described in the first aspect above;

[0069] The adjustment module is used to adjust the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model;

[0070] The target image acquisition module is used to input the image to be processed of the target user into the target image generation model to obtain the target image.

[0071] Optionally, the device further includes:

[0072] The second acquisition module is used to acquire the image to be processed input by the target user before the target image acquisition module performs the step of inputting the image to be processed of the target user into the target image generation model to obtain the target image;

[0073] or,

[0074] Obtain the image to be processed from the target user stored locally.

[0075] Optionally, the device further includes:

[0076] The second image generation model determination module is used to determine the image generation model corresponding to the user identifier of the target user in a preset database, based on the pre-recorded correspondence between user identifiers and image generation models, before the adjustment module performs the adjustment of the model parameters of the deployed image generation model based on the first image generation model to obtain the target image generation model.

[0077] The adjustment module is specifically used to: calculate the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model to obtain the fusion model parameters; and adjust the model parameters of the deployed image generation model to the fusion model parameters to obtain the target image generation model.

[0078] In a fifth aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0079] Memory, used to store computer programs;

[0080] When a processor executes a program stored in memory, it implements the image generation model training method steps described in any of the first aspects above, or implements the image generation method steps described in any of the second aspects above.

[0081] In a sixth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the image generation model training method steps described in any of the first aspects above, or implements the image generation method steps described in any of the second aspects above.

[0082] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the image generation model training method steps described in any of the first aspects above, or to implement the image generation method steps described in any of the second aspects above.

[0083] This invention provides an image generation model training method, an image generation method, an apparatus, and an electronic device. The image generation model training method is applied to a central server in an image generation model training system, which also includes multiple training servers. The method includes: acquiring multiple first training tasks; wherein, a first training task represents training a first image generation model corresponding to the first training task; the multiple first training tasks correspond to different image types; detecting the operating status of multiple training servers; wherein, the operating status of the training servers includes: a working state and an idle state; the working state indicates that the training server is executing a first training task; the idle state indicates that the training server is not executing a first training task; for each first training task, selecting a first training server from the multiple training servers in the idle state to execute the first training task; sending the first training task to the first training server so that the first training server executes the received first training task, obtaining a trained second image generation model, storing the second image generation model in a preset database, and returning the storage address of the second image generation model to the central server.

[0084] Based on the above processing, training efficiency can be improved by executing the training tasks corresponding to each image type separately through multiple training servers. Furthermore, for each image type, a first image generation model corresponding to that image type can be trained to obtain a trained image generation model for that image type. Subsequently, users can use this image generation model to obtain the target image of that image type, thus meeting their needs.

[0085] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0086] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0087] Figure 1 This is a first flowchart of an image generation model training method provided in an embodiment of the present invention;

[0088] Figure 2 This is a second flowchart of the image generation model training method provided in an embodiment of the present invention;

[0089] Figure 3 This is a third flowchart of the image generation model training method provided in the embodiments of the present invention;

[0090] Figure 4 This is a fourth flowchart of the image generation model training method provided in the embodiments of the present invention;

[0091] Figure 5 This is a fifth flowchart of the image generation model training method provided in an embodiment of the present invention;

[0092] Figure 6 This is a sixth flowchart of the image generation model training method provided in an embodiment of the present invention;

[0093] Figure 7 The seventh flowchart of the image generation model training method provided in the embodiments of the present invention;

[0094] Figure 8 The eighth flowchart of the image generation model training method provided in the embodiments of the present invention;

[0095] Figure 9 This is a first flowchart of an image generation method provided in an embodiment of the present invention;

[0096] Figure 10 This is a second flowchart of the image generation method provided in an embodiment of the present invention;

[0097] Figure 11 This is a structural diagram of an image generation model training system provided in an embodiment of the present invention;

[0098] Figure 12 A structural diagram of a machine learning subsystem provided in an embodiment of the present invention;

[0099] Figure 13 A structural diagram of an image generation model training device provided in an embodiment of the present invention;

[0100] Figure 14 A structural diagram of an image generation apparatus provided in an embodiment of the present invention;

[0101] Figure 15 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0102] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0103] In the existing technology, when training a model with an initial structure based on training samples containing multiple types of sample images, the training time is too long due to the large number of training samples containing multiple types of sample images, resulting in low efficiency in generating a well-trained image generation model.

[0104] To address the aforementioned problems, this invention provides an image generation model training method applied to a central server in an image generation model training system. The system further includes multiple training servers. The method includes: acquiring multiple first training tasks; wherein each first training task represents training a first image generation model corresponding to that first training task; the multiple first training tasks correspond to different image types; detecting the operating status of the multiple training servers; wherein the operating status of the training servers includes a working state and an idle state; the working state indicates that the training server is executing a first training task; the idle state indicates that the training server is not executing a first training task; for each first training task, selecting a first training server from the multiple idle training servers to execute that first training task; sending the first training task to the first training server so that the first training server executes the received first training task, obtaining a trained second image generation model, storing the second image generation model in a preset database, and returning the storage address of the second image generation model to the central server. This method can improve the efficiency of executing multiple first training tasks, reduce training time, and improve the efficiency of obtaining a trained second image generation model.

[0105] See Figure 1 , Figure 1 This is a first flowchart of an image generation model training method provided in an embodiment of the present invention. The method may include the following steps:

[0106] S101: Obtain multiple first training tasks.

[0107] Here, a first training task means training the first image generation model corresponding to the first training task; multiple first training tasks correspond to different image types.

[0108] S102: Detect the running status of multiple training servers.

[0109] The operating status of the training server includes: working status and idle status; working status indicates that the training server is executing the first training task; idle status indicates that the training server is not executing the first training task.

[0110] S103: For each first training task, select a first training server from among the multiple training servers that are in an idle state to perform the first training task.

[0111] S104: Send the first training task to the first training server so that the first training server can execute the received first training task, obtain the trained second image generation model, store the second image generation model in a preset database, and return the storage address of the second image generation model to the central server.

[0112] Based on the image generation model training method provided in this embodiment of the invention, training efficiency can be improved by having multiple training servers execute the training tasks corresponding to each image type separately. Furthermore, for each image type, a first image generation model corresponding to that image type can be trained to obtain a trained image generation model for that image type. Subsequently, users can obtain the target image of that image type using the image generation model corresponding to that image type, thus meeting user needs.

[0113] Regarding step S101, the image generation model training system can be a server cluster. The servers in the image generation model training system may include: a central server for managing and scheduling training tasks, training servers for executing training tasks, and storage servers for storing data, etc.

[0114] The central server acquires multiple first training tasks, which can be pre-set by technicians. Each first training task corresponds to an image type, and each first training task corresponds to a first image generation model. The correspondence between a first training task and an image type means that the second image generation model obtained by training the first image generation model corresponding to that first training task is used to generate images of that image type. During the training of a first image generation model, nearly a hundred parameters within that model need to be adjusted. For example, the first image generation model can be a diffusion model, including models such as Stable Diffusion, Midjourney, and Dali-Wall E2.

[0115] The image types corresponding to the multiple first training tasks are different. For example, the image type corresponding to the first training task 1 can be: a comic-style image; the image type corresponding to the first training task 2 can be: an image in the painting style of artist A; the image type corresponding to the first training task 3 can be: an image including a seaside background, etc.

[0116] Subsequently, after training the first image generation model, the image to be processed can be input into the trained first image generation model (i.e., the second image generation model). The second image generation model can output the target image obtained by processing the image to be processed according to the corresponding image type. For example, if the image to be processed contains a target person, and the image type corresponding to the second image generation model is an image including a seaside background, then the second image generation model can output a target image of the target person walking on the beach. Or, if the image type corresponding to the second image generation model is a cartoon-style image, then the second image generation model can output a target image displaying the target person in a cartoon style.

[0117] In steps S102 and S103, the central server establishes communication connections with multiple training servers. The central server can also monitor the operational status of these training servers. For example, the central server can monitor the operational status of multiple training servers when a monitoring period is reached. Alternatively, the central server can monitor the operational status of multiple training servers after receiving a training task.

[0118] In some embodiments, Figure 1 Based on this, see Figure 2 Step S102 may include the following steps:

[0119] S1021: Identify the training server that is running among multiple training servers.

[0120] S1022: For each identified training server, send running status detection information to that training server. When a response information carrying a status identifier is received from the training server, determine the running status of the training server based on the status identifier.

[0121] Specifically, when the status identifier is the first status identifier, it indicates that the training server is in a working state; when the status identifier is the second status identifier, it indicates that the training server is in an idle state.

[0122] To determine the operational status of multiple training servers, the central server can first identify the training servers that are currently running. These running training servers are those that are already started and can include both those currently executing training tasks and those that are not currently executing tasks.

[0123] To determine the operational status of each training server, the central server can send operational status detection information to that server. Upon receiving the detection information, the training server can respond to it based on its own operational status and return a response message to the central server, which includes a status identifier corresponding to that server's operational status.

[0124] After receiving the response information carrying a status identifier from the training server, the central server determines whether the training server is in a working state if the status identifier is the first status identifier, or in an idle state if the status identifier is the second status identifier. Thus, the central server obtains the operating status of multiple training servers.

[0125] In some embodiments, in order to reduce the operating cost of the image generation model training system and save the computing resources of the image generation model training system, the central server also needs to adjust the number of training servers in operation among the multiple training servers according to the operating status of the multiple training servers.

[0126] exist Figure 2 Based on this, see Figure 3 After step S1022, the method may further include the following steps:

[0127] S105: Determine whether the number of training servers in an idle state is less than the number of the first training tasks.

[0128] S106: If the number of training servers in an idle state is less than the number of first training tasks, calculate the first difference between the number of first training tasks and the number of training servers in an idle state, and calculate the sum of the number of training servers in a running state and the first difference to obtain the first sum.

[0129] S107: Calculate the ratio of the first sum to the preset load index to obtain the first ratio.

[0130] S108: Calculate the difference between the first ratio and the number of training servers in operation to obtain the first number.

[0131] S109: Start the first number of training servers that are currently in a closed state.

[0132] Idle training servers can be used to execute the acquired first training tasks. When the number of idle training servers is small, they cannot execute all the first training tasks simultaneously, resulting in low efficiency for the image generation model training system. When the number of idle training servers is large, some servers may be active but not executing training tasks, wasting the computational resources of the image generation model training system and increasing its operating cost. Therefore, the central server can determine whether the number of idle training servers is less than the number of first training tasks, i.e., whether the currently idle training servers can execute all the first training tasks simultaneously.

[0133] If the number of training servers in an idle state is less than the number of first training tasks, it indicates that the currently idle training servers cannot execute all the first training tasks simultaneously.

[0134] In one implementation, if the number of idle training servers is less than the number of first training tasks, the central server can sequentially determine the idle training servers to execute the first training task according to the training priority of the multiple first training tasks from high to low. The training priority of the training tasks can be referred to the relevant description in subsequent embodiments.

[0135] In another implementation, if the number of training servers in an idle state is less than the number of the first training tasks, the central server can turn on the training servers that are in a closed state.

[0136] The preset load metric is the upper limit of the load on the image generation model training system. The load of the image generation model training system is the ratio of the number of training servers in a working state to the number of training servers in a running state (this can be called the fifth ratio). When the fifth ratio is greater than the preset load metric, it indicates that the current load pressure on the image generation model training system is high, and training servers that are currently in a shutdown state need to be started to reduce the load pressure. When the fifth ratio is not greater than the preset load metric, it indicates that the current load pressure on the image generation model training system is not high, and training servers that are currently in an idle state can be shut down to reduce the operating cost of the image generation model training system and conserve its computing resources.

[0137] The central server can first calculate the first difference between the number of first training tasks and the number of training servers in an idle state. The first difference is the number of first training tasks that the currently idle training servers cannot execute, which is the minimum number of training servers that need to be started.

[0138] Then, the central server can calculate the sum of the number of training servers in operation and the first difference to obtain the first sum. The first sum is the number of training servers in operation in the image generation model training system when multiple first training tasks are processed.

[0139] Then, the central server can calculate the ratio of the first sum to a preset load index to obtain the first ratio. The first ratio is the number of training servers in the image generation model training system that are running when processing multiple first training tasks and the load of the image generation model training system does not exceed the preset load index.

[0140] Furthermore, the central server can calculate the difference between the first ratio and the number of training servers in operation to obtain the first number. This first number represents the number of training servers that need to be activated to ensure that idle training servers can simultaneously execute all first training tasks, and that the load on the image generation model training system does not exceed a preset load threshold. Therefore, the central server can activate the first number of training servers that are currently in a shutdown state. Subsequently, after activating the first number of training servers in a shutdown state, it is possible to ensure that idle training servers can simultaneously execute all first training tasks, and that the load on the image generation model training system does not exceed the preset load threshold.

[0141] In some embodiments, Figure 3 Based on this, see Figure 4 After step S105, the method may further include the following steps:

[0142] S110: If the number of training servers in an idle state is equal to the number of the first training tasks, calculate the ratio of the number of training servers in a running state to the preset load index to obtain the second ratio.

[0143] S111: Calculate the difference between the second ratio and the number of training servers in operation to obtain the second number.

[0144] S112: Start the second number of training servers that are currently in a closed state.

[0145] If the number of idle training servers equals the number of first training tasks, meaning the currently idle training servers can execute all first training tasks simultaneously, then all running training servers are in working condition. Therefore, to ensure the load on the image generation model training system does not exceed a preset load threshold, the central server needs to first calculate the ratio of the number of running training servers to the preset load threshold, obtaining a second ratio. This second ratio represents the number of running training servers when the load on the image generation model training system does not exceed the preset load threshold.

[0146] Then, the central server can calculate the difference between the second ratio and the number of training servers in operation to obtain the second number, which is the number of training servers that need to be started. The central server can then start the second number of training servers that were previously in a shutdown state. Subsequently, after starting the second number of training servers that were previously in a shutdown state, the idle training servers can simultaneously execute all the first training tasks, and the load on the image generation model training system will not exceed a preset load index.

[0147] In some embodiments, Figure 3 Based on this, see Figure 5 After step S105, the method may further include the following steps:

[0148] S113: If the number of idle servers is greater than the number of first training tasks, calculate the sum of the number of active training servers and the number of first training tasks to obtain the second sum.

[0149] S114: Calculate the ratio of the second sum to the number of training servers in operation to obtain the third ratio.

[0150] S115: Calculate the ratio of the second sum to the preset load index to obtain the fourth ratio.

[0151] S116: If the third ratio is greater than the preset load index, calculate the difference between the fourth ratio and the number of training servers in working state to obtain the third number; start the third number of training servers in the closed state; if the third ratio is less than the preset load index, calculate the difference between the number of training servers in working state and the fourth ratio to obtain the fourth number; shut down the fourth number of training servers in the idle state.

[0152] If the number of idle training servers exceeds the number of first training tasks (meaning the currently idle training servers can execute all first training tasks simultaneously), and besides the training servers processing the first training tasks, there are other idle training servers (which can be called training servers to be shut down), a large number of these servers wastes the computational resources of the image generation model training system, increasing its operating cost. Conversely, a small number of these servers results in a high load on the system. Therefore, the central server can first calculate the sum of the number of active training servers and the number of first training tasks to obtain a second sum. This second sum represents the number of active training servers in the image generation model training system when processing multiple first training tasks.

[0153] Then, the central server can calculate the ratio of the second sum to the number of training servers in operation, obtaining a third ratio. This third ratio represents the load on the image generation model training system when processing multiple acquired first training tasks. Furthermore, the central server can calculate the ratio of the second sum to a preset load metric, obtaining a fourth ratio. This fourth ratio represents the number of training servers in operation when the load on the image generation model training system equals the preset load metric.

[0154] Furthermore, the central server can compare the third ratio with a preset load index. When the third ratio is greater than the preset load index, it indicates that the number of training servers in an idle state is relatively small. Therefore, the central server can calculate the difference between the fourth ratio and the number of training servers in an active state to obtain the third number, which is the number of training servers that need to be started from the off state. After starting the third number of training servers in the off state, the training servers in the idle state can simultaneously execute all the first training tasks, and the load on the image generation model training system will not exceed the preset load index, thus avoiding excessive load pressure on the image generation model training system.

[0155] When the third ratio equals the preset load index, that is, when the training servers in an idle state execute the acquired multiple first training tasks, the load of the image generation model training system is equal to the preset load index. This prevents the image generation model training system from experiencing excessive load pressure and avoids wasting its computational resources. Therefore, the central server does not need to start training servers that are currently shut down, nor does it need to shut down training servers that are currently idle.

[0156] When the third ratio is less than the preset load index, it indicates that a large number of training servers are in an idle state. Therefore, to reduce the operating cost of the image generation model training system and conserve its computing resources, the central server can calculate the difference between the number of training servers in operation and the fourth ratio to obtain the fourth number. This fourth number represents the number of idle training servers that need to be shut down. After shutting down this fourth number of idle training servers, the idle servers can simultaneously execute all the first training tasks, and the load of the image generation model training system will not be less than the preset load index, thus reducing the operating cost and conserving its computing resources.

[0157] Based on the above processing, the running status of the training servers can be adjusted according to the number of training servers in idle state, the number of training servers in running state, the number of first training tasks, and the preset load index. This can reduce the operating cost of the image generation model training system and save the computing resources of the image generation model training system while avoiding excessive load pressure.

[0158] In step S103, after determining the training servers that are in an idle state, the central server can select any idle training server as the first training server to execute the first training task for each first training task.

[0159] In some embodiments, Figure 1 Based on this, see Figure 6 Step S103 may include the following steps:

[0160] S1031: For each first training task, according to the preset correspondence between training priority and training server, select a training server from multiple training servers in an idle state that corresponds to the training priority of the first training task, and use it as the first training server to execute the first training task.

[0161] For each first training task, the central server can first determine the training priority of the first training task based on its attributes. The attributes of a first training task include at least one of the following: the order of magnitude of the amount of training samples used when executing the first training task, and the business priority corresponding to the first training task. A higher business priority for a first training task indicates that more users are using the image generation model for that image type, and that users use that image generation model more frequently. A lower business priority for a first training task indicates that fewer users are using the image generation model for that image type, and that users use that image generation model less frequently.

[0162] The larger the order of magnitude of the training sample data used when executing the first training task, the higher the business priority of the first training task and the higher the training priority of the first training task; the smaller the order of magnitude of the training sample data used when executing the first training task, the lower the business priority of the first training task and the lower the training priority of the first training task.

[0163] Furthermore, for each training server, the central server can determine its training priority based on its configuration (e.g., memory, GPU (Graphics Processing Unit), CPU (Central Processing Unit), etc.). A higher configuration results in better computational performance and faster execution of training tasks, thus a higher-configuration server has a higher training priority. Conversely, a lower configuration results in worse computational performance and slower execution of training tasks, thus a lower-configuration server has a lower training priority. The training priority of a training server represents the highest priority training task that server can execute.

[0164] Furthermore, for each first training task, the central server can select a training server with a training priority no less than that of the first training task from among multiple training servers in an idle state. This determines the training server corresponding to the training priority of the first training task, which is then used as the first training server to execute the first training task.

[0165] Based on the above processing, the central server can use a better-configured training server to execute training tasks with higher training priority. This avoids the situation where a lower-configured training server is unable to execute training tasks with higher training priority due to insufficient configuration. This can improve the efficiency of executing multiple training tasks and reduce training time.

[0166] In step S104, for each first training task, after selecting a first training server to execute the first training task, the central server can send the first training task to the selected first training server. Then, the first training server can execute the first training task, thereby training the first image generation model corresponding to the first training task to obtain a trained second image generation model.

[0167] In some embodiments, the first training task carries the storage address of training samples for training the corresponding first image generation model. The training samples are multiple training samples obtained in a real-world environment, and each training sample displays people. The characteristics of the people displayed in each training sample are different; for example, the training samples may include: training samples displaying people with yellow skin, training samples displaying people with white skin, training samples displaying people with large eyes, training samples displaying people with small eyes, training samples displaying tall people, training samples displaying short people, training samples displaying overweight people, training samples displaying thin people, etc.

[0168] Furthermore, for each training sample, the training sample can include a sample image, sample text, sample parameters, etc. The sample text is used to describe the content displayed in the sample image. For example, if the sample image shows a woman wearing a red hat, the sample text could be: A woman wearing a red hat.

[0169] Then, the first training server can obtain the training samples according to the storage address of the training samples carried in the first training task, and use the obtained training samples to train the first image generation model corresponding to the first training task to obtain the trained second image generation model.

[0170] In some embodiments, for each training sample, the first training server can input the sample image and sample text from the training sample into the first image generation model, and the first image generation model outputs prediction parameters. Then, based on the difference between the prediction parameters and the sample parameters, the model parameters of the first image generation model are adjusted to train the first image generation model.

[0171] Furthermore, the first training server can store the second image generation model in a pre-defined database and return the storage address of the second image generation model to the central server. The pre-defined database can be deployed on the storage server included in the image generation model training system for storing data.

[0172] In some embodiments, Figure 1 Based on this, see Figure 7 After step S104, the method may further include the following steps:

[0173] S117: Obtain the second training task.

[0174] The second training task involves training a second image generation model corresponding to a specified image type using images from a specified user.

[0175] S118: Based on the operating status of multiple training servers, select a second training server from the training servers that are in an idle state to perform the second training task.

[0176] S119: Send a second training task to the second training server so that the second training server can execute the received second training task, obtain the trained third image generation model, store the third image generation model in a preset database, and return the storage address of the third image generation model to the central server.

[0177] The specified user's image is an image containing a person, input by the specified user. For example, the specified user's image could be a selfie, and the person displayed in the image would be the specified user. Alternatively, the specified user's image could be an image of celebrity A, and the person displayed in the image would be celebrity A.

[0178] A designated user can input an image of that user into their user device and select a specific image type from the image types corresponding to the pre-trained second image model. Correspondingly, the user device can send the designated user's image and the specified image type to the image generation model training system. The central server can then obtain a second training task indicating that the second image generation model corresponding to the specified image type is being trained using the designated user's image.

[0179] Furthermore, the central server can select a second training server from the idle training servers according to the operating status of the multiple training servers to perform the second training task. The method by which the central server selects the second training server is similar to the method of selecting the first training server in the previous embodiments, and can be referred to the relevant description in the previous embodiments.

[0180] In some embodiments, the second training task carries the storage address of the image of the specified user and the specified image type.

[0181] Step S122 may include the following steps:

[0182] Step 1: Send a second training task to the second training server, so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from the preset database according to the specified image type carried in the second training task. Then, use the specified user's image to train the second image generation model corresponding to the specified image type to obtain the trained third image generation model, store the third image generation model in the preset database, and return the storage address of the third image generation model to the central server.

[0183] The second training task includes the storage address of an image for a specified user. The second training server can retrieve the image for that user based on this storage address. The image of the specified user displays a person and can be stored in a pre-defined database. Furthermore, the second training server can retrieve the second image generation model corresponding to the specified image type from the pre-defined database, based on the specified image type specified in the second training task.

[0184] Then, the second training server can use images from a specified user to train a second image generation model corresponding to a specified image type. That is, the second training server uses multiple images displaying the same person to train the second image generation model, resulting in a third image generation model. Using the third image generation model, the target image of the person displayed in the specified user's image can be obtained.

[0185] Furthermore, the second training server can store the third image generation model in a pre-defined database and return the storage address of the third image generation model to the central server. The image type corresponding to the third image generation model is the image type after adjusting the specified image type according to the specified user's image. For example, if the specified image type is: comic-style image; then the image type corresponding to the third image generation model is: an image of the specified user drawn in a comic style.

[0186] In some embodiments, the central server includes a control server and a management server.

[0187] exist Figure 7 Based on this, see Figure 8 Before step S118, the method may further include the following steps:

[0188] S120: The management server calculates the similarity between a specified image type and the image type corresponding to the already trained image generation model.

[0189] S121: From the already trained image generation models, determine the image generation model with the highest similarity to the specified image type for the corresponding image type, and use it as the candidate image generation model.

[0190] S122: When the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, the candidate image generation model is determined as the third image generation model corresponding to the second training task.

[0191] Accordingly, step S118 may include the following steps:

[0192] S1181: When the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, select a second training server from the training servers in the idle state to perform the second training task according to the running status of multiple training servers.

[0193] The management server can communicate with user devices. When a user device sends an image of a specified user and a specified image type to the image generation model training system, the management server first obtains the image and image type. To improve the utilization of the image generation model training system and avoid wasting its computational resources, the management server can first calculate the similarity between the specified image type and the corresponding image type in the already trained image generation model.

[0194] The trained image generation models include the second image generation model obtained based on the first training task in the foregoing embodiments, and the third image generation model obtained based on the second training task in the foregoing embodiments.

[0195] In one implementation, for each trained image generation model (which can be called the image generation model to be matched), the management server can use a preset encoding method to encode the image type corresponding to the image generation model to be matched, obtaining a first vector representing the image type corresponding to the image generation model to be matched. Furthermore, the management server can also use a preset encoding method to encode a specified image type, obtaining a second vector representing the specified image type. Then, the vector similarity between the first vector and the second vector is calculated as the similarity between the image type corresponding to the image generation model to be matched and the specified image type. For example, the preset encoding method can be a One-Hot encoding method.

[0196] In another implementation, for each image generation model to be matched, the management server can directly compare whether the specified image type is the same as the image type corresponding to the image generation model to be matched. If the specified image type is the same as the image type corresponding to the image generation model to be matched, it is determined that the specified image type is highly similar to the image type corresponding to the image generation model to be matched; if the specified image type is different from the image type corresponding to the already trained image generation model, it is determined that the specified image type is less similar to the image type corresponding to the image generation model to be matched.

[0197] The management server can select the image generation model with the highest similarity to a specified image type from the already trained image generation models, and use it as a candidate image generation model. Then, it determines whether the similarity between the candidate image generation model and the specified image type is greater than a preset threshold.

[0198] When the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, it indicates that the image generation model training system has trained the second image generation model corresponding to the specified image type using images from the specified user, thus obtaining the candidate image generation model. Therefore, the control server can directly determine the candidate image generation model as the third image generation model corresponding to the second training task.

[0199] When the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, it means that the image generation model training system has not yet used the specified user's image to train the second image generation model corresponding to the specified image type. Therefore, the control server can use the specified user's image to train the second image generation model corresponding to the specified image type. First, according to the running status of multiple training servers, the second training server to perform the second training task is selected from the training servers in the idle state.

[0200] In some embodiments, after step S125, the method may further include the following step: recording the correspondence between the third image generation model corresponding to the second training task and the specified user.

[0201] The central server can record the correspondence between the third image generation model corresponding to the second training task and the specified user in a pre-set database. Subsequently, when it is necessary to use the third image generation model corresponding to the second training task to generate a target image, the central server can directly retrieve the third image generation model corresponding to the user identifier of the specified user who needs to obtain the target image from the pre-set database.

[0202] Based on the above processing, the central server can use the image of the specified user to further train the already trained second image generation model, resulting in a third image generation model that is even more closely matched to the specified user. Subsequently, when generating and displaying target images containing the specified user using the third image generation model, the accuracy is higher and the user experience is better.

[0203] In some embodiments, the central server can statistically analyze the success rate and time consumption when executing any step in the image generation model training method described in the foregoing embodiments. Subsequently, based on the statistically obtained success rate and time consumption, the optimization direction of the image generation model training system can be determined to obtain a more efficient and stable image generation model training system. For example, if the success rate of the first training server in obtaining training samples from the preset database is low, the stability of the communication connection between the first training server and the preset database can be optimized; if the central server takes a long time to detect the running status of multiple training servers, the configuration of the central server can be improved, etc.

[0204] See Figure 9 , Figure 9 This is a first flowchart of an image generation method provided in an embodiment of the present invention. The method may include the following steps:

[0205] S901: Obtain the target image type input by the target user.

[0206] S902: Based on the pre-recorded correspondence between image types and image generation models, determine the image generation model in the preset database that corresponds to the target image type, and use it as the first image generation model.

[0207] The image generation model stored in the preset database is obtained based on any of the image generation model training methods described in the foregoing embodiments.

[0208] S903: Based on the model parameters of the first image generation model, adjust the model parameters of the deployed image generation model to obtain the target image generation model.

[0209] S904: Input the target user's image to be processed into the target image generation model to obtain the target image.

[0210] Based on the image generation method provided in this embodiment of the invention, an electronic device can obtain a target image generation model according to the target image type, and then input the image to be processed into the target image generation model to obtain a target image that displays the content of the image to be processed according to the target image type, thereby meeting user needs.

[0211] The image generation method provided in this embodiment of the invention can be applied to a user terminal. The target user can input the target image type into the user terminal, and then the user terminal can obtain the target image according to the image generation method provided in this embodiment of the invention.

[0212] Alternatively, the image generation method provided in this embodiment of the invention can be applied to a server. A target user can input a target image type to a user terminal. The user terminal can then send the target image type input by the target user to the server. The server then obtains the target image according to the image generation method provided in this embodiment of the invention and sends the target image to the user terminal. The server can be the server used to run the image generation model in the image generation model training system described in the foregoing embodiments.

[0213] Regarding step S901, the target user is the user who needs to acquire the target image. The user device's display interface can show the image type corresponding to the generated image generation model. The target user can select the appropriate target image type from multiple image types, thus inputting the target image type into the user device. Subsequently, the electronic device can acquire the target image type input by the target user.

[0214] Regarding step S902, in some embodiments, the electronic device can perform mirroring processing on the image generation model stored in the preset database, and use the mirroring processing result as the first image generation model.

[0215] Regarding steps S903 and S904, after the electronic device obtains the first image generation model, due to the complexity of its model structure and the large number of parameters used, deploying the first image generation model would require deleting other already deployed image generation models before deploying the first model, which would consume considerable time and result in low image generation efficiency. Therefore, the electronic device can obtain the model parameters of the first image generation model and then adjust the model parameters of the already deployed image generation models based on these parameters to obtain the target image generation model. Then, by inputting the image to be processed into the target image generation model, the target image output by the target image generation model can be directly obtained. This eliminates the need to delete other already deployed image generation models before deploying the first image generation model, saving time and improving image generation efficiency.

[0216] In some embodiments, prior to step S904, the method may further include the following steps: acquiring the image to be processed input by the target user; or acquiring the image to be processed of the target user stored locally.

[0217] The image to be processed, which needs to be input into the target image generation model, can be input by the target user. The target user can also input the image to be processed when inputting the target image type. Correspondingly, the electronic device can use the target image generation model to process the image to be processed, obtaining a target image that displays the content of the image to be processed according to the target image type.

[0218] Alternatively, the image to be processed can be an image stored locally on the electronic device. Figure 9 Based on this, see Figure 10 Before step S903, the method may further include the following steps:

[0219] S905: Based on the pre-recorded correspondence between user identifiers and image generation models, determine the image generation model in the preset database that corresponds to the user identifier of the target user, and use it as the second image generation model.

[0220] Accordingly, step S903 may include the following steps:

[0221] S9031: Calculate the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model to obtain the fusion model parameters.

[0222] S9032: Adjust the model parameters of the deployed image generation model to the fusion model parameters to obtain the target image generation model.

[0223] After the target user inputs the target image type, the electronic device can also receive the target user's user identifier. Accordingly, the electronic device can search for the image to be processed corresponding to the target user's user identifier locally. The target user can be the designated user in the aforementioned embodiments, and the image to be processed corresponding to the target user's user identifier can be the image of the designated user in the aforementioned embodiments.

[0224] In other words, the target user can correspond to the third image generation model corresponding to the second training task in the aforementioned embodiments, and the third image generation model corresponding to the target user can be used to generate a target image displaying the target user. Therefore, the electronic device can obtain the second image generation model corresponding to the target user's user identifier from a preset database based on the target user's user identifier. The second image generation model corresponding to the target user's user identifier can be the third image generation model corresponding to the second training task in the aforementioned embodiments.

[0225] Furthermore, the electronic device can also obtain the model parameters of the second image generation model corresponding to the user identifier of the target user. The method by which the electronic device obtains the model parameters of the second image generation model is similar to the method of obtaining the model parameters of the first image generation model in the foregoing embodiments, and can be referred to the relevant descriptions in the foregoing embodiments.

[0226] Furthermore, in order to obtain a target image generation model based on the model parameters of the first image generation model and the second image generation model, the electronic device can calculate a weighted sum of the model parameters of the first and second image generation models according to preset weights to obtain the fused model parameters. The preset weights can be determined when the image generation model training system performs the second training task to train the second image generation model in the aforementioned embodiment. For example, the preset weights could be: a weight of 0.4 for the model parameters of the first image generation model and a weight of 0.6 for the model parameters of the second image generation model.

[0227] Since the image generation model has nearly 1 billion parameters, when calculating the weighted sum of the parameters of the first and second image generation models, we can first determine the correspondence between the model parameters, and then calculate the weighted sum of the corresponding model parameters. For example, if the model parameters of the first image generation model include: model parameter 1, parameter value a1; model parameter 2, parameter value b1; and the model parameters of the second image generation model include: model parameter 1, parameter value a2; and model parameter 2, parameter value b2, then for model parameter 1, we calculate the weighted sum of parameter values ​​a1 and a2 to obtain the fusion parameter value A of model parameter 1; and for model parameter 2, we calculate the weighted sum of parameter values ​​b1 and b2 to obtain the fusion parameter value B of model parameter 2. Therefore, the fusion model parameters would then include: model parameter 1, parameter value A; and model parameter 2, parameter value B.

[0228] Furthermore, electronic devices can adjust the model parameters of the deployed image generation model to the parameters of the fusion model to obtain the target image generation model.

[0229] Based on the above processing, the electronic device can fuse the first image model and the second image model to obtain the target image model, which has higher accuracy.

[0230] In some embodiments, see Figure 11 , Figure 11 This is a structural diagram of an image generation model training system provided in an embodiment of the present invention.

[0231] Figure 11 In this system, the image generation model training system comprises four subsystems: a multi-tenant management subsystem, a resource management subsystem, a machine learning subsystem, and a logic control subsystem.

[0232] The multi-tenant management subsystem manages tenants who train the image generation model system. This subsystem can include multiple servers, and other electronic devices can interact with it via a REST API (Representational State Transfer Application Programming Interface). Each tenant has a unique identity, and the multi-tenant management subsystem records this identity and its corresponding configuration.

[0233] The multi-tenant management subsystem is the management server in the aforementioned embodiments. Tenants may include technicians, designated users, target users, etc., as described in the aforementioned embodiments. Furthermore, tenants may also include an application (APP) that provides services to the first user. The first user can input the target image type into the APP, and then the APP sends the target image type to the electronic device. The identity identifier is the user identifier in the aforementioned embodiments, and the configuration corresponding to the tenant's identity identifier is the storage address of the third image generation model corresponding to the second training task in the aforementioned embodiments.

[0234] The multi-tenant management subsystem determines candidate image generation models based on the similarity between a specified image type and the image type corresponding to a trained image generation model. If the similarity between the candidate image generation model and the specified image type exceeds a preset threshold, the candidate image generation model is designated as the third image generation model. If the similarity is not greater than the preset threshold, a second training server is selected, and a second training task is sent to it. The second training server then executes the received second training task to obtain the third image generation model corresponding to the second training task. This improves the utilization rate of the image generation model training system, enabling natural system reuse and supporting homogeneous needs across various application scenarios.

[0235] Furthermore, the multi-tenant management subsystem can improve data security. When tenants use the image generation model to train the system, each tenant can only access its own corresponding data, avoiding data confusion and interference between different tenants, protecting the data security of each tenant, preventing one tenant from accessing other tenants' data, and reducing the risk of data leakage.

[0236] Furthermore, the multi-tenant management subsystem enhances system flexibility. Each tenant can upload its own image and specify the image type. Subsequently, the image generation model training system can generate a personalized image generation model for each tenant. This personalized image generation model corresponds to the third image generation model for the second training task in the aforementioned embodiment. In other words, it allows each tenant to create its own structured configuration information based on its needs, which is then passed as system function parameters, enabling system function customization.

[0237] The resource management subsystem stores resources. Other electronic devices can interact with the resource management subsystem via a REST API. Resources are the key data involved in business processes. The resource management subsystem allows for structured data modeling of resources, meaning different types of resources are stored separately. Resources include: users, photo sets, models, style templates, training samples, and monitoring metrics.

[0238] The resource management subsystem is the preset database in the aforementioned embodiments.

[0239] The user is the subject of the personalized photo, which includes attributes such as gender, ethnicity, and the app tenant they belong to. The user can be the designated user in the aforementioned embodiments, or the first user. The personalized photo is the image of the designated user in the aforementioned embodiments.

[0240] The photo set is a group of photos, which may include user-uploaded selfies, pre-processed intermediate results, pre-set training category image sets, generated stylized photos, etc. In other words, the photo set may include images of a specified user, images to be processed, target images, etc., as described in the previous embodiments.

[0241] The model is a usable diffusion model, which can be an official base model, a pre-trained style model, or a character optimization model generated in real-time. In other words, the model can include the first image generation model, the second image generation model, the third image generation model, etc., as described in the previous embodiments.

[0242] The style template is the configuration information used for image output, including the character optimization model, style model, the fusion method of the character and style models, prompt words, and other diffusion model output parameters. In other words, the style template can include the model parameters of the first image model and the second image model used when executing the image generation method in the aforementioned embodiments, the preset weights, and the sample text included in the training samples when executing the image generation model training method in the aforementioned embodiments.

[0243] The training template contains configuration information for character optimization training. Each time real-time optimization training is initiated based on a set of user selfies, the configuration parameters contained within the training template are used. In other words, the training template includes the model parameters obtained when training the second image generation model based on the images of a specified user.

[0244] Monitoring metrics are used to record performance data such as success rate and time taken at each processing stage. Specifically, these are the success rates and time taken when executing any step in the image generation model training method described in the aforementioned embodiments, as statistically analyzed by the central server. Subsequently, based on the statistically obtained success rates and time taken, the optimization direction of the image generation model training system can be determined, resulting in a more efficient and stable image generation model training system.

[0245] The machine learning subsystem is used for image generation and training based on a diffusion model. Other electronic devices can interact with the machine learning subsystem via a REST API. The computational demands of the machine learning subsystem are enormous; therefore, compared to other subsystems in the image generation model training system, it is equipped with a high-memory GPU, resulting in higher hardware costs. Furthermore, training the model using the machine learning subsystem requires adjusting nearly a hundred parameters, and a single training session can take 10-30 minutes. When processing the image using the target image generation model, it requires nearly one billion parameters to obtain the target image, and processing the image using the target image generation model takes 5-10 seconds. Therefore, the machine learning subsystem uses an asynchronous mechanism to execute received tasks.

[0246] The machine learning subsystem may include multiple servers, which may include the training server and the server for running the image generation model as described in the foregoing embodiments. The asynchronous mechanism is the selection of a training server from among the multiple idle training servers described in the foregoing embodiments to perform the training task.

[0247] The machine learning subsystem completely encapsulates the complex machine learning node deployment and start / stop mechanisms internally. Logically, it is equivalent to a simple single machine learning service node. Externally, it provides services such as querying, starting, and recycling machine nodes, as well as creating and querying inference training tasks through a REST API service with a fixed address, thus simplifying the usage complexity for external callers to the greatest extent.

[0248] See Figure 12 , Figure 12 This is a structural diagram of a machine learning subsystem provided in an embodiment of the present invention.

[0249] The machine learning subsystem includes an API layer and a multi-cloud adaptation layer.

[0250] The API layer is a module that enables the machine learning subsystem to interact with other subsystems besides the machine learning subsystem. It includes an asynchronous task management module and a node scaling module.

[0251] The node scaling module is responsible for managing machine learning GPU nodes, including node creation, recycling, and status queries. Relevant information is updated to the node database in real time. A node is essentially a running training server as described in the previous embodiments. The module obtains the running status of these training servers and, when a training server is started or stopped (i.e., its running status changes), it can acquire information about the changed training server. The node database is a pre-defined database as described in the previous embodiments.

[0252] The asynchronous task management module includes a training task submodule and an inference task submodule, and can obtain machine learning tasks from a task database. The task database is a preset database as described in the foregoing embodiments.

[0253] The training task submodule is used for fine-tuning the diffusion model, which is equivalent to performing the second training task in the aforementioned embodiment. The inference task submodule is used for inferring and generating images using the diffusion model, which is equivalent to inputting the target user's image to be processed into the target image generation model to obtain the target image in the aforementioned embodiment.

[0254] The creation and querying of tasks in the task database, as well as the opening and closing of nodes in the node database, are handled by the multi-cloud adaptation layer. In other words, the multi-cloud adaptation layer can adjust the running status of training servers based on the number of idle training servers, the number of running training servers, the number of initial training tasks, and preset load indicators. This allows the multi-cloud adaptation layer to reduce the operating cost of the image generation model training system and conserve its computational resources while avoiding excessive load on the system.

[0255] Furthermore, the multi-cloud adaptation layer adapts to the APIs of various cloud platforms, enabling task and node management on each platform. Logically, it encapsulates homogeneous operations across different cloud platforms, shielding them from platform differences. In other words, through the multi-cloud adaptation layer, the image generation model training system can use servers on different cloud platforms. Additionally, the multi-cloud adaptation layer can adjust the distribution ratio of control machine nodes across various cloud platforms according to a preset distribution strategy. This means it can control the number of servers used on each cloud platform, and consequently, adjust the distribution strategy based on the cost-effectiveness and stability of each cloud platform, reducing operating costs.

[0256] The logic control subsystem is responsible for implementing the core control logic of the business logic. It interacts with the multi-tenant subsystem and resource management subsystem based on business requests to acquire and record necessary data; generates specific machine learning training and inference tasks; and, through interaction with the machine learning subsystem, completes the optimization training and rendering of the diffusion model. It also monitors the online operation and load status of the machine nodes in the machine learning subsystem in real time, and dynamically reclaims idle machines or starts new nodes based on load indicators, thereby implementing an elastic strategy. The logic control subsystem is the central server in the aforementioned embodiment. After obtaining the first training task, the central server sends it to the first training server to execute the training task and obtain a trained second image generation model. After the target user inputs the target image type, the central server can forward the target image type to the server running the image generation model. The server running the image generation model then outputs the target image based on the target image type, achieving image rendering. The elastic strategy, as described in the aforementioned embodiment, involves the central server detecting the operating status of multiple training servers and adjusting the number of running training servers in the image generation model training system based on their operating status, thereby adjusting the load of the image generation model training system.

[0257] Based on the above processing, training efficiency can be improved by executing the training tasks corresponding to each image type separately through multiple training servers. Furthermore, for each image type, a first image generation model corresponding to that image type can be trained to obtain a trained image generation model for that image type. Subsequently, users can use this image generation model to obtain the target image of that image type, thus meeting their needs.

[0258] Furthermore, the image generation model training system can run 24 / 7, shortening the template creation and optimization cycle, and resulting in high-quality and stable target images. Additionally, the system can support multiple apps providing stylized avatars to the first user, and its flexible mechanism can adapt to actual traffic changes, reducing the GPU cost of the training system.

[0259] Based on the same inventive concept as the image generation model training method described above, this embodiment of the invention also provides an image generation model training apparatus. The apparatus is applied to the central server of an image generation model training system, which further includes multiple training servers. (See also...) Figure 13 , Figure 13 This is a structural diagram of an image generation model training device provided in an embodiment of the present invention. The device includes:

[0260] The first acquisition module 1301 is used to acquire multiple first training tasks; wherein, a first training task represents training a first image generation model corresponding to the first training task; and the multiple first training tasks correspond to different image types.

[0261] The detection module 1302 is used to detect the operating status of multiple training servers; wherein the operating status of the training servers includes: working status and idle status; the working status indicates that the training server is executing the first training task; the idle status indicates that the training server is not executing the first training task.

[0262] The first selection module 1303 is used to select a first training server from a plurality of training servers in an idle state for each first training task.

[0263] The first sending module 1304 is used to send the first training task to the first training server so that the first training server executes the received first training task, obtains the trained second image generation model, stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the central server.

[0264] Optionally, the detection module 1302 is specifically used for: determining which training servers among a plurality of training servers are in a running state; for each determined training server, sending running state detection information to that training server; and when receiving response information carrying a status identifier from that training server, determining the running state of that training server based on the status identifier; wherein, when the status identifier is a first status identifier, it indicates that the training server is in a working state; and when the status identifier is a second status identifier, it indicates that the training server is in an idle state.

[0265] Optionally, the device further includes:

[0266] The judgment module is used to send running status detection information to each identified training server when the detection module 1302 performs the operation. When the response information carrying the status identifier is received from the training server, the module determines the running status of the training server based on the status identifier and then determines whether the number of training servers in the idle state is less than the number of the first training tasks.

[0267] The first sum calculation module is used to calculate the first difference between the number of first training tasks and the number of training servers in the idle state if the number of training servers in the idle state is less than the number of first training tasks, and to calculate the sum of the number of training servers in the running state and the first difference to obtain the first sum.

[0268] The first ratio calculation module is used to calculate the ratio of the first sum to a preset load index to obtain the first ratio.

[0269] The first number calculation module is used to calculate the difference between the first ratio and the number of training servers in operation to obtain the first number;

[0270] The first startup module is used to start the first number of training servers that are in a closed state.

[0271] Optionally, the device further includes:

[0272] The second ratio calculation module is used to calculate the ratio of the number of training servers in the running state to a preset load index after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of training servers in the idle state is equal to the number of the first training tasks, the module obtains the second ratio.

[0273] The second number calculation module is used to calculate the difference between the second ratio and the number of training servers in operation to obtain the second number;

[0274] The second startup module is used to start the second number of training servers that are in a closed state.

[0275] Optionally, the device further includes:

[0276] The second sum calculation module is used to calculate the sum of the number of training servers in the working state and the number of the first training tasks after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of servers in the idle state is greater than the number of the first training tasks, the module calculates the sum of the number of training servers in the working state and the number of the first training tasks to obtain the second sum.

[0277] The third ratio calculation module is used to calculate the ratio of the second sum to the number of training servers in operation, and obtain the third ratio.

[0278] The fourth ratio calculation module is used to calculate the ratio of the second sum to the preset load index to obtain the fourth ratio.

[0279] The third number calculation module is used to calculate the difference between the fourth ratio and the number of training servers in working state if the third ratio is greater than the preset load index, so as to obtain the third number.

[0280] The third startup module is used to start the third number of training servers that are in a closed state;

[0281] The fourth number calculation module is used to calculate the difference between the number of training servers in working state and the fourth ratio if the third ratio is less than the preset load index, and obtain the fourth number.

[0282] The first shutdown module is used to shut down the fourth number of training servers that are in an idle state.

[0283] Optionally, the first selection module 1303 is specifically used to: for each first training task, according to a preset correspondence between training priority and training server, select a training server from multiple training servers in an idle state that corresponds to the training priority of the first training task, and use it as the first training server for executing the first training task.

[0284] Optionally, the device further includes:

[0285] The second acquisition module is used to acquire a second training task after the first sending module 1304 executes the sending of the first training task to the first training server; wherein the second training task represents training the second image generation model corresponding to a specified image type using images from a specified user;

[0286] The second selection module is used to select a second training server from the training servers that are in an idle state to perform the second training task, according to the running status of multiple training servers.

[0287] The second sending module is used to send the second training task to the second training server, so that the second training server executes the received second training task, obtains the trained third image generation model, stores the third image generation model in a preset database, and returns the storage address of the third image generation model to the central server.

[0288] Optionally, the second training task carries the storage address and specified image type of the specified user's image;

[0289] The second sending module is specifically used to: send the second training task to the second training server, so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from a preset database according to the specified image type carried in the second training task, and use the image of the specified user to train the second image generation model corresponding to the specified image type to obtain a trained third image generation model.

[0290] Optionally, the central server includes a control server and a management server;

[0291] The device further includes:

[0292] The similarity calculation module is used to calculate the similarity between the specified image type and the image type corresponding to the already trained image generation model before the second selection module performs the step of selecting a second training server from the training servers in an idle state according to the running status of multiple training servers to perform the second training task.

[0293] The first determining module is used to determine, from the already trained image generation models, the image generation model with the highest similarity to the specified image type, as a candidate image generation model;

[0294] The second determining module is used to determine the candidate image generation model as the third image generation model corresponding to the second training task when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold.

[0295] The second selection module is specifically used to: when the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, select a second training server from the training servers in an idle state to perform the second training task according to the running status of multiple training servers.

[0296] Optionally, the device further includes:

[0297] The recording module is used to record the correspondence between the third image generation model and the specified user after the second determining module determines the candidate image generation model as the third image generation model when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold.

[0298] Based on the image generation model training apparatus provided in this embodiment of the invention, training efficiency can be improved by having multiple training servers execute training tasks corresponding to each image type. Furthermore, for each image type, a first image generation model corresponding to that image type can be trained to obtain a trained image generation model for that image type. Subsequently, users can obtain the target image of that image type using the image generation model corresponding to that image type, thus meeting user needs.

[0299] Based on the same inventive concept as the image generation method described above, embodiments of the present invention also provide an image generation apparatus. See [link to previous document]. Figure 14 , Figure 14This is a structural diagram of an image generation apparatus provided in an embodiment of the present invention, the apparatus comprising:

[0300] The first acquisition module 1401 is used to acquire the target image type input by the target user;

[0301] The first image generation model determination module 1402 is used to determine, based on the pre-recorded correspondence between image types and image generation models, an image generation model corresponding to the target image type in a preset database, and use it as the first image generation model; wherein, the image generation model stored in the preset database is obtained based on any of the image generation model training methods described in the first aspect above.

[0302] The adjustment module 1403 is used to adjust the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model;

[0303] The target image acquisition module 1404 is used to input the image to be processed of the target user into the target image generation model to obtain the target image.

[0304] Optionally, the device further includes:

[0305] The second acquisition module is used to acquire the image to be processed input by the target user before the target image acquisition module 1404 performs the step of inputting the image to be processed of the target user into the target image generation model to obtain the target image;

[0306] or,

[0307] Obtain the image to be processed from the target user stored locally.

[0308] Optionally, the device further includes:

[0309] The second image generation model determination module is used to determine the image generation model corresponding to the user identifier of the target user in the preset database as the second image generation model before the adjustment module 1403 performs the adjustment of the model parameters of the deployed image generation model based on the first image generation model to obtain the target image generation model.

[0310] The adjustment module 1403 is specifically used for: calculating the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model to obtain the fusion model parameters; adjusting the model parameters of the deployed image generation model to the fusion model parameters to obtain the target image generation model.

[0311] Based on the image generation apparatus provided in this embodiment of the invention, an electronic device can obtain a target image generation model according to the target image type, and then input the image to be processed into the target image generation model to obtain a target image that displays the content of the image to be processed according to the target image type, thereby meeting user needs.

[0312] This invention also provides an electronic device, such as... Figure 15 As shown, it includes a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504. The processor 1501, communication interface 1502, and memory 1503 communicate with each other via the communication bus 1504.

[0313] Memory 1503 is used to store computer programs;

[0314] When the processor 1501 executes the program stored in the memory 1503, it implements the steps of any of the image generation model training methods in the above embodiments, or implements the steps of any of the image generation methods in the above embodiments.

[0315] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0316] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0317] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0318] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0319] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image generation model training methods, or implements the steps of any of the above-described image generation methods.

[0320] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image generation model training methods in the above embodiments, or to implement any of the above image generation methods.

[0321] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0322] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0323] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0324] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for training an image generation model, characterized in that, The method is applied to the central server of an image generation model training system, which further includes multiple training servers. The method includes: Multiple first training tasks are obtained; wherein, a first training task represents training a first image generation model corresponding to the first training task; the multiple first training tasks correspond to different image types; The system monitors the running status of multiple training servers. The running status of a training server includes: working status and idle status. The working status indicates that the training server is executing the first training task; the idle status indicates that the training server is not executing the first training task. For each first training task, select a first training server from among the multiple training servers that are in an idle state to perform the first training task; The first training task is sent to the first training server so that the first training server executes the received first training task, obtains the trained second image generation model, stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the central server. The step of selecting a first training server from a plurality of idle training servers for each first training task includes: For each first training task, the training priority of the first training task is determined according to its attributes; wherein, the attributes of a first training task include at least one of the following: the order of magnitude of the amount of training sample data used when executing the first training task, and the business priority corresponding to the first training task; the higher the business priority corresponding to a first training task, the more users use the image generation model of the image type corresponding to the first training task, and the higher the frequency of users using the image generation model of the image type corresponding to the first training task. For each training server, the training priority is determined based on its configuration; the higher the configuration of a training server, the higher its training priority. From multiple training servers that are in an idle state, select a training server whose training priority is not less than that of the first training task, and obtain the training server corresponding to the training priority of the first training task, and use it as the first training server to execute the first training task.

2. The method according to claim 1, characterized in that, The detection of the operating status of multiple training servers includes: Identify the training servers that are currently running among multiple training servers; For each identified training server, a running status detection message is sent to that training server. When a response message carrying a status identifier is received from the training server, the running status of the training server is determined according to the status identifier. Specifically, when the status identifier is a first status identifier, it indicates that the training server is in a working state; when the status identifier is a second status identifier, it indicates that the training server is in an idle state.

3. The method according to claim 2, characterized in that, After sending runtime status detection information to each identified training server, and upon receiving a response carrying a status identifier from the training server, determining the runtime status of the training server based on the status identifier, the method further includes: Determine if the number of training servers in an idle state is less than the number of the first training tasks; If the number of training servers in an idle state is less than the number of first training tasks, calculate the first difference between the number of first training tasks and the number of training servers in an idle state, and calculate the sum of the number of training servers in a running state and the first difference to obtain the first sum. Calculate the ratio of the first sum to the preset load index to obtain the first ratio; wherein, the preset load index is the load limit of the image generation model training system; the load of the image generation model training system is the ratio of the number of training servers in the image generation model training system that are in working state to the number of training servers that are in running state. Calculate the difference between the first ratio and the number of training servers in operation to obtain the first number; Start the first number of training servers that are currently in a closed state.

4. The method according to claim 3, characterized in that, After determining whether the number of training servers in an idle state is less than the number of the first training tasks acquired, the method further includes: If the number of training servers in an idle state is equal to the number of the first training tasks, calculate the ratio of the number of training servers in a running state to the preset load index to obtain the second ratio. The difference between the second ratio and the number of training servers in operation is calculated to obtain the second number; Start the second number of training servers that are currently in a closed state.

5. The method according to claim 3, characterized in that, After determining whether the number of training servers in an idle state is less than the number of the first training tasks acquired, the method further includes: If the number of idle servers is greater than the number of the first training tasks, calculate the sum of the number of active training servers and the number of the first training tasks to obtain the second sum. Calculate the ratio of the second sum to the number of training servers in operation to obtain the third ratio; Calculate the ratio of the second sum to the preset load index to obtain the fourth ratio; If the third ratio is greater than the preset load index, calculate the difference between the fourth ratio and the number of training servers in working state to obtain the third number; Start the third number of training servers that are currently in a closed state; If the third ratio is less than the preset load index, calculate the difference between the number of training servers in working state and the fourth ratio to obtain the fourth number; Shut down the fourth number of training servers that are in an idle state.

6. The method according to claim 1, characterized in that, After sending the first training task to the first training server, the method further includes: Obtain a second training task; wherein, the second training task means training the second image generation model corresponding to a specified image type using images from a specified user; Based on the running status of multiple training servers, a second training server is selected from the training servers that are in an idle state to perform the second training task. The second training task is sent to the second training server so that the second training server executes the received second training task, obtains the trained third image generation model, stores the third image generation model in a preset database, and returns the storage address of the third image generation model to the central server.

7. The method according to claim 6, characterized in that, The second training task carries the storage address and specified image type of the image of the specified user; The step of sending the second training task to the second training server, so that the second training server executes the received second training task to obtain the trained third image generation model, includes: The second training task is sent to the second training server so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from the preset database according to the specified image type carried in the second training task, and train the second image generation model corresponding to the specified image type using the image of the specified user to obtain the trained third image generation model.

8. The method according to claim 6, characterized in that, The central server includes a control server and a management server; Before selecting a second training server from idle training servers according to the operating status of multiple training servers, the method further includes: The management server calculates the similarity between the specified image type and the image type corresponding to the already trained image generation model; From the already trained image generation models, determine the image generation model that has the greatest similarity to the specified image type for the corresponding image type, and use it as a candidate image generation model; When the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, the candidate image generation model is determined as the third image generation model corresponding to the second training task. The step of selecting a second training server from the idle training servers according to the operating status of multiple training servers to perform the second training task includes: When the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, a second training server for executing the second training task is selected from the training servers in an idle state according to the running status of multiple training servers.

9. The method according to claim 8, characterized in that, After determining the candidate image generation model as the third image generation model corresponding to the second training task when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold, the method further includes: Record the correspondence between the third image generation model corresponding to the second training task and the specified user.

10. An image generation method, characterized in that, The method includes: Obtain the target image type input by the target user; Based on the pre-recorded correspondence between image types and image generation models, an image generation model corresponding to the target image type in a preset database is determined as the first image generation model; wherein, the image generation model stored in the preset database is obtained based on the image generation model training method described in any one of claims 1-9; Based on the model parameters of the first image generation model, the model parameters of the deployed image generation model are adjusted to obtain the target image generation model; The target user's image to be processed is input into the target image generation model to obtain the target image.

11. The method according to claim 10, characterized in that, Before inputting the target user's image to be processed into the target image generation model to obtain the target image, the method further includes: Obtain the image to be processed input by the target user; or, Obtain the image to be processed from the target user stored locally.

12. The method according to claim 10, characterized in that, Before adjusting the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model, the method further includes: Based on the pre-recorded correspondence between user identifiers and image generation models, the image generation model corresponding to the user identifier of the target user in the preset database is determined as the second image generation model; The step of adjusting the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model includes: The fusion model parameters are obtained by calculating the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model. The model parameters of the deployed image generation model are adjusted to the parameters of the fusion model to obtain the target image generation model.

13. An image generation model training device, characterized in that, The device is used as a central server in an image generation model training system, which further includes multiple training servers. The device comprises: The first acquisition module is used to acquire multiple first training tasks; wherein, a first training task represents training a first image generation model corresponding to the first training task; and the multiple first training tasks correspond to different image types. The detection module is used to detect the running status of multiple training servers. The running status of the training servers includes: working status and idle status. The working status indicates that the training server is executing the first training task; the idle status indicates that the training server is not executing the first training task. The first selection module is used to select a first training server from a plurality of training servers that are in an idle state for each first training task. The first sending module is used to send the first training task to the first training server so that the first training server executes the received first training task, obtains the trained second image generation model, stores the second image generation model in a preset database, and returns the storage address of the second image generation model to the central server. The first selection module is specifically used for: For each first training task, the training priority of the first training task is determined according to its attributes; wherein, the attributes of a first training task include at least one of the following: the order of magnitude of the amount of training sample data used when executing the first training task, and the business priority corresponding to the first training task; the higher the business priority corresponding to a first training task, the more users use the image generation model of the image type corresponding to the first training task, and the higher the frequency of users using the image generation model of the image type corresponding to the first training task. For each training server, the training priority is determined based on its configuration; the higher the configuration of a training server, the higher its training priority. From multiple training servers that are in an idle state, select a training server whose training priority is not less than that of the first training task, and obtain the training server corresponding to the training priority of the first training task, and use it as the first training server to execute the first training task.

14. The apparatus according to claim 13, characterized in that, The detection module is specifically used for: Identify the training servers that are currently running among multiple training servers; For each identified training server, a running status detection message is sent to that training server. When a response message carrying a status identifier is received from the training server, the running status of the training server is determined according to the status identifier. Specifically, when the status identifier is a first status identifier, it indicates that the training server is in a working state; when the status identifier is a second status identifier, it indicates that the training server is in an idle state.

15. The apparatus according to claim 14, characterized in that, The device further includes: The judgment module is used to send running status detection information to each identified training server when the detection module performs the operation. When the response information carrying the status identifier is received from the training server, the module determines the running status of the training server based on the status identifier and then determines whether the number of training servers in the idle state is less than the number of the first training tasks. The first sum calculation module is used to calculate the first difference between the number of first training tasks and the number of training servers in the idle state if the number of training servers in the idle state is less than the number of first training tasks, and to calculate the sum of the number of training servers in the running state and the first difference to obtain the first sum. The first ratio calculation module is used to calculate the ratio of the first sum to a preset load index to obtain the first ratio; wherein, the preset load index is the load limit of the image generation model training system; the load of the image generation model training system is the ratio of the number of training servers in the image generation model training system that are in working state to the number of training servers that are in running state. The first number calculation module is used to calculate the difference between the first ratio and the number of training servers in operation to obtain the first number; The first startup module is used to start the first number of training servers that are in a closed state.

16. The apparatus according to claim 15, characterized in that, The device further includes: The second ratio calculation module is used to calculate the ratio of the number of training servers in the running state to a preset load index after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of training servers in the idle state is equal to the number of the first training tasks, the module obtains the second ratio. The second number calculation module is used to calculate the difference between the second ratio and the number of training servers in operation to obtain the second number; The second startup module is used to start the second number of training servers that are in a closed state.

17. The apparatus according to claim 15, characterized in that, The device further includes: The second sum calculation module is used to calculate the sum of the number of training servers in the working state and the number of the first training tasks after the judgment module performs the judgment on whether the number of training servers in the idle state is less than the number of the first training tasks. If the number of servers in the idle state is greater than the number of the first training tasks, the module calculates the sum of the number of training servers in the working state and the number of the first training tasks to obtain the second sum. The third ratio calculation module is used to calculate the ratio of the second sum to the number of training servers in operation, and obtain the third ratio. The fourth ratio calculation module is used to calculate the ratio of the second sum to the preset load index to obtain the fourth ratio. The third number calculation module is used to calculate the difference between the fourth ratio and the number of training servers in working state if the third ratio is greater than the preset load index, so as to obtain the third number. The third startup module is used to start the third number of training servers that are in a closed state; The fourth number calculation module is used to calculate the difference between the number of training servers in working state and the fourth ratio if the third ratio is less than the preset load index, and obtain the fourth number. The first shutdown module is used to shut down the fourth number of training servers that are in an idle state.

18. The apparatus according to claim 13, characterized in that, The device further includes: The second acquisition module is used to acquire a second training task after the first sending module executes the sending of the first training task to the first training server; wherein the second training task represents training the second image generation model corresponding to a specified image type using images from a specified user; The second selection module is used to select a second training server from the training servers that are in an idle state to perform the second training task, according to the running status of multiple training servers. The second sending module is used to send the second training task to the second training server, so that the second training server executes the received second training task, obtains the trained third image generation model, stores the third image generation model in a preset database, and returns the storage address of the third image generation model to the central server.

19. The apparatus according to claim 18, characterized in that, The second training task carries the storage address and specified image type of the image of the specified user; The second sending module is specifically used for: The second training task is sent to the second training server so that the second training server can obtain the image of the specified user according to the storage address of the specified user's image carried in the second training task, and obtain the second image generation model corresponding to the specified image type from the preset database according to the specified image type carried in the second training task, and train the second image generation model corresponding to the specified image type using the image of the specified user to obtain the trained third image generation model.

20. The apparatus according to claim 18, characterized in that, The central server includes a control server and a management server; The device further includes: The similarity calculation module is used to calculate the similarity between the specified image type and the image type corresponding to the already trained image generation model before the second selection module performs the step of selecting a second training server from the training servers in an idle state according to the running status of multiple training servers to perform the second training task. The first determining module is used to determine, from the already trained image generation models, the image generation model with the highest similarity to the specified image type, as a candidate image generation model; The second determining module is used to determine the candidate image generation model as the third image generation model corresponding to the second training task when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold. The second selection module is specifically used for: When the similarity between the image type corresponding to the candidate image generation model and the specified image type is not greater than a preset threshold, a second training server for executing the second training task is selected from the training servers in an idle state according to the running status of multiple training servers.

21. The apparatus according to claim 20, characterized in that, The device further includes: The recording module is used to record the correspondence between the third image generation model and the specified user after the second determining module determines the candidate image generation model as the third image generation model when the similarity between the image type corresponding to the candidate image generation model and the specified image type is greater than a preset threshold.

22. An image generation apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the target image type input by the target user; The first image generation model determination module is used to determine, based on the pre-recorded correspondence between image types and image generation models, an image generation model corresponding to the target image type in a preset database, as the first image generation model; wherein, the image generation model stored in the preset database is obtained based on the image generation model training method according to any one of claims 1-9; The adjustment module is used to adjust the model parameters of the deployed image generation model based on the model parameters of the first image generation model to obtain the target image generation model; The target image acquisition module is used to input the image to be processed of the target user into the target image generation model to obtain the target image.

23. The apparatus according to claim 22, characterized in that, The device further includes: The second acquisition module is used to acquire the image to be processed input by the target user before the target image acquisition module performs the step of inputting the image to be processed of the target user into the target image generation model to obtain the target image; or, Obtain the image to be processed from the target user stored locally.

24. The apparatus according to claim 22, characterized in that, The device further includes: The second image generation model determination module is used to determine the image generation model corresponding to the user identifier of the target user in a preset database, based on the pre-recorded correspondence between user identifiers and image generation models, before the adjustment module performs the adjustment of the model parameters of the deployed image generation model based on the first image generation model to obtain the target image generation model. The adjustment module is specifically used for: The fusion model parameters are obtained by calculating the weighted sum of the model parameters of the first image generation model and the model parameters of the second image generation model. The model parameters of the deployed image generation model are adjusted to the parameters of the fusion model to obtain the target image generation model.

25. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-9, or claims 10-12.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to claims 1-9, or any one of claims 10-12.

Images