AIGC image generation control method, control device and storage medium

Abstract

The invention provides an AIGC (Artificial Intelligence Generation Content) image generation control method and device and a storage medium, and relates to the technical field of artificial intelligence. The AIGC image generation control method comprises the following steps: acquiring an image generation request through an interaction entrance; determining an algorithm service address of a model corresponding to the image generation request; calling a preprocessing algorithm of the model according to the algorithm service address, and obtaining a preprocessing result; calling a generation algorithm of the model according to the algorithm service address and the preprocessing result, and obtaining a generation result; calling a post-processing algorithm of the model according to the algorithm service address and the generation result, and obtaining a post-processing result; calling an auditing system to audit the post-processing result, and obtaining an auditing result; the post-processing result and the audit result are output through an interaction outlet, the pre-processing algorithm and the post-processing algorithm run in the CPU, and the generation algorithm runs in the GPU.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an AIGC (Artificial Intelligence Generated Content) image generation control method, control device and storage medium. Background Art

[0002] In the field of AI (Artificial Intelligence) big models, new models emerge in an endless stream. Using a trained model to provide online services to users, such as supporting users to initiate requests for images generated by AI big models, has some engineering difficulties.

[0003] In the related technology, there are some AI image generation tools that provide services to users by cooperating with channels in the community. Users interact with the robots of the AI ​​image generation tools in the community channels and enter prompts and some other parameters according to the commands provided by the robots. The robots will then return the generated images to the channel. Users need to wait for a while during this interaction, usually within one minute. However, this form of interaction is not a common technical solution for AI image generation, and is limited by the community platform, making it difficult to directly deploy and use on other platforms or enterprises. In addition, the response time of ordinary business interfaces on the Internet is generally in milliseconds, while AI image generation is a long-term task. GPU inference services take a long time, generally in seconds, and functions that support online image generation generally consider asynchronous interaction, which is relatively complex. Summary of the invention

[0004] One purpose of the present disclosure is to improve the scheduling flexibility of the AI ​​image generation algorithm and improve resource utilization.

[0005] According to one aspect of some embodiments of the present disclosure, an AIGC image generation control method is proposed, including: obtaining an image generation request through an interactive entrance; determining the algorithm service address of a model corresponding to the image generation request; calling the preprocessing algorithm of the model according to the algorithm service address, and obtaining the preprocessing result; calling the generation algorithm of the model according to the algorithm service address and the preprocessing result, and obtaining the generation result; calling the post-processing algorithm of the model according to the algorithm service address and the generation result, and obtaining the post-processing result; calling the audit system to audit the post-processing result, and obtaining the audit result; outputting the post-processing result and the audit result through an interactive exit, wherein the pre-processing algorithm and the post-processing algorithm run in a CPU (Central Processing Unit), and the generation algorithm runs in a GPU (Graphics Processing Unit).

[0006] In some embodiments, determining the algorithm service address of the model corresponding to the image generation request includes: querying registration information according to at least one of the business tag and the model name, and obtaining the registration information entry of the corresponding model, wherein the registration information is obtained during the algorithm registration process and stored in the cache, and each registration information includes at least one of the business tag and the model name, and the correspondence between the address of the preprocessing algorithm, the address of the generation algorithm, or the address of the post-processing algorithm of the model; and generating a list of algorithm service addresses in the order of the address of the preprocessing algorithm, the address of the generation algorithm, and the address of the post-processing algorithm.

[0007] In some embodiments, calling the preprocessing algorithm of the model according to the algorithm service address and obtaining the preprocessing result includes: calling the preprocessing algorithm of the model according to the address of the preprocessing algorithm in the list of algorithm service addresses, and obtaining the output of the preprocessing algorithm as the preprocessing result; calling the generation algorithm of the model according to the algorithm service address and the preprocessing result, and obtaining the generation result includes: calling the generation algorithm of the model according to the address of the generation algorithm in the list of algorithm service addresses, and using the preprocessing result as the input of the generation algorithm, and obtaining the output of the generation algorithm as the generation result; calling the post-processing algorithm of the model according to the algorithm service address and the generation result, and obtaining the post-processing result includes: calling the post-processing algorithm of the model according to the address of the post-processing algorithm in the list of algorithm service addresses, and using the generation result as the input of the post-processing algorithm, and obtaining the output of the post-processing algorithm as the post-processing result.

[0008] In some embodiments, outputting the post-processing results and the auditing results through the interactive exit includes: sending the post-processing results and the auditing results to an output message queue, wherein the messages in the output message queue are output one by one through the interactive exit.

[0009] In some embodiments, the registration information also includes a version identifier, and obtaining the registration information entry of the corresponding model includes: obtaining the registration information entry of the latest version of the corresponding model according to the version identifier.

[0010] In some embodiments, the method also includes: in the event of a failure in generating an image, stopping subsequent operations and returning a predetermined business code; retrying to generate the image according to the predetermined business code by executing an algorithm service corresponding to the model, calling an HTTP retry, or adding the image generation request to a retry queue; in the event that the number of retries exceeds a threshold and image generation fails, calling the image in the gallery to output it through an interactive export.

[0011] In some embodiments, the method also includes: when executing a new version test, determining the target version of the model for processing the current image generation request in proportion, wherein the target version is the latest version or the next latest version, and determining the algorithm service address of the model corresponding to the image generation request includes: obtaining the registration information entry of the target version of the corresponding model; when the generated image quality of the latest version of the model is higher than the generated image quality of the next latest version of the model, determining that the latest version test has passed.

[0012] According to one aspect of some embodiments of the present disclosure, an image generation control device is proposed, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the above AIGC image generation control methods based on instructions stored in the memory.

[0013] According to one aspect of some embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, any one of the AIGC image generation control methods described above is implemented.

[0014] According to one aspect of some embodiments of the present disclosure, a computer program product is proposed, including a computer program or instructions, and when the computer program or instructions are executed by a processor, any one of the above AIGC image generation control methods is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute improper limitations on the present disclosure.

[0016] Figure 1 The following is a flowchart of some embodiments of the AIGC image generation control method disclosed in the present invention.

[0017] Figure 2 Flowcharts of other embodiments of the AIGC image generation control method disclosed in the present invention.

[0018] Figure 3 The figure is a schematic diagram of the operation logic of the AIGC image generation control method disclosed in the present invention.

[0019] Figure 4 Schematic diagrams of some embodiments of the AIGC image generation control device disclosed in the present invention.

[0020] Figure 5 Schematic diagrams of other embodiments of the AIGC image generation control device disclosed in the present invention.

[0021] Figure 6 Schematic diagrams of some further embodiments of the AIGC image generation control device disclosed in the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.

[0023] Currently, there can be many business scenarios of AIGC raw image services within the same platform, and each business has its own service entry and exit. The coupling between algorithms and engineering is relatively serious. In addition to training models and being responsible for the effects of the models, algorithm engineers also need to customize the development of each business. It takes a long time to invest in business research and development, and it also takes a lot of energy to maintain the stability of existing online services.

[0024] A complete AIGC algorithm image generation process P mainly consists of three steps: preprocessing algorithm A, generating algorithm B, and post-processing algorithm C. The input of algorithm B depends on the output of algorithm A, and the input of algorithm C depends on the output of algorithm B. In related technologies, algorithm engineers need to build an engineering flow of A->B->C for internal interaction of the algorithm each time they implement a generation process P, and also need to provide an entry and exit for external interaction. There may be multiple image generation processes P for one business. At the same time, AIGC image generation will also have multiple business lines. If the above A, B, and C algorithms are merged into a large algorithm, although the investment time of research and development is reduced for algorithm engineers, the entire algorithm needs to occupy GPU resources to run, and algorithms A and C are CPU services, only the generation algorithm B is a GPU service. When a request comes, the GPU during preprocessing and post-processing is idle, which will cause a waste of GPU resources.

[0025] In response to the above problems, the present invention proposes an AIGC image generation control method, which can manage various algorithms used for AIGC image generation and flexibly schedule corresponding services for user requests; provide a unified entry and exit for image generation, reduce the pressure of business access, improve the efficiency of business launch, and improve the effective utilization of GPU resources.

[0026] The flowcharts of some embodiments of the AIGC image generation control method disclosed in the present invention are as follows: Figure 1 In some embodiments, the AIGC image generation control method is executed by the upper-layer master control system of the AIGC image generation algorithm, which has a unified input and output interface of the system, and the user obtains the AIGC image generation service by calling the interface of the master control system.

[0027] In step S11, an image generation request is obtained through an interactive portal. The interactive portal is an input interface of the master control system. In some embodiments, the image generation request includes at least one of a business tag or a model name. The image generation request may also include user requirements and description information for the generated image, so as to serve as input information of the model to generate an image that meets the user's requirements.

[0028] In some embodiments, the business tag or model name may be determined based on the business line of the user initiating the request to generate an image, or the user's requirement for generating an image, or both.

[0029] In step S12, the algorithm service address of the model corresponding to the image generation request is determined.

[0030] In some embodiments, registration information may be pre-stored in the system, and the registration information may be reported and stored during the algorithm registration process. For example, when an algorithm engineer launches an algorithm, the registration information is reported. In some embodiments, the registration information may be stored in a cache to improve query efficiency. Through the method in this embodiment, multiple algorithms required for a model can be decoupled, which is conducive to the reuse of algorithms, reduces the amount of algorithms stored in the platform, and is also conducive to improving the efficiency of service launch.

[0031] In some embodiments, each piece of registration information includes at least one of a business tag and a model name, and a corresponding relationship with the address of the preprocessing algorithm, the address of the generation algorithm, or the address of the post-processing algorithm of the model. In some embodiments, the registration information also includes the type of the algorithm, which is a preprocessing algorithm, a generation algorithm, or a post-processing algorithm. For example, for a certain preprocessing algorithm, the registration information includes the business tag / model name, the address of the preprocessing algorithm, and the preprocessing algorithm type identifier. In some embodiments, the registration information may also include a version identifier to distinguish the version of the algorithm corresponding to the registration information.

[0032] In some embodiments, the calling interface can register the algorithm (service) to the master control system, and the service registration can rely on Redis implementation. For example, the Key in the data structure (KV) is "asc:url:{label}:{model}:{version}:pre", where the K prefix is ​​fixed to the constant asc, which is the Java system name; url is the table name, indicating that the service url is stored; [label] is a variable to distinguish business scenarios; [model] is a variable to distinguish model names; [version] is the model version used by the current generated service; [pre] is an enumeration type to distinguish processing stages. The current enumeration set is pre, gen, and post, corresponding to the three stages of preprocessing, generation, and post-processing respectively; V is a specific algorithm service address, which can be a URL.

[0033] In the above step S12, the registration information is queried according to at least one of the business tag and the model name to obtain the corresponding model registration information entry. In some embodiments, for an image generation request, at least three registration information entries need to be matched to obtain the addresses of three types of algorithms, including the address of the pre-processing algorithm, the address of the generation algorithm, and the address of the post-processing algorithm; then the algorithm service address is in the order of the address of the pre-processing algorithm, the address of the generation algorithm, and the address of the post-processing algorithm to generate a list of algorithm service addresses.

[0034] In some embodiments, when the registration information includes a version identifier, the latest version of the registration information entry of the corresponding model is obtained according to the version identifier so as to adopt the latest version of the algorithm.

[0035] In steps S13-S15 shown below, the algorithms are called in the order from first to last in the list of algorithm service addresses.

[0036] In step S13, the preprocessing algorithm of the model is called according to the algorithm service address, and the preprocessing result is obtained.

[0037] In some embodiments, the preprocessing algorithm of the model is called according to the address of the preprocessing algorithm in the list of algorithm service addresses, the requirement information provided by the user in the image generation request is input, and the output of the preprocessing algorithm is obtained as the preprocessing result.

[0038] In step S14, the generation algorithm of the model is called according to the algorithm service address and the preprocessing result, and the generation result is obtained.

[0039] In some embodiments, the generation algorithm of the model is called according to the address of the generation algorithm in the list of algorithm service addresses, and the preprocessing result in step S13 is used as the input of the generation algorithm to obtain the output of the generation algorithm as the generation result.

[0040] In step S15, the post-processing algorithm of the model is called according to the algorithm service address and the generated result, and the post-processing result is obtained.

[0041] In some embodiments, the post-processing algorithm of the model is called according to the address of the post-processing algorithm in the list of algorithm service addresses, and the generated result in step S14 is used as the input of the post-processing algorithm to obtain the output of the post-processing algorithm as the post-processing result, which is the final processing result of the model.

[0042] In some embodiments, the above-mentioned preprocessing algorithm and post-processing algorithm run in the CPU, and the generation algorithm runs in the GPU, so that the GPU can be allocated to other image generation requests during the execution stage of the preprocessing algorithm and post-processing algorithm, avoiding waste of resources caused by the GPU being idle.

[0043] The algorithm is called through the above steps S13-S15. Further, step S16 is executed.

[0044] In step S16, the audit system is called to audit the post-processing results and obtain the audit results.

[0045] In step S17, the post-processing result and the audit result are outputted through the interactive outlet, which is a unified output port of the master control system.

[0046] In some embodiments, the post-processing results and the audit results can be sent to an output message queue, and the messages in the output message queue are output one by one through the interactive export. Through such a method, the results can be output one by one in the case of relatively busy business, which improves reliability and avoids confusion.

[0047] Based on the method in the embodiment shown above, it is possible to flexibly schedule corresponding services for user requests, decouple the relationship between algorithms, and improve the efficiency of algorithm launch; provide a unified entry and exit for image generation, reduce the pressure of business access, and improve the efficiency of business launch; and improve the effective utilization of GPU resources.

[0048] In some embodiments, based on the method shown above, the AIGC image generation control method of the present disclosure may also include: Figure 2 The process shown in .

[0049] In step 201, during the execution of the task of generating an image, it is determined whether a failure to generate an image occurs. In some embodiments, the failure may be caused by a busy GPU. If a failure to generate an image occurs, step 202 is executed, otherwise the subsequent process of generating an image continues until the image is output.

[0050] In some embodiments, the task execution failure may be generated during the operation of the generation algorithm. In some embodiments, the task execution failure may be generated during the operation of the generation algorithm. Figure 1 Occurs at any step in the process shown.

[0051] In step 202, the subsequent operation is stopped and the predetermined service code is returned. In some embodiments, the subsequent operation may be Figure 1 In the operation steps shown in , specifically, the subsequent operations refer to the subsequent steps starting from the step where the failure to generate an image occurs.

[0052] In some embodiments, the predetermined service code may be a pre-set service code for identifying busyness, so as to identify image generation failure caused by GPU busyness.

[0053] In step 203, when it is determined that the predetermined service code is triggered, the image generation is retried. In some embodiments, the image generation can be retried in any one or more of the following ways: by re-calling the algorithm service corresponding to the model, calling HTTP retry, and adding the image generation request to the retry queue.

[0054] In some embodiments, the method selected for retrying to generate an image may be determined according to the business corresponding to the image generation request, so that the retry method matches the corresponding business scenario.

[0055] In the related technology, it is difficult to perceive the real-time online image generation. Based on the method in the embodiment shown above, it is possible to quickly respond to the failure of image generation, understand the cause of failure through the feedback business code, and improve the perception of business operation status; through retry, the probability of successfully feeding back images that meet user needs can be increased.

[0056] In some embodiments, Figure 2 As shown, the AIGC image generation control method disclosed in the present invention also includes step 204 and step 205.

[0057] In steps 204-205, it is determined whether the retry to generate the image fails. If the retry to generate the image fails and the number of retries does not exceed the number threshold, the process returns to step 203; if the number of retries exceeds the number threshold, the process proceeds to step 205; if the retry to generate the image succeeds, the process proceeds according to Figure 1 The steps in the illustrated embodiment execute subsequent operations until an image is output.

[0058] In step 206, the image in the gallery is called and outputted through the interactive outlet. In some embodiments, the gallery may include multiple stored images, and the target image is determined in the gallery randomly, sequentially, or according to the matching relationship between the image and the business, and outputted through the interactive outlet.

[0059] Based on the method in the embodiment shown above, when multiple retries still fail, a fallback operation can be performed through the gallery, thereby improving the user-friendliness of the system.

[0060] In some embodiments, a unified monitoring point can be set up in the above-mentioned master control system, with a matching monitoring dashboard and alarm strategy to monitor the real-time request volume, sending volume and image generation link time, improve the ability to grasp the business operation status, improve operation reliability, and also help staff to discover and eliminate problems in a timely manner.

[0061] In some embodiments, the AIGC image generation control method disclosed in the present invention can also assist the process of algorithm launch and perform AB testing. For example, when performing a new version test, the image generation request is diverted, some of which call the latest version of the algorithm that needs to be tested, and some call the next latest version of the algorithm. The effects of the images generated by the two algorithms are compared to determine whether the latest version of the algorithm can achieve the optimization effect, thereby realizing the testing of the new version of the algorithm. In some embodiments, the latest version of the algorithm can be called for a small number (for example, 20%) of image generation requests, so as to maintain the stability of the service and reduce the impact of the test on user use.

[0062] For a single image generation request during the AB test, the target version of the model for processing the current image generation request is determined in proportion, and the target version is the latest version or the next latest version. Then, in the process of determining the algorithm service address of the model corresponding to the image generation request, the registration information entry of the target version of the model corresponding to the request is obtained according to the version identifier identified in the registration information entry. By processing image generation requests for services associated with the newly launched algorithm within a predetermined number or a predetermined time length, if the quality of the generated images of the latest version of the model is higher than the quality of the generated images of the next latest version of the model, it is determined that the latest version has passed the test.

[0063] Based on the method in the embodiment shown above, the reliability of the online algorithm can be improved and the efficiency of version testing can be improved.

[0064] The operation logic diagram of the AIGC image generation control method disclosed in the present invention is as follows: Figure 3 As shown in . The control method and image generation method of AIGC image generation run in the AIGC service center. When the business system receives the image generation request, it sends the message to the message queue Kafka for storage. In addition, the business system sends Http signaling to call the Rest controller.

[0065] The image generation requests in the queue are executed one by one, triggering the scheduler of the master control system, submitting tasks to the resource library to the thread pool, and the thread pool assigns the execution tasks. In some embodiments, concurrent scheduling can be used, such as multiple threads calling services at the same time, using CountDownLatch and CompetableFuture to obtain asynchronous thread execution results for aggregation, to achieve the purpose of speeding up, thereby improving service efficiency.

[0066] The task execution process may be as shown in any of the above embodiments. In some embodiments, the generated final image is output via the message queue Kafka.

[0067] Based on the method in the embodiment shown above, it can be responsible for the interaction with the business system externally, provide a unified entry and exit, and reduce the pressure of business access; the process of internal algorithm interaction is shielded externally, and the business system only needs to submit a request for image generation, and then obtain the result from the specified message queue; it is responsible for scheduling each algorithm module internally, and meeting the input and output dependencies between each algorithm module, decoupling algorithms and projects, and reusing common logic, such as input parameter verification, audit services, etc., performing pre-interception and post-processing, so that the algorithm block is more focused and its responsibilities are unified.

[0068] In some embodiments, the AIGC image generation control method disclosed in the present invention can also provide a function of switching the traffic of the new and old versions of the model online in real time, and support the registration information of multiple versions of model services online at the same time. The model version information here will be passed in when calling the algorithm service registration interface. The version of each model that is initially launched for the first time is the initial value (for example, 0). After that, the registration interface can be called to register the +1 version for the iterative model effect. When a raw image request is received, multiple versions of a model under a certain business tag will be queried when querying the cache. By default, the service registration URL list with the largest version will be selected for calling. For models with the AB experiment function enabled, ASC will randomly distribute user traffic according to the proportion requirements based on the proportion in the dynamic configuration, such as 20% of the traffic to the latest version model and 80% of the traffic to the next latest version model. In this way, when the new version model is launched to replace the old version, a small amount of user traffic can be allocated to the new version online first, and the picture effect finally sent by ASC can be compared and analyzed. After comprehensively evaluating the new version of the model, all traffic can be switched to the new version.

[0069] In some embodiments, the AIGC image generation control method disclosed herein may also provide image generation retry and image library backup functions. Due to the characteristics of the GPU image generation service, one image generation will increase the GPU utilization rate to more than 90%. Sometimes the online traffic is too large, and the image generation service of a certain model is called too intensively. The GPU will be busy and cause the image generation to fail. In this case, the algorithm service will return immediately with a business code indicating busyness. The ASC will retry the image generation service based on this business code. Depending on the scenario, some will call HTTP retries, and some will join the retry queue. The algorithm module pulls requests based on its own consumption capacity to ensure the success of the image generation. The image library backup is to provide a backup image and return it to the user after the retry fails.

[0070] Based on the method in the embodiment shown above, it is possible to realize flexible registration and calling of multi-version algorithm services, flexible plug-in and unplug of algorithm services, and build a unified service chain. There is no need to develop external interactions and interactions between algorithm modules separately for each business line. Each business line only needs to agree on a business label or model identifier to access the AIGC online image generation service of an existing model online, which is conducive to improving the efficiency of business launch.

[0071] After testing on the platform, using the master control system running the AIGC image generation control method disclosed in this disclosure, in the first scenario, the average daily online image generation volume is about 2.4W, and the image generation success rate reaches 99.981%; in the second scenario, the average daily image generation volume supported by ASC is about 1.6W, and the image generation success rate reaches 99.957%. The P99 time of image generation in both scenarios is within 15s, and the average time fluctuates around 3.5s.

[0072] The flowcharts of some embodiments of the AIGC image generation control device disclosed in the present invention are as follows: Figure 4 In some embodiments, the AIGC image generation control device may be located in or serve as a master control system of the AIGC image generation platform.

[0073] The input unit 41 can obtain an image generation request through an interactive portal. The interactive portal is an input interface of the master control system. In some embodiments, the image generation request includes at least one of a business tag or a model name. The image generation request may also include the user's requirements and description information for the generated image, so as to serve as input information of the model to generate an image that meets the user's requirements. In some embodiments, the above-mentioned business tag or model name can be determined based on the business line of the user who initiated the image generation request, or the user's requirements for the generated image, or both.

[0074] The address determination unit 42 can determine the algorithm service address of the model corresponding to the image generation request. The address determination unit 42 queries the registration information according to at least one of the business tag and the model name to obtain the registration information entry of the corresponding model. In some embodiments, for an image generation request, it is necessary to match at least three registration information entries to obtain the addresses of three types of algorithms, including the address of the preprocessing algorithm, the address of the generation algorithm, and the address of the post-processing algorithm; then the algorithm service address is in the order of the address of the preprocessing algorithm, the address of the generation algorithm, and the address of the post-processing algorithm to generate a list of algorithm service addresses.

[0075] In some embodiments, when the registration information includes a version identifier, the latest version of the registration information entry of the corresponding model is obtained according to the version identifier so as to adopt the latest version of the algorithm.

[0076] The algorithm calling unit 43 can call the algorithms in the order from first to last in the list of algorithm service addresses.

[0077] First, the algorithm calling unit 43 calls the preprocessing algorithm of the model according to the algorithm service address and obtains the preprocessing result. In some embodiments, the preprocessing algorithm of the model is called according to the address of the preprocessing algorithm in the list of the algorithm service address, the requirement information provided by the user in the image generation request is input, and the output of the preprocessing algorithm is obtained as the preprocessing result.

[0078] Further, the algorithm calling unit 43 calls the generation algorithm of the model according to the algorithm service address and the preprocessing result, and obtains the generation result. In some embodiments, the generation algorithm of the model is called according to the address of the generation algorithm in the list of algorithm service addresses, and the preprocessing result is used as the input of the generation algorithm, and the output of the generation algorithm is obtained as the generation result.

[0079] Finally, the algorithm calling unit 43 calls the post-processing algorithm of the model according to the algorithm service address and the generated result, and obtains the post-processing result. In some embodiments, the post-processing algorithm of the model is called according to the address of the post-processing algorithm in the list of the algorithm service address, and the generated result is used as the input of the post-processing algorithm, and the output of the post-processing algorithm is obtained as the post-processing result, which is the final processing result of the model.

[0080] In some embodiments, the above-mentioned preprocessing algorithm and post-processing algorithm run in the CPU, and the generation algorithm runs in the GPU, so that the GPU can be allocated to other image generation requests during the execution stage of the preprocessing algorithm and post-processing algorithm, avoiding waste of resources caused by the GPU being idle.

[0081] The audit unit 44 can call the audit system to audit the post-processing results and obtain the audit results.

[0082] The output unit 45 can output the post-processing results and the audit results through the interactive export. The interactive export is a unified output port of the master control system. In some embodiments, the post-processing results and the audit results can be sent to the output message queue, and the messages in the output message queue are output one by one through the interactive export. Through such a method, the results can be output one by one when the business is relatively busy, which improves reliability and avoids confusion.

[0083] Such an AIGC image generation control device can flexibly schedule corresponding services for user requests, decouple the relationship between algorithms, and improve the efficiency of algorithm launch; provide a unified entry and exit for image generation, reduce the pressure of business access, and improve business launch efficiency; and improve the effective utilization of GPU resources.

[0084] In some embodiments, the AIGC image generation control device may further include an exception handling unit, which can determine whether a failure to generate an image occurs during the execution of the task of generating an image. If a failure to generate an image occurs, subsequent operations are stopped and a predetermined business code is returned. In some embodiments, the predetermined business code may be a pre-set business code that identifies busyness, so as to identify a failure to generate an image due to a busy GPU. When it is determined that the predetermined business code is triggered, the image is retried. In some embodiments, retrying to generate an image may be performed in any one or more of the following ways: by executing the algorithm service corresponding to the re-calling model, calling HTTP retry, and adding the image generation request to the retry queue. In some embodiments, the method selected for retrying to generate an image can be determined based on the business corresponding to the image generation request, so that the retry method matches the corresponding business scenario.

[0085] Based on the device in the embodiment shown above, it is possible to quickly respond to the situation of image generation failure, understand the cause of failure through the feedback business code, and improve the ability to perceive the business operation status; through retries, the probability of successfully feeding back an image that meets user needs can be increased.

[0086] In some embodiments, the exception handling unit can also determine whether a retry to generate an image fails. If the image generation fails after multiple retries, the image in the gallery is called and output through the interactive export. In some embodiments, the gallery may include multiple stored images, and the target image is determined in the gallery by random, sequential, or according to the matching relationship between the image and the business, and output through the interactive export.

[0087] Based on the method in the embodiment shown above, in the event of a retry failure, a fallback operation can be performed through the gallery, thereby improving the user-friendliness of the system.

[0088] In some embodiments, the AIGC image generation control device may also include a monitoring unit, which can monitor the embedding points, monitor the real-time request volume, the sending volume and the time consumption of the image generation link, and schedule the alarm strategy, thereby improving the ability to grasp the business operation status, improving the operation reliability, and also facilitating the staff to discover and eliminate problems in a timely manner.

[0089] In some embodiments, the AIGC image generation control device may also include a testing unit that can assist in the process of algorithm launch and perform AB testing. For example, when performing a new version test, the image generation request is diverted, some of which call the latest version of the algorithm that needs to be tested, and some call the next latest version of the algorithm. The effects of the images generated by the two algorithms are compared to determine whether the latest version of the algorithm can achieve the optimization effect, thereby testing the new version of the algorithm. In some embodiments, the latest version of the algorithm can be called for a small number (e.g., 20%) of image generation requests, so as to maintain the stability of the service and reduce the impact of the test on user use.

[0090] For a single image generation request during the AB test, the test unit determines the target version of the model for processing the current image generation request in proportion, and the target version is the latest version or the next latest version. Then, in the process of determining the algorithm service address of the model corresponding to the image generation request, the registration information entry of the target version of the model corresponding to the request is obtained according to the version identifier identified in the registration information entry. By processing image generation requests for services associated with the newly launched algorithm within a predetermined number or a predetermined time length, if the quality of the generated images of the latest version of the model is higher than the quality of the generated images of the next latest version of the model, it is determined that the latest version test has passed.

[0091] Such a device can improve the reliability of the online algorithm and increase the efficiency of version testing.

[0092] A structural schematic diagram of an embodiment of the image generation control device disclosed in the present invention is as follows Figure 5 As shown. The image generation control device includes a memory 501 and a processor 502. Wherein: the memory 501 can be a disk, a flash memory or any other non-volatile storage medium. The memory is used to store the instructions in the corresponding embodiment of the AIGC image generation control method above. The processor 502 is coupled to the memory 501 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 502 is used to execute the instructions stored in the memory, which can improve the scheduling flexibility of the AI ​​picture generation algorithm and improve the utilization of resources.

[0093] In one embodiment, it is also possible to Figure 6 As shown, the image generation control device 600 includes a memory 601 and a processor 602. The processor 602 is coupled to the memory 601 via a BUS bus 603. The image generation control device 600 can also be connected to an external storage device 605 via a storage interface 604 to call external data, and can also be connected to a network or another computer system (not shown) via a network interface 606. No further detailed description will be given here.

[0094] In this embodiment, by storing data instructions in a memory and then processing the instructions through a processor, the scheduling flexibility of the AI ​​image generation algorithm can be improved and resource utilization can be improved.

[0095] In another embodiment, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method in the corresponding embodiment of the AIGC image generation control method. Those skilled in the art should understand that the embodiments of the present disclosure can be provided as methods, devices, or computer program products. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0100] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0101] It should be noted that the terms "first", "second", etc. in the specification, claims, and drawings of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present disclosure rather than to limit it. Although the present disclosure has been described in detail with reference to the preferred embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present disclosure, which should be included in the scope of the technical solution for protection of the present disclosure.

Claims

1. A method for controlling the generation of an AIGC image, comprising: Obtain image generation request through interactive entry; Determine the algorithm service address of the model corresponding to the image generation request; Calling the preprocessing algorithm of the model according to the algorithm service address and obtaining the preprocessing result; According to the algorithm service address and the preprocessing result, calling the generation algorithm of the model and obtaining the generation result; Calling the post-processing algorithm of the model according to the algorithm service address and the generated result, and obtaining the post-processing result; Calling the audit system to audit the post-processing results and obtain the audit results; Output the post-processing result and the audit result through an interactive export, The pre-processing algorithm and the post-processing algorithm are run in a central processing unit (CPU), and the generation algorithm is run in a graphics processing unit (GPU).

2. The image generation control method according to claim 1, wherein: The determining of the algorithm service address of the model corresponding to the image generation request includes: Querying registration information according to at least one of the business tag and the model name, and obtaining a registration information entry of the corresponding model, wherein the registration information is obtained and stored in the cache during the algorithm registration process, and each piece of the registration information includes at least one of the business tag and the model name, and a corresponding relationship between the address of the preprocessing algorithm, the address of the generation algorithm, or the address of the post-processing algorithm of the model; The algorithm service addresses are arranged in the order of the address of the pre-processing algorithm, the address of the generating algorithm and the address of the post-processing algorithm to generate a list of the algorithm service addresses.

3. The image generation control method according to claim 2, wherein: The calling the preprocessing algorithm of the model according to the algorithm service address and obtaining the preprocessing result comprises: calling the preprocessing algorithm of the model according to the address of the preprocessing algorithm in the list of the algorithm service address, and obtaining the output of the preprocessing algorithm as the preprocessing result; The calling the generation algorithm of the model according to the algorithm service address and the preprocessing result and obtaining the generation result comprises: calling the generation algorithm of the model according to the address of the generation algorithm in the list of the algorithm service address, and using the preprocessing result as the input of the generation algorithm, and obtaining the output of the generation algorithm as the generation result; Calling the post-processing algorithm of the model according to the algorithm service address and the generated result, and obtaining the post-processing result includes: calling the post-processing algorithm of the model according to the address of the post-processing algorithm in the list of the algorithm service address, and using the generated result as the input of the post-processing algorithm, and obtaining the output of the post-processing algorithm as the post-processing result.

4. The image generation control method according to claim 1, wherein: Outputting the post-processing result and the audit result through the interactive export includes: The post-processing result and the audit result are sent to an output message queue, wherein the messages in the output message queue are output one by one through the interaction exit.

5. The image generation control method according to claim 2, wherein: The registration information also includes a version identifier. The obtaining of the registration information entry of the corresponding model includes: obtaining the registration information entry of the latest version of the corresponding model according to the version identifier.

6. The image generation control method according to any one of claims 1 to 5, further comprising: In the case of image generation failure, subsequent operations are stopped and the predetermined service code is returned; Retrying to generate an image by re-calling the algorithm service corresponding to the model, calling HTTP retry, or adding the image generation request to a retry queue according to the predetermined business code; When the number of retries exceeds a threshold and image generation fails, the image in the call gallery is output through the interactive exit.

7. The image generation control method according to claim 5, further comprising: In case of new version testing, Determining a target version of a model for processing a current image generation request according to the ratio, wherein the target version is a latest version or a second latest version, and determining an algorithm service address of a model corresponding to the image generation request comprises: obtaining a registration information entry of the target version of the corresponding model; In a case where the quality of the generated image of the latest version of the model is higher than the quality of the generated image of the second latest version of the model, it is determined that the test of the latest version has passed.

8. An image generation control device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method according to any one of claims 1 to 7 based on instructions stored in the memory.

9. A non-transitory computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images