A method and device for scheduling a request for a graph
By using a dynamic mapping table to achieve precise scheduling of text image requests, the performance bottleneck of the traditional text image service architecture in high-concurrency scenarios is solved, and the system's processing efficiency and response speed are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NEW H3C TECH CO LTD
- Filing Date
- 2026-02-25
- Publication Date
- 2026-06-12
Smart Images
Figure CN122195645A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a text-based image request scheduling method and apparatus. Background Technology
[0002] With the rapid development of AI-generated content technology, text-based graph models have been widely applied. However, current mainstream text-based graph service architectures are generally designed for single-concurrency or limited-concurrency scenarios, making it difficult to handle the demands of large-scale concurrent user access. Specifically, traditional solutions typically employ a single-concurrency processing mode, where subsequent requests must enter a waiting queue for sequential processing; even in multi-concurrency scenarios, they can only rely on the load balancing mechanisms of systems such as Kubernetes for simple request distribution and instance round-robin.
[0003] This traditional approach has significant drawbacks: when faced with high concurrency requests, the system service response latency is noticeable, and the system processing efficiency is low. Even with scaling up by adding service instances, new requests are still distributed to each instance via a round-robin method. Faced with different model requirements, service instances need to frequently switch between different models, resulting in a large amount of overhead for model switching, loading, and initialization. This severely consumes inference computing resources, leading to overall low efficiency of the Wensheng Graph service system and an inability to effectively support continuous high concurrency requests. Summary of the Invention
[0004] This application provides a text graph request scheduling method and apparatus to solve the problem of huge overhead and response delay caused by frequent service instance switching in high-concurrency scenarios in traditional text graph service architecture.
[0005] Specifically, this application provides the following technical solution: Firstly, this application provides a method for scheduling text-to-image requests, applied to the service interface of a text-to-image service system, the method comprising: Parse the text graph request to determine the target access model of the text graph request; By querying the dynamic mapping table, it is determined whether there is an idle service instance with the effective model being the target access model. The dynamic mapping table is used to record the effective models of each service instance in the text image service system in real time. If it exists, the text image request will be forwarded to the idle service instance for processing; If it does not exist, the text image request will be forwarded to any idle service instance of the text image service system to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
[0006] Secondly, this application provides a text image request scheduling device, applied to the service interface of a text image service system, the device comprising: The first module is used to parse the text image request in order to determine the target access model of the text image request; The second module is used to determine whether there is an idle service instance with the target access model as the effective model by querying the dynamic mapping table. The dynamic mapping table is used to record the effective model of each service instance in the text image service system in real time. The third module is used to forward the text image request to an idle service instance for processing if there is an idle service instance with the effective model being the target access model. The fourth module is used to forward the text image request to any idle service instance of the text image service system if there is no idle service instance with the effective model being the target access model, so as to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
[0007] Thirdly, this application provides a text-to-image service system, including a service interface and multiple backend service instances; The service interface is used to schedule text-based image requests in the following manner: parsing the text-based image request to determine the target access model of the text-based image request; querying a dynamic mapping table to determine whether there is an idle service instance with the target access model as the effective model, wherein the dynamic mapping table is used to record the effective model of each service instance in real time; if it exists, forwarding the text-based image request to the idle service instance for processing; if it does not exist, forwarding the text-based image request to any idle service instance to trigger the idle service instance to switch the effective model to the target access model and process the text-based image request.
[0008] Fourthly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described above.
[0009] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described above.
[0010] The technical solution provided in this application has the following beneficial effects: This application uses a dynamic mapping table to perceive the effective model of each service instance in real time. During the scheduling of text graph requests, requests are forwarded to idle service instances that have loaded the target model first, avoiding unnecessary model switching overhead. When there is no corresponding idle service instance, the idle service instance is triggered to switch models, thus achieving accurate matching between requests and service instances.
[0011] Compared to the traditional load balancing mechanism that relies on round-robin, this application can effectively cope with large-scale high-concurrency scenarios, significantly improve the overall processing efficiency and response speed of the text image service system, avoid the waste of computing resources caused by frequent switching, and provide efficient and stable technical support for continuous high-concurrency text image services.
[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0013] The accompanying drawings, which are incorporated in and form part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0014] Figure 1 A flowchart illustrating the text-based image request scheduling method provided in this application embodiment; Figure 2 A schematic diagram illustrating the text-based image request scheduling process provided in an embodiment of this application; Figure 3 A schematic diagram of a text image request scheduling device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the text-to-image service system provided in an embodiment of this application. Detailed Implementation
[0015] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the embodiments of this application is only used to describe specific embodiments of this application and is not intended to limit this application.
[0016] This application aims to address the performance bottleneck of traditional text graph service architectures in high-concurrency scenarios. Specifically, the traditional polling mechanism distributes requests evenly based on the number of instances without considering the current model state loaded by each service instance. As a result, requests may be assigned to instances that have not loaded the target model, causing service instances to frequently switch between different text graph models. This generates a large amount of overhead for model loading, initialization, and memory switching, which severely reduces system throughput and response speed.
[0017] Therefore, this application provides a text image request scheduling method and apparatus to achieve the following objectives: significantly improve the ability of the text image service system to support large-scale concurrent requests; optimize request forwarding and processing efficiency to maximize the utilization of computing resources; and improve user experience and response time in high-concurrency scenarios.
[0018] The practical application of this application will be described in detail below through specific embodiments.
[0019] This application provides a text-based image request scheduling method, such as... Figure 1 As shown, the method may include the following steps: Step 110: Parse the text-based graph request to determine the target access model of the text-based graph request; Step 120: By querying the dynamic mapping table, determine whether there is an idle service instance with the target access model as the effective model. The dynamic mapping table is used to record the effective models of each service instance in the text image service system in real time. If it exists, proceed to step 130; otherwise, proceed to step 140. Step 130: Forward the text image request to the idle service instance for processing; Step 140: Forward the text image request to any idle service instance of the text image service system to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
[0020] This application applies to a text-based image service system designed for large-scale concurrency. For example... Figure 2 As shown, the system includes a unified external service interface for receiving text graph requests submitted by users, and multiple backend service instances for carrying out specific inference tasks. Each service instance loads and runs only one specific text graph model at any given time.
[0021] The service interface of this application maintains a dynamic mapping table in real time. The core function of this table is to record the currently effective text graph model of each service instance in the system in real time, that is, to establish a one-to-one correspondence between "service instance identifier - effective model identifier". The table will be updated in real time as the service instance model switching process is carried out to ensure the accuracy of the mapping relationship.
[0022] In this application, the request content of the Wenshengtu request includes the target model identifier (which can be a type identifier) and a prompt word, and the specific format can be request(model, prompt). Generally speaking, different users will request different models and prompt words, but the initial effective model of each service instance in the Wenshengtu service system is consistent.
[0023] The service interface of this application acts as an intelligent scheduler, responsible for dispatching text-to-image requests to appropriate service instances for processing. Specifically, for a text-to-image request to be processed, the service interface determines the target access model (i.e., the model actually requested by the request) by parsing the request; then, the service interface queries its own maintained dynamic mapping table to select an idle service instance whose current effective model is the target access model; finally, the text-to-image request is forwarded to the idle service instance for processing. After receiving the text-to-image request, since the current effective model is the target access model of the text-to-image request, there is no need to perform a model switching operation. It directly calls the current effective model to perform text-to-image inference calculation based on the prompt words in the text-to-image request, generates the image result corresponding to the text-to-image request, and returns the image result to the user through the service interface, thus completing the entire text-to-image request processing flow.
[0024] Accordingly, if the service interface, after querying the dynamic mapping table, finds that there is no idle service instance with the target access model as the effective model, it forwards the text image request to any idle service instance to trigger that idle service instance to switch its current effective model to the target access model and process the text image request. Along with the model switching process of the idle service instance, the service interface synchronously updates the dynamic mapping table.
[0025] Optionally, to improve request scheduling efficiency, the above-mentioned forwarding of the text image request to any idle service instance can specifically be: finding an idle service instance of the text image service system and forwarding the text image request to the first found idle service instance.
[0026] Furthermore, to cope with high-concurrency scenarios, this application supports batch processing of text image requests.
[0027] Specifically, when the number of pending text image requests reaches a preset threshold, the service interface executes a batch processing procedure; wherein, the batch processing procedure includes: For the current batch of text-based image requests, determine the target access model for each request and obtain the correspondence between the model and the request list; determine the effective model for each service instance in the text-based image service system and obtain the correspondence between the model and the instance list; select each model as the current model in descending order of the number of requests in the request list. If the current model has a corresponding instance list, then batch assign the text-based image requests in the request list corresponding to the current model to the service instances in the instance list corresponding to the current model; otherwise, batch assign the text-based image requests in the request list corresponding to the current model to any idle service instance in the text-based image service system.
[0028] The above batch processing procedure can be specifically divided into the following steps: Step 1: For the current batch of text-based image requests, determine the target access model for each text-based image request, summarize the access requests for each text-based image model, and obtain the correspondence between the model and the request list. The above correspondence between models and request lists refers to the categorization of all text image requests within the current batch according to their target access models, forming a mapping relationship from "model identifier" to "request set". Assuming that in the current batch, the target access model for text image requests 1, 2, and 3 is model A, and the target access model for requests 4 and 5 is model B, then the correspondence can be generated as follows: Model A -> {Request 1, Request 2, Request 3}, Model B -> {Request 4, Request 5}.
[0029] Optionally, all pending text image requests at the current moment can be used as the text image requests for the current batch, or a preset number of pending text image requests can be used as the text image requests for the current batch, such as processing 20 text image requests per batch.
[0030] Step 2: Determine the effective models for each service instance in the text graph service system, summarize the service instances of each text graph model, and obtain the correspondence between the model and instance list; The above correspondence between models and instance lists refers to classifying all service instances of the Wenshengtu service system according to their effective models, forming a mapping relationship from "model identifier" to "instance set". Assuming that in the Wenshengtu service system, the effective model for service instances 1, 2, and 3 is model A, and the effective model for instance 4 is model C, then the following correspondence can be generated: Model A → {Instance 1, Instance 2, Instance 3}, Model C → {Instance 4}.
[0031] Step 3: From the correspondence between the models and the request list, select the model with the most requests and denote it as the current model. Obtain the request list corresponding to the current model. Based on the correspondence between the models and the instance list, determine whether there is an instance list corresponding to the current model. If there is, proceed to step 4; otherwise, proceed to step 5. Step 4: Batch distribute the text image requests from the request list corresponding to the current model to the instances in the instance list corresponding to the current model for processing; Step 5: Batch distribute the text image requests in the request list corresponding to the current model to any idle service instance, trigger the idle service instance to switch the effective model to the current model, and update the correspondence between the model and the instance list; Preferably, the idle service instance with the lowest load (no active model) or the lowest model switching cost (shortest switching time, all requests for the currently active model have been processed) in the Wenshengtu service system can be identified, and the Wenshengtu requests in the request list corresponding to the current model can be batch-assigned to the idle service instance for processing.
[0032] Step 6: After allocating all text image requests in the request list corresponding to the current model, remove the current model and its corresponding request list from the model-request list correspondence. Step 7: Determine whether there are any unprocessed models in the correspondence between the model and the request list; if so, return to step 3 until there are no unprocessed models in the correspondence between the model and the request list; if not, proceed to step 8. Step 8: Confirmation. The batch processing of the current batch of text image requests has now been completed.
[0033] Assuming the text image service system has four text image models, and the batch processing workflow processes 20 text image requests per batch, then the following scenarios are included: If the number of requests for each text graph model is equal, and each model has 5 requests, then the 5 requests for model A can be batch-distributed to each instance with model A as the effective model, then the 5 requests for model B can be batch-distributed to each instance with model B as the effective model, and so on.
[0034] If the number of requests for each text image is unequal, then in descending order of the number of requests, the model with the most requests is determined in each round, and the requests from the request list corresponding to that model are assigned to the instances whose effective model is that model. For example, if model A has 15 requests and model B has 5 requests, then the 15 requests for model A are first assigned to the instances whose effective model is model A, and then the 5 requests for model B are assigned to the instances whose effective model is model B.
[0035] In extreme cases, if the target access model of the requests in the current batch is the same, the requests in the current batch will be distributed to all instances for processing, triggering all instances to switch the effective model to the target model together.
[0036] As can be seen, this application further introduces a batch processing flow when the number of pending requests reaches a preset threshold: first, requests are categorized according to the target access model; then, instances are categorized according to the effective model of the service instance; and finally, requests are batch-allocated in descending order of the number of requests. This approach can maximize the utilization of service instances that have already loaded the target model, significantly reduce the frequency of model switching, and thus greatly reduce the fixed overhead of model loading and initialization.
[0037] Furthermore, this application supports sequential processing of text image requests according to priority.
[0038] In this application, the request content of the text image request may include not only the target model identifier and prompt words mentioned above, but also the priority identifier qos, and the specific format may be request(model, prompt, qos).
[0039] Specifically, the service interface of this application is also used to perform the following process: parsing the text image request to determine the priority of the text image request; when there are multiple text image requests to be processed, the text image requests of each priority are sequentially used as the current scheduling target in descending order of priority.
[0040] As an example, the value of the priority parameter QoS can be set to 0, 1, or 2. Here, 0 represents the highest priority, 1 the medium priority, and 2 the lowest priority. Therefore, in practical applications, the service interface first schedules requests with priority 0, then schedules requests with priority 1, and finally schedules requests with priority 2.
[0041] Based on the same inventive concept, this application also provides a text-based image request scheduling device, which is applied to the service interface of a text-based image service system.
[0042] like Figure 3 As shown, the device includes: The first module is used to parse the text image request in order to determine the target access model of the text image request; The second module is used to determine whether there is an idle service instance with the target access model as the effective model by querying the dynamic mapping table. The dynamic mapping table is used to record the effective model of each service instance in the text image service system in real time. The third module is used to forward the text image request to an idle service instance for processing if there is an idle service instance with the effective model being the target access model. The fourth module is used to forward the text image request to any idle service instance of the text image service system if there is no idle service instance with the effective model being the target access model, so as to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
[0043] Optionally, the fourth module forwards the text image request to any idle service instance in the following way: Find an idle service instance of the Wensheng Image Service System and forward the Wensheng Image request to the first found idle service instance.
[0044] Optionally, the device may also include: The fifth module is used to execute a batch processing procedure when the number of pending text image requests reaches a preset threshold; wherein the batch processing procedure includes: For the current batch of text-based image requests, determine the target access model for each request and obtain the correspondence between the model and the request list; determine the effective model for each service instance in the text-based image service system and obtain the correspondence between the model and the instance list; select each model as the current model in descending order of the number of requests in the request list. If the current model has a corresponding instance list, then batch assign the text-based image requests in the request list corresponding to the current model to the service instances in the instance list corresponding to the current model; otherwise, batch assign the text-based image requests in the request list corresponding to the current model to any idle service instance in the text-based image service system.
[0045] Optionally, the above batch processing flow also includes: Use all pending text image requests at the current moment as the text image requests for the current batch, or use a preset number of pending text image requests as the text image requests for the current batch.
[0046] Optionally, the above-mentioned batch allocation of Wensheng image requests from the request list corresponding to the current model to any idle service instance of the Wensheng image service system can be as follows: Identify the idle service instance with the lowest load or the lowest model switching cost in the text image service system, and batch assign the text image requests from the request list corresponding to the current model to that idle service instance for processing.
[0047] Optionally, the device may also include: The sixth module is used to parse the text-to-image request to determine its priority; The seventh module is used to, when there are multiple pending text image requests, sequentially select text image requests of each priority level as the current scheduling target in descending order of priority.
[0048] This application also provides a text-based image service system, such as... Figure 4 As shown, it includes service interfaces and multiple backend service instances; The service interface is used to schedule text-based image requests in the following manner: parsing the text-based image request to determine the target access model of the text-based image request; querying a dynamic mapping table to determine whether there is an idle service instance with the target access model as the effective model, wherein the dynamic mapping table is used to record the effective model of each service instance in real time; if it exists, forwarding the text-based image request to the idle service instance for processing; if it does not exist, forwarding the text-based image request to any idle service instance to trigger the idle service instance to switch the effective model to the target access model and process the text-based image request.
[0049] This application also provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the various functions or steps of the above method embodiments.
[0050] The aforementioned computer-readable storage media include, but are not limited to, any of the following: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media capable of storing program code.
[0051] This application also provides a computer program product that, when run on a computer, causes the computer to perform various functions or steps of the above method embodiments.
[0052] The computer-readable storage medium and computer program product provided in the embodiments of this application are used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0053] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various variations or substitutions can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A text-based graph request scheduling method, characterized in that, A service interface applied to a text-based image service system, the method comprising: Parse the text graph request to determine the target access model of the text graph request; By querying the dynamic mapping table, it is determined whether there is an idle service instance with the effective model being the target access model. The dynamic mapping table is used to record the effective models of each service instance in the text image service system in real time. If it exists, the text image request will be forwarded to the idle service instance for processing; If it does not exist, the text image request will be forwarded to any idle service instance of the text image service system to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
2. The method according to claim 1, characterized in that, The step of forwarding the text image request to any idle service instance includes: Find an idle service instance of the Wensheng Image Service System and forward the Wensheng Image request to the first found idle service instance.
3. The method according to claim 1, characterized in that, The method further includes: When the number of pending text image requests reaches a preset threshold, a batch processing procedure is executed. The batch processing procedure includes: For each batch of raw image requests, determine the target access model for each raw image request and obtain the correspondence between the model and the request list; Determine the effective model for each service instance in the text image service system, and obtain the correspondence between the model and the instance list; In descending order of the number of requests in the request list, each model is selected as the current model. If a corresponding instance list exists for the current model, the text image requests in the request list corresponding to the current model are batch-assigned to the service instances in the instance list corresponding to the current model. Otherwise, the text image requests in the request list corresponding to the current model are batch-assigned to any idle service instance of the text image service system.
4. The method according to claim 3, characterized in that, The batch processing procedure also includes: Use all pending text image requests at the current moment as the text image requests for the current batch, or use a preset number of pending text image requests as the text image requests for the current batch.
5. The method according to claim 3, characterized in that, The step of batch allocating the Wensheng image requests from the request list corresponding to the current model to any idle service instance of the Wensheng image service system includes: Identify the idle service instance with the lowest load or the lowest model switching cost in the text image service system, and batch assign the text image requests from the request list corresponding to the current model to that idle service instance for processing.
6. The method according to claim 1, characterized in that, The method further includes: Parse the text-based image request to determine its priority; When there are multiple pending text graph requests, text graph requests of each priority level are selected as the current scheduling targets in descending order of priority.
7. A text-based image request scheduling device, characterized in that, The device, which is used as a service interface for a text-based image service system, includes: The first module is used to parse the text image request in order to determine the target access model of the text image request; The second module is used to determine whether there is an idle service instance with the target access model as the effective model by querying the dynamic mapping table. The dynamic mapping table is used to record the effective model of each service instance in the text image service system in real time. The third module is used to forward the text image request to an idle service instance for processing if there is an idle service instance with the effective model being the target access model. The fourth module is used to forward the text image request to any idle service instance of the text image service system if there is no idle service instance with the effective model being the target access model, so as to trigger the idle service instance to switch the effective model to the target access model and process the text image request.
8. A text-to-image service system, characterized in that, This includes service interfaces and multiple backend service instances; The service interface is used to schedule text-based image requests in the following manner: parsing the text-based image request to determine the target access model of the text-based image request; querying a dynamic mapping table to determine whether there is an idle service instance with the target access model as the effective model, wherein the dynamic mapping table is used to record the effective model of each service instance in real time; if it exists, forwarding the text-based image request to the idle service instance for processing; if it does not exist, forwarding the text-based image request to any idle service instance to trigger the idle service instance to switch the effective model to the target access model and process the text-based image request.
9. A computer-readable storage medium comprising computer instructions, characterized in that, When the computer instructions are executed on the electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-6.
10. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-6.