An algorithm module scheduling method
By introducing the principle of aligning drafting depth and page granularity as integer multiples in the multi-module algorithm framework, the problem of unbalanced utilization of cached page resources is solved, achieving efficient utilization of cached pages and improving the stability of system performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING DARONGXIN TECHNOLOGY GROUP CO LTD
- Filing Date
- 2025-12-12
- Publication Date
- 2026-06-02
AI Technical Summary
In existing technologies, multi-module algorithm frameworks suffer from unbalanced resource utilization in cache page management, leading to fluctuations in task batch loading and affecting system scheduling balance and performance.
By introducing the principle of aligning drafting depth with page granularity as an integer multiple, unified scheduling parameters for the drafting and verification modules are determined, ensuring that the length of the generated result matches the page allocation granularity and stabilizing the system's parallel batch processing capability.
It achieves high consistency in cache page utilization, reduces storage waste, improves maximum concurrent micro-batch capacity and end-to-end throughput, and stabilizes system performance.
Smart Images
Figure CN121742999B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of algorithm module scheduling technology, and more specifically, to an algorithm module scheduling method. Background Technology
[0002] In recent years, multi-module algorithm frameworks (especially inference systems) have required multiple model or algorithm stages to work collaboratively in a single task. For example, in speculative generative inference, the front-end module is responsible for generating draft results, while the back-end module is responsible for verifying and retaining the acceptable parts. While this mechanism improves average execution speed, due to statistical differences in the runtime and output length of the two modules, the system must use cached pages to store intermediate results to maintain the continuity of the data flow. Existing systems typically manage cache space with fixed page sizes for fast memory allocation and reclamation. However, the length of the draft generated by the algorithm is not fixed, and the acceptance rate varies significantly between different requests. As a result, when there is a deviation between the generated content and the page granularity, some pages cannot be fully filled, and the remaining space cannot be reused immediately, causing the actual number of available pages to fluctuate over time. This fluctuation directly affects the loading capacity of task batches, thereby disrupting the system's scheduling balance.
[0003] Specifically, the drafting module reserves multiple pages at once to store temporary results when generating drafts; after the verification module confirms some results, only some pages are retained indefinitely, while the remaining pages, although marked as recyclable, often cannot be immediately reallocated because a page is the smallest unit of management; the uneven distribution of acceptance rates for different tasks causes the occupancy and release rhythm of cached pages to become uneven, resulting in periodic peaks and troughs; the scheduler fails to recognize this periodic resource occupancy pattern and continues to schedule batch processing sizes according to the average value, causing the system to experience increased latency when page utilization is low and resource contention when utilization is high. Summary of the Invention
[0004] This invention provides an algorithm module scheduling method that solves the technical problem of how to determine the unified scheduling parameters of the drafting and verification modules under the condition of limited cache page resources, so that the length of the generated result and the page allocation granularity are periodically matched, thereby stabilizing the parallel batch processing capability of the system and reducing latency fluctuations.
[0005] This invention provides an algorithm module scheduling method, comprising:
[0006] Get the page size of the key-value cache, the total page budget, and the draft acceptance rate;
[0007] Based on the draft acceptance rate, determine the target drafting depth so that the expected length of the accepted prefix is aligned with an integer multiple of the page size;
[0008] Based on the peak page occupancy of a single request corresponding to the target drafting depth, the maximum concurrent micro-batch capacity is calculated using the total page budget.
[0009] The cycle time is determined based on the service rates of the drafting module and the verification module.
[0010] Based on the maximum concurrent micro-batch capacity and cycle time, requests are mapped to specific time coordinates and slots in the order of arrival, and the scheduling results containing the target drafting depth and mapping relationship are output.
[0011] The beneficial effects of this invention include: by introducing an integer multiple alignment principle between drafting depth and page granularity during the algorithm module scheduling process, a definite correspondence is established between the length of the retained draft prefix and the size of the key-value cache page, achieving precise utilization and periodic stable allocation of page resources. This method can maintain high consistency in cache page utilization under different draft acceptance rates, significantly reducing storage waste caused by intra-page fragmentation; simultaneously, by establishing a mathematical correspondence between the scheduling cycle period and the concurrent micro-batch capacity, the execution rhythm between multiple modules is synchronized in both time and space dimensions, thereby stably improving the maximum concurrent micro-batch capacity and end-to-end throughput without increasing page resources, and improving the capacity oscillation and performance degradation problems that occur in traditional scheduling when page allocation granularity is mismatched. Attached Figure Description
[0012] Figure 1 The paging resonance effect of this invention can significantly improve the maximum concurrent capacity graph;
[0013] Figure 2 This is a heatmap comparing the memory page utilization rates of the paging resonance scheduling of the present invention with those of the traditional scheduling.
[0014] Figure 3 This is a flowchart of an algorithm module scheduling method according to the present invention. Detailed Implementation
[0015] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.
[0016] like Figure 1 As shown, Figure 1 The scheduling method and comparison method of this invention are presented at different draft acceptance rates. Maximum concurrent micro-batch capacity Relationship curve; experimental parameters fixed at page size Maximum drafting depth Total budget The dashed lines represent a comparison method, following... The increase is gradual; the solid line represents the method of this invention, and the curve shows a plateau and a step-like trend with obvious capacity peaks in several intervals. The gray area in the figure represents the capacity gain compared to the comparative method; the positions marked as resonance peaks correspond to when the expected length of the retained prefix is close to an integer multiple of the page size, resulting in a significant increase in the capacity of concurrent micro-batches with minimal in-page space; the resonance gain area between adjacent peaks represents the segment where the capacity is consistently better than the comparative method; the step effect is at the integer multiple alignment boundary, where if the drafting depth is not selected according to the rules of this invention, in-page fragmentation increases and causes capacity to fall back. Figure 1 This demonstrates that, given page size, maximum drafting depth, and total page budget, deterministically selecting the drafting depth to match the desired retention prefix with the page granularity can transform the discrete constraints of page resources into predictable capacity leaps, thereby significantly improving the maximum concurrent microbatch capacity across a wide range of draft acceptance rates.
[0017] like Figure 2 As shown, Figure 2 The distribution of memory page utilization in a two-dimensional space, representing draft acceptance rate and drafting depth, was compared between the traditional alignment method and the scheduling method of this invention, under the same page size of 16. The left figure shows the traditional method, with the horizontal axis representing draft acceptance rate and the vertical axis representing drafting depth. Grayscale indicates actual page utilization. Large areas of low to medium utilization and significant fragmentation are visible, indicating that when the length of the retained prefix does not match the page size, the free space within the page is difficult to reuse, leading to long-term inefficiency. The right figure shows the method of this invention, exhibiting multiple bright high-utilization bands under the same coordinate system and color scale, with utilization approaching 100%. These high-utilization bands correspond to the calculation of the expected length of the retained prefix and the selection of a drafting depth that matches this length to an integer multiple of the page size. This ensures stable matching of page allocation and release within the timeframe, significantly reducing fragmentation. Figure 2 This demonstrates that, with a wide range of draft acceptance rates and drafting depths, the present invention can transform previously scattered low-utilization areas into predictable high-utilization zones, enabling full and stable use of memory page resources.
[0018] like Figure 3 As shown, an algorithm module scheduling method includes:
[0019] Get the page size of the key-value cache, the total page budget, and the draft acceptance rate;
[0020] Based on the draft acceptance rate, determine the target drafting depth so that the expected length of the accepted prefix is aligned with an integer multiple of the page size;
[0021] Based on the peak page occupancy of a single request corresponding to the target drafting depth, the maximum concurrent micro-batch capacity is calculated using the total page budget.
[0022] The cycle time is determined based on the service rates of the drafting module and the verification module.
[0023] Based on the maximum concurrent micro-batch capacity and cycle time, requests are mapped to specific time coordinates and slots in the order of arrival, and the scheduling results containing the target drafting depth and mapping relationship are output.
[0024] In one embodiment of the invention, page size, total page budget, and draft acceptance rate include:
[0025] The page size is the number of key-value entries that a single paginated key-value cache page can hold;
[0026] The total page budget is the total number of physical pages in system memory that can be allocated and used simultaneously.
[0027] The draft acceptance rate is the statistical value of the probability that a single draft entry will be retained by the validation module.
[0028] Define three physical and statistical parameters for the scheduling method of this algorithm:
[0029] Page size is denoted as Unit: Items / Page;
[0030] Total page budget is recorded as Unit: Page;
[0031] The draft acceptance rate is recorded as ; Unit: dimensionless;
[0032] It should be noted that existing large language model inference engines (such as the PagedAttention mechanism) do not allocate GPU memory per token, but rather use pagination management. Ignoring page size ( This can lead to a mismatch between the granularity of memory allocation and reclamation, resulting in severe internal fragmentation.
[0033] Total page budget ( The key-value cache (KVCache) is a hard physical limit imposed by the hardware (GPU memory) on the key-value cache, which determines the maximum capacity of the system.
[0034] Draft acceptance rate ( ) is an attribute in the speculative decoding scenario, reflecting the probability of passing the small model drafting and large model validation.
[0035] In the specific implementation of the project, and Directly read the configuration files of the self-inference engine (such as vLLM's block_size and gpu_memory_utilization). This is obtained through runtime sliding window statistics. Specifically, (initialization) a fixed-length window is maintained in memory. First-in-first-out queue (e.g.) (Record) Each reasoning step. After completion, the number of draft tokens for this step ( ) and "Number of accepted tokens" Enqueue; if the queue is full, remove the oldest set of data. (Calculate) Calculate the average acceptance rate within the window in real time: (Update) The calculated The value is directly used as the draft acceptance rate for the next scheduling cycle.
[0036] In one embodiment of the present invention, determining a target drafting depth, based on the draft acceptance rate, to align the desired length of the accepted prefix with an integer multiple of the page size includes:
[0037] For each candidate drafting depth, subtract the draft acceptance rate raised to the power of the candidate drafting depth from the first value to obtain the first value; multiply the draft acceptance rate by the first value to obtain the second value; subtract the draft acceptance rate from the first value to obtain the third value; divide the second value by the third value to obtain the expected length of the continuous prefix retained by the verification module.
[0038] Divide the expected length of the consecutive prefixes by the page size, round the quotient to the nearest integer, and multiply the rounded result by the page size to obtain an integer multiple of the alignment length.
[0039] The absolute value of the difference between the expected length of consecutive prefixes and the integer multiple of the alignment length is calculated, and the candidate drafting depth that minimizes the absolute value of the difference is determined as the target drafting depth.
[0040] Finding the optimal target drafting depth is denoted as .
[0041] Step 1: Calculate the expected length of the consecutive prefixes ;
[0042] Subtracting the candidate drafting depth power of the draft acceptance rate from the value one... yields the expected length of the consecutive prefixes retained by the validated module. ;
[0043] in, Indicates the depth of candidate drafting. The candidate drafting depth is raised to the power of the draft acceptance rate. Indicates the first value. Indicates the second value. This represents the third numerical value.
[0044] Step 2: Calculate the integer multiple alignment length ;
[0045] Divide the expected length of the consecutive prefixes by the page size, round the quotient, and so on to obtain an integer multiple of the alignment length. ;
[0046] in, This indicates the rounding operation. Indicates the page size.
[0047] Step 3: Determine the target drafting depth ;
[0048] Calculate the absolute value of the difference between the expected length of consecutive prefixes and the integer multiple of the alignment length... and determine it as the target drafting depth. ;
[0049] in, This represents the absolute value of the difference.
[0050] It should be noted that in speculative decoding, the actual length of the accepted token sequence is a random variable, following a truncated geometric distribution, with its mean being... .when Not divisible by page size This means that, statistically, each request will generate a non-full tail page (i.e., internal fragmentation). This fragmentation is amplified in continuous batch processing, significantly reducing memory utilization.
[0051] In one embodiment of the present invention, the peak page occupancy per single request corresponding to the target drafting depth includes:
[0052] Divide the target drafting depth by the page size, and round up the resulting quotient to get the number of pages occupied by the draft.
[0053] Divide the expected length of the consecutive prefixes retained by the verified module corresponding to the target drafting depth by the page size, and round up the resulting quotient to obtain the number of pages to be retained.
[0054] Add the number of pages used for drafting to the number of pages used for retention to get the peak page usage per request.
[0055] Through mathematical optimization, the amount of effective data generated by statistical expectation is forced to be limited. ) and the storage boundary of physical pages ( Alignment.
[0056] When the absolute value of the difference is at its minimum, the statistical level of memory fragmentation is reduced to a minimum, and the system enters a resonance state, thus enabling more concurrent requests to be packed into the limited memory.
[0057] In the worst-case scenario, the peak page occupancy of a single request under resonant conditions is calculated and denoted as... .
[0058] Step 1: Calculate the number of pages required for the draft. ;
[0059] Divide the target drafting depth by the page size and round the resulting quotient up. ;
[0060] Step 2: Calculate the number of pages to be reserved ;
[0061] Divide the expected length corresponding to the target drafting depth by the page size, and round up the resulting quotient. ;
[0062] Step 3: Calculate peak page usage per single request ;
[0063] Add the number of pages used for drafting to the number of pages used for retention. ;
[0064] Drafting occupancy ( This means that during the drafting phase, the system must pre-allocate memory space for all generated draft tokens, regardless of whether they are ultimately accepted. This is a rigid requirement for physical allocation.
[0065] Reserved ( This indicates that after the verification phase, the system needs to reserve memory for the surviving valid prefixes.
[0066] It should be noted that in the engineering implementation of pipelined parallel or asynchronous recycling, in order to ensure the absolute safety of scheduling, we adopt a conservative estimation strategy. That is, within this scheduling cycle, there is an overlap peak between the space required for drafting and the space reserved after the previous round of verification.
[0067] This represents the depth of drafting for this optimized target. The maximum page-level resource budget required to process a request.
[0068] In one embodiment of the present invention, the maximum concurrent micro-batch capacity is calculated using the total page budget based on the peak page occupancy of a single request corresponding to the target drafting depth, including:
[0069] Divide the total page budget by the peak page usage per request to obtain the quotient. Perform a floor operation on the quotient and use the result of the floor operation as the maximum concurrent micro-batch capacity.
[0070] Since it has already been done Resonant alignment was performed, at this point It is actually near the integer boundary of a local minimum. Therefore, based on this... The calculated peak occupancy is the minimum value after fragmentation optimization, which lays the foundation for maximizing batch processing capacity in the future.
[0071] Calculate the maximum parallel processing capability of a system constrained by video memory, i.e., the maximum concurrent micro-batch size. .
[0072] in, This indicates the maximum concurrent micro-batch capacity (unit: number of requests / batch). This represents the total page budget (in pages), corresponding to the total number of physical pages in the weighted system memory that can be allocated and used simultaneously. This represents the peak page usage per request (unit: pages / request), corresponding to the sum of the number of pages used for drafting and the number of pages used for reservation. This indicates a floor function, which performs a floor operation on the quotient value.
[0073] It's important to note that in inference systems, video memory (KVCache) is the absolute bottleneck resource. To ensure system stability, video memory overflow (OOM) is strictly prohibited. Therefore, the maximum concurrency of the system is... Must meet .
[0074] By division and rounding down Calculate the total budget under the worst-case scenario (PeakUsage). The number of integer requests that can be safely handled.
[0075] By optimizing the target drafting depth This makes the page occupancy of a single request... It reached a local minimum (i.e., eliminated intra-page fragmentation).
[0076] because It was reduced (as a denominator). (As a merchant) it will increase accordingly.
[0077] It should be noted that, due to The existence of right The changes exhibit a step-function response. Resonant alignment is precisely to ensure... Falling into the ability to make The optimal range for leaping to the next level is found, thereby maximizing the system's throughput.
[0078] In one embodiment of the present invention, determining the cycle time based on the service rates of the drafting module and the verification module includes:
[0079] Divide the target drafting depth by the drafting module service rate to obtain the time required for the drafting stage; divide the expected length of the continuous prefix retained by the verification module corresponding to the target drafting depth by the verification module service rate to obtain the time required for the verification stage; select the larger value between the time required for the drafting stage and the time required for the verification stage to determine the cycle time.
[0080] A unified time base for scheduling algorithm modules is determined, namely the cycle time (denoted as ). ).
[0081] Step 1: Calculate the time required for the drafting phase. ;
[0082] Step 2: Calculate the time required for the verification phase. ;
[0083] Step 3: Determine the beat period ;
[0084] in, Indicates the target drafting depth (unit: entries). This indicates the drafting module service rate (unit: entries / second), meaning the number of tokens generated by the drafting model per second. Indicates the time required for the drafting phase (in seconds). Indicates the expected length (in entries) of the consecutive prefixes retained by the module being verified. This indicates the service rate of the verification module (unit: items / second), meaning the number of tokens verified by the verification model per second. Indicates the time required for the verification phase (in seconds). This indicates the cycle time (in seconds), which is the length of the time window for unified scheduling. This indicates that the operation will select the one with the larger value.
[0085] It should be noted that an architecture combining continuous batching and speculative decoding is employed. To achieve deterministic scheduling (i.e., without complex dynamic queue checks), the system requires a unified heartbeat or beat. Within the same beat, the system needs to ensure that both drafting and verification actions have sufficient time to complete; otherwise, pipeline congestion or data races may occur.
[0086] It should be noted that, It is generated The time it takes to draft a document It is a verification The time of each valid token. Take the larger of the two. This means that the system sets a uniform pace according to the speed of the slowest component. Although this may seem to be limited by the slowest module, combined with off-peak slot mapping, this deterministic waiting eliminates the additional overhead caused by resource contention (RaceCondition) and frequent context switching (ContextSwitching) in dynamic scheduling.
[0087] In one embodiment of the present invention, requests are mapped to determined time coordinates and slots in the order of arrival based on the maximum concurrent micro-batch capacity and cycle time, and a scheduling result containing the target drafting depth and mapping relationship is output, including:
[0088] For each request numbered from zero in arrival order, divide the request number by the maximum concurrent micro-batch capacity to obtain the quotient and remainder; round down the quotient to determine the batch index of the request; determine the slot index of the request in the corresponding tick of the batch index; multiply the batch index by the uniform tick period to obtain the drafting start time of the request.
[0089] Add the drafting start time to the drafting phase duration to obtain the verification start time of the request;
[0090] Construct and output the scheduling results, which include the target drafting depth, maximum concurrent micro-batch capacity, cycle time, and a spatiotemporal mapping relationship for any request number consisting of the drafting start time, the verification start time, and the slot index.
[0091] Establish a unique and deterministic mapping from the order of request arrival to the spatiotemporal coordinates of execution.
[0092] set up The request index is numbered from zero according to the arrival order. ).
[0093] Step 1: Calculate the batch index With slot index ;
[0094] Divide the request number by the maximum concurrent micro-batch capacity to obtain the quotient and remainder; round down the quotient to determine the batch index of the request; determine the slot index of the request in... , ; Equivalent to That is, remainder operation;
[0095] Step 2: Calculate the drafting start time ;
[0096] Multiply the batch index by the cycle time to obtain the start time of the request's drafting. ;
[0097] Step 3: Calculate the start time of the verification ;
[0098] Add the drafting start time to the required drafting duration to obtain the verification start time of the request. ;
[0099] Step 4: Construct the scheduling result ;
[0100] Construct and output scheduling results...including target drafting depth, maximum concurrent micro-batch capacity, cycle time, and... spatiotemporal mapping relationship. ;
[0101] in, The request number (a dimensionless integer) represents the global order in which the request arrives in the system. This indicates the maximum concurrent micro-batch capacity (unit: number of requests / batch). The batch index (a dimensionless integer) represents the time window batch to which the request was assigned. The slot index (a dimensionless integer) represents the specific channel location in video memory occupied by the request (range of values). to ), Indicates the beat period (unit: seconds). Indicates the start time of drafting (in seconds), referring to the absolute point in time when the request entered the drafting module. Indicates the time required for the drafting phase (in seconds). Indicates the start time of verification (unit: seconds), referring to the absolute point in time when the request entered the verification module. This represents the scheduling result, and is a collection object containing static parameters and dynamic mapping functions.
[0102] It's important to note that traditional scheduling methods typically employ a First-Come, First-Served (FCFS) queue combined with dynamic memory allocation via a memory pool. When a request arrives, the scheduler needs to check in real-time whether there is available memory; if not, it blocks the request. This decision-making logic involves numerous conditional branches (if has_memory then run else wait), which can lead to unpredictable tail latency under high concurrency.
[0103] Calculated using the preceding steps (Maximum capacity based on paging resonance optimization) and (Beat) constructs a virtual two-dimensional grid coordinate system. The horizontal axis represents time (in seconds). The vertical axis represents the step size; the horizontal axis represents the space (with the vertical axis representing the step size). (For modulus). Any request Without needing to query the current system status, the corresponding spatiotemporal coordinates can be directly calculated using modular arithmetic based solely on the identifier. .
[0104] It should be noted that, due to yes right Take the remainder, and It is based on the total page budget The rigorously calculated safety upper limit is therefore mapped to the same beat. within Each request will inevitably consume a certain amount of video memory. This ensures that there is no need for runtime memory checks and that OOM (Out of Memory) will never occur.
[0105] It should be noted that, The design ensures that the verification module starts immediately after the drafting module. Because... The definition covers the slowest link, and this rigid timing ensures that no congestion occurs between the two modules, thus achieving the theoretical maximization of throughput.
[0106] Scheduling results As a wrapper object, it provides all the necessary control parameters, allowing the downstream inference engine (Executor) to execute only by table.
[0107] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.
Claims
1. A method for scheduling algorithm modules, characterized in that, include: Retrieve the page size, total page budget, and draft acceptance rate of the key-value cache, including: The page size is the number of key-value entries that a single paginated key-value cache page can hold; The total page budget is the total number of physical pages in system memory that can be allocated and used simultaneously. The draft acceptance rate is the statistical value of the probability that a single draft entry will be retained by the validation module. Based on the draft acceptance rate, determine the target drafting depth so that the expected length of the accepted prefix is aligned to an integer multiple of the page size, including: For each candidate drafting depth, subtract the draft acceptance rate raised to the power of the candidate drafting depth from the first value to obtain the first value; multiply the draft acceptance rate by the first value to obtain the second value; subtract the draft acceptance rate from the first value to obtain the third value; divide the second value by the third value to obtain the expected length of the continuous prefix retained by the verification module. Divide the expected length of the consecutive prefixes by the page size, round the quotient to the nearest integer, and multiply the rounded result by the page size to obtain an integer multiple of the alignment length. Calculate the absolute value of the difference between the expected length of consecutive prefixes and the integer multiple of the alignment length, and determine the candidate drafting depth that minimizes the absolute value of the difference as the target drafting depth; Based on the peak page occupancy of a single request corresponding to the target drafting depth, the maximum concurrent micro-batch capacity is calculated using the total page budget. The cycle time is determined based on the service rates of the drafting module and the verification module. Based on the maximum concurrent micro-batch capacity and cycle time, requests are mapped to specific time coordinates and slots in the order of arrival, and the scheduling results containing the target drafting depth and mapping relationship are output.
2. The algorithm module scheduling method according to claim 1, characterized in that, The peak page usage per request corresponding to the target drafting depth includes: Divide the target drafting depth by the page size, and round up the resulting quotient to get the number of pages occupied by the draft. Divide the expected length of the consecutive prefixes retained by the verified module corresponding to the target drafting depth by the page size, and round up the resulting quotient to obtain the number of pages to be retained. Add the number of pages used for drafting to the number of pages used for retention to get the peak page usage per request.
3. The algorithm module scheduling method according to claim 2, characterized in that, Based on the peak page occupancy per single request corresponding to the target drafting depth, the maximum concurrent micro-batch capacity is calculated using the total page budget, including: Divide the total page budget by the peak page usage per request to obtain the quotient. Perform a floor operation on the quotient and use the result of the floor operation as the maximum concurrent micro-batch capacity.
4. The algorithm module scheduling method according to claim 3, characterized in that, The cycle time is determined based on the service rates of the drafting and verification modules, including: Divide the target drafting depth by the drafting module service rate to obtain the time required for the drafting stage; divide the expected length of the continuous prefix retained by the verification module corresponding to the target drafting depth by the verification module service rate to obtain the time required for the verification stage; select the larger value between the time required for the drafting stage and the time required for the verification stage to determine the cycle time.
5. The algorithm module scheduling method according to claim 4, characterized in that, Based on the maximum concurrent micro-batch capacity and cycle time, requests are mapped to defined time coordinates and slots in arrival order, and the scheduling results, including the target drafting depth and mapping relationships, are output, including: For each request numbered from zero in arrival order, divide the request number by the maximum concurrent micro-batch capacity to obtain the quotient and remainder; round down the quotient to determine the batch index of the request; determine the slot index of the request in the corresponding tick of the batch index; multiply the batch index by the uniform tick period to obtain the drafting start time of the request. Add the draft start time to the required drafting time to obtain the verification start time of the request; Construct and output the scheduling results, which include the target drafting depth, maximum concurrent micro-batch capacity, cycle time, and a spatiotemporal mapping relationship for any request number consisting of the drafting start time, the verification start time, and the slot index.