Service scheduling method and system based on thought chain decision enhancement

Abstract

This invention relates to the field of data processing technology and discloses a service scheduling method and system based on enhanced thinking chain decision-making. It addresses the shortcomings of traditional methods, such as the lack of dynamic detection, local rollback, and more refined correction capabilities for intermediate processes within the thinking chain. This invention aligns execution time and establishes node profiles for distributed microservice clusters, forming task semantics and resource constraints for concurrent requests. It drives a large language model to generate sub-scheduling actions segment by segment. Combined with node state detection, resource verification, snapshot rollback, correction injection, and scheduling review, it verifies and corrects the scheduling chain segment by segment, generating and distributing a complete scheduling sequence. This achieves stable, real-time, and reliable service scheduling in a high-concurrency distributed cluster environment.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and specifically to a service scheduling method and system based on enhanced decision-making of the chain of thought. Background Art

[0002] In the scenario of high-concurrency distributed clusters, service scheduling for complex business requests not only needs to complete task decomposition and execution orchestration, but also needs to be consistent with the real-time resource status of underlying microservice nodes; in the prior art, there have been technical solutions that use large models for task planning and service orchestration; for example, the published patent application CN117370638B discloses a method and device for task decomposition and scheduling of a basic model enhanced by thought graph prompts, and another example is the published patent application CN117472552B that discloses a method, device, equipment and medium for intelligent orchestration and dynamic scheduling of service scenarios; the above technical solutions can use large models to generate task steps or orchestration plans, and to a certain extent improve the automation degree of service scheduling; however, such solutions mainly focus on task generation, plan orchestration and execution result feedback, and lack consideration of the coordination relationship between the reasoning process and the underlying resource status, making it difficult to achieve dynamic scheduling control for real-time resource status changes; However, in a real production environment, long chain of thought reasoning usually has the characteristics of a long generation cycle, many link levels and continuous changes in intermediate states, while the computing power, memory, queue and link status of underlying microservice nodes often change dynamically at the millisecond level; due to the temporal deviation between the reasoning process and the resource status, existing solutions are prone to allocate resources based on lagged status information, resulting in inaccurate resource constraint judgment, and further causing scheduling instructions to exceed the actual carrying capacity of physical nodes; at the same time, when a local step execution fails or the operating environment changes, existing solutions usually lack the ability to dynamically detect the intermediate process of the chain of thought, perform local rollback and more refined correction, and often can only trigger global re-pushing, further causing key-value cache idling, scheduling delay accumulation and concurrent request cascading timeouts, and in severe cases, it may even cause system-level deadlocks; therefore, how to achieve the linkage verification between the intermediate reasoning result and the underlying real-time state detection result during the chain of thought scheduling process, and perform local reconstruction and rollback correction when a deviation is found, so as to reduce the computing power loss caused by ineffective re-pushing, and suppress resource constraint distortion and cascading blockages, has become a technical problem to be solved in this field. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention provides a service scheduling method and system based on enhanced decision-making of the chain of thought, which solves the problem of the lack of the ability to dynamically detect the intermediate process of the chain of thought, perform local rollback and more refined correction in traditional methods.

[0004] To achieve the above object, the present invention provides the following technical solutions: Service scheduling methods based on thought chain-based decision enhancement include: S1. Perform full network time alignment on the distributed microservice cluster and register the identification information and mapping relationship of each node; S2. Receive and parse concurrent service requests to determine the tasks to be executed and the corresponding resource constraints; S3. Drive the large language model to generate sub-scheduling actions step by step. After generating each sub-scheduling action, pause the current inference process and save the corresponding context snapshot. S4. Determine the target node based on the sub-scheduling action, perform state detection on the target node, and verify the detected node state against resource constraints; S5. Control the subsequent inference process based on the verification result. If the verification passes, continue to generate subsequent sub-scheduling actions. If the verification fails, roll back to the corresponding context snapshot and inject correction information until a complete scheduling sequence is generated and issued for execution.

[0005] Furthermore, S1 includes: After detecting a cold start or node change event, identify the active node and perform time synchronization correction on the active node; Perform consistency checks on node attribute information and generate node index codes; Establish a node association mapping table and node status identifier based on the node index code; Collect resource status according to the keep-alive cycle and form a node status baseline; Nodes with abnormal attributes, time synchronization, or keep-alive status will be subject to pending review or isolation.

[0006] Furthermore, S2 includes: Perform window trigger identification, independence determination, header splitting, integrity verification, and anomaly correction on requests; By combining request semantics, service level information, business category and historical operation profile, determine latency sensitivity, task level, resource demand tendency, resource boundary constraints, candidate business images and constraint version numbers; The relevant information is registered in the task management table, and the buffer occupancy status is monitored and eviction is controlled.

[0007] Furthermore, S3 includes: For tasks entering the inference queue, the inference input is organized according to the business category, task semantics, resource constraints, node state baseline summary, and verified action summary; Perform segment-by-segment decoding and determination based on predefined boundary markers and action reference distribution matching; During the segment-by-segment decoding process, the generation state is frozen and a snapshot index is registered according to the action sequence number. Abnormal generation is subject to quantity constraints, duplicate identification, deduplication constraint injection, and timeout diversion processing.

[0008] Furthermore, during the segment-by-segment decoding process, the generation state is frozen and a snapshot index is registered according to the action sequence number. Abnormal generation is handled by performing quantity constraints, duplicate identification, deduplication constraint injection, and timeout routing, including: Record the current sub-scheduling action text, decoding position, hidden state, key-value cache, context offset position, and task identifier, and build a snapshot index according to the action sequence number; When the number of generated contents exceeds the limit, the process is transferred to a review process. When generated content is repeatedly repeated across action categories and target nodes, deduplication constraints are written. If no valid action is performed within the predetermined time limit, the preset traffic splitting process will be invoked.

[0009] Furthermore, S4 includes: Extract entities from sub-scheduling actions, and filter and sort candidate nodes based on logical hostnames, business instance relationships, or resource allocation intentions; Out-of-band probing is performed on candidate nodes, and the nodes are then smoothed and normalized to form node state vectors. Verification is performed based on resource boundary conditions and overall matching degree; When an anomaly is detected, supplementary detection or status marking is performed; when data is missing, supplementary data collection is performed; and when verification fails, resource diagnostic information is generated.

[0010] Furthermore, out-of-band probing is performed on candidate nodes, and the nodes are smoothed and normalized to form node state vectors, including: Bypassing the conventional container control interface, construct a status probe message containing task identifier, action sequence number, probe type, sending timestamp and verification information, and send it to the candidate node via User Datagram Protocol; The kernel-mode probes in the candidate nodes read the CPU utilization, task queue length, number of consecutive available memory page frames, network queuing latency, or video memory usage status. The results are then combined with multiple sampling results and the node state baseline for smoothing and normalization mapping to form a node state vector.

[0011] Furthermore, S5 includes: Update the verified action chain based on the verification flag, and update the temporary resource status copy according to resource usage and release status; Perform a partial rollback based on the previous security snapshot or the initial snapshot, write the resource over-limit dimension, the current available upper limit and the alternative node constraint, apply candidate suppression, and continue processing according to the partial retry rules or the basic traffic splitting rules; After the scheduling chain is closed, the execution sequence conflict review, resource duplication review, and region jump review are performed, and then the confirmation and submission process is followed.

[0012] Furthermore, the verified action chain is updated based on the verification flag, and the temporary resource state copy is updated according to resource usage and release status, including: After verification, the pause state of the current context snapshot is released, the action summary of the corresponding sub-scheduling action is appended to the verified action chain, and the resource allocation record of the action is determined without rewriting the node state baseline. Perform deduction or reclamation registrations on the processor available share, contiguous available memory, network budget, or video memory available share in the temporary resource status copy.

[0013] On the other hand, the present invention provides a service scheduling system based on thought chain decision enhancement, comprising: Time benchmark and node documentation module: used to perform network-wide time alignment on the distributed microservice cluster, register the identification information and mapping relationship of each node, and establish node association relationship and node status baseline; Request parsing module: Used to receive and parse concurrent service requests to determine the tasks to be executed; Resource constraint generation module: used to determine the resource constraints corresponding to the task to be executed based on the parsed request content, service level information and business type; Sub-scheduling generation and snapshot management module: This module is used to drive the large language model to generate sub-scheduling actions step by step based on the tasks to be executed and resource constraints, and to pause the current inference process after generating each sub-scheduling action and save the corresponding context snapshot. Node detection and verification module: used to determine the target node based on the sub-scheduling action, perform status detection on the target node, and verify the detected node status against resource constraints; Rollback Correction and Execution Control Module: Used to control the subsequent inference process based on the verification results. When the verification passes, it continues to generate subsequent sub-scheduling actions. When the verification fails, it rolls back to the corresponding context snapshot and injects correction information. After forming a complete scheduling sequence, it performs review and distribution.

[0014] Compared with existing technologies, this invention provides a service scheduling method and system based on enhanced decision-making using thought chains, which has the following beneficial effects: 1. This invention establishes a unified time base and node association relationship for a distributed microservice cluster, and forms a joint representation of task semantics and resource constraints for concurrent requests. During the generation of sub-scheduling actions segment by segment in the large language model, it introduces action-level state detection, resource constraint linkage verification, context snapshot freezing, and local rollback and correction information injection mechanisms. This ensures that each sub-scheduling action corresponds to the real-time computing power, memory, queue, and link status of the underlying nodes, reducing resource constraint judgment errors caused by resource allocation based on lagging states and situations where scheduling instructions exceed physical carrying capacity. Furthermore, when a local step fails or the operating environment changes, it performs fixed-point correction and continues reasoning along the previous safe position, reducing key-value cache idle time, accumulated scheduling latency, cascading timeouts of concurrent requests, and system blocking caused by regenerating the entire thought chain. This improves the stability, real-time performance, and reliability of service scheduling in a high-concurrency distributed cluster environment.

[0015] 2. This invention, by performing window trigger identification, independence judgment, task level classification, resource demand tendency extraction, and resource boundary constraint generation on concurrent service requests, and by executing sequence conflict verification, resource duplication verification, regional jump verification, and confirmation submission process after the scheduling chain is closed, can distinguish the urgency and resource demand differences of different business requests during the scheduling generation stage, and verify the sequential relationship, resource occupation relationship, and cross-regional routing relationship between actions during the scheduling issuance stage. This reduces scheduling conflicts, resource duplication, and cross-regional switching caused by mixed requests, inconsistent constraint standards, or mismatches in the action chain, thereby improving the consistency, controllability, and execution accuracy of service scheduling in complex business scenarios. Attached Figure Description

[0016] Figure 1 A schematic diagram of the service scheduling method based on thought chain decision enhancement of the present invention is provided; Figure 2 A schematic diagram of time alignment and node archiving in the distributed microservice cluster of this invention is provided; Figure 3 A schematic diagram illustrating the concurrent service request parsing and resource constraint generation of this invention is provided; Figure 4 A schematic diagram of the sub-scheduling action segmentation generation and context snapshot management of the present invention is provided; Figure 5 A schematic diagram of out-of-band detection of target nodes and resource constraint verification in this invention is provided; Figure 6 A schematic diagram of the local rollback correction and scheduling chain closure verification of the present invention is provided; Figure 7 A schematic diagram of the service scheduling system structure based on the enhanced decision-making of the thinking chain according to the present invention is given; Figure 8A schematic diagram illustrating an example of cross-availability zone hybrid service scheduling according to the present invention is provided. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: Figure 1 - Figure 6 A service scheduling method based on thought chain decision enhancement is presented, including: S1. Perform full network time alignment on the distributed microservice cluster and register the identification information and mapping relationship of each node; S2. Receive and parse concurrent service requests to determine the tasks to be executed and the corresponding resource constraints; S3. Drive the large language model to generate sub-scheduling actions step by step. After generating each sub-scheduling action, pause the current inference process and save the corresponding context snapshot. S4. Determine the target node based on the sub-scheduling action, perform state detection on the target node, and verify the detected node state against resource constraints; S5. Control the subsequent reasoning process based on the verification result. If the verification passes, continue to generate subsequent sub-scheduling actions. If the verification fails, roll back to the corresponding context snapshot and inject correction information until a complete scheduling sequence is generated and issued for execution. This invention addresses the high-frequency, dynamic, distributed microservice cluster operation scenario. The main components involved in the processing include a unified application gateway, a scheduling control plane, a large language model inference engine, multiple underlying microservice computing nodes, and status probes deployed in the kernel state of each computing node. Input information includes at least two types: one is high-concurrency service requests continuously initiated by external terminals, and the other is resource status information periodically fed back by each computing node. This resource status information includes at least CPU utilization, the number of available memory page frames, task queue length, and network transmission / reception queuing latency. During processing, the large language model progressively generates scheduling decisions, and the scheduling control plane integrates intermediate inference... The content is linked to the real-time resource status of the underlying nodes for verification. When they match, subsequent scheduling continues. When they do not match, only the current erroneous branch is rolled back and corrected, without triggering a full re-push of the entire scheduling chain. After the linkage verification and local correction are completed, an executable scheduling sequence is formed and sent to the container orchestration interface or task execution interface to complete resource allocation and service routing control. The entire process revolves around network-wide time alignment, request parsing, segmented reasoning, state detection, constraint verification, local rollback, and scheduling deployment. The intermediate results of the previous stage serve as the input for the next stage, thus forming a closed-loop scheduling process. Specifically, such as Figure 2 As shown: After the scheduling control plane completes a cold start, or when the number of newly accessed nodes reaches 5% of the total number of currently active nodes, or when it reaches 10 nodes, the baseline reconstruction process is initiated. If events such as motherboard power failure, changes in switching equipment connection relationships, container migration, or regional network switching occur simultaneously, the baseline reconstruction process is also initiated. The reason for setting the number of newly accessed nodes that trigger reconstruction to the above range is that when the scale of node changes is less than this range, the existing node registration results and node associations can usually still support the current scheduling. When the scale of node changes reaches this range, the node distribution, availability zone load, and service carrying location may change significantly. Continuing to use the original registration results may cause the subsequent candidate node selection to deviate from the actual operating environment. The scheduling control plane first determines the range of active nodes. Active nodes can be selected as nodes that have returned acknowledgment information for the most recent three consecutive keep-alive signal cycles and are not currently marked as isolated or disabled. Three consecutive cycles are used as the active determination criterion because a single keep-alive signal anomaly may be caused by instantaneous network jitter, busy node scheduling, or short-term drive blockage, while three consecutive cycles can more stably distinguish between normal fluctuations and actual disconnection. Subsequently, the scheduling control plane broadcasts a precise time protocol synchronization message to each active node and, in conjunction with the network card hardware timestamp, round-trip delay measurement results, symmetric link delay compensation results, and round-by-round correction results, uniformly corrects the local clock of each node. The precise time protocol is used instead of the ordinary network time synchronization method because there is a microsecond-level timing correlation between subsequent state detection, context snapshot freezing, and resource verification. If the time base deviation is too large, it will be difficult to establish an accurate correspondence between the model context corresponding to the same scheduling action and the node state sampling results. Within the same time domain supporting hardware timestamps, the time error between the scheduling control plane and each active node can be controlled within 2 microseconds. Setting 2 microseconds as the upper limit for time error is because microservice container creation and destruction, request state transitions, and kernel probe responses can all occur within the microsecond range. If the time error between nodes exceeds 2 microseconds, the state sampling results corresponding to the same scheduling action may span different scheduling segments, increasing resource judgment bias. To ensure the time synchronization process has a clear stopping condition, the scheduling control plane performs a maximum of 5 rounds of correction for each node. When the time deviation between two adjacent correction rounds is less than 200 nanoseconds, the node is considered... The clock enters a stable state and ends the calibration; 5 rounds are set as the upper limit of calibration because the improvement in deviation usually decreases significantly after 5 rounds, and continuing calibration will increase the synchronization overhead of the control plane; 200 nanoseconds are set as the convergence threshold because this value is significantly lower than the upper limit of 2 microsecond time error, indicating that the node clock has entered the stable correction range; if a node still cannot meet the 2 microsecond time error requirement after completing 3 resynchronizations, the node is marked as isolated and removed from the current scheduling candidate range; for cross-campus nodes that are not in the same time domain, they can first complete regional time alignment within their respective regions, and then participate in subsequent scheduling based on the regional time base; After time alignment is completed, the scheduling control plane reads node attribute information from the infrastructure asset repository and container registry, respectively, and performs consistency checks on fields such as Internet Protocol address, media access control address, high availability zone identifier, rack location, service image identifier, and node role. When there are field conflicts between multiple data sources, the record with the newest update time is used, and the conflict record is written to the review queue. After the attribute checks pass, the node attributes are concatenated according to a fixed field order to form a node attribute string, and a digest encoding operation is performed on the node attribute string to generate a 256-bit node index code for the corresponding node. The node index code length is set to 256 bits because this length can reduce the probability of index collisions in scenarios with a large number of nodes and many attribute fields, while not significantly increasing the index storage and retrieval overhead of the control plane, which is in line with the engineering usage habits of existing cluster management systems for fixed-length node identifiers. The scheduling control plane establishes a node association mapping table based on node index codes. The node association mapping table includes at least the correspondence between logical hostnames and real routing addresses, the correspondence between service instances and physical nodes, node proximity relationships, and node status identifiers. Node proximity relationships can be determined according to at least one of the following conditions: nodes are located in the same rack, nodes are connected to the same switching device, nodes are located in the same high availability zone, or the network round-trip latency between nodes is no higher than 1 millisecond. 1 millisecond is used as the latency judgment value for proximity relationships because this value can cover most low-latency interconnected nodes within the same data center, and can exclude nodes across regions or long links from the nearest neighbor range, making it easier to perform local refreshes only on affected nodes and their nearest neighbors. The node status identifier can be selected from five values: normal, pending review, disconnected, isolated, and disabled. Among them, normal indicates that it can participate in scheduling, pending review indicates that it will not participate in the first round of scheduling but retains registration information, disconnected indicates that the keep-alive signal has timed out, isolated indicates that scheduling access is prohibited, and disabled indicates that the node has left the current cluster service range. After initial node registration, the scheduling control plane continuously collects CPU utilization, available memory, task queue length, and network queuing latency over five keep-alive signal cycles. A node status baseline is generated using a moving average, serving as reference data for subsequent request parsing and target node verification. Five sampling cycles are chosen because too few cycles are susceptible to transient jitter, while too many cycles reduce baseline update speed. With a single keep-alive signal cycle of 200 milliseconds, five consecutive cycles correspond to approximately a one-second observation window, balancing stability and timeliness. A keep-alive signal cycle of 200 milliseconds is optional; if no data is received within three consecutive keep-alive signal cycles... Upon node confirmation, the node is determined to be in a disconnected state and a partial update process is triggered, refreshing only the data of that node and its nearest neighbors. The keep-alive signal period is set to 200 milliseconds because, at a node scale of hundreds to thousands, a period shorter than 100 milliseconds would significantly increase the keep-alive processing pressure on the control plane; if the period is longer than 500 milliseconds, the node's online state reversal will be significantly delayed, which is not conducive to timely updating of node relationships. Three consecutive periods of no confirmation correspond to approximately 600 milliseconds, which can both filter out single short-term anomalies and is suitable for background node online state maintenance. This disconnection determination is only used for node online state maintenance and is not used as the real-time resource verification time limit for subsequent individual scheduling actions. If a node has missing attributes, unresolved data source conflicts, failed time synchronization, or unstable keep-alive signal status during the initial registration phase, it will be marked as pending review and excluded from the first round of scheduling candidates. If a node still fails to meet the time error requirements after repeated synchronization, it will be marked as isolated. Isolated nodes can re-enter the node registration process after manual review or normal time synchronization. After the above processing, a unified time base, node registration results, node association mapping table, and node status baseline are generated and used as the basic input for subsequent request parsing, candidate node screening, and target node determination.

[0019] Specifically, such as Figure 3As shown: After completing network-wide time alignment, node registration, and node status baseline establishment, the unified application gateway continuously receives external service requests and writes them into a circular buffer according to their arrival time. When the unified application gateway receives 300 independent requests within a continuous 50-millisecond sliding time window, it initiates a centralized resolution process. It also initiates a centralized resolution process when it receives a single service level agreement (SLA) request with a time limit not exceeding 50 milliseconds. The sliding time window is set at 50 milliseconds, taking into account that the one-way fiber optic transmission latency between active-active data centers in the same city is typically about 2 milliseconds. 50 milliseconds can cover the aggregation process of burst traffic within a short time window without causing requests to wait too long before entering the scheduling chain. The request quantity threshold is set at 300, considering that this number corresponds to a high inbound concurrency density within the 50-millisecond window. If the process continues to be handled one by one, it is easy to increase queuing and analysis lag; therefore, centralized resolution is more suitable for high-concurrency scenarios. Independent requests are preferentially identified according to their request identifiers. When a request identifier is missing, it is distinguished according to a combination of source address, service category, and millisecond-level arrival timestamp to avoid the same request being counted repeatedly. After entering the centralized parsing process, the unified application gateway splits the request into request header information and business payload. The request header information includes at least the fields of request identifier, business category, user domain, priority flag, service level agreement tag, expected response time limit, source address, and retry flag. The business payload includes at least the fields of task semantic description, call context, attached parameters, and data object identifier. The scheduling control plane first performs integrity and format verification on the request header information, and then performs structure parsing and content extraction on the business payload. If there are missing fields but the core business semantics can still be identified, the missing fields are filled in according to the pre-configured default rules, and the initial priority of the request is reduced. If there are tag conflicts, the one with stricter constraints is used as the effective tag for the current request. If there are any cases of inconsistent checksums, failure to parse key fields, or business content format not conforming to any of the predefined protocol formats, the business payload is determined to be damaged and unrecoverable, and a retransmission instruction is returned to the unified application gateway. After request splitting, the scheduling control plane invokes the request encoding model to extract features from the business payload. The input of the request encoding model includes at least the request text content, structured tags, and numerical parameters, and the output includes at least the task semantic representation result and the resource demand tendency vector. During processing, the request text is first mapped to high-dimensional word vectors, and then jointly encoded with the business category, priority marker, and service level protocol tag in the request header to obtain a unified request representation result. Then, a resource demand tendency vector is generated based on the unified request representation result. The resource demand tendency vector can be selected with four dimensions, corresponding to computing resource tendency, memory resource tendency, network input / output resource tendency, and storage access resource tendency, respectively. The number of dimensions is set to 4, which comprehensively considers that subsequent scheduling generation, node selection, and resource verification mainly revolve around the four basic resources of computing, memory, network, and storage. Using this dimension setting can cover the main scheduling objects, while avoiding setting too many dimensions and increasing the parsing complexity. The values ​​of each dimension are normalized and take values ​​from 0 to 1, with larger values ​​indicating a stronger demand tendency for resources. When setting resource constraints, the scheduling control plane generates a latency sensitivity parameter based on the Service Level Agreement (SLA) tag and expected response time limit in the request header. The latency sensitivity parameter takes a continuous value between 0 and 1; a higher value indicates a stricter requirement for the response time limit. This parameter is generated using a piecewise linear mapping: when a request requires a response within 50 milliseconds, the latency sensitivity value is 0.9 to 1.0; when a request requires a response between 50 and 100 milliseconds, the latency sensitivity value is 0.8 to 0.9; when a request requires a response between 100 and 200 milliseconds, the latency sensitivity value is... The sensitivity value ranges from 0.6 to 0.8; when a request requires a response time exceeding 200 milliseconds, or when there is no strict time constraint, the latency sensitivity value ranges from 0 to 0.5. The above ranges are set in this way mainly because: requests within 50 milliseconds are usually time-sensitive tasks, requests within 100 milliseconds are usually real-time interactive tasks, requests between 100 and 200 milliseconds still need to be processed relatively quickly, and requests exceeding 200 milliseconds can usually be treated as latency-tolerant tasks. The specific values ​​within the ranges are linearly mapped to the corresponding ranges according to the actual time limit to reflect the differences in urgency between different requests. After obtaining the latency sensitivity parameter, the scheduling control plane determines the task level based on this parameter. When the latency sensitivity is not lower than 0.8, the request is marked as a high-priority task; when the latency sensitivity is higher than 0.5 but lower than 0.8, the request is marked as a normal real-time task; when the latency sensitivity is not higher than 0.5, the request is marked as a task that can be delayed. Setting 0.8 and 0.5 as classification thresholds is mainly considered because 0.8 corresponds to the boundary between time-sensitive services and general real-time services, and 0.5 corresponds to the boundary between general real-time services and delayed services, and is consistent with the aforementioned response time limit range. Through the above processing, the time limit information in the request header can be converted into quantifiable task priority judgment parameters, which facilitates the subsequent scheduling control plane to perform unified scheduling judgments based on resource demand trends and node status information. In addition to latency constraints, the scheduling control plane also generates resource boundary constraints based on the service category. These resource boundary constraints are formed in the following order: first, the corresponding resource template is found based on the service category; then, the template parameters are modified based on historical operational profiles; finally, the resource boundary constraints for the current request are formed according to pre-configured scheduling rules. For database write tasks, constraints on contiguous memory and storage bandwidth are added; for streaming media forwarding tasks, constraints on network queue latency and network throughput are added; for batch computing tasks, constraints on CPU share or GPU share are added; and for model inference tasks, constraints on GPU memory usage and inference concurrency are added. After the above processing, the business semantics are further converted into verifiable resource conditions, facilitating direct comparison with subsequent node status detection results. To facilitate direct invocation during subsequent reasoning and verification, the scheduling control plane writes the parsing results into the task management table. The task management table includes at least the following fields: task identifier, arrival time, task semantic representation result, resource demand tendency vector, latency sensitivity, business priority, candidate business image, call context summary, and constraint version number. Among them, the candidate business image is generated according to the business category and image mapping rules. The constraint version number is used to correspond to the node status baseline established above, so as to ensure that the same batch of tasks refer to the same resource benchmark at the same time during subsequent reasoning, detection, and verification, and avoid inconsistencies in judgment due to updates to the node status baseline. During the continuous writing of tasks, the unified application gateway synchronously monitors the usage rate of the ring buffer. When the ring buffer usage rate reaches 80%, the least recently used eviction policy is activated, prioritizing the cleanup of request records that have been parsed and finished processing. However, high-priority tasks and tasks in the inference process are not evicted. The usage rate threshold is set at 80%. Considering all factors, 20% of the remaining space is reserved to receive newly arriving burst requests and requests that are being parsed but have not yet completed status marking. This avoids the ring buffer being instantly filled during high-concurrency phases, which would affect the access of subsequent requests. After the above processing, a set of tasks to be executed and the resource constraint information corresponding to each task are formed, which are used as input for subsequent reasoning to generate sub-scheduling actions, node selection and status verification. Through this process, the original concurrent request traffic is converted into a unified task representation, and the abstract business semantics are further converted into verifiable resource constraint information, thereby providing direct input for the subsequent generation of scheduling chains.

[0020] Specifically, such as Figure 4 As shown: After tasks to be scheduled exist in the task management table, and the corresponding tasks are prioritized and enter the inference queue, the scheduling control plane extracts target tasks from the inference queue in sequence and organizes the input content in a fixed order. The input content includes at least the task identifier, business category, task semantic representation result, resource constraint information, node status baseline summary, and currently validated action summary. The main considerations for organizing the input content in the above order are: first clarify the scheduling object and business semantics, then introduce resource constraints and node status boundaries, and then supplement the generated action context to maintain the context continuity and constraint consistency in the long chain inference process. The scheduling control plane sends the organized input content to the large language model inference engine, which generates the scheduling text segment by segment using an autoregressive method, instead of outputting the complete scheduling scheme at once. To break down long-chain inference results into multiple independently verifiable scheduling units, the scheduling control plane sets up action boundary monitoring logic at the model decoder output. This logic simultaneously determines boundaries based on predefined boundary markers and action semantic matching results. Predefined boundary markers include at least newline characters, predefined action end markers, and field end markers, with these markers pre-determined by the scheduling output template and action syntax rules. Action semantic matching results are obtained through an action reference distribution. This distribution is constructed by first collecting pre-configured standard action template text and historically executed and verified scheduling action texts, then using data consistent with the current inference process. The encoder encodes the action text and then groups it according to predefined action categories. The action categories include at least allocation of computing resources, allocation of storage resources, switching service routes, mounting of business images, binding of network addresses, migration of task replicas, delayed execution, and transfer to disaster recovery nodes. After calculating the center value of the encoding results within the same category and performing normalization, the action reference distribution for the corresponding category is obtained. The action reference distribution is jointly constructed using standard action template text and historical successful action text. The main considerations are that the former can provide a stable action structure, and the latter can reflect the expression differences in real-world operating scenarios. Combining the two helps to reduce the bias caused by a single text sample. During the action boundary determination process, the scheduling control plane continuously reads the decoding layer output at the current moment and normalizes the decoding layer output to the current output distribution. Then, it calculates the distribution difference between the current output distribution and the action reference distribution corresponding to each action category. After obtaining the distribution difference between the current output distribution and each action reference distribution, the scheduling control plane takes the minimum value as the current action boundary determination value. When the minimum distribution difference is less than 0.15 and the current output position falls at the predefined boundary marker, it is determined that the current text has formed an independently executable sub-scheduling action. The distribution difference threshold is set to 0.15, mainly because: this value comes from historical scheduling log playback and boundary recognition training results. Under this value condition, it can identify independent action boundaries with a high probability and is not easy to mistakenly segment continuous action text into invalid segments. If the minimum distribution difference is not less than 0.15, or if the minimum distribution difference is less than 0.15 but the current output is still located in the internal field of the action, then the autoregressive generation continues until the action boundary determination condition is met. After determining that a sub-scheduled action has been formed, the scheduling control plane immediately sends a pause command to the large language model inference engine and freezes the current generation state. The frozen content includes at least the current sub-scheduled action text, decoding position, hidden states of each layer, multi-head attention key-value cache, current input context offset position, and corresponding task identifier. The scheduling control plane writes the frozen content into a context snapshot and manages it through a snapshot index table. The snapshot index table includes at least the snapshot number, corresponding task identifier, action sequence number, generation timestamp, GPU memory address offset, action text summary, and snapshot validity status. To balance local rollback speed and storage overhead, snapshots are saved in order of action sequence number. The snapshots corresponding to the three most recent sub-scheduled actions are kept in high-speed GPU memory, while the remaining earlier snapshots are asynchronously transferred to main memory. The GPU memory retention value is set to 3, mainly because: according to the historical rollback position statistics, most local corrections occur within the range of the most recent sub-scheduled actions, and retaining the three most recent snapshots can cover most local rollback requirements without significantly increasing GPU memory usage; after the earlier snapshots are transferred to main memory, they can still be reloaded when needed to avoid losing the complete context. To prevent large language models from continuously expanding invalid branches during generation, the scheduling control plane sets action quantity constraints and a duplicate generation identification mechanism. The maximum number of sub-scheduling actions for a single task is 32. When the number of sub-scheduling actions generated for the same task exceeds 32, the scheduling control plane transfers the current task to the review queue. The maximum number of actions is set to 32, mainly because, according to the statistical results of historical task chain lengths, most scheduling tasks can be completed within 24 actions. Setting the maximum to 32 allows for expansion margins based on the completion length of most tasks, which can cover the scheduling needs of complex tasks and avoid the model continuously expanding invalid branches during long chain generation. If two consecutive sub-scheduling actions are the same in both action category and target node, the current generation process is determined to have entered a duplicate generation state. The number of consecutive determinations is set to 2, mainly because a single repetition may come from normal retries, while two consecutive repetitions better reflect that the model has entered a local loop state. After detecting a duplicate generation state, the scheduling control plane adds deduplication constraint information to the input content of the next round to prompt the model to avoid outputting the same action category and the same target node again. If the large language model inference engine fails to generate any sub-scheduled actions that meet the boundary judgment conditions within the predetermined time limit, the scheduling control plane will transfer the current task to the timeout queue and call the lightweight heuristic scheduling logic to generate a preset splitting action first. The predetermined time limit is set to 200 milliseconds. The main consideration for setting the time limit to 200 milliseconds is that, according to the historical inference time consumption statistics, the normal generation time of a single sub-scheduled action is usually less than 80 milliseconds. 200 milliseconds can provide sufficient time for the first action generation of complex tasks, while avoiding a single task occupying inference resources for a long time and causing subsequent tasks to queue up and amplify. After obtaining the preset splitting action, the task transferred to the timeout queue can continue to enter the offline analysis queue for subsequent parameter correction and strategy optimization. After the above processing, sub-scheduling action text and corresponding context snapshot are formed. The sub-scheduling action text serves as the input basis for subsequent target node identification and state verification, while the context snapshot serves as the basis for local rollback when verification failure occurs. Through the above processing, the continuous long-chain reasoning process can be converted into multiple independently verifiable, locally rollbackable, and reusable scheduling units, thereby providing support for subsequent state detection, constraint verification, and local correction.

[0021] Specifically, such as Figure 5As shown: After generating a sub-scheduling action and saving the corresponding context snapshot, the scheduling control plane determines the target node based on the current sub-scheduling action, performs state probing on the target node, and verifies the detected node state with resource constraint information; the current sub-scheduling action includes at least one of the following: logical hostname, service instance identifier, resource allocation intent; the scheduling control plane first performs entity extraction on the current sub-scheduling action text, and the extraction result includes at least the logical hostname, service instance name, action category, resource object, estimated resource requirement, and region identifier; entity extraction can be a combination of rule matching and dependency analysis, where rule matching is used to extract host identifier, container identifier, region identifier, and resource fields, and dependency analysis is used to identify the correspondence between action object and resource quantity, and between target region and service type; When a logical hostname is given directly in a sub-scheduling action, the scheduling control plane calls the previously established node association mapping table to map the logical hostname to the real physical routing address and identifies the corresponding node as the probe target node. When a service instance name is given but not a logical hostname in a sub-scheduling action, the scheduling control plane forms a candidate node set based on the mapping relationship between the service instance and the physical node. Then, it filters nodes that are in normal status and located in the target availability zone from the candidate node set, and sorts them in order of remaining contiguous available memory from high to low, current CPU utilization from low to high, and network round-trip latency from low to high. The nodes with higher rankings are identified as probe target nodes. The above sorting method is mainly based on the fact that service carrying capacity is first affected by contiguous available memory, then by the current computing load, and finally by network transmission latency. Therefore, prioritizing nodes with larger memory reserves, lower computing load, and lower network latency is more conducive to ensuring the executability and stability of subsequent scheduling actions. When a sub-scheduling action only provides the resource allocation intention without specifying the service instance name, the scheduling control plane forms a candidate node set based on the action category, service type, and candidate service images. It then filters out nodes that meet the deployment scope by combining the node status baseline and node relationships. Finally, it comprehensively sorts the remaining continuous available memory, current CPU occupancy, and network round-trip latency to determine the target node to be probed. The comprehensive sorting can be determined based on the weighted results of the above indicators, where the weight corresponding to the remaining continuous available memory is higher than the weight corresponding to the current CPU occupancy, and the weight corresponding to the current CPU occupancy is higher than the weight corresponding to the network round-trip latency. In one optional case, the above weight values ​​can be 0.45, 0.35, and 0.20. This setting is primarily based on the consideration that in most microservice scheduling scenarios, memory availability has the greatest impact on instance placement, followed by computational load, and then network latency. If multiple candidate nodes are formed simultaneously, they are probed sequentially according to the sorting results. If no identifiable target node is formed, the current sub-scheduling action is marked as an action to be corrected and proceeds to the subsequent correction process. After identifying the target node, the scheduling control plane sends a status probe request to the target node using out-of-band probing, instead of through the conventional container control interface, to reduce the impact of interface queuing on the probe results. The scheduling control plane constructs a status probe data packet, with the packet header including at least the task identifier, action sequence number, probe type, sending timestamp, and verification information, and sends it to the target node via the User Datagram Protocol (UDP). The length of the status probe data packet is no more than 64 bytes. The data packet length is controlled within this range mainly because this length can accommodate the necessary information required for task identification, action identification, and verification control, while not significantly increasing the network transmission burden in high-frequency probing scenarios. Upon receiving a status probe data packet, the kernel-mode probe in the target node directly reads the node's resource status without interrupting user-mode services. The read data includes at least one or more of the following metrics: current CPU utilization, task scheduling queue length, number of consecutive available physical memory pages, network transmit / receive queuing latency, and graphics processor memory utilization. After reading, smoothing and normalization are performed on each data point to form a node state vector. Specifically, the current CPU utilization, task scheduling queue length, and network transmit / receive queuing latency are taken as the moving average of the last three samples; the number of consecutive available physical memory pages is taken as the minimum of the last three samples; and the graphics processor memory utilization is taken as the maximum of the last three samples. The sampling count is set to 3, primarily because single sampling is susceptible to transient jitter, while three samples smooth out short-term fluctuations without significantly increasing action-level probe latency. Normalization can be achieved by proportionally mapping the corresponding field's upper or lower limit or baseline value in the node state baseline, ensuring that state values ​​of different dimensions participate in subsequent verification on a unified scale. To ensure timeliness, the scheduling control plane sets a timeout window for each state probe; the timeout window is set to 5 milliseconds. This 5-millisecond timeout is chosen because, in a dual-active, low-latency cluster environment within the same city, the sum of fiber optic round-trip transmission latency and kernel-mode read-only traversal time can usually be controlled within this range. If it exceeds this range, it usually indicates severe congestion, driver anomalies, or link isolation at the target node. If no state probe result is received within 5 milliseconds, a supplementary probe is performed. The number of supplementary probes is set to 1, mainly because a second probe can filter out instantaneous packet loss and short-term jitter, without significantly amplifying the action and verification latency as with multiple repeated probes. If no result is returned after the supplementary probe, the target node is marked as disconnected, and the current sub-scheduling action stops referencing that node. If two consecutive probes time out within the same task verification cycle, or if a probe timeout is accompanied by a link anomaly record, the target node is marked as isolated and removed from the current candidate node set. After the detection results are returned normally, the scheduling control plane combines the node state vector with the resource constraint information formed above as the feasibility judgment input. The resource constraint verification includes at least three physical boundary conditions: the first is the CPU utilization constraint, that is, the current CPU utilization plus the estimated load value must not exceed 80.5%; the second is the continuous available memory constraint, that is, after deducting the current service usage and system reserve, the remaining continuous available memory must not be less than 256MB; the third is the network queuing latency constraint, that is, the network transmit / receive queuing latency must not exceed 2 milliseconds; the upper limit of CPU utilization is set at 80.5%, mainly considering that... In a multi-core scheduling environment, once the CPU load exceeds this value, the cost of context switching and lock contention will increase significantly, easily leading to amplified request queuing. Setting the lower limit of continuous available memory to 256MB is primarily because, for ordinary microservice instances, basic cold starts and runtime metadata loading require this level of continuous memory. Setting the upper limit of network queuing latency to 2 milliseconds is mainly because, in a dual-active, low-latency cluster environment within the same city, once network card queuing exceeds this value, high-concurrency traffic can quickly evolve into queue backlog. For GPU-related tasks, additional GPU memory usage constraints can be added; for storage-sensitive tasks, additional storage bandwidth constraints can be added. In addition to physical boundary condition verification, the scheduling control plane also calculates the overall matching degree, which measures the overall carrying capacity of the current target node for the current sub-scheduling action. The overall matching degree ranges from 0 to 1. During the calculation, the individual matching values ​​corresponding to each resource dimension are first determined, and then weighted and summed according to preset weights to obtain the overall matching degree. The individual matching value is determined based on the deviation ratio between the real-time state and the corresponding resource constraints: when the real-time state meets the resource boundary and the remaining margin is large, the individual matching value is close to 1; as the real-time state gradually approaches the resource boundary, the individual matching value decreases accordingly; when the real-time state exceeds the resource boundary, the individual matching value is 0. For ordinary microservice tasks, the resource dimensions involved in computation include at least the CPU, contiguous available memory, and network queuing latency, with weights of 0.40, 0.35, and 0.25, respectively. For tasks involving graphics processing unit (GPU) requirements, the resource dimensions involved in computation include at least the CPU, contiguous available memory, network queuing latency, and GPU memory, with weights of 0.30, 0.30, 0.20, and 0.20, respectively. The weights are set in this way primarily because ordinary microservice tasks are mainly affected by CPU, memory, and network conditions, while GPU-related tasks also need to reflect the impact of GPU memory load on processing capacity. The overall matching degree threshold is set to 0.85. This threshold is set to 0.85 primarily because: based on the statistical results of the matching degree distribution of historical successful and failed scheduling samples, when the overall matching degree is below this value, the proportion of failed samples increases significantly; when the overall matching degree is near or above 0.85, it can better distinguish between node states that are safe to execute and those with significant risks; therefore, when the overall matching degree is below 0.85, the current sub-scheduling action is determined to be unsafe to execute; when the overall matching degree is not lower than 0.85, the current target node is determined to meet the carrying requirements of the current sub-scheduling action. After verification, the scheduling control plane outputs the verification result and resource diagnostic information. If the verification passes, a flag indicating that inference can continue is generated. If the verification fails, a resource diagnostic vector is generated. The resource diagnostic vector includes at least the out-of-limit dimension, the actual available upper limit, the exceedance range, and the node status identifier. This resource diagnostic vector is then directly sent to the subsequent correction process as input for local rollback and correction prompts. If a single data item is missing during the verification process, the missing item is sampled once. The number of sample collections is set to 1, mainly to filter out instantaneous sampling anomalies and avoid repeated sampling that increases the action-level verification latency. If the missing data still exists after the sample collection, it is judged according to stricter conservative constraints. After the above processing, the current sub-scheduling action can correspond to the actual physical state of the target node item by item, thereby blocking scheduling paths that deviate from the actual resource state at the action level.

[0022] Specifically, such as Figure 6As shown: After receiving the verification result corresponding to the current sub-scheduling action, the scheduling control plane controls the subsequent inference process according to the verification result; when the verification result is passed, the scheduling control plane releases the pause state of the current context snapshot, marks the current sub-scheduling action as a verified action, and appends the action summary to the verified action chain; at the same time, it updates the temporary resource state copy according to the resource allocation result corresponding to the action; the temporary resource state copy includes at least one or more of the following fields: the node's remaining CPU available share, the remaining contiguous available memory, the network queuing budget, and the GPU memory available share; during the update, the resources occupied or expected to be occupied by the current action are deducted and the resources released by the current action are reclaimed, so that the subsequent generation process can continue to advance based on the resource state after the formation of the verified action in the current task chain, without directly rewriting the node state baseline established above; Setting up a temporary resource state copy is mainly considered as the node state baseline reflecting the reference state before verification, and the temporary resource state copy reflecting the cumulative impact of verified actions on node resources in the current task chain, thereby maintaining the consistency of resource caliber in the continuous inference process of multiple actions; after the update is completed, the scheduling control plane resumes the generation process of the large language model inference engine, so that the large language model continues to generate subsequent sub-scheduling actions under the joint constraints of the verified action chain and the temporary resource state copy. When the verification result is unsuccessful, the scheduling control plane does not trigger a full re-push, but instead performs a partial rollback. The scheduling control plane first reads the snapshot index table to locate the previous safe snapshot corresponding to the current sub-scheduling action. The previous safe snapshot can be the context snapshot corresponding to the most recently verified sub-scheduling action, or the context snapshot corresponding to the most recently sub-scheduling action that did not trigger resource overruns or node anomalies. If there is no verified action snapshot before the current failed action, the plane rolls back to the starting snapshot corresponding to the initial input context of the current task. After locating the snapshot, the scheduling control plane restores the model context to the corresponding safe starting point based on the memory address offset and decoding position recorded in the snapshot. During the restoration process, only the hidden state and key-value cache after the safe starting point are discarded, while the context before the safe starting point remains unchanged to avoid recalculating the previous verified actions. After completing the partial rollback, the scheduling control plane converts the previously generated resource diagnostic vector into correction information and injects this correction information after the constraint segment in the model input context. The correction information can be a structured constraint fragment, templated prompt text, or a semi-structured set of correction fields. Its content includes at least the resource limit exceeding dimension, the current available upper limit, the node identifier prohibiting further selection, the allowed backup regions, and the backup node selection criteria. This injection method is primarily chosen because when the large language model regenerates the current action, it needs to know not only why the current action failed but also which nodes or regions to avoid and which alternative objects to prioritize. For example, when the resource diagnostic vector indicates that the target node's continuous available memory is lower than the current task's required continuous memory lower limit, the correction information can be written as: "The current target node has insufficient continuous available memory; it is not allowed to continue allocating the current database write task to this node. Backup nodes within the same availability zone whose continuous available memory meets the current task's lower limit requirement should be prioritized." The correction information participates preferentially in subsequent generation and interpretation within the current sub-scheduling action regeneration cycle to ensure that the correction constraints directly affect the current action's regeneration process. To reduce the probability of re-entering the same erroneous branch during regeneration, the scheduling control plane applies output suppression to the candidate items corresponding to the current failed node or failed region. Output suppression can be selected to reduce the output score of the candidate item, reduce the sampling probability of the candidate item, or directly block the candidate item within the current regeneration cycle. Output suppression only applies to the candidate items corresponding to the current failed node or failed region and does not affect the normal generation of other similar action categories. Output suppression remains effective at least until the current sub-scheduling action completes regeneration and re-enters verification. After the new sub-scheduling action passes verification, the suppression condition is removed or updated according to the current inference state. This setting is mainly considered to use correction information to clarify the constraint direction, and output suppression to reduce the probability of hitting the same failed object again. The combination of the two can improve the convergence efficiency after local rollback. To prevent the large language model from continuously entering a state of repeated trial and error during local correction, the scheduling control plane sets an upper limit on the number of local retries. The upper limit for the number of local retries is set to 3. The main consideration for setting this value to 3 is that, based on the statistical results of the distribution of successful local corrections in history, most correctable resource conflicts can be eliminated after the first 1 to 2 corrections. If a sub-scheduling action that meets the resource boundary constraints is not generated after 3 consecutive retries, the success rate of subsequent retries will decrease significantly, and the benefits of continuing local trial and error will be low. If the current sub-scheduling action still fails the verification after 3 consecutive local retries, the scheduling control plane will stop the regeneration process of the current task at that position and transfer the current task to the degraded processing path. After entering the degradation processing path, the scheduling control plane calls the pre-set scheduling logic to generate a basic traffic splitting scheme. The basic traffic splitting scheme includes at least one or more of the following methods: pushing the current task into an asynchronous buffer queue and waiting for subsequent available resources to be restored before execution; forwarding the current task to the disaster recovery area or the set of standby nodes; sending a capacity expansion request to the elastic expansion interface and re-entering the scheduling process after the expansion is completed. The selection of the basic traffic splitting scheme is executed in the following order: when there are standby nodes in the target area that meet the minimum resource threshold of the current task, the task is forwarded to the set of standby nodes first; when neither the target area nor the standby area meets the minimum resource threshold of the current task, the current task is pushed into the asynchronous buffer queue; when the length of the asynchronous buffer queue reaches the preset congestion threshold or the queue waiting time reaches the preset delay threshold, the elastic expansion interface is triggered. The above processing order is mainly considered to reduce switching costs by prioritizing the use of existing standby resources; when existing resources are insufficient, short-term buffering can avoid immediate task discarding; and triggering expansion when the buffer continues to accumulate is more conducive to ensuring the overall system stability. After multiple consecutive sub-scheduling actions pass verification, if the large language model outputs an end marker, it is determined that the scheduling chain corresponding to the current task has formed a complete scheduling sequence. Before formally issuing the data, the scheduling control plane performs a link-level review on the verified action chain. The link-level review includes at least the following: checking whether there are execution order conflicts between adjacent actions, checking whether the same resource identifier is repeatedly allocated within the same time window, and checking whether there are repeated jumps during cross-regional routing. Execution order conflicts are determined based on the mutual exclusion relationship between action types. Repeated resource allocation is determined based on the resource identifier and the preset time window. Repeated jumps during cross-regional routing are determined based on the number of region switches between adjacent actions. When the same resource identifier is repeatedly allocated within one action execution window, it is determined as resource duplication. When the number of consecutive region switches reaches 3 or more, it is determined as repeated jumps. The threshold for the number of region switches is set to 3, mainly considering that a single or two region switches may be caused by reasonable obstacle avoidance or load migration, while three or more consecutive switches usually indicate that the scheduling path has experienced significant oscillations. After the link-level review is passed, the verified action chain is encapsulated into a complete scheduling sequence. After completing the link-level review, the scheduling control plane sends the complete scheduling sequence to the task execution interface. The scheduling sequence can be distributed using a batch submission method with an acknowledgment mechanism: each execution node first returns a readiness confirmation message, and then the scheduling control plane sends the final submission command uniformly. For the batch submission process, an acknowledgment waiting window can also be set. If an execution node fails to return a readiness confirmation message within the acknowledgment waiting window, the final submission is paused, and the action corresponding to that node is marked as an action pending review. The batch submission method with an acknowledgment mechanism is mainly considered to reduce the risk of overall inconsistency caused by changes in the state of a single node during the execution phase. After the execution is completed, the scheduling control plane records the number of partial rollbacks, the number of actions that pass verification, the degradation processing flag, and the final execution status of this round of tasks for subsequent statistical analysis and parameter optimization. After the above processing, the verification result can be directly applied to the inference path; when the verification passes, the current action enters the verified action chain and participates in subsequent inference; when the verification fails, the system completes the local reconstruction through partial rollback, correction information injection, and error candidate output suppression, without having to return to the starting point to regenerate the entire scheduling chain; thus, it can control the inference overhead while ensuring the reliability of scheduling, and finally form a complete scheduling sequence that is executable, verifiable, and partially rollback-able.

[0023] Example 2: Figure 7 This invention presents a service scheduling system based on thought chain decision enhancement, comprising: Time benchmark and node documentation module: used to perform network-wide time alignment on the distributed microservice cluster, register the identification information and mapping relationship of each node, and establish node association relationship and node status baseline; Request parsing module: Used to receive and parse concurrent service requests to determine the tasks to be executed; Resource constraint generation module: used to determine the resource constraints corresponding to the task to be executed based on the parsed request content, service level information and business type; Sub-scheduling generation and snapshot management module: This module is used to drive the large language model to generate sub-scheduling actions step by step based on the tasks to be executed and resource constraints, and to pause the current inference process after generating each sub-scheduling action and save the corresponding context snapshot. Node detection and verification module: used to determine the target node based on the sub-scheduling action, perform status detection on the target node, and verify the detected node status against resource constraints; Rollback Correction and Execution Control Module: Used to control the subsequent inference process based on the verification results. When the verification passes, it continues to generate subsequent sub-scheduling actions. When the verification fails, it rolls back to the corresponding context snapshot and injects correction information. After forming a complete scheduling sequence, it performs review and distribution.

[0024] Example 3: Based on Examples 1 and 2, the specific application process of the service scheduling method and system based on thought chain decision enhancement is further explained: Specifically, such as Figure 8 As shown: Taking a mixed business scenario of database write and result query across availability zones as an example, this paper further explains the collaborative operation process of the service scheduling method and system based on the enhanced thinking chain decision. The operating environment includes a unified application gateway, a scheduling control plane, a large language model inference engine, a time base and node archiving module, a request parsing module, a resource constraint generation module, a sub-scheduling generation and snapshot management module, a node detection and verification module, and a rollback correction and execution control module. The underlying distributed microservice cluster is deployed in two availability zones. The first availability zone deploys database write instances, query instances, and log processing instances, while the second availability zone deploys database write standby instances, query standby instances, and disaster recovery routing instances. Each physical node deploys a kernel-state probe. The unified application gateway continuously receives external business requests. The scheduling control plane and the large language model inference engine work together to generate scheduling decisions. The node detection and verification module performs out-of-band state detection on candidate nodes. The rollback correction and execution control module decides whether to continue inference, perform partial rollback, or degrade traffic based on the verification results, thus forming a complete scheduling closed loop. Before starting this round of scheduling, the time reference and node documentation module first completes the cluster initialization process. This module performs full network time alignment on all active nodes in the two availability zones, and registers the identification information, node role, service mirror identifier, availability zone location, and routing address of each node, establishing a node association mapping table and a node status baseline. The node status baseline records at least basic status information such as CPU utilization, available memory, task queue length, and network queuing latency, serving as a reference resource environment for subsequent operation. After completing the above processing, the scheduling control plane obtains a unified time reference and node mapping relationship, and subsequent request parsing, sub-scheduling action generation, status detection, and resource verification are all performed under this unified reference. When external terminals continuously initiate mixed database write and query requests, the request parsing module begins operation. The unified application gateway receives external service requests and writes them into a circular buffer. The request parsing module reads the target request from the circular buffer, separates the request header information and business payload, and extracts the request identifier, business category, service level agreement tag, expected response time limit, call context, task semantic description, etc. If the request contains both database write and query linkage intents, the request parsing module identifies it as a composite business task and further forms a unified task semantic representation result. Based on this, the resource constraint generation module generates resource constraint information corresponding to the task according to the business type, service level information, and operational requirements of the request. For example, for database services with write priority, a higher continuous memory lower limit constraint and a stricter queuing latency constraint can be formed; for query linkage, a higher routing continuity requirement and a lower cross-region jump tolerance can be formed. After forming the task semantic representation result and resource constraint information, the request parsing module writes the relevant content into the task management table and establishes a correlation with the aforementioned node status baseline for direct inference in the future. After a task enters the inference queue, the sub-scheduling generation and snapshot management module drives the large language model inference engine to generate scheduling content. The scheduling control plane reads the target task from the task management table, organizes the task identifier, business category, task semantic representation result, resource constraint information, node status baseline summary, and currently validated action summary into input content, and sends it to the large language model inference engine. The large language model inference engine does not output the complete scheduling scheme at once, but generates scheduling text segment by segment according to an autoregressive method. As the decoding process progresses, the sub-scheduling generation and snapshot management module continuously performs action boundary monitoring at the output end. When the current output simultaneously meets the predefined boundary marker and action semantic matching conditions, it is determined that an independently executable sub-scheduling action has been formed, and the current inference process is immediately paused, the current generation state is frozen, and a context snapshot corresponding to the sub-scheduling action is formed. This context snapshot and the action text summary are registered together in the snapshot index table for subsequent action-level verification and partial rollback. For example, in this scenario, the first sub-scheduling action generated by the large language model inference engine could be to prioritize the allocation of database write tasks to write instance nodes with high contiguous available memory within the first availability zone, while temporarily binding query tasks to query nodes in the same zone for subsequent verification. After the sub-scheduling generation and snapshot management module completes the generation of this action, it immediately freezes the current context and hands the action over to the node detection and verification module for processing, instead of continuing to generate a complete chain. As a result, the scheduling chain generated by the large language model is split into multiple independently verifiable scheduling units, and each scheduling unit does not enter the final execution chain before passing the verification. After receiving the sub-scheduling action, the node detection and verification module determines the target node based on the service instance, resource preference, and availability zone identifier in the action text. If the logical hostname is given directly in the action, the corresponding physical node is determined by calling the node association mapping table. If the action only gives the write instance type and resource preference, candidate nodes are filtered by combining the service instance mapping relationship, node status baseline, and availability zone range. Then, they are sorted by remaining contiguous available memory, current CPU utilization, and network round-trip latency to determine the target node to be detected first. After determining the target node, the node detection and verification module sends a status probe request to the node through out-of-band probing. The kernel-mode status probe in the target node returns status data such as current CPU utilization, task scheduling queue length, number of contiguous available memory page frames, and network queuing latency, forming a node status vector. Subsequently, the node detection and verification module combines the node status vector with the aforementioned resource constraint information as a feasibility judgment input. First, it performs boundary condition verification of CPU utilization, contiguous available memory, and network queuing latency, and then calculates the comprehensive matching degree to determine whether the node meets the carrying requirements of the current sub-scheduling action. If the verification result shows that the target node meets the resource requirements of the current sub-scheduled action, the node detection and verification module returns a pass result to the rollback correction and execution control module. The rollback correction and execution control module then releases the pause state of the current context snapshot, writes the action into the verified action chain, and updates the temporary resource state copy based on the resource occupancy result corresponding to the action. For example, after a database write task has been determined to fall on a write node in the first availability zone, the rollback correction and execution control module registers the node's occupied CPU share, occupied contiguous memory, and occupied network budget in the temporary resource state copy. This ensures that the subsequent action generation process is no longer based on the initial node state baseline, but rather on the temporary resource state after accumulating the impact of previous actions. Subsequently, the scheduling control plane resumes the generation process of the large language model inference engine, allowing the large language model to continue generating the next sub-scheduled action for the composite task. If the verification result shows that the target node does not meet the resource requirements of the current sub-scheduling action, the rollback correction and execution control module does not trigger a full re-push of the entire link, but performs a partial rollback. The scheduling control plane reads the snapshot index table, locates the previous safe snapshot corresponding to the current failed action, and restores the model context to that safe position. At this time, the actions that have passed the verification before the safe position remain unchanged, and only the current failed action and the subsequent unverified decoding state are discarded. Subsequently, the rollback correction and execution control module converts the resource diagnostic vector returned by the node detection and verification module into correction information and injects it after the constraint segment in the model input context. For example, if the target write node in the first availability zone fails the verification due to insufficient continuous available memory, the correction information can instruct the large language model to avoid the node and prioritize the search for a backup node in the same availability zone that meets the continuous memory threshold of the current task. If there is no node that meets the conditions in the same zone, the database write backup instance node in the second availability zone is prioritized. At the same time, the rollback correction and execution control module applies output suppression to the candidate nodes corresponding to the failed node, so that the large language model reduces the probability of selecting the failed node again when regenerating the current action. In this scenario, if the large language model regenerates a new database write operation under the combined effect of corrective information and output suppression, and the new target node passes state detection and resource verification, then the operation enters the verified operation chain, and subsequently continues to generate query binding operations, log processing operations, and necessary cross-region synchronization operations. If an operation fails to pass verification after a preset number of consecutive local retries, the rollback correction and execution control module stops the continued regeneration at the current position and calls the pre-set scheduling logic to generate a basic traffic splitting scheme. For example, when there are no write nodes that meet the continuous available memory threshold in either of the two availability zones, the system can push the current task into the asynchronous buffer queue or prioritize forwarding it to the set of standby nodes in the disaster recovery area. When the length of the asynchronous buffer queue continues to increase, the elastic expansion interface is triggered to avoid the current task continuously occupying inference resources and blocking subsequent requests. After multiple consecutive sub-scheduling actions pass verification, if the large language model outputs an end marker, it indicates that the scheduling chain corresponding to the current composite task has generated a complete scheduling sequence. At this point, the rollback correction and execution control module performs a link-level review of the verified action chain. The link-level review checks for execution order conflicts between adjacent actions, whether the same resource is repeatedly allocated within the same time window, and whether cross-region routing involves repeated jumps. If the link-level review passes, the verified action chain is encapsulated into a complete scheduling sequence and sent to the task execution interface. To reduce the risk of inconsistency during the execution phase, the system can adopt a batch submission method with a confirmation mechanism. That is, each execution node first returns preparation confirmation information, and then the scheduling control plane uniformly sends the final submission command. Finally, each execution node performs actions such as database writing, query binding, log forwarding, and synchronous route configuration according to the scheduling sequence to complete this round of task processing. Through the above collaborative operation process, the system can perform action-level linkage verification and local correction on the long thought chain scheduling process of large language model generation in a distributed microservice cluster environment with high concurrency and dynamic resource fluctuations. This reduces the inference overhead caused by full push and improves the executability and operational stability of the scheduling sequence.

[0025] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0026] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented in whole or in part by a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions of the embodiments of this application are implemented in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted wirelessly or wiredly from one website, computer, server, or data center to another website, computer, server, or data center. Wired methods include optical fiber, twisted pair, coaxial cable, etc. Wireless methods include infrared, microwave, etc. Available media include any available media that can be accessed by a computer or data storage devices such as servers and data centers that contain one or more sets of available media. Available media can be magnetic media (floppy disks, hard disks, magnetic tapes), optical media (DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0027] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0028] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A service scheduling method based on enhanced decision-making via thought chain, characterized in that, include: S1. Perform full network time alignment on the distributed microservice cluster and register the identification information and mapping relationship of each node; S2. Receive and parse concurrent service requests to determine the tasks to be executed and the corresponding resource constraints; S3. Drive the large language model to generate sub-scheduling actions step by step. After generating each sub-scheduling action, pause the current inference process and save the corresponding context snapshot. S4. Determine the target node based on the sub-scheduling action, perform state detection on the target node, and verify the detected node state against resource constraints; S5. Control the subsequent inference process based on the verification result. If the verification passes, continue to generate subsequent sub-scheduling actions. If the verification fails, roll back to the corresponding context snapshot and inject correction information until a complete scheduling sequence is generated and issued for execution.

2. The service scheduling method based on thought chain decision enhancement according to claim 1, characterized in that, S1 includes: After detecting a cold start or node change event, identify the active node and perform time synchronization correction on the active node; Perform consistency checks on node attribute information and generate node index codes; Establish a node association mapping table and node status identifier based on the node index code; Collect resource status according to the keep-alive cycle and form a node status baseline; Nodes with abnormal attributes, time synchronization, or keep-alive status will be subject to pending review or isolation.

3. The service scheduling method based on thought chain decision enhancement according to claim 1, characterized in that, S2 includes: Perform window trigger identification, independence determination, header splitting, integrity verification, and anomaly correction on requests; By combining request semantics, service level information, business category and historical operation profile, determine latency sensitivity, task level, resource demand tendency, resource boundary constraints, candidate business images and constraint version numbers; The relevant information is registered in the task management table, and the buffer occupancy status is monitored and eviction is controlled.

4. The service scheduling method based on thought chain decision enhancement according to claim 1, characterized in that, S3 includes: For tasks entering the inference queue, the inference input is organized according to the business category, task semantics, resource constraints, node state baseline summary, and verified action summary; Perform segment-by-segment decoding and determination based on predefined boundary markers and action reference distribution matching; During the segment-by-segment decoding process, the generation state is frozen and a snapshot index is registered according to the action sequence number. Abnormal generation is subject to quantity constraints, duplicate identification, deduplication constraint injection, and timeout splitting.

5. The service scheduling method based on thought chain decision enhancement according to claim 4, characterized in that, During the segment-by-segment decoding process, the generation state is frozen and a snapshot index is registered according to the action sequence number. Abnormal generation is handled by performing quantity constraints, duplicate identification, deduplication constraint injection, and timeout routing, including: Record the current sub-scheduling action text, decoding position, hidden state, key-value cache, context offset position, and task identifier, and build a snapshot index according to the action sequence number; When the number of generated contents exceeds the limit, the process is transferred to a review process. When generated content is repeatedly repeated across action categories and target nodes, deduplication constraints are written. If no valid action is performed within the predetermined time limit, the preset traffic splitting process will be invoked.

6. The service scheduling method based on thought chain decision enhancement according to claim 1, characterized in that, S4 includes: Extract entities from sub-scheduling actions, and filter and sort candidate nodes based on logical hostnames, business instance relationships, or resource allocation intentions; Out-of-band probing is performed on candidate nodes, and the nodes are then smoothed and normalized to form node state vectors. Verification is performed based on resource boundary conditions and overall matching degree; When an anomaly is detected, supplementary detection or status marking is performed; when data is missing, supplementary data collection is performed; and when verification fails, resource diagnostic information is generated.

7. The service scheduling method based on thought chain decision enhancement according to claim 6, characterized in that, Out-of-band probing is performed on candidate nodes, and the nodes are then smoothed and normalized to form node state vectors, including: Bypassing the conventional container control interface, construct a status probe message containing task identifier, action sequence number, probe type, sending timestamp and verification information, and send it to the candidate node via User Datagram Protocol; The kernel-mode probes in the candidate nodes read the CPU utilization, task queue length, number of consecutive available memory page frames, network queuing latency, or video memory usage status. The results are then combined with multiple sampling results and the node state baseline for smoothing and normalization mapping to form a node state vector.

8. The service scheduling method based on thought chain decision enhancement according to claim 1, characterized in that, S5 includes: Update the verified action chain based on the verification flag, and update the temporary resource status copy according to resource usage and release status; Perform a partial rollback based on the previous security snapshot or the initial snapshot, write the resource over-limit dimension, the current available upper limit and the alternative node constraint, apply candidate suppression, and continue processing according to the partial retry rules or the basic traffic splitting rules; After the scheduling chain is closed, the execution sequence conflict review, resource duplication review, and region jump review are performed, and then the confirmation and submission process is followed.

9. The service scheduling method based on thought chain decision enhancement according to claim 8, characterized in that, Update the verified action chain based on the verification flag, and update the temporary resource state copy according to resource usage and release status, including: After verification, the pause state of the current context snapshot is released, the action summary of the corresponding sub-scheduling action is appended to the verified action chain, and the resource allocation record of the action is determined without rewriting the node state baseline. Perform deduction or reclamation registrations on the processor available share, contiguous available memory, network budget, or video memory available share in the temporary resource status copy.

10. A service scheduling system based on enhanced decision-making using thought chains, comprising the service scheduling method based on enhanced decision-making using thought chains as described in any one of claims 1-9, characterized in that, include: Time benchmark and node documentation module: used to perform network-wide time alignment on the distributed microservice cluster, register the identification information and mapping relationship of each node, and establish node association relationship and node status baseline; Request parsing module: Used to receive and parse concurrent service requests to determine the tasks to be executed; Resource constraint generation module: used to determine the resource constraints corresponding to the task to be executed based on the parsed request content, service level information and business type; Sub-scheduling generation and snapshot management module: This module is used to drive the large language model to generate sub-scheduling actions step by step based on the tasks to be executed and resource constraints, and to pause the current inference process after generating each sub-scheduling action and save the corresponding context snapshot. Node detection and verification module: used to determine the target node based on the sub-scheduling action, perform status detection on the target node, and verify the detected node status against resource constraints; Rollback Correction and Execution Control Module: Used to control the subsequent inference process based on the verification results. When the verification passes, it continues to generate subsequent sub-scheduling actions. When the verification fails, it rolls back to the corresponding context snapshot and injects correction information. After forming a complete scheduling sequence, it performs review and distribution.

Images