Heterogeneous computing reasoning scheduling method and system for distributed software architecture

Abstract

The application relates to the technical field of distributed computing resource scheduling. A heterogeneous computing power reasoning scheduling method and system for a distributed software architecture are provided, wherein the method comprises the following steps: performing time delay component separation on a real-time reasoning time delay sequence reported by a node through a time delay component separation model to obtain a jitter component confidence interval and a decay component trend coefficient; performing incremental time sequence alignment on a plurality of reasoning streams based on the jitter component confidence interval and the decay component trend coefficient to generate a multi-stream time sequence offset table; implementing closed-loop scheduling decision based on the multi-stream time sequence offset table to obtain a node-level scheduling instruction set; and collecting new time delay data after executing the node-level scheduling instruction set, and updating parameters of the time delay component separation model, so that the technical effects of improving time delay feature analysis precision, reducing multi-stream collaborative delay and enhancing dynamic adaptability of a scheduling strategy are achieved.

Description

Technical Field

[0001] This invention relates to the field of distributed computing resource scheduling technology, and in particular to a heterogeneous computing power inference scheduling method and system for distributed software architecture. Background Technology

[0002] With the widespread deployment of AI inference applications in edge computing, autonomous driving, and the Industrial Internet of Things, distributed software architecture has become a critical infrastructure supporting large-scale AI services. Efficiently scheduling inference tasks in heterogeneous computing environments is a core element in ensuring the service quality of real-time sensitive applications (such as autonomous driving decision-making and industrial quality inspection), and it is also a key focus and challenge in distributed system optimization.

[0003] Traditional distributed inference scheduling uses fixed thresholds for latency monitoring, which cannot break down the different characteristics of latency and cannot effectively identify the causes of various latency anomalies, significantly reducing the accuracy of scheduling decisions. Traditional multi-stream collaboration uses a batch processing-style time-series alignment mode, which relies on complete latency sequences to complete the operation, resulting in significant processing lag and making it difficult to adapt to the needs of real-time inference applications. In addition, traditional scheduling strategies maintain a static and fixed form for a long time, lacking dynamic closed-loop adjustment capabilities, and cannot adapt to dynamic changes in device status and network environment, making scheduling performance prone to continuous degradation. Summary of the Invention

[0004] Based on this, it is necessary to provide a heterogeneous computing power inference scheduling method and system for distributed software architecture to address the above-mentioned technical problems, so as to improve the accuracy of latency feature analysis, reduce multi-stream collaborative latency, and enhance the dynamic adaptability of scheduling strategies.

[0005] In a first aspect, this application provides a heterogeneous computing power inference scheduling method for distributed software architectures, the method comprising:

[0006] The real-time inference delay sequence reported by the node is separated into delay components by a delay component separation model to obtain the confidence interval of the jitter component and the trend coefficient of the attenuation component.

[0007] Based on the confidence interval of jitter component and the trend coefficient of attenuation component, incremental time alignment is performed on multi-stream inference streams to generate a multi-stream time offset table.

[0008] A closed-loop scheduling decision is made based on a multi-stream time offset table to obtain a node-level scheduling instruction set.

[0009] After executing the node-level scheduling instruction set, new latency data is collected, and the parameters of the latency component separation model are updated.

[0010] In one embodiment, a closed-loop scheduling decision is implemented based on a multi-stream timing offset table to obtain a node-level scheduling instruction set, including:

[0011] Synchronization error aggregation is performed on the multi-stream timing offset table to obtain the global synchronization error entropy;

[0012] Based on the global synchronization error entropy, proportional control conversion is performed to generate resource quota adjustment amounts;

[0013] Integral control conversion is performed based on global synchronization error entropy to generate migration target node identifiers;

[0014] Differential control transformation is performed based on global synchronization error entropy to generate buffer configuration parameters;

[0015] The rate of change of control signals is obtained by monitoring the rate of change of resource quota adjustment, migration target node identifier, and buffer configuration parameters.

[0016] When the rate of change of the control signal exceeds the preset threshold, stability protection is executed, and a conservative scheduling flag is obtained.

[0017] Combine resource quota adjustment amounts, migration target node identifiers, buffer configuration parameters, and conservative scheduling flags to generate a node-level scheduling instruction set.

[0018] In one embodiment, differential control transformation is performed based on global synchronization error entropy to generate buffer configuration parameters, including:

[0019] The instantaneous gradient is obtained by performing time window differentiation on the global synchronization error entropy;

[0020] Jitter pattern recognition is performed based on instantaneously changing gradients to generate jitter pattern classification labels;

[0021] Based on the jitter pattern classification identifier index, a preset buffer strategy library is used to obtain a buffer strategy template;

[0022] The parameters of the buffer strategy template are adjusted based on the instantaneously changing gradient to generate a set of buffer parameters;

[0023] Combine the buffer strategy template and the set of buffer parameters to generate buffer configuration parameters.

[0024] In one embodiment, when the rate of change of the control signal exceeds a preset threshold, stability protection is performed to obtain a conservative scheduling flag, including:

[0025] The rate of change of the control signal is compared with multiple thresholds to obtain the out-of-limit level indicator;

[0026] Based on the over-limit level identifier, select the protection strategy template and generate the strategy selection vector;

[0027] Based on the strategy, a vector is selected to execute the instruction freeze operation, and a freeze state record is obtained;

[0028] Historical state rollback points are generated based on the frozen state records to obtain the rollback baseline snapshot;

[0029] Combine the over-limit level identifier, strategy selection vector, and rollback baseline snapshot to generate a conservative scheduling flag.

[0030] In one embodiment, based on the jitter component confidence interval and the attenuation component trend coefficient, incremental time alignment is performed on multiple inference streams to generate a multi-stream time offset table, including:

[0031] Perform baseline anchor point selection on the preset flow priority label to obtain the baseline anchor point inference flow;

[0032] Establish a benchmark virtual timeline based on the benchmark anchor point inference flow;

[0033] When any inference stream generates a new data point, the penalized weight of the attenuation component trend coefficient is adjusted to generate a dynamic regularization weight.

[0034] The path boundary is defined based on the confidence interval of the jitter component to obtain the restricted path space;

[0035] Incremental path calculations are performed on the dynamically regularized weights and the constrained path space to obtain the path cost matrix;

[0036] Timing offset is calculated based on the path cost matrix and the baseline virtual time axis, generating a multi-stream timing offset table.

[0037] In one embodiment, after executing the node-level scheduling instruction set, new latency data is collected, and the parameters of the latency component separation model are updated, including:

[0038] Execute the node-level scheduling instruction set and generate instruction execution status records;

[0039] New latency data is collected based on instruction execution status records to obtain a new latency dataset.

[0040] Identify the migration instruction nodes in the instruction execution status record, reset the attenuation baseline of the relevant nodes, and generate the reset attenuation component sequence;

[0041] Identify the buffer injection nodes in the instruction execution status record, perform time delay compensation on the relevant nodes, and generate compensated time delay observations;

[0042] The time delay component separation model is updated using the new time delay dataset, the reset attenuation component sequence, and the compensated time delay observations, resulting in an updated time delay component separation model.

[0043] Secondly, this application also provides a heterogeneous computing power inference scheduling system for distributed software architecture, the system comprising:

[0044] The delay separation module is used to separate the real-time inference delay sequence reported by the node through the delay component separation model, and obtain the confidence interval of the jitter component and the trend coefficient of the attenuation component.

[0045] The timing alignment module is used to perform incremental timing alignment on multiple inference streams based on the confidence interval of jitter components and the trend coefficient of attenuation components, and generate a multi-stream timing offset table.

[0046] The scheduling decision module is used to implement closed-loop scheduling decisions based on the multi-stream time series offset table to obtain a node-level scheduling instruction set.

[0047] The feedback update module is used to collect new latency data after executing the node-level scheduling instruction set and update the parameters of the latency component separation model.

[0048] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0049] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods in the first aspect of this application.

[0050] This application provides a heterogeneous computing power inference scheduling method and system for distributed software architectures. It employs a latency component separation model to finely decompose the real-time inference latency sequence reported by nodes, obtaining the confidence interval of the jitter component and the trend coefficient of the decay component. This allows for a more comprehensive capture and analysis of latency characteristics, effectively improving the accuracy of latency feature analysis. Based on this, incremental time-series alignment is performed on multiple inference streams using the aforementioned two key parameters. This abandons the traditional batch processing operation mode that relies on complete latency sequences, significantly reducing redundant computational steps in the multi-stream collaboration process, thereby reasonably reducing multi-stream collaboration latency and simultaneously generating a highly accurate multi-stream time-series offset table.

[0051] Based on a multi-stream time-series offset table, a closed-loop scheduling decision is implemented, and a node-level scheduling instruction set is output. After the scheduling instructions are executed, the system collects new latency data and uses it in reverse to iteratively update the parameters of the latency component separation model, constructing a dynamic loop mechanism of "feature splitting - time-series alignment - scheduling execution - model optimization". This loop continuously strengthens the model's ability to perceive and adapt to dynamic changes in heterogeneous computing environments, thereby steadily enhancing the dynamic adaptability of the scheduling strategy and ensuring that the scheduling scheme can be continuously optimized and adjusted according to environmental changes. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a heterogeneous computing power inference scheduling method for distributed software architecture in one embodiment of the present invention.

[0054] Figure 2 This is a flowchart illustrating the process of generating buffer configuration parameters by performing differential control transformation based on global synchronization error entropy in one embodiment of the present invention.

[0055] Figure 3 This is a structural diagram of a heterogeneous computing power inference scheduling system for a distributed software architecture according to one embodiment of the present invention. Detailed Implementation

[0056] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0057] First, the application scenarios of the embodiments of this application are described. In the embodiments of this application, a heterogeneous computing power inference scheduling method and system for distributed software architecture is provided, applicable to but not limited to edge computing, intelligent driving, and industrial IoT scenarios.

[0058] In illustrative purposes, the heterogeneous computing power inference scheduling method and system for distributed software architecture provided in this application embodiment can also be applied to application scenarios such as smart cities, smart healthcare, and cloud-based AI service deployment. This is only an example and does not limit the specific application scenarios.

[0059] like Figure 1 As shown, this application provides a heterogeneous computing power inference scheduling method for distributed software architectures, the method comprising:

[0060] S101: The real-time inference delay sequence reported by the node is separated into delay components by using a delay component separation model to obtain the confidence interval of the jitter component and the trend coefficient of the attenuation component.

[0061] For example, the distributed scheduling and control terminal receives the real-time inference delay sequence reported by each node, performs integrity verification and validity verification on the received real-time inference delay sequence, and ensures that the real-time inference delay sequence meets the input requirements of the delay component separation model. The distributed scheduling and control terminal calls its own preset delay component separation model, completely inputs the verified real-time inference delay sequence into the delay component separation model, and starts the feature decomposition process.

[0062] The latency component separation model systematically extracts features from the input real-time inference latency sequence, identifying two core features: fluctuating features caused by instantaneous environmental interference and temporary resource contention, and progressive features caused by long-term equipment operating losses and transmission link attenuation. For each of the identified core features, the latency component separation model performs specific processing procedures: statistical characteristic analysis and interval definition for fluctuating features, and evolutionary pattern judgment and trend quantification for progressive features.

[0063] Through the above series of processes, the time delay component separation model completes the decomposition and feature quantization of the real-time inference time delay sequence, and obtains the confidence interval of the jitter component and the trend coefficient of the attenuation component.

[0064] S102: Based on the confidence interval of the jitter component and the trend coefficient of the attenuation component, perform incremental time alignment on the multi-stream inference streams to generate a multi-stream time offset table.

[0065] For example, using the jitter component confidence interval and the attenuation component trend coefficient as core references, the distributed scheduling control terminal collects multi-path inference stream data generated in the entire business scenario. At the same time, it uses the jitter component confidence interval to define the reasonable boundary of time series matching and uses the attenuation component trend coefficient to control the overall trend of time series dynamic evolution.

[0066] Following the principle of incremental iteration, the distributed scheduling and control terminal gradually corrects the timing deviations between multiple inference streams. The system sorts out the timing correspondences of each inference stream, integrates all timing difference information and performs regularization processing. Based on the orderly promotion of incremental timing alignment of multiple inference streams, a multi-stream timing offset table is generated.

[0067] S103: Implement closed-loop scheduling decisions based on multi-stream time offset tables to obtain a node-level scheduling instruction set.

[0068] For example, using a multi-stream timing offset table as the core input, the distributed scheduling control terminal analyzes the timing differences of each inference stream and aggregates them to obtain global synchronization error information. Combined with closed-loop control logic, the distributed scheduling control terminal derives the resource quota adjustment amount, migration target node identifier, and buffer configuration parameters based on the global synchronization error information.

[0069] The distributed scheduling and control terminal monitors the rate of change of various decision parameters, determines whether they exceed the stable operation threshold, and generates stability protection flags as needed. The distributed scheduling and control terminal integrates and encapsulates all decision parameters and stability protection flags according to a preset format to obtain a node-level scheduling instruction set.

[0070] S104: After executing the node-level scheduling instruction set, new delay data is collected, and the parameters of the delay component separation model are updated.

[0071] For example, the distributed scheduling control terminal executes the node-level scheduling instruction set, continuously senses the real-time operating status of heterogeneous computing power nodes, continuously collects the timing feedback information generated during the operation of various inference services, and collects and organizes various feedback information to obtain new latency data.

[0072] Based on the actual business operation status after scheduling execution and the newly collected latency data, the distributed scheduling control terminal, combined with the current operating characteristics of the heterogeneous computing environment, corrects the adaptation parameters of the original model and gradually completes the iterative correction of the internal parameters of the model, thereby completing the parameter update of the latency component separation model.

[0073] One embodiment of this application provides a heterogeneous computing power inference scheduling method for distributed software architectures. It uses a latency component separation model to finely decompose the real-time inference latency sequence reported by nodes, obtaining the confidence interval of the jitter component and the trend coefficient of the decay component. This allows for a more comprehensive capture and analysis of latency characteristics, effectively improving the accuracy of latency feature analysis. Based on this, incremental time-series alignment is performed on multiple inference streams using the aforementioned two key parameters. This abandons the traditional batch processing operation mode that relies on complete latency sequences, significantly reducing redundant computational steps in the multi-stream collaboration process, thereby reasonably reducing multi-stream collaboration latency and synchronously generating a highly accurate multi-stream time-series offset table.

[0074] Based on a multi-stream time-series offset table, a closed-loop scheduling decision is implemented, and a node-level scheduling instruction set is output. After the scheduling instructions are executed, the system collects new latency data and uses it in reverse to iteratively update the parameters of the latency component separation model, constructing a dynamic loop mechanism of "feature splitting - time-series alignment - scheduling execution - model optimization". This loop continuously strengthens the model's ability to perceive and adapt to dynamic changes in heterogeneous computing environments, thereby steadily enhancing the dynamic adaptability of the scheduling strategy and ensuring that the scheduling scheme can be continuously optimized and adjusted according to environmental changes.

[0075] In one embodiment, a closed-loop scheduling decision is implemented based on a multi-stream timing offset table to obtain a node-level scheduling instruction set, including:

[0076] (1) Perform synchronization error aggregation on the multi-stream timing offset table to obtain the global synchronization error entropy.

[0077] For example, based on all time-series record information stored in the multi-stream time-series offset table, the distributed scheduling control terminal extracts the time-series offset data generated by each inference stream during the collaborative scheduling process, unifies and standardizes the statistical dimensions and statistical standards of all time-series data, and eliminates computational interference caused by differences in data formats under heterogeneous computing power environments.

[0078] Based on a standardized entropy statistical algorithm, the distributed scheduling and control terminal integrates the time-series deviation data of dispersed multi-path inference streams across the entire domain. It quantifies the degree of time-series coordination differences between different inference streams, performs deep aggregation calculations of multi-dimensional synchronization errors, reflects the overall time-series imbalance of the inference business, and obtains the global synchronization error entropy, as shown in the following formula:

[0079]

[0080] In the formula, Represents the global synchronization error entropy, used to uniformly characterize the degree of disorder in the overall temporal synchronization of multi-path inference streams; This represents the probability density of synchronization error corresponding to a single inference stream. This represents the cumulative calculation of the error quantization results for all inference streams; This represents the total number of inference streams currently running in parallel in the system.

[0081] Among them, the global synchronization error entropy can reflect the overall temporal coordination state of multiple inference streams under the distributed architecture, and can be used as the basis for evaluation of closed-loop scheduling decisions.

[0082] (2) Based on the global synchronization error entropy, perform proportional control conversion to generate resource quota adjustment amount.

[0083] For example, with the global synchronization error entropy as the core input variable, the distributed scheduling and control terminal calls the built-in proportional control operation rules, and combines the resource allocation constraints of the heterogeneous computing power cluster to introduce a fixed preset proportional control coefficient for linear mapping operation.

[0084] Based on the severity of timing deviations reflected by the global synchronization error entropy, the distributed scheduling and control terminal adaptively matches the adjustment range and direction of computing resources, strictly follows the proportional control calculation logic to complete parameter conversion and operational constraint verification, and fully executes the entire proportional control conversion process to generate the resource quota adjustment amount, as shown in the following formula:

[0085]

[0086] In the formula, This represents the amount of resource quota adjustment, used to define the rules for increasing or decreasing resource allocation for heterogeneous computing power nodes; Represents the proportional control coefficient, which is a fixed control parameter pre-configured by the system; Representing the global synchronization error entropy, it is the core input data source for this proportional control conversion.

[0087] Among them, the resource quota adjustment carries the computing power allocation adjustment information of heterogeneous computing power nodes, which can be directly applied to the resource scheduling configuration of each distributed node.

[0088] (3) Perform integral control conversion based on global synchronization error entropy to generate migration target node identifier.

[0089] For example, the distributed scheduling and control terminal continuously records the long-term evolution of error values ​​based on the time-series data of global synchronization error entropy over a complete operating cycle, and performs statistical analysis on the error accumulation trend in the time-series dimension. After enabling the integral control operation rule, the distributed scheduling and control terminal accumulates and solves the global synchronization error entropy over continuous time periods. It then performs multi-dimensional screening and comparison based on the hardware computing power performance, real-time load pressure, and inference task adaptability of each heterogeneous computing power node to select the optimal operating node that matches the long-term error accumulation characteristics. The target node determination is completed strictly following the integral control conversion logic, and a migration target node identifier is generated. The formula is as follows:

[0090]

[0091] In the formula, This represents the integral control output value, used to characterize the long-term cumulative effect of the global synchronization error entropy; Represents the integral control coefficient; This represents the error integral operation over a continuous time interval. Represents the start time of the statistical period; Represents the current running time of the system; A continuous-time variable representing the integration process; Represents a time-varying infinitesimal variable.

[0092] Among them, the migration target node identifier is a unique identifier for heterogeneous computing power nodes, which can locate the migration carrier of the inference task.

[0093] (4) Perform differential control conversion based on global synchronization error entropy to generate buffer configuration parameters.

[0094] For example, focusing on the dynamic changes in global synchronization error entropy between adjacent time nodes, the distributed scheduling and control terminal identifies the fluctuation rate and evolution trend of timing synchronization errors, clearly distinguishing the differentiated characteristics of steady-state deviation and abrupt deviation. After initiating the differential control operation rules, the distributed scheduling and control terminal calculates the real-time gradient of the global synchronization error entropy. Combining business attributes such as the data transmission rate of multiple inference streams, the data cache capacity limit, and real-time interaction requirements, it matches an appropriate cache operation strategy. Based on the differential control transformation logic, it completes the adaptive conversion and correction of cache parameters, generating buffer configuration parameters, where the formula is as follows:

[0095]

[0096] In the formula, This represents the differential control output value, used to characterize the real-time rate of change of the global synchronization error entropy; Represents the differential control coefficient; The time-varying derivative of the global synchronization error entropy; This represents the global synchronization error entropy at the current moment; Represents a micro-element that changes over a short period of time; This represents the real-time fluctuation increment of the global synchronization error entropy.

[0097] Among them, the buffer configuration parameters cover a variety of configuration contents such as cache capacity, data refresh rate, and data queuing rules, which support stable cache interaction of inference stream data.

[0098] (5) Monitor the rate of change of resource quota adjustment, migration target node identifier and buffer configuration parameters to obtain the rate of change of control signal.

[0099] For example, by establishing a full-time parameter monitoring link, the distributed scheduling control terminal continuously collects the real-time update status of resource quota adjustments, migration target node identifiers, and buffer configuration parameters, and periodically retains complete parameter snapshots at different operating times. The distributed scheduling control terminal calculates the change magnitude and fluctuation frequency of the three types of scheduling parameters within a continuous period, introduces multi-dimensional weighting coefficients to weight and fuse the change characteristics of various parameters, comprehensively measures the overall severity of scheduling command changes, and uniformly completes the integration and calculation of multi-source change data to calculate the control signal change rate, as shown in the following formula:

[0100]

[0101] In the formula, It represents the rate of change of the control signal and is used to comprehensively quantify the overall volatility of the three types of scheduling parameters. This represents the weighting coefficient of the resource quota adjustment amount. It is a system-preset constant used to define the proportion and weight of the resource quota adjustment amount in the calculation of the rate of change. This represents the resource quota adjustment amount at the current monitoring moment, and is a real-time scheduling parameter generated during proportional control conversion; This represents the resource quota adjustment amount at the previous monitoring time, and is a scheduling parameter generated within the historical period; This represents a fixed monitoring time interval, which is the system's preset continuous data acquisition time period; This represents the weight coefficient of the target node identifier during migration. It is a system-preset constant used to define the weight of the target node identifier in the calculation of the rate of change. The representative condition indicator function is a logical decision function used to mark whether the node identifier has changed; The identifier of the migration target node represents the current monitoring time and is a real-time scheduling parameter generated for integral control conversion; This represents the logical inequality operator, used to determine whether there is a difference between the node identifiers at the current time and those at historical time. The identifier of the migration target node represents the previous monitoring time and is a scheduling parameter generated within the historical period; This represents the weighting coefficient of the buffer configuration parameters. It is a system-preset constant used to define the weighting of the buffer configuration parameters in the rate of change calculation. The buffer configuration parameters represent the current monitoring time and are real-time scheduling parameters generated by the differential control conversion; The buffer configuration parameters represent the previous monitoring time and are scheduling parameters generated within the historical period.

[0102] Among them, the rate of change of control signals can intuitively reflect the degree of adjustment of scheduling strategies and is the core basis for judging system stability protection.

[0103] (6) When the rate of change of the control signal exceeds the preset threshold, stability protection is performed to obtain the conservative scheduling flag.

[0104] For example, after the control signal change rate is calculated in real time, the distributed scheduling control terminal compares it with the multi-level stability thresholds preset by the system to determine whether the control signal change rate exceeds the safe operating range. When an over-limit state is detected, the distributed scheduling control terminal immediately activates the preset stability protection operation mechanism, restricts the frequent switching of high-frequency scheduling commands, synchronously records key information such as the current cluster load status, network transmission status, and inference task running status, generates a traceable system operation snapshot, marks the system as entering the stable protection scheduling mode, and obtains the conservative scheduling flag bit through the complete protection process.

[0105] The stability protection mechanism is a built-in protective operation logic in the distributed scheduling control terminal, possessing the basic capabilities to suppress scheduling oscillations, lock the operating status, and retain backtracking snapshots. The conservative scheduling flag is an independent status identifier parameter that carries key information such as the operating protection level, policy lock status, and historical snapshot index.

[0106] (7) Combine the resource quota adjustment amount, migration target node identifier, buffer configuration parameters and conservative scheduling flag to generate a node-level scheduling instruction set.

[0107] For example, following the standardized instruction compilation specifications of the distributed heterogeneous computing power scheduling system, the distributed scheduling control terminal orderly aggregates the resource quota adjustment amount, migration target node identifier, buffer configuration parameters, and generated conservative scheduling flags generated by computation. The distributed scheduling control terminal verifies the completeness, validity, and logical adaptability of various scheduling parameters item by item, investigates operational conflicts and adaptation defects between parameters, uniformly completes the standardized encapsulation and format adaptation of multiple scheduling elements, organizes all scheduling control information and integrates and arranges it in an integrated manner, and generates a node-level scheduling instruction set after compliance verification.

[0108] Among them, the node-level scheduling instruction set is a standardized set of control instructions that is adapted to the instruction parsing and execution standards of heterogeneous computing power nodes under a distributed software architecture.

[0109] like Figure 2 As shown, differential control transformation is performed based on global synchronization error entropy to generate buffer configuration parameters, including:

[0110] S201: Perform time window differentiation calculation on the global synchronization error entropy to obtain the instantaneous gradient.

[0111] For example, the distributed scheduling control terminal selects a time window of preset duration and extracts continuous global synchronization error entropy time-series data within that time window, ensuring that the extracted data is complete and uninterrupted in the time dimension, thus guaranteeing the validity of the underlying data for computation. The distributed scheduling control terminal performs a sliding differential operation on the extracted global synchronization error entropy data, calculating the difference in global synchronization error entropy between two adjacent data points within the time window. It then calculates the ratio of this difference to the time interval between the data points to obtain the local rate of change for each adjacent data point. Finally, it averages all local rates of change to filter out instantaneous random noise, obtaining the instantaneous gradient, as shown in the following formula:

[0112]

[0113] In the formula, Represents the instantaneous gradient, used to quantify the average rate of change of the global synchronization error entropy within the current time window; The total number of data points contained in the time window is a system-preset constant. This represents the summation of the local rates of change of all adjacent data points within a time window; Represents the current moment The previous The global synchronization error entropy corresponding to each sampling interval; Represents the current moment The previous The global synchronization error entropy corresponding to each sampling interval; The sampling time interval, representing the global synchronization error entropy, is a fixed time unit preset by the system. This represents the current computation time.

[0114] Among them, time window differential calculation is a computational method that performs local differential processing on time series data based on a sliding window of fixed duration. By integrating the local rate of change within the window, noise filtering and trend extraction are achieved, which can capture the short-term dynamic change characteristics of the global synchronization error entropy.

[0115] S202: Based on the instantaneous gradient change, jitter pattern recognition is performed to generate jitter pattern classification labels.

[0116] For example, the distributed scheduling and control terminal extracts core feature indicators from the instantaneously changing gradient, including key information such as gradient peak value, gradient duration, gradient fluctuation frequency, and gradient change amplitude, to ensure that the feature indicators can reflect the dynamic attributes of the gradient. The distributed scheduling and control terminal performs multi-dimensional comparisons between the extracted feature indicators and a preset jitter pattern feature library, which contains feature thresholds and judgment rules for various jitter patterns such as sudden jitter, gradual jitter, and steady-state jitter.

[0117] The distributed scheduling and control terminal determines the jitter mode corresponding to the global synchronization error entropy based on the comparison results. If the gradient peak exceeds the burst threshold and the duration is less than the preset duration, it is judged as burst jitter; if the gradient is gentle and changes continuously in the same direction, it is judged as gradual jitter; if the gradient fluctuation amplitude is within the steady-state threshold range, it is judged as steady-state jitter. A unique jitter mode classification identifier is generated based on the judgment results.

[0118] The jitter pattern feature library is a pre-built feature storage and judgment system, which includes typical feature parameters, threshold standards and matching rules for various jitter patterns, providing a standardized basis for jitter type judgment; the jitter pattern classification identifier is a unique identifier used to distinguish different jitter types, which includes key information such as jitter type encoding and feature matching degree, and is used to index the corresponding buffering strategy.

[0119] S203: Based on the jitter mode classification identifier index of the preset buffer strategy library, obtain the buffer strategy template.

[0120] For example, the distributed scheduling and control terminal invokes the system's built-in preset buffering strategy library. This strategy library uses jitter mode classification identifiers as index keywords to establish a one-to-one mapping relationship between various jitter modes and corresponding buffering strategy templates. The distributed scheduling and control terminal uses the generated jitter mode classification identifiers as search conditions to perform a search operation in the preset buffering strategy library, locating the buffering strategy template corresponding to the identifier. The buffering strategy template contains core content such as the buffering mechanism framework, parameter adjustment direction, and execution logic for a specific jitter mode, ensuring that the strategy template can adapt to the corresponding jitter characteristics, thus obtaining the buffering strategy template.

[0121] Among them, the preset buffer strategy library is a pre-built strategy storage system that covers the optimal buffer strategy framework under various jitter scenarios. Each strategy template clearly defines the core logic, applicable scenarios and parameter adjustment range of the buffer mechanism. The buffer strategy template is a basic strategy framework designed for a specific jitter mode, providing standardized structural support for subsequent parameter refinement adjustments.

[0122] S204: Adjust the parameters of the buffer strategy template based on the instantaneously changing gradient to generate a set of buffer parameters.

[0123] For example, the distributed scheduling control terminal uses the magnitude and trend of instantaneous gradient changes as the core basis for parameter adjustment, determining the adjustment direction and magnitude of each adjustable parameter in the buffer strategy template. The distributed scheduling control terminal converts the instantaneous gradient changes into parameter adjustment values ​​using a preset linear mapping rule. If the instantaneous gradient change is positive and large, parameters such as increasing the buffer capacity and data refresh frequency are adjusted; if the instantaneous gradient change is negative and flat, the buffer capacity and refresh frequency are appropriately reduced to ensure that the parameter adjustments are compatible with the jitter trend.

[0124] The distributed scheduling and control terminal adjusts each adjustable parameter in the buffer policy template, such as the cache capacity threshold, data queuing rules, cache refresh cycle, and data discard threshold, one by one. After adjustment, all parameters are integrated and standardized to generate a buffer parameter set. The core calculation of this parameter adjustment process is implemented through the following formula:

[0125]

[0126] In the formula, This represents the adjusted buffer parameter value; This represents the initial baseline value of this parameter in the buffer strategy template; This represents the gradient parameter adjustment coefficient, a system-preset constant used to adjust the weight of the influence of instantaneously changing gradients on parameter adjustment. This represents the instantaneous gradient and serves as the core input for parameter adjustment.

[0127] The buffer parameter set is a standardized set of specific executable parameters such as cache capacity threshold, data queuing rules, and cache refresh cycle. Each parameter has a specific value and can directly define the operation details of the buffer mechanism.

[0128] S205: Combine the buffer strategy template and the set of buffer parameters to generate buffer configuration parameters.

[0129] For example, the distributed scheduling control terminal performs an adaptability check on the buffer policy template and the buffer parameter set, verifying whether each parameter in the buffer parameter set conforms to the framework constraints of the buffer policy template, and ensuring that the parameter values ​​do not conflict or contradict the policy logic. For example, the cache capacity parameter must not exceed the adjustment range specified by the policy template, and the data refresh cycle must be compatible with the buffer mechanism in the policy logic.

[0130] The distributed scheduling and control terminal uses the verified buffer policy template as the basic operating framework, embeds the specific parameter values ​​in the buffer parameter set into the corresponding parameter fields of the framework, and performs standardized integration and format encapsulation according to the configuration protocol of the distributed system buffer to obtain configuration data containing complete information such as policy type identifier, parameter value, execution priority, and effective duration, and generates buffer configuration parameters.

[0131] Among them, the buffer configuration parameters are a standardized set of parameters used to directly guide the operation of the buffer. They integrate the strategy framework and specific parameter values, conform to the buffer parsing and execution standards under the distributed architecture, and can ensure that the buffer runs stably in a way that adapts to the jitter mode.

[0132] In one embodiment, when the rate of change of the control signal exceeds a preset threshold, stability protection is performed to obtain a conservative scheduling flag, including:

[0133] (1) Perform multi-level threshold comparison on the rate of change of the control signal to obtain the over-limit level identifier.

[0134] For example, the distributed scheduling control terminal retrieves a pre-configured multi-level stability threshold system from the system, and progressively inputs the control signal change rate acquired through real-time monitoring into the threshold judgment logic, sequentially performing item-by-item comparison and verification of different protection level thresholds. The distributed scheduling control terminal classifies the anomaly levels based on the deviation range of the control signal change rate, distinguishes between minor, moderate, and severe exceedances, and completes the level classification based on the difference in deviation magnitude, defining the severity of the current scheduling system anomaly and obtaining the exceedance level identifier, as shown in the following formula:

[0135]

[0136] In the formula, This represents the level of severity of the anomaly, used to mark the level of abnormality corresponding to the rate of change of the control signal. This represents a multi-level threshold determination function, used to complete numerical range matching and level classification; This represents the rate of change of the control signal and is the core input data for this threshold comparison. This represents the threshold for a level 1 mild over-limit. This represents the threshold for a moderate level 2 exceedance. This represents the threshold for Level 3 severe over-limit.

[0137] Among them, the multi-level stability threshold system is a pre-set layered protection standard that can classify different control levels according to different fluctuation intensities.

[0138] (2) Select a protection strategy template based on the over-limit level identifier and generate a strategy selection vector.

[0139] For example, based on the generated over-limit level identifier, the distributed scheduling and control terminal accesses the built-in stability protection strategy resource library to establish a fixed association mapping between the level identifier and the protection scheme. The distributed scheduling and control terminal matches the control logic that is suitable for the current anomaly level, selects and retrieves the protection strategy template with the corresponding protection strength, extracts the core attributes contained in the protection strategy template such as control dimensions, constraint scope, and execution strength, and uses standardized vector encoding to perform structured transformation and unified arrangement of the attribute information to generate a strategy selection vector.

[0140] Among them, the protection strategy template is a pre-set standardized protection execution framework, and different templates correspond to different scheduling constraint rules.

[0141] (3) Select the vector according to the strategy to execute the instruction freeze operation and obtain the freeze state record.

[0142] For example, using a pre-defined strategy selection vector as the basis for execution, the distributed scheduling control terminal initiates a permission locking mechanism for global scheduling instructions, restricting the continuous issuance and dynamic changes of high-frequency scheduling instructions. The distributed scheduling control terminal suspends task migration operations between heterogeneous computing power nodes, constrains frequent adjustments to real-time resource quotas, fixes the temporary running state of buffer configuration parameters, fully records key operational information such as instruction freeze start time, locked permission scope, and restricted operation types, continuously retains all operational logs during the control phase, and integrates all recorded content to complete standardized archiving, resulting in a frozen state record.

[0143] Among them, the instruction freeze operation is a temporary control measure of the scheduling system, which can effectively prevent frequent policy changes caused by drastic fluctuations.

[0144] (4) Generate historical state rollback points based on the frozen state records to obtain the rollback baseline snapshot.

[0145] For example, by combining the fully archived freeze status records, the distributed scheduling control terminal locks the key time node triggered by the instruction freeze, and collects the load data of heterogeneous computing power nodes, the time-series operation data of multi-path inference streams, the transmission status data of the entire network link, and the existing scheduling configuration data at that moment. The distributed scheduling control terminal encrypts, solidifies, and stores the multi-dimensional operation data in a fixed location, delineates historical state rollback nodes that can be traced back and restored at any time, uniformly encapsulates the full-domain operation snapshot data, retains the baseline configuration content for stable system operation, and obtains the rollback baseline snapshot.

[0146] Among them, the historical state rollback point is a restoration node reserved by the system, which can realize the rapid rollback of the running state when the scheduling anomaly intensifies.

[0147] (5) Combine the over-limit level identifier, strategy selection vector and rollback baseline snapshot to generate conservative scheduling flag.

[0148] For example, following the system's conservative scheduling identifier generation specifications, the distributed scheduling control terminal sequentially aggregates three core information categories: over-limit level identifier, strategy selection vector, and rollback baseline snapshot, verifying the completeness and logical consistency of each type of information. The distributed scheduling control terminal unifies the encapsulation format of various data categories, calibrates the correlation logic between different data modules, completes the integrated arrangement of multiple types of protection information, integrates all control elements of anomaly level, protection strategy, and rollback baseline, and generates conservative scheduling flag bits.

[0149] Among them, the conservative scheduling flag is a dedicated status identifier for the stable protection mode of the scheduling system, which can provide a mandatory constraint basis for subsequent closed-loop scheduling decisions.

[0150] In one embodiment, based on the jitter component confidence interval and the attenuation component trend coefficient, incremental time alignment is performed on multiple inference streams to generate a multi-stream time offset table, including:

[0151] (1) Perform baseline anchor point selection on the preset flow priority label to obtain the baseline anchor point inference flow.

[0152] For example, relying on a preset flow priority tagging system, the distributed scheduling and control terminal fully reads the preset flow priority tags associated with each inference flow and extracts core attributes included in the tags, such as business importance level, data transmission stability rating, and inference latency sensitivity threshold. The distributed scheduling and control terminal performs standardized and quantified processing on various attributes, obtains the comprehensive priority score of each inference flow through weighted summation, and then performs a secondary verification by combining the historical time-series stability data of each inference flow, selecting the inference flow with the highest comprehensive priority and the smallest time-series fluctuation to obtain the benchmark anchor inference flow, where the formula is as follows:

[0153]

[0154] In the formula, Representing the The comprehensive priority score of the inference flow is used to measure the suitability of the inference flow as a benchmark anchor point; Weighting coefficients representing the level of business importance. The weighting coefficients representing the data transmission stability rating The weight coefficients representing the inference latency sensitivity threshold are all system-preset constants and satisfy the following conditions: ; Representing the Quantification of the business importance level of the inference flow; Representing the The quantitative value of data transmission stability rating for inference streams; Representing the The inference latency sensitivity threshold quantization value of the inference flow.

[0155] Among them, the preset flow priority label system is a set of attribute identifiers pre-configured for each inference flow, used to distinguish the priority and adaptation characteristics of the inference flow in the business scenario; the benchmark anchor inference flow is a reference benchmark for time alignment, providing a time dimension calibration basis for other inference flows.

[0156] (2) Establish a benchmark virtual time axis based on the benchmark anchor point inference flow.

[0157] For example, based on the temporal characteristics of the benchmark anchor inference stream, the distributed scheduling and control terminal extracts the original timestamp information of all data points in the benchmark anchor inference stream, performs format standardization processing on the original timestamps, and eliminates the time format differences caused by different data acquisition devices. Based on the standardized timestamp sequence, the distributed scheduling and control terminal uses linear interpolation to supplement the missing nodes in the time series, generating a continuous and uniform time reference frame. Combining the business attributes such as the inference task cycle and data sampling frequency of the benchmark anchor inference stream, the terminal adjusts the scale precision and granularity of the time axis to establish a benchmark virtual time axis, where the formula is as follows:

[0158]

[0159] In the formula, Represents the first virtual timeline on the baseline The time value of each scale division; A standardized timestamp representing the earliest data point in the baseline anchor inference stream; The scale index representing the baseline virtual time axis; The scale interval representing the baseline virtual time axis is determined by a combination of the data sampling frequency of the baseline anchor inference flow and the inference task cycle.

[0160] Among them, the benchmark virtual time axis is a virtual time reference system built based on the benchmark anchor inference stream, providing a unified time comparison standard for multiple inference streams; the linear interpolation method is a numerical calculation method used to supplement missing nodes in the time series, which can ensure the continuity and uniformity of the time axis.

[0161] (3) When any inference stream generates a new data point, the penalty weight of the attenuation component trend coefficient is adjusted to generate dynamic regularization weight.

[0162] For example, the distributed scheduling control terminal monitors the data generation status of each inference stream in real time. When it detects that any inference stream generates a new data point, it immediately extracts the attenuation component trend coefficient corresponding to that inference stream and analyzes the time-series attenuation rate and cumulative effect reflected by the attenuation component trend coefficient.

[0163] The distributed scheduling and control terminal dynamically adjusts the penalty weight of the attenuation component trend coefficient according to the time deviation direction between the new data point and the reference virtual time axis. If the new data point lags behind the reference time axis, the penalty weight of the attenuation trend is increased; if the new data point is ahead of the reference time axis, the penalty weight is appropriately reduced. The impact of time attenuation on the alignment result is balanced by adjusting the weight, and dynamic regularization weight is generated.

[0164] Among them, the dynamic regularization weight is a weight parameter that is dynamically adjusted according to the temporal decay characteristics and time deviation, which can adaptively optimize the path selection for temporal alignment.

[0165] (4) The path boundary is defined based on the confidence interval of the jitter component to obtain the restricted path space.

[0166] For example, the distributed scheduling control terminal retrieves the acquired jitter component confidence intervals, analyzes the upper and lower boundary thresholds of the confidence intervals, and uses these thresholds as constraints for the time-series alignment paths. The distributed scheduling control terminal combines historical time-series offset data from multiple inference streams to analyze the reasonable range of time-series fluctuations. Based on the boundary thresholds of the jitter component confidence intervals, it further limits the upper and lower limits of the time-series offset of each inference stream relative to the baseline virtual time axis, eliminates abnormal alignment paths exceeding the confidence intervals, and constructs a set of paths that conform to the jitter characteristic constraints, resulting in a constrained path space, where the formula is as follows:

[0167]

[0168] In the formula, Represents a restricted path space; Represents a single timing alignment path; The lower boundary value representing the confidence interval of the jitter component; This represents the upper boundary value of the confidence interval for the jitter component; This represents the timing offset corresponding to a single timing matching path.

[0169] Among them, the constrained path space is a set of time-series alignment paths that meet the jitter feature constraints, which can filter out abnormal paths to ensure alignment accuracy; the boundary threshold of the jitter component confidence interval is a constraint standard determined based on the temporal fluctuation characteristics of the inference stream, used to define a reasonable range of temporal offset.

[0170] (5) Perform incremental path calculation on the dynamic regularized weights and the restricted path space to obtain the path cost matrix.

[0171] For example, combining the generated dynamic warp weights and the constructed constrained path space, the distributed scheduling control terminal employs an incremental dynamic time warp algorithm, calculating only the local paths corresponding to newly generated data points, without needing to retraceive the entire path space. The distributed scheduling control terminal uses the dynamic warp weights as adjustment factors for path costs, calculating the cost of each local path in the constrained path space. The cost includes multiple dimensions such as time deviation cost and attenuation adaptation cost. A weighted fusion is used to obtain the comprehensive cost of each path. The comprehensive costs of all paths are then warped and arranged in matrix form to obtain the path cost matrix, where the formula is as follows:

[0172]

[0173] In the formula, The path cost matrix represents the first... Line number The comprehensive value of the series; Represents dynamic regularization weights; Representing the Road inference flow data points and reference time Timing deviation of the scale; The coefficient representing the attenuation component trend; This represents the temporal span parameter of the corresponding matching path.

[0174] Among them, the incremental dynamic time warping algorithm is an efficient algorithm that only calculates the local path corresponding to the newly added data, which can reduce the computational complexity of time alignment; the path cost matrix is ​​a matrix structure used to record the comprehensive cost of each alignment path, providing data support for the selection of the optimal alignment path.

[0175] (6) Calculate the time offset based on the path cost matrix and the baseline virtual time axis to generate a multi-stream time offset table.

[0176] For example, the distributed scheduling and control terminal traverses all elements in the path cost matrix and selects the path corresponding to the element with the minimum overall cost. This path is the optimal time-series alignment path for the current new data point. The distributed scheduling and control terminal extracts the time-series offset corresponding to the optimal alignment path and associates and stores this offset with the identification information, data point index, and baseline virtual time axis scale information of the corresponding inference stream. The distributed scheduling and control terminal repeats the above process for all new data points in all inference streams, continuously updating the time-series offset records of each inference stream, and integrating and arranging all time-series offset data in a unified format to generate a multi-stream time-series offset table.

[0177] Among them, the multi-stream timing offset table is a standardized table that records the timing offset data of each inference stream relative to the baseline virtual time axis. It includes key information such as inference stream identifier, data point index, baseline time scale, and timing offset, providing a basis for timing differences for subsequent closed-loop scheduling decisions.

[0178] In one embodiment, after executing the node-level scheduling instruction set, new latency data is collected, and the parameters of the latency component separation model are updated, including:

[0179] (1) Execute the node-level scheduling instruction set and generate instruction execution status records.

[0180] For example, the distributed scheduling and control terminal sends the node-level scheduling instruction set to the corresponding heterogeneous computing power nodes one by one according to the standardized instruction transmission protocol, ensuring that there is no packet loss or error during the instruction transmission process. The distributed scheduling and control terminal monitors the execution progress and feedback results of each instruction in real time, records the instruction issuance time, target node response time, execution start time, execution completion time, and execution status, and collects related data such as changes in node resource usage, load fluctuations, and network transmission latency during instruction execution. All monitoring information is classified and organized according to instruction identifiers to form a structured record document, generating an instruction execution status record.

[0181] Among them, the instruction execution status record is a structured data document that reflects the execution status of the node-level scheduling instruction set. It includes core modules such as basic instruction information, execution timing data, node running status data, and execution result identifiers, providing an execution background basis for subsequent data collection and model updates.

[0182] (2) Collect new delay data based on instruction execution status records to obtain a new delay dataset.

[0183] For example, based on the instruction execution status record, the distributed scheduling and control terminal initiates a full-domain latency data collection process after detecting that all instructions have been executed or that a preset collection node has been reached. The distributed scheduling and control terminal collects multi-dimensional latency data for each heterogeneous computing power node, including transmission latency (time spent on data transmission between nodes), processing latency (time spent on inference operations within a node), and caching latency (time spent waiting for data in the buffer), ensuring that the collected latency data corresponds one-to-one with the instruction execution scenario. The distributed scheduling and control terminal performs outlier filtering (removing extreme data caused by network jitter or hardware failure) and format standardization on the collected raw latency data, classifying and integrating it according to node identifier, task type, and time series to obtain a new latency dataset.

[0184] Among them, the new latency dataset is a multi-dimensional, structured latency data collection acquired after instruction execution, covering latency information of the entire process of data transmission, task processing, and buffer waiting, providing the latest measured data support for the latency component separation model.

[0185] (3) Identify the migration instruction nodes in the instruction execution status record, reset the attenuation baseline of the relevant nodes, and generate the reset attenuation component sequence.

[0186] For example, the distributed scheduling control terminal parses the instruction type field in the instruction execution status record, filters out scheduling instructions identified as "task migration," and extracts the target node identifier (i.e., the migration instruction node) corresponding to this type of instruction. The distributed scheduling control terminal retrieves the historical attenuation baseline (a benchmark value reflecting the long-term temporal attenuation characteristics of the node) of the migration instruction node before instruction execution, and, in conjunction with the execution content of the migration instruction (such as the amount of task load migration and the proportion of resource reallocation), resets and calibrates the historical attenuation baseline to eliminate the impact of abrupt changes in attenuation characteristics caused by task migration. Based on the reset attenuation baseline and combined with the latency evolution trend of the node in the new latency data, the distributed scheduling control terminal generates a continuous and smooth attenuation characteristic sequence, obtaining the reset attenuation component sequence.

[0187] Among them, the migration instruction node is a heterogeneous computing power node that receives scheduling instructions of the "task migration" type. Its attenuation characteristics will change abruptly due to changes in task load. The attenuation baseline is a benchmark parameter that characterizes the temporal attenuation law during long-term stable operation of the node, providing a basic reference for attenuation component calculation. The reset attenuation component sequence is an attenuation characteristic data sequence adapted to the node's operating state after task migration, ensuring that the attenuation components are consistent with the actual operating state of the node.

[0188] (4) Identify the buffer injection nodes in the instruction execution status record, perform time delay compensation on the relevant nodes, and generate the compensated time delay observation value.

[0189] For example, the distributed scheduling control terminal continues to parse the instruction execution status record, identifies the scheduling instruction marked "buffer configuration adjustment" based on the instruction type field, and extracts the target node identifier (i.e., the buffer injection node) corresponding to this type of instruction. The distributed scheduling control terminal analyzes the changes in the buffer configuration parameters of the buffer injection node (such as buffer capacity adjustment, refresh cycle change), calculates the impact of the configuration change on latency (i.e., the latency deviation introduced by the buffer). If the buffer capacity increases, the latency deviation is negative (the buffer waiting time decreases); if the buffer capacity decreases, the latency deviation is positive (the buffer waiting time increases). The distributed scheduling control terminal subtracts the corresponding latency deviation from the original latency observation value of this node in the new latency dataset, completes the latency compensation calculation, and generates the compensated latency observation value.

[0190] Among them, the buffer injection node is a heterogeneous computing power node that receives the "buffer configuration adjustment" type scheduling instruction, and its cache latency will change due to configuration changes; latency compensation is the operation process to eliminate the interference of buffer configuration adjustment on the original latency observation value, and ensure that the latency data can truly reflect the timing characteristics of the node.

[0191] (5) Update the time delay component separation model using the new time delay dataset, the reset attenuation component sequence and the compensated time delay observations to generate the updated time delay component separation model.

[0192] For example, the distributed scheduling and control terminal uses the new delay dataset, the reset attenuation component sequence, and the compensated delay observations as joint update data sources, inputting them into the delay component separation model. The distributed scheduling and control terminal calls the model's built-in parameter iteration algorithm, using the compensated delay observations as the target output, the multi-dimensional delay data in the new delay dataset as input features, and the reset attenuation component sequence as constraints, to calculate the fitting error between the model's current parameters and the measured data. The distributed scheduling and control terminal continuously adjusts the model's core parameters, such as weight parameters, attenuation component calculation coefficients, and jitter component fitting parameters, using gradient descent until the fitting error is less than a preset threshold or the maximum number of iterations is reached, completing the optimization and update of the model parameters and generating the updated delay component separation model.

[0193] Among them, the delay component separation model is an algorithm model used to separate core components such as jitter component and attenuation component in delay data. Its core parameters include weight coefficient, fitting coefficient, and constraint threshold. Parameter updates can improve the model's adaptability to the timing characteristics of the current heterogeneous computing power environment. The updated delay component separation model is an optimized model calibrated with measured data, which can more accurately complete subsequent delay component separation tasks.

[0194] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0195] In one embodiment, such as Figure 3 As shown, this application also provides a heterogeneous computing power inference scheduling system 300 for distributed software architecture, the system 300 including:

[0196] The delay separation module 301 is used to separate the real-time inference delay sequence reported by the node through the delay component separation model to obtain the jitter component confidence interval and the attenuation component trend coefficient.

[0197] The timing alignment module 302 is used to perform incremental timing alignment on multiple inference streams based on the confidence interval of jitter components and the trend coefficient of attenuation components, and generate a multi-stream timing offset table.

[0198] The scheduling decision module 303 is used to implement closed-loop scheduling decisions based on the multi-stream time offset table to obtain a node-level scheduling instruction set.

[0199] The feedback update module 304 is used to collect new delay data after executing the node-level scheduling instruction set and update the parameters of the delay component separation model.

[0200] Specifically, the distributed scheduling and control terminal includes a delay separation module 301, a timing alignment module 302, a scheduling decision module 303, and a feedback update module 304.

[0201] The latency separation module 301 receives the real-time inference latency sequence reported by heterogeneous computing nodes, identifies and eliminates abnormal data caused by hardware failures and network interruptions, and unifies the data format and time granularity. Then, it calls the latency component separation model, uses a component decomposition algorithm to decompose the jitter component and attenuation component, performs statistical analysis on the jitter component to generate a jitter component confidence interval, and performs trend fitting on the attenuation component to generate an attenuation component trend coefficient.

[0202] Among them, the delay separation module 301 is responsible for the separation of delay data components. The real-time inference delay sequence is the continuous delay data reported by the node at a fixed period. The delay component separation model integrates data preprocessing and component identification functions.

[0203] The timing alignment module 302 takes the jitter component confidence interval and the attenuation component trend coefficient as input, selects the benchmark anchor point inference stream according to the stream priority rule, and constructs a benchmark virtual time axis based on its time characteristics. When the inference stream generates new data points, it adjusts the normalization weights in combination with the attenuation component trend coefficient, limits the alignment path range with the jitter component confidence interval, selects the optimal path through incremental path calculation, and integrates the timing offset data to generate a multi-stream timing offset table.

[0204] Among them, the timing alignment module 302 focuses on timing calibration of multi-path inference streams. The benchmark virtual time axis is a unified time reference frame, and incremental calculation can avoid repeated processing of historical data and improve alignment efficiency.

[0205] After acquiring the multi-stream time-series offset table, the scheduling decision module 303 aggregates the time-series offset data to calculate the inference stream synchronization deviation. It then quantifies the overall discrete state using the entropy method to generate a global synchronization error entropy. Based on this entropy, it obtains the resource quota adjustment amount, migration target node identifier, and buffer configuration parameters through proportional, integral, and derivative control transformations, respectively. It monitors the rate of change of the decision parameters to generate the control signal change rate; if the rate exceeds the limit, it activates stability protection to generate a conservative scheduling flag. Finally, it integrates and encapsulates these parameters to generate a node-level scheduling instruction set.

[0206] Among them, the scheduling decision module 303 is the core module for formulating scheduling strategies, the global synchronization error entropy is the global deviation basis for scheduling decisions, and the node-level scheduling instruction set is a standardized set of executable instructions.

[0207] The feedback update module 304 issues a node-level scheduling instruction set, monitors the instruction execution status, and records the issuance, response, and execution results, generating an instruction execution status record. After instruction execution, multi-dimensional new latency data is collected, outliers are filtered, and standardized to form a new latency dataset. The attenuation baseline of the migration instruction node is reset, the latency deviation of the injected node in the compensation buffer is compensated, and the relevant data is input into the latency component separation model to iteratively optimize the parameters, generating an updated latency component separation model.

[0208] Among them, the feedback update module 304 realizes the closed-loop optimization of scheduling. The new delay dataset reflects the latest operating status of the node, and the updated delay component separation model has better accuracy, providing data support for subsequent processes.

[0209] The scheduling decision module 303 is also used for:

[0210] Synchronization error aggregation is performed on the multi-stream timing offset table to obtain the global synchronization error entropy;

[0211] Based on the global synchronization error entropy, proportional control conversion is performed to generate resource quota adjustment amounts;

[0212] Integral control conversion is performed based on global synchronization error entropy to generate migration target node identifiers;

[0213] Differential control transformation is performed based on global synchronization error entropy to generate buffer configuration parameters;

[0214] The rate of change of control signals is obtained by monitoring the rate of change of resource quota adjustment, migration target node identifier, and buffer configuration parameters.

[0215] When the rate of change of the control signal exceeds the preset threshold, stability protection is executed, and a conservative scheduling flag is obtained.

[0216] Combine resource quota adjustment amounts, migration target node identifiers, buffer configuration parameters, and conservative scheduling flags to generate a node-level scheduling instruction set.

[0217] The scheduling decision module 303 is also used for:

[0218] The instantaneous gradient is obtained by performing time window differentiation on the global synchronization error entropy;

[0219] Jitter pattern recognition is performed based on instantaneously changing gradients to generate jitter pattern classification labels;

[0220] Based on the jitter pattern classification identifier index, a preset buffer strategy library is used to obtain a buffer strategy template;

[0221] The parameters of the buffer strategy template are adjusted based on the instantaneously changing gradient to generate a set of buffer parameters;

[0222] Combine the buffer strategy template and the set of buffer parameters to generate buffer configuration parameters.

[0223] The scheduling decision module 303 is also used for:

[0224] The rate of change of the control signal is compared with multiple thresholds to obtain the out-of-limit level indicator;

[0225] Based on the over-limit level identifier, select the protection strategy template and generate the strategy selection vector;

[0226] Based on the strategy, a vector is selected to execute the instruction freeze operation, and a freeze state record is obtained;

[0227] Historical state rollback points are generated based on the frozen state records to obtain the rollback baseline snapshot;

[0228] Combine the over-limit level identifier, strategy selection vector, and rollback baseline snapshot to generate a conservative scheduling flag.

[0229] Timing alignment module 302 is also used for:

[0230] Perform baseline anchor point selection on the preset flow priority label to obtain the baseline anchor point inference flow;

[0231] Establish a benchmark virtual timeline based on the benchmark anchor point inference flow;

[0232] When any inference stream generates a new data point, the penalized weight of the attenuation component trend coefficient is adjusted to generate a dynamic regularization weight.

[0233] The path boundary is defined based on the confidence interval of the jitter component to obtain the restricted path space;

[0234] Incremental path calculations are performed on the dynamically regularized weights and the constrained path space to obtain the path cost matrix;

[0235] Timing offset is calculated based on the path cost matrix and the baseline virtual time axis, generating a multi-stream timing offset table.

[0236] Feedback update module 304 is also used for:

[0237] Execute the node-level scheduling instruction set and generate instruction execution status records;

[0238] New latency data is collected based on instruction execution status records to obtain a new latency dataset.

[0239] Identify the migration instruction nodes in the instruction execution status record, reset the attenuation baseline of the relevant nodes, and generate the reset attenuation component sequence;

[0240] Identify the buffer injection nodes in the instruction execution status record, perform time delay compensation on the relevant nodes, and generate compensated time delay observations;

[0241] The time delay component separation model is updated using the new time delay dataset, the reset attenuation component sequence, and the compensated time delay observations, resulting in an updated time delay component separation model.

[0242] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0243] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0244] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0245] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A heterogeneous computing power inference scheduling method for distributed software architecture, characterized in that, The method includes: The real-time inference delay sequence reported by the node is separated into delay components by a delay component separation model to obtain the confidence interval of the jitter component and the trend coefficient of the attenuation component. Based on the jitter component confidence interval and the attenuation component trend coefficient, incremental time alignment is performed on the multi-stream inference streams to generate a multi-stream time offset table. Based on the multi-stream time offset table, a closed-loop scheduling decision is made to obtain a node-level scheduling instruction set. After executing the node-level scheduling instruction set, new latency data is collected, and the parameters of the latency component separation model are updated.

2. The heterogeneous computing power inference scheduling method for distributed software architecture according to claim 1, characterized in that, The closed-loop scheduling decision based on the multi-stream time offset table yields a node-level scheduling instruction set, including: The synchronization error is aggregated by the multi-stream timing offset table to obtain the global synchronization error entropy; Based on the global synchronization error entropy, a proportional control conversion is performed to generate a resource quota adjustment amount; Based on the global synchronization error entropy, integral control conversion is performed to generate the migration target node identifier; Based on the global synchronization error entropy, differential control transformation is performed to generate buffer configuration parameters; The rate of change of the resource quota adjustment, the migration target node identifier, and the buffer configuration parameters is monitored to obtain the rate of change of the control signal. When the rate of change of the control signal exceeds a preset threshold, stability protection is performed, and a conservative scheduling flag is obtained. The node-level scheduling instruction set is generated by combining the resource quota adjustment amount, the migration target node identifier, the buffer configuration parameters, and the conservative scheduling flag.

3. The heterogeneous computing power inference scheduling method for distributed software architecture according to claim 2, characterized in that, The differential control transformation based on the global synchronization error entropy, generating buffer configuration parameters, includes: The instantaneous gradient is obtained by performing time window differentiation on the global synchronization error entropy. Based on the instantaneous gradient change, jitter pattern recognition is performed to generate jitter pattern classification labels; Based on the jitter mode classification identifier index, a preset buffer strategy library is used to obtain a buffer strategy template; Based on the instantaneous gradient change, the parameters of the buffer strategy template are adjusted to generate a buffer parameter set; The buffer strategy template and the buffer parameter set are combined to generate the buffer configuration parameters.

4. The heterogeneous computing power inference scheduling method for distributed software architecture according to claim 2, characterized in that, When the rate of change of the control signal exceeds a preset threshold, stability protection is executed, and a conservative scheduling flag is obtained, including: The rate of change of the control signal is compared with multiple thresholds to obtain the out-of-limit level identifier; Based on the over-limit level identifier, a protection strategy template is selected, and a strategy selection vector is generated. According to the strategy, a vector is selected to execute the instruction freeze operation, and a freeze state record is obtained; Based on the frozen state record, a historical state rollback point is generated to obtain a rollback baseline snapshot; The conservative scheduling flag is generated by combining the over-limit level identifier, the strategy selection vector, and the rollback baseline snapshot.

5. The heterogeneous computing power inference scheduling method for distributed software architecture according to claim 1, characterized in that, The step of incrementally aligning the multi-stream inference streams based on the jitter component confidence interval and the attenuation component trend coefficient to generate a multi-stream timing offset table includes: Perform baseline anchor point selection on the preset flow priority label to obtain the baseline anchor point inference flow; A benchmark virtual timeline is established based on the benchmark anchor point inference flow; When any inference stream generates a new data point, the penalty weight of the attenuation component trend coefficient is adjusted to generate a dynamic regularization weight. Based on the confidence interval of the jitter components, the path boundary is defined to obtain the restricted path space; Incremental path calculations are performed on the dynamic regularization weights and the constrained path space to obtain the path cost matrix; Based on the path cost matrix and the baseline virtual time axis, the time series offset is calculated to generate the multi-stream time series offset table.

6. The heterogeneous computing power inference scheduling method for distributed software architecture according to claim 1, characterized in that, After executing the node-level scheduling instruction set, new latency data is collected, and the parameters of the latency component separation model are updated, including: Execute the node-level scheduling instruction set and generate an instruction execution status record; New latency data is collected based on the instruction execution status record to obtain a new latency dataset. Identify the migration instruction nodes in the instruction execution status record, reset the attenuation baseline of the relevant nodes, and generate a reset attenuation component sequence; Identify the buffer injection nodes in the instruction execution status record, perform time delay compensation on the relevant nodes, and generate compensated time delay observations; The time delay component separation model is updated using the new time delay dataset, the reset attenuation component sequence, and the compensated time delay observations to generate the updated time delay component separation model.

7. A heterogeneous computing power inference scheduling system for distributed software architecture, characterized in that, The system includes: The delay separation module is used to separate the real-time inference delay sequence reported by the node through the delay component separation model, and obtain the confidence interval of the jitter component and the trend coefficient of the attenuation component. The timing alignment module is used to perform incremental timing alignment on multiple inference streams based on the jitter component confidence interval and the attenuation component trend coefficient, and generate a multi-stream timing offset table. The scheduling decision module is used to implement closed-loop scheduling decisions based on the multi-stream time offset table to obtain a node-level scheduling instruction set; The feedback update module is used to collect new latency data after executing the node-level scheduling instruction set and update the parameters of the latency component separation model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the heterogeneous computing power inference scheduling method for distributed software architecture as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the heterogeneous computing power inference scheduling method for distributed software architecture as described in any one of claims 1 to 6.

Images