Job evolution management method integrating random queuing semantics and variational bayesian ensemble learning

Abstract

The application discloses a job evolution management and control method fusing random queuing semantics and variational Bayesian ensemble learning. The method firstly converts discrete queuing reason codes of jobs into theoretical waiting time features with physical meanings by using M / M / c / K queuing theory, and constructs a physical enhanced feature manifold space by exponential smoothing weighting; then a Bayesian ensemble regression model enhanced by variational inference is used to model the probability distribution of job completion time ETA, and the mean, standard deviation and dynamic confidence interval are output; then explicit statistical hard threshold judgment and isolated forest implicit anomaly detection are executed in parallel, and a double-track risk identification result is generated by using a correction factor that automatically translates with uncertainty; finally, the analytic hierarchy process is called to output the optimal disposal strategy, and the model parameters, prior hyperparameters and risk discrimination threshold are updated online based on the completion residual through the stochastic gradient descent operator. The application realizes the probabilistic prediction and uncertainty quantification of ETA.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of distributed computing, cluster scheduling, and probabilistic machine learning, and in particular to a job evolution management method based on stochastic queuing semantics and variational Bayesian ensemble learning. Background Technology

[0002] With the continuous expansion of cloud computing, big data processing, and artificial intelligence training tasks, the job scheduling efficiency of distributed clusters has become a key factor affecting overall computing performance. Accurate prediction of the Estimated Time of Arrival (ETA) is a prerequisite for efficient scheduling. However, existing monitoring and prediction systems for heterogeneous clusters generally suffer from the following three technical limitations when facing highly dynamic, high-load, and inconsistent node configurations in real-world production environments.

[0003] First, shallow statistical features are insufficient to characterize complex queuing dynamics. Most existing prediction methods only use shallow statistical features such as average load and historical averages for regression modeling, neglecting the actual physical processes jobs undergo while queuing in the resource pool, such as resource fragmentation, preemption by high-priority tasks, and blocking by cross-task dependency chains. These factors can cause non-linear waiting delays, leading to significant deviations in the predictions of traditional models whenever resource contention occurs in the cluster, making them unsuitable for adapting to real-world cluster congestion.

[0004] Second, point estimates cannot quantify the reliability of predictions. Mainstream deep learning or regression models only output a specific numerical value (i.e., a point estimate), without providing the scheduler with a confidence boundary for that prediction. However, distributed systems are inherently filled with random noise (such as network jitter, node failures, and resource contention), and models have inherent limitations in their understanding of environmental conditions. A single number cannot reflect these sources of uncertainty, causing downstream schedulers to blindly trust predictions when faced with highly volatile tasks, unable to perform differentiated risk mitigation operations (such as resource reservation, task degradation, or proactive migration) based on confidence levels.

[0005] Third, risk management mechanisms are outdated and lack closed-loop adaptive capabilities. Current risk monitoring largely relies on a set of hard-coded static thresholds, triggering alarms only when timeouts or failures occur, failing to identify progressive risks that "have not yet crossed the boundary but have deviated from the normal trajectory." Furthermore, existing systems generally lack closed-loop calibration capabilities; model parameters are not updated online with actual execution feedback, leading to a continuous decline in prediction accuracy after long-term operation, and the effectiveness of scheduling decisions diminishes with environmental drift.

[0006] In summary, there is an urgent need for an operation control method that can integrate physical modeling and probabilistic machine learning, possess uncertainty quantification capabilities, and a closed-loop evolution mechanism to overcome the aforementioned limitations. Summary of the Invention

[0007] The purpose of this invention is to provide a job evolution control method that integrates random queuing semantics and variational Bayesian ensemble learning, in order to solve at least some of the problems existing in the prior art.

[0008] This invention provides a job evolution management method based on stochastic queuing semantics and variational Bayesian ensemble learning, comprising: Based on the discrete semantic identifiers and arrival features of the job queuing stage, the steady-state distribution probability of the system is calculated using the M / M / c / K queuing dynamics theoretical model, and the theoretical waiting time features with physical meaning are reduced to generate, thus constructing a physically enhanced feature manifold space. By combining physical enhancement features with basic job attribute data to form a complete input vector, an ensemble regression model enhanced by variational inference algorithm is used to model the probability distribution of job completion time, and output the mean, standard deviation and dynamic confidence interval. Based on the dynamic confidence interval and the load characteristics during operation, explicit statistical hard threshold judgment and isolated forest hidden abnormal path detection are performed in parallel. The prediction uncertainty correction factor is used to dynamically shift the early warning limit to generate dual-track risk identification results. The analytic hierarchy process (AHP) is used to construct a multi-criteria decision matrix based on the dual-track risk identification results, and the optimal handling strategy is output and executed. After the operation is completed, the model parameters, prior hyperparameters, and risk discrimination thresholds are updated online using an incremental learning operator based on the residual between the actual completion time and the mean of the predicted probability distribution.

[0009] Furthermore, the construction of the physically enhanced feature manifold space includes: Collect dynamic configuration data of the resource pool at the time of job submission to obtain the number of active service nodes, average job arrival rate, and average service rate per node; The system service intensity is calculated by dividing the average arrival rate of the job by the product of the number of active service nodes and the average service rate of a single node, and is used as a scalar indicator to characterize the busyness of the system. Substitute the system service intensity and the number of active service nodes into the steady-state probability series summation formula of the M / M / c / K queuing dynamics theoretical model to calculate the probability that all nodes are simultaneously in an idle state, which is taken as the system idle probability. Based on the system idle probability, the system service intensity, and the number of active service nodes, the average queue length calculation formula of the M / M / c / K queuing dynamics theoretical model is used to calculate the average number of queued tasks in the system, which is then used as the queue backlog length. Based on the queue backlog length and the average arrival rate of the job, the theoretical waiting time is obtained by dividing the two according to Little's law. The theoretical waiting time and the historical observation mean are weighted and fused with a first-order exponential smoothing coefficient to obtain a continuous real-valued physical enhancement feature.

[0010] Furthermore, the probability distribution modeling of the job completion time (ETA) and the output of the mean, standard deviation, and dynamic confidence interval include: The physical enhancement features and the basic attribute data of the operation are concatenated using multi-source features and vectorized encoding to obtain a complete input feature vector. Based on the input feature vector and the predicted output of the ensemble regression model, variational inference calculation to maximize the lower bound of evidence is performed to obtain the cognitive uncertainty value of the model for heterogeneous noise environment, which is internalized into the model parameters. The model parameters are sampled or analyzed using a posterior distribution to obtain the mean and standard deviation of the distribution. The dynamic confidence interval is obtained by performing interval estimation based on the mean, the standard deviation, and the quantiles corresponding to the preset confidence level.

[0011] Furthermore, the parallel execution of explicit statistical hard threshold assessment and isolated forest latent anomaly path detection generates a dual-track risk identification result, including: Construct a correction factor that decays exponentially with increasing standard deviation, and use the correction factor to dynamically shift the base SLA threshold to obtain a dynamic SLA threshold. Compare the upper limit of the dynamic confidence interval with the dynamic SLA threshold to obtain the explicit timeout risk assessment result and the excess amount. Based on the time-series data of CPU utilization, memory usage and IO throughput during the job operation phase, sliding window tensor sampling and isolated forest inference are performed to obtain latent anomaly scores that characterize the degree to which the operation trajectory deviates from the normal distribution. The explicit timeout risk assessment result and the implicit anomaly score are weighted and fused according to risk level to obtain the dual-track risk identification result.

[0012] Furthermore, it also includes adaptive updates to the correction factor: The standard deviation of the collected prediction output and the scene calibration reference scale; Divide the scene calibration reference scale by the standard deviation, take the negative value and use it as the exponential part of the exponential function for mapping calculation, to obtain a dynamic correction factor that automatically shifts with the prediction uncertainty, which is used to adjust the sensitivity limit of the statistical hard threshold in real time.

[0013] Furthermore, the invocation of the analytic hierarchy process (AHP) to construct a multi-criteria decision matrix and output the optimal handling strategy includes: Construct a three-dimensional judgment matrix that includes the success rate of handling, system execution cost and intervention timeliness, solve for the maximum eigenvalue of the matrix, and calculate the consistency ratio; After verifying that the consistency ratio is less than a preset threshold, output the strategy weight vector and select the processing strategy with the highest weight as the optimal processing strategy. The optimal handling strategy is sent to the underlying controller of the cluster for execution via the state anchoring interface.

[0014] Furthermore, the online updating of model parameters, prior hyperparameters, and risk discrimination thresholds using incremental learning operators includes: Based on the actual completion time after the task is completed and the mean of the predicted probability distribution, residual calculation is performed to obtain the scalar residual; Based on the scalar residual, a stochastic gradient descent backpropagation operation is performed to update three sets of variables: the attention weights of the ensemble regression model, the prior hyperparameters of the KL divergence term, and the warning limits corresponding to the correction factors. The updated prediction model parameters and risk assessment parameters for the next cycle are obtained.

[0015] Furthermore, the ensemble regression model employs an extreme random tree architecture enhanced by the Bayesian optimization algorithm: it searches for the globally optimal deep structure and feature sampling strategy of the extreme random tree in the continuous hyperparameter manifold space through a Gaussian process surrogate model, and outputs a posterior distribution with KL divergence constraints under the variational inference framework.

[0016] Furthermore, it also includes establishing a snapshot management mechanism for heterogeneous task load distribution: while performing online updates of model parameters, it saves the current ensemble regression model parameter topology sequence, KL divergence prior hyperparameter set, and risk discrimination threshold vector at a preset period to support lossless rollback of state based on snapshots during drastic environmental migrations.

[0017] Furthermore, the method is deployed as a whole in the core scheduling link layer of the distributed computing system: it performs real-time bidirectional data interaction with the resource controller and monitoring probe at the bottom layer of the resource pool through a high-performance state anchoring interface, receives job queuing semantic tags and runtime monitoring indicators, and issues the optimal handling strategy determined by the analytic hierarchy process.

[0018] Compared with existing technologies, this invention has at least the following beneficial effects: by integrating queuing theory physical priors with variational Bayesian ensemble learning, it achieves probabilistic prediction and uncertainty quantification of job completion time. Furthermore, by combining dual-track risk identification and residual-driven closed-loop incremental evolution, it improves the prediction accuracy of job scheduling, the initiative of risk warning, and the long-term adaptive capability of the system in heterogeneous cluster environments. Attached Figure Description

[0019] Figure 1This is a flowchart of the overall solution of the present invention.

[0020] Figure 2 This is a flowchart of the process for constructing a physically enhanced feature manifold space according to the present invention.

[0021] Figure 3 This is a flowchart of the output mean, standard deviation, and dynamic confidence interval of this invention.

[0022] Figure 4 This is a flowchart illustrating how the present invention generates dual-track risk identification results.

[0023] Figure 5 This is a flowchart of the optimal handling strategy output by the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] like Figure 1 As shown, this embodiment describes in detail the data processing flow of a job evolution control method that integrates stochastic queuing semantics and variational Bayesian ensemble learning, specifically including: S1: Based on the discrete semantic identifiers and arrival features of the job queuing stage, the steady-state distribution probability of the system is calculated using the M / M / c / K queuing dynamics theoretical model, and the theoretical waiting time features with physical meaning are generated by reduction, thus constructing a physically enhanced feature manifold space.

[0026] In this embodiment, when a computing job is submitted to the distributed resource pool, the discrete semantic identifier generated during the queuing phase and the dynamic parameters of the resource pool at the arrival time are first captured. Specifically, the discrete semantic identifier includes enumerated reason codes such as "preempted by a high-priority task", "blocked by dependent data", and "insufficient resource quota"; the arrival parameters include the number of currently active service nodes, the average arrival rate of the job, and the average service rate of a single node.

[0027] According to one aspect of this application, in order to transform these discrete, qualitative queuing information into continuous numerical features with clear physical meaning, the system service intensity is first calculated using arrival parameters. This indicator directly reflects the current load saturation level of the resource pool. Further, the service intensity and the number of service nodes are substituted into the steady-state distribution probability solution framework of the M / M / c / K queuing dynamics theory model to calculate the probability that all service nodes are simultaneously idle. Based on this, the average queue length calculation formula in queuing theory is used to calculate the average number of tasks waiting in the queue under the current load conditions.

[0028] Subsequently, according to Little's law, the average queue length is divided by the average arrival rate of the job to obtain the theoretical waiting time, representing the expected waiting time of the job under the ideal queuing model. In a preferred embodiment, to balance the physical model and real observations, the historical observation mean corresponding to the queuing reason code is further collected, and a first-order exponential smoothing coefficient is introduced to weight and fuse the theoretical waiting time with the historical observation mean. The fused value is the final physical augmentation feature. This feature replaces the original discrete reason code and becomes a key input dimension of the subsequent probabilistic prediction model, thereby constructing the feature manifold space of physical augmentation.

[0029] S2: During operation, the physical enhancement features and basic job attributes are combined to form a complete input vector. An ensemble regression model enhanced by variational inference algorithm is used to model the probability distribution of job completion time (ETA, Estimated Time of Arrival), and output the mean, standard deviation and dynamic confidence interval.

[0030] In this scheme, after completing the physical augmentation mapping of queuing semantics, the probabilistic prediction stage of job completion time begins. Specifically, the physical augmentation features obtained in the previous stage are concatenated with the job's basic attribute data. Basic attributes include, but are not limited to: the job's resource profile (estimated CPU core count, memory size, GPU requirements), dependency graph (upstream task completion status), user quota limits, and job type labels. For example, these heterogeneous data, after vectorization encoding and normalization, are integrated into a complete high-dimensional input feature vector.

[0031] According to a further improvement of this embodiment, an ensemble regression model enhanced with a variational inference algorithm is used to process the input vector. Preferably, this ensemble regression model uses an extreme random tree as the base learner and embeds a Bayesian inference framework. During model inference, instead of outputting a single deterministic value, the completion time of the task is modeled as a random variable following a certain probability distribution (e.g., a Gaussian distribution). To achieve this probabilistic output, the model approximates the true posterior distribution of the parameters by maximizing the lower bound of evidence. The KL divergence term in the lower bound of evidence is used to measure the model's cognitive uncertainty regarding the current heterogeneous noisy environment.

[0032] In one alternative implementation, for the current job to be predicted, the model outputs two key parameters from the aforementioned approximate posterior distribution: the mean μ and the standard deviation σ. The mean μ represents the most likely completion time according to the model; the standard deviation σ quantifies the model's confidence in the prediction: the larger σ is, the less reliable the prediction is, and the more likely it is to be affected by factors such as resource competition or random disturbances.

[0033] Furthermore, to provide the downstream scheduler with an operable decision boundary, a dynamic confidence interval is constructed based on the mean μ, standard deviation σ, and the quantiles corresponding to the preset confidence level. Specifically, the lower limit of the confidence interval is the mean minus the product of the quantiles and the standard deviation, and the upper limit is the mean plus the product of the quantiles and the standard deviation. This interval directly defines a probabilistic envelope range for the job completion time, and the interval width is a quantitative indicator of the uncertainty of the current job, allowing downstream modules to judge the reliability of the prediction results.

[0034] S3: Based on the dynamic confidence interval of the output and the load characteristics during operation, explicit statistical hard threshold judgment and isolated forest hidden anomaly path detection are performed in parallel. The early warning limit is dynamically shifted using the prediction uncertainty correction factor to generate dual-track risk identification results.

[0035] After obtaining the probability distribution and dynamic confidence interval of the ETA, this scheme further performs real-time risk perception on the execution process of the operation. According to one aspect of this application, risk identification is divided into two parallel tracks: the explicit track is responsible for judging whether there is a risk of timeout default, while the implicit track is responsible for capturing abnormal deviation patterns in the operation trajectory.

[0036] For explicit timeouts, the upper limit of the dynamic confidence interval output from the previous stage is first taken as the worst-case completion time. Simultaneously, a basic SLA (Service Level Agreement) threshold is preset based on the business scenario. Considering that the standard deviation given by the prediction model reflects the uncertainty of the current environment, the static threshold cannot adapt to such fluctuations; therefore, a correction factor that adaptively changes with the standard deviation is introduced. Preferably, the construction of this correction factor follows an exponential weighting mapping rule: when the prediction uncertainty is large, the correction factor tends to a small positive number, thus shifting the static threshold in a more lenient direction; conversely, when the uncertainty is small, the correction factor tends to 1, and the threshold is close to the original setting. Subsequently, the upper limit of the confidence interval is compared with the dynamically shifted SLA threshold: if the former is greater than the latter, an explicit timeout risk is determined, and the excess amount is calculated for subsequent decision-making modules to assess the severity of the risk.

[0037] In parallel with explicit path detection, the hidden anomaly path detection employs the Isolation Forest algorithm. For example, during job execution, time-series load features such as CPU utilization, memory usage, and I / O (Input / Output) throughput (including read / write rates and latency) are continuously collected. These features are tensor-sampled in a sliding window format, with each sampling window forming a multi-dimensional feature vector. These vectors are input into a pre-trained Isolation Forest model, which constructs multiple isolated trees by randomly partitioning the feature space. For the current job's load feature vector, the path length from the root node to a leaf node in each tree is calculated, thus determining the average path depth. Simultaneously, the average structure depth of the forest is calculated based on the subsample size. Finally, the average path depth and average structure depth are compared and mapped to an anomaly score between 0 and 1: a score closer to 1 indicates a sample is more likely to be isolated (i.e., a higher degree of anomaly), while a score closer to 0 indicates a sample tends towards a normal pattern.

[0038] In a preferred embodiment, the timeout risk assessment result (including the excess amount) from the explicit track output is weighted and fused with the anomaly score from the implicit track output. The fused result is the dual-track risk identification result, which will be sent to the next stage decision-making module to trigger specific governance strategies.

[0039] S4: The analytic hierarchy process (AHP) is used to construct a multi-criteria decision matrix based on the dual-track risk identification results, output the optimal handling strategy and execute it; after the operation is completed, the model parameters, prior hyperparameters and risk discrimination thresholds are updated online using the incremental learning operator based on the residual between the actual completion time and the predicted mean.

[0040] After obtaining the dual-track risk identification results, this solution needs to select the optimal option from a pre-defined treatment strategy library for execution. Based on a further improvement in this embodiment, the analytic hierarchy process (AHP) is used for multi-criteria decision-making. Specifically, a three-dimensional pairwise comparison judgment matrix is ​​first constructed, with the three dimensions being: treatment success rate (the probability that the strategy can alleviate the current risk), system execution cost (the additional resources consumed), and intervention timeliness (the delay required from decision to implementation). For the specific risk scenario, the relative importance scale between each dimension is provided by expert experience or historical statistical data and filled into the judgment matrix.

[0041] Furthermore, the largest eigenvalue of the judgment matrix is ​​calculated, and a consistency index is determined accordingly. Simultaneously, a corresponding random consistency index is found based on the matrix dimension, and the consistency index is divided by the random consistency index to obtain the consistency ratio. Preferably, the judgment matrix is ​​considered to have satisfactory consistency, and its output weight vector is considered reliable, only when the consistency ratio is less than 0.1. After the consistency verification passes, the weight values ​​of each candidate strategy are extracted from the matrix's eigenvectors to form a strategy weight vector. The strategy with the highest weight is selected as the optimal handling strategy. Finally, this strategy is sent to the resource controller at the cluster's underlying layer for execution via the state anchoring interface, achieving proactive intervention in the job's runtime environment.

[0042] Once the task is completed, this solution enters the closed-loop evolution phase. Specifically, the actual completion time of the task is collected and compared with the mean of the probability distribution predicted in phase S2 to calculate the residual. This residual reflects the systematic bias of the current prediction model: a positive residual indicates that the model tends to underestimate the completion time; a negative residual indicates that the model tends to overestimate it.

[0043] Based on this residual, stochastic gradient descent is used for online incremental learning. In one optional implementation, this operator simultaneously updates three sets of variables: the first set is the attention weights of each basic learner in the ensemble regression model, enabling the model to adjust its attention to each feature when encountering similar tasks in the future; the second set is the prior hyperparameters of the KL divergence term in the variational inference framework, which affects the subsequent measurement of uncertainty; the third set is the warning threshold corresponding to the correction factor used in the risk assessment process, making the dynamic shift of the explicit threshold more closely match the observed deviation trend. Each task completion feedback triggers the above updates, thereby continuously converging the prediction model, uncertainty measurement, and risk discrimination threshold towards the distribution of the real environment. Thus, a complete closed-loop governance loop of "physical feature enhancement - probabilistic prediction - dual-track risk identification - multi-criteria decision-making - incremental feedback evolution" is formed.

[0044] According to another aspect of this application, such as Figure 2 As shown, a physically enhanced feature manifold space is constructed, including: Step S11: Collect dynamic configuration data of the resource pool at the time of job submission to obtain the number of active service nodes, average job arrival rate, and average service rate per node.

[0045] In this embodiment, when any computing job is submitted to the scheduling queue of the distributed resource pool, a data acquisition process is first triggered. Specifically, a monitoring probe deployed at the cluster scheduling layer captures a snapshot of the current dynamic configuration of the resource pool in real time. The dynamic configuration data includes three core parameters: the number of active service nodes, the average job arrival rate, and the average service rate per node.

[0046] Preferably, the number of active service nodes (denoted as c) is obtained by counting the number of physical machines or container instances that are currently online and can accept new task scheduling, excluding nodes that are unavailable due to failure, maintenance, or network isolation. This value is refreshed every fixed period (e.g., 5 seconds) and sampled instantaneously when a job arrives.

[0047] Furthermore, the average job arrival rate (denoted as λ) is calculated using a sliding time window method. For example, the total number of jobs successfully submitted to the resource pool within the past T minutes (e.g., 10 minutes) is selected and divided by the time window length to obtain the average number of arriving jobs per unit time. To avoid drastic fluctuations caused by sudden bursts of traffic, an exponentially weighted moving average (EWMA) can be used to smooth the raw arrival rate.

[0048] Average service rate per node (denoted as μ) s The acquisition of queuing rate parameters relies on historical execution logs. For each active service node, the number of jobs completed within a past period (e.g., the past hour) is counted, and this number is divided by the node's total active time during that period (excluding idle waiting time) to obtain the node's average service rate. Then, the arithmetic mean of the service rates of all active service nodes is taken to obtain the global average service rate per node. These three parameters constitute the basic input data for subsequent queuing theory calculations.

[0049] Step S12: Divide the average job arrival rate by the product of the number of active service nodes and the average service rate per node to calculate the system service intensity, which serves as a scalar indicator representing the system's busyness.

[0050] After collecting the three basic parameters, this scheme enters the calculation stage of the core indicators of queuing theory. Specifically, the obtained average job arrival rate λ is used as the numerator, and the number of active service nodes c and the average service rate per node μ are calculated. s The product of the two is used as the denominator, and the ratio of the two is calculated. The result is the system service strength, denoted by ρ.

[0051] According to one aspect of this application, the system service intensity ρ represents the ratio between the maximum number of jobs that all service nodes can process and the actual number of jobs arriving per unit time. When ρ is less than 1, it indicates that the system's service capacity exceeds the arrival load, and the queue length tends to stabilize within a finite time, satisfying the steady-state condition of the M / M / c / K queuing dynamics theoretical model. When ρ is close to 1, it indicates that the system is operating under high load, at which point the queue length is extremely sensitive to parameter disturbances, and even small fluctuations in the arrival rate can cause significant queuing delays. When ρ is greater than or equal to 1, it indicates that the system is continuously overloaded, and the queue will grow indefinitely, requiring the triggering of additional elastic expansion mechanisms under this condition.

[0052] For example, assuming the number of active service nodes collected is c=8, the average job arrival rate is λ=12 per minute, and the average service rate per node is μ s =1.6129 per minute, then the calculation process for service intensity ρ is as follows: first calculate c and μ s The product of these values ​​is 12.9032. Dividing λ=12 by this product yields ρ≈0.93. This value indicates that the current resource pool is at a high load level of 93%, which is a typical congested condition. The subsequent queuing theory derivation has significant engineering implications.

[0053] Step S13: Substitute the system service intensity and the number of service nodes into the steady-state probability series summation formula of the M / M / c / K queuing dynamics theoretical model to calculate the probability that all nodes are in an idle state at the same time, which is used as the system idle probability.

[0054] After obtaining the system service intensity ρ, this scheme further solves for a key intermediate quantity in the queuing theory model: the system idle probability (denoted as P0), which represents the probability that, under steady-state conditions, all c service nodes in the resource pool are simultaneously idle (i.e., no jobs are being processed). In high-load scenarios, P0 is usually a very small positive number, reflecting the degree to which system resources are fully utilized.

[0055] According to a further improvement in this embodiment, the calculation of P0 is based on the steady-state probability series summation formula of the M / M / c / K queuing dynamics theoretical model (in this scheme, the M / M / c / K model without capacity limit is adopted, that is, K tends to infinity; in actual engineering, a finite queue length K can also be set). Specifically, the numerator of the formula is 1, and the denominator consists of two parts: the first part is a summation term from k=0 to k=c-1, each term being (cρ) raised to the power of k divided by the factorial of k; the second part is a correction term, which is the product of (cρ) raised to the power of c divided by the factorial of c and (1-ρ).

[0056] Preferably, to avoid numerical overflow, when cρ is large (e.g., exceeding 20), a recursive relation or logarithmic transformation can be used for numerical stabilization. In an optional implementation, for the scenario where c=8 and ρ=0.93, first calculate cρ=7.44, and then calculate the terms (cρ) sequentially for k=0 to 7. 0 / 0! = 1, (cρ) 1 / 1! = 7.44, (cρ) 2 / 2! ≈ 27.68, and so on, accumulating to obtain an approximate value of approximately 908.32 for the first part. The second part calculates (cρ). 8 =7.44 8 Approximately 9.4 × 10 6 Dividing by the product of 40320 and 0.07, we get approximately 3331.6. The sum of the denominators is approximately 4240, and taking the reciprocal, we get P0 as approximately 2.35 × 10⁻⁶. -4 This extremely small idle probability verifies that it is almost impossible for all nodes to be idle simultaneously under high load, thus providing an accurate normalized basis for subsequent queue length calculation.

[0057] Step S14: Based on the system idle probability, system service intensity, and number of service nodes, calculate the average number of tasks in the queue according to the average queue length calculation formula in queuing theory, and use this as the queue backlog length.

[0058] After calculating the system idle probability P0, this scheme further calculates the average queue length (denoted as Lq). This metric directly characterizes the number of preceding tasks that a newly arriving job is expected to wait for before it officially receives service. According to classic conclusions of queuing theory, the average queue length is determined by P0, service intensity ρ, number of service nodes c, and the square of (1-ρ).

[0059] Specifically, the formula for calculating the average queue length Lq is: the numerator is P0 multiplied by (cρ) raised to the power of c, then multiplied by ρ; the denominator is the factorial of c multiplied by the square of (1-ρ). In this embodiment, the obtained P0≈2.35×10 -4 、(cρ) c =7.44 8 ≈9.4×10 6 Substituting ρ=0.93 into the numerator, we get a numerator ≈ 2.35e -4 ×9.4e 6 ×0.93≈2054. Denominator: c! =40320, (1-ρ) 2 = 0.07 2 = 0.0049, the product of the numerator and denominator is approximately 40320 × 0.0049 ≈ 197.6. Dividing the numerator by the denominator, we get Lq ≈ 10.4. This result indicates that under the current high load condition of 93%, Job2026 On average, there are about 10 tasks waiting in the queue for service in the arrival time system.

[0060] In a preferred embodiment, to verify the reasonableness of the calculation results, the average system queue length L = Lq + cρ can be calculated simultaneously, where cρ represents the average number of tasks being served (equal to the service intensity multiplied by the number of nodes), and the sum of the two is the total number of tasks in the system. This value should satisfy the self-consistency test of Little's Law.

[0061] Step S15: Based on the queue backlog length and the average job arrival rate, divide the two according to Little's law to obtain the theoretical waiting time. Use a first-order exponential smoothing coefficient to weight and fuse the theoretical waiting time with the historical observation mean to obtain a continuous real-valued physical enhancement feature.

[0062] After obtaining the average queue length Lq, this scheme transforms it into a theoretical waiting time with a definite temporal meaning. According to Little's Law, in a steady-state queuing system, the average queue length Lq is equal to the average job arrival rate λ multiplied by the average waiting time Wq. Therefore, the theoretical waiting time Wq is equal to Lq divided by λ.

[0063] For example, dividing the obtained Lq≈10.4 tasks by the average job arrival rate λ=0.2 tasks per second (the arrival rate per minute of 12 needs to be converted to 0.2 tasks per second), we get Wq≈52.0 seconds. This duration represents: ignoring random fluctuations, the Job... 2026 It is expected to wait approximately 52 seconds in the queue before being processed.

[0064] However, purely theoretical models may deviate from actual observations. Therefore, this scheme introduces a first-order exponential smoothing mechanism for calibration. Specifically, it first queries the historical database for the historical observation mean corresponding to the current queuing reason code for the job (e.g., "preempted by a higher priority", reason code number 101). This mean is obtained by arithmetically averaging the actual wait times of all jobs carrying the same reason code over a past period, for example, 44 seconds. Then, a smoothing coefficient α (typically 0.85) between 0 and 1 is selected, which determines the confidence weight of the theoretical model relative to historical observations. The final physical enhancement feature w... i The calculation method is as follows: α multiplied by the theoretical waiting time Wq, plus (1-α) multiplied by the historical observation average. In this example, w i = 0.85×52.0 + 0.15×44 = 50.8 seconds, rounded down to 51.0 seconds.

[0065] This fusion feature retains the physical prior knowledge of the queuing theory model (through α-weighting) while also calibrating the model bias using historical data (through (1-α) weighting). The final real-valued feature w is obtained. i This will replace the original discrete cause code and, together with other basic attributes of the task, form the input vector for the subsequent probabilistic prediction model.

[0066] According to another aspect of this application, such as Figure 3 As shown, a probability distribution model is performed on the ETA (Earning Time to Complete) of the job, outputting the mean, standard deviation, and dynamic confidence interval, including: Step S21: Perform multi-source feature concatenation and vectorization encoding on the physical enhancement features and basic job attribute data to obtain the complete input feature vector.

[0067] After completing the physical augmentation mapping of queuing semantics and obtaining the real-valued feature w i Next, this scheme enters the feature engineering stage of the probabilistic prediction model. Specifically, w i Multi-source fusion is performed with other basic attribute data of the job. Basic attributes include, but are not limited to: job resource profile (estimated number of CPU cores, estimated memory size, estimated number of GPUs), dependency graph (completion rate of upstream jobs, number of blocked tasks), user quota (remaining available CPU time, memory quota balance), and job type label (compute-intensive, IO-intensive, memory-intensive).

[0068] Preferably, the numerical features (such as the number of CPU cores and memory size) in the above basic attributes are directly subjected to max-min normalization to map their value range to the [0,1] interval, so as to eliminate the influence of the difference in units on model training. Categorical features (such as job type labels) are converted into binary vectors using one-hot encoding. For example, "computation-intensive" is mapped to [1,0,0], "IO-intensive" is mapped to [0,1,0], and "memory-intensive" is mapped to [0,0,1].

[0069] In one alternative implementation, the physical enhancement feature w i (Unit: seconds) Since its physical meaning is consistent with the ETA dimension, no additional normalization is needed; the original value is directly retained as a one-dimensional feature. Subsequently, all encoded and normalized feature vectors are concatenated along the feature dimension to form a unified high-dimensional dense vector, denoted as X. The dimension of this input feature vector X depends on the number of features enabled in the specific scenario, typically ranging from tens to hundreds of dimensions. Finally, X is fed into the subsequent Bayesian ensemble regression model for probabilistic prediction.

[0070] Step S22: Based on the input feature vector and the predicted output of the ensemble regression model, perform variational inference calculation to maximize the lower bound of evidence, obtain the cognitive uncertainty value of the model for heterogeneous noise environment, and internalize it into the model parameters.

[0071] After constructing the input feature vector X, this scheme employs an ensemble regression model enhanced with variational inference. Unlike traditional point estimation regression models, this scheme models the job completion time as a random variable and approximates its posterior distribution through variational inference.

[0072] Specifically, let y be the actual completion time of the task (the target variable to be predicted), and z be the latent variable in the model (which can be understood as the model's internal representation or attention weights). The variational inference method is as follows: a parameterized variational distribution q(z|X) is used to approximate the true but difficult-to-solve posterior distribution p(z|X,y). To achieve this approximation, the model optimizes the variational parameters by maximizing the lower bound of evidence (ELBO).

[0073] According to another aspect of this application, the lower bound of evidence (ELBO) consists of two terms: the first is the expected log-likelihood term, E{q(z|X)}[log p(y|X,z)], which measures the accuracy of the model's fit to the training data under the current variational distribution: the higher the expected value, the more accurate the model's prediction of the actual completion time; the second is the negative KL divergence term, -KL(q(z|X) || p(z)), where the KL divergence measures the difference between the variational distribution q(z|X) and the prior distribution p(z). Combining the two terms, ELBO = expected log-likelihood - KL divergence.

[0074] During model training, stochastic gradient descent (SGD) or its variants (such as the Adam optimizer) are used to maximize the ELBO. In each iteration, a batch of samples is sampled from the training dataset, the ELBO value with respect to the current parameters and its gradient with respect to the model parameters are calculated, and then the parameters are updated along the gradient ascent direction. When the model converges, the value of the KL divergence term directly reflects the model's cognitive uncertainty regarding the current heterogeneous noise environment: if the environmental noise is high, the training data coverage is insufficient, or the mapping relationship between features and the target is ambiguous, the KL divergence term will tend to have a large positive value, resulting in an increase in the standard deviation σ of the subsequent output. This cognitive uncertainty is internalized in the model's variational parameters and is explicitly output during the prediction phase.

[0075] Step S23: Sample or analyze the posterior distribution of the model parameters to obtain the mean and standard deviation of the distribution.

[0076] Once the variational inference model has completed training (or online prediction), this scheme extracts the specific probability distribution parameters from the approximate posterior distribution. In a preferred embodiment, it is assumed that the conditional likelihood p(y|X,z) follows a Gaussian distribution, and the variational distribution q(z|X) is also constrained to a Gaussian form. Under this assumption, the marginal distribution of the predicted output is also a Gaussian distribution, and its parameters can be directly calculated analytically.

[0077] Specifically, for a given input feature vector X, the model first calculates the parameters of the variational distribution through forward propagation: that is, the mean vector μ of the latent variable z. z The covariance matrix Σ z (Usually simplified to diagonal covariance). Then, the predictions from multiple base learners (e.g., multiple extreme random trees) in the ensemble regression are aggregated. Each base learner outputs a sub-prediction value, and the arithmetic mean of all sub-prediction values ​​is the mean μ of the final prediction; the sample standard deviation of all sub-prediction values ​​(after Bessel correction) is the standard deviation σ of the final prediction.

[0078] In another alternative implementation, if the model uses Monte Carlo sampling for prediction, then T latent variable samples (e.g., T=100) are drawn from the variational distribution q(z|X), and for each latent variable sample, the corresponding predicted value y is calculated. t Then, the empirical mean of these T predicted values ​​is taken as μ, and the empirical standard deviation as σ. The advantage of the sampling method is that it does not require making a Gaussian assumption about the form of the posterior distribution, and it is applicable to more complex distribution forms. Regardless of the method used, the final output consists of two key values: the mean μ (in seconds, representing the most likely completion time) and the standard deviation σ (in seconds, representing the uncertainty of the prediction).

[0079] Step S24: Based on the mean, standard deviation, and quantiles corresponding to the preset confidence level, perform interval estimation to obtain the dynamic confidence interval.

[0080] After obtaining the mean μ and standard deviation σ, this scheme further constructs dynamic confidence intervals with clear probabilistic significance. The purpose of the confidence interval is to provide a safe and redundant boundary for the downstream scheduler: given the model's current understanding, the probability that the actual completion time falls within this interval is equal to the preset confidence level.

[0081] According to a further improvement of this embodiment, a confidence level is first selected, for example, 95%. Under the assumption of a standard normal distribution, the two-sided quantile corresponding to the 95% confidence level is Z≈1.96 (more precisely, this can be obtained by consulting the standard normal distribution cumulative function table). Subsequently, the lower limit of the confidence interval is L = μ - Z × σ, and the upper limit of the confidence interval is U = μ + Z × σ.

[0082] For example, assuming the model output has a mean μ = 300 seconds, a standard deviation σ = 15 seconds, and a confidence level of 95%, then Z = 1.96. Calculating L = 300 - 1.96 × 15 = 270.6 seconds, and U = 300 + 1.96 × 15 = 329.4 seconds, this approach asserts that with a 95% probability, the actual completion time of the task will be between 270.6 seconds and 329.4 seconds. The interval width W = U - L = 2 × Z × σ = 58.8 seconds. This width intuitively reflects the uncertainty of the current prediction: the larger σ is, the wider the interval, indicating that the model is less confident about the prediction result.

[0083] In one optional implementation, multiple confidence level intervals (e.g., 68%, 95%, 99.7%) can be output simultaneously, allowing downstream schedulers to flexibly choose based on the risk tolerance of the business scenario. For example, for high-priority jobs, a narrower 68% confidence interval can be used to reduce the conservatism of resource reservations; for critical tasks, a 99.7% interval can be used to ensure that SLA defaults are almost impossible. The upper and lower limits of the dynamic confidence interval are then sent to the risk perception module for threshold comparison of explicit timeout risks.

[0084] According to another aspect of this application, such as Figure 4 As shown, explicit statistical hard threshold assessment and isolated forest latent anomaly path detection are performed in parallel to generate dual-track risk identification results, including: Step S31: Construct a correction factor that increases with the standard deviation. Use this correction factor to dynamically shift the base SLA threshold to obtain the dynamic SLA threshold. Compare the upper limit of the confidence interval with the dynamic SLA threshold to obtain the result of the explicit timeout risk assessment and the excess amount.

[0085] After obtaining the upper limit of the dynamic confidence interval U (i.e., ETA) max Following this, the scheme proceeds to the assessment phase of explicit timeout risk. Considering that static SLA thresholds cannot adapt to fluctuations in environmental uncertainty, this scheme introduces a correction factor κ that adaptively varies with the prediction standard deviation σ.

[0086] Specifically, the construction of the correction factor κ follows an exponential weighting mapping rule. First, a scene calibration reference scale σ is obtained. ref This value can be obtained through offline calibration experiments: for example, in a standard noise environment, record the typical standard deviation of the model output, and satisfy σ ref The relationship is = -σ × ln(2 / 3). Then, σ ref Divide by the standard deviation σ of the current predicted output, take the negative value as the exponential part of the natural constant e, and calculate, i.e., κ = exp(-σ ref / σ). When σ is large, σref When σ is small, the exponential part approaches 0, and κ approaches 1. The correction factor κ increases with increasing σ (approaching 1) and decreases with decreasing σ (approaching 0), indicating that: when the model uncertainty is high, κ approaches 1, and the dynamic threshold shift is limited; when the model is very deterministic, κ is small, and the dynamic threshold is close to the base threshold. In an optional implementation, the correction factor is used to shift the threshold T. SLA = T SLA0 +(1-κ)·σ, when the standard deviation σ increases, the correction factor κ increases and approaches 1. Although 1-κ decreases accordingly, the dynamic translation (1-κ)·σ still increases with the increase of σ and gradually approaches the scene calibration reference scale σref. Therefore, the dynamic threshold gradually shifts towards a more relaxed direction relative to the basic threshold, and the translation amount has an upper limit. When σ is small, κ is small and (1-κ) is large, the threshold is close to the basic value.

[0087] Furthermore, the upper limit of the confidence interval ETA is... max With T SLA Compare. If ETA max >T SLA If so, it is determined that there is an explicit timeout risk, and the excess amount ΔT = ETA is calculated. max - T SLA Otherwise, it is determined that there is no explicit timeout risk. The value of this excess will serve as an important input dimension of the subsequent analytic hierarchy process (AHP) decision matrix, reflecting the severity of the risk.

[0088] Step S32: Based on the time-series data of CPU utilization, memory usage and IO throughput during the job execution phase, perform sliding window tensor sampling and isolated forest inference to obtain latent anomaly scores that characterize the degree to which the execution trajectory deviates from the normal distribution.

[0089] In parallel with the assessment of explicit timeout risks, this solution utilizes the Isolation Forest algorithm to detect implicit anomalies in the job's execution trajectory. Specifically, after the job starts, its runtime monitoring data is continuously collected, including: CPU utilization (percentage), memory usage (percentage), IO read rate (bytes per second), IO write rate (bytes per second), and IO latency (milliseconds). This data is reported by monitoring probes deployed on each compute node at a fixed frequency (e.g., once per second).

[0090] According to another aspect of this application, in order to capture the dynamic changes during job execution, a sliding window is used for tensor sampling. A time window width W (e.g., 60 seconds) is set, and it slides on the time axis with a step size S (e.g., 10 seconds). The time-series data within each window constitutes a multi-dimensional feature vector: each timestamp corresponds to a multi-dimensional point (CPU, memory, IO read, IO write, IO latency), and the entire window forms a tensor with a dimension of (number of points in the window × feature dimension). This tensor is flattened into a one-dimensional feature vector, which is used as an input sample for the isolated forest.

[0091] The Isolation Forest model is pre-trained using historical running data from normal jobs. During training, the model randomly selects feature dimensions and split values, recursively dividing the feature space into multiple hyperrectangular regions to construct multiple isolated trees (e.g., 100 trees). For a sliding window sample x of the current job, the model calculates its path length h(x) on each isolated tree: the number of edges traversed from the root node to a leaf node. Then, the average path depth E(h(x)) is obtained by averaging across all trees. Simultaneously, the average structural depth c(n) of the forest is calculated based on the subsample size n (the size of each subsample set during training), with a theoretical expected value of 2 × H. n-1 - 2×(n-1) / n, where H n-1 It is the (n-1)th harmonic number, approximately ln(n-1)+0.577.

[0092] Finally, the formula for calculating the anomaly score s is: s = 2 -E(h(x)) / c(n) When E(h(x)) is much smaller than c(n), the exponential part is a large negative number, and s is close to 1, indicating that the sample is very easily isolated, i.e., the degree of anomaly is high; when E(h(x)) is close to c(n), s is approximately equal to 0.5, indicating that the sample is on the edge of the normal distribution; when E(h(x)) is much larger than c(n), s is close to 0, indicating that the sample is highly normal. This scheme presets an anomaly warning threshold (e.g., 0.6). When s exceeds this threshold, it is determined that there is a hidden trajectory anomaly.

[0093] Step S33: Perform risk level weighted fusion of the explicit timeout risk assessment result and the implicit anomaly score to obtain the dual-track risk identification result.

[0094] After obtaining the explicit timeout risk assessment result (including the excess amount ΔT) and the implicit anomaly score s respectively, this scheme fuses the two signals to generate a comprehensive risk situation awareness result. The purpose of fusion is to avoid false alarms or missed alarms from a single path and to improve the accuracy of risk identification through cross-validation of multi-dimensional evidence.

[0095] Specifically, the first step is to quantify the explicit risk outcomes: if the ETA max ≤T SLAThe explicit risk value R explicit = 0; if ETAmax > T SLA Then R explicit = min(1, ΔT / T SLA The implicit risk value R is calculated by truncating the excess amount relative to the base threshold to the [0,1] range. implicit Take the abnormal score s directly (which is already in the [0,1] interval).

[0096] In a preferred embodiment, a weighted sum is used for fusion: the comprehensive risk score R total =w explicit ×R explicit + w implicit ×R implicit , where w explicit and w implicit These are preset weighting coefficients, and their sum is 1. For example, w can be set based on historical experience. explicit =0.6, w implicit =0.4, meaning the weight of explicit timeout risk is slightly higher than that of implicit anomalies, because timeouts are directly associated with SLA defaults. Then, according to R total The risk level is output based on the threshold range it falls within: for example, R total <0.3 indicates "low risk", 0.3≤R total <0.6 indicates "medium risk", R total A value ≥0.6 is considered "high risk". Simultaneously, the fusion result also retains the original two independent signals (whether an explicit alarm was triggered, and whether a implicit alarm was triggered), so that the subsequent decision-making module can refer to the specific risk characteristic attribution when calling the Analytic Hierarchy Process (AHP). Finally, the dual-track risk identification result (risk level, R...) total (ΔT, s, trigger flag) are encapsulated into a data structure and passed to the decision governance module.

[0097] According to another aspect of this application, such as Figure 5 As shown, the Analytic Hierarchy Process (AHP) is used to construct a multi-criteria decision matrix and output the optimal response strategy, including: Step S41: Construct a three-dimensional judgment matrix that includes the success rate of handling, system execution cost and intervention timeliness, solve for the maximum eigenvalue of the matrix, and calculate the consistency ratio.

[0098] When the dual-track risk identification results reach the medium or high risk level, this plan uses the Analytic Hierarchy Process (AHP) to select the optimal treatment strategy. First, a three-dimensional pairwise comparison matrix A is constructed, with dimensions 3×3, corresponding to three criteria: treatment success rate (C1), system execution cost (C2), and intervention timeliness (C3). The element a in the matrix... ijThe importance scale of criterion i relative to criterion j is represented by a 1-9 scale: 1 indicates equal importance, 3 indicates slightly important, 5 indicates significantly important, 7 indicates strongly important, 9 indicates extremely important, and 2, 4, 6, and 8 are intermediate values.

[0099] For example, in a scenario with explicit timeout risk (ΔT = 4.4 seconds) and implicit resource contention anomaly (s = 0.764), the success rate of handling the situation is considered the most important, followed by the timeliness of intervention, while the system execution cost is relatively the least important. Therefore, we can set: a 12 =3 (C1 is slightly more important than C2), a 13 =5 (C1 is significantly more important than C3), a 23 =2 (C2 is slightly more important than C3). Then fill in the remaining elements of the matrix according to symmetry: a ji =1 / aij And the diagonal element a ii =1.

[0100] Next, we solve for the largest eigenvalue λ of the judgment matrix. max Approximate calculations can be performed using the power method or the sum-product method. The steps of the sum-product method are as follows: normalize each column of the matrix, then sum the results for each row, and divide by the matrix dimension to obtain an approximate value of the eigenvector. Finally, calculate A multiplied by the eigenvector, divide each component of the resulting vector by its corresponding eigenvector component, and take the average to obtain λ. max We obtain λ. max Then, calculate the consistency index CI = (λ) max - n) / (n-1), where n=3. Then, find the average random consistency index RI based on n=3 (the standard value given by Saaty: RI=0.58 when n=3). Finally, calculate the consistency ratio CR= CI / RI. If CR<0.1, the judgment matrix is ​​considered to have satisfactory consistency and the weight vector is reliable; otherwise, the scale values ​​in the judgment matrix need to be readjusted until the consistency check is passed.

[0101] Step S42: After verifying that the consistency ratio is less than the preset threshold, output the strategy weight vector and select the processing strategy with the highest weight as the optimal processing strategy.

[0102] After confirming CR < 0.1, this scheme extracts the policy weight vector from the judgment matrix. Specifically, it sums each row of the normalized judgment matrix and then divides by n to obtain the weight w for each criterion. i(i=1,2,3). The sum of these three weight values ​​is 1. Then, for each candidate strategy in the preset handling strategy library (e.g., "dynamic hot expansion of CPU quota", "job migration to idle node", "increase job priority", "wait without intervention", etc.), a score is given for the strategy on the three criteria based on expert knowledge or historical statistical data. The comprehensive score of each strategy is equal to the sum of the weights of each criterion multiplied by the scores under that criterion. After normalizing the comprehensive scores of all candidate strategies, the strategy weight vector is obtained.

[0103] Preferably, the strategy with the highest weight in the strategy weight vector is selected as the optimal handling strategy. For example, in a latency scenario caused by resource contention, "dynamic hot expansion of CPU quota" may receive the highest weight of 0.88; while in an IO bottleneck scenario, "priority enhancement of storage read / write" may be the optimal choice. This strategy is then output to the execution interface.

[0104] Step S43: Send the optimal handling strategy to the underlying controller of the cluster for execution through the state anchoring interface.

[0105] After determining the optimal handling strategy, this solution sends the strategy command to the underlying resource controller of the cluster through a predefined state anchoring interface. The state anchoring interface is a standardized API for bidirectional communication, supporting both synchronous and asynchronous calling modes. Specifically, the parameters received by the interface include: Job ID, strategy type code (e.g., "HOT_EXPAND_CPU"), strategy parameters (e.g., the number of CPU cores to be expanded, the list of target nodes), and execution time limit (e.g., completion within 5 seconds).

[0106] Upon receiving the instruction, the underlying controller parses the policy type and invokes the corresponding execution module. For example, for hot scaling of CPU quotas, the controller sends a resource update request to the container orchestration engine (such as Kubernetes) to modify the CPU limit value of the Pod where the job resides. After execution, the controller returns the execution result (success / failure, actual effective time, and amount of resource changes) through a status anchoring interface. The risk decision module in this solution can make secondary decisions based on the returned results (e.g., if hot scaling fails, a job migration strategy is implemented as a fallback). The design of this interface ensures the decoupling of this solution from the underlying infrastructure and is adaptable to various cluster management platforms.

[0107] This invention has the following advantages: First, by introducing the M / M / c / K queuing dynamics theory model to physically model the semantics of job queuing, discrete reason codes are transformed into continuous waiting time features with clear physical meanings. This solves the problem that traditional shallow statistical features cannot characterize the queuing dynamics process, improving the accuracy of completion time prediction under high load and congestion conditions. Second, by employing a variational inference-enhanced Bayesian ensemble regression model, the prediction results are expanded from a single numerical value to a complete probability distribution including the mean, standard deviation, and dynamic confidence interval. This achieves explicit quantification of prediction uncertainty in the job scheduling field, enabling downstream schedulers to perform differentiated risk avoidance operations based on confidence boundaries. Third, it constructs an "explicit statistical hard threshold + The dual-track risk perception system of "isolated forest latent anomaly detection," coupled with a correction factor that automatically shifts with uncertainty, can capture progressive hidden dangers that have deviated from the normal trajectory but have not yet triggered hard thresholds, upgrading passive post-event remediation to proactive early intervention. Fourth, the online update mechanism of stochastic gradient descent based on completed residuals realizes the closed-loop co-evolution of prediction model weights, KL prior hyperparameters, and risk discrimination thresholds, enabling the system to have continuous adaptive capability under task flow distribution drift and avoiding accuracy decay in long-term operation. Fifth, the introduction of the analytic hierarchy process (AHP) for multi-criteria decision-making, combined with consistency verification, ensures the rationality of the handling strategy, while the snapshot management mechanism provides fault tolerance capability for lossless rollback of state in the face of drastic environmental changes. In summary, this invention forms a complete technical closed loop from feature enhancement, probabilistic modeling, risk perception, closed-loop evolution to decision execution, achieving breakthrough progress in prediction accuracy, uncertainty quantification, proactive risk management, and system robustness.

[0108] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A job evolution control method based on random queuing semantics and variational Bayesian ensemble learning, characterized in that, include: Based on the discrete semantic identifiers and arrival features of the job queuing stage, the steady-state distribution probability of the system is calculated using the M / M / c / K queuing dynamics theoretical model, and the theoretical waiting time features with physical meaning are reduced to generate, thus constructing a physically enhanced feature manifold space. During operation, the system combines physical enhancement features with basic job attribute data to form a complete input vector. An integrated regression model enhanced by variational inference algorithm is used to model the probability distribution of job completion time, and outputs the mean, standard deviation and dynamic confidence interval. Based on the dynamic confidence interval and the load characteristics during operation, explicit statistical hard threshold judgment and isolated forest hidden abnormal path detection are performed in parallel. The prediction uncertainty correction factor is used to dynamically shift the early warning limit to generate dual-track risk identification results. The analytic hierarchy process (AHP) is used to construct a multi-criteria decision matrix based on the dual-track risk identification results, and the optimal handling strategy is output and executed. After the operation is completed, the model parameters, prior hyperparameters, and risk discrimination thresholds are updated online using an incremental learning operator based on the residual between the actual completion time and the mean of the predicted probability distribution.

2. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The construction of the physically enhanced feature manifold space includes: Collect dynamic configuration data of the resource pool at the time of job submission to obtain the number of active service nodes, average job arrival rate, and average service rate per node; The system service intensity is calculated by dividing the average arrival rate of the job by the product of the number of active service nodes and the average service rate of a single node, and is used as a scalar indicator to characterize the busyness of the system. Substitute the system service intensity and the number of active service nodes into the steady-state probability series summation formula of the M / M / c / K queuing dynamics theoretical model to calculate the probability that all nodes are simultaneously in an idle state, which is taken as the system idle probability. Based on the system idle probability, the system service intensity, and the number of active service nodes, the average queue length calculation formula of the M / M / c / K queuing dynamics theoretical model is used to calculate the average number of queued tasks in the system, which is then used as the queue backlog length. Based on the queue backlog length and the average arrival rate of the job, the theoretical waiting time is obtained by dividing the two according to Little's law. The theoretical waiting time and the historical observation mean are weighted and fused with a first-order exponential smoothing coefficient to obtain a continuous real-valued physical enhancement feature.

3. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The method of probabilistically modeling the ETA of the job completion time and outputting the mean, standard deviation, and dynamic confidence interval includes: The physical enhancement features and the basic job attribute data are concatenated using multi-source features and vectorized encoding to obtain a complete input feature vector. Based on the input feature vector and the predicted output of the ensemble regression model, variational inference calculation to maximize the lower bound of evidence is performed to obtain the cognitive uncertainty value of the model for heterogeneous noise environment, which is internalized into the model parameters. The model parameters are sampled or analyzed using a posterior distribution to obtain the mean and standard deviation of the distribution. The dynamic confidence interval is obtained by performing interval estimation based on the mean, the standard deviation, and the quantiles corresponding to the preset confidence level.

4. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The parallel execution of explicit statistical hard threshold assessment and isolated forest latent anomaly path detection generates dual-track risk identification results, including: Construct a correction factor that increases with the standard deviation, and use the correction factor to dynamically shift the base SLA threshold to obtain the dynamic SLA threshold. Compare the upper limit of the dynamic confidence interval with the dynamic SLA threshold to obtain the explicit timeout risk assessment result and the excess amount. Based on the time-series data of CPU utilization, memory usage and IO throughput during the job operation phase, sliding window tensor sampling and isolated forest inference are performed to obtain latent anomaly scores that characterize the degree to which the operation trajectory deviates from the normal distribution. The explicit timeout risk assessment result and the implicit anomaly score are weighted and fused according to risk level to obtain the dual-track risk identification result.

5. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 4, characterized in that, It also includes adaptive updates to the correction factor: The standard deviation of the collected prediction output and the scene calibration reference scale; Divide the scene calibration reference scale by the standard deviation, take the negative value and use it as the exponential part of the exponential function for mapping calculation, to obtain a dynamic correction factor that automatically shifts with the prediction uncertainty, which is used to adjust the sensitivity limit of the statistical hard threshold in real time.

6. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The analytic hierarchy process (AHP) is used to construct a multi-criteria decision matrix and output the optimal handling strategy, including: Construct a three-dimensional judgment matrix that includes the success rate of handling, system execution cost and intervention timeliness, solve for the maximum eigenvalue of the matrix, and calculate the consistency ratio; After verifying that the consistency ratio is less than a preset threshold, output the strategy weight vector and select the processing strategy with the highest weight as the optimal processing strategy. The optimal handling strategy is sent to the underlying controller of the cluster for execution via the state anchoring interface.

7. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The method of using incremental learning operators to update model parameters, prior hyperparameters, and risk discrimination thresholds online includes: Based on the actual completion time after the task is completed and the mean of the predicted probability distribution, residual calculation is performed to obtain the scalar residual; Based on the scalar residual, a stochastic gradient descent backpropagation operation is performed to update three sets of variables: the attention weights of the ensemble regression model, the prior hyperparameters of the KL divergence term, and the warning limits corresponding to the correction factors. The updated prediction model parameters and risk assessment parameters for the next cycle are obtained.

8. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The ensemble regression model employs an extreme random tree architecture enhanced by a Bayesian optimization algorithm: it searches for the globally optimal deep structure and feature sampling strategy of the extreme random tree in a continuous hyperparameter manifold space through a Gaussian process surrogate model, and outputs a posterior distribution with KL divergence constraints within a variational inference framework.

9. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, It also includes establishing a snapshot management mechanism for heterogeneous task load distribution: while performing online updates of model parameters, it saves the current ensemble regression model parameter topology sequence, KL divergence prior hyperparameter set, and risk discrimination threshold vector at a preset period to support lossless rollback of state based on snapshots during drastic environmental migrations.

10. The job evolution control method based on stochastic queuing semantics and variational Bayesian ensemble learning according to claim 1, characterized in that, The method is deployed in the core scheduling link layer of the distributed computing system: it performs real-time bidirectional data interaction with the resource controller and monitoring probe at the bottom layer of the resource pool through a high-performance state anchoring interface, receives job queuing semantic tags and runtime monitoring indicators, and issues the optimal handling strategy determined by the analytic hierarchy process.

Images