Real-time analysis and intelligent optimization system for software operation data based on machine learning

Abstract

This invention provides a real-time analysis and intelligent optimization system for software operation data based on machine learning, comprising: a data acquisition module for acquiring real-time operational data of the software operation system, including request arrival logs, service processing logs, and resource usage information; a mechanism model construction module for establishing a queuing control mechanism model based on the real-time operational data and defining a set of operational operations within the mechanism model; an operational state estimation module for estimating the real-time operational data based on the mechanism model to obtain request arrival rate intervals, service capacity intervals, and burst traffic intervals, forming a set of uncertainty intervals; and a service level target constraint module for selecting tail performance indicators as service level targets based on the set of uncertainty intervals. This invention can continuously guarantee service level targets under dynamic fluctuations and uncertain environments, improving the stability and resource utilization of the software operation system.

Description

Technical Field

[0001] This invention relates to the field of operational data analysis and optimization technology, and in particular to a real-time analysis and intelligent optimization system for software operational data based on machine learning. Background Technology

[0002] As internet applications continue to expand, software operating systems need to maintain stable operation in environments with high concurrency, high traffic, and dynamic fluctuations. Existing operation and maintenance methods typically rely on static threshold monitoring and manual intervention. When there is a sudden increase in request traffic or fluctuations in service capacity, it is often difficult to adjust in a timely manner, which can easily lead to increased system latency, increased request drop rates, or even service unavailability.

[0003] Existing technologies have attempted to introduce queuing models or scheduling algorithms to describe the operational status, but most of them only focus on average indicators and lack effective constraints on tail performance indicators (such as P95 / P99 latency and availability default rate), which leads to the risk of performance default in extreme cases. Therefore, we propose a real-time analysis and intelligent optimization system for software operation data based on machine learning.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a real-time analysis and intelligent optimization system for software operation data based on machine learning, thereby solving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A machine learning-based real-time analysis and intelligent optimization system for software operation data includes:

[0008] The data acquisition module is used to acquire real-time operating data of the software operation system, including request arrival logs, service processing logs, and resource usage information.

[0009] The mechanism model construction module is used to establish a queuing control mechanism model based on the real-time running data, and to define a set of running operations in the mechanism model;

[0010] The operation status estimation module is used to estimate real-time operation data based on the mechanism model to obtain the request arrival rate range, service capacity range, and burst traffic range, forming a set of uncertainty ranges;

[0011] The service level target constraint module is used to select tail performance indicators as service level targets based on the set of uncertainty intervals, and combine them with stability constraints to form a set of constraints.

[0012] The constraint reinforcement learning optimization module is used to generate operating strategies using reinforcement learning methods under the set of constraints, and to embed the mechanism model into the strategy evaluation and improvement process.

[0013] The strategy security verification module is used to perform formal verification, simulation verification and gray-scale verification on the operation strategy, and to verify whether the operation strategy meets the service level target and stability constraints within the set of uncertainty intervals.

[0014] The deployment and closed-loop update module is used to deploy the verified and qualified operation strategy to the software operation system for execution, collect the execution results of the operation strategy as new real-time operation data, and form a closed loop for continuous real-time analysis and optimization of the operation strategy.

[0015] The steps to establish a queuing control mechanism model include:

[0016] Collect real-time operational data of the software operation system, including request arrival logs, service processing logs, and resource usage information;

[0017] Based on the aforementioned operational data, operational parameters such as request arrival rate, service rate, concurrency, buffer capacity, and throttling threshold are extracted.

[0018] A queuing control mechanism model is established based on the aforementioned operating parameters to characterize the relationship between request arrival, service processing, and queuing.

[0019] The mechanism model defines a set of running operations, which includes at least scheduling, degradation, caching, and traffic splitting operations, and sets trigger conditions for each operation.

[0020] The running parameters are bound to the set of running operations to form a basic set of running constraints.

[0021] The estimation steps of the operating state estimation module include:

[0022] Based on the aforementioned mechanism model, real-time operational data is statistically analyzed to obtain the average and fluctuation of request arrival rates.

[0023] The request arrival rate range is calculated based on the statistical results to reflect the normal fluctuation range of request traffic.

[0024] By analyzing changes in service processing capacity, a service capacity range can be determined.

[0025] Based on distribution characteristics, burst traffic is identified, and burst traffic intervals are obtained;

[0026] The request arrival rate range, service capacity range, and burst traffic range are merged to form an uncertainty range set, and a mapping relationship is established with the running operation set.

[0027] The constraint steps of the service level target constraint module include:

[0028] Based on the set of uncertainty intervals, a tail performance metric is selected as the service level target, and the tail performance metric includes at least one of percentile latency or availability default rate;

[0029] Quantitative constraints are set for the service level targets to limit the target range of latency or availability metrics;

[0030] The service level target is transformed into a probabilistic constraint to describe the performance requirements under a preset confidence level;

[0031] Stability constraints are introduced into the constraints to limit the dynamic fluctuations of the operating strategy;

[0032] The service level target constraints and stability constraints are unified to form a set of constraints for optimizing the operation strategy.

[0033] The optimization steps of the constraint reinforcement learning optimization module include:

[0034] Under the set of constraints, a running state space and an action space are defined. The state space includes request arrival rate, service rate, queue length, average latency, and resource utilization. The action space includes scheduling, degradation, caching, and traffic splitting operations.

[0035] The state space and action space are modeled using reinforcement learning, the action selection probability is generated using a policy network, and the state value is evaluated using a value network.

[0036] Construct a reward function that includes delay penalties, default penalties, and resource consumption to guide the optimization direction of the execution strategy;

[0037] Stability constraints are introduced during the strategy update process to ensure that the running strategy meets stability requirements during optimization.

[0038] Through iterative training until the convergence condition is met, an operational strategy is generated that can adaptively select a combination of scheduling, degradation, caching, or traffic splitting actions.

[0039] The security verification steps of the policy security verification module specifically include:

[0040] Formal verification of the operation strategy is performed, a verification model is established based on the mechanism model, and the operation strategy is verified to meet the service level objectives and stability constraints.

[0041] The operation strategy is simulated and verified in a simulation environment using historical or synthetic operational data to evaluate its performance under different load conditions.

[0042] In the production environment, perform canary-scale verification of the operation strategy, select a portion of traffic for strategy testing and collect real-time metrics;

[0043] Set a rollback threshold; when the running metrics exceed the preset threshold, an automatic rollback to the original strategy will be triggered.

[0044] When a running strategy passes formal verification, simulation verification, and grayscale verification simultaneously, the running strategy is determined to be a feasible running strategy.

[0045] The closed-loop update steps of the deployment and closed-loop update module include:

[0046] Deploy the validated operational strategies to the software operation system and implement them, using rolling updates, blue-green deployments, or canary expansion methods to achieve a smooth launch.

[0047] During the execution of the operational strategy, real-time operational data is continuously collected, including key indicators such as latency distribution, error rate, and throughput.

[0048] The mechanism model and uncertainty interval set are dynamically updated based on the collected real-time operating data, and the update is triggered periodically or when the indicators are abnormal.

[0049] The updated mechanism model and the set of uncertainty intervals are fed back into the reinforcement learning optimization process to iteratively generate new operating strategies, forming a continuous real-time analysis and optimization closed loop.

[0050] The beneficial effects of this invention are as follows:

[0051] This invention constructs a queuing-control mechanism model by collecting request arrival logs, service processing logs, and resource usage information. This model includes request arrival rate, service rate, concurrency, buffer capacity, and throttling thresholds. Furthermore, it forms a set of uncertainty intervals, dynamically reflecting the system's fluctuation characteristics under different loads. Compared to existing operation and maintenance methods that rely on static thresholds, this approach not only characterizes the system's average performance but also captures the impact of sudden traffic surges on the operational state, thus providing a more realistic and reliable modeling foundation for subsequent strategy optimization.

[0052] This invention selects percentile latency and availability default rate as service level targets under the premise of an uncertain interval set, and limits them through probabilistic and stability constraints, so that the operation strategy can still control tail performance under extremely high load. Compared with the existing technology that only focuses on average latency, this invention effectively reduces the risk of excessively high P95 and P99 latency and availability default, avoids service level agreement (SLA) default due to extreme traffic, and protects user experience and platform reputation.

[0053] This invention utilizes constrained reinforcement learning to generate operating policies and embeds mechanism models and stability constraints during policy evaluation and improvement. This ensures that policy optimization fully leverages the exploratory capabilities of machine learning while remaining true to the system's actual constraints. Compared to existing optimization schemes based solely on reinforcement learning, this invention avoids problems such as unstable training processes or unenforceable policy results. It can adaptively select combinations of actions such as scheduling, degradation, caching, and traffic splitting to achieve more intelligent and controllable policy optimization.

[0054] This invention establishes three levels of verification—formal verification, simulation verification, and gray-scale verification—before the formal deployment of the operational strategy, and combines these with preset rollback thresholds for security assurance. Formal verification ensures that the strategy logic conforms to stability constraints; simulation verification verifies the strategy's performance under historical or synthetic data; and gray-scale verification conducts low-traffic testing in a real environment, triggering automatic rollback when metrics become abnormal. Compared to existing "one-step deployment" methods, this effectively reduces the risk of system failures or large-scale performance fluctuations after strategy deployment. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the software operation data real-time analysis and intelligent optimization system module based on machine learning of the present invention;

[0056] Figure 2 This is a schematic diagram of the real-time analysis and intelligent optimization method for software operation data based on machine learning, as described in this invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1: As Figure 1 As shown, this embodiment provides a method for real-time analysis and intelligent optimization of software operation data based on machine learning, including the following steps:

[0059] S1. Collect the operating data of the software operation system, establish a mechanism model including request arrival rate, service rate, concurrency, buffer capacity and throttling threshold, and define the set of operation operations such as scheduling, degradation, caching and traffic splitting.

[0060] S2. Estimate the operational data based on the mechanism model to obtain the request rate range, service capacity range, and burst traffic range, forming an uncertainty range set, and establish a mapping relationship with the operational set.

[0061] S3. Based on the set of uncertainty intervals, select percentile delay or availability default rate as service level target, and introduce stability constraints to form a set of constraints for the operation strategy.

[0062] S4. Under the constraints, a running strategy is generated using reinforcement learning methods, which enables the strategy to adaptively select a combination of scheduling, degradation, caching, or diversion actions, and ensures that the strategy optimization process meets the constraints.

[0063] S5. Perform formal verification, simulation verification and grayscale verification on the operation strategy. Trigger rollback when the verification result does not meet the preset threshold.

[0064] S6. Closed-loop update: Deploy and execute verified operational strategies, and update the mechanism model and uncertainty interval set based on new operational data to achieve continuous real-time analysis and optimization closed loop.

[0065] S1 specifically includes the following sub-steps:

[0066] S110. Collect real-time operational data: Collect real-time operational data of the software operation system, including: Request arrival log: Records the timestamp, request type and request source of each request; Service processing log: Records the processing time, completion status and error code of each service node; Resource usage information: Including CPU utilization, memory utilization, disk I / O utilization and network bandwidth utilization.

[0067] Data collection method: Data is collected periodically using monitoring tools (such as Prometheus, ELK, or a self-developed monitoring agent). The sampling period is set to 1s–10s, and the sampling window length is set to 1min–5min to ensure the timeliness and representativeness of the data.

[0068] S120. Extract operating parameters: Based on the collected real-time operating data, extract the following operating parameters: Request arrival rate λ (req / s):

[0069]

[0070] in Request arrival rate, which represents the number of requests arriving at the system per unit time, is a core indicator for measuring system load intensity. This represents the total number of requests counted within the time interval Δt; Δt represents the time interval (e.g., 1 second, 1 minute, etc.). Therefore... This represents the average number of requests per unit of time.

[0071] In detail, the above formula is the foundation of queuing theory and performance modeling, used to calculate the arrival rate and service rate of a system. (The number of requests that can be processed per unit time) are used together to determine whether the system has the risk of queuing, congestion, and overload; when This indicates that the request arrival rate exceeds the system's processing capacity, leading to a continuous increase in queue length, increased response latency, and a risk of performance breach; for example: in If 6000 requests are received within 60 seconds, then =100req / s.

[0072] Service speed (req / s):

[0073]

[0074] in (Service Rate) represents the average processing capacity of a single service instance or a single "service channel", that is, the number of requests that can be completed per unit of time; This represents the total number of requests processed by the system within the time interval Δt; k represents the number of parallel service instances or servers; therefore This represents the average processing capacity of each service instance.

[0075] To elaborate: (Arrival rate) and The comparison of (total system processing capacity) determines whether the system is stable: if If the system has sufficient processing capacity, the queue will not grow indefinitely; if If the request arrival rate exceeds the processing capacity, the system will gradually become congested. Combined with... and It can be used to build queuing models (such as the M / M / k queue model) to evaluate key performance indicators such as latency, queue length, and drop rate.

[0076] For example, if k=6, and 7200 requests are completed within 60 seconds, then... =20req / s; Concurrency k (number of servers): obtained from the system deployment configuration file, e.g., k=6; Buffer capacity B (number of requests): set by queue parameters, e.g., the maximum message queue length is B=200; Throttling threshold (Dimensionless): Used to define the high water level of the load, initially set to... After stress testing and calibration, the range is 0.85k–0.95k to balance resource utilization and system stability.

[0077] S130. Establish a queuing-control mechanism model: Based on the operating parameters, establish an M / M / k / B queue model, assuming that: the request arrival process follows a Poisson distribution; the service time follows an exponential distribution; and the system supports a maximum of B requests in the queue.

[0078] The model metrics are calculated as follows: The Erlang-C formula is used to calculate the probability that a request needs to wait. :

[0079]

[0080] in It is the request arrival rate, which is the average number of requests entering the system per second; k is the service rate of a single service node; k is the number of parallel service nodes. It is the total system service rate; It is the queuing probability, that is, the probability that once a request arrives, it needs to enter the queue to wait;

[0081] The above formula measures the degree of congestion in the system: The larger the value, the more likely the request will need to wait, and the more congested the system is. The smaller the value, the more sufficient the system's processing capacity, and the more requests can be served directly; it is a fundamental parameter for calculating average waiting time and queue length.

[0082] Average queue length:

[0083]

[0084] in This represents the average number of queued requests in the system (i.e., the average number of requests waiting for service). Indicates the probability of queuing; It is the request delivery rate; The value of k represents the service rate of a single service node; k represents the number of parallel service nodes. This indicates the system's remaining processing capacity (i.e., "total service capacity arrival rate").

[0085] The above formula is the core result of the M / M / k queue model, used to calculate the expected value of the queue length; if A large number indicates a large number of requests queued in the system, suggesting a potentially long response time; if A value close to 0 indicates that most system requests can be served directly.

[0086] Average waiting time:

[0087]

[0088] in It is the average waiting time, that is, the average length of time a request needs to wait in the queue; It is the average number of queued requests in the system (calculated from the formula above); It is the effective arrival rate, that is, the rate at which requests actually enter the system and are served (usually equal to...). (If no request is dropped).

[0089] Average response time:

[0090]

[0091] in It represents the average system response time (also called average dwell time or average completion time), which is the total average time from when a user request enters the system to when the service is completed. This represents the average waiting time, which is the time it takes for a request to be queued. This represents the average service time, which is the average time it takes for a single request to be processed at a service node. The formula above indicates that: Average response time = Average waiting time + Average service time.

[0092] Drop probability:

[0093]

[0094] in Request drop rate represents the proportion of total requests that are rejected or dropped; that is, the proportion of incoming requests that are dropped or cannot be processed. This represents the total request arrival rate, which is the number of requests arriving at the system per unit of time. This represents the effective arrival rate, which is the number of requests that the system actually receives and processes. The effective processing rate is equal to the arrival rate multiplied by the "percentage of requests not dropped";

[0095] For example when =100req / s When =20req / s, k=6, B=200 ≈0.42, ≈12.5, ≈0.125s, ≈0.175s, ≈0.03; Those skilled in the art can calculate the system's operating performance indicators using formulas and examples.

[0096] S140. Define the set of runtime operations and triggering conditions: Define the set of runtime operations in the mechanism model, including scheduling, degradation, caching, and traffic splitting operations, and set the triggering conditions as follows:

[0097] Scheduling operation: Condition: When Triggered on time; Meaning: Redistribute requests to idle nodes to avoid overloading some nodes.

[0098] Caching operation: Condition: Triggered when queue length ≥ 0.7B and duration Δt ≥ 5s; Meaning: Temporarily store requests in a caching layer (such as Redis) to buffer burst traffic.

[0099] Offloading operation: Condition: Triggered when μ≤0.8λ / k; Meaning: Offload some requests to backup service or degrade channel.

[0100] Degradation operation: Conditions: Triggered when resource utilization is ≥90% and a high-risk surge in traffic is met; High-risk conditions: Surge rate R ≥ 50%, kurtosis K ≥ 3.5. Meaning: Reduce the service quality of non-critical functions to ensure core functions.

[0101] For example: when =110、 =20, k=6, B=200, calculate =5.5, while 0.9k=5.4, which satisfies the scheduling triggering condition.

[0102] S150. Forming a basic set of operational constraints: Binding the set of operational operations to operational parameters to form a basic set of operational constraints. For example, when Pd > 0.05, a traffic splitting operation must be triggered to reduce the drop rate; when Ws > 200ms, a degradation operation must be triggered to ensure the response time target. These constraints are supported by model formulas, threshold settings, and calculation examples, enabling the operational strategy to be executed dynamically and ensuring system stability.

[0103] S2 specifically includes the following sub-steps:

[0104] S210. Real-time operational data statistics: Based on the mechanism model, a sliding window method is used to statistically analyze real-time operational data: Window size T: 30s–300s, configurable, recommended value is 60s; Calculation content: Calculate the mean and standard deviation of the number of requests arriving within each window; Formula:

[0105]

[0106] in is the request arrival rate within the i-th observation window; n is the number of observation windows; This represents the average request delivery rate, which is the average value over n windows. Standard deviation is used to measure the fluctuation in request delivery rate; mean is used to measure the average. Reflects overall flow level; standard deviation Reflecting traffic fluctuations The larger the value, the more drastic the fluctuation in request arrival rate, and the more prone the system is to sudden pressure. For example, in a window of T=60s, collecting n=60 points, the average... =100req / s, standard deviation =10, then =100, =10.

[0107] S220. Calculate the request arrival rate interval: Calculate the confidence interval based on the statistical results, using a 95% confidence level.

[0108]

[0109] The above is a traffic estimation formula based on confidence intervals, used to construct statistical intervals for request delivery rates; This represents the average request delivery rate. 1.96 is the standard deviation, used to measure the fluctuation of request arrival rate; 1.96 is the coefficient under the standard normal distribution, corresponding to a 95% confidence level. This indicates the lower limit of the interval. Upper limit of the interval This means there is a 95% certainty that the actual average request delivery rate falls within this range. For example, when =100, When =10 and n=60:

[0110]

[0111] This means we have a 95% confidence level that the actual request delivery rate is 97.5%– Between req / s.

[0112] S230. Estimating Service Capacity Range: Predicting the trend of service rate μ using moving average and linear regression methods: Prediction time span: 1–5 min, recommended value 2 min; Update rule: If the prediction results show that μ will be in the future… If the internal rate decreases by ≥10%, then update. The lower limit of the interval is .

[0113] For example, the current =20req / s, the prediction result shows a 15% decrease within 2 minutes, then The interval is updated to [17, 20].

[0114] S240. Identify burst traffic characteristics: Perform distribution characteristic analysis on real-time operational data, using skewness S and kurtosis K: Skewness S:

[0115]

[0116] The above is the formula for calculating skewness, which measures the symmetry of the distribution of a random variable X; X represents a random variable (such as request arrival rate, response time, etc.). The mean of a random variable; The standard deviation of a random variable; This indicates the expectation operator.

[0117] When S>0 (positively skewed): the right tail of the distribution is longer, which means that most requests are faster, but a small number of requests have particularly long response times; S<0 (negatively skewed): the left tail of the distribution is longer, which means that most requests are slower, but a small number of requests are abnormally fast; S=0: the distribution is approximately symmetrical (such as a normal distribution).

[0118] S>1 indicates that the flow distribution is right-skewed; kurtosis K:

[0119]

[0120] The above is the formula for calculating kurtosis, which is used to measure the kurtosis and tail thickness of the distribution of a random variable X; X represents a random variable (such as request arrival rate, response time, etc.). The mean of a random variable; The standard deviation of a random variable; Represents the expectation operator;

[0121] K≈3: Close to a normal distribution, with moderate tail thickness (often called "mid-kurtosis"); K>3: Thick tails and high peaks, indicating a sharp distribution, with extreme values ​​(high delay / high flow) more likely to occur; K<3: Thin tails and low peaks, indicating a relatively flat distribution, with a lower probability of extreme values. In practical calculations, excess kurtosis = K–3 is usually used to directly represent the degree of tail thickness relative to a normal distribution.

[0122] K > 3.5 indicates the presence of a peaked distribution; spury rate R:

[0123]

[0124] The above is the sudden change rate of request arrival rate, which measures the magnitude of increase or decrease in request rate over a time interval. This represents the request arrival rate at the current time t; This represents the request arrival rate at the previous moment (or the previous observation window); Δt is the time interval; R is the rate of change (percentage); when R>0, it indicates that the request rate is increasing (a sudden surge in traffic); R<0 indicates that the request rate is decreasing (a drop in traffic); a large |R| indicates drastic fluctuations and the system is at risk of sudden traffic surges.

[0125] When R ≥ 50% and S > 1 and K > 3.5, it is determined to have entered a high-risk zone for sudden traffic flow; for example: in =Within 60s, If the value increases from 100 to 160, then R = 60%; if S = 1.2 and K = 4.0 simultaneously, it is considered a sudden high-risk event; Modeling method: The flow distribution in the high-risk area is fitted using a Gaussian mixture model (GMM), the parameter estimation method is the expectation-maximization (EM) algorithm, the maximum number of iterations is 100, and the convergence threshold is... .

[0126] S250, Forming a set of uncertainty intervals: Determining the request arrival rate intervals service range, service capacity range The intervals and burst flow intervals are merged to form a set of uncertain intervals; and a mapping relationship is established between the set of uncertain intervals and the set of operational processes.

[0127] For example: if The interval is [97.5, 102.5], and the actual value is... =101 (high); The range is [17, 20] (below the normal range of 20, considered low); the burst traffic is judged as normal; indicating that a traffic splitting operation has been triggered.

[0128] S3 specifically includes the following sub-steps:

[0129] S310. Selecting Tail Performance Metrics: Based on the set of uncertainty intervals, select tail performance metrics as service level targets. Tail performance metrics are used to measure the system's performance under extreme conditions and include at least: P95 latency: 95% of request response times do not exceed a threshold, typically ranging from 50 to 1000 ms; P99 latency: 99% of request response times do not exceed a threshold, typically ranging from 100 to 2000 ms; Availability default rate: The proportion of requests exceeding a preset latency threshold to the total number of requests, typically ranging from 0.01% to 1%.

[0130] For example, if 80 out of 10,000 requests have a response time exceeding 200ms, then the default rate is 0.8%.

[0131] S320: Set quantitative constraints: For tail performance metrics, set clear quantitative constraints: P99 latency ≤ 200ms; P95 latency ≤ 150ms; availability default rate ≤ 1%. These ranges are determined through stress testing and historical operational experience: In large-scale business systems (such as e-commerce flash sales), P99 latency exceeding 200ms will significantly impact user experience; a default rate higher than 1% is likely to trigger Service Level Agreement (SLA) compensation.

[0132] For example, if 80 out of 10,000 requests have a latency greater than 200ms, then the default rate is 0.8%, which satisfies the ≤1% constraint.

[0133] S330: Transform into opportunity constraint form: Convert the above service level objectives into probability constraints, i.e., opportunity constraints:

[0134]

[0135] The above is an opportunity constraint, commonly used in the mathematical expression of SLAs (Service Level Agreements) or reliability constraints. Delay represents the system response time or request completion time. This indicates the upper limit threshold for latency (e.g., 200ms); p represents the confidence level / probability guarantee (e.g., 0.99, indicating that 99% of requests will meet the requirements). It is a probability operator; it indicates that the system needs to guarantee that for at least p requests, the latency is less than or equal to a threshold. .

[0136] For example, when setting =200ms, p=0.99, if 19800 / 20000 requests in the simulation sample have a latency ≤200ms, then the constraint condition is satisfied:

[0137]

[0138] S340: Introducing stability constraints: Based on chance constraints, Lyapunov stability constraints are introduced, using the drift function. Measure the trend of system queue backlog:

[0139]

[0140] The above is an expected value of delay variation, used to measure the average trend of system delay variation between adjacent time periods; where This is the queue backlog function, reflecting the queue length or load level of the system at time t; Indicates the system delay at the next moment; Indicates the expected delay increment;

[0141] An increase in latency indicates increased system load or performance degradation. The decreasing latency indicates that the optimization strategy or scheduling measures are effective. The delay remains stable.

[0142] Regulation , The range was set to 0–0.05, determined through stress testing: when The system is stable when ≤0.03; when When the value is ≥0.07, the system enters an unstable state; therefore =0.05 was selected as the safety threshold; for example, in a test, when =120, =123, then =3; if the system's average queue backlog is 100, then =0.03, which satisfies the stability constraint.

[0143] S350: Unifying the Constraint Set: Unifying opportunity constraints and stability constraints to form a set of constraints for optimizing the operational strategy, using a weighted function:

[0144]

[0145] in This indicates the degree to which the system meets the service level objectives; if the actual delay probability is higher than the expected p, this item is positive, increasing the reward; if it is lower than p, this item is negative, incurring a penalty. Stability constraints representing system delay; It is the trend of delay. This is the allowable threshold; if the delay fluctuation is less than the threshold, it indicates stability, and this term is positive; if the fluctuation is too large, it indicates instability, and this term is negative. F is a comprehensive objective function / reward function; and F is a weight parameter, ranging from 0 to 1, used to balance "satisfying SLA" and "maintaining stability"; the larger F is, the better the current strategy is.

[0146] When F≥0, the operating strategy is deemed feasible. For example, let's assume... =0.6, =0.4, =200ms, p=0.99; Actual measurement (Delay ≤ 200ms) = 0.995 =0.03, =0.05; then F=0.011>0; therefore, the running strategy satisfies the optimization constraints.

[0147] S4 specifically includes the following sub-steps:

[0148] S410: Define the state space and action space: Initialize the reinforcement learning method under the set of constraints for running the policy optimization: State space S: includes the following state variables: request arrival rate (req / s); Service rate (req / s); Queue length Q (number of requests); Average latency D (ms); Resource utilization U (%). Action space A: Corresponding set of operations: Scheduling (redistributing requests to different service nodes); Degradation (reducing the service quality of non-critical functions); Caching (temporarily storing requests in the cache layer); Traffic splitting (transferring requests to backup nodes or other channels).

[0149] For example when =120req / s When the response rate is 20req / s, Q=180, D=220ms, and U=92%, the state is s=(120,20,180,220,92%). The action space can be selected as A={scheduling, degradation, buffering, and offloading}.

[0150] S420, Employing the Actor-Critic reinforcement learning method: Policy Network Input state s, output the probability of choosing each action a; value network : Input state s, output the value estimate of that state.

[0151] In detail, both the policy network and the value network adopt a three-layer fully connected neural network structure: Input layer: Input vector s, including the request arrival rate Service speed The algorithm has five dimensions: queue length Q, average latency D, and resource utilization U; hidden layers: 128 neurons per layer, using ReLU activation function; output layer: policy network. Output the probability distribution of the action space A = {scheduling, degradation, buffering, offloading}, using Softmax activation; value network. Output a single scalar value for this state, with a linear activation function.

[0152] The training conditions are set as follows: the number of iterations is no less than 5000; when the average reward fluctuation of 100 consecutive iterations is less than 1% and the stability constraint ΔL ≤ 0.05, it is considered to have converged.

[0153] In the TensorFlow implementation, the learner Adam has a learning rate of 0.001 and a discount factor of [missing information]. =0.95, experience replay buffer size 10 4The batch size is 64. Experiments have verified that the above parameters can achieve convergence in a simulation environment, resulting in a stable operating strategy.

[0154] S430: Define the policy reward function: The policy reward consists of three parts: delay penalty, default penalty, and resource consumption.

[0155]

[0156] The above is a negative form of a reward function (penalty function), used for reinforcement learning or optimization decision-making; R represents the reward value; latency penalty: a penalty term generated when the request response time is too high; breach penalty: a penalty term when the SLA (such as P99 latency ≤200ms) is not met; resource consumption: the resource cost occupied during operation (such as CPU, memory, bandwidth, etc.). , , These are weight parameters that control the importance of the three penalty terms respectively; the negative sign before the formula indicates that the reward function R will decrease due to the increase in delay, default, and resource consumption.

[0157] If delays and defaults are severe, the R value becomes very small or even negative, indicating that the strategy is not advisable; if delays are low, defaults are few, and resource utilization is reasonable, the R value is large, indicating that the strategy is excellent.

[0158] Delay penalty: Delay penalty = D is the average delay. Latency threshold (e.g., 200ms); Default penalty: Default rate × 100; Resource consumption: Weighted sum of CPU and memory utilization; Weight parameters , , The range is [0,1], recommended value: =0.5, =0.3, =0.2. For example, D=250ms, =200ms, delay penalty=50; default rate=0.5%, default penalty=0.5; resource consumption=70%; then R=-(0.5×50+0.3×0.5+0.2×0.7)=-25.29.

[0159] S440: Embedded stability constraints: When updating the policy gradient, the Lyapunov drift function ΔL is introduced as a constraint term.

[0160]

[0161] The above is a policy gradient update formula with stability constraints, used for policy optimization in reinforcement learning; The objective function J represents the policy parameters. The gradient (i.e., the policy update direction); This represents the gradient of the reward function R with respect to the policy parameters (the regular policy gradient). The weighting coefficients for stability constraints range from 0.01 to 0.1, with recommended values. =0.05; It is the delay change (the expected delay difference, which has been defined in the previous formula).

[0162] Conventional policy gradient methods only consider the reward function R; the above methods further subtract from this. This indicates that if the policy causes a continuous increase in latency (ΔL>0), it will negatively impact the update direction, thereby suppressing unstable policies. In detail, the above formula is a correction term that incorporates stability constraints into reinforcement learning.

[0163] S450: Iterative Training and Convergence Criterion: The training process combines online interaction with a simulation environment, with the following specific conditions: Number of iterations: no less than 5000; Convergence Condition 1: Average reward fluctuation over 100 consecutive iterations <1%; Convergence Condition 2: Constraints are satisfied. and .

[0164] For example: after 5200 training rounds, the reward value fluctuation range is 0.5% over 100 consecutive iterations, satisfying convergence condition one; simultaneously detecting... All satisfy condition two → the training of the judgment strategy has converged; the final running strategy is an adaptive combination of scheduling, degradation, caching and diversion operations, which can be dynamically switched in different states.

[0165] S5 specifically includes the following sub-steps:

[0166] S510. Formal Verification: Abstract the operating strategy into a Markov Decision Process (MDP) model, defined as a quadruple:

[0167]

[0168] Where S represents the state space, describing all possible states the system can be in, and S includes the request arrival rate. Service speed , queue length Q, resource utilization U, etc.; A represents the action space, which is the operations that the system can take in a certain state. In this embodiment, A is the set of running operations such as scheduling, degradation, caching, and traffic splitting; P represents the state transition probability, which describes the probability that the system transfers from one state to another after taking a certain action. For example: Taking the "traffic splitting" action in a high-load state may increase the probability of the delay state transferring to the "normal range"; R is the reward function, which describes the reward or punishment obtained by the system after taking a certain action. In this embodiment, R is related to delay constraints, stability constraints, resource consumption, etc.

[0169] MDP is the mathematical basis for reinforcement learning modeling; it formalizes the software operation optimization problem into a closed-loop process of "state - action, transition, reward"; the goal of the algorithm is to find a policy that selects an appropriate action sequence in the sense of maximizing the long-term return expectation.

[0170] Verification tool: Use the PRISM probabilistic model checking tool to write a model file (in.prism format) and define properties:

[0171]

[0172] The above describes how to use the PRISM probabilistic model checking tool for formal verification to ensure that the running policy meets the service level objective constraints and stability constraints.

[0173] Among them represents "the probability that the final system delay does not exceed is greater than or equal to 0.99"; for example, ensuring that 99% of the request delays ≤ 200ms (i.e., P99 SLA).

[0174] represents "during the entire operation, the delay change remains within with a probability greater than or equal to 0.95"; that is, ensuring that the delay change is controllable 95% of the time, reflecting the system stability. The logical AND means that only when both the SLA constraint and the stability constraint are satisfied can it be considered qualified.

[0175] Among them = 200ms, = 0.05; for example, running the verification in the PRISM tool, the output result is: Pr(delay ≤ 200ms) = 0.992 ≥ 0.99; Pr(ΔL ≤ 0.05) = 0.962 ≥ 0.95; meeting the service level objective and stability constraints.

[0176] S520: Simulation verification: Verify the running policy in the simulation environment using the historical log replay method: Data scale: at least include 105 Request records; Coverage time: 1h–24h; Simulation method: Input log data one by one into the mechanism model to calculate latency distribution, drop probability and availability index.

[0177] For example, in 200,000 request logs, the statistical results are as follows: P99 latency = 180ms (less than the threshold of 200ms); drop rate = 0.8% (less than 1%); availability breach rate = 0.7% (less than the SLA limit of 1%). Conclusion: The strategy meets the simulation verification requirements.

[0178] S530: Gray-scale Validation: Deploy and run the strategy in the production environment, selecting 1%–5% of system traffic as gray-scale traffic, with a recommended value of 2%. Gray-scale validation period: 30 minutes–2 hours; Real-time collected metrics: average latency, error rate, throughput. For example, running for 60 minutes with a gray-scale traffic ratio of 2%, the collected results are: average latency 195ms; error rate 1.5%; throughput consistent with the baseline (fluctuation ≤2%); Conclusion: Gray-scale validation results are qualified.

[0179] S540. Set Rollback Threshold: To ensure the safety of the verification process, the following rollback threshold is set: When the average latency increases by ≥5%; or the error rate increases by ≥2%; or the throughput decreases by ≥10%, an automatic rollback is immediately triggered, restoring the system to the old strategy. For example, in the 20th minute of the gray-scale test, if the average latency increases from 190ms to 210ms, the increase is (210-190) / 190 = 10.5% > 5%, and the system triggers the rollback mechanism to restore the original strategy.

[0180] S550: Strategy Qualification Criteria: A strategy is deemed feasible if it meets preset conditions in formal verification, simulation verification, and grayscale verification. The criteria are as follows:

[0181] Formal verification: Pr(delay≤200ms)≥0.99; Pr(ΔL≤0.05)≥0.95.

[0182] Simulation verification: P99 latency ≤ 200ms; availability default rate ≤ 1%; drop rate ≤ 1%.

[0183] Gray-scale validation: Average latency increase <5%; error rate increase <2%; throughput decrease <10%; and it remains stable for 30 consecutive minutes.

[0184] For example, the results of a certain strategy verification are as follows: Formal verification: passed; Simulation verification: P99 latency = 180ms, default rate = 0.7%, drop rate = 0.8%; Gray-scale verification: latency increased by 3%, error rate increased by 1.2%, throughput decreased by 2%. → The strategy is determined to be a feasible operational strategy.

[0185] S6 specifically includes the following sub-steps:

[0186] S610: Deployment and Operation Strategy: Deploy the operational strategy, which has passed formalization, simulation, and canary release verification, to the production environment, employing multiple rollback-capable deployment methods: Rolling Update: Replace service instances in batches, with a typical batch size of 5%–20%; Blue-Green Deployment: Maintain both the old and new versions simultaneously, allowing for one-click rollback when switching traffic routes; Canary Release: Gradually increase the traffic share of the new strategy under the existing strategy, ranging from 1%–20%. Tool Support: Can be implemented using Kubernetes, DockerSwarm, or a self-developed scheduling platform. For example, in a Kubernetes cluster, rolling updates are used, replacing 10% of instances in each batch to ensure 90% of nodes remain available, with an update completion time of approximately 5 minutes.

[0187] S620: Collect real-time operational data: During the execution of the operational strategy, continuously collect the following metrics: Latency distribution D (ms): including average latency, P95 latency, and P99 latency;

[0188] Error rate E (%): Calculated as follows

[0189]

[0190] Where E is the error rate, representing the proportion of failed requests in the system; Error Requests: the number of requests that returned incorrect results or failed to process within the monitoring period; Total Requests: the total number of requests received by the system within the same time period. A lower E indicates better system stability and reliability; a higher E indicates more errors during system operation and poorer request processing quality. Generally, SLAs have requirements for the error rate, such as <0.1%.

[0191] Throughput T (req / s): The number of requests completed per unit of time.

[0192] Data collection method: Monitoring is performed using Prometheus + Grafana, or real-time collection is achieved through the ELK logging system. Sampling frequency: 1s–10s; Window length: 1min–5min. For example, results collected in a 1min window: P99 latency = 190ms, error rate = 0.6%, throughput = 950req / s.

[0193] S630: Update Mechanism Model and Uncertainty Interval Set: Dynamically update the queuing-control mechanism model and uncertainty interval set based on real-time collected data: Periodic update: periodic range 5-10 min, recommended value 10 min; event-triggered update: update immediately when one of the following conditions is met: average latency > 200 ms; error rate > 1%; throughput decrease > 10%.

[0194] Update method: Recalculate Parameters, correct model formulas, update interval, Intervals and burst traffic intervals. For example, if the current average latency increases from 180ms to 220ms, an event update is triggered, the queue model is recalculated, and the triggering conditions are adjusted.

[0195] S640: Closed-Loop Optimization and Convergence Determination: The updated mechanism model and uncertainty interval set are fed back to the reinforcement learning method to enter the next iteration cycle: Iteration cycle: 10–30 min, recommended value 15 min; Optimization process: Reinforcement learning retrains and runs the policy, embedding the latest constraint set; Convergence criterion: When the fluctuation range of P99 latency, error rate, and throughput is all <1% within three consecutive iteration cycles, the optimization process is determined to have converged; For example, within three consecutive 15-min iteration cycles: P99 latency fluctuation: 190ms→192ms→191ms (fluctuation range 1%); Error rate fluctuation: 0.7%→0.8%→0.7% (fluctuation range 0.1%); Throughput fluctuation: 950→948→952req / s (fluctuation range 0.4%). → If the convergence criterion is met, the closed-loop optimization of the policy is determined to be complete.

[0196] Example 2: This example provides a real-time analysis and intelligent optimization system for software operation data based on machine learning, including:

[0197] The data acquisition module is used to acquire real-time operational data of the software operating system. The operational data includes request arrival logs, service processing logs, and resource usage information.

[0198] The mechanism model construction module is used to build a queuing-control mechanism model based on real-time running data. The mechanism model includes running constraints such as request arrival rate, service rate, concurrency, buffer capacity and throttling threshold, and defines a set of running operations in the mechanism model. The set of running operations includes at least scheduling, degradation, caching and diversion operations.

[0199] The operational status estimation module is used to estimate real-time operational data based on the mechanism model to obtain the request arrival rate range, service capacity range, and burst traffic range, forming an uncertainty range set. The uncertainty range set is then mapped to the operational set to define the state range of feasible operational strategies.

[0200] The service level target constraint module is used to select tail performance indicators as service level targets based on the set of uncertainty intervals. The tail performance indicators include at least one of percentile latency or availability default rate, and are combined with stability constraints to form a set of constraints to limit the search space of the operation strategy.

[0201] The constraint reinforcement learning optimization module is used to generate running strategies using reinforcement learning methods under the action of a set of constraints. The running strategies can adaptively select a combination of scheduling, degradation, caching, or diversion actions from the set of running operations based on the state. By embedding the mechanism model into the strategy evaluation and improvement process, it is ensured that the running strategies continuously meet the constraints during the optimization process.

[0202] The policy security verification module is used to perform formal verification, simulation verification and gray-scale verification of the running policy, to check whether the running policy meets the service level target and stability constraints within the set of uncertainty intervals, and to trigger automatic rollback when the preset rollback threshold is exceeded.

[0203] The deployment and closed-loop update module is used to deploy the verified and qualified operation strategy to the software operation system for execution, collect the execution results of the operation strategy as new real-time operation data, and update the uncertainty interval set and reinforcement learning optimization process based on the mechanism model, thereby forming a continuous real-time analysis and optimization closed loop for the operation strategy.

[0204] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0205] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0207] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time analysis and intelligent optimization system for software operation data based on machine learning, characterized in that: include: The data acquisition module is used to acquire real-time operating data of the software operation system, including request arrival logs, service processing logs, and resource usage information. The mechanism model construction module is used to establish a queuing control mechanism model based on the real-time running data, and to define a set of running operations in the mechanism model; The operation status estimation module is used to estimate real-time operation data based on the mechanism model to obtain the request arrival rate range, service capacity range, and burst traffic range, forming a set of uncertainty ranges; The service level target constraint module is used to select tail performance indicators as service level targets based on the set of uncertainty intervals, and combine them with stability constraints to form a set of constraints. The constraint reinforcement learning optimization module is used to generate operating strategies using reinforcement learning methods under the set of constraints, and to embed the mechanism model into the strategy evaluation and improvement process. Specifically, this includes defining a running state space and an action space under the set of constraints. The state space includes request arrival rate, service rate, queue length, average latency, and resource utilization. The action space includes scheduling, degradation, caching, and traffic splitting operations. The state space and action space are modeled using reinforcement learning, the action selection probability is generated using a policy network, and the state value is evaluated using a value network. Construct a reward function that includes delay penalties, default penalties, and resource consumption to guide the optimization direction of the execution strategy; Introducing stability constraints during policy gradient updates will use a drift function based on a mechanism model to measure the trend of queue backlog changes. Introducing the constraint term, the update formula for the policy gradient is: in The gradient of the objective function with respect to the policy parameters. The gradient of the reward function with respect to the policy parameters. The weighting coefficients are used to define stability constraints, thereby embedding the mechanism model into the policy evaluation and improvement process. Through iterative training until the convergence condition is met, a running strategy that can adaptively select a combination of scheduling, degradation, caching, or traffic splitting actions is generated. The strategy security verification module is used to perform formal verification, simulation verification, and grayscale verification on the operation strategy to check whether the operation strategy meets the service level target and stability constraints within the set of uncertainty intervals.

2. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 1, characterized in that, It also includes a deployment and closed-loop update module, which is used to deploy the verified and qualified operation strategy to the software operation system for execution, collect the execution results of the operation strategy as new real-time operation data, and form a closed loop for continuous real-time analysis and optimization of the operation strategy.

3. The steps for establishing a queuing control mechanism model in the real-time analysis and intelligent optimization system for software operation data based on machine learning as described in claim 1 include: Collect real-time operational data of the software operation system, including request arrival logs, service processing logs, and resource usage information; Based on the aforementioned operational data, operational parameters such as request arrival rate, service rate, concurrency, buffer capacity, and throttling threshold are extracted. A queuing control mechanism model is established based on the aforementioned operating parameters to characterize the relationship between request arrival, service processing, and queuing. The mechanism model defines a set of running operations, which includes at least scheduling, degradation, caching, and traffic splitting operations, and sets trigger conditions for each operation. The running parameters are bound to the set of running operations to form a basic set of running constraints.

4. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 1, characterized in that, The estimation steps of the operating state estimation module include: Based on the aforementioned mechanism model, real-time operational data is statistically analyzed to obtain the average and fluctuation of request arrival rates. The request arrival rate range is calculated based on the statistical results to reflect the normal fluctuation range of request traffic. By analyzing changes in service processing capacity, a service capacity range can be determined. Based on distribution characteristics, burst traffic is identified, and burst traffic intervals are obtained; The request arrival rate range, service capacity range, and burst traffic range are merged to form an uncertainty range set, and a mapping relationship is established with the running operation set.

5. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 1, characterized in that, The constraint steps of the service level target constraint module include: Based on the set of uncertainty intervals, a tail performance metric is selected as the service level target, and the tail performance metric includes at least one of percentile latency or availability default rate; Quantitative constraints are set for the service level targets to limit the target range of latency or availability metrics; The service level target is transformed into a probabilistic constraint to describe the performance requirements under a preset confidence level; Stability constraints are introduced into the constraints to limit the dynamic fluctuations of the operating strategy; The service level target constraints and stability constraints are unified to form a set of constraints for optimizing the operation strategy.

6. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 1, characterized in that, The security verification steps of the policy security verification module specifically include: Formal verification of the operation strategy is performed, a verification model is established based on the mechanism model, and the operation strategy is verified to meet the service level objectives and stability constraints. The operation strategy is simulated and verified in a simulation environment using historical or synthetic operational data to evaluate its performance under different load conditions. In the production environment, perform canary testing of the operational strategy, select a portion of traffic for strategy testing, and collect real-time metrics.

7. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 6, characterized in that, Also includes: Set a rollback threshold; when the running metrics exceed the preset threshold, an automatic rollback to the original strategy will be triggered. When a running strategy passes formal verification, simulation verification, and grayscale verification simultaneously, the running strategy is determined to be a feasible running strategy.

8. The real-time analysis and intelligent optimization system for software operation data based on machine learning according to claim 2, characterized in that, The closed-loop update steps of the deployment and closed-loop update module include: Deploy the validated operational strategies to the software operation system and implement them, using rolling updates, blue-green deployments, or canary expansion methods to achieve a smooth launch. During the execution of the operational strategy, real-time operational data is continuously collected, including key indicators such as latency distribution, error rate, and throughput. The mechanism model and uncertainty interval set are dynamically updated based on the collected real-time operating data, and the update is triggered periodically or when the indicators are abnormal. The updated mechanism model and the set of uncertainty intervals are fed back into the reinforcement learning optimization process to iteratively generate new operating strategies, forming a continuous real-time analysis and optimization closed loop.

Images