A dynamic adaptive replanning method, system, and product for intelligent applications

Abstract

This invention relates to the field of computing resource allocation technology, specifically disclosing a dynamic adaptive replanning method, system, and product for intelligent applications. The method includes: continuously monitoring at least one state dimension of a running intelligent application to obtain real-time runtime state data; based on the runtime state data, predicting the cumulative deviation cost that will result from maintaining the current execution plan within a preset time window, and estimating the one-time overhead required to perform a replanning operation; when the predicted cumulative deviation cost exceeds the one-time overhead, triggering a replanning operation on the current execution plan to invoke the global planner, and providing the real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan adapted to the current state; smoothly migrating the intelligent application from the current execution plan to the new execution plan; and improving the performance stability and resource utilization efficiency of the intelligent application.

Description

Technical Field

[0001] This invention relates to the field of computing resource allocation technology, and in particular to a dynamic adaptive replanning method, system, and product for intelligent applications. Background Technology

[0002] In existing technologies, for intelligent applications such as real-time risk control, streaming data analysis, or online services, an optimal execution plan is typically generated based on static information before execution. This plan considers the application's logical structure, physical implementation, and initial Service Level Objectives (SLOs), including dimensions such as latency, cost, and result quality. However, because these applications often need to run continuously, external environmental factors such as API performance fluctuations, changes in input data characteristics, or adjustments to business priorities can cause the actual operating state to deviate from the initial assumptions, leading to problems such as performance degradation, cost overruns, or substandard quality.

[0003] Existing technologies mainly include the following solutions: First, reactive autoscaling based on technical metrics, such as Kubernetes horizontal Pod Autoscaling (HPA) and event-based autoscaling (KEDA). These solutions dynamically adjust the number of service instances by monitoring metrics such as CPU utilization and queue length, achieving resource expansion or contraction. Second, the adaptive capabilities of mainstream big data stream processing frameworks, such as Apache Flink or Spark Streaming. These frameworks optimize resource efficiency for data processing tasks and support a certain degree of runtime adjustment, such as task redistribution or load balancing. Third, static optimization techniques for execution plans, which generate an initial plan before application startup, considering multi-dimensional constraints to achieve optimal configuration. Fourth, adaptive decision-making methods based on reinforcement learning, which model system optimization as an intelligent agent, and adjust to the dynamic environment through trial and error learning strategies.

[0004] However, these existing technologies have significant drawbacks. First, reactive autoscaling decision-making is limited to a single dimension, focusing only on technical metrics and failing to fully understand multi-dimensional business SLOs. For example, it cannot balance complex trade-offs between cost and quality, and its actions are limited to instance scaling, unable to fundamentally change the application structure. Second, while big data frameworks support adaptability, they are designed for general data processing, not for orchestrating heterogeneous AI services. They lack awareness of AI-specific costs (such as token-based pricing) or semantic quality, and cannot handle the unique challenges of generative AI. Third, static optimization is limited to the initial stage and cannot cope with dynamic changes during runtime. Once the environment drifts, the plan quickly becomes ineffective, and there is no mechanism for continuous adjustment. Finally, while reinforcement learning methods are cutting-edge, the decision-making process is black-boxed and lacks interpretability; the reward function is difficult to design, easily leading to deviations from business objectives; furthermore, training costs are high, and initial exploratory actions may disrupt online services, making it difficult to implement in production environments.

[0005] In summary, existing technologies are insufficient to achieve comprehensive dynamic adaptation to long-running intelligent applications, particularly in terms of multi-dimensional SLO monitoring, drift detection, and global adjustment. This makes it difficult for the system to maintain a long-term optimal state, and a more effective solution is urgently needed to overcome these limitations. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by proposing a dynamic adaptive replanning method, system, and product for intelligent applications. To achieve the above objective, the embodiments of this invention employ the following technical solutions:

[0007] In a first aspect, embodiments of the present invention propose a dynamic adaptive replanning method for intelligent applications, comprising the following steps:

[0008] Continuously monitor at least one state dimension of a running intelligent application to obtain real-time runtime state data, wherein the state dimension includes: end-to-end cost of the application, semantic quality of the results produced by the application, or data distribution drift of the application input data.

[0009] Based on the runtime state data, predict the cumulative deviation cost that will result from maintaining the current execution plan within a preset time window, and estimate the one-time cost required to perform a replanning.

[0010] When the cumulative deviation cost exceeds the one-time overhead, a replanning of the current execution plan is triggered to invoke the global planner and provide real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan adapted to the current state.

[0011] Smoothly migrate intelligent applications from the current execution plan to the new execution plan.

[0012] Preferably, the state dimension also includes: performance metrics for applying a single physical operator, including response time or error rate.

[0013] Preferably, the data distribution drift of the input data includes: based on real-time monitoring of the distribution characteristics of the input data, when a significant deviation of the distribution characteristics from a preset benchmark distribution is detected, determining the data distribution drift.

[0014] Preferably, the cumulative deviation cost includes: quantified business losses due to cost overruns, processing delays exceeding service level targets, or decreased result quality.

[0015] Preferably, the one-time overhead includes at least one of the following: the computational overhead of calling the global planner to generate a new plan, the instantaneous service jitter overhead caused by the execution plan migration, and the resource migration overhead.

[0016] Preferably, before smoothly migrating the intelligent application from the current execution plan to the new execution plan, the process further includes:

[0017] Compare the current execution plan with the new execution plan to identify the specific operator nodes that need to be changed;

[0018] When the specific operator node to be changed is a stateless operator, a new resource is created for the new execution plan, and the newly flowing data is directed to the new resource through a traffic switching mechanism to achieve blue-green deployment or canary release migration.

[0019] Preferably, when the specific operator node to be changed is a stateful operator, the current state data of the stateful operator in the current execution plan is migrated to the corresponding operator in the new execution plan through the checkpoint mechanism, so as to achieve stateful hot migration.

[0020] Preferably, the intelligent application is a streaming intelligent application that needs to run for a long time, including real-time risk control, streaming data analysis, or online services.

[0021] Secondly, embodiments of the present invention propose a dynamic adaptive replanning system for intelligent applications for executing the above-described dynamic adaptive replanning method for intelligent applications, comprising:

[0022] The runtime state data module is used to continuously monitor at least one state dimension of the running intelligent application to obtain real-time runtime state data. The state dimensions include: end-to-end cost of the application, semantic quality of the results produced by the application, or data distribution drift of the application input data.

[0023] The cost assessment module is used to predict the cumulative deviation cost that will result from maintaining the current execution plan within a preset time window based on the runtime state data, and to estimate the one-time cost required to perform a replanning.

[0024] The online replanning module is used to trigger a replanning of the current execution plan when the cumulative deviation cost of the prediction exceeds the one-time overhead. This involves calling the global planner and providing real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan that adapts to the current state.

[0025] The online migration module is used to smoothly migrate intelligent applications from the current execution plan to a new execution plan.

[0026] Thirdly, embodiments of the present invention provide a computer program product including a non-transitory computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, cause the processor to execute the aforementioned dynamic adaptive replanning method for intelligent applications.

[0027] Beneficial effects:

[0028] This invention significantly improves the performance stability and resource utilization efficiency of long-running applications through real-time monitoring and intelligent decision-making mechanisms of the application's operational status. This method overcomes the limitations of static planning in existing technologies, which cannot adapt to dynamic changes. It proactively triggers global replanning upon detecting state drift, ensuring the application continuously approaches its optimal execution state, thereby effectively avoiding performance degradation, cost overruns, or substandard quality. Specifically, this method achieves a shift from passive adjustment to proactive strategic optimization. By predicting and comparing the cumulative deviation cost with the replanning overhead, a transparent white-box decision-making process is formed, avoiding the shortcomings of traditional methods such as single decision dimensions and unexplainable black-box approaches, thus improving system reliability and auditability. Simultaneously, using real-time status data as planning input ensures that the new execution plan is more adaptable to the current environment, significantly improving the satisfaction of business SLOs, such as better balancing cost and result quality. Furthermore, this method introduces a smooth migration strategy, ensuring the continuity of streaming applications, reducing service interruption risks, and is suitable for scenarios such as real-time risk control or online services. Attached Figure Description

[0029] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0030] Figure 1 This is a flowchart illustrating a dynamic adaptive replanning method for intelligent applications provided in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the structure of a dynamic adaptive replanning system for intelligent applications provided in an embodiment of the present invention. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0033] For the first aspect, please refer to... Figure 1This invention proposes a dynamic adaptive replanning method for intelligent applications, applicable to a dynamic adaptive replanning system for intelligent applications. The dynamic adaptive replanning system for intelligent applications can be executed by, but is not limited to, computer devices with certain computing resources, such as personal computers (PCs, which are multi-purpose computers suitable for personal use in terms of size, price, and performance; desktops, laptops, mini-laptops, tablets, and ultrabooks all fall under the category of PCs), smartphones, personal digital assistants (PADs), or platform servers. This embodiment provides a dynamic adaptive replanning method for intelligent applications, aiming to solve the problems of static planning in existing technologies, such as inability to adapt to dynamic changes during runtime, limited decision dimensions, and lack of awareness of multi-dimensional business SLOs.

[0034] This method achieves dynamic adaptive adjustment of long-running intelligent applications by continuously monitoring multi-dimensional state dimensions, triggering intelligent decisions based on cost-benefit analysis, online replanning, and a smooth migration mechanism, ensuring that the application maintains its optimal operating state throughout its entire lifecycle. The specific implementation steps of this embodiment are described in detail below. These steps are implemented based on a Runtime Adaptation Controller (AAC), which acts as the upper-level control brain, periodically or event-drivenly executing a complete intelligent control closed loop. Its core modules include an AI-SLO monitoring module, a cost-benefit-based replanning triggering module, an online replanning module, and a smooth online migration module for execution plans. These modules work together to form a closed-loop system of "monitoring-detection-decision-execution".

[0035] Step S1: Continuously monitor at least one state dimension of the running intelligent application to obtain real-time runtime state data, wherein the state dimension includes: end-to-end cost of the application, semantic quality of the results produced by the application, or data distribution drift of the application input data.

[0036] In this embodiment, intelligent applications specifically refer to a collection of AI operators and general computing tasks orchestrated to achieve specific business objectives. Their structure is typically represented as a directed acyclic graph (DAG). Particular attention is paid to streaming intelligent applications that require long-term operation, such as real-time risk control systems, streaming data analytics platforms, or online service applications. During operation, external environmental factors such as API performance fluctuations, changes in input data characteristics, or adjustments to business priorities can cause state drift, meaning that the actual system performance indicators or the inherent characteristics of the input data stream significantly deviate from the expected state upon which the initial execution plan was based.

[0037] To detect these changes in real time, this step deploys the AI-SLO monitoring module as the system's "eyes," responsible for continuously and in real-time collecting multi-dimensional status data directly related to business objectives. Service Level Objectives (SLOs) are quantitative descriptions of the business goals that intelligent applications must meet during operation, including at least latency, cost, and quality. The monitoring process uses a lightweight agent deployed in the application execution environment to collect data periodically (e.g., every 5 seconds) through hook functions or API calls.

[0038] Specifically, end-to-end cost monitoring involves aggregating the monetary consumption of all physical operators. For example, for APIs using Large Language Models (LLMs) priced by token (such as GPT-4 or Llama3 models), the system calculates the cumulative cost by real-time statistics of token usage multiplied by the unit price. If the initial SLO sets a cost cap of $5 / hour, the monitoring module tracks the actual cost curve and compares it to the target. Semantic quality monitoring of application outputs targets generative AI tasks, using automated evaluation metrics such as BLEU scores (for translation tasks) or semantic similarity (based on the cosine similarity between the output and the reference result calculated using an embedding model such as BERT). For example, in real-time risk control applications, the system monitors the accuracy and recall of fraud detection; if the quality drops below a threshold (e.g., accuracy below 95%), it is marked as a state deviation.

[0039] Monitoring the data distribution drift of the application's input data is crucial, and statistical methods are employed to monitor the distribution characteristics of the input data in real time. Preferably, the data distribution drift of the application's input data includes: based on real-time monitoring of the input data's distribution characteristics, determining data distribution drift when a significant deviation from a preset benchmark distribution is detected. Specifically, the Kolmogorov-Smirnov (KS) test or Wasserstein distance is used as a metric. The preset benchmark distribution is based on the statistical characteristics of the initial training data or the previous period, such as the mean, variance, or histogram distribution of the input data. For streaming data analytics applications, the input data may be real-time sensor data streams. If the mean of feature X is detected to drift from 0.5 to 0.8 (KS statistic > 0.1), it is determined to be data distribution drift. This can lead to performance degradation of AI models, such as a decrease in click-through rates for recommendation models in online services.

[0040] Preferably, the state dimension further includes performance metrics for individual physical operators, such as response time or error rate. For more granular monitoring, the module also tracks the performance of individual physical operators (e.g., model APIs) that make up the application. For example, for a digest operator using the GPT-4 API, its average response time (time from request to response reception) and error rate (percentage of failed requests) are monitored. If the response time increases from 200ms to 500ms, or the error rate exceeds 5%, it is considered a degradation in physical operator performance, possibly caused by fluctuations in external APIs. Monitoring data is collected using tools such as Prometheus and aggregated in a central storage to form a real-time runtime state data stream. This data is stored in JSON format, such as {"cost":4.2,"quality":0.92,"data_drift_ks":0.12,"api_latency":450,"error_rate":0.03}, for use in subsequent steps.

[0041] Through the aforementioned monitoring mechanism, this step ensures the system's sensitive perception of dynamic changes, avoiding the limitations of existing technologies that rely solely on technical metrics (such as CPU utilization). In actual deployment, for a real-time risk control application, the monitoring module runs in a Kubernetes cluster, generating a status report every minute and supporting event-driven triggering (such as immediate reporting when drift exceeds a threshold). Furthermore, to address deeper logical changes, this step can also monitor concept drift. For example, in fraud detection applications, the definition of "fraud" itself evolves over time. The monitoring module can detect whether core concepts have drifted by continuously tracking the confidence distribution of the model output or introducing a bypass verification model, treating this as important runtime state data.

[0042] Step S2: Based on the running state data, predict the cumulative deviation cost that will result from maintaining the current execution plan within a preset time window, and estimate the one-time cost required to perform a replanning.

[0043] This step corresponds to the cost-effective replanning trigger module, achieving a paradigm shift from a "learning-driven black-box strategy" to an "economics-driven white-box decision-making" approach. This module is responsible for intelligently determining whether and when intervention is needed, avoiding system instability caused by frequent replanning.

[0044] First, the deviation cost is quantified. Based on the runtime state data collected by S1, the module uses time series forecasting models (such as Autoregressive Integrated Moving Average (ARIMA) or Long Short-Term Memory (LSTM)) to analyze historical data and trends, predicting the cumulative business loss if the current execution plan continues for a preset time window (e.g., the next 24 hours). Preferably, the cumulative deviation cost includes: quantified business losses due to cost overruns, processing delays exceeding service level targets, or decreased result quality. Specifically, cost overruns are calculated as the difference between the predicted cost and the SLO limit, multiplied by the time window. For example, if the predicted cost for 24 hours is $150, and the SLO limit is $120, the overrun is $30. Increased user churn due to processing delays exceeding the SLO (e.g., latency > 500ms) can be quantified as business loss. For example, based on historical data, each 1% latency overrun results in 0.1% user churn, multiplied by the user base and average value to calculate the loss. A decline in quality (such as an accuracy rate dropping from 95% to 90%) can lead to missed fraud detections in real-time risk control. The quantified loss is the number of missed events multiplied by the average loss amount.

[0045] For data distribution drift or physical operator performance degradation, the module integrates these factors into the prediction model. For example, if the data drift KS > 0.1, combined with historical observations, the prediction quality decreases by 5%, and the cumulative loss is calculated. The prediction model is trained based on state data from the past week, and parameters are updated hourly to ensure accuracy.

[0046] Secondly, estimate the re-planning overhead. Preferably, the one-time overhead includes at least one of the following: computational overhead of calling the global planner to generate a new plan, instantaneous service jitter overhead caused by plan migration, and resource migration overhead. In specific implementations, computational overhead is estimated based on the planner's historical execution time and resource consumption; for example, calling the global planner once requires 10 seconds of CPU time, costing $0.05. Instantaneous service jitter overhead is quantified as the increase in latency or decrease in throughput during migration; for example, blue-green deployment may cause a 1-second service interruption, resulting in a business loss of $1 affecting user experience. Resource migration overhead includes network / storage consumption for data transmission and state synchronization; for example, migrating 1GB of state data costs $0.1. Overhead estimation uses empirical formulas or simulation models and is dynamically adjusted according to the application scale.

[0047] Through these quantifications, this step outputs two economic indicators: a predicted cumulative deviation cost (e.g., $50) and a one-time expense (e.g., $10), providing a basis for S3's decision-making. The process is completely transparent, and the calculation logic at each step is auditable, ensuring that decisions align with business economic rationality.

[0048] Step S3: When the predicted cumulative deviation cost is higher than the one-time overhead, a replanning of the current execution plan is triggered to invoke the global planner and provide the real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan adapted to the current state.

[0049] When the cumulative deviation cost calculated by S2 is significantly higher than the one-time cost (e.g., deviation > 1.5 times the cost, to avoid edge cases), replanning is triggered. This threshold is configurable to ensure that action is taken only when the intervention benefit is significant, avoiding frequent system jitter.

[0050] Upon triggering, the online replanning module is invoked. This module is responsible for calculating the new execution plan, first passing the latest runtime state data as constraint information to the global planner. For example, "The current average latency of API-A is 500ms, not 200ms in the performance model," or "The mean of feature X of the input data has drifted to 0.8." The global planner is an optimization engine (such as a tool based on genetic algorithms or linear programming). The input includes the DAG structure of the application, available physical operator options (different models such as GPT-4 vs Llama3), and updated constraints, with the goal of maximizing SLO satisfaction (weighted cost, latency, quality).

[0051] The process of generating candidate plans involves enumerating physical implementations, such as replacing expensive operators with cheaper alternatives, or adjusting parallelism. The planner returns a completely new physical execution plan, for example, switching the summary operator from GPT-4 to Llama3 to reduce costs while ensuring quality does not fall below a threshold. The new plan is output in YAML format, containing operator mappings and resource configurations.

[0052] This step achieves the evolution from "static optimality" to "continuous optimality," ensuring that replanning adapts to real-time changes. For streaming data analytics applications, if data drift leads to a decline in quality, the planner can introduce data augmentation operators as new nodes.

[0053] Step S4: Smoothly migrate the intelligent application from the current execution plan to the new execution plan.

[0054] This step is executed by the smooth online migration module of the execution plan. As the system's "hands and feet," it is responsible for the smooth implementation of strategic adjustments, which is crucial for the 24 / 7 uninterrupted streaming intelligent application. Preferably, before smoothly migrating the intelligent application from the current execution plan to the new execution plan, it also includes: comparing the current execution plan with the new execution plan to identify the specific operator nodes that need to be changed. Specifically, it calculates the plan difference (Diff) by comparing DAG nodes and physical mappings to identify changing operators, such as replacing or adding nodes.

[0055] Different migration strategies are adopted depending on the type of change. Preferably, when the specific operator node to be changed is a stateless operator, a new resource is created for the new execution plan, and the newly flowing data is directed to the new resource through a traffic switching mechanism to achieve blue-green deployment or canary release migration. For example, for stateless classification operators, a new Pod is created using Kubernetes Service Mesh (such as Istio), with 100% of the initial traffic directed to the old plan, and a gradual switch (such as a 10% canary test) is performed before a full switch is completed once the system is stable. Blue-green deployment involves running the old and new plans in parallel, and destroying the old resources after the switch to ensure zero interruption.

[0056] Preferably, when the specific operator node to be changed is a stateful operator, a checkpointing mechanism is used to migrate the current state data of the stateful operator in the current execution plan to the corresponding operator in the new execution plan, thereby achieving stateful hot migration. Stateful operators, such as aggregation or windowing operations, involve internal states (such as counters or buffers). Migration process: First, the old operator serializes its current state (using checkpointing, such as the mechanism in Apache Flink) and saves it to shared storage (such as Redis); then, the new operator recovers from this state point and continues to process the data stream, ensuring that data is not lost and calculations are not duplicated. For example, in real-time risk control, state migration may involve the transfer of fraud counters, taking less than 1 second.

[0057] Preferably, the intelligent application is a streaming intelligent application that needs to run for a long time, including real-time risk control, streaming data analysis, or online services. These applications have high requirements for data flow continuity, and migration strategies prioritize minimizing interruptions. For example, in online services, gradual traffic switching is used to avoid user perception.

[0058] Through the steps described above, this embodiment achieves dynamic adaptive replanning for intelligent applications. In practical applications, for a streaming data analytics application, the initial plan uses an efficient API; during operation, a cost overrun of $20 / hour, a drift cost prediction of $100, and an overhead of $15 are detected, triggering replanning. A new plan is generated, switching to a cheaper model, and then migrated via blue-green deployment. The entire process improves SLO satisfaction rate, reduces costs by 15%, and maintains quality above 92%.

[0059] To further expand on the details of this embodiment, the following provides an in-depth description from aspects such as system architecture, algorithm implementation, example scenarios, and performance analysis:

[0060] First, at the system architecture level, this method is embedded in a runtime adaptive controller and integrates with frameworks such as Kubernetes or Apache Flink. The AI-SLO monitoring module uses Prometheus to collect metrics and supports custom exporters to expose state data. The cost-benefit triggering module uses Python to implement the prediction model; the LSTM network takes the state sequence of the past 100 time points as input and outputs a deviation curve. The online replanning module calls a planner such as the Optuna optimization library, and constraints are integrated into the objective function. The migration module utilizes Istio traffic management to ensure migration atomicity.

[0061] In terms of algorithm implementation, data distribution drift detection uses the KS test: given a sample distribution P and a benchmark Q, the statistical formula D = sup|F_P(x) - F_Q(x)| is used. If D > the critical value (calculated based on sample size), then there is a drift. Cumulative cost prediction: linear regression is used to fit the deviation trend, with the formula Deviation(t) = at + b, where a and b are fitted from historical data; cumulative = ∫Deviation(t)dtover window. Cost estimation: the empirical model Overhead = c1 planning time + c2 data migration volume + c3 jitter duration, where c is the business coefficient.

[0062] Example Scenario: A real-time risk control application using a Directed Acyclic Graph (DAG) includes data acquisition, feature extraction, model inference, and decision output. The initial plan uses a high-precision model at a cost of $4 / hour. After one hour of operation, data drift is detected (KS = 0.15), costing $5.2, with a quality of 0.88 and a latency of 450ms. S2 predicts 24-hour deviation at a cost of $60 (overspending $40 + quality loss $20) and overhead $8 (planning $3 + migration $5). S3 triggers the planner to generate a new plan: switching to a cheaper model and adding a drift correction operator. The new plan has a quality of 0.91 and a cost of $3.5. S4 migration: stateless feature extraction uses a canary algorithm (20% throughput initially), while stateful decision-making uses a checkpoint algorithm (state size 500KB, migration 0.5 seconds). All optimized details are seamlessly embedded in the steps; for example, expanding the state dimension improves monitoring comprehensiveness, and the migration strategy ensures streaming continuity. Overall, this embodiment provides a transparent and cost-controllable solution, overcoming the shortcomings of existing technologies.

[0063] Secondly, based on the foregoing embodiments, this embodiment further provides a dynamic adaptive replanning system for intelligent applications, used to execute the aforementioned dynamic adaptive replanning method for intelligent applications. This system aims to address the problems of static planning in existing technologies, such as its inability to adapt to dynamic changes during runtime, its single decision-making dimension, and its lack of awareness of multi-dimensional business SLOs. Through a combination of hardware and software, it achieves dynamic adaptive adjustment of intelligent applications running for extended periods, ensuring that the application maintains its optimal operating state throughout its entire lifecycle. This system can be, but is not limited to, integrated into computer devices with computing resources, such as servers, cloud computing platforms, or embedded systems, and supports integration with frameworks such as Kubernetes or Apache Flink.

[0064] Please see Figure 2 This is a schematic diagram of the structure of a dynamic adaptive replanning system for intelligent applications provided in this embodiment. The system includes a runtime state data module, an overhead evaluation module, an online replanning module, and an online migration module. These modules correspond to the core components of the Runtime Adaptation Controller, forming a complete "monitoring-detection-decision-execution" closed-loop system. Each module communicates with each other through API interfaces or message queues (such as Kafka) to ensure real-time data flow. The system's hardware infrastructure may include processors (such as CPUs / GPUs), memory, storage devices, and network interfaces. The software portion is implemented using Python or Java, supporting distributed deployment.

[0065] The runtime state data module continuously monitors at least one state dimension of the running intelligent application to obtain real-time runtime state data. This state dimension includes: end-to-end application cost, semantic quality of application outputs, or data distribution drift of application input data. This module corresponds to the AI-SLO monitoring module and acts as the system's "eyes," responsible for real-time collection of multi-dimensional state data directly related to business objectives. An intelligent application is defined as a set of AI operators and computational tasks that achieve specific business objectives. It is typically a directed acyclic graph (DAG) structure, particularly for long-running streaming intelligent applications such as real-time risk control, streaming data analysis, or online services. These applications are susceptible to fluctuations in external API performance, drift in input data characteristics, or adjustments in business priorities, leading to state drift—that is, actual performance or data characteristics deviating from the initial execution plan.

[0066] In its implementation, this module deploys lightweight agents (such as the Prometheus-based Exporter) to collect data periodically (e.g., every 5 seconds) or in an event-driven manner. Cost monitoring aggregates the monetary consumption of all physical operators. For example, for token-based LLM APIs (such as GPT-4 or Llama3), it calculates token usage and cumulative costs in real time, comparing them to the user-defined SLO limit (e.g., $5 / hour). Semantic quality monitoring, for generative AI, uses BLEU scores or semantic similarity (calculated based on the cosine similarity between the output and reference using BERT embeddings). In real-time risk control, it monitors precision and recall; if they fall below 95%, the labeling is considered off-target. Data distribution drift monitoring uses the KS test or Wasserstein distance to monitor the deviation of the input distribution features from a preset baseline (initial training data or the previous period). For example, if the feature mean drifts from 0.5 to 0.8 (KS>0.1), it is considered drift, which may lead to model performance degradation, such as a decrease in click-through rates for online service recommendations.

[0067] To enhance comprehensiveness, this module can also extend the monitoring of performance metrics for individual physical operators in the application, including response time or error rate. For example, it monitors the average response time (time from request to response) and error rate (failure percentage) of the GPT-4 API; an increase in response time from 200ms to 500ms, or an error rate >5%, is considered a degradation. Monitoring data is aggregated to a central storage (such as InfluxDB) and output in JSON format, such as {"cost":4.2,"quality":0.92,"data_drift_ks":0.12,"api_latency":450,"error_rate":0.03}. Furthermore, this module supports concept drift monitoring; for tasks such as fraud detection, it detects the evolution of core concepts through model confidence or bypass verification models. The implementation of this module ensures sensitive awareness of dynamic changes, overcoming the limitations of existing technologies that rely solely on technical metrics such as CPU utilization. In streaming data analytics applications, a status report is generated every minute, supporting threshold-triggered reporting.

[0068] The cost assessment module, based on runtime state data, predicts the cumulative deviation cost of maintaining the current execution plan within a preset time window and estimates the one-time cost of performing a replanning operation. This cost-effective replanning trigger module is the "decision core" of the system, achieving a paradigm shift from black-box learning to white-box decision-making. It quantifies the necessity of intervention using economic principles, avoiding system instability caused by frequent replanning.

[0069] Specifically, deviation cost quantification uses time series forecasting models (such as ARIMA or LSTM) to predict business losses in future windows (e.g., 24 hours) based on historical data. Cumulative deviation costs include cost overruns (the difference between forecast cost and the SLO cap), quantified losses due to processing latency exceeding the SLO (e.g., latency > 500ms causing a 0.1% user churn, multiplied by user value), and losses due to decreased result quality (e.g., a 5% drop in accuracy leading to missed fraud detections, multiplied by average event loss). Data drift and physical operator performance are integrated; for example, a 5% drop in prediction quality if KS > 0.1. Model parameters are updated hourly, trained based on data from the past week.

[0070] The replanning overhead estimate includes the computational cost of calling the global planner (e.g., 10 seconds of CPU time, $0.05), the instantaneous service jitter during migration (e.g., 1 second of interruption, resulting in a business loss of $1), and the resource migration overhead (e.g., transferring 1GB of state, $0.1). The empirical formula used is: Overhead = c1 planning time + c2 migration data volume + c3 * jitter duration, where c is the business coefficient. This module outputs two metrics, such as deviation of $50 and overhead of $10, providing transparent information for subsequent decisions. The decision logic is auditable to ensure economic rationality. In real-time risk control scenarios, if the cost exceeds the budget by $20 / hour, the prediction deviation is $60, and the overhead is $8, then a trigger is prepared.

[0071] The online replanning module is used to trigger a replanning of the current execution plan when the cumulative deviation cost of the prediction exceeds the one-time overhead. This involves invoking the global planner and providing the real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan adapted to the current state. This module is responsible for strategic adjustments, triggered when the deviation exceeds 1.5 times the overhead (a configurable threshold), to avoid edge jitter.

[0072] Upon triggering, the global planner (such as an Optuna-based optimization engine) is invoked, taking into account the application's DAG, physics options, and real-time constraints (such as "API latency 500ms" and "data mean 0.8"). The objective is to maximize SLO satisfaction, weighting cost, latency, and quality. Candidate plans are generated, such as switching the summarization operator from GPT-4 to Llama3 to reduce cost, or adding drift correction nodes. The new plan is output as YAML, containing operator mappings and resource configurations. This module enables evolution from static to continuous optimization; in streaming analysis, if drift leads to quality degradation, the planner introduces enhancement operators.

[0073] The online migration module is used to smoothly migrate the intelligent application from the current execution plan to the new execution plan. This module acts as the "hands and feet," ensuring uninterrupted operation 24 / 7. Before migration, the old and new plans are compared to identify the changed operator nodes.

[0074] For stateless operators, create new resources and redirect new inflow data via traffic switching (such as Istio's blue-green deployment or canary release); initially use 100% old plans, gradually switch (test with 10% first), and destroy old resources after stabilization. For stateful operators (such as aggregation), serialize the old state using checkpointing and save it to shared storage; the new operator restores it, ensuring no data loss or duplication; in real-time risk control, the migration of fraud counters takes less than 1 second. This module is suitable for streaming applications, such as online services using gradual switching to minimize disruption.

[0075] This system achieves dynamic adaptation through module collaboration. Example: In streaming data analysis, the initial plan uses an efficient API; during operation, drift is monitored, and if the deviation exceeds $100 and the cost is $15, a replanning is triggered to generate a low-cost model plan, enabling migration to a blue-green deployment. SLO satisfaction rate increases by 25%, and costs decrease by 18%. The system overcomes existing shortcomings, providing a transparent and controllable solution.

[0076] To elaborate further, the system is integrated with Kubernetes, supporting distributed scaling. The runtime state data module uses a custom Exporter; the cost evaluation module implements LSTM in Python; the online replanning module uses Optuna for optimization; and the online migration module is managed by Istio. Algorithm details: KS test D = sup|F_P - F_Q|; Deviation prediction Deviation(t) = a*t + b; the cost formula is dynamically adjusted. The online service DAG includes data collection, inference, and output. After drift, the system predicts the loss, generates a new plan, adds validation nodes, and migrates the canary. Performance: After 1000 hours of simulation and 15 replanning iterations, the benefit is more than twice the cost. It outperforms the black-box nature of RL and has no training cost.

[0077] This invention also discloses a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the steps of the above-described dynamic adaptive replanning method for intelligent applications provided by this invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.

[0078] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A dynamic adaptive replanning method for intelligent applications, characterized in that, Includes the following steps: Continuously monitor at least one state dimension of a running intelligent application to obtain real-time runtime state data, wherein the state dimension includes: end-to-end cost of the application, semantic quality of the results produced by the application, or data distribution drift of the application input data. Based on the runtime state data, predict the cumulative deviation cost that will result from maintaining the current execution plan within a preset time window, and estimate the one-time overhead required to perform a replanning; wherein, the cumulative deviation cost includes: overspending based on the cost, quantitative business losses caused by processing delays exceeding service level targets, or deterioration in result quality; the one-time overhead includes: at least one of the following: computational overhead of calling the global planner to generate a new plan, instantaneous service jitter overhead caused by execution plan migration, and resource migration overhead; When the cumulative deviation cost of the prediction exceeds the one-time overhead, a replanning of the current execution plan is triggered to invoke the global planner and provide the real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan adapted to the current state. The intelligent application is smoothly migrated from the current execution plan to the new execution plan.

2. The method according to claim 1, characterized in that, The state dimension also includes: performance metrics for applying a single physical operator, which include response time or error rate.

3. The method according to claim 2, characterized in that, The data distribution drift of the application input data includes: based on real-time monitoring of the distribution characteristics of the input data, when a significant deviation of the distribution characteristics from a preset benchmark distribution is detected, the data distribution drift is determined.

4. The method according to claim 1, characterized in that, Before smoothly migrating the intelligent application from the current execution plan to the new execution plan, the process also includes: Compare the current execution plan with the new execution plan to identify the specific operator nodes that need to be changed; When the specific operator node that needs to be changed is a stateless operator, a new resource is created for the new execution plan, and the newly flowing data is directed to the new resource through a traffic switching mechanism to achieve blue-green deployment or canary release migration.

5. The method according to claim 4, characterized in that, When the specific operator node that needs to be changed is a stateful operator, the current state data of the stateful operator in the current execution plan is migrated to the corresponding operator in the new execution plan through the checkpoint mechanism to achieve stateful hot migration.

6. The method according to claim 1, characterized in that, The intelligent application is a streaming intelligent application that needs to run for a long time, including real-time risk control, streaming data analysis, or online services.

7. A dynamic adaptive replanning system for intelligent applications, used to execute the dynamic adaptive replanning method for intelligent applications according to any one of claims 1 to 6, comprising: The runtime state data module is used to continuously monitor at least one state dimension of the running intelligent application to obtain real-time runtime state data, wherein the state dimension includes: end-to-end cost of the application, semantic quality of the results produced by the application, or data distribution drift of the application input data. The overhead assessment module is used to predict the cumulative deviation cost caused by maintaining the current execution plan within a preset time window based on the runtime state data, and to estimate the one-time overhead required to perform a replanning; wherein, the cumulative deviation cost includes: overspending based on the cost, quantitative business loss caused by processing delays exceeding service level targets, or deterioration in result quality; the one-time overhead includes at least one of the following: computational overhead of calling the global planner to generate a new plan, instantaneous service jitter overhead caused by execution plan migration, and resource migration overhead; The online replanning module is used to trigger a replanning of the current execution plan when the cumulative deviation cost of the prediction is higher than the one-time overhead. This involves calling the global planner and providing the real-time runtime state data as new constraint information to the global planner, thereby generating a new execution plan that adapts to the current state. An online migration module is used to smoothly migrate the intelligent application from the current execution plan to the new execution plan.

8. A computer program product, characterized in that, The method includes a non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the dynamic adaptive replanning method for intelligent applications as described in any one of claims 1 to 6.

Images