A cloud-edge collaborative task scheduling method based on non-intrusive sensing and value evaluation

Abstract

The application belongs to the technical field of edge computing, cloud computing and task scheduling, and particularly relates to a cloud-edge collaborative task scheduling method based on non-intrusive sensing and value evaluation. The method comprises the following steps: S1, non-intrusive cloud performance sensing and modeling; S2, task value evaluation based on urgency; S3, adaptive update of edge node value threshold: the edge node realizes adaptive resource reservation by maintaining a dynamically changing value threshold. Only the tasks with a value higher than the threshold are allowed to occupy local resources; S4, multi-level collaborative scheduling decision: the scheduler executes hierarchical scheduling according to the real-time sensed state and calculated indexes in priority order. The application solves the problem that it is difficult to cope with public cloud performance fluctuations under limited control authority, optimizes the space-time configuration of computing power resources, can provide more stable service quality guarantee at a lower cost, and has important practical application value.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of edge computing, cloud computing, and task scheduling, and specifically relates to a cloud-edge collaborative task scheduling method based on non-intrusive perception and value assessment. Background Technology

[0002] With the rapid development of IoT, 5G communication, industrial internet, and smart city technologies, massive amounts of terminal devices are continuously generating massive amounts of sensing data and computing tasks, gradually revealing the significant technical shortcomings of traditional centralized cloud computing models. Cloud computing centers are physically far from terminal devices, and all massive amounts of raw data are uploaded to the cloud for centralized processing. This not only consumes a large amount of core network bandwidth resources and results in high transmission latency, making it difficult to meet the stringent requirements of low latency and high real-time performance for applications such as industrial control, intelligent security, vehicle networking, and real-time monitoring; but also poses risks of data leakage and compliance security due to the long-distance uploading and storage of sensitive and private data.

[0003] Edge computing pushes computing power, storage, and service capabilities to the network edge, enabling it to handle terminal service requests locally, effectively reducing transmission latency, alleviating pressure on backbone networks, and protecting local data privacy. However, edge nodes generally suffer from heterogeneous resources, limited computing power, limited storage capacity, and dispersed deployment. A single edge node struggles to handle high computing power, heavy loads, and complex interconnected tasks, and lacks global resource coordination capabilities, easily leading to problems such as excessive load on local nodes, uneven resource idleness, task blocking, and low scheduling success rates.

[0004] Cloud-edge collaboration is a novel collaborative working model between edge computing nodes and cloud computing centers, enabling task scheduling, resource complementarity, and data interaction via network. While numerous studies have explored cloud-edge collaboration methods, most approach them from the cloud vendor's perspective. These studies assume the algorithm has full access to the underlying hardware infrastructure, enabling it to obtain information such as the total global resource volume and utilization, the mapping between task loads and underlying resources, the length of the task queue, hardware performance, and global load. However, in non-intrusive cloud-edge collaboration scenarios, the algorithm is run by the cloud vendor's user and can only access limited information provided by the vendor through its API, such as instance specifications and resource unit prices. Therefore, these methods cannot be used in such scenarios.

[0005] Skycomputing, as an emerging approach, does not require modification to the underlying public cloud infrastructure. Instead, it adapts itself to the differences between various cloud vendors. By using middleware to connect the API interfaces exposed to users by various public clouds, it integrates the different interfaces of various cloud vendors and provides users with a unified skycomputing interface. This non-intrusive solution to the problem of multi-cloud collaboration makes it the most suitable method for this scenario.

[0006] In this context, the two existing technical solutions that are closest to the present invention are as follows: 1. SkyPilot SkyPilot, an emerging cloud-to-cloud proxy framework, focuses on automating the efficient execution of user tasks in multi-cloud environments. The solution first requires users to provide detailed task specifications, including required computing resources (such as vCPU, memory, GPU, etc.), software environment configuration, and specific execution instructions. Based on these specifications, SkyPilot calls the API interfaces of major public cloud vendors in real time to obtain current instance prices, specifications, and real-time resource availability. Next, the system's built-in optimizer comprehensively considers the cost and performance of multi-cloud nodes, automatically selecting cloud instances that meet the user's task requirements and offer the lowest price per unit time or the shortest execution time. After node selection, SkyPilot also handles automated resource allocation, environment setup, data migration, and task monitoring, greatly simplifying the complexity of utilizing heterogeneous public cloud resources.

[0007] 2. SkyCap SkyCap is a loss-aware coordinator for sky computing, primarily focused on optimizing the revenue of service deployment in multi-cloud environments. Its online scheduling algorithm abstracts task deployment across different cloud nodes into a multi-armed slot machine model, treating each available cloud node as an "arm." The system continuously records and saves samples of the actual revenue or profit gained from deploying tasks on each cloud node in the past, and calculates the average expected revenue for each cloud node based on these historical observations, using this as a basis for evaluating node performance. When a task request is received from an edge node, SkyCap selects the cloud instance with the highest expected revenue for deployment through a decision-making mechanism that balances exploration and utilization, aiming to maximize the system's long-term total revenue.

[0008] The main shortcomings of existing technologies are as follows: (1) Because public cloud service providers only provide users with highly abstract interfaces to simplify resource encapsulation, users cannot obtain underlying state information such as the actual load and memory bandwidth of physical machines. This "black box" characteristic, along with interference factors such as resource overselling, leads to significant random fluctuations in the performance of public cloud computing. Existing non-intrusive scheduling methods usually only estimate tasks based on static parameters or simple historical averages, which cannot accurately capture and model this performance fluctuation. This directly leads to serious deviations in the system's prediction of task execution time, making it impossible to effectively avoid the risk of timeouts during performance troughs. Ultimately, this results in insufficient execution guarantee capabilities for latency-sensitive mobile tasks, leading to high timeout default costs. This is the most significant drawback of existing technologies.

[0009] (2) Existing non-intrusive cloud collaboration frameworks focus on workload migration between multiple clouds or one-way offloading from the edge to the cloud, neglecting the collaboration potential between geographically adjacent edge nodes. In real mobile networks, the lack of an "edge-to-edge collaboration" mechanism between edge nodes makes it easy for a single edge node to run into resource exhaustion when dealing with sudden traffic with uneven spatial and temporal distribution, while the redundant computing power of neighboring nodes cannot be allocated in time to share the pressure, thus causing a large number of tasks to be forced to queue or fail due to resource scarcity.

[0010] (3) Because existing scheduling algorithms fail to quantify the value and prioritize heterogeneous mobile tasks and lack dynamic resource reservation strategies, the scarce computing power of edge nodes is easily exhausted prematurely by a large number of low-urgency short tasks during peak load periods. Since this indiscriminate allocation mode fails to reserve the necessary resource space for high-value tasks, critical tasks have to be unloaded to the cloud or even abandoned directly when they arrive, which not only significantly increases cloud service costs but also increases the total default loss of the system. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the present invention aims to provide a non-intrusive task processing algorithm, BridgeScheduler, based on perception and collaborative scheduling. This algorithm addresses the cloud-edge collaborative task scheduling problem under conditions of public cloud performance fluctuations and uneven edge resource allocation in scenarios with limited control permissions, minimizing the total task cost. By constructing a non-intrusive perception module, historical execution records are used to statistically model cloud node performance fluctuations, thereby accurately predicting task timeout risks. Simultaneously, an adaptive resource reservation mechanism based on task value and a cloud-edge / edge-edge collaborative mechanism are introduced. This ensures that high-urgency tasks receive priority access to local low-latency resources while achieving globally optimized allocation of edge and cloud computing power, ultimately reducing the total system default cost and fully utilizing edge resources.

[0012] Technical solution A cloud-edge collaborative task scheduling method based on non-intrusive perception and value assessment includes the following steps: S1, Non-intrusive Cloud Performance Awareness and Modeling: Obtain cloud performance data through non-intrusive means and statistically model its volatility to assess the expected execution time and timeout risk of tasks in the cloud.

[0013] S2, Task Value Assessment Based on Urgency: To prioritize high-urgency tasks when resources are limited, the task value is calculated for each task. This comprehensively reflects the urgency of the task and the cost-effectiveness of resource utilization.

[0014] S3, Adaptive Update of Edge Node Value Threshold: Edge nodes maintain a dynamically changing value threshold. This enables adaptive resource reservation. Only tasks with a value exceeding a certain threshold are allowed to occupy local resources.

[0015] S4, Multi-level Cooperative Scheduling Decision: The scheduler executes hierarchical scheduling according to priority order based on the real-time sensed status and calculated indicators.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) To address the problem that performance fluctuations in existing non-intrusive cloud-edge collaboration technologies cannot be accurately perceived due to the "black box" nature of public cloud resources, leading to a high risk of task timeout defaults, this invention proposes a non-intrusive public cloud performance perception method based on historical data statistical modeling. By constructing a dynamic sliding window and a normal distribution model, it achieves accurate quantification of cloud performance fluctuations and network latency without obtaining access to underlying physical hardware. This method not only overcomes the limitations of traditional non-intrusive solutions that rely solely on static parameters or simple mean predictions, but also accurately identifies high-risk periods by calculating expected timeout penalties, thereby avoiding performance traps during the scheduling decision-making stage and significantly reducing the timeout default rate of latency-sensitive tasks.

[0017] (2) To address the problem in existing technologies where edge nodes lack horizontal collaboration mechanisms, leading to resource exhaustion at a single point when facing sudden traffic surges, while the computing power of neighboring nodes remains idle and wasted, this invention proposes a cloud-edge and edge-edge collaborative scheduling mechanism for heterogeneous latency-sensitive tasks, establishing a hierarchical decision-making system that integrates local execution, edge offloading, and cloud-edge collaboration. By introducing the "edge-edge collaboration" mechanism, the resource silo effect of a single edge node is broken, enabling tasks to flexibly flow between geographically adjacent edge nodes. This innovation allows the system to prioritize the use of low-latency idle computing power on the edge side when dealing with mobile loads with uneven spatiotemporal distribution, not only alleviating the pressure of single-point overload but also significantly reducing unnecessary high-latency cloud offloading, thereby improving the overall system throughput and response speed.

[0018] (3) To address the problem that existing scheduling algorithms lack value quantification for task heterogeneity sensitivity, which often leads to the forced failure of high-urgency tasks due to resource occupation by low-value tasks, this invention proposes an edge node adaptive resource reservation method based on task value assessment. This method defines task value indicators. And combined with value threshold Dynamic prediction enables fine-grained access control of traffic. This method adaptively raises or lowers the access threshold based on real-time load changes, ensuring that low-value tasks are automatically filtered out when resources are scarce, while reserving critical computing power for high-value bursts of traffic. This mechanism not only maximizes the utilization efficiency of edge resources but also effectively reduces the total default cost incurred by the system due to the failure of critical tasks.

[0019] In summary, this invention provides a more reliable and economical solution to the shortcomings of existing cloud-edge collaborative environments by introducing a non-intrusive performance-aware model, an adaptive resource reservation mechanism, and a multi-layered collaborative scheduling strategy between cloud and edge nodes. These innovations not only address the difficulty of handling public cloud performance fluctuations under limited control, but also optimize the spatiotemporal allocation of computing resources through horizontal collaboration between edge nodes and value-based resource allocation. These improvements enable mobile computing systems to provide more stable quality of service guarantees at a lower cost when facing highly dynamic loads and complex network environments, demonstrating significant practical application value. Attached Figure Description

[0020] Figure 1 This is an application scenario diagram of the present invention; Figure 2 This is a system architecture diagram of the present invention; Figure 3 This is a flowchart of the method of the present invention; Figure 4 This is a comparison chart of the total cost of the method of the present invention and the traditional method, where the solid line represents the BridgeScheduler algorithm of the present invention, the dashed line represents the SkyPilot algorithm, and the dotted line represents the SkyCap algorithm. Figure 5 This is a comparison chart of the edge deployment rates of the method of this invention and traditional methods for high-urgency tasks. The solid line represents the BridgeScheduler algorithm of this invention, the dashed line represents the SkyPilot algorithm, and the dotted line represents the SkyCap algorithm. Detailed Implementation

[0021] The technical solution provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0022] The application scenarios and system topology of this invention are as follows: Figure 1 As shown, it includes a three-layer architecture: user layer, edge layer, and cloud layer.

[0023] in: User layer: This includes user terminals distributed across different geographical locations, responsible for submitting computing tasks to the nearest edge node.

[0024] Edge layer: Composed of multiple interconnected edge nodes. Edge nodes have local computing capabilities, which can either directly process user tasks or, through edge-to-edge collaboration, schedule tasks to nearby idle edge nodes to balance the load.

[0025] Cloud layer: Contains multiple public cloud nodes. When edge layer resources are insufficient, edge nodes can offload tasks to cloud nodes for execution through cloud-edge collaboration (as shown by the green one-way arrow in the figure).

[0026] This invention proposes a cloud-edge collaborative task scheduling method based on non-intrusive perception and value assessment, aiming to solve the problems of performance fluctuation perception and task assurance in public cloud environments.

[0027] like Figure 2 As shown, the method of the present invention is mainly implemented by two core modules: a non-invasive sensing module ( Figure 2 (Left half) and cloud-edge collaborative scheduling module ( Figure 2 (Right half)

[0028] Specifically as follows: Non-intrusive perception module: Responsible for statistically modeling the performance fluctuations of cloud nodes and perceiving the network conditions (latency and bandwidth) between nodes by analyzing historical task records and actively probing, without having low-level hardware permissions.

[0029] To address the performance opacity issue caused by the "black box" nature of public clouds, this invention designs a perception method based on historical information reuse and statistical modeling, the key points of which are as follows: (1) Time window selection: The system dynamically selects an optimal historical time window. This aims to strike a balance between data freshness and sample sufficiency, ensuring the accuracy of subsequent estimates.

[0030] (2) Performance distribution modeling: The system is based on By combining historical task records within a time period with computational characteristics such as task resource requests and total computational load, as well as disk I / O characteristics, the actual execution time of historical tasks is normalized into samples of time required per unit of CPU computation and time required per unit of I / O transfer. This allows for statistical modeling of the performance distribution of cloud nodes, thereby depicting the average performance level and fluctuation range of cloud nodes.

[0031] (3) Timeout Risk Assessment: For newly arrived tasks, the system combines their computational load with the above performance distribution to predict the execution time distribution of the task, and accordingly calculates the expected timeout penalty for the task to be executed in the cloud. This serves as a risk basis for subsequent scheduling decisions.

[0032] The cloud-edge collaborative scheduling module includes an adaptive resource reservation submodule and a task scheduling submodule. The former reserves edge resources for high-urgency tasks by calculating task value and dynamically adjusting value thresholds; the latter decides whether to execute the task locally, offload it to other edge nodes (edge-to-edge collaboration), offload it to a cloud node (cloud-edge collaboration), or abandon it based on perceived information and value judgment.

[0033] Specifically, value-based task valuation and adaptive resource reservation: To ensure the success of high-urgency tasks, the concept of "task value" is introduced and dynamic access control is implemented: (1) Task value calculation: The system calculates the value index for each task. This indicator comprehensively reflects both the urgency of the task and the cost-effectiveness of resource utilization.

[0034] (2) Dynamic adjustment of value threshold: The edge node maintains a dynamic value threshold. The system analyzes the optimal allocation boundary of edge resources in different time periods based on historical task load and resource consumption, and predicts the appropriate value threshold for the current moment.

[0035] (3) Adaptive resource reservation: When a task reaches an edge node, it only reserves resources if its value is available. Not lower than the value threshold of the current edge node Furthermore, the edge node resources meet the requirements of this task. Only when resources are scarce will the task be prioritized for local execution. This mechanism automatically reserves edge resources for high-value tasks when resources are limited.

[0036] Cloud-edge collaboration and edge-edge collaboration scheduling strategies: Based on the perceived risk information and task value, the scheduling module makes decisions according to the following logic: (1) Local acceptance judgment: when the task Upon arrival, assess the mission's value. Is it not lower than the local value threshold? And whether there are enough available resources locally. If so, execute locally.

[0037] (2) Edge-to-edge collaboration (unloading to other edges): If the local area cannot accept the task, the system queries the value threshold of the nearby edge nodes. For candidate edge nodes with sufficient resources and value thresholds that allow the task to enter, the system further compares the expected total cost of the task execution on them and prioritizes the edge nodes with lower expected total costs for unloading.

[0038] (3) Cloud-edge collaboration (offloaded to the cloud): If the edge layer cannot accept the application, the system traverses all available cloud nodes. Combining the expected timeout penalty provided by the perception module and the cloud resource price, the expected total cost of execution on each cloud node is estimated. Select the cloud node with the lowest cost. If the expected total cost of the cloud node with the lowest cost is lower than the maximum default cost of the task... If the task is successfully executed, it will be offloaded to that cloud node for execution; otherwise, the task will be abandoned.

[0039] Example 1 A cloud-edge collaborative task scheduling method based on non-intrusive perception and value assessment, such as... Figure 3 This includes the following steps: S1, Non-intrusive Cloud Performance Awareness and Modeling: Since edge nodes have no underlying control over public cloud nodes (i.e., "black box" characteristics), cloud performance data is obtained through non-intrusive means, and its volatility is statistically modeled to assess the expected execution time and timeout risk of tasks in the cloud.

[0040] S11, Time Window Choice: Define metric P1 as the percentage difference between the sample means of the first and second halves of the window, used to measure the historical data staleness error caused by an excessively long window; define metric P2 as the percentage of the standard error of the sample mean relative to the sample mean, used to measure the sample size insufficient error caused by an excessively short window. The candidate window that minimizes P1 + P2 is the current optimal historical time window, thus achieving a balance between data freshness and sample size.

[0041] S12, Historical performance data collection and normalization: for windows Internal cloud nodes The completed historical mission records are statistically analyzed. For any historical mission... Based on its computational complexity Disk I / O volume Samples of time required per unit of CPU computation and time required per unit of I / O transfer were constructed respectively: in, For historical mission In cloud nodes The time required for a unit of CPU computation during execution; For historical mission In cloud nodes Time required for a unit I / O transfer during execution; For historical mission In cloud nodes A sample of the time required for the corresponding unit of CPU computation; For historical mission In cloud nodes The sample of time required for the corresponding unit I / O transfer volume.

[0042] S13, Cloud Node Performance Distribution Modeling: Establish statistical distribution models for the time required per unit of CPU computation and the time required per unit of I / O transfer volume. in, For cloud nodes The time required for unit CPU computation is a random variable; For cloud nodes The time required to process a unit of I / O is a random variable; For cloud nodes The average time required per unit of CPU computation; For cloud nodes The standard deviation of time required per unit of CPU computation; For cloud nodes The average time required to process a unit of I / O; For cloud nodes The standard deviation of time required to process one unit of I / O; Represents a normal distribution, where This represents the mean. Indicates variance.

[0043] This model simultaneously characterizes the average level and fluctuations of CPU and I / O performance of cloud nodes.

[0044] S14 further includes an active detection mechanism: to ensure the accuracy of the statistical model, a minimum sample threshold is set. If the current time window If the number of passive historical samples collected internally is insufficient, the edge nodes will automatically generate and send lightweight active probing tasks to the cloud nodes to supplement performance sampling points and ensure the accuracy of performance estimation.

[0045] The active probing task is a lightweight benchmark task constructed by the edge node and sent to the cloud node for execution. It is used to simultaneously measure the CPU service capability and I / O service capability of the cloud node. The benchmark test is run multiple times on the cloud node to collect performance points and thus supplement the number of performance samples.

[0046] S15, Completion Time Distribution Calculation: Based on the above model, for newly arrived tasks... If its computational complexity is Disk I / O quantity is Predict its location in this cloud node The random variable of expected completion time Random variable with time required per unit CPU computation and the random variable of time required to process a unit of I / O The relationship is: Its mean With variance They are respectively: In subsequent modules, the expected timeout penalty for tasks executed in the cloud is calculated based on the completion time distribution.

[0047] S2, Task Value Assessment Based on Urgency: To prioritize high-urgency tasks when resources are limited, the concept of "task value" is introduced. This indicator.

[0048] S21, Task Parameter Definition: Each task Includes resource requirements Calculation workload Disk I / O volume Input data volume Response time requirements Timeout penalty coefficient and the maximum cost of default .

[0049] S22, Estimated Total Cost Definition: Task at Node The estimated total cost during execution is denoted as Specifically, when a task is executed on an edge node, the expected timeout penalty is considered; when a task is executed on a cloud node, both the expected timeout penalty and cloud service fees are considered.

[0050] Specifically, set tasks At the node The random variable for the completion time during execution is: Its probability density function is The task response time requirement is... The timeout penalty coefficient is The maximum cost of default is .

[0051] Task at node Timeout penalty function during execution For (of which) (Completion time) Therefore, the task is at the node The expected timeout penalty during execution is: When calculating the expected timeout penalty for cloud nodes, the completion time follows a normal distribution. The mean value calculated according to the method in step S1 With variance Then, the expected timeout penalty for the portion exceeding the deadline is calculated; however, when calculating the expected timeout penalty for edge nodes, it degenerates into the case of a fixed completion time, that is, the fixed completion time of the edge node task is directly used. Substitution calculate.

[0052] Only when the task Scheduled to cloud node Cloud service fees are incurred only during execution. (Set up cloud nodes.) The unit CPU resource price, unit memory resource price, and unit disk resource price are respectively , and Then the task In cloud nodes The estimated cloud service cost during execution is: S23, the difference between the optimal total cost on the cloud side and the cost on the edge side: Let the task... At the current edge node The estimated total cost of execution is In all candidate cloud nodes The minimum estimated total cost is: The estimated total cost savings from keeping the task on the edge side are: S24, Normalized resource time overhead and task value calculation: Let the estimated completion time of the task on the current edge node be... Then its normalized resource time cost is in , , These represent the total number of CPU cores, memory capacity, and disk capacity of the edge nodes, respectively.

[0053] The value of the task is defined as: The higher the value of a task, the more worthwhile it is to prioritize the use of edge resources for that task.

[0054] S3, Adaptive Update of Edge Node Value Threshold: Edge nodes maintain a dynamically changing value threshold. This enables adaptive resource reservation. Only tasks with a value exceeding a certain threshold are allowed to occupy local resources.

[0055] S31, Historical value threshold statistics of edge nodes: Review the task records of multiple past time slices with a fixed length of time slice. For any time slice, filter the active tasks that overlap with the time slice, calculate the normalized resource time occupied by them in the time slice, and sort them from high to low according to their task value. Then, load them into the available resource capacity of the time slice in turn. The value of the last task that can still be loaded is taken as the ideal value threshold of the time slice. S32, Prediction of Future Thresholds: Based on the historical value threshold sequence obtained through backtracking calculations, time series forecasting algorithms (such as the ARIMA model) are used to analyze its changing trends and predict the optimal value threshold for the current time period. And apply it to the current scheduling decisions.

[0056] S4, Multi-level Cooperative Scheduling Decision: The scheduler executes hierarchical scheduling according to priority order based on the real-time sensed status and calculated indicators.

[0057] S41, Local Acceptance Decision: When the task... Reaching the edge node When doing so, first determine whether local resources are sufficient (i.e., the available resource vector of the current edge node E). Resource requirement vector not less than task T And whether the task value meets the standard ( ) If both conditions are met, the task is deployed locally. This ensures that the limited resources of edge nodes are prioritized for high-value tasks.

[0058] S42, Edge-to-edge collaborative offloading: Edge nodes periodically exchange their current value thresholds through the network. and resource status.

[0059] S421, If ​​the local node cannot accept the request, the scheduler iterates through the list of other candidate edge nodes. Filter out those that meet the requirements And there are sufficient candidate edge nodes.

[0060] S422, attempt to offload the task to a candidate edge node (prioritizing the node with the lowest network latency or lightest load). If offloading is successful, the task is executed on the neighboring edge node.

[0061] S43, Cloud-based Collaborative Unloading: If all edge nodes are unable to accept the task, the cloud-based unloading process will begin.

[0062] S431, Traverse all available cloud nodes Based on the performance model established in steps S1 and S2, the total cost of the task on each cloud node is calculated. .

[0063] S432, select the cloud node with the lowest total cost. .

[0064] S433, if the minimum cost is still higher than the maximum penalty for direct failure of the task. If the task fails, the task is abandoned; otherwise, it is rescheduled to a cloud node. implement.

[0065] Example 2 This embodiment provides a verification test and effect analysis of the non-intrusive cloud-edge collaborative scheduling method described above. The significant advancements of this invention compared to existing technologies are verified by constructing a prototype system. First, to verify the effectiveness of the scheduling algorithm, a prototype system was built using Python. Local nodes were used as edge nodes, and Alibaba Cloud's Function Compute services in Hangzhou, Beijing, and Shanghai (three different regions) were used as cloud nodes. A test scenario was constructed containing 30 users, 5 edge nodes, and 3 cloud nodes.

[0066] Regarding load data, the computational load was sampled by referring to the function execution time distribution in the Azure Functions Trace 2019 and Huawei Private Cloud 2023 datasets, and the spatiotemporal unevenness of the load on each edge node was simulated based on the distribution of the number of user requests received daily by the base station in The Telecom Dataset. The task set was also set to 50% high-urgency tasks and 50% low-urgency tasks, with high-urgency tasks having low latency tolerance and high timeout penalties, to test the algorithm's ability to distinguish heterogeneous tasks.

[0067] The experiment selected SkyPilot and SkyCap, two representative non-intrusive scheduling methods, as comparison benchmarks. SkyPilot focuses only on minimizing cross-cloud costs without considering edge nodes, while SkyCap, although supporting cloud-edge offloading, lacks edge-edge collaboration and relies solely on historical averages for performance prediction. Figure 4This chart compares the total cost of the BridgeScheduler algorithm, SkyPilot, and SkyCap algorithms. The left subplot shows the results from Azure Functions Trace 2019, and the right subplot shows the results from Huawei Private Cloud 2023. The horizontal axis represents the load factor (the ratio of the total task load to the total resources of the edge nodes), and the vertical axis represents the total cost. The solid line represents the BridgeScheduler algorithm, the dashed line represents the SkyPilot algorithm, and the dotted-dash line represents the SkyCap algorithm.

[0068] Experimental results show that, at different load ratios from 1.0x to 5.0x, the BridgeScheduler algorithm proposed in this invention significantly reduces the total cost (including timeout penalties and cloud service fees) after task generation by 68.8%~93.8% and 68.4%~92.9% respectively compared to SkyPilot and SkyCap.

[0069] In terms of mission assurance, this invention maintains a high "deployment rate at the edge of high-urgency tasks," such as... Figure 5 This chart compares the edge deployment rates of high-urgency tasks using the BridgeScheduler algorithm of this invention with those using the SkyPilot and SkyCap algorithms (higher rates are better). The left subplot shows the results from Azure Functions Trace 2019, and the right subplot shows the results from Huawei Private Cloud 2023. The horizontal axis represents the load factor (the ratio of total task load to total edge node resources), and the vertical axis represents the edge deployment rate of high-urgency tasks (%). The solid line represents the BridgeScheduler algorithm of this invention, the dashed line represents the SkyPilot algorithm, and the dotted-dash line represents the SkyCap algorithm.

[0070] Experimental results demonstrate that the value threshold mechanism successfully reserves edge resources for high-value tasks, effectively preventing performance crashes caused by low-value tasks crowding out resources.

[0071] Furthermore, ablation experiments showed that disabling the value threshold module would reduce system performance by approximately 34.9% to 61.4%, and disabling the non-intrusive sensing module would reduce system performance by approximately 37.2% to 71.4%. This further validates that using normal distribution modeling to perceive the performance of "black box" cloud nodes can effectively avoid the risk of unexpected timeouts caused by performance fluctuations in the cloud.

[0072] Example 3 This embodiment provides a task collaborative scheduling system in a heterogeneous computing environment. The system is not limited to specific physical devices, but is applicable to hybrid computing scenarios that include local controllable computing power and remote black-box computing power. It mainly includes a first type of computing node, a second type of computing node, and a user terminal.

[0073] The first type of computing node serves as the access point and control center for tasks. In practical applications, it can be an edge computing server, a 5G base station, an IoT gateway, an enterprise local data center, or a smart terminal—any device with local resource control. This node deploys a non-intrusive sensing agent module and a collaborative scheduling controller. The non-intrusive sensing agent module is responsible for performance monitoring of the second type of computing node. By recording historical task execution times or sending probe messages, it constructs a performance fluctuation model (such as a normal distribution model) for the second type of computing node without underlying permissions. The collaborative scheduling controller uses a value assessment unit to calculate the urgency value of tasks and a threshold decision unit to maintain a value threshold in real time based on a time series analysis algorithm, thereby achieving adaptive resource reservation. Furthermore, multiple first-type computing nodes network through communication interfaces, periodically exchanging their respective value thresholds and load states, thus supporting task migration between peer nodes, i.e., edge-to-edge collaboration.

[0074] The second type of computing node serves as an elastic scaling pool for the system. It presents a "black box" characteristic to the first type of computing nodes, as it cannot obtain information about underlying states such as CPU load. In practical applications, it can be a public cloud platform, a third-party computing power leasing platform, or a serverless computing instance. This node provides computing services through a standard API interface, exposing only basic billing information and task execution results to the first type of computing nodes. During system operation, the first type of computing nodes utilizes a value threshold mechanism to allow only high-urgency tasks with values ​​exceeding the current threshold to occupy local resources when local resources are scarce. Simultaneously, it uses non-intrusive data awareness for risk-avoidance offloading, fully considering the risks associated with performance fluctuations when deciding to schedule tasks to the second type of computing nodes. This breaks the limitation of traditional cloud-edge collaboration requiring intrusion into the underlying cloud infrastructure.

[0075] The scope of protection of the cloud-edge collaborative scheduling method based on non-intrusive perception and dynamic value threshold described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A cloud-edge collaborative task scheduling method based on non-intrusive perception and value assessment, characterized in that, Includes the following steps: S1, Non-intrusive Cloud Performance Awareness and Modeling: Obtain cloud performance data through non-intrusive means and statistically model its volatility to assess the expected execution time and timeout risk of tasks in the cloud; S2, Task Value Assessment Based on Urgency: To prioritize high-urgency tasks when resources are limited, the task value is calculated for each task. This comprehensively reflects the urgency of the task and the cost-effectiveness of resource utilization; S3, Adaptive Update of Edge Node Value Threshold: Edge nodes maintain a dynamically changing value threshold. This enables adaptive resource reservation; only tasks with a value exceeding this threshold are allowed to occupy local resources. S4, Multi-level Cooperative Scheduling Decision: The scheduler executes hierarchical scheduling according to priority order based on the real-time sensed status and calculated indicators. Step S2 specifically involves: S21, Task Parameter Definition: Each task Includes resource requirements Calculation workload Disk I / O volume Input data volume Response time requirements Timeout penalty coefficient and the maximum cost of default ; S22, Estimated Total Cost Definition: Task at Node The estimated total cost during execution is denoted as When a task is executed on an edge node, the expected timeout penalty is considered; when a task is executed on a cloud node, both the expected timeout penalty and the cloud service fee are considered. Specifically, set tasks At the node The random variable for the completion time during execution is: Its probability density function is The task response time requirement is... The timeout penalty coefficient is The maximum cost of default is ; Task at node Timeout penalty function during execution For, among which Completion time: Therefore, the task is at the node The expected timeout penalty during execution is: When calculating the expected timeout penalty for cloud nodes, the completion time follows a normal distribution. The calculated mean With variance Then, the expected timeout penalty for the portion exceeding the deadline is calculated; however, when calculating the expected timeout penalty for edge nodes, it degenerates into the case of a fixed completion time, that is, the fixed completion time of the edge node task is directly used. Substitution calculate; Only when the task Scheduled to cloud node Cloud service fees are incurred only during execution; cloud nodes are set up. The unit CPU resource price, unit memory resource price, and unit disk resource price are respectively , and Then the task In cloud nodes The estimated cloud service cost during execution is: S23, the difference between the optimal total cost on the cloud side and the cost on the edge side: Let the task... At the current edge node The estimated total cost of execution is In all candidate cloud nodes The minimum estimated total cost is: The estimated total cost savings from keeping the task on the edge side are: S24, Normalized resource time overhead and task value calculation: Let the estimated completion time of the task on the current edge node be... Then its normalized resource time cost is: in , , These are the total number of CPU cores, memory capacity, and disk capacity of the edge nodes, respectively. The value of the task is defined as: The higher the value of a task, the more worthwhile it is to prioritize the use of edge resources for that task; Step S4 specifically involves, S41, Local Acceptance Decision: When the task... Reaching the edge node When doing so, first determine whether local resources are sufficient, that is, the available resource vector of the current edge node E. Resource requirement vector not less than task T And whether the task value meets the standard: If both conditions are met, the task will be deployed locally. S42, Edge-to-edge collaborative offloading: Edge nodes periodically exchange their current value thresholds through the network. and resource status; S421, If ​​the local node cannot accept the request, the scheduler iterates through the list of other candidate edge nodes. Filter out those that meet the requirements And candidate edge nodes with sufficient resources; S422, attempt to unload the task to a candidate edge node. If the unloading is successful, the task will be executed on the neighboring edge node. S43, Cloud-based Collaborative Unloading: If all edge nodes are unable to accept the task, the cloud-based unloading process will begin. S431, Traverse all available cloud nodes Based on the performance model established in steps S1 and S2, the total cost of the task on each cloud node is calculated. ; S432, select the cloud node with the lowest total cost. ; S433, if the minimum cost is still higher than the maximum penalty for direct failure of the task. If the task fails, the task is abandoned; otherwise, it is rescheduled to a cloud node. implement.

2. The method according to claim 1, characterized in that, Step S1 specifically involves: S11, Time Window Choice: Define index P1 as the percentage difference between the sample means of the first half and the second half of the window, which is used to measure the historical data staleness error caused by an excessively long window; define index P2 as the percentage of the standard error of the sample mean relative to the sample mean, which is used to measure the sample size insufficient error caused by an excessively short window; then the candidate window that minimizes P1+P2 is the current best historical time window. S12, Historical performance data collection and normalization: for windows Internal cloud nodes The historical mission records completed above are statistically analyzed; for any historical mission... Based on its computational complexity Disk I / O volume Samples of time required per unit of CPU computation and time required per unit of I / O transfer were constructed respectively: in, For historical mission In cloud nodes The time required for a unit of CPU computation during execution; For historical mission In cloud nodes Time required for a unit I / O transfer during execution; For historical mission In cloud nodes A sample of the time required for the corresponding unit of CPU computation; For historical mission In cloud nodes Samples of the time required for the corresponding unit I / O transfer volume; S13, Cloud Node Performance Distribution Modeling: Establish statistical distribution models for the time required per unit of CPU computation and the time required per unit of I / O transfer volume. in, For cloud nodes The time required for unit CPU computation is a random variable; For cloud nodes The time required to process a unit of I / O is a random variable; For cloud nodes The average time required per unit of CPU computation; For cloud nodes The standard deviation of time required per unit of CPU computation; For cloud nodes The average time required to process a unit of I / O; For cloud nodes The standard deviation of time required to process one unit of I / O; Represents a normal distribution, where This represents the mean. Indicates variance; This model simultaneously characterizes the average level and fluctuations of cloud node CPU and I / O performance; S15, Completion Time Distribution Calculation: Based on the above model, for newly arrived tasks... If its computational complexity is Disk I / O quantity is Predict its location in this cloud node The random variable of expected completion time Random variable with time required per unit CPU computation and the random variable of time required to process a unit of I / O The relationship is Its mean With variance They are respectively: Subsequent modules calculate the expected timeout penalty for tasks executed in the cloud based on the completion time distribution.

3. The method according to claim 2, characterized in that, Step S1 also includes, S14, Active Probe Mechanism: To ensure the accuracy of the statistical model, a minimum sample threshold is set. If the current time window If the number of passive historical samples collected internally is insufficient, the edge nodes will automatically generate and send lightweight active probing tasks to the cloud nodes to supplement performance sampling points and ensure the accuracy of performance estimation. The active probing task is a lightweight benchmark task constructed by the edge node and sent to the cloud node for execution. It is used to simultaneously measure the CPU service capability and I / O service capability of the cloud node. The benchmark test is run multiple times on the cloud node to collect performance points and thus supplement the number of performance samples.

4. The method according to claim 1, characterized in that, Step S3 specifically involves: S31, Historical value threshold statistics of edge nodes: Review the task records of multiple past time slices with a fixed length of time slice. For any time slice, filter the active tasks that overlap with the time slice, calculate the normalized resource time occupied by them in the time slice, and sort them from high to low according to their task value. Then, load them into the available resource capacity of the time slice in turn. The value of the last task that can still be loaded is taken as the ideal value threshold of the time slice. S32, Prediction of Future Thresholds: Based on the historical value threshold sequence obtained through backtracking calculations, time series forecasting algorithms are used to analyze its changing trends and predict the optimal value threshold for the current time period. And apply it to the current scheduling decisions.

Images