Asynchronous task scheduling and computing resource joint optimization method based on cloud edge collaboration

Abstract

The invention relates to a cloud-side collaborative asynchronous task scheduling and computing resource joint optimization method, which comprises the following steps of: in a cloud-side collaborative computing network, acquiring asynchronously arriving task information, edge equipment state information and server resource state information in real time; calculating the priority weight of each task in the operation cycle according to the time characteristic and the calculation requirement of the task information, and executing preemptive reordering on the task queue based on the priority weights; and constructing a system state vector based on the task information, the priority weight, the edge device state information and the server resource state information, generating a mixed action through a pre-trained deep reinforcement learning model, and executing task scheduling and computing resource allocation in an operation cycle based on the mixed action. According to the method, joint optimization of asynchronous task scheduling and resource allocation is realized through deep reinforcement learning, and the performance of a cloud edge cooperative computing system is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing and edge computing technology, and in particular, it is a method for asynchronous task scheduling and joint optimization of computing resources in a cloud-edge collaborative manner. Background Technology

[0002] With the rapid development of 5G networks, the Internet of Things (IoT), and artificial intelligence (AI) technologies, applications such as autonomous driving and augmented reality are increasingly demanding real-time and computationally intensive tasks. Traditional cloud computing architectures suffer from latency issues due to long-distance data transmission, making it difficult to meet millisecond-level response requirements. Mobile edge computing (MEC) deploys computing nodes at the network edge, bringing computing power closer to terminal devices, significantly reducing network latency and alleviating backbone network pressure. However, MEC's ​​limited computing resources restrict its ability to handle complex tasks. Cloud-edge collaborative computing, combining the low latency of the edge with the powerful computing capabilities of the cloud, has become a key technology for resolving this contradiction. Currently, cloud-edge collaboration faces multiple challenges: task arrival times are random and asynchronous; edge devices rely on energy harvesting technology for power, resulting in dynamic energy status changes; edge servers have limited computing resources, while cloud processing is constrained by communication latency. Furthermore, task quality of service requirements vary; for example, real-time tasks require edge-priority processing, while computationally intensive tasks rely on cloud computing power. Traditional synchronous scheduling methods require waiting for all tasks to arrive before processing, leading to resource waste and timeouts for high-priority tasks; existing optimization algorithms have high computational complexity and are difficult to adapt to task asynchronicity and dynamic changes in device energy.

[0003] Therefore, there is an urgent need for an intelligent and flexible scheduling and resource allocation method to achieve efficient optimization of cloud-edge collaboration. Summary of the Invention

[0004] The purpose of this invention is to provide a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method to solve the problems of low efficiency in task scheduling and resource allocation caused by asynchronous task arrival, dynamic changes in edge device energy, and limited resources in existing cloud-edge collaborative computing networks.

[0005] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method, the method comprising: In a cloud-edge collaborative computing network, real-time acquisition of asynchronously arriving task information, edge device status information, and server resource status information is achieved. Based on the time characteristics and computational requirements of the task information, the priority weight of each task is calculated within the running cycle, and the task queue is preemptively reordered based on the priority weight. Based on task information, priority weights, edge device status information, and server resource status information, a system state vector is constructed. Hybrid actions are generated through a pre-trained deep reinforcement learning model. Based on the hybrid actions, task scheduling and computing resource allocation are performed within the runtime cycle.

[0006] As a further improvement to one embodiment of the present invention, the real-time acquisition of asynchronously arriving task information, edge device status information, and server resource status information includes, upon system startup, initializing the cloud-edge collaborative computing network and modeling asynchronous task arrival using a Poisson distribution; wherein the task arrival interval Δ It follows an exponential distribution Exp(λ), where k is the task and λ is the average arrival rate; The runtime T is divided into N+2 time slots using a time-division multiple access (TDMA) mechanism, including: Time slots are used for edge devices to obtain initial energy from mobile edge computing servers via wireless power transfer; A time slot, comprising multiple sub-time slots, is used for task offloading; each edge device offloads tasks to the edge server via a wireless channel within its assigned sub-time slot, and each sub-time slot is allocated to only one edge device, generating an initial task queue. Time slot to It is used for edge servers and cloud servers to process tasks in parallel and update task status information and resource usage in real time. Time slot It is used to aggregate task execution results between edge servers and cloud servers, including task completion time. and the number of tasks completed Where N is the preset number of parallel processing time slots; Edge device status information is acquired at a fixed sampling frequency; the edge device status information includes energy status and channel status. ;in This represents the channel gain between edge device i and edge server e. This represents the channel gain between edge device i and cloud server c; the energy state includes the current energy level and energy harvesting rate of the edge device. Record the remaining computing resources of the edge server and remaining computing resources of cloud servers .

[0007] As a further improvement to one embodiment of the present invention, the method further includes, in that, calculating the priority weight of each task includes, Time slot Within, based on the initial task queue, the arrival time of task k is... Remaining deadline And calculating urgency Normalization was performed to generate arrival time factors. Remaining time factor and urgency factor The formula is: Where T is the operating cycle, This is the threshold for task processing time. At the highest level of urgency; The priority weights are calculated by weighted summation, using the following formula: in, Let k be the priority weight. , , These are the weighting coefficients.

[0008] As a further improvement to one embodiment of the present invention, the method further includes the preemptive reordering of the task queue based on priority weights. In the time slot to Within the current task, when the priority weight of a newly arrived task exceeds the priority weight of the currently executing task, and the difference is greater than a preset threshold ΔP, the currently executing task is interrupted and its calculation progress is saved to a memory queue. The task queue for newly arrived tasks is reordered in descending order according to priority weights to generate an optimized task queue, and the corresponding priority weights are stored.

[0009] As a further improvement to one embodiment of the present invention, the method further includes, in the step of constructing the system state vector and generating hybrid actions through a pre-trained deep reinforcement learning model, the following steps are taken: In the time slot to Internally, construct the system state vector. The system state vector Including task information Channel state Remaining computing resources of edge servers and cloud servers and the remaining time of the running cycle T ;in, Let k be the computational cost of task k. The resource requirements for task k; The system state vector Input a pre-trained deep reinforcement learning model to generate hybrid actions Where t is the current time slot index; The hybrid action include, User scheduling decisions Used to determine time slots The edge device i that offloads the task within sub-time slot n; where, Indicates that device i is in time slot The nth sub-slot unloading task k within the time slot, Indicates idle; the deep reinforcement learning model is based on the system state vector. Priority weights in Prioritize selecting high-efficiency tasks from the optimized task queue. Task k is assigned to device i; Unloading target decision , used to determine the processing location of task k; where, This indicates that task k is processed on the edge server. This indicates that the data is being routed to the cloud server; when the channel gain... Exceeding the predetermined threshold or remaining computing resources When sufficient, the deep reinforcement learning model is based on the system state vector. Priority weights in And edge server status, prioritize high. Task k is assigned to the edge server; Computing resource allocation decision This is used to allocate computing resources from edge servers or cloud servers to task k, and to satisfy the maximum computing resource constraint. and ; The user scheduling decision and unloading target decision are generated using an ε-greedy strategy, selecting the optimal action with a probability of 1−ε and randomly exploring with a probability of ε; the computing resource allocation decision is generated by outputting the mean of the actor network and superimposing Gaussian noise, ensuring that the allocated resources do not exceed the remaining resources of the edge server or cloud server.

[0010] As a further improvement to one embodiment of the present invention, the method further includes, the training step of the deep reinforcement learning model includes, In the interaction with the cloud-edge collaborative computing network, in time slots to Internal execution action Then, generate the state transition quadruple. And store it in the experience pool; among which, For the reward function; When the number of samples in the experience pool is not less than a preset threshold, a preset number of samples are randomly sampled from it. The temporal difference error is calculated based on the sampled data and Bellman equation, and the critic network is updated to minimize the mean square error loss. The actor network is updated using the policy gradient ascent method to optimize the selection strategy of mixed actions until the model converges or reaches the predetermined number of training steps.

[0011] As a further improvement to one embodiment of the present invention, the method further includes that the deep reinforcement learning model is optimized based on a reward function, the formula of which is: in, As an energy consumption reward, For time slots to The total energy consumption of the inner edge server or cloud server to complete the task; Rewards for timely completion of tasks. For time slots to The actual completion time of each task within the task. The maximum delay specified; Incentives for resource utilization For time slots to The total amount of computing resources actually used internally. This represents the maximum computing resources available to all servers. C is the base reward value for completing the task. For time slots to The number of tasks completed within the specified timeframe; , , , These are the weighting coefficients.

[0012] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization system, the system comprising a data acquisition module, a reordering module, and a task scheduling and resource allocation module; The data acquisition module is used to acquire asynchronously arriving task information, edge device status information, and server resource status information in real time in the cloud-edge collaborative computing network. The reordering module is used to calculate the priority weight of each task within the running cycle according to the time characteristics and computing requirements of the task information, and to perform preemptive reordering of the task queue based on the priority weight. The task scheduling and resource allocation module is used to construct a system state vector based on task information, priority weights, edge device status information, and server resource status information. It generates hybrid actions through a pre-trained deep reinforcement learning model and performs task scheduling and allocation of computing resources based on the hybrid actions during the runtime cycle.

[0013] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides an electronic device, including a memory and a processor, characterized in that the memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method as described above.

[0014] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides a storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method described above.

[0015] Compared with existing technologies, this invention provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method. Through deep reinforcement learning, it achieves joint optimization of asynchronous task scheduling and resource allocation, significantly improving the performance of cloud-edge collaborative computing systems. The method can perceive task characteristics and device status in real time, dynamically adjust scheduling strategies and resource allocation, reduce latency for high-priority tasks, and ensure service quality; simultaneously, it optimizes system energy consumption and improves resource utilization. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of the asynchronous task scheduling and computing resource joint optimization method for cloud-edge collaboration described in this invention.

[0017] Figure 2 This is a schematic diagram of asynchronous multi-device task computation in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method described in this invention.

[0018] Figure 3 This is a flowchart of the training process of the deep reinforcement learning model for the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method described in this invention.

[0019] Figure 4 This is a schematic diagram of the architecture of the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization system described in this invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0021] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0022] In Embodiment 1 of the present invention, the present invention provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method, such as... Figure 1 As shown, the method includes, S1: In the cloud-edge collaborative computing network, acquire asynchronously arriving task information, edge device status information, and server resource status information in real time; S2: Based on the time characteristics and computational requirements of the task information, calculate the priority weight of each task within the running cycle, and perform preemptive reordering of the task queue based on the priority weight. S3: Based on task information, priority weights, edge device status information, and server resource status information, a system state vector is constructed. Hybrid actions are generated through a pre-trained deep reinforcement learning model. Based on the hybrid actions, task scheduling and computing resource allocation are performed within the runtime cycle.

[0023] In one specific embodiment of the present invention, asynchronously arriving task information, edge device status information, and server resource status information are acquired in real time. Specifically, Upon system startup, the cloud-edge collaborative computing network is initialized, and asynchronous task arrival is modeled using a Poisson distribution; where the task arrival interval Δ It follows an exponential distribution Exp(λ), where k is the task and λ is the average arrival rate; The runtime T is divided into N+2 time slots using a time-division multiple access (TDMA) mechanism, including: Time slots are used for edge devices to obtain initial energy from mobile edge computing servers via wireless power transfer; A time slot, comprising multiple sub-time slots, is used for task offloading; each edge device offloads tasks to the edge server via a wireless channel within its assigned sub-time slot, and each sub-time slot is allocated to only one edge device, generating an initial task queue. Time slot to It is used for edge servers and cloud servers to process tasks in parallel and update task status information and resource usage in real time. Time slot It is used to aggregate task execution results between edge servers and cloud servers, including task completion time. and the number of tasks completed Where N is the preset number of parallel processing time slots; Edge device status information is acquired at a fixed sampling frequency; the edge device status information includes energy status and channel status. ;in This represents the channel gain between edge device i and edge server e. This represents the channel gain between edge device i and cloud server c; the energy state includes the current energy level and energy harvesting rate of the edge device. Record the remaining computing resources of the edge server and remaining computing resources of cloud servers .

[0024] It should be noted that, in order to realize the method of asynchronous task scheduling and joint optimization of computing resources in cloud-edge collaborative computing network, this invention obtains asynchronously arriving task information, edge device status information and server resource status information by initializing cloud-edge collaborative computing network, modeling task arrival, adopting time-division multiple access mechanism and real-time monitoring of edge device and server status when the system starts.

[0025] Furthermore, during the system startup phase, the cloud-edge collaborative computing network is initialized to ensure the normal operation of task scheduling and resource allocation. The cloud-edge collaborative computing network comprises multiple edge devices, edge servers (i.e., mobile edge computing servers, MEC servers), and cloud servers. Edge devices are energy-harvesting devices that obtain energy from edge servers via wireless power transfer technology. The initialization process includes the following steps: Network architecture configuration: Configure the set of edge devices Where I represents the number of edge devices; initialize the multi-core CPU computing resources of the edge server and record the maximum computing resources. Initialize large-scale computing resources on the cloud server and record the maximum computing resources. ; Communication channel configuration: Establish wireless communication channels between edge devices, edge servers, and cloud servers, and record the initial channel gain. .

[0026] Operation cycle division: such as Figure 2As shown, the system operation period T is set, and the period T is divided into N+2 time slots through the time division multiple access (TDMA) mechanism. The time slot division provides a time framework for task unloading, processing and result aggregation through the time division multiple access mechanism, and generates an initial task queue, task status information and result data, which respectively support priority weight calculation, system state vector construction and reward function optimization.

[0027] Furthermore, to simulate the randomness of asynchronous task arrival, a Poisson distribution is used to model the task arrival process. The task arrival interval Δ The distribution follows an exponential distribution Exp(λ), where k is the number of tasks and λ is the average arrival rate; λ represents the expected frequency of task arrivals per unit time. The specific implementation is as follows: Edge devices generate computing tasks based on business needs, and task characteristics include arrival time. Remaining deadline Calculation workload and computing resource requirements .

[0028] The edge server records task arrival times in real time through a task monitoring module. The system generates a task arrival sequence based on a Poisson distribution. This sequence is stored in a task queue for subsequent priority calculation and scheduling.

[0029] To ensure modeling accuracy, the average arrival rate λ is dynamically adjusted according to the application scenario (such as real-time tasks or computationally intensive tasks). For example, λ is increased for high-frequency task scenarios to simulate dense task arrivals.

[0030] Furthermore, to coordinate task offloading and processing among edge devices, a Time Division Multiple Access (TDMA) mechanism is adopted, dividing the runtime T into N+2 time slots to ensure the efficiency and orderliness of task processing. The specific time slot functions are as follows: Time slot (Energy Harvesting Phase): Edge devices acquire initial energy from the edge server via wireless energy transmission technology; the edge server is equipped with a wireless energy transmission module to send energy signals to the edge devices, and the devices convert and store the energy through an energy receiving module (such as an RF energy harvester); the energy harvesting rate is dynamically adjusted according to the device distance, channel status, and transmission power.

[0031] Time slot (Task Offloading Phase): Each edge device offloads its task to the edge server via a wireless channel within its allocated sub-slot. A TDMA mechanism is employed to ensure that each sub-slot is allocated to only one edge device; the allocation decision is made using variables. The number of sub-time slots is determined by the edge server based on the number of devices and the bandwidth capacity of each sub-time slot. After task unloading is complete, the edge server determines the processing location (edge ​​server or cloud server) based on the task characteristics. After task unloading is complete, the edge server generates an initial task queue, containing task arrival times. Calculation workload and resource requirements It is used for priority weight calculation and system state vector construction.

[0032] Time slot to (Task Processing Phase): Edge servers and cloud servers process tasks in parallel. Edge servers handle tasks with lower computational demands or those sensitive to latency, while cloud servers handle tasks with higher computational demands. The two servers synchronously update task status, including task progress and resource usage, via a high-speed communication link to ensure coordinated task processing. Task status information generated during the task processing phase (including task progress and resource usage) is updated in real-time to the task management database, serving as input to the system state vector for generating hybrid actions in the deep reinforcement learning model. Simultaneously, task processing data supports energy consumption and timeliness calculations for the reward function.

[0033] Time slot (Results Aggregation Phase): Edge servers and cloud servers aggregate task execution results, including metrics such as task completion time, energy consumption, and resource utilization. The aggregated results are used to verify that all tasks were completed within the runtime T and serve as input data for subsequent optimizations (such as training deep reinforcement learning models). The metrics generated during the results aggregation phase are stored in the task management database and used as the reward function. The input is used to evaluate system performance and guide the training of deep reinforcement learning models.

[0034] Furthermore, edge device status information, including energy status and channel status, is acquired in real time via sensors to support task scheduling and resource allocation. The specific implementation is as follows: Energy Status Acquisition: Edge devices are equipped with voltage and current sensors to collect energy status data, including the current energy value, at a predetermined frequency. and energy harvesting rate Sensor data is transmitted to the edge server via a low-power communication protocol. The edge server maintains the device's state of energy table, which is used to dynamically adjust task offloading and processing strategies.

[0035] Channel State Acquisition: The edge server uses a Channel State Information (CSI) measurement module to acquire the channel gain between the edge device, the edge server, and the cloud server in real time. The channel gain changes dynamically based on device location, environmental interference, and transmission power, and is updated periodically (e.g., once per time slot) to ensure communication reliability.

[0036] Furthermore, the resource status information for edge servers and cloud servers includes remaining computing resources. This information is obtained in real time through the resource monitoring module. The specific implementation is as follows: Edge server resource monitoring: The edge server is equipped with a resource management module that records the remaining computing resources of multi-core CPUs in real time. It is dynamically updated based on the progress of task processing.

[0037] Cloud server resource monitoring: Cloud servers monitor the remaining capacity of large-scale computing resources through a distributed computing framework. It records the number of available virtual machines or containers and synchronizes resource status with edge servers via a high-speed network.

[0038] Data storage and updates: Server resource status information is stored in a resource status database and updated periodically (e.g., after each time slot or after each task is completed) to support dynamic decision-making by deep reinforcement learning models.

[0039] Furthermore, task information, edge device status information, and server resource status information are integrated through the edge server's task management module to generate a unified data structure for subsequent priority calculation and deep reinforcement learning model use. Data transmission employs an efficient protocol to ensure low latency and high reliability. The edge server synchronizes data using timestamps to ensure the temporal consistency of task information, status information, and resource information.

[0040] In one specific embodiment of the present invention, the priority weight of each task is calculated, specifically as follows: Time slot Within, based on the initial task queue, the arrival time of task k is... Remaining deadline And calculating urgency Normalization was performed to generate arrival time factors. Remaining time factor and urgency factor The formula is: Where T is the operating cycle, This is the threshold for task processing time. At the highest level of urgency; The priority weights are calculated by weighted summation, using the following formula: in, Let k be the priority weight. , , These are the weighting coefficients.

[0041] Preferred, , , The method for determining the weighting coefficients is as follows: By using the analytic hierarchy process (AHP), the relative importance of weight coefficients is determined based on the weighted impact of task type on system performance. For real-time tasks, increase the weighting coefficient. Prioritize remaining deadlines to ensure time slots are available. to Timely handling of internal tasks; For computationally intensive tasks, increase the weight coefficient. Prioritize computational urgency and support time slots. to Efficient resource allocation within the organization; For typical periodic tasks, the weighting coefficients are balanced. To ensure fairness in arrival times and maintain fair scheduling of task queues.

[0042] The task queue is preemptively reordered based on priority weights. Specifically: In the time slot to Within the current task, when the priority weight of a newly arrived task exceeds the priority weight of the currently executing task, and the difference is greater than a preset threshold ΔP, the currently executing task is interrupted and its calculation progress is saved to a memory queue. The task queue for newly arrived tasks is reordered in descending order according to priority weights to generate an optimized task queue, and the corresponding priority weights are stored.

[0043] It should be noted that, in order to achieve dynamic scheduling of asynchronous tasks in cloud-edge collaborative computing networks, this invention optimizes the task processing order by normalizing task information, calculating priority weights, executing preemptive interrupts, and reordering task queues, thereby improving system energy efficiency and task timeliness.

[0044] Furthermore, based on time slots The generated initial task queue is used to extract task information, including arrival time. Remaining deadline And calculating urgency To ensure the comparability of these parameters, normalization is used to generate factors with uniform dimensions, as specifically implemented below: Arrival time factor generation: Obtain the arrival time of each task through the task management module. And calculate the arrival time factor based on the operating cycle T. .

[0045] Remaining deadline factor generation: Records the remaining deadline for each task. And based on the task processing time threshold (Calculate the remaining time factor based on the task type, such as a strict deadline for a real-time task) .

[0046] Urgency factor generation: Calculates the urgency level of the task. (Determined by task type, e.g., real-time tasks are high urgency, and ordinary tasks are low urgency), combined with the maximum urgency level. (System-defined upper limit for urgency) Calculate urgency factor .

[0047] Data storage: Normalized factors , and The data is stored in the task management database for subsequent priority weight calculations. Normalization is performed automatically by the task scheduling module of the edge server at a fixed frequency (e.g., once per time slot) to ensure real-time performance.

[0048] Furthermore, based on the normalized factors, the priority weight of each task is calculated through weighted summation to comprehensively assess the importance and urgency of the tasks. The specific implementation is as follows: Determining the weighting coefficients: Weighting coefficients , , The relative importance of system performance objectives (such as energy efficiency, timeliness, and resource utilization) is assessed based on task type using the Analytic Hierarchy Process (AHP).

[0049] Preferably, differentiated settings are implemented for different task types: In real-time tasks (such as autonomous driving decision-making) The highest weight is given to tasks that focus on remaining deadline constraints; in computationally intensive tasks (such as big data analytics) For tasks with higher weights, the impact of urgency on subsequent queues should be considered first; in ordinary periodic tasks... The weights are appropriate, balancing the fairness of arrival time.

[0050] Weighted summation: The task scheduling module calculates the priority weights according to the formula, satisfying... To ensure weight normalization. Calculation results. Stored in the task management database, each task is marked with its priority.

[0051] Dynamic updates: priority weights The system is dynamically updated at each time slot or when a new task arrives to reflect the latest state of the task queue. The update process is efficiently executed by the computing modules of the edge server, ensuring low computational overhead.

[0052] Furthermore, to prioritize high-urgency tasks, the system employs a preemptive scheduling mechanism, specifically implemented as follows: Priority comparison: When a new task arrives, the task scheduling module calculates its priority weight. and the priority weight of the currently executing task. Compare them. Retrieved from the task management database, reflecting the highest priority task being processed in the current time slot.

[0053] Interruption determination: If > If the difference exceeds a predetermined threshold ΔP, a preemption mechanism is triggered. The currently executing task is interrupted, its computation process is paused, and the task status (including completed computations and intermediate results) is stored in a memory queue via the task management module of the edge server. The memory queue uses an efficient data structure (such as a double-ended queue) to support fast access and recovery. The preemption threshold ΔP is determined by system performance requirements; for example, a smaller value is used for real-time tasks, and a larger value is used for ordinary tasks. This value is dynamically adjusted by the task management module based on the task type.

[0054] Interruption Management: The storage of interrupted tasks includes task identifiers, current computation progress, and required resource information to ensure seamless recovery upon resumption. The system tracks interruption events through a logging module for performance analysis and optimization.

[0055] Furthermore, to ensure that tasks are executed in priority order, the quicksort algorithm is used to reorder the task queue in descending order, as implemented below: Queue Updates: The task queue contains newly arrived tasks and interrupted tasks, stored in the task management database on the edge server. Each task in the queue is associated with its priority weight. .

[0056] Quick sort implementation: The quick sort algorithm is used, with priority weights. Use the sort key to sort the task queue in descending order.

[0057] Reordering Triggered: Reordering is triggered after a new task arrives or a task is interrupted, ensuring that the queue always reflects the latest priority order. The sorting results are updated to the task management database for use in task scheduling of subsequent time slots.

[0058] Optimization measures: To reduce sorting overhead, the system can switch to the insertion sort algorithm when the number of tasks is small, and adopt parallel sorting optimization (such as multi-threaded processing) in high-load scenarios.

[0059] Furthermore, the reordered task queue includes priority weights. Stored in the task management database as a system state vector Components used by deep reinforcement learning models to generate hybrid actions .

[0060] Priority weight The reordered task queues are integrated with the task scheduling module of the edge server to generate a task scheduling table containing task identifiers and priority weights. This scheduling table is transmitted in real-time to the deep reinforcement learning model, serving as a crucial component of the system state vector for generating hybrid actions (user scheduling decisions, offloading target decisions, and computational resource allocation decisions). Data transmission employs an efficient protocol to ensure low latency and high reliability. The task scheduling table is updated synchronously with edge device status information and server resource status information to maintain temporal consistency.

[0061] In one specific embodiment of the present invention, a system state vector is constructed, and hybrid actions are generated through a pre-trained deep reinforcement learning model, specifically, In the time slot to Internally, construct the system state vector. The system state vector Including task information Channel state Remaining computing resources of edge servers and cloud servers and the remaining time of the running cycle T ;in, Let k be the computational cost of task k. The resource requirements for task k; The system state vector Input a pre-trained deep reinforcement learning model to generate hybrid actions Where t is the current time slot index; The hybrid action include, User scheduling decisions Used to determine time slots The edge device i that offloads the task within sub-time slot n; where, This indicates that device i unloads task k in the nth sub-time slot within the time slot. Indicates idle; the deep reinforcement learning model is based on the system state vector. Priority weights in Prioritize selecting high-efficiency tasks from the optimized task queue. Task k is assigned to device i; Unloading target decision , used to determine the processing location of task k; where, This indicates that task k is processed on the edge server. This indicates that the data is being routed to the cloud server; when the channel gain... Exceeding the predetermined threshold or remaining computing resources When sufficient, the deep reinforcement learning model is based on the system state vector. Priority weights in And edge server status, prioritize high. Task k is assigned to the edge server; Computing resource allocation decision This is used to allocate computing resources from edge servers or cloud servers to task k, and to satisfy the maximum computing resource constraint. and ; The user scheduling decision and unloading target decision are generated using an ε-greedy strategy, selecting the optimal action with a probability of 1−ε and randomly exploring with a probability of ε; the computing resource allocation decision is generated by outputting the mean of the actor network and superimposing Gaussian noise, ensuring that the allocated resources do not exceed the remaining resources of the edge server or cloud server.

[0062] It should be noted that, in order to realize the dynamic scheduling and resource allocation of asynchronous tasks in the cloud-edge collaborative computing network, this invention optimizes the task scheduling and resource allocation strategy by constructing a system state vector and using a pre-trained deep reinforcement learning (DRL) model to generate hybrid actions.

[0063] Furthermore, the system state vector Integrating time slots in cloud-edge collaborative computing networks to The dynamic information within is used as input to the deep reinforcement learning model to generate scheduling and resource allocation decisions. The specific implementation is as follows: Task information integration: Task information includes task computation volume Task resource requirements and priority weight This information is retrieved from the task management database on the edge server and updated in real time to reflect the status of newly arriving tasks or interrupted tasks.

[0064] Channel state acquisition: Channel state includes the channel gain of edge device i and edge server i. and channel gain with cloud servers Channel state information (CSI) is acquired periodically through the edge server's channel state information (CSI) measurement module. Channel gain data is stored in the communication state database, reflecting dynamic changes in the wireless communication environment.

[0065] Server resource status acquisition: The remaining computing resources of edge servers and cloud servers are respectively represented as follows: and The resource management module monitors resources in real time. Edge servers record the available resources of multi-core CPUs, while cloud servers record the available virtual machine or container capacity through a distributed computing framework.

[0066] Remaining time record: Remaining time This represents the remaining available time within the current running period T, calculated by the task scheduling module of the edge server based on the current time slot and period T.

[0067] State vector construction: System state vector Integrate into structured data The state vectors are stored in a state management database on the edge server. These state vectors are updated periodically via an efficient protocol to ensure temporal consistency with the states of tasks, channels, and resources.

[0068] Furthermore, the pre-trained deep reinforcement learning model receives the system state vector. Generate hybrid actions This optimizes task scheduling and resource allocation. The specific implementation is as follows: Model Architecture: The DRL model employs an Actor-Critic framework, comprising an actor network and a critic network. The actor network is responsible for generating hybrid actions, while the critic network evaluates the expected rewards of the actions. During the training phase, the model is optimized through interaction with a cloud-edge collaborative computing network environment and stored in a model database on an edge server.

[0069] Input processing: State vector The data is normalized using a data preprocessing module to improve the model's convergence and stability. The normalized data is then processed. Input the actor network through an efficient interface.

[0070] Model Execution: Actor Network Based on Generate hybrid actions The critic network outputs a value estimate of the action, which is used to guide online decision-making. The model execution process runs on the high-performance computing unit of the edge server to ensure real-time performance.

[0071] Furthermore, mixed actions It includes user scheduling decisions, offloading target decisions, and computing resource allocation decisions, specifically implemented as follows: User scheduling decisions Generated by the actor network of a deep reinforcement learning model, based on the system state vector. Priority weights in Prioritize selecting high-efficiency tasks from the optimized task queue. Task k is assigned to device i, and settings are configured. This indicates that device i unloads task k in the nth sub-time slot (n∈{1,2,...,M}) of time slot 1; if no task is assigned, then... , indicating idle. Optimized task queues are ordered by... Sorted in descending order, stored in the task management database, high The task is at the top of the queue, with a priority threshold. (Initialized by the system based on the urgency of the task, with higher values ​​for real-time tasks and lower values ​​for ordinary tasks, and stored in the task management database) Used to filter high-priority tasks.

[0072] Unloading target decision Generated by an actor network, based on In and edge server status (channel gain) Remaining computing resources ), with higher priority Task( ≥ ) allocate edge servers ( ),when Higher than the predetermined threshold (Based on channel quality initialization, reflecting the median value of channel gain, determined by the channel state information measurement module) or Above the resource threshold (Based on edge server computing capability initialization, reflecting the minimum available resources of a multi-core CPU, determined by the resource management module) when; otherwise, The tasks are then routed to cloud servers. This decision-making process is executed within each sub-slot n, optimizing task unloading efficiency and reducing latency for high-priority tasks.

[0073] Computing resource allocation decision Allocate computing resources to edge servers or cloud servers for tasks to meet maximum computing resource constraints. and Resource allocation decisions are generated by the output layer of the actor network's continuous actions, outputting the mean of resource allocation and superimposing Gaussian noise (the standard deviation is preset by system performance requirements) to increase exploratory power. The decision results are validated by the resource management module to ensure... and .

[0074] Motion integration: Mixed motion Integrate into The task scheduling database is used to store data in the scheduling execution module, which transmits it to the edge server and cloud server via an efficient protocol.

[0075] Furthermore, to balance exploration and exploitation, the ε-greedy strategy and the Gaussian noise mechanism for resource allocation are implemented in the following ways: ε-greedy strategy: The ε value for user scheduling decisions and unloading target decisions is dynamically adjusted based on system load and determined by the task scheduling module according to task queue length and resource utilization. The strategy is implemented through a random number generator (based on a pseudo-random algorithm) to ensure the fairness of random exploration.

[0076] Gaussian noise generation: Gaussian noise for resource allocation decisions is generated using a normal distribution sampler in the actor network, with the noise standard deviation dynamically adjusted based on the remaining server resources. The noise generation process is executed by the edge server's computing module to optimize real-time performance.

[0077] Constraint Validation: Resource allocation decisions are validated through the resource management module to ensure that allocated resources do not exceed the limit. and If the constraints are exceeded, the assigned value is mapped to the feasible region using a projection method.

[0078] Furthermore, mixed actions The task scheduling module integrates these into scheduling commands, including device selection. Processing location and resource allocation Instructions are stored in the task scheduling database and transmitted via efficient protocols to the execution modules of edge servers and cloud servers for task unloading, processing, and resource allocation. Action generation results and state vectors are also included. Timestamp synchronization ensures that scheduling decisions are consistent with the system state.

[0079] In one specific embodiment of the present invention, the training steps of the deep reinforcement learning model are as follows: In the interaction with the cloud-edge collaborative computing network, in time slots to Internal execution action Then, generate the state transition quadruple. And store it in the experience pool; among which, For the reward function; When the number of samples in the experience pool is not less than a preset threshold, a preset number of samples are randomly sampled from it. The temporal difference error is calculated based on the sampled data and Bellman equation, and the critic network is updated to minimize the mean square error loss. The actor network is updated using the policy gradient ascent method to optimize the selection strategy of mixed actions until the model converges or reaches the predetermined number of training steps.

[0080] It should be noted that, in order to train a deep reinforcement learning (DRL) model that jointly optimizes asynchronous task scheduling and computing resources in a cloud-edge collaborative computing network, this invention optimizes the decision-making ability of the DRL model by performing actions, generating state transition quadruples, sampling experience, and updating network parameters in a cloud-edge collaborative computing network environment or a simulation environment.

[0081] Furthermore, such as Figure 3 As shown, the training of the DRL model is completed through interaction with the cloud-edge collaborative computing network environment or its simulated environment, generating state transition quadruples to record training data. The specific implementation is as follows: Environment Interaction: In a cloud-edge collaborative computing network environment, the agent of the DRL model (running on the edge server) performs hybrid actions. Including user scheduling decisions Unloading target decision and computing resource allocation decisions .action Based on the current system state vector The generated state vector contains task information, channel state, remaining server resources, and remaining time. In a simulation environment, the behavior of a real network environment is modeled using a task arrival model based on a Poisson distribution and channel state simulation.

[0082] Feedback Acquisition: Action Execution Afterwards, the environment returns a reward value. and the next state Reward Value The next state is calculated using the task management and resource management modules of the edge server. Updated via sensors and communication modules.

[0083] Quadruple generation: Given the current state ,action ,award and the next state The state transition quadruple is generated by the training data management module of the edge server. .

[0084] Experience pool storage: Quadruples are stored in an experience pool of size B (e.g., B = 10000, predetermined by system memory capacity), managed using a first-in, first-out (FIFO) strategy. When the experience pool is full, a new quadruplet overwrites the oldest sample. The experience pool is deployed in an in-memory database on an edge server, supporting efficient access.

[0085] Furthermore, to ensure training stability and efficiency, the DRL model obtains training data from the experience pool through random sampling. The specific implementation is as follows: Sampling Trigger: The training module monitors the number of samples in the experience pool. When the number of samples reaches or exceeds a predetermined threshold τ based on the system performance requirements (determined by the experience pool capacity B and the training convergence speed, and preset by the training module according to the system performance requirements), the sampling process is triggered.

[0086] Random sampling: b samples are randomly selected from the experience pool (based on the experience pool capacity and training batch size), using a uniform random sampling algorithm (based on a pseudo-random number generator) to reduce sample correlation. The sampling process is executed by the training module of the edge server to ensure efficiency.

[0087] Sample Management: Samples are stored in a temporary training buffer and structured as follows: The batch data is used for subsequent network updates. The training module periodically cleans up the buffer to free up memory.

[0088] Furthermore, the Critic network is used to evaluate the expected reward of an action by updating the network parameters by minimizing the temporal difference error. The specific implementation is as follows:

[0089] Temporal Difference Error Calculation: Based on the sampled data and the Bellman equation, the temporal difference (TD) error is calculated. The Bellman equation uses the current reward... Next state The target value is calculated using a value estimate (output by the critic network) and a discount factor γ (e.g., γ = 0.99, based on a long-term benefit tradeoff). The TD error is the difference between the target value and the current value estimate.

[0090] Loss function optimization: The Critic network aims to minimize the mean squared error (MSE) loss, defined as the sum of squared TD errors. Optimization employs a gradient descent algorithm (such as the Adam optimizer) to update the parameters of the Critic network. The update process runs on the high-performance computing units of edge servers, updating once per batch of samples.

[0091] Parameter storage: Updated critic network parameters are stored in the model database and backed up periodically to support recovery after training interruptions.

[0092] Furthermore, the Actor network is used to generate hybrid actions, and the action selection policy is optimized using the policy gradient ascent method. The specific implementation is as follows: Policy gradient calculation: Based on the value estimation of the sampled samples and the critic network, the policy gradient is calculated. The gradient is determined by multiplying the logarithm of the action probability by the TD error, and the policy gradient ascent method is used to maximize the expected return.

[0093] Network Update: The parameters of the Actor network are updated using the gradient ascent algorithm to optimize the selection strategy for hybrid actions (including user scheduling, offloading targets, and resource allocation decisions). The update frequency is synchronized with the critic network, updating once per batch of samples.

[0094] Convergence control: The training process continues until the model converges (e.g., the rate of change of the policy gradient falls below a predetermined threshold) or a predetermined number of training steps is reached (based on system performance requirements). The training module tracks gradient changes through a convergence monitor and automatically terminates training.

[0095] Parameter storage: The optimized actor network parameters are stored in the model database and updated synchronously with the critic network parameters for use in online task scheduling.

[0096] Furthermore, the DRL model is trained and deployed on a dedicated training module on the edge server, supporting real-time interaction with the cloud-edge collaborative computing network. Training data (quadruples, sampled samples) and model parameters are synchronized between the edge server and the cloud server through an efficient protocol, ensuring model consistency in the distributed environment. After training, the optimized DRL model is deployed to the task scheduling module through the model loading interface for generating hybrid actions.

[0097] In one specific embodiment of the present invention, the deep reinforcement learning model is based on a reward function. The optimized formula is as follows: in, As an energy consumption reward, For time slots to The total energy consumption of the inner edge server or cloud server to complete the task; Rewards for timely completion of tasks. For time slots to The actual completion time of each task within the task. The maximum delay specified; Incentives for resource utilization This represents the total amount of computing resources actually used within a time slot. This represents the maximum computing resources available to all servers. C is the base reward value for completing the task. For time slots to The number of tasks completed within the specified timeframe; These are the weighting coefficients.

[0098] It should be noted that, to achieve the optimization of a deep reinforcement learning (DRL) model that jointly optimizes asynchronous task scheduling and computing resources in a cloud-edge collaborative computing network, this invention designs a reward function that includes energy consumption, task timeliness, resource utilization, and task completion rewards. This function is used to evaluate the effectiveness of actions and guide model training. The weight coefficients are optimized through grid search combined with the control variable method to balance system performance objectives.

[0099] Furthermore, energy consumption rewards Used to assess the total energy consumption of a server in completing a task, and to encourage energy reduction. Total Energy Consumption The energy consumption of tasks processed by edge servers and cloud servers is calculated by summing the data. Edge servers monitor the operating power of multi-core CPUs and task processing time through their resource management modules to calculate energy consumption at the edge. Cloud servers record the energy consumption of virtual machines or containers through a distributed computing framework, and these data are aggregated into the total energy consumption. .

[0100] Furthermore, rewards for timely task completion. Used to evaluate task completion time and encourage compliance with task latency requirements. Actual completion time for each task. Recorded by the task management module of the edge server, calculated based on the task completion timestamp. The specified maximum latency. (Predefined by task type) Stored in the task management database.

[0101] Furthermore, resource utilization incentives Used to evaluate the efficiency of computing resource utilization and encourage maximizing resource utilization. Total amount of computing resources actually used. The resource allocation based on task distribution is statistically analyzed by the resource management modules of edge servers and cloud servers. Cumulative calculation. Maximum computing resources across all servers. (Determined during system initialization) Stored in the resource management database.

[0102] Furthermore, there are rewards for completing the task. Used to incentivize task completion and improve system throughput. Number of completed tasks. The task management module compiles statistics based on task completion status (e.g., task results summary completed). The basic reward value C for task completion (predetermined by system performance requirements) is stored in the reward parameter database.

[0103] Furthermore, total reward The DRL model optimization is guided by a weighted summation of energy consumption, timeliness, resource utilization, and task completion rewards. The total reward is calculated by the reward calculation module based on each reward component and its weight coefficients μ, ν, ω, and δ. The weight coefficients satisfy μ + ν + ω + δ = 1 to ensure reward normalization.

[0104] Weight optimization: The weight coefficients are optimized using a combination of grid search and the controlled variable method. The steps are as follows: Grid search: Search within a predefined weight space (e.g., μ,ν,ω,δ∈[0,1], step size 0.1) to generate candidate weight combinations and evaluate the overall impact of each weight combination on system performance (energy efficiency, timeliness, resource utilization).

[0105] Controlled variable method: Fix some weights (such as μ, ν), adjust the remaining weights (such as ω, δ), test performance indicators (such as average task latency, total energy consumption) in a simulated environment, and select the optimal weight combination.

[0106] Scenario Adjustment: Dynamically adjust weights based on the application scenario (e.g., scenarios dominated by real-time tasks or compute-intensive tasks). For example, increase ν in real-time task scenarios to prioritize timeliness, and increase ω in compute-intensive task scenarios to optimize resource utilization.

[0107] Furthermore, the components of the reward function Total Rewards The reward database, stored on an edge server, uses a structured format with timestamps. The reward data is recorded. Reward data is synchronized with the cloud server via an efficient protocol to ensure consistency in reward calculation within the distributed environment. The reward calculation module collaborates with the task management and resource management modules to update reward values ​​in real time, supporting the training of DRL models and the generation of mixed actions.

[0108] In a second embodiment of the present invention, the present invention provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization system, such as... Figure 4 As shown, the system includes a data acquisition module 1, a reordering module 2, and a task scheduling and resource allocation module 3; The data acquisition module 1 is used to acquire asynchronously arriving task information, edge device status information, and server resource status information in real time in the cloud-edge collaborative computing network. The reordering module 2 is used to calculate the priority weight of each task within the running cycle according to the time characteristics and computing requirements of the task information, and to perform preemptive reordering of the task queue based on the priority weight. The task scheduling and resource allocation module 3 is used to construct a system state vector based on task information, priority weights, edge device status information and server resource status information, generate hybrid actions through a pre-trained deep reinforcement learning model, and perform task scheduling and computing resource allocation based on the hybrid actions within the running cycle.

[0109] In a third embodiment of the present invention, the present invention provides an electronic device, including a memory and a processor, characterized in that the memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method as described above.

[0110] In Embodiment 4 of the present invention, the present invention provides a storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method as described above.

[0111] In summary, this invention provides a cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method. Through deep reinforcement learning, it achieves joint optimization of asynchronous task scheduling and resource allocation, significantly improving the performance of cloud-edge collaborative computing systems. The method can perceive task characteristics and device status in real time, dynamically adjust scheduling strategies and resource allocation, reduce latency for high-priority tasks, and ensure service quality; simultaneously, it optimizes system energy consumption and improves resource utilization.

[0112] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the modules described above can be referred to the corresponding process in the aforementioned method implementation, and will not be repeated here.

[0113] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0114] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0115] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer system (which may be a personal computer, server, or network system, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method, characterized in that: include, In a cloud-edge collaborative computing network, real-time acquisition of asynchronously arriving task information, edge device status information, and server resource status information is achieved. Based on the time characteristics and computational requirements of the task information, the priority weight of each task is calculated within the running cycle, and the task queue is preemptively reordered based on the priority weight. Based on task information, priority weights, edge device status information, and server resource status information, a system state vector is constructed. Hybrid actions are generated through a pre-trained deep reinforcement learning model. Based on the hybrid actions, task scheduling and computing resource allocation are performed within the runtime cycle.

2. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 1, characterized in that: The real-time acquisition of asynchronously arriving task information, edge device status information, and server resource status information includes... When the system starts up, the cloud-edge collaborative computing network is initialized, and asynchronous task arrival is modeled using Poisson distribution. The task arrival interval Δ It follows an exponential distribution Exp(λ), where k is the task and λ is the average arrival rate; The runtime T is divided into N+2 time slots using a time-division multiple access (TDMA) mechanism, including: Time slots are used for edge devices to obtain initial energy from mobile edge computing servers via wireless power transfer; A time slot, comprising multiple sub-time slots, is used for task offloading; each edge device offloads tasks to the edge server via a wireless channel within its assigned sub-time slot, and each sub-time slot is allocated to only one edge device, generating an initial task queue. Time slot to It is used for edge servers and cloud servers to process tasks in parallel and update task status information and resource usage in real time. Time slot It is used to aggregate task execution results between edge servers and cloud servers, including task completion time. and the number of tasks completed Where N is the preset number of parallel processing time slots; Edge device status information is acquired at a fixed sampling frequency; the edge device status information includes energy status and channel status. ;in This represents the channel gain between edge device i and edge server e. This represents the channel gain between edge device i and cloud server c; the energy state includes the current energy level and energy harvesting rate of the edge device. Record the remaining computing resources of the edge server and remaining computing resources of cloud servers .

3. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 2, characterized in that: The calculation of the priority weight for each task includes, Time slot Within, based on the initial task queue, the arrival time of task k is... Remaining deadline And calculating urgency Normalization was performed to generate arrival time factors. Remaining time factor and urgency factor The formula is: ， ， ， Where T is the operating cycle, This is the threshold for task processing time. At the highest level of urgency; The priority weights are calculated by weighted summation, using the following formula: ， in, Let k be the priority weight. , , These are the weighting coefficients.

4. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 3, characterized in that: The preemptive reordering of the task queue based on priority weights includes, In the time slot to Within the current task, when the priority weight of a newly arrived task exceeds the priority weight of the currently executing task, and the difference is greater than a preset threshold ΔP, the currently executing task is interrupted and its calculation progress is saved to a memory queue. The task queue for newly arrived tasks is reordered in descending order according to priority weights to generate an optimized task queue, and the corresponding priority weights are stored.

5. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 4, characterized in that: The construction of the system state vector, and the generation of hybrid actions through a pre-trained deep reinforcement learning model, includes: In the time slot to Internally, construct the system state vector. The system state vector Including task information Channel state Remaining computing resources of edge servers and cloud servers and the remaining time of the running cycle T ;in, Let k be the computational cost of task k. The resource requirements for task k; The system state vector Input a pre-trained deep reinforcement learning model to generate hybrid actions Where t is the current time slot index; The hybrid action include, User scheduling decision Used to determine time slots The edge device i that offloads the task within sub-time slot n; where, Indicates that device i is in time slot The nth sub-slot unloading task k within the time slot, Indicates idle; the deep reinforcement learning model is based on the system state vector. Priority weights in Prioritize selecting high-efficiency tasks from the optimized task queue. Task k is assigned to device i; Unloading target decision , used to determine the processing location of task k; where, This indicates that task k is processed on the edge server. This indicates that the data is being routed to the cloud server; when the channel gain... Exceeding the predetermined threshold or remaining computing resources When sufficient, the deep reinforcement learning model is based on the system state vector. Priority weights in And edge server status, prioritize high. Task k is assigned to the edge server; Computing resource allocation decision This is used to allocate computing resources from edge servers or cloud servers to task k, and to satisfy the maximum computing resource constraint. and ; The user scheduling decision and unloading target decision are generated using an ε-greedy strategy, selecting the optimal action with a probability of 1−ε and randomly exploring with a probability of ε; the computing resource allocation decision is generated by outputting the mean of the actor network and superimposing Gaussian noise, ensuring that the allocated resources do not exceed the remaining resources of the edge server or cloud server.

6. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 5, characterized in that: The training steps of the deep reinforcement learning model include: In the interaction with the cloud-edge collaborative computing network, in time slots to Internal execution action Then, generate the state transition quadruple. And store it in the experience pool; among which, For the reward function; When the number of samples in the experience pool is not less than a preset threshold, a preset number of samples are randomly sampled from it. The temporal difference error is calculated based on the sampled data and Bellman equation, and the critic network is updated to minimize the mean square error loss. The actor network is updated using the policy gradient ascent method to optimize the selection strategy of mixed actions until the model converges or reaches the predetermined number of training steps.

7. The cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method according to claim 6, characterized in that: The deep reinforcement learning model is optimized based on a reward function, the formula of which is: ， in, As an energy consumption reward, For time slots to The total energy consumption of the inner edge server or cloud server to complete the task; Rewards for timely completion of tasks. For time slots to The actual completion time of each task within the task. The maximum delay specified; Incentives for resource utilization For time slots to The total amount of computing resources actually used internally. This represents the maximum computing resources available to all servers. C is the base reward value for completing the task. For time slots to The number of tasks completed within the specified timeframe; , , , These are the weighting coefficients.

8. A cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization system, characterized in that: It includes a data acquisition module, a reordering module, and a task scheduling and resource allocation module; The data acquisition module is used to acquire asynchronously arriving task information, edge device status information, and server resource status information in real time in the cloud-edge collaborative computing network. The reordering module is used to calculate the priority weight of each task within the running cycle according to the time characteristics and computing requirements of the task information, and to perform preemptive reordering of the task queue based on the priority weight. The task scheduling and resource allocation module is used to construct a system state vector based on task information, priority weights, edge device status information, and server resource status information. It generates hybrid actions through a pre-trained deep reinforcement learning model and performs task scheduling and allocation of computing resources based on the hybrid actions during the runtime cycle.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores a computer program that can run on the processor. When the program is executed on the processor, it implements the steps in the cloud-edge collaborative asynchronous task scheduling and computing resource joint optimization method as described in any one of claims 1-7.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps in the asynchronous task scheduling and computing resource joint optimization method for cloud-edge collaboration as described in any one of claims 1-7.

Images