Edge-Cloud Collaborative Configuration and Scheduling Methods for Video Analytics Tasks

Abstract

This invention discloses an edge-cloud collaborative configuration and scheduling method for video analysis tasks, belonging to the configuration adjustment and resource scheduling technology of the Internet application layer. First, an edge-cloud collaborative scheduler based on a reinforcement learning network is constructed. An actor-critic model is introduced into the edge-cloud collaborative architecture. The actor model is deployed on edge nodes as the scheduler, while the critic model is deployed in the analyzer module on any node in the cloud. The analyzer evaluation results are fed back to the actor models on each edge node for training. Then, an optimization objective is established and solved to obtain discrete decision and bandwidth decision, which are applied to the scheduler's video analysis task. After the current time slot step is completed, the remaining tasks are uploaded to the cloud node, and the task processing status is used to train the actor-critic system. Training is iterated until the actor-critic system converges. This invention achieves efficient resource utilization, improves system scalability and robustness, and reduces energy consumption.

Description

Technical Field

[0001] This invention pertains to configuration adjustment and resource scheduling technology at the Internet application layer, specifically a collaborative configuration and scheduling method for edge-cloud applications oriented towards video analytics tasks. Background Technology

[0002] With the development of the Internet of Things (IoT), a large number of surveillance cameras have been deployed in various fields, including security monitoring, crime prevention, and smart commerce. It is reported that by 2023, the number of surveillance cameras had reached 25 million, and the global surveillance camera market size is projected to reach $45.93 billion by 2027. These cameras generate a large amount of video streams, which require timely analysis. Requests to analyze video streams for specific purposes are commonly referred to as video analytics tasks. These tasks typically use deep neural networks to process video streams, demanding significant computing resources. However, many terminal devices lack sufficient computing resources to run deep neural networks in real time. Therefore, offloading video analytics tasks to edge computing or cloud computing servers is a common solution, making video analytics a "killer application" of cloud computing.

[0003] Existing cloud computing resources are trending towards Sky Computing, which abstracts the heterogeneity of cloud resources and provides a universal interface for accessing computing resources to ensure applications can run across clouds, thereby effectively improving application flexibility, scalability, and disaster recovery capabilities. To achieve this goal, effective scheduling of resources within the network is crucial. This involves not only addressing the heterogeneity of computing resources but also the diverse network environments of servers. Task performance preferences also vary depending on the task scenario and the preferences of the user issuing the task. For example, some tasks prioritize high accuracy, while others may prioritize lower processing latency rather than accuracy. Meeting these different task preferences is essential for improving Quality of Experience (QoE).

[0004] Furthermore, in video analytics tasks, video streams can be encoded and uploaded using various parameters (such as resolution and frame rate), and tasks can be divided into multiple steps and scheduled to run on different computing nodes. Changing these configurations and scheduling strategies can lead to significant differences in performance and affect the demand for computing and network resources. Therefore, it is necessary to jointly optimize video stream configurations and task scheduling strategies to meet the performance goals of each task and ensure that the usage of each edge or cloud computing resource does not exceed its capacity.

[0005] Existing cloud computing solutions for video analytics dynamically adjust video configuration parameters by analyzing the performance requirements of different tasks (including key indicators such as accuracy, latency, and cost), effectively reducing the amount of data that needs to be transmitted and processed, thus avoiding resource waste. Meanwhile, some research focuses on scheduling computing devices distributed across different geographical locations to meet the huge computing resource demands of video analytics tasks, achieving efficient resource utilization. However, the former only considers the simple case of a single edge or cloud node, which is not suitable for large-scale task deployment; the latter does not consider adjusting video configuration according to task requirements, which may lead to suboptimal task results and a certain degree of resource waste. Summary of the Invention

[0006] This invention proposes an edge-cloud collaborative configuration and scheduling method for video analytics tasks. It optimizes configuration and scheduling while ensuring that application performance meets task requirements, so as to achieve the goal of fully utilizing device resources and improving QoE.

[0007] The edge-cloud collaborative configuration and scheduling method for video analytics tasks is implemented according to the following steps:

[0008] Step 1: Build a collaborative video analytics architecture involving the edge, device, and cloud, including end devices, edge devices, and cloud devices;

[0009] The edge device is responsible for generating a video stream according to the configuration, and simultaneously generating a video analysis task. It then submits the video stream and video analysis task to the edge device. The edge device decides the configuration of the current video stream, schedules the task through the scheduler, processes part of the task based on the scheduling result, and submits the remaining task content to the cloud device. The cloud device's video analysis model generates analysis results and transmits the analysis results to the analyzer to evaluate the task execution. The analysis results are then fed back to the edge device's scheduler, which optimizes the task scheduling decision.

[0010] Step 2: Introduce an actor-critic system into the edge-cloud collaborative video analysis architecture to obtain an edge-cloud collaborative scheduler based on reinforcement learning network.

[0011] The actor-critic system is as follows: the actor model is deployed on edge nodes and acts as a scheduler, determining the configuration and scheduling strategy of the video stream in the next time slot at each time slot; the critic model is deployed on any node in the cloud during the training phase and acts as an analyzer to evaluate the task execution of each node at each time point, and feeds back the evaluation results to the actor models on each edge node for training.

[0012] The actor-critic neural network architecture uses an LSTM module to capture the time information of historical decisions as a feature vector, and concatenates other dimension vectors in the observation space, the number of tasks submitted to the edge nodes, and the preferences of each task into the feature vector.

[0013] Step 3: Establish an optimization objective for the edge-cloud collaborative scheduler based on reinforcement learning network, aiming to maximize accuracy a while minimizing latency l and processing overhead o for each task i.

[0014] Suppose there are M edge nodes, and each edge node manages n i Given N tasks in a system with a total runtime of T, in each time slot t, we need to determine the resolution r, frame rate f, number of steps s on the edge node, cloud node c for processing the task, and allocated bandwidth b for each task's video stream. The optimization objective is then expressed as:

[0015]

[0016]

[0017] Where, the subscript i,t indicates that the parameter belongs to task i in the t-th time slot, the superscript E indicates the edge node, and the superscript C indicates the cloud node; Indicates end-to-side bandwidth. Indicates edge-cloud bandwidth;

[0018] Constraints C1 to C4 confirm that the resolution, frame rate, number of steps running on edge nodes, and cloud nodes used to process the task are within an optional range. Constraints C5 to C... 10 This ensures that the device resources used by each node are within limits.

[0019] The GPU memory h and memory m used by task i on the edge node and cloud node are related to its resolution r, frame rate f, and number of steps s on the edge node, and are expressed as:

[0020]

[0021] Step four: Solve the optimization problem by decomposing it into decisions about discrete quantities and decisions about bandwidth b.

[0022] (1) Decision-making for discrete quantities

[0023] Discrete decision-making maximizes the utility of each task by determining the video stream configuration and task scheduling for each task, as expressed by the formula:

[0024]

[0025] stC1,C2,C3,C4,C7,C8,C9,C 10

[0026] The discrete decision problem can be modeled as a partially observable Markov Decision Process (POMDP) ​​as follows:

[0027] Bandwidth, GPU memory, RAM, and GPU utilization are represented as B, H, M, and G, respectively.

[0028] State Space: The optimal scheduling decision for each edge node is determined by the resource capacity of each compute node, the preference p of each task, and the scheduling strategies σ of other edge nodes. Therefore, the state space is defined as S' = {S n}, where S n ={(B E H E M E G E B C H C M C G C ,p,σ)}.

[0029] Observation Space: Since the scheduling strategies of other edge nodes are unobservable, we can only infer their scheduling strategy by observing the scheduling decision x made by the edge node at the previous time step and the corresponding reward r'. Therefore, the observation space is defined as O = {O n}, where O n ={(B E H E M E G E B C H C M C G C ,p,x t-1 ,r' t-1 )}.

[0030] Observe the function: Y is a function S×O→[0,1]. The function Y=P(x k,t ,R′ k,t |σ k ) represents the condition σ in task k. k Make a decision x k,t And receive reward R′ k,t The probability. After all edge nodes make scheduling decisions at time t, the observation will transform into one that satisfies...

[0031] Decision space: The decision space is the configuration of the video stream and the scheduling of tasks, i.e., A = {x} n}={(r n ,f n ,s n ,c n )}.

[0032] Reward Space: If the task in time slot t is successfully completed, the reward is the same as in the problem statement. If certain scheduling decisions fail due to resource allocation exceeding capacity on a cloud node, i.e., C7, C8, C9, and C are not satisfied... 10 For any condition in the reward space, a negative constant -C is assigned to the reward. Therefore, the reward space is represented as R′={p i,1 a i -p i,2 l i -p i,3 o i}∪{-C}.

[0033] Applying the above modeling process to the actor-critic system in step two, allowing the actor model to make decisions in each time slot, is the solution method for discrete quantities.

[0034] (2) Decision on bandwidth b

[0035] The decision regarding the continuous quantity, i.e., bandwidth b, is to minimize latency by determining the bandwidth allocation for each task, as expressed by the following formula:

[0036]

[0037] stC5,C6

[0038] The optimal bandwidth allocation for each task at the corresponding edge node is obtained using Cauchy's inequality. ω(s) represents the ratio of the data size after s steps of processing the video stream at the edge node to the original data size. The resulting bandwidth allocation for task i at edge node e is... i The optimal bandwidth allocation is as follows:

[0039]

[0040] Step 5: Based on the discrete decision and bandwidth decision obtained from solving the optimization problem, calculate the delay and overhead of task i in time slot t;

[0041] (1) The time delay of task i in time slot t is expressed as:

[0042]

[0043] Among them, the transmission delay between the end and the edge For coding coefficients; edge-to-cloud transmission delay These are the coding coefficients; ρ(s) represents the computational delay. i,t This indicates the proportion of data processed on edge devices.

[0044] (2) The cost of task i in time slot t is expressed as:

[0045]

[0046] in, This represents the unit price for transmission from edge node i to cloud node j. This represents the rental unit price of the edge node. This indicates the rental price per cloud node.

[0047] Step 6: The scheduler executes the corresponding video analysis tasks according to the discrete decision quantities. After the current time slot step is completed, the remaining tasks are uploaded to the cloud node.

[0048] Step 7: The cloud node processes the remaining tasks and uses the task processing results to train the actor-critic system;

[0049] The training process for the actor-critic system is as follows:

[0050] The critic model takes the average of all task feature vectors at each node as input to the fully connected layer to generate estimates needed to calculate the evaluation value, including modeling factors for cost and latency, task accuracy, and task success rate. The modeling factors are used to calculate the corresponding latency and cost, and the estimated reward is calculated by combining the accuracy. The completion rate is then multiplied by the estimated reward to obtain the corresponding evaluation value.

[0051] The actor model uses fully connected layers to take the feature vectors of each task as input and output a probability map for each dimension of the decision space.

[0052] The evaluation values ​​obtained from the critic model are used to optimize the actor model and update its parameters. Meanwhile, the execution results of the task after the actor model takes action, as well as the actual latency and cost obtained in step five, will be used to optimize the critic model so that the evaluation values ​​it generates are closer to the actual benefits.

[0053] Step 8: During each iteration of the actor-critic system training process, the scheduler reconfigures the task generator, returns to step 3, and repeats the above steps until the actor-critic system converges.

[0054] The advantages and beneficial effects of this invention are as follows:

[0055] (1) Based on the actor-critic deep reinforcement learning architecture, this invention proposes a distributed task configuration and scheduling optimization algorithm, which realizes efficient resource utilization in a multi-device environment and improves overall performance while ensuring the scalability of the entire system.

[0056] (2) The method of the present invention improves the scalability and robustness of the system, significantly reduces energy consumption, and ensures that the performance of the video analysis task can meet the needs of the task. Attached Figure Description

[0057] Figure 1 It is a video analytics architecture model that integrates edge, cloud, and endpoint technologies.

[0058] Figure 2 This is a flowchart of video analytics workflow involving edge-cloud collaboration;

[0059] Figure 3 This is a schematic diagram of the overall architecture of the actor-critic system;

[0060] Figure 4 This is a schematic diagram of the structure of a reinforcement learning network. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0062] like Figure 1 As shown, the edge-cloud collaborative architecture for video analytics tasks analyzes different types of surveillance videos, mainly object detection, object tracking, and traffic analysis. The proposed system has a three-layer structure consisting of a client, edge devices, and cloud devices. Video analytics tasks primarily rely on deep neural network models that require significant computational and storage resources, making them difficult to deploy independently to resource-constrained edge servers. Furthermore, due to the massive data volume of video streams, transmitting all videos to the cloud platform for analysis incurs unnecessary transmission and computational costs. Therefore, this invention segments the models used in video analytics tasks, allowing edge devices to perform partial data processing before offloading subsequent steps to cloud devices, thus enabling flexible resource allocation across the entire system.

[0063] Cloud nodes possess most of the network's computing resources. These nodes are located in different regions and have varying rental fees, resulting in different network environments and usage costs. This creates trade-offs for tasks with different preferences. For example, some tasks are more sensitive to system response latency, so the system prefers to submit these tasks to physically closer cloud nodes for lower transmission latency; while other tasks prioritize cost savings, making it a better choice to submit them to physically farther but lower-cost nodes.

[0064] Making decisions about the cloud nodes called by the system from the perspective of task preferences can effectively improve system efficiency and reduce costs. However, in actual system operation, a more important issue is to make reasonable task allocation based on the resource capacity and utilization of each device to avoid problems such as some nodes exceeding the device resource capacity when there are too many tasks, and some devices being idle, resulting in resource waste.

[0065] The edge-cloud collaborative configuration and scheduling method for video analytics tasks proposed in this invention comprises the following steps:

[0066] Step 1: Build a collaborative video analytics architecture involving the edge, device, and cloud, including end devices, edge devices, and cloud devices;

[0067] like Figure 2 As shown, the client device generates a video stream through its camera and submits the video analysis task to the nearest edge node. A scheduler is deployed at the edge node, which adjusts the video stream configuration on the edge side in real time based on resource availability and task requirements, determining the task steps to be executed on the edge device and submitting subsequent tasks to the cloud node. After completing its own task, the edge node uploads the remaining data to the cloud node, where the cloud node's video analysis model generates analysis results. The analyzer evaluates the execution of the task and finally feeds the results back to the edge device's scheduler, which then optimizes its decisions based on these results.

[0068] Step 2: Introduce an actor-critic system into the edge-cloud collaborative video analysis architecture to obtain an edge-cloud collaborative scheduler based on reinforcement learning network.

[0069] The overall architecture of the actor-critic system is as follows: Figure 3 As shown, specifically: the actor model is deployed on edge nodes and acts as a scheduler, determining the configuration and scheduling strategy of the video stream in the next time slot at each time slot; the critic model is deployed in the analyzer module on any node in the cloud during the training phase, acting as an analyzer to evaluate the task execution of each node at each time point, and feeding the evaluation results back to the actor model on each edge node for training.

[0070] The replay buffer within the analyzer records resource utilization, task execution results, and actions taken by the actor model. This information is used to optimize the critic model and generate corresponding evaluation values ​​for each time slot, which are then used to train the actor model. This invention uses a centralized critic and analyzer module because the centralized critic module can capture information from all edge nodes in the network and makes the training process easier to converge. The centralized analyzer and critic modules are only used during the training phase; therefore, the system can operate entirely in a distributed manner during the inference phase, which does not conflict with the goal of designing a distributed scheduling system in this invention.

[0071] The actor-critic neural network architecture uses an LSTM module to capture the time information of historical decisions as a feature vector, and concatenates other dimension vectors in the observation space, the number of tasks submitted to the edge nodes, and the preferences of each task into the feature vector.

[0072] Specific reinforcement learning network structures such as Figure 4 As shown, since the observation space includes the historical policies and rewards of all schedulers, utilizing the temporal information in the history is crucial. Therefore, this invention uses an LSTM module to capture temporal information as a feature vector and embeds other dimensions in the observation space (the number of tasks submitted to the scheduler and the preference for each task) into the feature vector. For the actor model, a fully connected layer is used to take the feature vector as input and output a probability map for each dimension in the decision space. The critic model utilizes similar embeddings and an LSTM module to process task information and generate feature vectors. The average of these vectors is used as input to a fully connected layer to generate estimates needed to calculate the evaluation value, including accuracy, success rate, and other modeling factors. The corresponding latency and cost are calculated using modeling formulas for latency and overhead. Accuracy is used to calculate the estimated reward according to the method defined in the reward space. Finally, the success rate is multiplied by the estimated reward to obtain the corresponding evaluation value.

[0073] Step 3: Establish an optimization objective for the edge-cloud collaborative scheduler based on reinforcement learning network, aiming to maximize accuracy a while minimizing latency l and processing overhead o for each task i.

[0074] Suppose there are M edge nodes, and each edge node manages n i There are N tasks in the system, and the total system runtime is T. In each time slot t, the resolution r, frame rate f, number of steps s on the edge node, cloud node c for processing the task, and allocated bandwidth b of the video stream for each task need to be determined. The objective of this invention is to maximize accuracy a while minimizing latency l and processing overhead o. Since each task has different preferences for these three factors, the benefit of each task can be expressed as a weighted linear combination of these three factors. Therefore, the optimization objective is expressed as:

[0075]

[0076]

[0077] Where, the subscript i,t indicates that the parameter belongs to task i in the t-th time slot, the superscript E indicates the edge node, and the superscript C indicates the cloud node; Indicates end-to-side bandwidth. Indicates edge-cloud bandwidth;

[0078] Constraints C1 to C4 confirm that the resolution, frame rate, number of steps running on edge nodes, and cloud nodes used to process the task are within an optional range. Constraints C5 to C... 10 This ensures that the device resources used by each node are within limits.

[0079] The video analysis task of this invention is executed on a GPU; therefore, the main resources considered in this invention include GPU utilization g, GPU memory h, RAM m, and network bandwidth b. Among these resources, the usage of GPU memory h and RAM m is determined by video configuration and scheduling decisions; if resources are insufficient, the task will fail. The usage of these resources can be determined during the training phase. The GPU memory h and RAM m occupied by task i on edge nodes and cloud nodes are related to its resolution r, frame rate f, and the number of steps s executed on the edge nodes, expressed as:

[0080]

[0081] Regarding GPU core utilization, since GPU vendors do not provide APIs for fine-grained allocation of GPU cores, GPU cores are typically allocated equally to each task running on that GPU. Therefore, GPU utilization can be modeled using the following formula, where c and e represent the steps required for the task on the edge node and cloud node, G represents the GPU capacity of the node, and ∑ j [e j =e i ] and ∑ j [c j,t =c i,t The number of tasks assigned to that node is indicated by the symbol ].

[0082]

[0083]

[0084] Step four: Solve the optimization problem by decomposing it into decisions about discrete quantities and decisions about bandwidth b.

[0085] (1) Decision-making for discrete quantities

[0086] Discrete decision-making maximizes the utility of each task by determining the video stream configuration and task scheduling for each task, as expressed by the formula:

[0087]

[0088] stC1,C2,C3,C4,C7,C8,C9,C 10

[0089] Decision-making for discrete quantities is NP-hard because it is an integer programming problem with a nonlinear objective. Furthermore, each decision-maker can only collect partial information due to environmental limitations; moreover, variable available resources and task preferences further complicate the situation. Therefore, the discrete quantity decision-making problem is modeled as a partially observable Markov Decision Process (POMDP). POMDP is a better modeling scheme because it allows for decision-making under uncertainty and incomplete information. To this end, this invention proposes a reinforcement learning-based algorithm to solve this problem. The POMDP process is described as follows:

[0090] Bandwidth, GPU memory, RAM, and GPU utilization are represented as B, H, M, and G, respectively.

[0091] State Space: The optimal scheduling decision for each edge node is determined by the resource capacity of each compute node, the preference p of each task, and the scheduling strategies σ of other edge nodes. Therefore, the state space is defined as S' = {S n}, where S n ={(B E H E M E G E B C H C M C G C ,p,σ)}.

[0092] Observation Space: Since the scheduling strategies of other edge nodes are unobservable, we can only infer their scheduling strategy by observing the scheduling decision x made by the edge node at the previous time step and the corresponding reward r'. Therefore, the observation space is defined as O = {O n}, where O n ={(B E H E M E G E B C H C M C G C ,p,x t-1 ,r' t-1 )}.

[0093] Observe the function: Y is a function S×O→[0,1]. The function Y=P(x k,t ,R′ k,t |σ k ) represents the condition σ in task k. k Make a decision x k,tAnd receive reward R′ k,t The probability. After all edge nodes make scheduling decisions at time t, the observation will transform into one that satisfies...

[0094] Decision space: The decision space is the configuration of the video stream and the scheduling of tasks, i.e., A = {x} n}={(r n ,f n ,s n ,c n )}.

[0095] Reward Space: If the task in time slot t is successfully completed, the reward is the same as in the problem statement. If certain scheduling decisions fail due to resource allocation exceeding capacity on a cloud node, i.e., C7, C8, C9, and C are not satisfied... 10 For any condition in the reward space, a negative constant -C is assigned to the reward. Therefore, the reward space is represented as R′={p i,1 a i -p i,2 l i -p i,3 o i}∪{-C}.

[0096] Applying the above modeling process to the actor-critic system in step two, allowing the actor model to make decisions in each time slot, is the solution method for discrete quantities.

[0097] (2) Decision on bandwidth b

[0098] The decision regarding the continuous quantity, i.e., bandwidth b, is to minimize latency by determining the bandwidth allocation for each task, as expressed by the following formula:

[0099]

[0100] stC5,C6

[0101] The optimal bandwidth allocation for each task at the corresponding edge node is obtained using Cauchy's inequality. ω(s) represents the ratio of the data size after s steps of processing the video stream at the edge node to the original data size. The resulting bandwidth allocation for task i at edge node e is... i The optimal bandwidth allocation is as follows:

[0102]

[0103] Step 5: Based on the discrete decision and bandwidth decision obtained from solving the optimization problem, calculate the delay and overhead of task i in time slot t;

[0104] (1) The time delay of task i in time slot t is expressed as:

[0105]

[0106] Bandwidth allocation affects transmission latency, which in turn affects overall task performance. Therefore, this invention models transmission latency as a function determined by bandwidth and data volume, and further depends on the video stream resolution r and frame rate f. Based on previous research, r can be used... 2 f models the data size. The amount of data that needs to be transmitted from the end to the edge also depends on the video coding algorithm used on the end side, for which a coding coefficient C is required. enc This represents the ratio of the amount of data after encoding to the amount of data before encoding. Therefore, the end-to-end transmission delay can be expressed by the following formula:

[0107]

[0108] The amount of data that needs to be transmitted between the edge and the cloud depends on the size of the feature map obtained after the edge device performs s-step processing on the original data. A coefficient can also be used to determine this. This represents the ratio of the amount of data before and after processing. Therefore, the edge-cloud transmission latency can be expressed by the following formula:

[0109]

[0110] The latency of each video segment includes transmission latency, propagation latency, and computation latency. (Using...) To represent the propagation delay between edge nodes and cloud nodes; ρ(s) i,t This represents the proportion of data processed on edge devices, which can then be used to... This is used to represent computational delay.

[0111] (2) The cost of each video segment consists of transmission cost and computation cost. Both types of cost are linearly related to the amount of data. The cost of task i in time slot t is expressed as:

[0112]

[0113] in, This represents the unit price for transmission from edge node i to cloud node j. This represents the rental unit price of the edge node. This indicates the rental price per cloud node.

[0114] Step 6: The scheduler executes the corresponding video analysis tasks according to the discrete decision quantities. After the current time slot step is completed, the remaining tasks are uploaded to the cloud node.

[0115] Step 7: The cloud node processes the remaining tasks and uses the task processing results to train the actor-critic system;

[0116] The training process for the actor-critic system is as follows:

[0117] The critic model takes the average of all task feature vectors at each node as input to the fully connected layer to generate estimates needed to calculate the evaluation value, including modeling factors for cost and latency, task accuracy, and task success rate. The modeling factors are used to calculate the corresponding latency and cost, and the estimated reward is calculated by combining the accuracy. The completion rate is then multiplied by the estimated reward to obtain the corresponding evaluation value.

[0118] The actor model uses fully connected layers to take the feature vectors of each task as input and output a probability map for each dimension of the decision space.

[0119] The evaluation values ​​obtained from the critic model are used to optimize the actor model and update its parameters. Meanwhile, the execution results of the task after the actor model takes action, as well as the actual latency and cost obtained in step five, will be used to optimize the critic model so that the evaluation values ​​it generates are closer to the actual benefits.

[0120] Step 8: During each iteration of the actor-critic system training process, the scheduler reconfigures the task generator, returns to step 3, and repeats the above steps until the actor-critic system converges.

Claims

1. A method for edge-cloud collaborative configuration and scheduling for video analytics tasks, characterized in that, Includes the following steps: Step 1: Construct a video analytics architecture that integrates edge devices and cloud devices, and introduce an actor-critic system to obtain an edge-cloud collaborative scheduler based on reinforcement learning networks. The actor-critic system is as follows: the actor model is deployed on edge nodes and acts as a scheduler, determining the configuration and scheduling strategy of the video stream in the next time slot at each time slot; the critic model is deployed on any node in the cloud during the training phase and acts as an analyzer to evaluate the task execution of each node at each time point, and feeds back the evaluation results to the actor models on each edge node for training. The actor-critic neural network structure uses an LSTM module to capture the time information of historical decisions as a feature vector, and concatenates other dimension vectors in the observation space, the number of tasks submitted to the edge nodes, and the preferences of each task into the feature vector; Step 2: Establish an optimization objective for the edge-cloud collaborative scheduler based on reinforcement learning network, aiming to maximize accuracy a while minimizing latency l and processing overhead o for each task i. Suppose there are M edge nodes, and each edge node manages n i Given N tasks in a system with a total runtime of T, in each time slot t, we need to determine the resolution r, frame rate f, number of steps s on the edge node, cloud node c for processing the task, and allocated bandwidth b for each task's video stream. The optimization objective is then expressed as: Where, the subscript i,t indicates that the parameter belongs to task i in the t-th time slot, the superscript E indicates the edge node, and the superscript C indicates the cloud node; Indicates end-to-side bandwidth. Indicates edge-cloud bandwidth; Constraints C1 to C4 confirm that the resolution, frame rate, number of steps running on edge nodes, and cloud nodes used to process the task are within an optional range. Constraints C5 to C... 10 This ensures that the device resources used by each node are within limits; The GPU memory h and memory m used by task i on the edge node and cloud node are related to its resolution r, frame rate f, and number of steps s on the edge node, and are expressed as: Step 3: Solve the optimization problem by decomposing it into decisions about discrete quantities and decisions about bandwidth. (1) Decision-making for discrete quantities Discrete decision-making maximizes the utility of each task by determining the video stream configuration and task scheduling for each task, as expressed by the formula: s.t.C1,C2,C3,C4,C7,C8,C9,C 10 Modeling the discrete decision problem as a partially observable Markov decision process (POMDP) ​​and applying the modeling process to the actor-critic system, where the actor model makes a decision in each time slot, is the solution method for discrete quantities. (2) Bandwidth decision The decision regarding the continuous quantity, i.e., bandwidth b, is to minimize latency by determining the bandwidth allocation for each task, as expressed by the following formula: stC5,C6 The optimal bandwidth allocation for each task on the corresponding edge node is obtained by using Cauchy's inequality. Step four: Based on the discrete decision and bandwidth decision obtained from solving the optimization problem, calculate the delay and overhead of task i in time slot t; (1) The time delay of task i in time slot t is expressed as: Among them, the end-to-edge transmission delay C enc For coding coefficients; edge-to-cloud transmission delay These are the coding coefficients; ρ(s) represents the computational delay. i,t This indicates the proportion of data processed on edge devices; These represent the GPU utilization of the task at the edge node and the cloud node, respectively. Used to represent the propagation delay between edge nodes and cloud nodes; (2) The cost of task i in time slot t is expressed as: in, This represents the unit price for transmission from edge node i to cloud node j. This represents the rental unit price of the edge node. This indicates the rental price per cloud node; Step 5: The scheduler executes the corresponding video analysis tasks according to the discrete decision quantities. After the current time slot step is completed, the remaining tasks are uploaded to the cloud node. Step six: The cloud node processes the remaining tasks and uses the task processing results to train the actor-critic system; The training process for the actor-critic system is as follows: The critic model takes the average of all task feature vectors at each node as input to the fully connected layer to generate the estimated value needed to calculate the evaluation value, including the modeling factors of cost and latency, task accuracy, and task success rate; calculates the corresponding latency and cost through the modeling factors, calculates the estimated reward by combining the accuracy, and multiplies the completion rate by the estimated reward to obtain the corresponding evaluation value; The actor model uses fully connected layers to take the feature vectors of each task as input and output a probability map of each dimension in the decision space. The evaluation values ​​obtained by the critic model are used to optimize the actor model and update the parameters of the actor model. At the same time, the execution results of the task after the actor model takes action, as well as the actual latency and cost, will be used to optimize the critic model so that the evaluation values ​​it generates can be closer to the actual benefits. Step 7: During each iteration of the actor-critic system training process, the scheduler reconfigures the task generator, returns to step 2, and repeats the above steps until the actor-critic system converges.

2. The edge-cloud collaborative configuration and scheduling method for video analytics tasks according to claim 1, characterized in that, The edge-cloud collaborative video analytics architecture is as follows: the edge device is responsible for generating a video stream according to the configuration, and simultaneously generating a video analytics task, and submitting the video stream and video analytics task to the edge device; the edge device decides the configuration of the current video stream, schedules the task through the scheduler, processes part of the task content according to the scheduling result, and submits the remaining task content to the cloud device; the cloud device's video analytics model generates analysis results, transmits the analysis results to the analyzer to evaluate the task execution, and feeds back the evaluation results to the scheduler of the edge device, and the scheduler optimizes the task scheduling decision.

3. The edge-cloud collaborative configuration and scheduling method for video analytics tasks according to claim 1, characterized in that, The modeling process of Partially Observable Markov Decision Processes (POMDPs) is as follows: Bandwidth, GPU memory, RAM, and GPU utilization are represented as B, H, M, and G, respectively. State space: The optimal scheduling decision for each edge node is determined by the resource capacity of each compute node, the preference p of each task, and the scheduling strategies σ of other edge nodes; therefore, the state space is defined as S' = {S n }, where S n ={(B E H E M E G E B C H C M C G C ,p,σ)}; Observation space: Since the scheduling strategies of other edge nodes are unobservable, we can only infer their scheduling strategies by observing the scheduling decision x and the corresponding reward r' made by the edge node in the previous time step. Therefore, the observation space is defined as O = {O n }, where O n ={(B E H E M E G E B C H C M C G C ,p,x t-1 ,r' t-1 )}; Observe the functions: Y is a function S×O→[0,1]; the function Y=P(x k,t ,R′ k,t |σ k ) represents the condition σ in task k. k Make a decision x k,t And receive reward R′ k,t The probability; after all edge nodes make scheduling decisions at time t, the observation will transform into one that satisfies Decision space: The decision space is the configuration of the video stream and the scheduling of tasks, i.e., A = {x} n }={(r n ,f n ,s n ,c n )}; Reward space: If the task in time slot t is successfully completed, the reward is the same as in the problem statement; if some scheduling decisions fail due to resource allocation exceeding capacity of a cloud node, i.e., C7, C8, C9, and C are not satisfied. 10 For any condition in the reward space, a negative constant -C is assigned; therefore, the reward space is represented as R′={p i,1 a i -p i,2 l i -p i,3 o i }∪{-C}.

4. The edge-cloud collaborative configuration and scheduling method for video analytics tasks according to claim 1, characterized in that, The process of solving for the optimal bandwidth allocation using Cauchy's inequality is as follows: Let ω(s) represent the ratio of the data size of the video stream after s steps of processing at the edge node to the original data size. The resulting value for task i at edge node e is... i The optimal bandwidth allocation is as follows:

Images