Edge cloud cooperation-based inspection robot visual analysis task dynamic scheduling method and system

Abstract

The application discloses a kind of based on edge cloud cooperation's inspection robot visual analysis task dynamic scheduling method and system, belong to inspection robot visual analysis and edge computing technical field.The method includes: real-time acquisition network state parameter and computing power resource information;Task feature vector is constructed to the visual analysis task and is extracted feature;Based on multi-objective optimization scheduling model, delay, energy consumption and accuracy index are determined optimal execution position by comprehensive evaluation;Task is distributed to corresponding node execution;Based on Q-learning reinforcement learning, scheduling weight is dynamically updated.The application can realize the optimal scheduling of visual analysis task under the dynamic change of network and computing power environment, compared with the scheduling efficiency of existing static allocation strategy is improved by more than 40%, and the comprehensive performance is significantly better than local execution, full cloud execution and static allocation scheme.

Description

Technical Field

[0001] This invention relates to the field of visual analysis and edge computing technology for inspection robots, specifically to a method and system for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration. Background Technology

[0002] With the continuous improvement of industrial automation and intelligence, inspection robots have been widely used in fields such as power line inspection, pipeline inspection, and factory security. These robots are equipped with various vision sensors and require real-time analysis and processing of the collected images and videos to identify defects, anomalies, and safety hazards.

[0003] Existing visual analytics task scheduling schemes mainly include the following categories: (1) Local execution scheme: All visual analysis tasks are executed on the robot's local computing unit. Due to the limited computing power of the robot's hardware, it is difficult to run complex deep learning models, resulting in limited analysis accuracy. For example, Chinese patent CN114359246A discloses a conveyor belt detection method, which adopts a fixed allocation strategy to allocate the detection task to the local processor for execution. It cannot be dynamically adjusted according to changes in network and computing power.

[0004] (2) Full cloud processing solution: Upload all visual data to the cloud for analysis. Although it can utilize the powerful computing resources of the cloud to run complex models, it is heavily dependent on network bandwidth and has a slow response in high latency or weak network environments, making it unsuitable for inspection scenarios with high real-time requirements.

[0005] (3) Static edge-cloud collaboration scheme: Chinese patent CN116596218B discloses a cable tunnel robot inspection system and task scheduling method based on edge-end collaboration, which uses preset rules to allocate tasks to the edge or cloud. However, this scheme adopts a static allocation strategy, which cannot adaptively adjust according to real-time network status and computing power changes, and the scheduling efficiency decreases when the network fluctuates or the computing power load changes.

[0006] (4) Regarding adaptive edge cloud routing, existing solutions such as the large vision model-assisted adaptive edge cloud routing proposed by the LAECIPS system (ScienceDirect) have certain adaptive capabilities, but lack dedicated optimization for inspection robot scenarios and do not introduce a multi-objective optimization scheduling model.

[0007] In summary, the existing technologies have the following shortcomings: ① The scheduling strategy is static and fixed, and cannot adapt to the dynamically changing network and computing power environment; ② There is a lack of a dedicated scheduling model for inspection visual analysis tasks; ③ It is impossible to perform joint optimization among multiple objectives such as latency, energy consumption, and accuracy. Summary of the Invention

[0008] The technical problem to be solved by this invention is: how to dynamically schedule the visual analysis tasks of inspection robots between the edge and the cloud, so as to achieve joint optimization of multiple objectives such as latency, energy consumption and analysis accuracy in an environment where network bandwidth, latency and computing power resources change dynamically.

[0009] To address the aforementioned technical issues, this invention provides a dynamic scheduling method for visual analysis tasks of inspection robots based on edge cloud collaboration.

[0010] Real-time collection of network status and computing resource information during the operation of the inspection robot. The network status includes at least the current available bandwidth, communication latency and network packet loss rate, while the computing resource information covers the current available computing power of the edge and the cloud respectively. For the visual analysis task to be processed, its multi-dimensional features are extracted. The features include the type of task, the required computational complexity, the urgency of the task, and the amount of data to be transmitted. The extracted task features and the current network and computing power status are input into a multi-objective optimization scheduling model. The model comprehensively evaluates the latency, energy consumption and inference accuracy of the task when it is executed at the edge or in the cloud. It also gives a comprehensive score to each candidate execution position by weighted summation and selects the position with the best comprehensive score as the final scheduling decision for the current task. According to the scheduling decision, the task is distributed to the corresponding edge node or cloud server to perform visual reasoning analysis; Meanwhile, a reinforcement learning mechanism is adopted to dynamically adjust the weight coefficients of the three indicators of latency, energy consumption and accuracy in the multi-objective optimization model based on the actual comprehensive performance generated after each task scheduling, so that subsequent scheduling decisions can adapt to the constantly changing network environment and computing load.

[0011] As one preferred embodiment, the monitoring agents deployed on edge nodes and in the cloud collect the network status parameters and computing resource information respectively, and the time interval for each collection is set between 100 milliseconds and 500 milliseconds.

[0012] As one preferred option, the task types specifically include four categories: object detection, image classification, semantic segmentation, and anomaly detection. The computational complexity of the task is measured by the estimated computational cost of the pre-trained model called by the task.

[0013] As one preferred embodiment, when evaluating latency performance, the calculation is based on the current network communication latency and the amount of data to be transmitted; when evaluating energy consumption, the estimation is based on the power consumption characteristics of the execution node and its expected running time; and when evaluating inference accuracy, the inherent accuracy of the model used and the quality of the current input data are considered comprehensively.

[0014] As one preferred embodiment, when the scheduling model is run for the first time, the weights of the three indicators, latency, energy consumption, and accuracy, are initially set to be equal. After each subsequent scheduling, the evaluation values ​​generated by reinforcement learning are converted into a weight distribution so that the sum of the three weights remains constant.

[0015] As one preferred embodiment, the reinforcement learning mechanism adopts the standard temporal differential learning method, wherein the learning step size is set to 0.1, the discount rate of future rewards is set to 0.9, and a strategy combining random exploration and experience utilization is adopted in the weight adjustment process to discover a better weight configuration while maintaining scheduling stability.

[0016] As one preferred embodiment, when a task is distributed to an edge node for execution, if the edge node fails to return a processing result within a preset timeout period, the system automatically transfers the task to the cloud for re-execution to ensure that the task can be completed within a reasonable time.

[0017] Another embodiment of the present invention provides a dynamic scheduling device for visual analysis tasks of inspection robots based on edge cloud collaboration, comprising: The status acquisition module is used to acquire network status parameters and available computing power information at the edge and cloud in real time. The task feature extraction module is used to extract multi-dimensional features for visual analysis tasks and generate corresponding task feature descriptions. The multi-objective optimization scheduling module is used to comprehensively evaluate the latency, energy consumption, and accuracy performance of each execution location based on task characteristics and current resource status, and output the optimal scheduling decision. The task distribution module is used to send tasks to be processed to designated edge nodes or the cloud for execution according to the scheduling decision results; It also includes a reinforcement learning weight update module, which dynamically adjusts the weight coefficients of each evaluation index during the multi-objective optimization process based on the actual feedback from historical scheduling results.

[0018] Another embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the claims as described above.

[0019] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the claims as stated above.

[0020] Compared with the prior art, the technical solution of this invention has the following advantages: (1) Dynamic adaptive scheduling: The present invention can dynamically adjust the execution position of visual analysis tasks according to the real-time network status and changes in computing resources. Compared with static allocation strategies (such as CN116596218B), the scheduling efficiency is improved by more than 40%.

[0021] (2) Multi-objective joint optimization: This invention establishes a multi-objective optimization scheduling model that includes three dimensions: delay, energy consumption and accuracy. It can achieve the optimal balance among multiple optimization objectives and improve the overall performance by 35% compared with a single-objective optimization scheme.

[0022] (3) Reinforcement learning adaptation: This invention introduces the Q-learning reinforcement learning algorithm to dynamically update the scheduling weights, enabling the system to continuously learn and adapt to environmental changes during long-term operation, and the scheduling accuracy continues to improve with running time.

[0023] (4) Optimization for inspection scenarios: This invention has made special optimization designs for the visual analysis scenarios of inspection robots, including task feature extraction and execution position evaluation. Compared with general edge cloud routing solutions (such as LAECIPS system), the task completion rate is improved by 25% in the inspection scenario.

[0024] (5) Low latency and high reliability: This invention reduces the average response latency to less than 200ms while ensuring the accuracy of analysis through the coordinated cooperation of edge preprocessing and cloud deep analysis, thus meeting the real-time requirements of inspection. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall system architecture of the present invention, showing the three-layer architecture of cloud server cluster, edge computing nodes and inspection robot terminal and their data interaction relationship.

[0026] Figure 2 This diagram illustrates the core principle of the dynamic scheduling algorithm of this invention, showing the overall structure of the input layer, the multi-objective optimization scheduling model, and the output layer.

[0027] Figure 3 The flowchart of the method of the present invention illustrates the complete scheduling process from state acquisition to weight update.

[0028] Figure 4 The graph shows a performance comparison between the present invention and existing solutions, illustrating the overall score comparison results.

[0029] Figure 5 This is a flowchart of a method and system for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration, according to one embodiment of the present invention. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0031] Example 1: Power Line Inspection Scenario This embodiment applies the present invention to the dynamic scheduling of visual analysis tasks for power line inspection robots. The inspection robot is equipped with a binocular camera and an infrared thermal imager, and is required to perform defect detection and temperature anomaly analysis on power lines.

[0032] System deployment environment: The cloud is deployed in a telecom-grade data center, equipped with 8 NVIDIA A100 GPUs; the edge computing nodes are deployed in the substation computer rooms along the line, equipped with NVIDIA Jetson AGX Xavier; the inspection robot communicates with the edge nodes through a 5G network.

[0033] In step S1, the monitoring agent collects network status parameters every 200ms: 5G network bandwidth B(t) fluctuates between 50-200Mbps, latency D(t) between 5-30ms, and packet loss rate P(t) between 0.01%-0.5%. The available computing power Ce(t) at the edge fluctuates between 20-32 TOPS, and the available computing power Cc(t) in the cloud fluctuates between 200-2000 TOPS.

[0034] In step S2, the visual analysis task is divided into four categories: insulator damage detection (f_type=1, f_complexity=0.8, f_urgency=0.9, f_datasize=5MB), conductor strand breakage detection (f_type=2, f_complexity=0.7, f_urgency=0.95, f_datasize=8MB), temperature anomaly detection (f_type=3, f_complexity=0.4, f_urgency=0.7, f_datasize=2MB), and passage obstacle detection (f_type=4, f_complexity=0.6, f_urgency=0.85, f_datasize=3MB).

[0035] In step S3, the multi-objective optimization scheduling model calculates the combined score for edge execution and cloud execution for each task. Taking insulator damage detection as an example, when the network latency is 10ms: edge execution S_edge = 0.33×0.3 + 0.33×0.5 + 0.33×0.6 = 0.467, cloud execution S_cloud = 0.33×0.7 + 0.33×0.3 + 0.33×0.2 = 0.400, therefore cloud execution is selected.

[0036] In step S4, the task is distributed to the corresponding node via the gRPC protocol. TensorRT is used at the edge to accelerate inference, and PyTorch is used in the cloud for deep analysis.

[0037] In step S5, the Q-learning state space contains discretized values ​​of 3 network parameters and 2 computing power parameters, and the action space is {edge execution, cloud execution}. After 1000 rounds of iterative training, the weights converge.

[0038] Performance test results: In a continuous 24-hour line patrol task, the average scheduling latency of the method of this invention is 156ms, the task completion rate is 98.7%, the comprehensive energy efficiency ratio is 0.85, the scheduling efficiency is improved by 42% compared with the static edge allocation scheme, and the latency is reduced by 68% compared with the full cloud scheme.

[0039] Example 2: Pipeline Inspection Scenario This embodiment applies the present invention to the visual analysis task scheduling of a pipeline internal inspection robot. The pipeline inspection robot is equipped with a high-definition camera and LiDAR, and needs to detect corrosion, cracks, and deformation on the inner wall of the pipeline.

[0040] System deployment environment: The cloud is deployed on the enterprise's private cloud, equipped with 4 NVIDIA V100 GPUs; the edge computing node is deployed at the pipeline inlet workstation, equipped with an Intel Movidius VPU array; the inspection robot communicates with the edge node via Wi-Fi 6.

[0041] In this scenario, due to the unstable Wi-Fi signal within the pipeline, network bandwidth fluctuates significantly (5-80Mbps), and latency fluctuates between 20-150ms. The Q-learning module of this invention can quickly adapt to network changes, automatically switching the analysis task from the cloud to the edge when bandwidth suddenly drops.

[0042] Performance comparison test results: For the same pipeline inspection task set (500 visual analysis tasks), the performance of each solution is shown in the table below: Local execution: average latency 520ms, analysis accuracy 72%, task completion rate 85%, overall score 50 points.

[0043] Full cloud execution: average latency 380ms, analysis accuracy 94%, task completion rate 78%, overall score 65.

[0044] Static allocation: average latency 340ms, analysis accuracy 83%, task completion rate 88%, overall score 55 points.

[0045] The method of this invention has the following characteristics: average latency of 186ms, analysis accuracy of 92%, task completion rate of 97%, and overall score of 95.

[0046] Therefore, it can be seen that the method of the present invention improves the overall performance by 42%-90% compared with the existing solution in pipeline inspection scenarios, which fully verifies the effectiveness and practicality of the present invention.

Claims

1. A dynamic scheduling method for visual analysis tasks of inspection robots based on edge cloud collaboration, characterized in that, The method includes: Real-time collection of network status and computing resource information during the operation of the inspection robot. The network status includes at least the current available bandwidth, communication latency and network packet loss rate, while the computing resource information covers the current available computing power of the edge and the cloud respectively. For the visual analysis task to be processed, its multi-dimensional features are extracted. The features include the type of task, the required computational complexity, the urgency of the task, and the amount of data to be transmitted. The extracted task features and the current network and computing power status are input into a multi-objective optimization scheduling model. The model comprehensively evaluates the latency, energy consumption and inference accuracy of the task when it is executed at the edge or in the cloud. It also gives a comprehensive score to each candidate execution position by weighted summation and selects the position with the best comprehensive score as the final scheduling decision for the current task. According to the scheduling decision, the task is distributed to the corresponding edge node or cloud server to perform visual reasoning analysis; Meanwhile, a reinforcement learning mechanism is adopted to dynamically adjust the weight coefficients of the three indicators of latency, energy consumption and accuracy in the multi-objective optimization model based on the actual comprehensive performance generated after each task scheduling, so that subsequent scheduling decisions can adapt to the constantly changing network environment and computing load.

2. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration as described in claim 1, characterized in that, The monitoring agents deployed on edge nodes and in the cloud via the network collect the network status parameters and computing resource information, respectively, with the time interval for each collection set between 100 milliseconds and 500 milliseconds.

3. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration according to claim 2, characterized in that, The task types specifically include four categories: object detection, image classification, semantic segmentation, and anomaly detection. The computational complexity of a task is measured by the estimated computational cost of the pre-trained model called by that task.

4. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration according to claim 3, characterized in that, When evaluating latency performance, it is calculated based on the current network communication latency and the amount of data to be transmitted; when evaluating energy consumption, it is estimated based on the power consumption characteristics of the execution node and its expected running time. When evaluating inference accuracy, the inherent accuracy of the model used and the quality of the current input data are taken into account.

5. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration according to claim 4, characterized in that, When the scheduling model is run for the first time, the weights of the three indicators, latency, energy consumption and accuracy, are initially set to be equal. After each subsequent scheduling, the evaluation values ​​generated by reinforcement learning are converted into a weight distribution so that the sum of the three weights remains constant.

6. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration according to claim 5, characterized in that, The reinforcement learning mechanism adopts the standard temporal differential learning method, in which the learning step size is set to 0.1, the discount rate of future returns is set to 0.9, and a strategy combining random exploration and experience utilization is adopted in the weight adjustment process to discover a better weight configuration while maintaining scheduling stability.

7. The method for dynamic scheduling of visual analysis tasks for inspection robots based on edge cloud collaboration according to claim 6, characterized in that, When a task is distributed to an edge node for execution, if the edge node fails to return a processing result within a preset timeout period, the system will automatically transfer the task to the cloud for re-execution to ensure that the task can be completed within a reasonable time.

8. A dynamic scheduling device for visual analysis tasks of inspection robots based on edge cloud collaboration, characterized in that, The device includes: The status acquisition module is used to acquire network status parameters and available computing power information at the edge and cloud in real time. The task feature extraction module is used to extract multi-dimensional features for visual analysis tasks and generate corresponding task feature descriptions. The multi-objective optimization scheduling module is used to comprehensively evaluate the latency, energy consumption, and accuracy performance of each execution location based on task characteristics and current resource status, and output the optimal scheduling decision. The task distribution module is used to send tasks to be processed to designated edge nodes or the cloud for execution according to the scheduling decision results; It also includes a reinforcement learning weight update module, which dynamically adjusts the weight coefficients of each evaluation index during the multi-objective optimization process based on the actual feedback from historical scheduling results.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images