An event semantic priority cloud-edge collaborative video monitoring dynamic scheduling method and system

Abstract

The application provides a cloud-edge collaborative video monitoring dynamic scheduling method and system with event semantic priority, which comprises obtaining video stream data of one or more monitoring cameras and performing basic analysis and processing; an edge node sends a structured event summary to a cloud semantic analysis module for semantic analysis and risk assessment, generating event priority information; the edge node collects local real-time resource state information; a scheduling decision module of the edge node performs joint dynamic scheduling on the to-be-processed tasks according to the event priority information and the resource state information; when the task meets a preset compensation trigger condition, the edge node sends relevant data of the task to a cloud compensation processing module, and outputs a compensation processing result; the task state is updated according to the processing result, and the local task queue is rearranged. The application can improve the response timeliness and task scheduling accuracy of high-priority events, and reduce the system resource overhead caused by indiscriminate cloud uploading and invalid queuing.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to the fields of intelligent video surveillance, edge computing, cloud computing collaborative processing and task scheduling, and particularly to a cloud-edge collaborative video surveillance dynamic scheduling method and system based on event semantic priority. This method is applicable to various intelligent monitoring systems deployed on edge nodes and cloud platforms, and is used to improve the priority determination capability, resource scheduling capability, and real-time response capability of monitoring tasks in complex scenarios. Background Technology

[0002] With the widespread deployment of video surveillance systems, edge computing devices, and cloud computing platforms, intelligent video analytics applications are increasingly used in scenarios such as public safety, park management, traffic control, and industrial inspection. Compared to traditional methods that rely solely on manual review of surveillance videos, intelligent video surveillance systems typically combine technologies such as target detection, target tracking, abnormal behavior recognition, person re-identification, and event retrieval to automatically analyze targets and events in the monitored footage, thereby improving the automation level and event response efficiency of the surveillance system.

[0003] In recent years, with the development of artificial intelligence and multimodal semantic analysis technologies, an increasing number of intelligent video surveillance systems have adopted a cloud-edge collaborative architecture of "preliminary analysis at edge nodes + deep processing in the cloud." In this architecture, edge nodes are typically deployed near cameras or on local servers, responsible for performing tasks with high real-time requirements such as target detection, target tracking, and basic event recognition. The cloud platform, on the other hand, utilizes its strong computing power to perform more complex tasks such as semantic understanding, event analysis, cross-device collaborative analysis, and global resource scheduling. This approach can, to a certain extent, balance real-time performance, accuracy, and system scalability.

[0004] In existing technologies, common approaches for cloud-edge collaborative processing of intelligent video surveillance tasks include: pre-dividing video analysis tasks into edge-side execution tasks and cloud-side execution tasks according to task type; or offloading some tasks to the cloud for processing based on the current load of edge nodes; or, in multi-node scenarios, distributing and migrating tasks based on information such as node computing power, network bandwidth, and the number of tasks. Some solutions also introduce the concept of task priority, for example, setting fixed priorities for tasks based on task category, business rules, or preset levels, and determining the execution order or migration strategy of tasks accordingly.

[0005] However, the aforementioned existing technologies generally still have the following problems in intelligent video surveillance scenarios: (1) Lack of dynamic priority determination mechanism based on event semantic content Most existing systems primarily rely on task type, static rules, or node resource usage for task scheduling, typically failing to dynamically determine task priority based on the semantic content, risk level, and actual urgency of the monitored events. For example, different monitoring tasks may belong to the same type, but their corresponding event risk levels and handling time requirements may differ significantly. If only fixed priorities or uniform scheduling rules are used, truly high-risk, high-urgency events may not be prioritized for handling.

[0006] (2) Lack of a joint scheduling mechanism that combines event priority with edge resource status While some existing solutions consider factors such as edge node computing resources, cache usage, network bandwidth, or task queue length, they typically optimize primarily from the perspective of resource allocation or task offloading, failing to jointly analyze the semantic priority of events with the real-time resource status of edge nodes, task waiting time, and processing latency constraints. This often leads scheduling strategies to favor static load balancing or simple resource optimization, making it difficult to balance the urgency of events and overall system processing efficiency under resource constraints and a large number of concurrent tasks.

[0007] (3) Lack of a condition-triggered cloud-edge compensation mechanism for high-priority tasks When edge nodes lack local resources, task concurrency increases, or a sudden surge of high-risk events occurs, existing systems typically handle situations by queuing, migrating all tasks, or using fixed offloading strategies. These approaches either cause high-priority tasks to wait for extended periods locally or increase bandwidth consumption and cloud processing pressure due to indiscriminate cloud migration. There is a lack of a mechanism that can selectively migrate and compensate for high-priority tasks based on real-time resource conditions, thus impacting the timeliness of response and scheduling accuracy for critical events.

[0008] (4) Lack of a mechanism for updating task status and reordering queues after processing results are returned. In some existing cloud-edge collaboration solutions, even when tasks are migrated to the cloud for processing, the focus is often only on task unloading itself, with insufficient consideration given to how to update event states, cancel duplicate tasks, and adjust the execution order of subsequent tasks after the cloud processing results are returned. This makes it difficult for the system to adaptively adjust the scheduling queue in dynamic scenarios based on the latest analysis results, affecting the overall collaborative processing effect.

[0009] In summary, existing technologies in the cloud-edge collaborative processing of intelligent video surveillance lack a method that can dynamically determine the priority of events based on the semantic content of monitoring events, jointly schedule events in conjunction with the real-time resource status of edge nodes, implement condition-triggered cloud-edge compensation processing for high-priority tasks under resource-constrained conditions, and support dynamic scheduling methods that update task status and reorder queues after the processing results are returned. Summary of the Invention

[0010] This invention provides a cloud-edge collaborative video surveillance dynamic scheduling method and system based on event semantic priority. It uses structured event summaries generated by edge nodes as a foundation, leverages cloud-based semantic analysis results to determine the risk level and urgency of monitored events, and combines this with the real-time resource status of edge nodes to dynamically decide the execution order, processing location, and migration method of monitoring tasks. When edge resources are insufficient or the expected processing latency of high-priority tasks exceeds limits, selective cloud-edge compensation processing is triggered. After the processing results are returned, the task status is updated and the scheduling queue is rearranged, thereby improving the response timeliness and task scheduling accuracy of high-priority events, while reducing the system resource overhead caused by indiscriminate cloud migration and invalid queuing.

[0011] The present invention achieves the above objectives through the following technical solutions: A cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority includes: Step S1: The edge node acquires video stream data from one or more surveillance cameras, performs basic analysis processing on the video stream data, including at least target detection, target tracking and basic event recognition, and generates a structured event summary based on the analysis results; Step S2: The edge node sends the structured event summary to the cloud semantic analysis module. The cloud semantic analysis module performs semantic parsing and risk assessment on the structured event summary and generates event priority information, which includes event priority score and / or priority category. Step S3: The edge node collects local real-time resource status information, which includes at least one or more of the following: CPU utilization, GPU utilization, memory usage, local task queue length, network link latency, and available upload bandwidth. Step S4: The scheduling decision module of the edge node performs joint dynamic scheduling on the tasks to be processed based on the event priority information and resource status information, determines the execution order and processing position of each task, and the scheduling decision is at least one of the following: execute immediately locally, queue locally for execution, forward to a nearby edge node for execution, send to the cloud for compensation processing, or postpone execution and wait for the next round of scheduling. Step S5: When the task meets the preset compensation triggering conditions, the edge node sends the relevant data of the task to the cloud compensation processing module. The cloud compensation processing module performs further analysis and processing on the task and outputs the compensation processing results. Step S6: The edge node receives the processing result returned by the cloud compensation processing module, updates the task status according to the processing result, and reorders the local task queue.

[0012] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority is provided. The structured event summary includes at least one or more of the following: event number, event occurrence timestamp, edge node number, camera number, target category, number of targets, behavior label, location area identifier, event duration and initial anomaly score, video segment identifier, target location coordinates or region box information, target trajectory summary information, and local processing status identifier.

[0013] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority is provided. In step S1, after the edge node generates a structured event summary, it prioritizes uploading the structured event summary to the cloud semantic analysis module. After receiving the event priority information returned by the cloud semantic analysis module, the scheduling decision module generates a scheduling decision based on the event priority information and resource status information. When the scheduling decision is to send the data to the cloud for compensation processing, the edge node selectively uploads the video clip, target cropped image, or feature representation corresponding to the current task to the cloud compensation processing module according to the scheduling decision; when the scheduling decision is to execute the data locally immediately or to queue the data locally for execution, the edge node does not upload the video clip, target cropped image, or feature representation.

[0014] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method for event semantic priority is provided. In step S2, the cloud semantic analysis module uses a large language model, a visual language model, a rule engine, a knowledge graph reasoning module or a combination thereof to realize semantic parsing and risk assessment. After receiving the structured event summary, the risk level of the event is assessed by combining the preset risk rules, semantic tag mapping relationship, historical event pattern or scene knowledge. Event priority information is output in any of the following formats: Discrete priority levels, including high priority, medium priority, and low priority; Numerical priority score; Multidimensional priority description, including a combination of urgency, danger level, and timeliness requirements.

[0015] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority is provided. In step S2, the cloud semantic analysis module outputs an event priority score P.i and the corresponding priority category Label i ; When P i When the value exceeds the first preset threshold, it is marked as a high-priority task; When P i Tasks falling between the first and second preset thresholds are marked as medium priority tasks. When P i When the priority is less than the second preset threshold, it is marked as a low-priority task.

[0016] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority is provided. In step S3, the edge node periodically collects resource status information, or triggers real-time collection when a new task arrives, a task is completed, or a sudden change in system load is detected. The resource status information includes at least one or more of the following: CPU utilization, GPU utilization, memory usage, video memory usage, local task queue length, current cache usage, network link latency, currently available upload bandwidth, average task processing time in the recent period, and cumulative waiting time of high-priority tasks. Step S3 also includes collecting task status information for each task. The task status information includes one or more of the following: task arrival time, waiting time, estimated computational cost, required model type, local executability flag, and migration permission flag.

[0017] According to the present invention, a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority is provided. In step S4, the joint dynamic scheduling comprehensively considers at least one of the following factors: event semantic priority, task waiting time, expected task processing delay, current CPU utilization of edge nodes, current GPU utilization of edge nodes, queue length, network link status, task migration cost, and task timeliness constraints. The scheduling decision module constructs a comprehensive scheduling index D for each task. i Comprehensive scheduling index D i It is used to reflect the current urgency of task execution and is obtained through rule calculation, weighted summation, hierarchical judgment, policy table matching, or learning model output.

[0018] According to the cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority provided by the present invention, in step S4, based on comprehensive scheduling indicators, the edge node assigns at least one of the following scheduling actions to each task: Local immediate execution: Executes when the task is of high priority and local resources are sufficient; Local queuing for execution: Execute when the task is of medium to low priority and local resources are scarce; Forward to a nearby edge node for execution: Execute when a nearby edge node has available resources; Upload to the cloud for compensation processing: This is executed when the task is of high priority but local resources are scarce and the network status meets the migration conditions. Execution is suspended and awaits the next round of scheduling: Execute when the task is of medium to low priority and local resources are scarce.

[0019] According to the cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority provided by the present invention, in step S5, the preset compensation triggering conditions include at least one or more of the following: The task priority is higher than the preset threshold; High-priority tasks are expected to wait longer than a preset latency threshold; The utilization rate of the edge node's graphics processor or central processing unit exceeds the preset resource threshold. The local task queue length exceeds the preset length threshold; The current network bandwidth meets the upload requirements; The task was deemed suitable for execution in the cloud; Multiple high-priority tasks arrive in quick succession; The necessary data sent from the edge node to the cloud compensation processing module includes structured event summaries, corresponding video clips, target cropped images, feature vectors, and local preliminary recognition results.

[0020] A cloud-edge collaborative video surveillance dynamic scheduling system based on event semantic priority, which is applied to the aforementioned cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority, includes: The video access module is used to access video streams from one or more surveillance cameras and send the video streams to the edge node processing module; The edge awareness module performs object detection, object tracking, and basic event recognition, generating structured event summaries. The event summary management module is used to encapsulate, cache, number, and control the upload of structured event summaries, and maintain the correspondence between tasks and video clips; The cloud-based semantic analysis module is used to receive structured event summaries and perform semantic parsing, risk assessment, and priority generation on the events. The resource status acquisition module is used to collect resource information such as CPU, GPU, memory, bandwidth, and queue length of edge nodes in real time. The scheduling decision module is used to output scheduling actions such as local execution, queuing, neighbor forwarding, or cloud compensation based on event semantic priority and resource status. The cloud-based compensation processing module is used to perform further detailed analysis on high-priority tasks that meet the conditions and output the compensation processing results. The result feedback and queue update module is used to receive cloud processing results, update task status, and rearrange the local task queue. The alarm and result output module is used to send the final event judgment result to the alarm terminal, management platform or other upper-level application system.

[0021] Therefore, compared with existing technologies, the cloud-edge collaborative video surveillance dynamic scheduling method and system based on event semantic priority proposed in this invention have the following advantages: 1. This invention utilizes structured event summaries generated by edge nodes and cloud-based semantic analysis results to construct a dynamic scheduling mechanism based on event semantic priority. This enables task scheduling to move beyond task categories or static resource allocation, thereby determining the task execution order and processing location based on actual risk levels and urgency, and improving the handling and guarantee capabilities for high-priority events.

[0022] 2. This invention dynamically adjusts the scheduling strategy based on event priority, resource load status, and task waiting status, and combines it with a condition-triggered cloud-edge compensation mechanism. Compared with fixed priority or simple offloading methods, it can effectively control edge node resource occupation, cloud processing pressure, and overall collaborative overhead while ensuring the timeliness of critical task response.

[0023] 3. This invention achieves dynamic closed-loop correction of scheduling strategy through task status update and queue reordering mechanism after result feedback. It can adjust the priority of subsequent tasks and remove duplicate or invalid tasks in a timely manner according to the latest analysis results, avoid invalid resource occupation, and further improve the system's adaptive scheduling capability in dynamic and complex scenarios.

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0025] Figure 1 This is a flowchart of an embodiment of a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority according to the present invention.

[0026] Figure 2 This is a flowchart illustrating an embodiment of the cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority according to the present invention.

[0027] Figure 3 This is a system schematic diagram of an embodiment of the cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority according to the present invention.

[0028] Figure 4 This is a logical principle diagram of scheduling decisions in an embodiment of the cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority according to the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] See Figures 1 to 4 This embodiment provides a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority, including: Step S1: The edge node acquires video stream data from one or more surveillance cameras, performs basic analysis processing on the video stream data, including at least target detection, target tracking and basic event recognition, and generates a structured event summary based on the analysis results; Step S2: The edge node sends the structured event summary to the cloud semantic analysis module. The cloud semantic analysis module performs semantic parsing and risk assessment on the structured event summary and generates event priority information, which includes event priority score and / or priority category. Step S3: Edge nodes collect local real-time resource status information to obtain the resource status information required for current scheduling; Step S4: The scheduling decision module of the edge node performs joint dynamic scheduling on the tasks to be processed based on the event priority information and resource status information, determines the execution order and processing position of each task, and the scheduling decision is at least one of the following: execute immediately locally, queue locally for execution, forward to a nearby edge node for execution, send to the cloud for compensation processing, or postpone execution and wait for the next round of scheduling. Step S5: When the task meets the preset compensation triggering conditions, the edge node sends the relevant data of the task to the cloud compensation processing module. The cloud compensation processing module performs further analysis and processing on the task and outputs the compensation processing results. Step S6: After the cloud completes the compensation process, the edge node receives the processing result returned by the cloud compensation processing module, updates the task status according to the processing result, and reorders the local task queue.

[0032] In step S1, target detection is used to identify target objects in video frames. Target objects may include people, vehicles, items, or preset interest categories. Target tracking is used to maintain the temporal correlation of the same target between consecutive video frames. Basic event recognition is used to make preliminary judgments on the target's behavior, area entry status, dwell status, movement status, or other preset monitoring events.

[0033] The edge nodes generate a structured event summary based on the above analysis results. The structured event summary includes at least one or more of the following: event number, event timestamp, edge node number, camera number, target category, number of targets, behavior label, location area identifier, event duration and initial anomaly score, video segment identifier, target location coordinates or region box information, target trajectory summary information, and local processing status identifier.

[0034] In one implementation, after generating a structured event summary, edge nodes do not directly upload the original video stream. Instead, they prioritize uploading the structured event summary to the cloud-based semantic analysis module. Upon receiving event priority information from the cloud-based semantic analysis module, the scheduling decision module generates a scheduling decision based on the event priority information and resource status information. When further analysis is required, the scheduling decision determines whether to upload the corresponding video segment, target cropped image, or feature representation. This approach reduces network bandwidth consumption and minimizes the transmission burden caused by uploading irrelevant video data.

[0035] Specifically, when the scheduling decision is to send the data to the cloud for compensation processing, the edge node selectively uploads the video clip, target cropped image, or feature representation corresponding to the current task to the cloud compensation processing module according to the scheduling decision; when the scheduling decision is to execute locally immediately or queue locally for execution, the edge node does not upload the video clip, target cropped image, or feature representation.

[0036] The output of step S1 is the set of tasks to be scheduled, T = T1, T2, ..., T. n Each task T i Corresponding to a structured event summary E i .

[0037] In step S2, the cloud-based semantic analysis module uses a large language model, a visual language model, a rule engine, a knowledge graph reasoning module, or a combination thereof to achieve semantic parsing and risk assessment. In order to avoid limiting the present invention to a specific model, the present invention focuses on protecting the processing mechanism for generating semantic priorities based on event summaries, rather than a specific model itself.

[0038] After receiving structured event summaries, the cloud-based semantic analysis module assesses the risk level of an event by combining preset risk rules, semantic tag mapping relationships, historical event patterns, or scenario knowledge. Event priority information is output in any of the following formats: (1) Discrete priority levels, including high priority, medium priority and low priority; (2) Numerical priority scores, such as risk scores between 0 and 1; (3) Multidimensional priority description, including a combination of urgency, danger and timeliness requirements.

[0039] In one preferred implementation, the cloud-based semantic analysis module outputs an event priority score P. i and the corresponding priority category Label i ,For example: When P i When the value exceeds the first preset threshold, it is marked as a high-priority task; When P i Tasks falling between the first and second preset thresholds are marked as medium priority tasks. When P i When the priority is less than the second preset threshold, it is marked as a low-priority task.

[0040] To facilitate subsequent scheduling, the cloud can also output semantic explanation information for each task, such as tags for suspected armed behavior, abnormal gathering in key areas, and repeated appearances of key personnel, to enhance the interpretability of scheduling decisions.

[0041] The output of step S2 is a set of tasks T'=T1',T2',…,T with event priority identifiers. n '.

[0042] In step S3, the edge node periodically collects resource status information, or triggers real-time collection when a new task arrives, a task is completed, or the system load changes suddenly; the resource status information includes at least one or more of the following: CPU utilization, GPU utilization, memory usage, video memory usage, local task queue length, current cache usage, network link latency, currently available upload bandwidth, average task processing time in the recent period, and cumulative waiting time of high-priority tasks; Among them, edge nodes can periodically collect resource status information, or trigger real-time collection when a new task arrives, a task is completed, or the system load changes suddenly.

[0043] Step S3 also includes synchronously collecting task status information for each task. The task status information includes one or more of the following: task arrival time, waiting time, estimated computational cost, required model type, local executability flag, and migration permission flag.

[0044] Resource status information and task status information together constitute the scheduling decision input set S. i The output of step S3 is the resource state set R. s and task state set Q s .

[0045] In step S4, the joint dynamic scheduling is not based on a single factor for sorting, but takes into account at least the following factors: event semantic priority, task waiting time, expected task processing latency, current CPU utilization of edge nodes, current GPU utilization of edge nodes, queue length, network link status, task migration cost, and task timeliness constraints. In one implementation, the scheduling decision module constructs a comprehensive scheduling index D for each task. i Comprehensive scheduling index D i This indicator reflects the current urgency of task execution and is obtained through rule calculation, weighted summation, hierarchical judgment, policy table matching, or learning model output. This embodiment does not limit the specific mathematical expression of the comprehensive scheduling indicator, as long as it enables joint decision-making based on event priority and resource status.

[0046] In step S4, based on comprehensive scheduling metrics, the edge node assigns at least one of the following scheduling actions to each task: Local immediate execution: Executes when the task is of high priority and local resources are sufficient; Local queuing for execution: Execute when the task is of medium to low priority and local resources are scarce; Forward to a nearby edge node for execution: Execute when a nearby edge node has available resources; Upload to the cloud for compensation processing: This is executed when the task is of high priority but local resources are scarce and the network status meets the migration conditions. Execution is suspended and awaits the next round of scheduling: Execute when the task is of medium to low priority and local resources are scarce.

[0047] In this way, the system can prioritize the timeliness of processing critical events without increasing the burden of processing all tasks in the cloud.

[0048] The output of step S4 is the task scheduling decision result set A = a1, a2, ..., a n .

[0049] In step S5, the preset compensation triggering conditions include at least one or more of the following: The task priority is higher than the preset threshold; High-priority tasks are expected to wait longer than a preset latency threshold; The utilization rate of the edge node's graphics processor or central processing unit exceeds the preset resource threshold. The local task queue length exceeds the preset length threshold; The current network bandwidth meets the upload requirements; The task was deemed suitable for execution in the cloud; Multiple high-priority tasks arrive in quick succession; When the above conditions are met, the edge node will send the necessary data related to the task to the cloud compensation processing module. The necessary data may include structured event summaries, corresponding video clips, target cropped images, feature vectors, and local preliminary recognition results.

[0050] After receiving the above data, the cloud-based compensation processing module calls a higher-precision or more computationally expensive analysis model to perform further processing, including one or more of the following: more refined abnormal behavior recognition, higher-precision person re-identification, cross-camera target association analysis, semantic-level event confirmation, and multi-source information joint analysis. In this embodiment, the cloud-based compensation process does not perform remote analysis on all tasks uniformly, but only performs selective migration and compensation on high-priority tasks that meet the conditions, thereby balancing critical task protection with overall system overhead control.

[0051] The output of step S5 is the cloud compensation result set C. r .

[0052] In step S6, the processing result includes at least one or more of the following: the final event determination result, the updated risk level, the target re-identification result, the cross-device association result, the follow-up tracking suggestion, and the control instruction for whether to generate an alarm. The edge node performs at least one of the following operations based on the processing result: updates the task status to completed, confirmed, or alarmed; removes duplicate or expired tasks from the local task queue; recalculates the priority of related tasks that are still in the waiting state; adjusts the execution order of subsequent tasks; and pushes the processing result to the alarm module, the display module, or the upper-layer business system.

[0053] In one implementation, if the cloud returns a result indicating that the event has been confirmed as a high-risk anomaly, the edge node immediately raises the priority of subsequent tasks related to the event; if the cloud returns a result indicating that the event is a false alarm, the edge node lowers the priority of related tasks or directly terminates subsequent processing to reduce the consumption of ineffective resources.

[0054] Through step S6, the system can continuously update the task status in dynamic and complex scenarios and optimize subsequent scheduling strategies based on the latest results, thus forming a complete closed loop.

[0055] In practical applications, such as Figure 2As shown, the overall system of this invention includes a front-end acquisition layer, edge nodes, a cloud platform, and an application output layer. The front-end acquisition layer provides raw video streams to the edge nodes via surveillance cameras and video stream inputs. The edge nodes include a video access module, an edge perception module, an event summary management module, a resource status acquisition module, a scheduling decision module, a local execution / queue processing module, and a result feedback and queue update module. The cloud platform includes a semantic analysis and priority generation module and a cloud compensation processing module. The application output layer includes an alarm and result output module and an upper-layer business platform.

[0056] The process involves several steps: the raw video stream is input to the edge node via the video access module; the edge perception module performs target detection, target tracking, and basic event recognition on the video stream; the event summary management module generates an event summary and sends it to the cloud platform for semantic analysis and priority generation; the resource status acquisition module sends the resource status information of the edge node to the scheduling decision module; the scheduling decision module combines the event semantic priority and the edge resource status to decide whether to execute the task locally, queue it, or perform cloud compensation processing; the cloud compensation processing module further analyzes high-priority or complex tasks and returns the processing results to the result feedback and queue update module; finally, the alarm and result output module outputs alarm information and processing results to the upper-layer business platform.

[0057] like Figure 3 As shown, the method flow of the present invention includes the following steps: First, video access and event summary generation are performed (S1), target detection, target tracking and basic event recognition are performed on the input video stream to generate a structured event summary; then, cloud semantic parsing and priority generation are performed (S2), the risk level of the event is assessed and the priority score or priority category is output; next, edge resource status acquisition is performed (S3), resource status information such as CPU utilization, GPU utilization, bandwidth, queue length, estimated waiting time and available computing power are obtained; then, event priority and resource status are jointly scheduled (S4), the priority and resource status information are fused to determine the task execution order and processing location.

[0058] After step S4, it is determined whether the task meets the cloud-edge compensation conditions. If not, it proceeds to local execution or queue waiting. If it meets the conditions, it proceeds to high-priority task cloud-edge compensation processing (S5), uploading event summaries and related data for cloud-based compensation analysis. Regardless of whether the task is executed locally or processed via cloud compensation, it ultimately enters the result feedback, status update, and queue reordering steps (S6) to update the task status and resource usage, adjust subsequent queues, output alarm results, and finally output the results and proceed to the next round of scheduling.

[0059] like Figure 4As shown, the scheduling decision module receives two types of input information: one is event task information, including event priority, task waiting time, estimated processing latency, and whether migration is allowed; the other is edge resource status, including CPU utilization, GPU utilization, queue length, and network status. Based on the above inputs, the scheduling decision module performs a joint priority-resource evaluation and generates corresponding scheduling actions.

[0060] Scheduling actions include immediate local execution, local queuing, forwarding to nearby edges, cloud-based compensation processing, and deferred execution. Among these, cloud-based compensation processing can be conditionally triggered when the task is of high priority, edge resources are scarce, and network conditions are met. Figure 3 This is used to illustrate the logical relationship in which the scheduling decision module in this invention generates various processing actions based on task information and resource status.

[0061] An Implementation Example of a Cloud-Edge Collaborative Video Surveillance Dynamic Scheduling System Based on Event Semantic Priority A cloud-edge collaborative video surveillance dynamic scheduling system based on event semantic priority, which is applied to the aforementioned cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority, includes: The video access module is used to access video streams from one or more surveillance cameras and send the video streams to the edge node processing module; The edge awareness module performs object detection, object tracking, and basic event recognition, generating structured event summaries. The event summary management module is used to encapsulate, cache, number, and control the upload of structured event summaries, and maintain the correspondence between tasks and video clips; The cloud-based semantic analysis module is used to receive structured event summaries and perform semantic parsing, risk assessment, and priority generation on the events. The resource status acquisition module is used to collect resource information such as CPU, GPU, memory, bandwidth, and queue length of edge nodes in real time. The scheduling decision module is used to output scheduling actions such as local execution, queuing, neighbor forwarding, or cloud compensation based on event semantic priority and resource status. The cloud-based compensation processing module is used to perform further detailed analysis on high-priority tasks that meet the conditions and output the compensation processing results. The result feedback and queue update module is used to receive cloud processing results, update task status, and rearrange the local task queue. The alarm and result output module is used to send the final event judgment result to the alarm terminal, management platform or other upper-level application system.

[0062] Furthermore, the structured event summary generated by the edge perception module includes at least one or more of the following: event number, event timestamp, edge node number, camera number, target category, number of targets, behavior label, location area identifier, event duration, initial anomaly score, video segment identifier, target location coordinates or region box information, target trajectory summary information, and local processing status identifier.

[0063] Furthermore, the cloud-based semantic analysis module employs a large language model, a visual language model, a rule engine, a knowledge graph reasoning module, or a combination thereof, to output event priority scores and / or priority categories, and to output semantic explanation information.

[0064] Furthermore, the scheduling decision module comprehensively considers at least one of the following factors: event semantic priority, task waiting time, expected task processing latency, current CPU utilization of edge nodes, current GPU utilization of edge nodes, queue length, network link status, task migration cost, and task timeliness constraints. It constructs a comprehensive scheduling index for each task and determines the scheduling action based on the comprehensive scheduling index.

[0065] In practical applications, the method provided in this embodiment is deployed in an intelligent video surveillance system consisting of surveillance cameras, edge nodes, cloud servers, and an upper-layer business platform. Edge nodes may include one or more edge computing devices, and cloud servers may include one or more servers with high computing power. Each device may include one or more processors, memory, network communication interfaces, and a software environment for running relevant program instructions.

[0066] in: Surveillance cameras are used to capture real-time video streams or video clips; Edge nodes are used to perform target detection, target tracking, basic event recognition, resource status acquisition, and local scheduling decisions; The cloud server is used to perform event semantic analysis, risk level assessment, and compensation processing for high-priority tasks; The upper-level business platform is used to receive alarm information, display processing results, or execute further business linkages.

[0067] The system is divided into several logical modules according to their functions: Video access module: Used to receive video stream data from one or more surveillance cameras and send the video stream data to the edge perception module for preprocessing and analysis.

[0068] Edge perception module: Used to perform target detection, target tracking and basic event recognition on the input video stream, extract target information, behavior information and event information in the monitoring scene, and generate structured event summaries.

[0069] Event summary management module: used to encapsulate, number, cache and upload structured event summaries, and establish mapping relationships between event summaries and corresponding video clips, target cropped images or feature representations.

[0070] Cloud-based semantic analysis module: Used to receive structured event summaries, perform semantic parsing and risk assessment on monitored events, and output event priority scores, priority categories, and semantic tag information.

[0071] Resource Status Acquisition Module: Used to collect real-time resource status information of edge nodes, including CPU utilization, GPU utilization, memory usage, task queue length, network latency, and available bandwidth.

[0072] Scheduling Decision Module: This module dynamically decides on the task execution order, execution location, and migration method based on the event priority information output by the cloud semantic analysis module and the resource status information output by the resource status acquisition module.

[0073] Cloud-based compensation processing module: When the compensation triggering conditions are met, it is used to perform further analysis and processing on high-priority tasks, including higher-precision abnormal behavior recognition, person re-identification, cross-camera correlation analysis, or semantic-level event confirmation.

[0074] The result feedback and queue update module is used to receive cloud processing results, update task status, and perform reordering, deduplication, or priority adjustment on the local task queue of edge nodes.

[0075] Alarm and Result Output Module: Used to output the final event judgment result, alarm information and relevant location information to the monitoring management platform, alarm terminal or other upper-level application systems.

[0076] The modules described above can be implemented in software or a combination of software and hardware, and can work together through inter-process communication, message queue communication, or network communication. In this embodiment, the method of the present invention can be applied to monitoring scenarios in key areas of a park.

[0077] After receiving a video stream from a camera, an edge node detects three people moving rapidly and briefly gathering within a key area A. The edge perception module generates a corresponding structured event summary, which includes: event timestamp 2026-03-09 10:15:30, target category as people, behavior tags as running rapidly and briefly gathering, area identifier as key area A, number of targets as 3, and event duration as 8 seconds.

[0078] Subsequently, the edge node sends the structured event summary to the cloud-based semantic analysis module. The cloud-based semantic analysis module performs semantic parsing and risk assessment on the event based on the target category, behavioral tags, region identifiers, and target quantity in the event summary, and outputs a priority score of 0.92, a high priority category, and a semantic interpretation tag of "abnormal clustering event in key areas."

[0079] Meanwhile, the resource status acquisition module obtains the following resource status of the current edge node: the graphics processor utilization rate is 93%, the local task queue length is 16, and the currently available upload bandwidth meets the upload requirements. Based on the judgment that the task is of high priority, the edge node resources are scarce, and the network status meets the migration conditions, the scheduling decision module determines to perform the scheduling action of "uploading to the cloud for compensation processing" for this task.

[0080] The edge node then uploads a structured event summary and corresponding video clips related to the task to the cloud compensation processing module. After further analysis of the uploaded data, the cloud compensation processing module outputs the following results: the final determination is a high-risk abnormal event, the alarm is generated in the yes case, and the subsequent tracking suggestion is to continuously track the relevant targets in adjacent cameras.

[0081] After receiving the above processing results, the edge node updates the task status to "alarmed" and raises the subsequent analysis task corresponding to the relevant target in the camera near key area A to high priority. At the same time, it sends alarm information to the upper-layer business platform, thereby realizing the rapid handling of critical events.

[0082] Optional implementation method: Parameter and threshold settings In one implementation, the structured event summary includes at least the event occurrence timestamp, target category, and behavior label, and may further include one or more of the following: event number, edge node number, camera number, number of targets, area identifier, event duration, initial anomaly score, video segment identifier, target location coordinates or region box information, target trajectory summary information, and local processing status identifier.

[0083] In one implementation, the cloud-based semantic analysis module outputs an event priority score and priority category, and can classify the priority category into high priority, medium priority, and low priority. For example, when the priority score is greater than a first preset threshold, it is determined to be high priority; when the priority score is between the first and second preset thresholds, it is determined to be medium priority; and when the priority score is less than the second preset threshold, it is determined to be low priority.

[0084] In one implementation, the resource thresholds for edge nodes can be set as follows: GPU utilization threshold of 85%, CPU utilization threshold of 80%, local task queue length threshold of 10, and expected waiting time threshold for high-priority tasks of 2 seconds. When a high-priority task arrives, if GPU utilization exceeds 85% or local task queue length exceeds 10, and the current network status meets the migration requirements, cloud compensation processing is triggered.

[0085] The parameters and thresholds mentioned above are merely examples. Those skilled in the art can adjust them according to different deployment scenarios, hardware conditions, and business needs without affecting the implementation of the method of the present invention.

[0086] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority, characterized in that, include: Step S1: The edge node acquires video stream data from one or more surveillance cameras, performs basic analysis processing on the video stream data, including at least target detection, target tracking and basic event recognition, and generates a structured event summary based on the analysis results; Step S2: The edge node sends the structured event summary to the cloud semantic analysis module. The cloud semantic analysis module performs semantic parsing and risk assessment on the structured event summary and generates event priority information, which includes event priority score and / or priority category. Step S3: The edge node collects local real-time resource status information, which includes at least one or more of the following: CPU utilization, GPU utilization, memory usage, local task queue length, network link latency, and available upload bandwidth. Step S4: The scheduling decision module of the edge node performs joint dynamic scheduling on the tasks to be processed based on the event priority information and resource status information, determines the execution order and processing position of each task, and the scheduling decision is at least one of the following: execute immediately locally, queue locally for execution, forward to a nearby edge node for execution, send to the cloud for compensation processing, or postpone execution and wait for the next round of scheduling. Step S5: When the task meets the preset compensation triggering conditions, the edge node sends the relevant data of the task to the cloud compensation processing module. The cloud compensation processing module performs further analysis and processing on the task and outputs the compensation processing results. Step S6: The edge node receives the processing result returned by the cloud compensation processing module, updates the task status according to the processing result, and reorders the local task queue.

2. The method according to claim 1, characterized in that: The structured event summary includes at least one or more of the following: event number, event timestamp, edge node number, camera number, target category, number of targets, behavior label, location area identifier, event duration and initial anomaly score, video segment identifier, target location coordinates or region bounding box information, target trajectory summary information, and local processing status identifier.

3. The method according to claim 1, characterized in that: In step S1, after the edge node generates a structured event summary, it first uploads the structured event summary to the cloud semantic analysis module. After receiving the event priority information returned by the cloud semantic analysis module, the scheduling decision module generates a scheduling decision based on the event priority information and resource status information. When the scheduling decision is to send the data to the cloud for compensation processing, the edge node selectively uploads the video clip, target cropped image, or feature representation corresponding to the current task to the cloud compensation processing module according to the scheduling decision. When the scheduling decision is to execute immediately locally or queue locally for execution, the edge node will not upload video clips, target cropped images, or feature representations for the time being.

4. The method according to claim 1, characterized in that: In step S2, the cloud-based semantic analysis module uses a large language model, a visual language model, a rule engine, a knowledge graph reasoning module, or a combination thereof to achieve semantic parsing and risk assessment. After receiving a structured event summary, the risk level of the event is assessed by combining preset risk rules, semantic tag mapping relationships, historical event patterns or scenario knowledge; Event priority information is output in any of the following formats: Discrete priority levels, including high priority, medium priority, and low priority; Numerical priority score; Multidimensional priority description, including a combination of urgency, danger level, and timeliness requirements.

5. The method according to claim 1, characterized in that: In step S2, the cloud-based semantic analysis module outputs the event priority score P. i and the corresponding priority category Label i ; When P i When the value exceeds the first preset threshold, it is marked as a high-priority task; When P i Tasks falling between the first and second preset thresholds are marked as medium priority tasks. When P i When the priority is less than the second preset threshold, it is marked as a low-priority task.

6. The method according to claim 1, characterized in that: In step S3, the edge node periodically collects resource status information, or triggers real-time collection when a new task arrives, a task is completed, or the system load changes suddenly; the resource status information includes at least one or more of the following: CPU utilization, GPU utilization, memory usage, video memory usage, local task queue length, current cache usage, network link latency, currently available upload bandwidth, average task processing time in the recent period, and cumulative waiting time of high-priority tasks; Step S3 also includes collecting task status information for each task. The task status information includes one or more of the following: task arrival time, waiting time, estimated computational cost, required model type, local executability flag, and migration permission flag.

7. The method according to claim 1, characterized in that: In step S4, the joint dynamic scheduling comprehensively considers at least one of the following factors: event semantic priority, task waiting time, expected task processing latency, current CPU utilization of edge nodes, current GPU utilization of edge nodes, queue length, network link status, task migration cost, and task timeliness constraints. The scheduling decision module constructs a comprehensive scheduling index D for each task. i Comprehensive scheduling index D i It is used to reflect the current urgency of task execution and is obtained through rule calculation, weighted summation, hierarchical judgment, policy table matching, or learning model output.

8. The method according to claim 7, characterized in that: In step S4, based on comprehensive scheduling metrics, the edge node assigns at least one of the following scheduling actions to each task: Local immediate execution: Executes when the task is of high priority and local resources are sufficient; Local queuing for execution: Execute when the task is of medium to low priority and local resources are scarce; Forward to a nearby edge node for execution: Execute when a nearby edge node has available resources; Upload to the cloud for compensation processing: This is executed when the task is of high priority but local resources are scarce and the network status meets the migration conditions. Execution is suspended and awaits the next round of scheduling: Execute when the task is of medium to low priority and local resources are scarce.

9. The method according to any one of claims 1 to 8, characterized in that: In step S5, the preset compensation trigger conditions include at least one or more of the following: The task priority is higher than the preset threshold; High-priority tasks are expected to wait longer than a preset latency threshold; The utilization rate of the edge node's graphics processor or central processing unit exceeds the preset resource threshold. The local task queue length exceeds the preset length threshold; The current network bandwidth meets the upload requirements; The task was deemed suitable for execution in the cloud; Multiple high-priority tasks arrive in quick succession; The necessary data sent from the edge node to the cloud compensation processing module includes structured event summaries, corresponding video clips, target cropped images, feature vectors, and local preliminary recognition results.

10. A cloud-edge collaborative video surveillance dynamic scheduling system with event semantic priority, characterized in that, This system is applied to a cloud-edge collaborative video surveillance dynamic scheduling method based on event semantic priority as described in any one of claims 1 to 9, the system comprising: The video access module is used to access video streams from one or more surveillance cameras and send the video streams to the edge node processing module; The edge awareness module performs object detection, object tracking, and basic event recognition, generating structured event summaries. The event summary management module is used to encapsulate, cache, number, and control the upload of structured event summaries, and maintain the correspondence between tasks and video clips; The cloud-based semantic analysis module is used to receive structured event summaries and perform semantic parsing, risk assessment, and priority generation on the events. The resource status acquisition module is used to collect resource information such as CPU, GPU, memory, bandwidth, and queue length of edge nodes in real time. The scheduling decision module is used to output scheduling actions such as local execution, queuing, neighbor forwarding, or cloud compensation based on event semantic priority and resource status. The cloud-based compensation processing module is used to perform further detailed analysis on high-priority tasks that meet the conditions and output the compensation processing results. The result feedback and queue update module is used to receive cloud processing results, update task status, and rearrange the local task queue. The alarm and result output module is used to send the final event judgment result to the alarm terminal, management platform or other upper-level application system.

Images