Unmanned aerial vehicle alarm closed-loop disposal system based on cloud-side cooperation and disposal method thereof

Abstract

The invention discloses an unmanned aerial vehicle alarm closed-loop disposal system based on cloud-side cooperation and a disposal method thereof. The system comprises a plurality of unmanned aerial vehicle modules which are responsible for collecting video streams, receiving closed-loop instructions and executing automatic disposal actions; the edge computing node is deployed at a near field, receives a video stream with low delay, identifies an alarm condition in real time through the AI analysis unit, filters the alarm condition through the event duplicate removal unit to determine a new event, and the alarm condition generation unit generates a structured alarm condition message and a closed-loop control instruction in the case of the new event to drive the unmanned aerial vehicle to perform disposal; and the cloud platform is responsible for receiving, storing and displaying the management alarm information. The unmanned aerial vehicle alarm condition closed-loop disposal system realizes an automatic closed loop of alarm condition sensing, decision making and action.

Description

Technical Field

[0001] This invention relates to the field of computer vision and drone control technology, and in particular to an automated patrol and intelligent emergency response system and method for drones based on artificial intelligence, edge computing and Internet of Things communication. Background Technology

[0002] In recent years, with the increasing demand for smart cities and public safety, drones have been increasingly widely used in areas such as police patrols, security maintenance, major event security, traffic management, and disaster response. Drones possess advantages such as high mobility, rapid deployment, and wide field of view, gradually becoming an important supplement to ground patrol forces. However, current mainstream drone patrol technologies still heavily rely on human observation and manual operation by pilots, failing to truly realize the automation and intelligence potential of drones.

[0003] Currently, most technical solutions adopt a "manual monitoring + manual operation" mode. The drone transmits real-time images to the backend, where professional pilots or monitoring personnel monitor the screen for extended periods, manually identifying any suspicious individuals, abnormal behaviors, or dangerous situations in the footage. Once a potential alarm is detected, the monitoring personnel must make a manual judgment and confirmation before manually inputting commands to control the drone, such as performing actions like approaching for reconnaissance, hovering for observation, issuing verbal warnings to drive it away, taking photos and videos, or tracking.

[0004] While some solutions incorporate artificial intelligence models for basic target detection (such as people, vehicles, and fire detection), most remain at the level of "identification and alarm," failing to enable drones to make automatic decisions and take automatic actions based on the scenario. Therefore, they cannot form a complete intelligent closed loop of "perception → decision → action."

[0005] Therefore, under this model, the existing system faces several core technological bottlenecks. Drone pilots and monitoring personnel need to maintain high concentration for extended periods, constantly monitoring video feeds, which can easily lead to visual fatigue, resulting in missed reports, false alarms, or delayed judgments. This problem is particularly severe in long-duration patrols, nighttime missions, or high-frequency patrol scenarios. Secondly, the entire process chain—from AI or human detection of suspicious situations to human judgment, manual operation, and finally, drone execution—is too long. Delays in issuing actions often lead to missed opportunities for optimal intervention or evidence collection, especially evident in security cases requiring rapid response.

[0006] Furthermore, in city-level or multi-point simultaneous patrol missions, manual operation is insufficient to manage a large number of patrol drones simultaneously. The system's lack of automation limits patrol efficiency and scalability. Existing AI detection models are mostly static, repeatedly issuing alarms for persistently existing targets (such as the same pedestrian or vehicle), generating numerous redundant notifications. Back-end personnel need to spend extra time filtering and identifying these notifications, increasing their workload.

[0007] Furthermore, current AI patrol systems are mostly "alarm-based" rather than "autonomous." They lack capabilities such as automatic tracking, behavior analysis, dynamic decision-making, and task coordination, making it impossible to establish a truly intelligent drone patrol system. Overall, current drone patrol systems remain in a "semi-automatic" stage, failing to meet the demands of policing and public safety for efficient, automated, and large-scale collaborative patrols. To achieve truly intelligent patrols, it is necessary to move beyond the current model that relies solely on manual labor and simple AI detection, and develop a complete intelligent closed-loop technology that integrates perception, understanding, decision-making, and action. Summary of the Invention

[0008] To address the issues of slow response, low efficiency, low intelligence, and redundant alarm information in drone patrols mentioned in the background, this invention aims to provide a cloud-edge collaborative drone emergency response closed-loop system and method.

[0009] This invention provides a cloud-edge collaborative drone-based emergency response system, comprising: multiple drone modules, an edge computing node, and a cloud platform. The multiple drone modules are configured to acquire real-time video streams of a target area and receive a closed-loop control command from the drone, executing automated response actions corresponding to the control command. Furthermore, the edge computing node is communicatively connected to the multiple drone modules and deployed within their coverage area, enabling it to receive the real-time video streams with low latency. The edge computing node includes: a video processing unit, an artificial intelligence analysis unit, an event deduplication unit, an alarm generation unit, and a command sending unit. The video processing unit is configured to receive real-time video streams transmitted by the multiple drone modules. The artificial intelligence analysis unit is communicatively connected to the video processing unit and is configured to perform real-time analysis of the real-time video stream based on a pre-trained artificial intelligence model to identify alarm events in the footage. The event deduplication unit is connected to the artificial intelligence analysis unit and is configured to deduplicate the alarm events based on spatiotemporal features, target identifiers, or continuous detection information to determine whether the alarm event is a new event. The alarm generation unit is connected to the event deduplication unit and, when the alarm event is determined to be a new event, is configured to generate a structured alarm message and the corresponding closed-loop control command. The command sending unit is connected to the alarm generation unit and is configured to transmit the closed-loop control command to at least one drone among the multiple drone modules. Furthermore, the cloud platform is communicatively connected to the edge computing node and is configured to receive and store the structured alarm message; and to display and manage the alarm message.

[0010] In some embodiments, the event deduplication unit is configured to: assign a unique target identifier to the first identified alarm event, and track the target with the unique target identifier in subsequent video frames based on a target tracking algorithm, so as to determine whether the alarm event is a reported event.

[0011] In some embodiments, the target tracking algorithm includes calculating the intersection-over-union (IoU) ratio of the target bounding box or extracting visual features for comparison.

[0012] In some embodiments, the automated handling actions include: controlling the plurality of drone modules to perform precise target tracking, optically zooming the target to fix evidence, and automatically issuing persuasive warnings via onboard speakers.

[0013] In some embodiments, the instruction sending unit is configured to transmit the closed-loop control instructions to the plurality of drone modules based on the MQTT communication protocol.

[0014] In some embodiments, the edge computing node further includes a video annotation unit, configured to: overlay the target bounding box and its category annotation of the alarm event onto the real-time video stream to generate an annotated video stream; and push the annotated video stream to the streaming media server for the cloud platform to access.

[0015] This invention also provides a closed-loop handling method for drone-based emergency incidents based on cloud-edge collaboration, comprising the following steps: Receiving step: An edge computing node receives a real-time video stream from a drone; Perception step: The edge computing node performs artificial intelligence analysis on the real-time video stream to identify emergency incidents in the footage; Decision step: The identified emergency incidents are deduplicated to determine whether they are new incidents; Generation step: When a new incident is identified, the edge computing node generates a structured emergency message and a drone closed-loop control command; Action step: The closed-loop control command is transmitted to the drone, and the drone executes automated handling actions associated with the command; Reporting step: The structured emergency message is transmitted to a cloud platform.

[0016] In some embodiments, the decision-making steps include: assigning a unique target identifier to the first identified alarm event; and tracking the target with the identifier in subsequent video frames based on a target tracking algorithm to determine whether it is a reported event.

[0017] In some embodiments, the tracking algorithm includes IoU calculation or visual feature comparison.

[0018] In some embodiments, the automated handling actions in the action steps include: controlling the drone to perform precise target tracking, optically zooming the target to secure evidence; and automatically issuing persuasive warnings via an onboard loudspeaker.

[0019] In some embodiments, the closed-loop control commands for the UAV are transmitted to the UAV via the MQTT protocol during the action steps.

[0020] In some embodiments, the method further includes: the edge computing node overlays the target bounding box and category annotation information of the alarm event onto the real-time video stream in real time, generating a real-time annotated video stream, and pushing it to the streaming media server for the cloud platform to access.

[0021] The present invention has the following beneficial effects:

[0022] 1. Achieve automated closed-loop control of "perception → decision → action" at the edge, completely changing the traditional model that relies on manual observation and operation in the background, and solving the problems of long links and high latency in the existing technology from discovery to handling; by completing AI analysis and control decisions directly at the edge node close to the data source, perception and action are closely linked, greatly shortening the response time of the alarm and significantly improving patrol efficiency and combat capability.

[0023] 2. This invention introduces a target tracking-based alarm event deduplication mechanism, which effectively eliminates the common problem of duplicate alarms in existing AI alarm solutions. By assigning a unique identifier to each alarm event and continuously tracking it, it ensures that alarms are only triggered for the first occurrence of an event, successfully filtering out a large number of invalid alarms, reducing the workload of the backend and improving the accuracy of alarm identification.

[0024] 3. This invention adopts a cloud-edge collaborative architecture, which fully leverages the low-latency analysis capabilities of the edge and the centralized management capabilities of the cloud. It not only meets the real-time requirements of emergency response but also provides unified storage, display, and scheduling functions, combining real-time performance and manageability, which is superior to deployment modes that rely solely on the cloud or a single terminal.

[0025] 4. This invention can automatically complete the handling and evidence collection operations of drones, including zoom magnification, target tracking, verbal persuasion, and automatic high-definition evidence capture, realizing a functional upgrade from "alarm" to "automated evidence collection", giving drone patrols higher intelligence and practical value. Attached Figure Description

[0026] Embodiments of the present invention will be described with reference to the accompanying drawings, which are briefly described below.

[0027] Figure 1 This is a schematic diagram illustrating the closed-loop emergency response system for drones based on cloud-edge collaboration, as described in Embodiment 1 of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the closed-loop emergency response system for drones based on cloud-edge collaboration, as described in Embodiment 2 of the present invention.

[0029] Figure 3 The flowchart illustrates the closed-loop handling method for drone-based emergency situations based on cloud-edge collaboration in Embodiment 3 of the present invention.

[0030] Figure 4 The flowchart illustrates the closed-loop handling method for drone-based emergency situations based on cloud-edge collaboration in Embodiment 4 of the present invention.

[0031] In the accompanying drawings, the same reference numerals generally denote the same, functionally similar and / or structurally similar elements.

[0032] Figure labeling: Drone module-10; Edge computing node-20; Video processing unit-21; Artificial intelligence analysis unit-22; Event deduplication unit-23; Alarm generation unit-24; Command sending unit-25; Video annotation unit-26; Cloud platform-30. Detailed Implementation

[0033] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate preferred embodiments of the application. However, this application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0035] Embodiments of this application are described herein with reference to cross-sectional views, which are preferred embodiments of this application. Variations in the illustrated shapes due to, for example, manufacturing techniques and / or tolerances are expected. Therefore, embodiments of this application should not be limited to the specific shapes of the regions shown herein, but include shape deviations due to, for example, manufacturing techniques. The regions shown in the figures are substantially schematic, and their shapes do not represent the actual shapes of regions of the device, nor do they limit the scope of embodiments of this application.

[0036] In this article, cloud-edge collaboration refers to a computing model where a cloud platform and edge computing nodes work together. The cloud platform is responsible for global data management and user interaction, while edge computing nodes are deployed close to the data source (such as drones) and are responsible for real-time data processing and rapid response. Closed-loop emergency response refers to the fully automated process from automatically detecting an emergency event through video streams, to automatic system decision-making, automatic control of drones to execute on-site actions, and finally, reporting the emergency information. Edge computing nodes are the core computing units deployed at the mission site or near-field data center, used to receive and process drone video streams in real time. Structured emergency messages refer to data that organizes emergency information into a fixed format (such as JSON), typically including key fields such as the time of the emergency, geographical coordinates, emergency type, and on-site captured images.

[0037] Example 1

[0038] Please see Figure 1 , Figure 1 This diagram illustrates the closed-loop emergency response system for drone-based incidents based on cloud-edge collaboration, as described in this invention.

[0039] like Figure 1 The present invention provides a closed-loop emergency response system for drone-based police incidents based on cloud-edge collaboration, comprising: multiple drone modules 10, edge computing nodes 20, and a cloud platform 30. The multiple drone modules 10 are configured to acquire real-time video streams of the target area, and to receive a closed-loop control command from the drone and execute automated response actions corresponding to the closed-loop control command; wherein the automated response actions include: controlling the multiple drone modules 10 to perform precise target tracking, optically zooming the target to secure evidence, and automatically issuing warnings via onboard loudspeakers.

[0040] Secondly, the edge computing node 20 is communicatively connected to the plurality of drone modules 10 and deployed within the coverage area of ​​the plurality of drone modules 10, enabling it to receive the real-time video stream with low latency. The edge computing node 20 includes: a video processing unit 21, an artificial intelligence analysis unit 22, an event deduplication unit 23, an alarm generation unit 24, and a command sending unit 25; wherein the video processing unit 21 is configured to receive the real-time video stream transmitted by the plurality of drone modules 10. The artificial intelligence analysis unit 22 is communicatively connected to the video processing unit 21 and is configured to perform real-time analysis of the real-time video stream based on a pre-trained artificial intelligence model to identify alarm events in the footage. The artificial intelligence module includes models such as YOLO or Faster R-CNN, but the present invention is not limited to these. The event deduplication unit 23 is connected to the artificial intelligence analysis unit 22 and is configured to perform deduplication processing on the alarm events based on spatiotemporal features, target identifiers, or continuous detection information to determine whether the alarm event is a new event. The event deduplication unit 23 is configured to: assign a unique target identifier to the first identified alarm event, and track the target with the unique target identifier in subsequent video frames based on a target tracking algorithm to determine whether the alarm event is a reported event; wherein the target tracking algorithm includes calculating the intersection-over-union (IoU) ratio of the target bounding box or extracting visual features for comparison. The alarm events may include: gatherings of people, illegally parked vehicles, vehicles with specific license plates, motorcyclists or electric bike riders not wearing helmets, illegal fishing, road obstacles, or banners containing specific text information, etc.

[0041] The alarm generation unit 24 is connected to the event deduplication unit 23. When the alarm event is determined to be a new event, it is configured to generate a structured alarm message and the corresponding closed-loop control command. The command sending unit 25 is connected to the alarm generation unit 24. It is configured to transmit the closed-loop control command to at least one of the plurality of drone modules 10. The command sending unit is configured to transmit the closed-loop control command to the plurality of drone modules 10 based on the MQTT communication protocol.

[0042] In addition, the cloud platform 30 is communicatively connected to the edge computing node 20 and is configured to receive and store the structured alarm messages; and to display and manage the alarm messages.

[0043] Example 2

[0044] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the closed-loop emergency response system for drones based on cloud-edge collaboration, as described in Embodiment 2 of the present invention.

[0045] like Figure 2As shown, the edge computing node 20 also includes a video annotation unit 26, configured to: overlay the target bounding box and its category annotation of the alarm event onto the real-time video stream to generate an annotated video stream; and push the annotated video stream to the streaming media server for the cloud platform 30 to access.

[0046] Example 3

[0047] Please see Figure 3 , Figure 3 The flowchart illustrates the closed-loop handling method for drone-based emergency situations based on cloud-edge collaboration in Embodiment 3 of the present invention.

[0048] like Figure 3 As shown, this invention provides a closed-loop handling method for drone-based emergency situations based on cloud-edge collaboration, which includes the following steps:

[0049] Receiving step S1: The edge computing node receives a real-time video stream from the drone.

[0050] Perception step S2: The edge computing node performs artificial intelligence analysis on the real-time video stream to identify police incidents in the video.

[0051] Decision step S3: Deduplicating the identified alarm events to determine whether they are new events; wherein decision step S3 includes: assigning a unique target identifier to the first identified alarm event; and tracking the target with the identifier in subsequent video frames based on a target tracking algorithm to determine whether it is a reported event. The tracking algorithm includes IoU calculation or visual feature comparison.

[0052] Step S4: When a new event is identified, the edge computing node generates a structured alarm message and a drone closed-loop control command. The structured alarm message includes one or more pieces of information, such as the time of the alarm occurrence, geographical coordinates, alarm type, and high-definition images captured at the scene. The drone closed-loop control command is used to respond to automated handling actions in response to the alarm event. Drone closed-loop control refers to the edge computing node sending closed-loop control commands to the drone via low-latency, high-reliability IoT protocols such as MQTT. After receiving the command, the drone automatically executes the associated handling actions. Automated handling actions may include: controlling the drone to accurately track the target, optically zooming the target to secure evidence, and automatically issuing warnings via onboard speakers.

[0053] Action step S5: The closed-loop control command is transmitted to the drone, and the drone executes automated actions associated with the command; wherein the automated actions in action step S5 include: controlling the drone gimbal to perform precise target tracking, optically zooming the target to secure evidence; and automatically issuing warnings via an onboard speaker. In action step S5, the closed-loop control command is transmitted to the drone via the MQTT protocol.

[0054] Reporting step S6: Transmit the structured alarm message to the cloud platform. The edge computing node 20 reports the generated structured alarm message to the cloud platform or terminal application via MQTT or HTTP POST request for review and further processing by backend personnel.

[0055] Example 4

[0056] Please see Figure 4 , Figure 4 The flowchart illustrates the closed-loop handling method for drone-based emergency situations based on cloud-edge collaboration in Embodiment 4 of the present invention.

[0057] like Figure 4 As shown, the processing method further includes: video annotation step S7: the edge computing node overlays the target bounding box and category annotation information of the alarm event onto the real-time video stream in real time, generating a real-time annotated video stream, and pushes it to the streaming media server for the cloud platform to access. After identifying the alarm event, the edge computing node can overlay the bounding box and category annotation information of the alarm target onto the original video stream in real time, generating a real-time annotated video stream, and push it to the streaming media server for the cloud platform or terminal application to access at any time. This upgrades the processing method from an "alarm system" to an "automated evidence collection system".

[0058] While various embodiments of the invention have been described above, it should be understood that these are presented by way of example only and not as limiting. Therefore, the scope of the invention should not be limited to the above embodiments, but should be defined solely by the claims and their equivalents.

Claims

1. A cloud-edge collaborative-based unmanned aerial vehicle (UAV) emergency response closed-loop handling system, characterized in that, Comprise: a plurality of unmanned aerial vehicle modules configured to collect real-time video streams of a target area, and receive closed-loop control instructions of the unmanned aerial vehicle modules and execute automatic handling actions corresponding to the closed-loop control instructions; an edge computing node in communication connection with the plurality of unmanned aerial vehicle modules, deployed within the coverage range of the plurality of unmanned aerial vehicle modules, so as to be able to receive the real-time video streams in a low-latency manner, the edge computing node comprising: a video processing unit configured to receive the real-time video streams transmitted by the plurality of unmanned aerial vehicle modules; an artificial intelligence analysis unit in communication connection with the video processing unit, configured to perform real-time analysis on the real-time video streams based on a pre-trained artificial intelligence model, to identify police event in the picture; an event deduplication unit connected with the artificial intelligence analysis unit, configured to perform deduplication processing on the police event based on spatiotemporal features, target identification or continuous detection information, to determine whether the police event is a new event; a police event generation unit connected with the event deduplication unit, configured to generate a structured police event message and corresponding closed-loop control instructions when the police event is determined to be a new event; and an instruction sending unit connected with the police event generation unit, configured to transmit the closed-loop control instructions to at least one unmanned aerial vehicle in the plurality of unmanned aerial vehicle modules; and a cloud platform in communication connection with the edge computing node, configured to receive and store the structured police event message, and display and manage the police event message.

2. The treatment system of claim 1, characterized by The event deduplication unit is configured to assign a target unique identifier to a first-identified police event, and track a target having the target unique identifier in subsequent video frames based on a target tracking algorithm, to determine whether the police event is a reported event.

3. The treatment system of claim 2, wherein, The target tracking algorithm includes calculating the intersection over union (IoU) of the target box or extracting visual features for comparison.

4. The treatment system of claim 1, wherein, The automatic handling actions include controlling the plurality of unmanned aerial vehicle modules to perform precise target tracking, optically zooming in on the target to fix evidence, and automatically shouting to persuade through an on-board loudspeaker.

5. The treatment system of claim 1, wherein, The instruction sending unit is configured to transmit the closed-loop control instructions to the plurality of unmanned aerial vehicle modules based on the MQTT communication protocol.

6. The treatment system of claim 1, wherein, The edge computing node further comprises a video annotation unit configured to superimpose a target bounding box and its class label of the police event in the picture of the real-time video stream, to generate an annotated video stream, and push the annotated video stream to a streaming server for retrieval by the cloud platform.

7. A cloud-edge collaboration-based unmanned aerial vehicle alarm loop handling method, characterized in that, The method comprises the following steps: a receiving step: the edge computing node receives real-time video streams from unmanned aerial vehicles; a perception step: the edge computing node performs artificial intelligence analysis on the real-time video streams to identify police events in the picture; a decision step: the identified police events are subjected to deduplication processing to determine whether the police events are new events; a generation step: when determined to be new events, the edge computing node generates a structured police event message and unmanned aerial vehicle closed-loop control instructions; Action step: transmitting the closed-loop control instruction to the UAV, the UAV performing an automated handling action associated with the instruction; and Reporting step: transmitting the structured police case message to the cloud platform.

8. The treatment method according to claim 7, characterized in that, The decision step includes assigning a target unique identifier to the first-identified police case event, and tracking the target with the identifier in subsequent video frames based on a target tracking algorithm to determine whether it is a reported event.

9. The treatment method according to claim 8, characterized in that, The tracking algorithm includes IoU calculation or visual feature comparison.

10. The treatment method of claim 7, wherein, The automated handling action in the action step includes controlling the UAV to perform precise target tracking, optically zooming in on the target to fix evidence, and automatically shouting to persuade through the on-board speaker.

11. The treatment method according to claim 7, characterized by The closed-loop control instruction in the action step is transmitted to the UAV via the MQTT protocol.

12. The treatment method of claim 7, wherein, The handling method further includes: the edge computing node superimposes the target bounding box and label information such as the category of the police case event on the picture of the real-time video stream in real time to generate a real-time labeled video stream, and pushes it to a streaming media server for review by the cloud platform.

Images