Traffic congestion intelligent detection and early warning method and system based on unmanned aerial vehicle aerial video, electronic device and storage medium

Abstract

The application discloses a kind of traffic congestion intelligent detection and early warning method, system based on unmanned aerial vehicle aerial video, electronic equipment and storage medium, it is related to unmanned aerial vehicle application technical field, method includes: S10, obtain the aerial video of the area to be surveyed of unmanned aerial vehicle along the preset route shooting;S20, extract key frame image in aerial video, and obtain the GPS information of key frame image;S30, vehicle in each key frame image is identified and tracked, and the moving direction of vehicle is judged;S40, constructs vehicle chain, judges whether there is congestion event in each key frame image, if there is, then trigger early warning;S50, after triggering early warning, extract all congestion vehicle identification set of existence congestion event, and remove duplicate processing is carried out to congestion vehicle identification set;S60, generate early warning data packet containing congestion event key information, and send to command control platform.The beneficial effects of the present application: improve the efficiency and accuracy of traffic state monitoring.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) application technology, and more specifically, to a method, system, electronic device, and storage medium for intelligent detection and early warning of traffic congestion based on UAV aerial video. Background Technology

[0002] With the continuous growth of highway mileage and the rapid increase in motor vehicle ownership in my country, traffic congestion has become a prominent problem affecting urban operational efficiency and residents' travel experience. Timely and accurate detection and early warning of traffic congestion are prerequisites for implementing effective traffic control and alleviating congestion.

[0003] Traditional traffic congestion detection methods primarily rely on fixed sensor networks deployed along roads, such as geomagnetic coils, microwave radar, and video surveillance cameras. These technologies have significant limitations in practical applications: First, the installation, wiring, and maintenance costs of fixed detection equipment are high, making it difficult to achieve full coverage of the road network and resulting in numerous monitoring blind spots. Second, the detection data typically only reflects local traffic parameters (such as flow rate, speed, and occupancy) at the installation point, failing to capture the spatial spread and length of congestion, as well as the internal structure of queued vehicles, making it difficult to make a comprehensive and accurate assessment of congestion status.

[0004] In recent years, the rapid development and widespread adoption of drone technology has provided new technological approaches for traffic condition monitoring. Drones offer significant advantages such as flexible deployment, high mobility, wide field of view, and broad coverage. Equipped with high-definition camera modules, they can provide aerial observation of traffic flow, effectively avoiding vehicle obstruction issues present by fixed ground-based cameras. Combined with computer vision and deep learning technologies, drone aerial video can automatically detect, track, and perform basic condition analysis of traffic targets (primarily vehicles), providing a new data foundation and technological possibilities for intelligent traffic congestion detection.

[0005] However, most existing drone-based traffic analysis solutions still only involve simple vehicle identification and counting, or rely on pre-defined lane markings and other static elements for vehicle behavior analysis. Their applicability is limited in complex scenarios with unclear lane markings or turbulent traffic flow.

[0006] To this end, the present invention provides a method, system, electronic device and storage medium for intelligent detection and early warning of traffic congestion based on drone aerial video. It can analyze the spatial following relationship and chain structure characteristics between vehicles in real time, promptly detect and determine traffic congestion events, and realize automatic early warning, thereby improving the efficiency and accuracy of traffic condition monitoring. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a method, system, electronic device, and storage medium for intelligent detection and early warning of traffic congestion based on drone aerial video. It can analyze the spatial following relationship and chain structure characteristics between vehicles in real time, promptly detect and determine traffic congestion events, and achieve automatic early warning, thereby improving the efficiency and accuracy of traffic condition monitoring.

[0008] The technical solution adopted by this invention to solve its technical problem is: a method for intelligent detection and early warning of traffic congestion based on drone aerial video, applied to a drone system, wherein the drone system includes at least a drone and a command and control platform, and its improvement lies in including the following steps: S10, acquire aerial video of the area to be surveyed by the drone flying at a constant speed along a preset route and using a vertical overhead shooting method; wherein, the drone is equipped with a high-definition camera for shooting video and a sensor for collecting real-time flight parameters. The sensor will transmit the collected real-time flight parameters to the command and control platform at a frequency of 0.5Hz, that is, transmit the current flight parameters every 2 seconds, and the command and control platform will integrate the received flight parameters into the drone's trajectory file. The flight parameters include at least the drone's GPS information, flight speed, flight attitude, gimbal angle, camera parameters, video timestamp, and battery level; S20: Extract key frame images from aerial video and obtain GPS information for the key frame images; S30: Identify and track vehicles in each keyframe image and determine the direction of vehicle movement; S40: Construct a vehicle chain based on vehicles with a following relationship in each key frame image. Determine whether there is a congestion event in each key frame image by comparing the parameters of the vehicle chain with a preset threshold. If there is, trigger an early warning. S50, after triggering the warning, extracts the tracking identifiers of all vehicles in the vehicle chain with congestion events to form a congested vehicle identifier set, and performs deduplication on the congested vehicle identifier set; S60 generates a warning data packet containing key information about the congestion event based on the deduplicated set of congested vehicle identifiers and sends it to the command and control platform.

[0009] Furthermore, in step S20, the specific method for extracting keyframe images from the aerial video and obtaining the GPS information of the keyframe images is as follows: S201, based on the drone's flight speed, uniformly extracts frames from aerial video at fixed time intervals to obtain keyframe images; S202, calculate the approximate GPS information of the keyframe based on the fixed-frequency data of the keyframe image. The specific calculation method is as follows: Let the frame number of the currently extracted keyframe in the aerial video be K, and the corresponding video timestamp be... The two times before and after this keyframe when the fixed-frequency data was recorded are: and , at all times and The corresponding recorded GPS information are as follows: and ; Keyframes at Moment The calculation expression is: longitude: ; latitude: ; high: ; in, .

[0010] Furthermore, in step S30, the specific method for identifying and tracking vehicles in each keyframe image and determining the direction of vehicle movement is as follows: S301 uses the YOLO26 detection model to perform forward inference on each keyframe image, identify vehicle targets contained in the keyframe image, and output the bounding box of each vehicle target. S302 uses an online multi-target tracking algorithm to associate data of the same vehicle target appearing in different keyframe images and assigns a unique and unchanging tracking identifier to each vehicle target. S303, extract the geometric center point pixels of the bounding rectangle of each vehicle target, and use the GPS information of the keyframe image corresponding to each vehicle target and the flight attitude and camera parameters of the UAV corresponding to the keyframe image to obtain the latitude and longitude data of each vehicle target through coordinate transformation. S304 determines the absolute geographic direction of movement of each vehicle target by measuring its geographic displacement in consecutive keyframe images, and converts the absolute geographic direction of movement into the direction of movement relative to the current monitoring screen by combining the real-time yaw angle of the camera.

[0011] Furthermore, in step S40, the construction of a vehicle chain based on vehicles with a following relationship in each keyframe image, and the determination of whether a congestion event exists in each keyframe image by comparing the parameters of the vehicle chain with a preset threshold, and if so, the specific method for triggering an early warning is as follows: S401, For each vehicle target with a determined direction of motion, construct a rectangular search neighborhood at the rear of the vehicle based on its direction of motion; S402, within the rectangular search neighborhood, the following relationship between vehicle targets is determined by calculating the Euclidean distance between the center points of each vehicle target bounding box, and a directed graph is constructed based on all following relationships, with all maximal paths in it serving as vehicle following chains; S403, determine whether the vehicle target in the vehicle following chain has reached a preset first threshold. If so, calculate the vertical distance from the bounding box of the tail of the vehicle following chain to the screen boundary in the opposite direction of its movement. and the arithmetic mean of the longer sides of the target bounding boxes of all vehicles within the vehicle following chain. And determine whether the vehicle following chain satisfies the inequality: ,in, This indicates the preset second threshold; If so, the chain is determined to be a congested chain, and the number of congested chains in the current video frame scene is counted. It is then determined whether the counted number of congested chains reaches or exceeds a preset third threshold. If so, a traffic congestion event is determined to have occurred in the current video frame scene, and the corresponding early warning recording and alarm output mechanism is triggered.

[0012] Furthermore, in step S50, the specific method for deduplicating the set of congested vehicle identifiers is as follows: Maintain a historical alarm record queue with a fixed maximum capacity. , Each element in the set is a collection of congested vehicle identifiers corresponding to the historical successful alarm times; Calculate the intersection of the current set of congested vehicle identifiers and every historical set in the historical alarm record queue. If there exists any historical set such that the cardinality of the intersection is greater than zero, then the condition is satisfied. If the current frame's congested vehicle combination overlaps with previously alerted historical congestion events, it is considered a continuation of the same congestion event, and no new alert is triggered. In the above formula, This represents any historical set in the historical alarm record queue. This represents the set of congested vehicle identifiers for the current frame; conversely, if... If the intersection of the current frame and all historical alarm records in the queue is empty, then the current frame is considered to have detected a new congestion event completely unrelated to previous alarms. In this case, a traffic congestion warning record is generated. It is added as a new element to the tail of the historical queue, and the queue length is updated synchronously to maintain the capacity constraint.

[0013] Furthermore, in step S60, the early warning data packet includes the current keyframe image, the video timestamp of the moment the congestion occurred, the corresponding geographical coordinates, and a list of vehicle identifiers involved in the congestion.

[0014] A traffic congestion intelligent detection and early warning system based on drone aerial video, used in the aforementioned traffic congestion intelligent detection and early warning method based on drone aerial video, is improved by including: The data acquisition module is used to acquire aerial videos of the area to be surveyed by a drone flying at a constant speed along a preset route and taking pictures from above in a vertical overhead position. The keyframe and spatiotemporal data processing module is used to extract keyframe images from aerial videos and obtain GPS information from the keyframe images. The vehicle detection, tracking, and motion analysis module is used to identify and track vehicles in each keyframe image and determine the direction of vehicle movement. The chain relationship construction and congestion determination module is used to construct a vehicle chain based on vehicles with a front-to-back following relationship in each key frame image. The module determines whether there is a congestion event in each key frame image by comparing the parameters of the vehicle chain with a preset threshold. If a congestion event is found, an early warning is triggered. The early warning deduplication module is used to extract the tracking identifiers of all vehicles in the vehicle chain with congestion events after an early warning is triggered, to form a congested vehicle identifier set, and to deduplicate the congested vehicle identifier set. The early warning generation and visualization output module is used to generate early warning data packets containing key information about congestion events based on the deduplicated set of congested vehicle identifiers, and send them to the command and control platform.

[0015] An electronic device, improved in that it includes at least one processor and at least one memory, wherein, The memory stores computer-readable instructions; The computer-readable instructions are executed by one or more processors, enabling the electronic device to implement the intelligent traffic congestion detection and early warning method based on drone aerial video as described above.

[0016] An improvement of a storage medium storing computer-readable instructions is that the computer-readable instructions are executed by one or more processors to implement the intelligent traffic congestion detection and early warning method based on drone aerial video as described above.

[0017] The beneficial effects of this invention are as follows: First, this invention acquires aerial video of the entire field of view through vertical overhead shooting and extracts key frames, providing a data foundation for real-time analysis. Next, it identifies, tracks, and determines the precise direction of vehicle movement, establishing a dynamic benchmark for analyzing the spatial relationships between vehicles. Then, based on vehicles with a following relationship in each key frame image, it constructs a vehicle chain, directly realizing real-time analysis of the spatial following relationship and chain structure characteristics between vehicles. Furthermore, by comparing the parameters of the vehicle chain with preset thresholds, it determines whether there are congestion events in each key frame image and triggers an early warning, thereby promptly detecting and determining traffic congestion events. Finally, through deduplication and automatic packaging and sending of early warning information, it avoids repeated alarms for the same event and automatically pushes the conclusions to the command and control platform, fully realizing the automation of the entire process from feature analysis and event determination to early warning output. Therefore, this invention can analyze the spatial following relationship and chain structure characteristics between vehicles in real time, promptly detect and determine traffic congestion events, and achieve automatic early warning, thereby improving the efficiency and accuracy of traffic condition monitoring. Attached Figure Description

[0018] Figure 1 This is an overall flowchart of a traffic congestion intelligent detection and early warning method based on drone aerial video according to the present invention; Figure 2 This is a block diagram of a traffic congestion intelligent detection and early warning method based on drone aerial video according to the present invention; Figure 3 This is a hardware structure diagram of an electronic device as an example embodiment; Figure 4 This is a block diagram illustrating an electronic device as an example embodiment. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

[0021] This invention discloses a method for intelligent detection and early warning of traffic congestion based on drone aerial video, applied to a drone system, which includes at least a drone and a command and control platform, and includes the following steps: S10: Acquire aerial video of the area to be surveyed by a drone flying at a constant speed along a preset route and using a vertical overhead shooting method; wherein, the drone is equipped with a high-definition camera for shooting video and a sensor for collecting real-time flight parameters. The sensor transmits the collected real-time flight parameters to the command and control platform at a frequency of 0.5Hz, that is, it transmits the current flight parameters every 2 seconds, and the command and control platform integrates the received flight parameters into a trajectory file of the drone. The flight parameters include at least the drone's GPS information, flight speed, flight attitude, gimbal angle, camera parameters, video timestamp, and battery level.

[0022] It should be noted that in this embodiment, the operator remotely controls a drone equipped with a high-definition camera and various sensors (such as GPS and inertial measurement units) to fly at a constant speed along a pre-planned route, keeping the gimbal vertically downward to collect aerial video of the area to be inspected. Simultaneously, the drone's sensors transmit real-time flight parameters (including but not limited to GPS position, flight speed, flight attitude, gimbal angle, camera parameters, video timestamps, and battery level) to the command and control platform at a fixed frequency of 0.5Hz (once every 2 seconds). After receiving this data, the command and control platform integrates and records it into a structured drone trajectory file. This file, along with the aerial video, serves as the basic data source for subsequent processing, providing accurate and synchronized spatiotemporal reference data for subsequent traffic analysis.

[0023] S20, extract keyframe images from the aerial video and obtain the GPS information of the keyframe images; specifically, the method for extracting keyframe images from the aerial video and obtaining the GPS information of the keyframe images is as follows: S201: Based on the drone's flight speed, frames are uniformly extracted from the aerial video at fixed time intervals to obtain keyframe images. During the frame extraction process, a small overlap between adjacent keyframe images is allowed. For example, if the drone completes the entire frame in 5 seconds, the sampling time interval can be set to 4.5 seconds. S202, calculate the approximate GPS information of the keyframe based on the fixed-frequency data of the keyframe image. It should be noted that since the fixed-frequency data in the trajectory file is recorded only once every 2 seconds, this means that the GPS information is also recorded every 2 seconds. The frame rate of the video to be detected is fps (fps is usually 30). There are 2 fps frames in a 2-second time period. Therefore, the probability of directly obtaining the GPS information of the currently extracted keyframe is only 1 / 2 fps; in most cases, it is missing. It is necessary to calculate the approximate GPS information of the keyframe. The specific calculation method is as follows: Let the frame number of the currently extracted keyframe in the aerial video be K, and the corresponding video timestamp be... The two times before and after this keyframe when the fixed-frequency data was recorded are: and , at all times and The corresponding recorded GPS information are as follows: and ;because and Two seconds apart, and considering that the drone maintains a constant speed and does not make any sudden, large movements during the patrol, the keyframes are set at... Moment It can be obtained through the following calculation expression: longitude: ; latitude: ; high: ; in, ; It should be noted that in this embodiment, since the frequency at which the drone records flight parameters such as GPS (e.g., 0.5Hz) is much lower than the video frame rate (usually 25-30Hz), most video frames do not have direct GPS data. This embodiment uses linear interpolation on the known, discrete GPS recording points along the time axis to assign an estimated geographic coordinate corresponding to the shooting time to each key frame. This establishes an accurate spatiotemporal reference for subsequent processing, enabling the system to associate the pixel positions in the video image with the real geographic coordinates. This is an essential prerequisite for accurately calculating the vehicle's geographic coordinates, direction of movement, and ultimately locating the congestion event.

[0024] S30, identify and track vehicles in each keyframe image, and determine the direction of vehicle movement; specifically, the method for identifying and tracking vehicles in each keyframe image and determining the direction of vehicle movement is as follows: S301 uses the YOLO26 detection model to perform forward inference on each keyframe image, identifies vehicle targets contained in the keyframe images, and outputs the bounding rectangle bounding box of each vehicle target; and records the pixel coordinates of the bounding rectangle bounding box (e.g., represented as...). ); S302 uses an online multi-target tracking algorithm to associate data of the same vehicle target appearing in different keyframe images and assigns a unique and unchanging tracking identifier to each vehicle target. It should be noted that, in this embodiment, the online multi-target tracking algorithm refers to the commonly used target tracking algorithm SORT. It maintains the identity of vehicle targets identified between consecutive frames, assigning an ID to each vehicle target to distinguish different vehicle targets. For example, in intersection monitoring, if a vehicle is briefly obscured and then reappears, SORT can maintain its ID without assigning a new ID.

[0025] S303, extract the geometric center point pixels of the bounding rectangle of each vehicle target, and use the GPS information of the keyframe image corresponding to each vehicle target and the flight attitude and camera parameters of the UAV corresponding to the keyframe image to obtain the latitude and longitude data of each vehicle target through coordinate transformation. It should be noted that, in this embodiment, the geometric center pixel of the circumscribed rectangular bounding box is calculated using the pixel coordinates of the circumscribed rectangular bounding box. For example, the geometric center pixel of the circumscribed rectangular bounding box of a vehicle target is represented as follows: ,in, , By taking the center point of the keyframe image where the vehicle target is located as the origin, the calculation is performed. The pixel offset relative to the origin, combined with the known ground resolution, is converted into a true horizontal displacement distance. The converted horizontal displacement distance is then compensated using the yaw, pitch, and roll angles transmitted in real time by the UAV flight control system to eliminate the viewing angle deviation caused by changes in the UAV's nose direction. Finally, using the GPS coordinates of the current video frame as the starting point, the system uses a geodesic algorithm to accurately solve the problem on the WGS-84 ellipsoid. Based on the compensated displacement direction and distance, the system directly calculates the latitude and longitude coordinates of the vehicle target, which are then used as the ground projection position of the vehicle target at the current moment, thus completing the mapping from two-dimensional pixels to a three-dimensional Earth.

[0026] S304 determines the absolute geographic direction of movement of each vehicle target by measuring its geographic displacement in consecutive keyframe images, and converts the absolute geographic direction of movement into the direction of movement relative to the current monitoring screen by combining the real-time yaw angle of the camera.

[0027] It should be noted that, in this embodiment, for any tracking identifier, a sequence of vehicle geographic coordinates obtained in step S303 is maintained over N consecutive keyframes. Calculate the geographic displacement vector from the first frame position to the last frame position in this sequence: , Based on displacement components and The sign and relative size of the vehicle's absolute geographical movement during the observation period are used to classify it into one of four directions: east, south, west, or north. The specific criterion is: comparison. and The magnitude of the component is taken, and the direction corresponding to the component with the larger absolute value is taken as the main motion axis. The specific direction along the axis is determined by the positive or negative sign of the component (positive value is east or north, negative value is west or south). Then, the absolute geographic direction is mapped to the current image field of view using the real-time yaw angle of the camera to obtain the four motion directions of the image relative to the image coordinate system: up, down, left, and right. These directions serve as the spatial reference for subsequent rear neighborhood search and vehicle chain construction. S40, a vehicle chain is constructed based on vehicles with a following relationship in each keyframe image. The presence of congestion events in each keyframe image is determined by comparing the parameters of the vehicle chain with a preset threshold. If a congestion event exists, an early warning is triggered. Specifically, the method for constructing a vehicle chain based on vehicles with a following relationship in each keyframe image and determining whether a congestion event exists in each keyframe image by comparing the parameters of the vehicle chain with a preset threshold, and triggering an early warning if a congestion event exists, is as follows: S401, for each vehicle target with a determined direction of motion, a rectangular search neighborhood is constructed at the rear of the vehicle based on its direction of motion; unlike the existing technology that relies on lane lines to determine vehicle following, this invention does not require any ground markings or lines, but only constructs an adaptive dynamic search area behind the vehicle based on the vehicle's own direction of motion vector from the UAV's top-down view. S402, within the rectangular search neighborhood, the following relationship between vehicle targets is determined by calculating the Euclidean distance between the center points of each vehicle target bounding box, and a directed graph is constructed based on all following relationships, with all maximal paths in it serving as vehicle following chains; It should be noted that, in this embodiment, for any vehicle target with a determined direction of movement, it is necessary to search the neighborhood behind the vehicle target, traverse all other tracked vehicle targets in the current frame except for the current vehicle, calculate the Euclidean distance between the center point of the current vehicle's bounding box and the center points of each candidate vehicle's bounding box, select the candidate vehicle with the smallest distance and whose center point falls within the search area, and determine it as the following vehicle behind the current vehicle. Thus, a one-way following relationship from the front vehicle to the rear vehicle is established. Based on all such following relationships in the current frame, a directed graph is constructed, and all maximal paths are identified as vehicle following chains. Isolated nodes containing only a single vehicle are discarded. For each vehicle following chain, the screen movement direction of each vehicle target within the vehicle following chain is counted, and the direction with the highest frequency of occurrence is taken as the overall movement direction of the chain.

[0028] S403, determine whether the vehicle target in the vehicle following chain has reached a preset first threshold. If so, calculate the vertical distance from the bounding box of the tail of the vehicle following chain to the screen boundary in the opposite direction of its movement. and the arithmetic mean of the longer sides of the target bounding boxes of all vehicles within the vehicle following chain. Determine whether the vehicle following chain satisfies the inequality: ,in, This indicates the preset second threshold; If so, the chain is determined to be a congested chain, and the number of congested chains in the current video frame scene is counted. It is then determined whether the counted number of congested chains reaches or exceeds a preset third threshold. If so, a traffic congestion event is determined to have occurred in the current video frame scene, and the corresponding early warning recording and alarm output mechanism is triggered.

[0029] S50, after triggering the warning, extract the tracking identifiers of all vehicles in the vehicle chain with congestion events to form a congested vehicle identifier set, and perform deduplication processing on the congested vehicle identifier set; wherein, the specific method of performing deduplication processing on the congested vehicle identifier set is as follows: Maintain a historical alarm record queue with a fixed maximum capacity. , Each element in the set is a collection of congested vehicle identifiers corresponding to the historical successful alarm times; Calculate the intersection of the current set of congested vehicle identifiers and every historical set in the historical alarm record queue. If there exists any historical set such that the cardinality of the intersection is greater than zero, then the condition is satisfied. If the current frame's congested vehicle combination overlaps with previously alerted historical congestion events, it is considered a continuation of the same congestion event, and no new alert is triggered. In the above formula, This represents any historical set in the historical alarm record queue. This represents the set of congested vehicle identifiers for the current frame; conversely, if... If the intersection of the current frame and all historical alarm records in the queue is empty, then the current frame is considered to have detected a new congestion event completely unrelated to previous alarms. In this case, a traffic congestion warning record is generated. It is added as a new element to the tail of the historical queue, and the queue length is updated synchronously to maintain the capacity constraint.

[0030] It should be noted that in this embodiment, when the system determines that there is a congestion event requiring warning in the current video frame based on preset conditions, the following operation process will be performed to achieve intelligent alarm deduplication: First, the tracking identifiers of all vehicle targets identified as part of a "valid congestion chain" are extracted and merged into a single set of vehicle identifiers representing the current congestion event. The system maintains a historical alarm record queue in memory with a fixed maximum length (e.g., capable of storing the 10 most recent alarm records). Before deciding to send a new alert, the system will iterate through this historical queue and select the current set. With each historical alarm set stored in the queue Compare the two sets and calculate their intersection. If any historical set exists... Make the intersection non-empty (i.e. If the current congestion event is determined to be a continuation of the event represented by this historical record, the system will suppress the alarm to avoid duplicate output; otherwise, if the current set If the intersection of this event with all sets in the historical queue is empty, it is determined to be a completely new and independent congestion event. In this case, the system will execute an alarm output: generate an early warning record containing information such as timestamp, geographical location, and a list of congested vehicle IDs, and push it to the command and control platform; simultaneously, the current set... Added as a new element to the history queue ( At the end of the queue, if the queue length exceeds the preset maximum capacity after adding a new element, the oldest record at the head of the queue will be automatically removed to maintain a fixed queue size. In terms of visualization, the system will highlight the congested vehicles in the current video frame that triggers the new alarm and overlay the warning text in a prominent position on the screen so that monitoring personnel can intuitively grasp the situation.

[0031] S60, based on the deduplicated set of congested vehicle identifiers, a warning data packet containing key information about the congestion event is generated and sent to the command and control platform; wherein, the warning data packet includes the current key frame image, the video timestamp of the moment the congestion occurred, the corresponding geographical coordinates, and a list of vehicle identifiers involved in the congestion.

[0032] It should be noted that in this embodiment, once the system confirms a new congestion event and completes the deduplication of warnings, it will immediately encapsulate and push the warning information. Specifically, the system first performs post-processing on the current keyframe image that triggered the warning, highlighting the bounding boxes of all congested vehicle chains using a predefined color (such as red), and overlaying the text "Traffic Congestion Warning" and the number of chains and vehicles currently involved in the congestion on top of the image. This allows monitoring personnel to intuitively identify the location and scale of the congestion event. Simultaneously, the system collects and encapsulates the following core data: the current keyframe image (overlaid with highlighted congestion chain bounding boxes and warning text), the video timestamp of the congestion occurrence, the corresponding geographical coordinates, and a list of vehicle identifiers involved in the congestion. Subsequently, this data is organized into a structured JSON object and sent in real-time to the pre-set API interface of the command and control platform via network protocols (such as HTTP POST or MQTT), thus completing a full warning report for traffic management personnel to view and process in real-time on the platform terminal.

[0033] Reference Figure 2 As shown, this invention also discloses a traffic congestion intelligent detection and early warning system based on drone aerial video, used in the traffic congestion intelligent detection and early warning method based on drone aerial video as described in the above embodiments, including: Data acquisition module 601 is used to acquire aerial video of the area to be surveyed by a drone flying at a constant speed along a preset route and taking pictures from a vertical top view. The keyframe and spatiotemporal data processing module 602 is used to extract keyframe images from aerial videos and obtain GPS information of the keyframe images. The vehicle detection, tracking and motion analysis module 603 is used to identify and track vehicles in each keyframe image and determine the direction of vehicle movement. The chain relationship construction and congestion determination module 604 is used to construct a vehicle chain based on vehicles with a front-to-back following relationship in each key frame image. It determines whether there is a congestion event in each key frame image by comparing the parameters of the vehicle chain with a preset threshold. If there is a congestion event, an early warning is triggered. The early warning deduplication module 605 is used to extract the tracking identifiers of all vehicles in the vehicle chain with congestion events after an early warning is triggered, so as to form a congested vehicle identifier set, and to perform deduplication processing on the congested vehicle identifier set. The early warning generation and visualization output module 606 is used to generate an early warning data packet containing key information about the congestion event based on the deduplicated set of congested vehicle identifiers, and send it to the command and control platform.

[0034] It should be noted that the traffic congestion intelligent detection and early warning strategy based on UAV aerial video provided in the above embodiments is only an example of the above functional module division. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the traffic congestion intelligent detection and early warning system 600 based on UAV aerial video will be divided into different functional modules to complete all or part of the functions described above. Furthermore, the traffic congestion intelligent detection and early warning system 600 based on UAV aerial video provided in the above embodiments and the traffic congestion intelligent detection and early warning method based on UAV aerial video belong to the same concept. The specific way each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0035] Figure 3 A schematic diagram of the structure of an electronic device according to an exemplary embodiment is shown.

[0036] It should be noted that this electronic device is merely an example adapted to the present invention and should not be construed as providing any limitation on the scope of use of the present invention. Furthermore, this electronic device should not be interpreted as requiring or depending on having... Figure 3 One or more components of the exemplary electronic device 2000 shown.

[0037] The hardware structure of electronic devices 2000 can vary significantly due to differences in configuration or performance, such as... Figure 3 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0038] Specifically, power supply 210 is used to provide operating voltage for various hardware devices on electronic device 2000.

[0039] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Of course, in other examples adapted to this invention, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 3 As shown, this does not constitute a specific limitation.

[0040] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0041] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0042] Application 253 is a computer-readable instruction based on operating system 251 that performs at least one specific task, and may include at least one module ( Figure 3 (Not shown), each module can contain computer-readable instructions for electronic device 2000. For example, a traffic congestion intelligent detection and early warning device based on drone aerial video can be considered as application 253 deployed on electronic device 2000.

[0043] Data 255 may be signal information, etc., and is stored in memory 250.

[0044] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer-readable instructions stored in the memory 250, thereby enabling the computation and processing of massive amounts of data 255 in the memory 250. For example, a method for intelligent detection and early warning of traffic congestion based on UAV aerial video can be implemented by reading a series of computer-readable instructions stored in the memory 250 through the central processing unit 270.

[0045] Furthermore, the present invention can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of the present invention is not limited to any specific hardware circuit, software, or combination thereof.

[0046] Please see Figure 4 This invention provides an electronic device 4000, which may include: a desktop computer, a laptop computer, a server, etc., with sensor recognition capabilities.

[0047] exist Figure 4 In this context, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0048] The data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0049] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0050] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0051] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 4000, but not limited thereto.

[0052] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002.

[0053] The computer-readable instructions are executed by one or more processors 4001 to implement the intelligent traffic congestion detection and early warning method based on UAV aerial video in the above embodiments.

[0054] Furthermore, this embodiment of the invention provides a storage medium storing computer-readable instructions, which are executed by one or more processors to realize the intelligent traffic congestion detection and early warning method based on UAV aerial video as described above.

[0055] Compared with existing technologies, the beneficial effects of this invention are as follows: First, by acquiring aerial video of the entire field of view through vertical overhead shooting and extracting key frames, a data foundation for real-time analysis is provided. Next, vehicles are identified, tracked, and their precise direction of movement is determined, establishing a dynamic benchmark for analyzing the spatial relationships between vehicles. Then, based on vehicles with a following relationship in each key frame image, a vehicle chain is constructed, directly realizing real-time analysis of the spatial following relationship and chain structure characteristics between vehicles. Furthermore, by comparing the parameters of the vehicle chain with preset thresholds, it is determined whether there is a congestion event in each key frame image and an early warning is triggered, thereby timely detection and determination of traffic congestion events. Finally, by deduplication and automatic packaging and sending of early warning information, duplicate alarms for the same event are avoided, and the conclusions are automatically pushed to the command and control platform. This fully realizes the automation of the entire process from feature analysis, event determination to early warning output. Therefore, it can analyze the spatial following relationship and chain structure characteristics between vehicles in real time, timely detect and determine traffic congestion events, and realize automatic early warning, thereby improving the efficiency and accuracy of traffic condition monitoring.

[0056] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for intelligent detection and early warning of traffic congestion based on drone aerial video, applied to a drone system, wherein the drone system includes at least a drone and a command and control platform, characterized in that, Includes the following steps: S10, acquire aerial video of the area to be surveyed by the drone flying at a constant speed along a preset route and using a vertical overhead shooting method; wherein, the drone is equipped with a high-definition camera for shooting video and a sensor for collecting real-time flight parameters. The sensor will transmit the collected real-time flight parameters to the command and control platform at a frequency of 0.5Hz, that is, transmit the current flight parameters every 2 seconds, and the command and control platform will integrate the received flight parameters into the drone's trajectory file. The flight parameters include at least the drone's GPS information, flight speed, flight attitude, gimbal angle, camera parameters, video timestamp, and battery level; S20: Extract key frame images from aerial video and obtain GPS information for the key frame images; S30: Identify and track vehicles in each keyframe image and determine the direction of vehicle movement; S40: Construct a vehicle chain based on vehicles with a following relationship in each key frame image. Determine whether there is a congestion event in each key frame image by comparing the parameters of the vehicle chain with a preset threshold. If there is, trigger an alert. S50, after triggering the warning, extracts the tracking identifiers of all vehicles in the vehicle chain with congestion events to form a congested vehicle identifier set, and performs deduplication on the congested vehicle identifier set; S60 generates a warning data packet containing key information about the congestion event based on the deduplicated set of congested vehicle identifiers and sends it to the command and control platform.

2. The intelligent traffic congestion detection and early warning method based on UAV aerial video as described in claim 1, characterized in that, In step S20, the specific method for extracting keyframe images from the aerial video and obtaining the GPS information of the keyframe images is as follows: S201, based on the drone's flight speed, uniformly extracts frames from aerial video at fixed time intervals to obtain keyframe images; S202, calculate the approximate GPS information of the keyframe based on the fixed-frequency data of the keyframe image. The specific calculation method is as follows: Let the frame number of the currently extracted keyframe in the aerial video be K, and the corresponding video timestamp be... The two times before and after this keyframe when the fixed-frequency data was recorded are: and , at all times and The corresponding recorded GPS information are as follows: and ; Keyframes at Moment The calculation expression is: longitude: ; latitude: ; high: ; in, .

3. The intelligent traffic congestion detection and early warning method based on UAV aerial video as described in claim 2, characterized in that, In step S30, the specific method for identifying and tracking vehicles in each keyframe image and determining the direction of vehicle movement is as follows: S301 uses the YOLO26 detection model to perform forward inference on each keyframe image, identify vehicle targets contained in the keyframe image, and output the bounding box of each vehicle target. S302 uses an online multi-target tracking algorithm to associate data of the same vehicle target appearing in different keyframe images and assigns a unique and unchanging tracking identifier to each vehicle target. S303, extract the geometric center point pixels of the bounding rectangle of each vehicle target, and use the GPS information of the keyframe image corresponding to each vehicle target and the flight attitude and camera parameters of the UAV corresponding to the keyframe image to obtain the latitude and longitude data of each vehicle target through coordinate transformation. S304 determines the absolute geographic direction of movement of each vehicle target by measuring its geographic displacement in consecutive keyframe images, and converts the absolute geographic direction of movement into the direction of movement relative to the current monitoring screen by combining the real-time yaw angle of the camera.

4. The intelligent traffic congestion detection and early warning method based on UAV aerial video as described in claim 3, characterized in that, In step S40, a vehicle chain is constructed based on vehicles with a following relationship in each keyframe image. The presence of a congestion event in each keyframe image is determined by comparing the parameters of the vehicle chain with a preset threshold. If a congestion event is detected, an early warning is triggered in the following manner: S401, For each vehicle target with a determined direction of motion, construct a rectangular search neighborhood at the rear of the vehicle based on its direction of motion; S402, within the rectangular search neighborhood, the following relationship between vehicle targets is determined by calculating the Euclidean distance between the center points of each vehicle target bounding box, and a directed graph is constructed based on all following relationships, with all maximal paths in it serving as vehicle following chains; S403, determine whether the vehicle target in the vehicle following chain has reached a preset first threshold. If so, calculate the vertical distance from the bounding box of the tail of the vehicle following chain to the screen boundary in the opposite direction of its movement. and the arithmetic mean of the longer sides of the target bounding boxes of all vehicles within the vehicle following chain. And determine whether the vehicle following chain satisfies the inequality: ,in, This indicates the preset second threshold; If so, the chain is determined to be a congested chain, and the number of congested chains in the current video frame scene is counted. It is then determined whether the counted number of congested chains reaches or exceeds a preset third threshold. If so, a traffic congestion event is determined to have occurred in the current video frame scene, and the corresponding early warning recording and alarm output mechanism is triggered.

5. The intelligent traffic congestion detection and early warning method based on UAV aerial video as described in claim 4, characterized in that, In step S50, the specific method for deduplicating the set of congested vehicle identifiers is as follows: Maintain a historical alarm record queue with a fixed maximum capacity. , Each element in the set is a collection of congested vehicle identifiers corresponding to the historical successful alarm times; Calculate the intersection of the current set of congested vehicle identifiers and every historical set in the historical alarm record queue. If there exists any historical set such that the cardinality of the intersection is greater than zero, then the condition is satisfied. If the current frame's congested vehicle combination overlaps with previously alerted historical congestion events, it is considered a continuation of the same congestion event, and no new alert is triggered. In the above formula, This represents any historical set in the historical alarm record queue. This represents the set of congested vehicle identifiers for the current frame; conversely, if... If the intersection of the current frame and all historical alarm records in the queue is empty, then the current frame is considered to have detected a new congestion event completely unrelated to previous alarms. In this case, a traffic congestion warning record is generated. It is added as a new element to the tail of the historical queue, and the queue length is updated synchronously to maintain the capacity constraint.

6. The intelligent traffic congestion detection and early warning method based on UAV aerial video as described in claim 5, characterized in that, In step S60, the early warning data packet includes the current key frame image, the video timestamp of the moment the congestion occurred, the corresponding geographical coordinates, and a list of vehicle identifiers involved in the congestion.

7. A traffic congestion intelligent detection and early warning system based on drone aerial video, used to implement the traffic congestion intelligent detection and early warning method based on drone aerial video as described in any one of claims 1-6, characterized in that, include: The data acquisition module is used to acquire aerial videos of the area to be surveyed by a drone flying at a constant speed along a preset route and taking pictures from above in a vertical overhead position. The keyframe and spatiotemporal data processing module is used to extract keyframe images from aerial videos and obtain GPS information from the keyframe images. The vehicle detection, tracking, and motion analysis module is used to identify and track vehicles in each keyframe image and determine the direction of vehicle movement. The chain relationship construction and congestion determination module is used to construct a vehicle chain based on vehicles with a front-to-back following relationship in each key frame image. The module determines whether there is a congestion event in each key frame image by comparing the parameters of the vehicle chain with a preset threshold. If a congestion event is found, an early warning is triggered. The early warning deduplication module is used to extract the tracking identifiers of all vehicles in the vehicle chain with congestion events after an early warning is triggered, to form a congested vehicle identifier set, and to deduplicate the congested vehicle identifier set. The early warning generation and visualization output module is used to generate early warning data packets containing key information about congestion events based on the deduplicated set of congested vehicle identifiers, and send them to the command and control platform.

8. An electronic device, characterized in that, Includes at least one processor and at least one memory, wherein, The memory stores computer-readable instructions; The computer-readable instructions are executed by one or more processors, enabling the electronic device to implement the intelligent traffic congestion detection and early warning method based on drone aerial video as described in any one of claims 1-6.

9. A storage medium having computer-readable instructions stored thereon, characterized in that, The computer-readable instructions are executed by one or more processors to implement the intelligent traffic congestion detection and early warning method based on drone aerial video as described in any one of claims 1-6.

Images