Unmanned aerial vehicle video region-of-interest automatic identification and management system in road traffic scene

Abstract

The invention discloses an unmanned aerial vehicle video region-of-interest automatic identification and management system in a road traffic scene, and the system is characterized in that a road coordinate conversion and data management module is used for obtaining image data of a target road in real time through an unmanned aerial vehicle, converting image coordinates in the image data of the target road into world coordinates, and transmitting the world coordinates to an unmanned aerial vehicle; the geometric feature information of the target road is extracted, stored and dynamically updated; the unmanned aerial vehicle position and attitude data management module is used for collecting geographic position data and attitude data of the unmanned aerial vehicle in real time; and the region-of-interest management module is used for determining a region of interest of the target road according to the geometric feature information of the target road and the geographic position data and attitude data of the unmanned aerial vehicle, and performing real-time monitoring and task event detection on the region of interest. The target road scene is automatically analyzed, the interest area is extracted, traffic target detection, event identification and other business application management are carried out on the interest area, the system resource overhead is reduced, and the efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of highway traffic management, and in particular to an automatic identification and management system for interest areas of unmanned aerial vehicle videos in road traffic scenes. Background Art

[0002] In the field of highway traffic management, timely identification and handling of various traffic incidents, such as illegally parked vehicles, spilled objects, and fog, are crucial to improving road safety and operating efficiency. Traditional methods for collecting road events mainly include fixed detectors and manual inspections, which have many limitations in practical applications. Fixed detectors mainly use cameras or ground sensors, which can only cover a limited area and have monitoring blind spots, making it difficult to achieve comprehensive monitoring of long distances and entire sections of highways. At the same time, the installation and maintenance costs of these devices are high, and they are easily disturbed in bad weather conditions. Manual inspections are inefficient and cannot be carried out around the clock and in real time, which can easily lead to the failure to detect and handle incidents in a timely manner.

[0003] In response to the above technical bottlenecks and shortcomings, the use of drones for traffic monitoring has become an emerging solution in recent years. Drones have the advantages of strong flexibility, wide coverage, and high real-time response capabilities, which can effectively make up for the shortcomings of traditional monitoring methods. However, most current drone monitoring methods still face significant technical challenges in automatically identifying and managing regions of interest. How to consider the business application needs of drones using videos for target or event detection during highway inspections, and combine drone location, posture and other data with road basic data to reasonably divide regions of interest driven by different applications and achieve effective management of each area is still an unresolved problem. Summary of the invention

[0004] The purpose of the present invention is to provide an automatic identification and management system for interest areas of drone videos in road traffic scenes to address the deficiencies in the prior art. It automatically analyzes the target road scene under the drone's acquisition perspective, extracts the interest area, and manages the interest area. The present invention can realize the automatic interest area division of drone detection video image data in highway scenes, and effectively reduce system resource overhead and improve image processing operation efficiency by carrying out business applications such as traffic target detection and event recognition based on deep learning or image recognition technology in the interest area.

[0005] An embodiment of the present application provides an automatic identification and management system for drone video interest areas in a road traffic scenario, the system comprising: Communication-connected road coordinate conversion and data management module, drone position and attitude data management module, and area of ​​interest management module; among them, The road coordinate conversion and data management module is used to obtain the image data of the target road in real time through the UAV, and convert the image coordinates in the target road image data into world coordinates to extract, store and dynamically update the geometric feature information of the target road; The UAV location and attitude data management module is used to collect the geographic location data and attitude data of the UAV in real time; The region of interest management module is used to determine the region of interest of the target road according to the geometric feature information of the target road, the geographic location data and the posture data of the drone, and perform real-time monitoring and mission event detection on the region of interest.

[0006] Optionally, the road coordinate conversion and data management module includes: A coordinate conversion preprocessing unit; wherein the coordinate conversion preprocessing unit is used to perform preprocessing of the image coordinates in the target road image data through the parameters of the image acquisition device built into the drone, so as to realize the conversion of the pixel points of the target road image plane into the three-dimensional coordinate system of the image acquisition device.

[0007] Optionally, the drone position and attitude data management module includes: A drone geographic location data determination unit and a drone attitude data determination unit; wherein, The UAV geographic location data determination unit is used to obtain the real-time geographic location data of the UAV by using a global navigation satellite system; wherein the geographic location data at least includes longitude, latitude and altitude information; The drone attitude data determination unit is used to obtain the attitude data of the drone through an inertial measurement unit; wherein the attitude data at least includes a pitch angle, a roll angle and a heading angle.

[0008] Optionally, the road coordinate conversion and data management module further includes: The data fusion and coordinate conversion unit is used to receive and fuse the real-time geographic location data and attitude data of the UAV through an extended Kalman filter algorithm, and convert the pixel points of the target road image plane into a world coordinate system through the parameters of the image acquisition device built into the UAV.

[0009] Optionally, the data fusion and coordinate conversion unit includes: The perspective transformation subunit is used to transform the pixel points of the target road image plane into the world coordinate system through a preset perspective transformation method, so as to align the target road image with the actual target road area.

[0010] Optionally, the interest area management module includes: An interest region division unit and an interest region detection and identification unit, wherein: The region of interest division unit is used to realize automatic region division of the target road according to the real-time geographic location data, posture data and target road map data of the UAV; The interest region detection and identification unit is used to perform interest region detection and identification using a deep learning model.

[0011] Optionally, the interest area management module further includes: The region of interest updating unit is used to optimize and update the region of interest using visual-inertial odometry technology.

[0012] Another embodiment of the present application provides a method for automatically identifying and managing interest areas of drone videos in a road traffic scenario, the method comprising: Acquire image data of a target road, and convert image coordinates in the image data of the target road into world coordinates to determine geometric feature information of the target road; Collect the geographic location data and attitude data of the drone; According to the geometric feature information of the target road, the geographic location data and the posture data of the UAV, the region of interest of the target road is determined, and real-time monitoring and detection of mission events are performed on the region of interest.

[0013] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to implement the above-mentioned method when executed.

[0014] Another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the above-mentioned method.

[0015] Compared with the prior art, the present invention provides an automatic identification and management system for drone video interest areas in road traffic scenes, wherein the road coordinate conversion and data management module is used to obtain the image data of the target road in real time through the drone, and convert the image coordinates in the target road image data into world coordinates to extract, store and dynamically update the geometric feature information of the target road; the drone position and attitude data management module is used to collect the geographic location data and attitude data of the drone in real time; the interest area management module is used to determine the interest area of ​​the target road according to the geometric feature information of the target road, the geographic location data and attitude data of the drone, and perform real-time monitoring and task event detection on the interest area. It automatically analyzes the target road scene under the drone acquisition perspective, extracts the interest area, and manages the interest area. By carrying out business applications such as traffic target detection and event recognition based on deep learning or image recognition technology in the interest area, the system resource overhead is effectively reduced and the image processing operation efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A hardware structure block diagram of a computer terminal for a method for automatically identifying and managing interest areas of drone videos in road traffic scenarios provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of an automatic identification and management system for interest areas of drone videos in road traffic scenarios provided by an embodiment of the present invention; Figure 3 A schematic diagram of a flow chart of a method for automatically identifying and managing regions of interest in drone videos in a road traffic scenario provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, but should not be construed as limiting the present invention.

[0018] The embodiment of the present invention first provides a method for automatically identifying and managing interest areas of drone videos in road traffic scenarios. The method can be applied to electronic devices, such as computer terminals, specifically ordinary computers, quantum computers, etc.

[0019] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal for automatically identifying and managing a drone video interest area in a road traffic scene provided by an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0020] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / modules corresponding to the method for automatically identifying and managing the area of ​​interest of drone video in the road traffic scene in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0021] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0022] See also Figure 2 , Figure 2A structural schematic diagram of an automatic identification and management system for drone video interest areas in a road traffic scenario provided by an embodiment of the present invention, wherein the automatic identification and management system 200 for drone video interest areas in a road traffic scenario may include a road coordinate conversion and data management module 201, a drone position and posture data management module 202, and an interest area management module 203 that are communicatively connected; wherein the road coordinate conversion and data management module 201 is used to acquire image data of a target road in real time through a drone, and convert the image coordinates in the target road image data into world coordinates to extract, store, and dynamically update the geometric feature information of the target road; the drone position and posture data management module 202 is used to collect the geographic location data and posture data of the drone in real time; the interest area management module 203 is used to determine the interest area of ​​the target road according to the geometric feature information of the target road, the geographic location data and posture data of the drone, and perform real-time monitoring and task event detection on the interest area.

[0023] Specifically, the road coordinate conversion and data management module 201 may include: a coordinate conversion preprocessing unit; wherein the coordinate conversion preprocessing unit is used to perform preprocessing of the image coordinates in the target road image data through the parameters of the image acquisition device built into the drone, so as to realize the conversion of the pixel points of the target road image plane into the three-dimensional coordinate system of the image acquisition device.

[0024] In an optional embodiment, the road coordinate conversion and data management module 201 may also include: a data fusion and coordinate conversion unit, which is used to receive and fuse the real-time geographic location data and posture data of the drone through an extended Kalman filter algorithm, and convert the pixel points of the target road image plane into the world coordinate system through the parameters of the image acquisition device built into the drone.

[0025] The UAV position and attitude data management module 202 may include: a UAV geographic location data determination unit and a UAV attitude data determination unit; wherein the UAV geographic location data determination unit is used to obtain the real-time geographic location data of the UAV by using a global navigation satellite system; wherein the geographic location data at least includes longitude, latitude and altitude information; the UAV attitude data determination unit is used to obtain the attitude data of the UAV through an inertial measurement unit; wherein the attitude data at least includes a pitch angle, a roll angle and a heading angle.

[0026] Wherein, the data fusion and coordinate conversion unit may include: a perspective transformation subunit, which is used to transform the pixel points of the target road image plane into the world coordinate system through a preset perspective transformation method to achieve alignment of the target road image with the actual target road area.

[0027] Specifically, the area of ​​interest management module 203 may include: an area of ​​interest division unit and an area of ​​interest detection and identification unit, wherein the area of ​​interest division unit is used to realize automatic area division of the target road according to the real-time geographic location data, posture data and target road map data of the drone; the area of ​​interest detection and identification unit is used to perform detection and identification of the area of ​​interest using a deep learning model.

[0028] The interest region management module 203 may further include: an interest region updating unit, wherein the interest region updating unit is used to optimize and update the interest region by using visual-inertial odometer technology.

[0029] In a specific implementation, the road coordinate conversion and data management module 201 is applied to highway traffic management, and the basic road position data may include geometric features such as the number of lanes, lane width, slope, and road markings. These data are important for monitoring and analyzing traffic conditions and identifying traffic events. The function of the road coordinate conversion and data management module 201 is to convert image coordinates into world coordinates from image data obtained by drones, thereby extracting and managing basic geometric feature information of roads, and realizing the storage and dynamic update of these data.

[0030] For example, this part can determine the camera's internal and external parameters through camera calibration. These parameters are used to convert the image coordinates collected by the drone through the video stream into the real world coordinates. The process of converting the world coordinates (X, Y, Z) to the image coordinates (u, v) is as follows: ; Among them, I represents the identity matrix, S represents the scale factor, which is used to process the proportional relationship of homogeneous coordinates, (X, Y, Z) represents the point in the world coordinate system, (u, v) represents the point in the image coordinate system, K represents the intrinsic parameter matrix of the camera, [R|t] represents the extrinsic parameter matrix of the camera, where R is the rotation matrix, which represents the rotation of the camera relative to the world coordinate system, and t is the translation vector, which is used to describe the translation of the camera position relative to the world coordinate system.

[0031] In order to convert the image coordinates (u, v) back to the world coordinates (X, Y, Z), we need to know the coordinate information of the point Z on a specific plane. For example, a road can be assumed to be a plane Z=Z 0, in this case, an inverse perspective transform can be used. In this case, the point in the world coordinate system is (X,Y,Z). Back-project the image coordinates onto this plane: ; Among them, H -1 Represents the inverse of the perspective transformation matrix.

[0032] The perspective transformation matrix H can be calculated from four known pairs of image coordinate points (u, v) and corresponding world coordinate points (X, Y, Z). Four pairs of points are the minimum requirement for perspective transformation, because the perspective transformation matrix H has 8 independent unknowns, and each pair of image coordinates and world coordinates provides 2 equations, so at least 4 pairs of points are needed to form a complete set of equations and solve the perspective transformation matrix.

[0033] Assuming that at 4 points and their corresponding points, the perspective transformation matrix can be obtained by solving the following linear equations: ; Among them, l represents the point in the image, and h is the element in the perspective transformation matrix.

[0034] Solving this system of equations yields the elements of the perspective transformation matrix H. These elements are then substituted into the inverse perspective transformation formula above for the transformation. When back-projecting to world coordinates, the scale factor S can be eliminated by assuming the depth of a specific point, such as the height of the road plane. The implementation process is shown in the following formula: ; The calculation of X and Y is as follows: ; ; Among them, Z 0 represents the assumed height of the road plane (i.e., the depth of the plane, the Z coordinate), r 31 、r 32 、r 33 ,t z Represents the rotation and translation parameters in the perspective transformation matrix, which is used to map image coordinates to world coordinates, c x 、c y represents the image coordinates of the camera optical center, that is, the camera's principal point offset, f x 、f y Represents the focal length of the camera, which is used to represent the scaling of the image in the x and y directions.

[0035] Through detailed formula derivation and step-by-step instructions, image coordinates can be converted into world coordinates to achieve automated management of basic road information.

[0036] The management of drone position and attitude data 202 is a key step in realizing the automatic identification and management of regions of interest in road traffic scenes. The drone collects real-time data through its sensors (such as GNSS and IMU), including position coordinates and attitude information (such as pitch angle, heading angle, roll angle, etc.), which are used to accurately identify and locate regions of interest to realize the automatic management of road scenes.

[0037] Drones use GNSS (Global Navigation Satellite System) to obtain real-time geographic location data, including longitude, latitude, and altitude information. These data can provide the drone with absolute position coordinates in three-dimensional space. The drone obtains its attitude data through IMU (Inertial Measurement Unit), including pitch angle, roll angle, and heading angle. These data are used to describe the spatial direction and rotation state of the drone. The extended Kalman filter is used to fuse the sensor data from GNSS and IMU to obtain the precise position and attitude estimate of the drone. The extended Kalman filter can effectively handle nonlinear motion models, provide continuous and smooth state estimation, and reduce noise and uncertainty.

[0038] In the extended Kalman filter EKF, it is necessary to define the state vector of the drone, which usually includes the position, speed, and attitude of the drone. A typical state vector can be: ; Among them, x, y, z represent the position of the drone in the world coordinate system. Indicates the speed of the drone in all directions, Represents the attitude of the drone, namely the roll angle, pitch angle and heading angle. The process model is used to predict the next state of the drone based on the acceleration and angular velocity data provided by the IMU. The IMU data has a high frequency but is prone to accumulate errors, so it needs to be corrected through state updates. The angular velocity w and acceleration α data provided by the IMU are integrated to update the attitude and velocity of the drone: ; Among them, f(x,u) is the nonlinear system state transfer function, u represents the IMU data, and w is the process noise.

[0039] In EKF, the measurement model is used to fuse GNSS data with the predicted state. GNSS measurements are less frequent but provide accurate absolute positions. By comparing the GNSS measurements with the position information predicted by the IMU, the EKF can correct the state of the drone.

[0040] The measurement equation is as follows: Z = h(x) + v, where h(x) is a nonlinear measurement equation that indicates how to obtain the measurement value from the state estimate, and v is the measurement noise.

[0041] The EKF process prediction and measurement update are executed in a loop. At each moment, the system predicts the state based on the IMU data and then corrects the state based on the GNSS data. The IMU provides high-frequency state updates, and the GNSS provides low-frequency but accurate position corrections. Through this fusion, the drone can obtain continuous and accurate position information and attitude estimation.

[0042] The image coordinates in the drone camera view are converted to world coordinates through the drone's position information (GNSS coordinates) and attitude data (IMU data), combined with the camera's internal and external parameters. The coordinates in the image are two-dimensional (in pixels). In order to convert them to three-dimensional coordinates in the world coordinate system, the pixel points in the image plane must first be converted to the camera's own three-dimensional coordinate system through the camera's internal parameters. Combined with the drone's attitude data and position information, the points in the camera coordinate system can be converted to the world coordinate system through the camera's external parameters.

[0043] Based on the position and attitude data of the drone, combined with high-precision map data and basic road position data (such as lane lines, boundaries, etc.), the position of the area of ​​interest is determined by converting the coordinates in the image to the world coordinates in the video stream intercepted by the drone, and the boundary and range of the area of ​​interest are automatically calculated through coordinate conversion.

[0044] During the real-time flight of the drone, the size and position of the region of interest are adjusted through the constantly updated position information and attitude data. Based on the real-time position and attitude data of the drone, traffic events in the region of interest (such as illegally parked vehicles, spilled objects, fog, etc.) are detected through the image data captured by the camera. During the drone cruising process, the real-time updated position and attitude data are used in combination with the recognition results of the region of interest to dynamically adjust the monitoring tasks and parameter settings. For example, according to different monitoring requirements (such as distance, environmental changes), the region of interest is re-divided to realize dynamic reporting and processing of events.

[0045] The management of the region of interest in the region of interest management module 203 can be divided into the following main steps: First, the regions of interest are divided, and the road scenes are zoned and managed according to the real-time location and attitude data of the drone and the high-precision map data. Different regions may correspond to different monitoring tasks, such as detecting illegally parked vehicles, spilled objects, fog, etc. The main task of the region of interest division is to automatically divide the road scene into regions according to the location information of the drone and the high-precision map data. For example, perspective transformation can be used to accomplish this.

[0046] Perspective transformation is used to convert points in the image coordinate system to the world coordinate system in order to align the image captured by the drone with the actual road area.

[0047] ; Among them, p w =[X,Y,Z,I] T is a point in the world coordinate system, p i =[u,v,I] T is a point in the image coordinate system, and H is the perspective transformation matrix, which can be calculated using known world coordinate points and image coordinate points.

[0048] Divide the transformed region of interest according to the distance from the drone, for example: The area between 0 and 50 meters from the drone: used for target detection (such as detecting vehicles, pedestrians, etc.); Area 50 to 100 meters from the drone: used for event detection (such as identifying traffic accidents or illegally parked vehicles); Area 100 to 150 meters from the drone: used for fog detection or weather monitoring.

[0049] After the region of interest is divided, it is necessary to detect and identify the region of interest in the video stream captured by the drone. This step uses a deep learning model for semantic segmentation and object detection. Semantic segmentation is used to classify each pixel in the video image into a specific category, such as road, lane, pedestrian, etc. A deep learning model is used to implement semantic segmentation. The output of the model is a feature map of the same size as the input image, and each pixel is assigned a category label.

[0050] The formula is: For each pixel p, the semantic segmentation model outputs a probability score P(c|p), where c is the category label: ; Among them, W and b are the weights and biases of the model, f(p) is the feature representation of the input image, and the Softmax function is used to convert the output of the model into a probability distribution. Object detection is used to identify and locate specific objects (such as illegally parked vehicles, obstacles, etc.) in an image. The output of the object detection model usually includes the location of a bounding box and a category label: The formula represents: The bounding box position is (x, y, w, h), where (x, y) is the coordinate of the center of the bounding box, and w and h are the width and height respectively.

[0051] The loss function of the target detection model usually includes classification loss and regression loss: ; Among them, L cls is the classification loss, p is the predicted category probability, p* is the true category label, L reg is the regression loss, t is the predicted bounding box parameter, t* is the true bounding box parameter, and λ cls and λ regis the weight parameter.

[0052] The goal of dynamic adjustment and management is to adjust the size and position of the region of interest according to the real-time position of the drone and environmental changes. The region of interest is dynamically adjusted according to the real-time position information and attitude data of the drone. Assume that the current position information of the drone is p=(X,Y,Z) and the attitude information is quaternion q=(q 0 ,q 1 ,q 2 ,q 3 ), the position of the adjusted region of interest can be expressed by rotation and translation: ; Among them, R(q) is the rotation matrix converted from the quaternion q, and t is the translation vector, which represents the relative displacement of the drone.

[0053] To ensure real-time updates of the region of interest, the following strategies can be used: Position prediction: Based on the current speed and direction of the drone, predict its position at the next moment, and update the region of interest based on the predicted position. Through the drone's sensors (such as GNSS and IMU), we can obtain the drone's current speed and flight direction in real time. This data can be used to predict where the drone will move to in a short period of time. Based on the drone's current speed (vector) and time interval, the drone's future position can be predicted by a simple kinematic formula. For example, assuming that the drone moves at a constant speed and direction, then we can estimate its position at the next moment by a linear prediction method. In this way, the region of interest can be updated and adjusted before the drone arrives.

[0054] Error Correction: Using the Extended Kalman Filter (EKF) allows for continuous correction of position errors by combining historical measurements with current state predictions. In this process, the EKF uses the drone’s sensor data (such as GNSS and IMU data) to make real-time corrections to the predicted position to improve accuracy in the area of ​​interest.

[0055] Region of interest management ensures that drones can efficiently identify and monitor traffic events in complex road scenes through precise division, detection and dynamic adjustment. Combining perspective transformation, semantic segmentation, target detection and real-time optimization algorithms, intelligent management and monitoring of highway scenes can be achieved.

[0056] It can be seen that the drone position and attitude data management method in this application realizes the accurate position and attitude estimation of the drone by fusing GNSS and IMU data and adopting the extended Kalman filter (EKF) algorithm, and dynamically adjusts the boundaries and ranges of the area of ​​interest in combination with high-precision map data. The automatic identification and dynamic management method of the area of ​​interest converts the image coordinates into world coordinates through perspective transformation, uses deep learning models (such as U-Net or DeepLab) for semantic segmentation, accurately identifies target areas such as roads and vehicles, and optimizes and updates the area of ​​interest using visual-inertial odometer (VIO) technology. The automatic inspection and event detection method based on drone cruising realizes early warning and dynamic reporting of traffic incidents by combining real-time data processing in different monitoring areas (such as close-range target detection, medium-range event detection, and long-range fog group identification). Multi-task management and hierarchical response strategies automatically adjust monitoring strategies according to monitoring task requirements to ensure the efficiency and flexibility of multi-task execution.

[0057] In summary, the present application discloses a method for extracting and dynamically managing regions of interest of unmanned aerial vehicles, a method for fusing position and attitude data, a method for perspective transformation and image processing, an event detection and partition management strategy, and a system integration architecture. The present application utilizes autonomous cruising of unmanned aerial vehicles to achieve real-time monitoring and multi-task event detection of multiple areas on the highway, such as illegal parking of vehicles, spillage, fog, etc., and can quickly process monitoring data, realize dynamic region of interest management and event detection, ensure the real-time nature of detection, and reduce dependence on fixed infrastructure, reduce installation and maintenance costs, and improve the economy and reliability of the system.

[0058] Compared with the prior art, the present invention provides an automatic identification and management system for drone video interest areas in road traffic scenes, wherein the road coordinate conversion and data management module is used to obtain the image data of the target road in real time through the drone, and convert the image coordinates in the target road image data into world coordinates to extract, store and dynamically update the geometric feature information of the target road; the drone position and attitude data management module is used to collect the geographic location data and attitude data of the drone in real time; the interest area management module is used to determine the interest area of ​​the target road according to the geometric feature information of the target road, the geographic location data and attitude data of the drone, and perform real-time monitoring and task event detection on the interest area. It automatically analyzes the target road scene under the drone acquisition perspective, extracts the interest area, and manages the interest area. By carrying out business applications such as traffic target detection and event recognition based on deep learning or image recognition technology in the interest area, the system resource overhead is effectively reduced and the image processing operation efficiency is improved.

[0059] See also Figure 3 , Figure 3A flowchart of a method for automatically identifying and managing interest areas of drone videos in a road traffic scenario provided by an embodiment of the present invention may include the following steps: S301: Acquire image data of a target road, and convert image coordinates in the target road image data into world coordinates to determine geometric feature information of the target road; S302: Collecting geographic location data and attitude data of the drone; S303: Determine the region of interest of the target road according to the geometric feature information of the target road, the geographic location data and the posture data of the UAV, and perform real-time monitoring and mission event detection on the region of interest.

[0060] Specifically, this application proposes an automatic extraction and management application method for interest areas of highway scenes based on drones, which automatically analyzes the road scenes under the drone's collection perspective, extracts interest areas, and manages the interest areas. By integrating the drone's perception attitude data, such as longitude and latitude, heading angle, pitch angle, speed and other information, with the basic road position data of the high-precision map, such as lane distribution, lane width, road direction and slope, the drone-collected video image is matched with the actual road. According to the distribution of the actual target road in the image, a road area segmentation method based on location information is proposed. On the basis of identifying the road area, the road scene is further divided into interest areas according to the target road drone traffic management requirements, and functional attributes are added to the interest areas, such as target extraction and refinement of vehicle operation parameters, abnormal road surface state detection in the close-range area, traffic event discrimination application in the medium-range area, and identification of abnormal environmental conditions such as fog at a long distance.

[0061] This application can realize the automatic region of interest division of drone detection video image data in the target road scene. By carrying out business applications such as traffic target detection and event recognition based on deep learning or image recognition technology in the region of interest, it can effectively reduce system resource overhead and improve image processing operation efficiency.

[0062] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to implement the steps in the above method embodiment when running.

[0063] Specifically, in this embodiment, the above storage medium may be configured to store a computer program for performing the following steps: S301: Acquire image data of a target road, and convert image coordinates in the target road image data into world coordinates to determine geometric feature information of the target road; S302: Collecting geographic location data and attitude data of the drone; S303: Determine the region of interest of the target road according to the geometric feature information of the target road, the geographic location data and the posture data of the UAV, and perform real-time monitoring and mission event detection on the region of interest.

[0064] Specifically, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0065] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in the above method embodiment.

[0066] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0067] Specifically, in this embodiment, the processor may be configured to perform the following steps through a computer program: S301: Acquire image data of a target road, and convert image coordinates in the target road image data into world coordinates to determine geometric feature information of the target road; S302: Collecting geographic location data and attitude data of the drone; S303: Determine the region of interest of the target road according to the geometric feature information of the target road, the geographic location data and the posture data of the UAV, and perform real-time monitoring and mission event detection on the region of interest.

[0068] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0069] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0070] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0071] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0072] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0073] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0074] The embodiments of the present invention are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An automatic identification and management system for interest areas of drone videos in road traffic scenes, characterized in that: The system comprises: Communication-connected road coordinate conversion and data management module, drone position and attitude data management module, and area of ​​interest management module; among them, The road coordinate conversion and data management module is used to obtain the image data of the target road in real time through the UAV, and convert the image coordinates in the target road image data into world coordinates to extract, store and dynamically update the geometric feature information of the target road; The UAV location and attitude data management module is used to collect the geographic location data and attitude data of the UAV in real time; The region of interest management module is used to determine the region of interest of the target road according to the geometric feature information of the target road, the geographic location data and the posture data of the drone, and perform real-time monitoring and mission event detection on the region of interest.

2. The system according to claim 1, characterized in that The road coordinate conversion and data management module includes: A coordinate conversion preprocessing unit; wherein the coordinate conversion preprocessing unit is used to perform preprocessing of the image coordinates in the target road image data through the parameters of the image acquisition device built into the drone, so as to realize the conversion of the pixel points of the target road image plane into the three-dimensional coordinate system of the image acquisition device.

3. The system according to claim 2, characterized in that The drone position and attitude data management module includes: A drone geographic location data determination unit and a drone attitude data determination unit; wherein, The UAV geographic location data determination unit is used to obtain the real-time geographic location data of the UAV by using a global navigation satellite system; wherein the geographic location data at least includes longitude, latitude and altitude information; The drone attitude data determination unit is used to obtain the attitude data of the drone through an inertial measurement unit; wherein the attitude data at least includes a pitch angle, a roll angle and a heading angle.

4. The system according to claim 3, characterized in that The road coordinate conversion and data management module further includes: The data fusion and coordinate conversion unit is used to receive and fuse the real-time geographic location data and attitude data of the UAV through an extended Kalman filter algorithm, and convert the pixel points of the target road image plane into a world coordinate system through the parameters of the image acquisition device built into the UAV.

5. The system according to claim 4, characterized in that The data fusion and coordinate conversion unit comprises: The perspective transformation subunit is used to transform the pixel points of the target road image plane into the world coordinate system through a preset perspective transformation method, so as to align the target road image with the actual target road area.

6. The system according to claim 4, characterized in that The interest area management module includes: An interest region division unit and an interest region detection and identification unit, wherein: The region of interest division unit is used to realize automatic region division of the target road according to the real-time geographic location data, posture data and target road map data of the UAV; The interest region detection and identification unit is used to perform interest region detection and identification using a deep learning model.

7. The system according to claim 5, characterized in that The interest area management module further includes: The region of interest updating unit is used to optimize and update the region of interest using visual-inertial odometry technology.

8. A method for automatically identifying and managing interest areas of drone videos in road traffic scenes, characterized in that: The method comprises: Acquire image data of a target road, and convert image coordinates in the image data of the target road into world coordinates to determine geometric feature information of the target road; Collect the geographic location data and attitude data of the drone; According to the geometric feature information of the target road, the geographic location data and the posture data of the UAV, the region of interest of the target road is determined, and real-time monitoring and detection of mission events are performed on the region of interest.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to implement the method described in claim 8 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method described in claim 8.

Images