A method and device for dynamic monitoring of vehicle axle type and speed based on unmanned aerial vehicle

By carrying lidar and motion cameras on the drone, combined with advanced identification algorithms, precise monitoring of vehicle shaft type, speed and traffic direction coefficients is achieved, and problems of road damage, construction difficulties and high costs in the existing technology are solved. , real-time and comprehensive traffic parameter monitoring is achieved.

CN115019264BActive Publication Date: 2024-11-19JIANGSU DONGCHUANG DAWEI INTELLIGENT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210707443.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-21
Publication Date
2024-11-19
Estimated Expiration
2042-06-21

AI Technical Summary

Technical Problem

The existing traffic parameter detection methods have problems such as causing damage to the road surface, difficulty in construction and maintenance, easy to affect the accuracy, high costs and only monitoring specific roads.

Method used

UAVs are equipped with lidar and motion cameras, and image information is obtained through lidar and motion cameras. Combined with ROS system, Point-RCNN and yolo image recognition algorithms, we realize the vehicle axis type, speed and traffic direction coefficients. Real-time monitoring and build a traffic database.

Benefits of technology

Accurate monitoring of any road traffic volume, vehicle speed, vehicle axle type and lane direction coefficient is achieved, avoiding road damage and high costs, and can monitor traffic parameters in real time and comprehensively.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115019264B_ABST
    Figure CN115019264B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for dynamically monitoring the axle type and speed of a vehicle based on an unmanned aerial vehicle. The device includes a ground control and storage system, an unmanned aerial vehicle, and a laser radar, an industrial computer, a motion camera, and a wireless data transmission device arranged on the unmanned aerial vehicle, wherein: the laser radar is used to collect point cloud information of road traffic; the motion camera is used to collect image information of road traffic; the industrial computer is used to complete the external parameter setting of the laser radar and the motion camera; the ground control and storage system is used to receive point cloud information and image information through the wireless data transmission device and perform data processing, and send the unmanned aerial vehicle flight plan. The present invention uses the unmanned aerial vehicle equipped with a laser radar and a motion camera, and utilizes the three-dimensional information of the object obtained by the laser radar and the two-dimensional information obtained by the camera to realize the monitoring of basic traffic parameters such as the traffic volume, vehicle speed, vehicle axle type, and lane direction coefficient of any road.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart transportation, and in particular to a method for dynamic monitoring of traffic parameters based on unmanned aerial vehicles. Background Art

[0002] Smart highways are a new form of highway that adapts to the current advanced electronic information era. As of the end of 2020, the total mileage of national highways is 5.1981 million kilometers, and the number of cars on the road has reached 270 million. Such a huge traffic volume requires more scientific monitoring methods as an aid to achieve dynamic monitoring of traffic parameters.

[0003] At present, most traffic parameter detection methods extract road traffic information by burying induction coils, infrared lasers or installing surveillance cameras. However, burying induction coils will damage the road surface itself, and this method requires a lot of construction and is difficult to maintain. Although infrared laser detection does not damage the road surface, it is easily affected by pedestrians and non-motor vehicles. The cost of installing surveillance cameras is high, and traffic information can only be extracted for a specific road. Summary of the invention

[0004] Purpose of the invention: In view of the above problems, the present invention proposes a traffic parameter dynamic monitoring technology using a drone equipped with a laser radar and a motion camera, which can monitor road traffic in real time and establish a corresponding traffic database to facilitate management by relevant departments.

[0005] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is:

[0006] A method for dynamic monitoring of vehicle axle type and speed based on drone, the specific steps are as follows:

[0007] (I) Setting up a drone equipped with laser radar and sports camera

[0008] Set the motion camera to be horizontal with the laser radar;

[0009] (II) External parameter setting of LiDAR and motion camera

[0010] Calibrate the laser radar and motion camera to unify the coordinate systems of the laser radar and motion camera;

[0011] The point cloud information of the laser radar and the image information taken by the motion camera are collected through the ROS system, and the point cloud information and image information with the same timestamp are extracted;

[0012] (III) Setting up the UAV flight plan

[0013] Build a 3D road model based on LiDAR and derive road data based on 2D maps;

[0014] Combined with the starting and ending points of the drone, the obstacles within the road clearance range are identified through the three-dimensional road model, and the corresponding drone waypoints are generated according to the horizontal and vertical curves of the road;

[0015] Import the drone waypoints into DJ Onboard SDK to complete the drone path planning task;

[0016] For vehicle axle type detection, the drone hovering position is also set;

[0017] (IV) Data processing

[0018] The LiDAR offset angle is calculated by solving the normal vector of the ground point cloud, and the point cloud information is preprocessed using the rotation matrix to make it horizontal;

[0019] Based on the lane lines, the pre-processed point cloud information is segmented using the point cloud segmentation algorithm to extract the road part, and the vehicle target is recognized through the Point-RCNN recognition algorithm;

[0020] The eight vertex coordinates of the three-dimensional object frame obtained by vehicle target recognition are obtained, and the speed of the vehicle target is detected by comparing the positions of the three-dimensional object frames of the same target in different image frames;

[0021] Based on the image information captured by the motion camera, the vehicle axle type is recognized through the YOLO image recognition algorithm.

[0022] Furthermore, the position of the motion camera is set to be spatially horizontal with the laser radar.

[0023] Furthermore, the method also includes: calculating the traffic direction coefficient of the road through a point cloud segmentation algorithm.

[0024] Furthermore, the collected point cloud information and image information are transmitted to the ground station to build a traffic parameter monitoring database.

[0025] Furthermore, through rotation and translation transformation, the laser radar coordinate system and the camera coordinate system are aligned in space.

[0026] Furthermore, the position of the same vehicle in different point cloud coordinates can be obtained through a 3D target tracking algorithm.

[0027] A dynamic monitoring device for vehicle axle type and speed based on an unmanned aerial vehicle comprises a ground control and storage system, an unmanned aerial vehicle, and a laser radar, an industrial computer, a motion camera, and a wireless data transmission device arranged on the unmanned aerial vehicle, wherein:

[0028] LiDAR, used to collect point cloud information of road traffic;

[0029] Motion cameras are used to collect image information of road traffic;

[0030] Industrial computer, used to complete the external parameter setting of laser radar and motion camera;

[0031] The ground control and storage system is used to receive point cloud information and image information through wireless data transmission equipment, perform data processing, and send the UAV flight plan.

[0032] Furthermore, the shooting frequency of the motion camera is the same as the transmitting frequency of the lidar.

[0033] Beneficial effects: The present invention uses a drone equipped with a laser radar and a motion camera, and utilizes the three-dimensional information of objects obtained by the laser radar and the two-dimensional information obtained by the camera to monitor basic traffic parameters such as the traffic volume, vehicle speed, vehicle axle type, lane direction coefficient, etc. on any road. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of a dynamic monitoring device for vehicle axle type and speed based on a drone;

[0035] Among them: 1-UAV, 2-UAV flight control unit, 3-LiDAR, 4-Industrial computer, 5-Sports camera, 6-Wireless data transmission equipment, 7-UAV landing gear, 8-Ground control and storage system;

[0036] Figure 2 Schematic diagram of the laser radar and motion camera coordinate system. DETAILED DESCRIPTION

[0037] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. The preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0038] See attached Figure 1The present invention discloses a dynamic monitoring device for vehicle axle type and speed based on a UAV, comprising: a UAV 1 mainly used to construct a flight platform equipped with a laser radar and a motion camera; a UAV flight control unit 1 used to receive instructions issued by a ground control and storage system and make corresponding adjustments to the instructions; a laser radar 3 used to be carried on the UAV to collect point cloud information of road traffic; an industrial computer 4 performs data analysis and preprocessing on the data collected by the laser radar and the motion camera; a motion camera 5 used to capture image information of road traffic, and the shooting frequency is the same as the transmission frequency of the laser radar; a wireless data transmission device 6 used to losslessly transmit the data collected by the laser radar and the motion camera to a ground workstation; a UAV landing gear 7 used to support the UAV to take off and land stably; a control system in a ground control and storage system 8 is used to control the UAV to ensure that the flight route and flight altitude of the UAV meet the preset requirements, and a storage system is used to store the collected traffic information to construct a traffic database based on the UAV.

[0039] As a further preferred embodiment, the drone is equipped with a laser radar and a motion camera for real-time photography of road traffic information.

[0040] As a further preferred embodiment, the laser radar can deeply perceive the position of the vehicle in space by collecting three-dimensional information of objects.

[0041] As a further preferred embodiment, by using laser radar to recognize vehicle targets and perceive the spatial information of the three-dimensional object frame, a more accurate measurement of the vehicle speed can be made.

[0042] As a further preferred embodiment, the vehicle axle type can be accurately estimated using image data captured by a motion camera.

[0043] The drone-based dynamic monitoring device for traffic parameters mentioned in the present invention first needs to determine the drone's hovering position or the pilot manually controls the drone to fly to a designated location, and collect road vehicle information through the drone's equipped laser radar and motion camera, and transmit the collected data to an industrial computer for preprocessing.

[0044] Secondly, the wireless data transmission equipment transmits the pre-processed image information and point cloud information to the ground control and storage system, which performs corresponding algorithm recognition on the transmitted data, and finally realizes real-time monitoring of traffic parameters. The image information and point cloud information are stored to build a corresponding traffic parameter database.

[0045] The present invention provides a method for dynamically monitoring axle type and speed of a vehicle based on an unmanned aerial vehicle, comprising the following steps:

[0046] (I) UAV equipped with LiDAR and sports camera solution

[0047] Step 1. Since the semi-solid laser radar has a vertical viewing angle of about 25°, it needs to be tilted downward at a certain angle when mounted on a drone in order to better collect road traffic information through the laser radar;.

[0048] Step 2. Set the position of the motion camera to be horizontal with the laser radar.

[0049] (II) External parameter setting of LiDAR and motion camera

[0050] Step 3. Collect lidar data and motion camera data through the ROS system;

[0051] Step 4. Use the calibration tool to unify the coordinate systems of the laser radar and the motion camera;

[0052] Step 5. Use technical means to extract device information of different frame rates from the collected rosbag package containing lidar and motion camera data, and finally extract radar and image information with the same timestamp.

[0053] (III) UAV flight plan

[0054] Step 6. Use LiDAR to build a 3D road model and combine it with a 2D map to export road data.

[0055] Step 7. Combine the starting and ending points of the drone and identify obstacles within the road clearance range through the three-dimensional road model, and generate corresponding drone waypoints according to the horizontal and vertical curves of the road;

[0056] Step 8. Import the drone waypoints into DJ Onboard SDK to complete the drone path planning task;

[0057] Step 9. For vehicle axle type detection, calculate the camera imaging area, set a reasonable drone hovering position, and collect vehicle axle type image information in the road area.

[0058] (IV) Data processing

[0059] Step 10. Since the airborne laser radar is not mounted horizontally, it is necessary to calculate the radar deviation angle through the normal vector of the ground point cloud, and use the rotation matrix to pre-process the point cloud data to make it horizontal;

[0060] Ste11. Use the point cloud segmentation algorithm to segment the rotated point cloud through the lane lines to extract the road part;

[0061] Step 12. Realize vehicle target recognition through Point-RCNN recognition algorithm;

[0062] Step 13. Obtain the coordinates of eight vertices of the three-dimensional object frame, and detect the speed of the vehicle target by comparing the positions of the three-dimensional object frames of the same target in different frames;

[0063] Step 14. Using the photos taken by the motion camera in continuous frames, the vehicle axle type is recognized by the YOLO image recognition algorithm;

[0064] Step 15. Calculate the traffic direction coefficient of the road through the point cloud segmentation algorithm and the vehicle target recognition algorithm.

[0065] (V) Building a traffic parameter monitoring database

[0066] Step 16. Transmit the captured point cloud information and photos to the database of the ground station via wireless data transmission equipment for storage.

[0067] Furthermore, the optimal inclination angle of the drone-mounted device is determined through the laser radar perspective to ensure comprehensive traffic information collection on different roads; the sports camera is placed horizontally with the laser radar space to ensure that the vehicle axle shape is captured.

[0068] Furthermore, the laser radar emission frequency and the sports camera shooting frequency are set to the same parameter to facilitate the subsequent confirmation and statistics of vehicle information; the purpose of setting time synchronization is to ensure that the vehicle point cloud information corresponds to the vehicle image information.

[0069] Furthermore, by comparing different shooting angles through the drone's hovering position, full collection of vehicle information within the road width can be achieved; by comparing the detection accuracy of the lidar and motion camera, the optimal flight altitude of the drone can be determined.

[0070] Furthermore, if Figure 2 As shown in the figure, the position of the object in the laser radar and camera coordinate systems is unified, and the point cloud scanned by the laser radar is located in a three-dimensional coordinate system centered on the laser radar. Since the laser radar and camera sensors are installed in different positions, it is necessary to convert the laser radar coordinate system OXYZ and the camera coordinate system oxyz. The conversion process includes rotation and translation transformation. The laser radar coordinate system needs to be rotated to the camera coordinate system position with the X-axis and Y-axis as the center through the rotation matrix R, and combined with the translation matrix T to align the laser radar coordinate system with the camera coordinate system in space.

[0071] Furthermore, the purpose of setting time synchronization is to ensure that the point cloud information and image information have the same timestamp. When collecting, the point cloud and image can be recorded simultaneously by recording the rosbag package. Since the frame rate of the lidar and the camera is different, and the camera frame rate is generally higher than the radar frame rate, the video can be extracted from the recorded rosbag package first, and the video frame rate can be controlled in the process of converting the video to the image, so that the data collected by the two sensors are synchronized in time.

[0072] Further, S5.1: the offset angle of the laser radar relative to the horizontal direction is calculated by solving the normal vector of the collected ground point cloud, and the original point cloud is preliminarily processed using the corresponding rotation matrix to make it horizontal;

[0073] S5.2: The rotated point cloud is further processed, and the road surface area in the point cloud is extracted by using lane lines and point cloud segmentation algorithms;

[0074] S5.3: Vehicle target recognition is achieved through the Point-RCNN recognition algorithm, and the corresponding 3D target tracking algorithm is combined to obtain the position of the same vehicle in different point cloud coordinates.

[0075] S5.3: By acquiring the three-dimensional object frame information of the same target vehicle in different frames, the eight vertex coordinates of the object frames of the previous and next two frames are compared, and the speed detection of the vehicle target is realized based on the transformation of the spatial position;

[0076] S5.4: Perform axle type recognition on the collected road vehicle photos through image recognition algorithm;

[0077] S5.5: Count the traffic volume in different driving directions and calculate the direction coefficient of the road.

[0078] It should be pointed out that the description of the above embodiments is only used to help understand the method of the present application and its core idea. For ordinary technicians in this technical field, several improvements and modifications can be made to the present application without departing from the principles of the present application. These improvements and modifications are also within the scope of protection of the claims of the present application.

Claims

1. A method for dynamic monitoring of vehicle axle type and speed based on drone, characterized in that: The specific steps are as follows: (I) Setting up a drone equipped with laser radar and sports camera Set the motion camera to be horizontal with the laser radar; (II) External parameter setting of LiDAR and motion camera Calibrate the laser radar and motion camera to unify the coordinate systems of the laser radar and motion camera; The point cloud information of the laser radar and the image information taken by the motion camera are collected through the ROS system, and the point cloud information and image information with the same timestamp are extracted; (III) Setting up the UAV flight plan Build a 3D road model based on LiDAR and derive road data based on 2D maps; Combined with the starting and ending points of the drone, the obstacles within the road clearance range are identified through the three-dimensional road model, and the corresponding drone waypoints are generated according to the horizontal and vertical curves of the road; Import the drone waypoints into DJ Onboard SDK to complete the drone path planning task; For vehicle axle type detection, the drone hovering position is also set; (IV) Data processing The LiDAR offset angle is calculated by solving the normal vector of the ground point cloud, and the point cloud information is preprocessed using the rotation matrix to make it horizontal; Based on the lane lines, the pre-processed point cloud information is segmented using the point cloud segmentation algorithm to extract the road part, and the vehicle target is recognized through the Point-RCNN recognition algorithm; The eight vertex coordinates of the three-dimensional object frame obtained by vehicle target recognition are obtained, and the speed of the vehicle target is detected by comparing the positions of the three-dimensional object frames of the same target in different image frames; Based on the image information captured by the motion camera, the vehicle axle type is recognized through the YOLO image recognition algorithm.

2. The method for dynamic monitoring of vehicle axle type and speed based on drone according to claim 1, characterized in that: The position of the motion camera is set to be horizontal in space with the laser radar.

3. The method for dynamic monitoring of vehicle axle type and speed based on drone according to claim 1, characterized in that: The method also includes: calculating the traffic direction coefficient of the road through a point cloud segmentation algorithm.

4. The method for dynamic monitoring of vehicle axle type and speed based on drone according to claim 1, characterized in that: The collected point cloud information and image information are transmitted to the ground control and storage system to build a traffic parameter monitoring database.

5. The method for dynamic monitoring of vehicle axle type and speed based on drone according to claim 1, characterized in that: Through rotation and translation transformation, the lidar coordinate system and the camera coordinate system are aligned in space.

6. The method for dynamic monitoring of vehicle axle type and speed based on drone according to claim 1, characterized in that: The 3D target tracking algorithm is used to obtain the position of the same vehicle in different point cloud coordinates.

7. A dynamic monitoring device for vehicle axle type and speed based on drone, characterized in that: The device is based on the dynamic monitoring of the vehicle axle type and speed completed by the method as claimed in any one of claims 1 to 6; The device includes a ground control and storage system, a drone, and a laser radar, an industrial computer, a motion camera, and a wireless data transmission device arranged on the drone, wherein: LiDAR, used to collect point cloud information of road traffic; Motion cameras are used to collect image information of road traffic; Industrial computer, used to complete the external parameter setting of laser radar and motion camera; The ground control and storage system is used to receive point cloud information and image information through wireless data transmission equipment, perform data processing, and send the UAV flight plan.

8. The dynamic monitoring device for vehicle axle type and speed based on drone according to claim 7, characterized in that: The shooting frequency of the action camera is the same as the transmitting frequency of the lidar.