Scene construction method and system based on unmanned aerial vehicle aerial survey and intelligent tire perception

Abstract

The invention provides a scene construction method and system based on unmanned aerial vehicle aerial survey and intelligent tire sensing, and belongs to the technical field of automatic driving. The system collects macroscopic traffic information through an unmanned aerial vehicle aerial survey module, wherein the macroscopic traffic information comprises road geometry, lane lines and traffic target object tracks; the intelligent tire sensing module obtains pavement microscopic characteristics such as friction coefficient, unevenness and the like by using a touch sensor and a data acquisition and processing unit; the data processing and fusion module fuses the two data, and constructs a key scene including road topology containing semantic attributes, traffic flow and pavement characteristics; and the scene analysis and risk assessment module identifies the risk scene, assesses the grade and outputs early warning information. According to the method and the system, unmanned aerial vehicle macroscopic observation and intelligent tire microcosmic perception are integrated, and comprehensive road condition information is provided for an automatic driving system.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a scene construction method and system based on drone aerial survey and intelligent tire perception. Background Art

[0002] With the rapid development of autonomous driving technology, vehicle safety and reliability in complex road conditions face significant challenges. Traditional autonomous driving systems primarily rely on the vehicle's own sensors (such as cameras and radar) for environmental perception, but their ability to identify even extremely harsh road conditions (such as black ice and flooding) is limited. Furthermore, vehicle sensors typically only perceive local environmental information and lack the ability to capture macroscopic traffic flow and road topology information.

[0003] In recent years, the development of drone-based aerial surveying and smart tire technology has provided new solutions to these problems. Drone surveys can provide macroscopic information on road geometry, lane markings, and the trajectory of traffic objects, while smart tire tactile sensors can directly perceive microscopic road surface properties (such as friction coefficient and roughness). However, how to effectively combine these two technologies to construct critical scenarios and enhance the risk mitigation capabilities of autonomous driving systems remains an urgent challenge. Summary of the Invention

[0004] The purpose of this invention is to provide a scene construction method and system based on drone aerial survey and intelligent tire perception, integrating drone macro observation and intelligent tire micro perception to provide comprehensive road condition information for the autonomous driving system.

[0005] To achieve the above objectives, the present invention provides a key scene construction system based on UAV aerial survey and intelligent tire joint perception, including: The drone aerial survey module collects macroscopic traffic information, including road geometry, lane lines, and traffic object trajectories; An intelligent tire sensing module, comprising a tactile sensor installed inside the tire, a data acquisition unit, a tire data processing unit, and a data transmission unit; The data processing and fusion module receives data from the drone aerial survey module and the intelligent tire perception module, and constructs key scenarios including road topology, traffic flow, and road surface micro-characteristics with semantic attributes through data fusion; The scenario analysis and risk assessment module identifies risk scenario types, assesses risk levels, and outputs warning information.

[0006] Preferably, the UAV aerial survey module includes a UAV platform and a data processing submodule.

[0007] Preferably, the UAV platform is equipped with a camera and a lidar.

[0008] Preferably, the data processing submodule performs: Road geometry extraction: Using image processing algorithms to extract road boundaries and lane lines from camera images, combined with LiDAR point cloud data, a 3D reconstruction algorithm is used to generate a 3D geometric model of the road. Road semantic map generation: Obtain road semantic information based on the OSM map, or after obtaining the traffic target trajectory, use trajectory classification to obtain the routes to which different trajectories belong and construct road semantic information; Lane detection: Use deep learning algorithms to detect lane lines in camera images; Traffic target trajectory tracking: The dynamic and static information of traffic targets is acquired through the camera, and the shape, size, position, speed and acceleration of the vehicle are obtained through target detection, target tracking and Kalman filter post-processing steps.

[0009] Preferably, the tactile sensor is a three-axis acceleration sensor, the data acquisition unit filters and normalizes the data collected by the tactile sensor, the tire data processing unit calculates the road friction coefficient and road roughness, and the data transmission unit transmits the data to the data processing and fusion module.

[0010] Preferably, the road friction coefficient is calculated based on the tire dynamics model and the longitudinal / lateral acceleration; and the road roughness is characterized by the power spectrum density and the root mean square value of the vertical acceleration.

[0011] Preferably, the data processing and fusion module adopts a Kalman filter algorithm or a deep learning model to integrate the macroscopic road information of the drone aerial survey and the microscopic road surface characteristics of the smart tire.

[0012] Preferably, the scenario analysis and risk assessment module identifies at least one of the following risk scenarios: low adhesion road surface, wading scene, uneven road surface, and complex traffic flow scene.

[0013] The present invention also provides a scene construction method based on UAV aerial survey and intelligent tire perception, including: Step S1: Plan the flight path of the UAV, synchronously collect road images and lidar point cloud data, perform image denoising and lane line detection on the images, perform 3D reconstruction on the point cloud to extract road slope and curvature, and track the trajectory of traffic targets based on camera data; Step S2: planning the vehicle's driving path, collecting signals through tactile sensors, estimating the road friction coefficient based on the tire dynamics model, and calculating the road roughness through fast Fourier transform analysis; Step S3: Time-synchronize and spatially align the drone data and tire data, integrate road geometry, traffic object trajectories, road friction coefficient, and road roughness, and construct a key scene dataset containing risk labels.

[0014] Preferably, the dataset is used to train autonomous driving perception models or verify control strategies; When applying, execute according to the risk level: Low risk: prompt the driver to pay attention; Medium risk: autonomous driving system adjusts strategy; High risk: Instruct to park or switch to safety mode.

[0015] Therefore, the present invention adopts the above-mentioned scene construction method and system based on drone aerial survey and intelligent tire perception, and the beneficial technical effects are as follows: Drone aerial surveys combined with smart tires can accurately identify complex scenarios such as low-adhesion roads (such as black ice and slippery roads) and wading, providing comprehensive road condition information to the autonomous driving system, thereby significantly improving the safety of autonomous driving.

[0016] Multi-sensor data fusion technology combines the macro observation of drones with the micro perception of smart tires, enabling the system to adapt to a variety of complex road conditions and have high robustness and adaptability.

[0017] In addition, the system supports the development of Safety of Intended Functionality (SOTIF) for high-level autonomous driving systems. By perceiving and evaluating key scenarios in real time, it enhances the autonomous driving system's ability to identify and respond to specific risk scenarios, thereby improving the functional safety of the system.

[0018] Offline, data collected by data collection vehicles and drones are used to build key scenario datasets, providing valuable training data and test scenarios for the development of vehicle dynamics and autonomous driving systems, helping to improve system performance and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the structure of the scene construction system based on UAV aerial survey and intelligent tire perception of the present invention; Figure 2 This is a flow chart of the scene construction method based on UAV aerial survey and intelligent tire perception of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0021] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0022] Example 1 1. System composition

[0023] like Figure 1As shown in the figure, the scene construction system based on drone aerial survey and intelligent tire perception includes the following modules: 1. UAV aerial survey module.

[0024] A drone platform equipped with a high-resolution camera and lidar. The camera is used to collect road image data, and the lidar is used to collect 3D point cloud data of the road.

[0025] The data processing submodule includes road geometry extraction, road semantic map generation, lane detection, and traffic object trajectory tracking. Road geometry extraction uses image processing algorithms (such as edge detection and Hough transform) to extract road boundaries and lane lines from camera images. Combined with LiDAR point cloud data, 3D reconstruction algorithms (such as voxel grid filtering and region growing segmentation) are used to generate a 3D road geometric model. Road semantic map generation involves two methods: first, obtaining road semantic information from OSM maps; second, after obtaining traffic object trajectories, using trajectory classification to identify the routes to which different trajectories belong and construct road semantic information. Lane detection uses deep learning algorithms (such as convolutional neural networks, or CNNs) to detect lane lines in camera images and output their precise position and orientation. The camera captures dynamic and static information about traffic objects (such as vehicles and pedestrians). Vehicle parameters such as shape, size, position, speed, and acceleration are determined through object detection, object tracking, and post-Kalman filtering.

[0026] 2. Intelligent tire sensing module.

[0027] A tactile sensor (such as a three-axis accelerometer) installed inside the tire is used to collect the acceleration signal of the tire.

[0028] The data acquisition unit is responsible for data collection and preliminary processing of the tactile sensor, such as filtering and normalization.

[0029] The tire data processing unit estimates the road friction coefficient and road roughness based on tactile sensor data. Friction coefficient estimation is based on the tire's longitudinal and lateral acceleration signals, combined with tire dynamics models (such as the Magic Formula, Brush Model, UniTire Model, and SWIFT Model) using the least squares method or Kalman filter algorithm. Road roughness estimation analyzes the frequency distribution and amplitude variations of the tire's vertical acceleration signal, extracts road roughness characteristics using the Fast Fourier Transform (FFT) algorithm, and estimates road roughness parameters (such as RMS value and power spectral density) using a road roughness model.

[0030] The data transmission unit transmits the processed data to the data processing and fusion module.

[0031] 3. Data processing and fusion module.

[0032] The data preprocessing unit preprocesses the drone aerial survey data and smart tire perception data, including data cleaning, format conversion and time synchronization.

[0033] The feature extraction unit extracts features from drone aerial survey data and intelligent tire sensing data to obtain macroscopic road and traffic flow and microscopic road surface conditions. The feature extraction from drone aerial survey data includes road geometry (extracting geometric parameters such as road slope and curvature from 3D point cloud data), lane line features (extracting the precise position, width, and direction of lane lines from camera images), and traffic object trajectory features (extracting the speed, acceleration, and position information of traffic objects from camera data). The feature extraction from intelligent tire sensing data includes road friction coefficient features (extracting a real-time estimate of the road friction coefficient from the tire acceleration signal) and road roughness features (extracting the frequency and amplitude characteristics of road roughness from the tire's vertical acceleration signal).

[0034] The data fusion unit uses the Kalman filter algorithm or multi-sensor data fusion method to fuse the UAV aerial survey data and the smart tire perception data to generate a complete road scene model. It can also use the deep learning fusion method to build a deep learning model (such as the long short-term memory network, LSTM), take the UAV aerial survey data and the smart tire perception data as input, train the model to learn the correlation features between the two, and output the fused road scene features.

[0035] 4. Scenario analysis and risk assessment module.

[0036] The scene construction module constructs a complete road scene model based on the fused data, including information such as road topology, traffic flow, and road surface microscopic characteristics.

[0037] The risk identification module identifies specific risk scenarios, such as low-adhesion road surfaces (black ice, slippery roads, etc.) and wading (wading depth, water accumulation areas, etc.).

[0038] The risk assessment module evaluates the identified risk scenarios, determines the risk level, and generates corresponding early warning information.

[0039] 2. Implementation process

[0040] like Figure 2 As shown in the figure, the scene construction method based on UAV aerial survey and intelligent tire perception includes: 1. Data collection.

[0041] UAV aerial survey data collection: Plan the drone’s flight path and mission based on the target area to ensure coverage of all lanes and critical areas.

[0042] The drone flies according to the planned path, using high-resolution cameras to collect road images and lidar to collect three-dimensional point cloud data.

[0043] The collected image data is denoised and enhanced to improve image quality; the point cloud data is downsampled and filtered to remove noise points; target detection and trajectory tracking are performed based on the image data to extract the motion characteristics of traffic targets.

[0044] Image processing algorithms are used to extract road boundaries and lane lines from images. The three-dimensional geometric model of the road is generated by combining the point cloud data of the lidar, and geometric parameters such as the slope and curvature of the road are extracted. The Kalman filter algorithm is used to smooth the trajectory of traffic targets and extract their motion characteristics.

[0045] Intelligent tire perception data collection: Plan the driving path of the data collection vehicle to ensure that the vehicle repeatedly drives in different lanes of the target area and different positions of the same lane to collect comprehensive road surface data.

[0046] A vehicle equipped with smart tires travels along a planned path, and tactile sensors collect acceleration signals from the tires.

[0047] Filter the acceleration signal to remove high-frequency noise.

[0048] Using tire dynamics models and signal processing algorithms, the acceleration signal is converted into physical quantities (such as road friction coefficient and roughness) and calculated.

[0049] The road friction coefficient is estimated from the acceleration signal using the least squares method or Kalman filter algorithm. The frequency and amplitude characteristics of the road roughness are extracted from the vertical acceleration signal using the fast Fourier transform (FFT) algorithm, and the power spectrum density and root mean square value of the roughness are calculated.

[0050] 2. Data processing and fusion.

[0051] The drone aerial survey data and smart tire perception data are synchronized in time and aligned in space to ensure that the data from the two can match.

[0052] The road geometry, lane markings, and traffic object trajectory information obtained from drone aerial surveys are integrated with road friction coefficient and roughness information sensed by intelligent tires. Kalman filtering algorithms or deep learning models (such as LSTM) can be used for data fusion to generate a complete road scene model.

[0053] Based on the fused data, key scenarios are constructed, including information such as road topology, traffic flow, and road surface microscopic characteristics.

[0054] 3. Scenario analysis and risk assessment.

[0055] Identify risk scenarios such as low-adhesion roads (such as black ice and slippery roads), water-wading scenarios (such as water depth and water accumulation areas), uneven roads (such as mud and sand), and complex traffic flow scenarios (such as congestion and intersections).

[0056] For complex traffic flow scenarios, a driving safety field can be combined to assess the driving risk of a vehicle within the traffic flow. From a scenario layering perspective, this comprehensively considers the acquired road structure layer (road material, topography, etc.), road facility layer (lane markings, traffic lights, speed limit signs, etc.), temporary event layer (road maintenance, accident handling, etc.), traffic participant layer (vehicles, pedestrians, etc.), environmental layer (rain, snow, light intensity, etc.), network information layer (V2V, V2N and other wireless communication information and GPS, BeiDou and other satellite positioning information), as well as driver and passenger status (such as driver fatigue, distraction, passenger distribution, etc.) and vehicle status (such as current speed, acceleration, braking status, steering angle, lighting usage, vehicle fault information, etc.). The driving safety field is constructed to accurately determine traffic risk by analyzing the vehicle's interactions with surrounding traffic participants and hazardous objects in the map. This is because the driving risk of the traffic flow in which the vehicle is located will have a significant impact on the vehicle's control and driving strategy optimization. Incorporating it into the risk assessment system can make the risk identification of complex traffic flow scenarios more comprehensive and accurate, and provide a more complete basis for subsequent risk level assessment and warning information output.

[0057] The risk level is assessed based on the identified risk scenarios. For example, for low-adhesion roads, if the road friction coefficient is below a certain threshold, it is determined to be a high-risk scenario; for wading scenarios, if the wading depth exceeds the vehicle's safe wading depth, it is determined to be a high-risk scenario.

[0058] 4. Offline dataset construction.

[0059] Data collection is performed using a data collection vehicle equipped with smart tires and a drone. The vehicle drives through different lanes and locations in the target area, collecting microscopic road surface characteristics; the drone simultaneously collects macroscopic traffic information.

[0060] The collected data is processed by the data processing and fusion module to construct a key scenario dataset. This dataset contains macroscopic traffic information (road geometry, lane markings, and traffic object trajectories) and microscopic road surface characteristics (such as road friction coefficient, roughness, and wading depth). Its high diversity and practicality can be used to train the autonomous driving system's perception models, path planning algorithms, and dynamic control strategies, improving the system's robustness and adaptability.

[0061] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0062] Therefore, the present invention adopts the above-mentioned scene construction method and system based on drone aerial survey and intelligent tire perception, integrating drone macro observation and intelligent tire micro perception to provide comprehensive road condition information for the automatic driving system.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A scene construction system based on drone aerial survey and intelligent tire perception, characterized by: include: The drone aerial survey module collects macroscopic traffic information, including road geometry, lane lines, and traffic object trajectories; An intelligent tire sensing module, comprising a tactile sensor installed inside the tire, a data acquisition unit, a tire data processing unit, and a data transmission unit; The data processing and fusion module receives data from the drone aerial survey module and the intelligent tire perception module, and constructs key scenarios including road topology, traffic flow, and road surface micro-characteristics with semantic attributes through data fusion; The scenario analysis and risk assessment module identifies risk scenario types, assesses risk levels, and outputs warning information.

2. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 1 is characterized in that: The UAV aerial survey module includes a UAV platform and a data processing submodule.

3. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 2 is characterized in that: The drone platform is equipped with cameras and lidar.

4. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 2 is characterized in that: The data processing submodule performs: Road geometry extraction: Using image processing algorithms to extract road boundaries and lane lines from camera images, combined with LiDAR point cloud data, a 3D reconstruction algorithm is used to generate a 3D geometric model of the road. Road semantic map generation: Obtain road semantic information based on the OSM map, or after obtaining the traffic target trajectory, use trajectory classification to obtain the routes to which different trajectories belong and construct road semantic information; Lane detection: Use deep learning algorithms to detect lane lines in camera images; Traffic target trajectory tracking: The dynamic and static information of traffic targets is acquired through the camera, and the shape, size, position, speed and acceleration of the vehicle are obtained through target detection, target tracking and Kalman filter post-processing steps.

5. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 1 is characterized in that: The tactile sensor is a three-axis acceleration sensor. The data acquisition unit filters and normalizes the data collected by the tactile sensor. The tire data processing unit calculates the road friction coefficient and road roughness. The data transmission unit transmits the data to the data processing and fusion module.

6. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 5 is characterized in that: The road friction coefficient is calculated based on the tire dynamics model and longitudinal / lateral acceleration; the road roughness is characterized by the power spectral density and root mean square value of the vertical acceleration.

7. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 1 is characterized in that: The data processing and fusion module uses the Kalman filter algorithm or deep learning model to integrate the macro road information obtained from drone aerial surveys with the micro road surface characteristics of smart tires.

8. The scene construction system based on drone aerial survey and intelligent tire perception according to claim 1 is characterized in that: The scenario analysis and risk assessment module identifies at least one of the following risk scenarios: low-adhesion road surface, wading scene, uneven road surface, and complex traffic flow scene.

9. A scene construction method based on drone aerial survey and intelligent tire perception, characterized in that: include: Step S1: Plan the flight path of the UAV, synchronously collect road images and lidar point cloud data, perform image denoising and lane line detection on the images, perform 3D reconstruction on the point cloud to extract road slope and curvature, and track the trajectory of traffic targets based on camera data; Step S2: planning the vehicle's driving path, collecting signals through tactile sensors, estimating the road friction coefficient based on the tire dynamics model, and calculating the road roughness through fast Fourier transform analysis; Step S3: Time-synchronize and spatially align the drone data and tire data, integrate road geometry, traffic object trajectories, road friction coefficient, and road roughness, and construct a key scene dataset containing risk labels.

10. The scene construction method based on drone aerial survey and intelligent tire perception according to claim 9, characterized in that: The dataset is used to train autonomous driving perception models or verify control strategies; When applying, execute according to the risk level: Low risk: prompt the driver to pay attention; Medium risk: autonomous driving system adjusts strategy; High risk: Instruct to park or switch to safety mode.

Images