Road Surface Classification Using Multi-Sensor Neural Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining road surface conditions lack accuracy and reliability, which affects the safety and performance of both autonomous and non-autonomous vehicles.
Innovation Solution
A method utilizing data from sensors of different types, such as optical cameras and lidar, to extract features using neural networks, fuse the data, and classify it into predefined road surface conditions, enhancing accuracy and computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data from multiple sensor types is fused, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent combines data from multiple sensor types (optical cameras, lidar, radar, infrared sensors) to fuse comprehensive road surface condition information. This merging of heterogeneous sensor data sources enables accurate classification of road conditions (dry, wet, snowy, icy, slushy) by leveraging the complementary strengths of each sensor type, thereby resolving the contradiction between improved measurement precision and increased device complexity through systematic data integration
Solution Approach 2:
The system employs a multi-functional data processing architecture that handles multiple sensor types and multiple road surface conditions using a unified classification framework. The neural network classifier is designed to process diverse input data formats and output multiple condition classifications, making the system universally applicable to various sensor configurations and road conditions without requiring separate specialized systems for each sensor type or condition
2Measurement precision
If feature extraction techniques are applied, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary feature extraction on raw sensor data before classification, pre-processing the data to extract relevant characteristics (such as optical properties, geometric features, intensity values) that are indicative of road surface conditions. This preliminary action prepares the data in advance for rapid classification, improving measurement precision while managing processing time by organizing features before the actual classification decision is required
Solution Approach 2:
The patent extracts specific relevant features from large volumes of raw sensor data, isolating only the most discriminative characteristics needed for road surface condition classification. By taking out only the essential features (such as reflectivity, texture, color intensity) rather than processing all raw data, the system improves measurement precision while reducing the computational burden and time loss associated with processing unnecessary data
3Reliability
If multiple sensors are used, then reliability is improved, but device complexity increases
Solution Approach 1:
The patent merges data from multiple sensor types (optical cameras, lidar, radar, infrared) into a unified road surface condition assessment system. By combining these sensors and their data streams, the system achieves improved reliability through cross-validation and complementary information, where the failure or limitation of one sensor can be compensated by others, thereby resolving the contradiction between enhanced reliability and increased device complexity through integrated multi-sensor operation
Data Source
AI summary
A method for determining a road surface condition, including obtaining first data and obtaining second data. The first data includes a representation of the road surface and originates from a sensor of a first type. The second data includes a representation of the road surface and originates from a sensor of a second type. The method further includes generating third data by applying a feature extraction technique on the first data and generating fourth data by applying a feature extraction technique on the second data. Additionally, the method includes generating fifth data by fusing the third data and the fourth data and determining the road surface condition by classifying the fifth data in at least one class of a set of predefined classes. Furthermore, a method for controlling a vehicle is presented. Also, a method for training a combination of artificial neural networks is described.