Extending Short-Range Lidar Detection Range via Neural Network Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Short-range lidar sensors have limited measurement ranges, making them inadequate for high-speed driving scenarios where objects outside their range may not be detected in time, limiting their effectiveness in autonomous driving applications.
Innovation Solution
A method using a neural network that combines data from a short-range sensor with a long-range auxiliary sensor and a ground truth sensor to extend the effective range of the short-range sensor, allowing for the reconstruction of the 3D environment beyond its typical range, utilizing anchor points and sensor fusion techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of stationary object
If a short-range lidar sensor is used, then installation space requirements are reduced and cost is decreased, but the measurement range is limited making it inadequate for high-speed driving scenarios
Solution Approach 1:
The patent combines a short-range lidar sensor with a long-range auxiliary sensor (radar or camera) to create a hybrid sensor system. The short-range lidar provides high-resolution depth information for nearby objects, while the long-range auxiliary sensor detects objects at greater distances. A neural network fuses these heterogeneous data sources to extend the effective measurement range beyond what the short-range lidar could achieve alone, resolving the contradiction between compact sensor size and extended detection range.
Solution Approach 2:
The patent introduces a neural network as an intermediary component that processes and fuses data from the short-range lidar and long-range auxiliary sensor. This neural network acts as a mediator that translates the limited range output of the short-range lidar and the lower-resolution long-range data into extended effective range measurements, enabling the system to detect objects at distances beyond the native capability of the short-range sensor.
2Reliability
If a long-range lidar sensor is used, then the measurement range is extended for high-speed scenarios, but installation space requirements and cost increase
Solution Approach 1:
The patent replaces expensive, complex long-range lidar sensors with a combination of a inexpensive short-range lidar and a long-range auxiliary sensor (radar or camera). The short-range lidar is a more affordable, compact component that, when combined with the auxiliary sensor and processed through a neural network, achieves detection reliability comparable to or exceeding that of a dedicated long-range lidar, thereby reducing system complexity and cost while maintaining reliability.
Solution Approach 2:
The patent uses the long-range auxiliary sensor (radar or camera) to capture information about distant objects, which is then processed by a neural network to generate depth information that complements the short-range lidar data. This approach creates a virtual representation of the long-range environment using more affordable sensor types, effectively copying the functionality of a long-range lidar without the associated cost and complexity.
3Length of stationary object
If a short-range sensor is used, then cost and installation space are reduced, but objects outside the range cannot be detected in time for high-speed driving
Solution Approach 1:
The patent employs a long-range auxiliary sensor that continuously monitors the environment for objects at distances beyond the short-range lidar's detection capability. This preliminary detection allows the system to identify potential hazards earlier, providing advance warning before objects enter the short-range sensor's detection zone. The neural network processes this early warning information to prepare appropriate responses, ensuring sufficient response time even for high-speed driving scenarios.
Solution Approach 2:
The patent implements a feedback mechanism where the neural network continuously processes data from both the short-range lidar and long-range auxiliary sensor, dynamically adjusting the system's detection and response behavior. When the auxiliary sensor detects objects at long range, the feedback loop triggers early alert conditions that inform the vehicle control system, allowing for proactive response planning. This feedback-driven approach ensures that the system maintains adequate response time by anticipating the entry of objects into the critical short-range detection zone.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the production vehicle to detect objects and reconstruct the environment at longer ranges using less expensive short-range lidar sensors in conjunction with cost-effective complementary sensors, enhancing safety and reducing installation space requirements.
Implementation Method 1
range extension is done with a neural network, wherein the neural network is trained using following steps: detecting reflections, hits or objects in the surrounding of the training vehicle by using sensor data of second sensor setup
Implementation Method 2
determining an anchor point of an reflection or object which is out of a effective range of short-range sensor of second sensor setup by using anchor point measurement of long-range auxiliary sensor of second sensor setup
Implementation Method 3
detecting reflections, hits or objects in the surrounding of the training vehicle by using sensor data of second sensor setup
Data Source
Figure 1a
Figure 1b
Figure 2a~2b
AI summary
A method, preferably a computer-implemented method, for extending the detection range of a short-range sensor, and a corresponding system are provided. The method comprises a first sensor setup for a production vehicle (1), comprising a short-range sensor (3), a long-range auxiliary sensor (4) which provides an anchor point measurement, and a second sensor setup for a training vehicle (11), comprising a short-range sensor (13), a long-range auxiliary sensor (14) which provides an anchor point measurement, and a long-range ground truth sensor (16), wherein the sensors of the first and second sensor setup having a sensor range and providing sensor data for reflection detection, the range extension is done with a neural network, wherein the neural network is trained using the following steps: - Detecting reflections (20, 21, 22) in the surrounding of the training vehicle (11) by using sensor data of the second sensor setup, - Determining an anchor point (28) of a reflection (21) which is out of a detection range (26) of the short-range sensor (13) of the second sensor setup by using the anchor point measurement of the long-range auxiliary sensor (14) of second sensor setup, - Providing ground truth by determining the distances (25, 33) between the anchor point (28) and the other reflections (20, 22) by using the sensor data of the long-range ground truth sensor (16), and the method is executed on basis of the trained neural network using the following steps: - Detecting reflections (20, 21, 22) in the surrounding of the production vehicle (1) by using sensor data of the first sensor setup, - Determining anchor points (28) of a reflection (20, 21) by using the anchor point measurement of the long-range auxiliary sensor (4) of the first sensor setup, - Determining the distances (33) between the anchor point (28) and other reflections (22) in the detection range of the short-range sensor (3) of the first sensor setup by using the sensor data of the short-range sensor (3) of the first sensor setup, and - Inferring the distances (25) between the anchor point (28) and other reflections (20) out of the detection range of the short-range sensor (3) of the first sensor setup by using the provided ground truth of the trained neural network.