Neural Network Training Using Point Cloud Rays for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous driving systems face challenges in collecting sufficient data for rare scenes, leading to inferior processing capabilities of deep learning models, particularly in simulating and rendering complex, wide-range scenes involving moving vehicles.
Innovation Solution
A method for training neural network models that combines image data with point cloud information, utilizing the sparsity and registrability of point clouds to accurately represent wide-range backgrounds and moving objects, and a method for generating images that leverages these characteristics to produce accurate image information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep learning models are trained using only real-world road test data, then the models can process common scenes well, but they fail to handle rare scenes due to insufficient data collection
Solution Approach 1:
The patent creates virtual copies of rare scenes through simulation platforms. Instead of relying on scarce real-world data, the system generates synthetic training data by copying and reconstructing rare scene scenarios in a virtual environment, allowing the neural network to learn from numerous simulated examples rather than limited real occurrences.
Solution Approach 2:
The patent performs preliminary scene construction and data generation before actual training needs arise. By pre-building comprehensive scene databases including rare scenarios through simulation, the system ensures sufficient training data is available beforehand, eliminating the data scarcity problem during model training.
2Adaptability or versatility
If autonomous driving simulation platforms model high-speed moving vehicles with wide range scenes, then the simulation coverage is comprehensive, but the rendering complexity and computational requirements increase significantly
Solution Approach 1:
The patent segments the complex rendering task into separate processing stages: scene construction, point cloud generation, and neural network training. By dividing the wide-range scene simulation into modular components, the system manages rendering complexity more effectively while maintaining comprehensive scene coverage.
Solution Approach 2:
The patent replaces traditional geometric rendering mechanisms with neural network-based synthesis. Instead of using complex physical rendering engines to simulate wide-range scenes, the system trains a neural network to generate scene representations, substituting mechanical rendering processes with learned patterns that reduce computational complexity.
3Ease of manufacture
If traditional data collection methods are used for autonomous driving, then the data collection process is simple, but sufficient data for rare scenes cannot be collected
Solution Approach 1:
The patent creates virtual copies of rare scenes through simulation platforms. Instead of relying on scarce real-world data, the system generates synthetic training data by copying and reconstructing rare scene scenarios in a virtual environment, allowing the neural network to learn from numerous simulated examples rather than limited real occurrences.
Solution Approach 2:
The patent changes the data collection approach from physical road testing to virtual parameter-based scene generation. By adjusting simulation parameters to create diverse rare scenarios, the system generates sufficient rare scene data without the complexity of extended physical testing, maintaining process simplicity while increasing data quantity.
Data Source
AI summary
The present disclosure relates to a method for training a neural network model and a method for generating an image. The method for training a neural network model includes: acquiring an image about a scene captured by a camera; determining a plurality of rays at least according to parameters of the camera when capturing the image; determining a plurality of sampling points according to a relative positional relationship between the rays and a point cloud, where the point cloud is associated with a part of the scene; determining color information of pixels of the image which correspond to the sampling points; and training the neural network model according to positions of the sampling points and the color information of the pixels.