Sensing Image Generation for Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for vehicle external recognition, such as those using cameras and radars, face challenges in accurately detecting objects, especially at night or in adverse weather conditions, as they rely on image recognition and struggle to effectively utilize speed information from millimeter wave radars for object detection.
Innovation Solution
An information processing apparatus and method that generates a sensing image from sensor data including speed information, using a learned model to detect objects by projecting three-dimensional point clouds onto a two-dimensional plane and adding texture information based on speed, allowing for improved object detection even in challenging conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional image recognition methods are used for object detection, then the system can process visual data from cameras, but detection accuracy deteriorates in nighttime and adverse weather conditions
Solution Approach 1:
The patent merges radar data with camera images by generating a composite sensing image that integrates both modalities. The radar-derived speed information is overlaid on camera images through projective transformation, creating a unified representation that leverages radar's all-weather capability while maintaining camera's visual detail for accurate object detection across all conditions
Solution Approach 2:
The patent introduces a projective transformation as an intermediary mechanism to bridge radar coordinate systems and camera image planes. This transformation serves as a mediator that accurately maps radar-detected objects onto camera images, enabling seamless integration of speed information from radar with visual context from cameras
2Loss of information
If radar data is integrated with camera images through projective transformation, then speed information can be visualized on camera images, but the complexity of data processing increases
Solution Approach 1:
The patent performs projective transformation and radar-camera registration in advance during system initialization or calibration phase. By pre-computing the transformation matrix and aligning coordinate systems beforehand, the complex mathematical operations are prepared ahead of time, allowing real-time object detection to proceed with simpler, pre-configured processing steps
3Measurement precision
If deep neural networks are trained with traditional image data only, then the model can recognize objects visually, but the model fails to effectively utilize speed information for improved detection
Solution Approach 1:
The patent modifies the input parameters of deep neural networks by feeding them composite sensing images that contain both visual information from cameras and speed information from radar. This parameter change transforms the model's input from traditional images to enriched multi-parameter data, enabling the network to learn and utilize speed information directly for improved object detection and classification
Data Source
AI summary
Provided is an information processing apparatus that processes sensor data including speed information of an object. The information processing apparatus includes a generation unit that generates a sensing image on the basis of the sensor data including the speed information of the object, and a detection unit that detects the object from the sensing image using a learned model. The generation unit projects the sensor data including a three-dimensional point cloud on a two-dimensional plane to generate the sensing image having a pixel value corresponding to the speed information. The detection unit performs object detection using the learned model learned to recognize the object included in the sensing image.