Dynamic 3D Environment Representation via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in generating high-fidelity three-dimensional representations of physical environments that include dynamic objects and scenarios efficiently, as they either rely on expensive professional-grade drones with high-fidelity sensors or low-fidelity consumer-grade drones that struggle to capture comprehensive data accurately.
Innovation Solution
A method combining 3D sensing devices (like LiDAR sensors) to capture high-fidelity 3D data and 2D sensing devices to collect dynamic object information, aligning the data to create a high-fidelity 3D representation of the environment, allowing for the integration of dynamic objects and scenarios with low operational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If professional-grade drones with high-fidelity 3D sensors (LiDAR, radar) are used, then measurement precision and manufacturing precision of environmental representation are improved, but device complexity and operational cost increase significantly
Solution Approach 1:
The system segments the sensing tasks by using different sensor types for different purposes: 2D sensors capture dynamic object information while 3D sensors capture static environmental structure. This division allows each sensor type to operate at optimal fidelity for its specific function, reducing overall system complexity while maintaining high measurement precision where needed.
Solution Approach 2:
The patent applies local quality by using high-fidelity 3D sensing only for static environmental elements that require precise geometric representation, while relying on lower-fidelity 2D sensing for dynamic objects. This selective application of sensing fidelity optimizes resource allocation and reduces device complexity without compromising overall representation quality.
2Reliability
If professional-grade drones with high-fidelity sensors are deployed regularly, then reliability of dynamic information gathering is improved, but operational cost increases
Solution Approach 1:
The system segments the data collection responsibility between 2D and 3D sensing devices, allowing consumer-grade drones equipped with simpler 2D sensors to perform dynamic object tracking reliably, while professional-grade drones with complex 3D sensors are used only when high-fidelity static environmental mapping is required.
Solution Approach 2:
The patent creates a composite environmental representation by copying and integrating data from multiple sensor sources. The 2D sensor data is projected onto and aligned with the 3D environmental model, effectively copying dynamic object information into the high-fidelity framework without requiring continuous deployment of expensive professional-grade equipment.
3Measurement precision
If large volume of three-dimensional data is processed regularly, then measurement precision of environmental representation is improved, but productivity decreases due to resource consumption
Solution Approach 1:
The system extracts only the essential 3D structural information from the environment and represents it in a compact point cloud format. Dynamic objects are detected and tracked using efficient 2D image processing algorithms, extracting only their relevant motion and appearance features rather than processing full 3D data for all objects.
Solution Approach 2:
The patent projects 2D dynamic object detections into the 3D environmental space, effectively translating problems from one dimension to another. This allows the system to leverage efficient 2D image processing for dynamic objects while maintaining accurate 3D representation, avoiding the computational burden of processing full 3D data for all elements.
4Device complexity
If consumer-grade drones with low-fidelity sensors are used, then device complexity and operational cost are reduced, but measurement precision of environmental representation deteriorates
Solution Approach 1:
The system merges data from 2D and 3D sensors to create a comprehensive environmental representation. The 3D sensor provides accurate depth and structural information for static elements, while the 2D sensor captures dynamic objects and their motion. By combining these complementary data sources, the system achieves high measurement precision for both static and dynamic elements using a mixed-sensor platform.
Solution Approach 2:
The patent uses an alignment and projection mechanism as an intermediary to integrate 2D and 3D data. The 2D sensor data is aligned with the 3D environmental model through pose estimation and projection, allowing low-fidelity 2D measurements to be accurately positioned within the high-fidelity 3D framework, effectively bridging the fidelity gap between sensor types.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the generation of comprehensive, high-fidelity 3D representations of environments with dynamic objects and scenarios, enhancing applications such as autonomous driving and surveillance while reducing operational costs by leveraging both high-fidelity and low-fidelity sensor data effectively.
Implementation Method 1
the set of three-dimensional data comprises one or more scanned points by a LiDAR sensor of the first sensing device
Data Source
AI summary
The present invention generally relates to generating a three-dimensional representation of a physical environment, which includes dynamic scenarios. An exemplary device comprises one or more processors; a memory; and one or more programs that includes instructions for: obtaining a set of three-dimensional data of the physical environment, wherein the three-dimensional data is associated with a first sensing device; based on the set of three-dimensional data, generating a three-dimensional representation of the physical environment; obtaining a set of two-dimensional data of the physical environment, wherein the set of two-dimensional data is associated with a second sensing device and wherein the set of two-dimensional data comprises information of a dynamic object in the physical environment; generating an alignment between the three-dimensional representation of the physical environment and the set of two-dimensional data; and based on the alignment, obtaining a set of three-dimensional information associated with the dynamic object.