3D Sensor Data Visualization With Gap-Filling Scene Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methodologies for remote review of previously captured image and sensor data are limited by the quality and fidelity of 3D renderings, often resulting in lossy point clouds with missing information, which restricts the ability to view objects or features from desired perspectives and angles, and require significant mental effort to integrate 2D and 3D information, leading to reduced information availability for tasks.
Innovation Solution
A method that synchronizes sensor data into a single coordinate system, allowing for concurrent display of 2D and 3D information on a user interface with dynamic navigation, enabling the user to view objects from any perspective and automatically generate synthetic images to fill gaps in data, thereby enhancing the quality and completeness of the information displayed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If 3D renderings are generated from previously captured sensor data, then remote review capability is enabled, but information completeness deteriorates due to lossy point clouds and missing data
Solution Approach 1:
The patent transitions from traditional 2D image review to immersive 3D virtual reality environments. By capturing spatial data in three dimensions and rendering it in a VR space, the system enables remote reviewers to navigate and inspect objects from multiple angles simultaneously, achieving complete information viewing without physically being onsite while maintaining full data fidelity through volumetric representation rather than projected 2D images.
2Loss of information
If multiple 2D and 3D information displays are provided concurrently, then information availability improves, but user cognitive load increases requiring significant mental effort to integrate information
Solution Approach 1:
The patent merges multiple separate information displays (2D images, 3D point clouds, measurements, annotations) into a single unified virtual reality environment. All these data types are integrated into the same spatial context, allowing users to view them simultaneously from appropriate perspectives without needing to mentally correlate separate displays. The VR headset presents a cohesive immersive view where all information coexists in its proper spatial relationships.
Solution Approach 2:
By moving all information displays into the third dimension within the VR environment, the system eliminates the cognitive burden of integrating 2D representations. Measurements, annotations, and multiple views are positioned in 3D space according to their actual spatial relationships, allowing intuitive understanding without mental transformation between different 2D projections and perspectives.
3Measurement precision
If high-quality 3D renderings are generated, then viewing quality improves, but data processing complexity increases
Solution Approach 1:
The patent performs data processing and coordinate system transformations during the initial data capture and preprocessing phase, before the actual review occurs. Spatial data from multiple sensors is registered, aligned, and converted into a common coordinate system in advance. This preliminary processing creates a ready-to-render 3D dataset that can be efficiently displayed in VR without requiring complex real-time processing during the review session, thus maintaining high viewing quality while reducing operational complexity.
Data Source
AI summary
The systems and methods herein provide improved methodologies for visualization on a user's display of sensor data (e.g., 2D and 3D information obtained from or derived from sensors) for objects, components, or features of interest in a scene. The previously acquired sensor data is processable for concurrent display of objects/features/scene or location visualizations to a user during their real-time navigation of a scene camera during a variety of user visualization activities. Sensor data can be acquired via the operation of vehicles configured with one or more sensors, such as unmanned aerial vehicles, or from other methodologies, or from any other suitable sensor data acquisition activities. Objects etc. for which acquired sensor data can be visualized by a user on a display includes buildings, parts of buildings, and infrastructure elements, among other things. The improved display of information to a user for visualization and information generation therefrom provides significant benefits over prior art display methodologies and exhibits notable utility for user activities such as, inspection, condition assessment, performance assessment, insurance applications, construction, inventorying, building information modeling, asset management and the like. Information derivable from the methodologies herein can be used for machine learning libraries and digital twin processes.