3D Sensor Data Visualization With Synchronized Multi-Viewport Inspection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methodologies for remote review of previously captured image and sensor data are limited by the quality and fidelity of the data, leading to 'lossy' 3D point clouds and insufficient detail for contextual information, especially when magnification is required, which can result in incomplete or inaccurate inspections of objects and features.
Innovation Solution
A method that synchronizes sensor data into a single coordinate system, allowing for concurrent display of 2D and 3D information on a user interface, enabling navigation and real-time updating of viewports to provide detailed, contextually relevant information from any perspective, using synthetic data generation when actual data is lacking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sensor data is captured from multiple sources and perspectives, then the quantity and coverage of information increases, but the complexity of data synchronization and coordinate system alignment increases
Solution Approach 1:
The patent segments sensor data from multiple sources (RGB cameras, LIDAR, thermal sensors) into separate data streams, each processed and transformed independently before integration. This segmentation allows complex multi-source data to be handled in manageable portions, reducing overall processing complexity while maintaining comprehensive data coverage.
Solution Approach 2:
The patent introduces a coordinate transformation system as an intermediary layer between raw sensor data and the final unified 3D representation. This intermediary performs coordinate system alignment and data registration, mediating the complexity of integrating heterogeneous sensor data while preserving the quantity and quality of information from all sources.
2Measurement precision
If 3D point cloud data is magnified to provide detailed views, then inspection detail improves, but data fidelity and accuracy are lost
Solution Approach 1:
The patent transitions from traditional 2D magnification of point cloud data to generating synthetic 2D images from the 3D point cloud data. This dimensionality change allows users to view detailed information in a different representation format without losing the underlying 3D data fidelity. The synthetic images provide inspection detail while the original 3D data remains intact for accurate measurements.
3Manufacturing precision
If synthetic RGB image data is generated from point cloud data, then data quality and detail are improved, but processing time and computational resources increase
Solution Approach 1:
The patent performs coordinate system transformation and data registration in advance, before synthetic image generation. This preliminary action prepares the point cloud data in an optimized format, reducing the computational burden during subsequent synthetic image generation and decreasing overall processing time while maintaining high data quality.
4Loss of information
If multiple viewports with different information types are displayed concurrently, then information completeness improves, but user interface complexity increases
Solution Approach 1:
The patent merges multiple information types (3D point cloud views, 2D synthetic images, measurement data, and inspection annotations) into a unified viewport system. This merging allows comprehensive information display while managing interface complexity through integrated presentation, where related data types are combined in coordinated views rather than separate complex panels.
Data Source
AI summary
The systems and methods herein provide improved methodologies for visualization on a user's display of sensor data (e.g., 2D and 3D information obtained/derived from sensors) for objects, components, or features of interest in a scene. Previously acquired sensor data is processable for concurrent display of objects/features/scene or location visualizations during real-time navigation of a scene camera during user visualization activities. Sensor data can be acquired via the operation of vehicles, or from other methodologies, or from any other suitable sensor data acquisition activities, etc. for which acquired sensor data can be visualized. The improved display of information provides significant benefits over prior art display methodologies and exhibits notable utility for user activities such as, inspection, condition assessment, performance assessment, insurance applications, construction, inventorying, building information modeling, asset management and the like. Information derivable from the methodologies herein can be used for machine learning libraries and digital twin processes.