Server-Side 3D Mesh Generation from RGB-Depth Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems on devices like smartphones and tablets generate point clouds relative to the device's position, limiting the ability to create and manipulate 3D models of environments accurately and interactively, especially when moving or viewing from different perspectives.
Innovation Solution
A system that captures images and AR data, using algorithms to generate a 3D mesh on a server, which can be textured and manipulated, allowing for interactive 3D model generation and expansion as the device moves, and enabling remote access and modification by service professionals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If 3D mesh generation is performed on mobile devices, then processing speed and interactivity are improved, but device power consumption and computational load increase
Solution Approach 1:
The system divides 3D mesh generation into two segments: initial mesh generation is performed on mobile devices for quick results, while refined mesh generation and complex processing are offloaded to cloud servers. This segmentation allows mobile devices to achieve interactivity without excessive power consumption by handling only the essential initial processing locally.
Solution Approach 2:
The patent introduces a cloud server as an intermediary between mobile devices and the 3D mesh generation process. The server handles computationally intensive tasks while mobile devices perform lighter preprocessing and postprocessing, enabling fast 3D model generation without overloading device resources or consuming excessive power.
2Manufacturing precision
If detailed 3D meshes are generated from point clouds, then manufacturing precision and model accuracy are improved, but computational complexity and processing time increase
Solution Approach 1:
The mesh generation process is segmented into multiple stages: point cloud processing, initial mesh generation, and refined mesh generation. Each stage processes data at appropriate levels of detail, with complex refined mesh generation performed on cloud servers rather than mobile devices, reducing computational complexity while maintaining model accuracy.
Solution Approach 2:
The system applies partial processing on mobile devices (preprocessing and postprocessing) and excessive processing on cloud servers (refined mesh generation). This distributed approach ensures high model accuracy is achieved through comprehensive processing without requiring full computational complexity on mobile devices.
Data Source
AI summary
Embodiments include systems and methods for generating a 3D mesh on a server from one or more images and other data of a physical space that is captured by a device. The device may capture images with RGB data that includes depth information for pixels of the one or more images. The server may then create or update a 3D mesh based upon the images and depth information. The server may also use received images to provide textures to the various surfaces of the 3D mesh. The server may then cause a 3D model of the physical space to be viewed by another device, either by transmitting the 3D mesh for the other device used to construct a 3D model, or by rendering the 3D model directly on the other device. Other embodiments may be described and/or claimed.