System and method for generating high-fidelity digital avatars from multi-sensor 3D captures for virtual and augmented reality applications.

Abstract

The present invention relates to an innovative method and system for generating high-fidelity digital avatars from multi-source 3D capture data, facilitating user integration into immersive virtual reality (VR) and augmented reality (AR) environments. This method utilizes depth sensors such as LiDAR, multi-camera photogrammetry, and advanced geometric reconstruction algorithms based on deep learning, ensuring a faithful match to the user's physical characteristics. The invention is distinguished by its ability to produce 3D avatars optimized in terms of geometry and textures, while enabling real-time rendering and animation via a portable and accessible solution.

Description

Title of the invention: System and method for generating high-fidelity digital avatars from multi-sensor 3D captures for virtual and augmented reality applications. Summary of the invention

[0001] The present invention relates to an innovative method and system for generating high-fidelity digital avatars from multi-source 3D capture data, facilitating user integration into immersive virtual reality (VR) and augmented reality (AR) environments. This method uses depth sensors such as LiDAR, multi-camera photogrammetry, and advanced geometric reconstruction algorithms based on deep learning (Deep Learning), ensuring a faithful match to the user's physical characteristics. The invention is distinguished by its ability to produce 3D avatars optimized in terms of geometry and textures, while enabling real-time rendering and animation via a portable and accessible solution. technical field

[0002] The present invention belongs to the field of 3D capture technologies, digital avatar modeling and their integration into immersive VR / AR environments. It is primarily intended for the entertainment, e-commerce (virtual product trials) and virtual social interaction sectors, allowing the visualization of high-fidelity digital copies of users in various scenarios. Detailed description of the invention Technical context

[0003] The evolution of virtual and augmented reality technologies has made it possible to create new types of interactions where the use of realistic digital avatars is crucial. Current solutions generally rely on fixed 3D scanners or complex multi-camera setups, making the user experience expensive and not readily accessible to the general public. Furthermore, these systems often require modeling expertise and bulky equipment to produce a satisfactory result. It is therefore necessary to develop a simple and economical system for creating realistic avatars from common devices, such as smartphones or tablets. Technical problem to solve

[0004] The invention aims to overcome the drawbacks of current systems by proposing a 3D capture and modeling method that utilizes the capabilities of modern mobile devices. The objective is to provide a system that allows the creation of realistic digital avatars in a few simple steps, without requiring significant investment in specialized equipment. Technical solution

[0005] The proposed system includes:

[0006] A 3D capture module:

[0007] Integrated into a smartphone or tablet, it includes a Lidar depth sensor or a camera compatible with photogrammetry. Data acquisition is performed by scanning the user's body surface.

[0008] A digital processing module:

[0009] This module performs the geometric reconstruction of the 3D model from the captured points. It uses deep learning algorithms to correct artifacts and generate a realistic, high-definition texture.

[0010] A physical simulation module:

[0011] Integrated into the system, it dynamically adjusts the model so that the avatar reacts in real time to movements detected by wearable sensors (VR gloves, motion sensors) or integrated into the mobile device.

[0012] A standardized export format:

[0013] The created avatars can be exported in standard formats such as VRML, OBJ, or FBX for integration into various VR / AR platforms. Competitive advantages

[0014] The invention is distinguished by its portability and its ability to generate 3D models in real time without the need for expensive equipment. Unlike existing solutions, it can be used with consumer devices while maintaining professional visual quality. The integrated deep learning algorithm reduces artifacts commonly observed in 3D models generated by mobile devices, ensuring a superior immersive experience. Examples of applications

[0015] The invention makes it possible to:

[0016] - Create digital avatars for VR meeting platforms with copies realistic to encourage interaction;

[0017] - Facilitating the virtual testing of products (clothing, shoes, etc.) using exact copies of the users;

[0018] - Develop simulation or vocational training environments with avatars that react in real time to the user's voice commands or gestures. Conclusion

[0019] This invention offers a new solution for creating digital copies of users with ease of use and high visual fidelity. The system is adaptable to a variety of applications, thus increasing the possibilities for immersive interactions in virtual environments.

Claims

Demands

1. A method for creating a high-fidelity digital avatar of a user, the method comprising the following steps: (a) Acquisition of three-dimensional data of the user via an integrated depth sensor or a photogrammetry module associated with a mobile device (smartphone, tablet, etc.); (b) Reconstruction of a 3D geometric model of the user from this data using deep learning algorithms, allowing for the correction of imperfections and the generation of an optimized high-resolution mesh; (c) Application of a dynamic texture to the 3D model, said texture being generated by the synthesis of images captured from different perspectives in order to increase visual fidelity; (d) Export of the 3D model in a standard format (VRML, OBJ, FBX) for integration into virtual or augmented reality environments.

2. A method according to claim 1, characterized in that the 3D model is optimized by a polygonal complexity reduction algorithm, using geometric simplification techniques based on minimizing projection errors.

3. A method according to claim 1 or 2, characterized in that the user can be captured in motion or in a static position, enabling the acquisition of data for facial expressions, gestures and body postures through dynamic detection by a Lidar depth sensor or a stereoscopic vision camera.

4. 3D capture and modeling system comprising: (a) A depth sensor integrated into a mobile device; (b) An embedded processing module for real-time data calculation; (c) A rendering engine enabling compatibility with third-party graphics engines (Unreal Engine, Unity) and VR / AR platforms.

5. Capture device according to claim 4, characterized in that the system is compatible with virtual commerce applications

6. allowing the user to try on clothing, accessories, or sports equipment in real time via their digital avatar. The dynamic update process for the digital avatar is characterized by the continuous capture of the user's movements via external sensors or VR gloves, all integrated into the 3D model to faithfully reproduce each movement.