Digital human collaborative interaction system in multimode microenvironment and virtual-real mapping method

Abstract

The invention provides a digital human collaborative interaction system in a multimode microenvironment and a virtual-real mapping method, which are used for driving physical environment feedback through digital human behaviors and realizing accurate virtual-real space mapping by utilizing an SLAM (Simultaneous Localization and Mapping) technology and edge calculation. The system has the innovation points of 1) a digital human-environment coupling architecture supporting five-sense linkage, 2) a low-cost consumption-level hardware adaptation scheme and 3) an emotion-environment two-way adjustment algorithm, is suitable for the fields of fitness, education, entertainment and the like, and promotes element universe interaction to evolve to high immersion and high tolerance.

Description

Technical Field

[0001] The present invention belongs to the field of metaverse interaction technology and intelligent environment control, and specifically relates to a collaborative system integrating a multimodal microenvironment generator and a digital human manifestation device. Through real-time environmental data synchronization, digital human behavior driving, and virtual-real mapping algorithms, it realizes a multi-sensory interaction experience between users and digital humans in the real physical space, and solves the problems of the disconnection between virtual and real environments and the single interaction feedback in the prior art. Background Art

[0002] The prior art has the following defects: Disconnection between virtual and real environments: Traditional metaverse devices (such as VR headsets) only provide audiovisual feedback, lacking multi-modal environment collaboration such as touch and smell, resulting in insufficient immersion; Limitations in digital human interaction: Existing digital human technologies (such as patent CN119400152A) can generate dynamic expressions, but cannot be linked with the physical environment, and users cannot perceive the impact of digital human actions on the real environment; Insufficient cross-device collaboration: Independent environment control devices (such as smart air conditioners and aromatherapy machines) and digital human systems lack a unified driving logic, and users need to manually switch modes; The present invention proposes a virtual-real environment coupling architecture, which drives microenvironment parameters through digital human behavior and maps virtual interactions to the real physical space to achieve a full closed-loop interaction of "digital human - environment - user". Summary of the Invention

[0003] The technical solution of the present invention includes a system architecture composed of a digital human manifestation device and a multi-modal microenvironment generator, and a core algorithm composed of a digital human - environment driving model and a multi-modal synchronization engine.

[0004] The digital human manifestation device includes a holographic projection module, a motion capture unit, and an emotion computing engine. Holographic projection module: Generates a naked-eye 3D digital human image based on a Fresnel lens array (resolution 8K, refresh rate 120Hz); Motion capture unit: Integrates millimeter-wave radar and TOF sensors to real-time track user gestures and positions (accuracy ±1 cm); Emotion computing engine: Dynamically adjusts the digital human behavior strategy through facial micro-expression recognition (referencing the Hefei artificial intelligence 3DMM technology) and voice emotion analysis.

[0005] The multi-modal microenvironment generator includes an environmental parameter library, a virtual-real mapping module, and a cross-device protocol. Environmental parameter library: Encapsulates atomic parameters such as smell, temperature and humidity, wind field, and light effect; Virtual-real mapping module: Converts digital human actions into environmental instructions (such as waving hands → increased wind speed, gazing → local temperature rise); Cross-device protocol: Supports Bluetooth 5.3, Wi-Fi 6E, and Li-Fi transmissions, and is compatible with home and commercial scenario deployments.

[0006] The behavior-environment mapping rules are defined in the digital human-environment driving model: python

if the digital human's action == "waving": the environmental parameter ["wind_speed"] += 2m / s, the environmental parameter ["scent"] = "ocean"; elif the user's heart rate > 120bpm: the environmental parameter ["temperature"] -= 2℃

[0007] The core innovations of this invention are reflected in three aspects: dynamic coupling of virtual and real environments, cross-modal emotion enhancement, and low-cost virtual-real mapping. Dynamic coupling of virtual and real environments: The world's first interactive logic of digital human actions driving physical environment changes (such as the digital human lighting a virtual bonfire → the real environment warms up + the release of pine scent); Compared with the patent CN119292466A (only supporting digital human voice interaction), this invention extends to multi-sensory dimensions such as touch and smell.

[0008] Cross-modal emotion enhancement: The emotional state of the digital human (such as joy, tension) is manifested through environmental parameters (joy → warm light + floral scent, tension → cold wind + metallic smell); The user's physiological data (heart rate, brain waves) reversely adjusts the digital human's behavior (such as when the user is anxious, the digital human switches to the soothing mode).

[0009] Low-cost virtual-real mapping: Using a consumer-grade projector (such as XGIMI H6) and an open-source SLAM algorithm (ORB-SLAM3), the hardware cost is significantly reduced.

[0010] The industrial design description of this invention is as follows. In terms of modular design: The digital human display device can be disassembled into a projection unit and a sensing module, which can adapt to different space sizes. The micro-environment generator supports plug-and-play capsule replacement (odor / atomizer); In terms of safety and compliance: The odor release concentration complies with the ISO 16890 air purification standard, and the emergency braking protocol (automatically shuts down when the temperature > 45℃ or CO 2 > 1500ppm).

[0011] The comparison between this invention and the existing invention CN119400152A is shown in the following table (see Table 1).

[0012] Table 1: Technical comparison and advantages Comparison item Prior art (CN119400152A) The present invention Interaction dimension Vision + voice Five-sense full modality + environment-driven Virtual-real coupling ability No physical environment linkage Digital human actions drive temperature / humidity / wind field in real time Deployment cost High-end equipment Consumer-grade hardware Description of the drawings

[0013] Figure 1: System architecture diagram (Digital human appearance device + Microenvironment generator + Cloud collaboration); Figure 2 : Virtual-real environment mapping flowchart (Digital human movement → Environmental parameter generation → Physical feedback); Figure 3 : Cross-device communication protocol stack (Li-Fi / Bluetooth / Wi-Fi multimode transmission); Figure 4 : Pseudo-code for emotion-environment coupling algorithm. Detailed implementation manners

[0014] Example 1: Virtual fitness coach (Real person + Digital human). Scene: The user conducts boxing training with the digital human coach. System response: The digital human punches → The microenvironment generator releases a 5m / s headwind (simulating resistance); When the user's movement deviation > 20%, the pneumatic unit of the seat applies a 50N reverse force; At the end of the training, the environment switches to the hot spring mode (38°C hot air + lavender fragrance).

[0015] Example 2: Remote collaborative puzzle-solving (Real person + Real person + Digital human). Scene: User A (in Shanghai) and User B (in London) crack the secret room mechanism through the digital human assistant. System response: A triggers the virtual flame → The environmental temperature of B rises to 40°C + firelight special effect; The digital human touches the password disk → The devices at both places vibrate synchronously to prompt; After the decryption is successful, the environment releases celebration light effects (RGB strobing) and champagne smell.

[0016] Example 3: Multi-person virtual travel (Digital human tour guide + Real person tourists). Scene: The user group explores the virtual rainforest, and the digital human tour guide explains ecological knowledge. System response: The digital human points to the tree → Locally release the pine fragrance + The humidity rises to 85%; The user touches the virtual waterfall → The hand tactile unit simulates the impact of water mist; When a rare plant is discovered, the environmental light effect focuses on a green spotlight.

Claims

1. A digital human collaborative interaction system and virtual-reality mapping method in a multi-mode micro-environment, characterized in that It includes a digital human display device, a multimodal microenvironment generator and a virtual-reality mapping engine, wherein: the digital human display device drives the change of environmental parameters through motion capture and emotional computing; the microenvironment generator maps virtual interaction to the physical space to achieve real-time feedback of odor, temperature and humidity, and wind field; the virtual-reality mapping engine calibrates the coordinates of the digital human and the physical space through the SLAM algorithm, and supports multi-user cross-device collaboration.

2. The system according to claim 1, characterized in that The digital human display device integrates holographic projection and millimeter-wave radar, supporting naked-eye 3D display and sub-centimeter motion tracking.

3. The system according to claim 1, characterized in that The virtual-reality mapping engine includes a behavior-environment mapping rule library and supports user-defined interaction logic (such as gesture triggering the release of a specific smell).

4. The system according to claim 1, characterized in that Introduce edge computing nodes to optimize data synchronization delay and ensure that parameter errors in multi-user environments are less than 5%.

5. The system according to claim 1, characterized in that The Li-Fi protocol enables interference-resistant data transmission in high-density scenarios (rate ≥ 10Gbps).

Images