Virtual reality interaction method for constructing multi-modal fusion based on application scene features

CN119473014APending Publication Date: 2025-02-18HARBIN AIWELL TECH CO LTD
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411607141.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-12
Publication Date
2025-02-18

AI Technical Summary

Technical Problem

The existing multimodal fusion technology faces large feature differences, data synchronization problems, and complex feature extraction and processing difficulties when processing data of different modalities, resulting in the inability to achieve ideal immersion and natural interactive experience.

Method used

A multimodal fusion method based on application scenario characteristics is adopted to enhance networks, spectrum and spatial attention optimization networks through cross-modal knowledge, and edge-guided learning networks to integrate and optimize visual, auditory and tactile features.

Benefits of technology

It effectively enhances the correlation and difference between different modes, optimizes the quality of fusion characteristics, improves the user's tactile perception experience and interaction nature, and is more flexible and adaptable than traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119473014A_ABST
    Figure CN119473014A_ABST
Patent Text Reader

Abstract

The invention discloses a virtual reality interaction method for constructing multi-modal fusion based on application scene features, and relates to the technical field of virtual reality. In order to solve the technical problem that in the prior art, when an existing multi-modal fusion technology is used for processing fusion of different modal data, the characteristic difference between different modals is large, the technical scheme provided by the invention comprises the following steps: acquiring and preprocessing multi-modal data; constructing a cross-modal knowledge enhancement network, extracting initial features, and enhancing the consistency and difference between visual and auditory features; important context information in fusion features is optimized and enhanced based on an optimization network of frequency spectrum and space attention; constructing an edge guide learning network, and combining the edge guide learning network to the tactile data; the visual, auditory and tactile features are fused and applied to a virtual reality scene; and optimizing the model through structural similarity loss, mean square error loss, significance weight loss and edge loss. The method is suitable for cooperative work of multi-modal features in a complex environment.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Thematic display method and system based on virtual reality and electronic equipment

    CN120122832A

  • Multi-camera imaging method and system for head-mounted device

    CN120128692A

  • Multi-camera imaging method and system for head-mounted device

    CN120128692B

  • Multi-modal video target identification method based on cross-modal spatio-temporal joint learning

    CN120236233A

  • VR interaction behavior prediction method based on AI

    CN120429585A