A VR space-time crossing interaction and AR real scene fusion system with text input linkage

Abstract

The application discloses a VR space-time crossing interaction and AR real scene fusion system with a text, and belongs to the technical field of virtual reality and augmented reality graphics processing. The application extracts scene tags in the text through a text semantic analysis module, automatically matches and calls VR historical scene resources; relies on a space coordinate unified mapping module to construct a virtual-real space conversion matrix, realizes alignment of a virtual scene and an AR real scene position; realizes VR virtual picture and real scene picture layer superposition through a virtual-real layer fusion module; and realizes virtual-real space scene smooth switching through a space-time transition rendering module. The application solves technical problems of traditional VR and AR systems, such as mutual independence, inability to automatically dispatch scenes by literary texts, virtual-real space dislocation, and scene switching stiffness, and can realize virtual-real fusion immersive travel interaction experience driven by the text, has good modularity architecture expansibility, and has engineering practical value.

Description

Technical Field

[0001] This invention belongs to the field of computer technology of virtual reality, augmented reality spatial graphics fusion text-driven scene scheduling, specifically involving an interactive system that automatically schedules VR virtual historical scenes based on the semantic analysis of ancient Chinese texts, and realizes the matching of virtual images with real-world AR spatial coordinates, layer fusion, and smooth switching of time and space scenes. Background Technology

[0002] In the field of immersive cultural tourism experiences, VR virtual scene systems and AR real-scene display systems have long operated independently, resulting in several technical shortcomings: 1. Traditional VR scenes rely on manual selection and retrieval of scene resources, failing to automatically trigger corresponding historical and spatial scenes based on literary text content, and lacking an automated scene scheduling mechanism driven by textual semantics; 2. VR virtual spacetime images and on-site AR real-scene images belong to two independent rendering pipelines, making it difficult to achieve a unified spatial coordinate system mapping and seamless overlay and fusion of virtual and real images; 3. There is no dynamic transition rendering logic for "time travel," resulting in abrupt screen transitions when switching from real-scene AR to historical VR virtual scenes, disrupting the immersive experience; 4. The geographical features and temporal scene information described in the text cannot be automatically parsed and mapped to 3D scene parameters, making scene construction and matching highly dependent on manual configuration, leading to high development costs and poor scalability. Existing solutions only address independent VR displays or independent AR real-scene overlays, lacking an integrated technical architecture that integrates textual semantic linkage, unified mapping of virtual and real coordinates, and smooth transitions between spacetime scenes. Therefore, new technical solutions are urgently needed to address these pain points. Summary of the Invention

[0003] The technical problems to be solved are: overcoming the technical defects of existing VR and AR systems, such as being independent of each other, being unable to automatically drive scene scheduling by text, difficulty in spatial matching between virtual scenes and real scenes, abrupt transitions between virtual and real scenes, and high costs of manual configuration of scene parameters; and providing an integrated computer system to realize text semantic parsing to drive the loading of VR historical scenes, unified spatial coordinate mapping, fusion of VR virtual images and AR real scene layers, and smooth transitions between virtual and real spatiotemporal scenes.

[0004] Technical Solution: A VR time-space travel interactive and AR real-scene fusion system with text-based linkage, characterized by comprising: a text semantic parsing module, a scene resource indexing module, a unified spatial coordinate mapping module, a VR virtual scene rendering module, an AR real-scene acquisition module, a virtual-real layer fusion module, a time-space transition rendering module, and an interactive response module; the text semantic parsing module is used to segment and recognize entities from the input text, extract geographical location, time sequence, and scene tags of objects, and output structured scene scheduling instructions; the scene resource indexing module receives the scene scheduling instructions and retrieves pre-stored corresponding historical time-space VR 3D scene resources based on tag matching; the unified spatial coordinate mapping module establishes the real-world AR real-scene geographical coordinates and the VR virtual historical scene 3D coordinates. The transformation matrix between coordinates completes the alignment of virtual and real spatial positions; the VR virtual scene rendering module loads the successfully matched 3D scene resources, completes spatial positioning according to the transformation matrix, and outputs VR virtual image data stream; the AR real scene acquisition module calls the terminal camera device to acquire real-time images of the scene and outputs the original real-scene image; the virtual and real layer fusion module overlays the VR virtual image layer onto the AR real scene image layer according to the transparency weight parameter to generate a virtual and real fused image; the spatiotemporal transition rendering module is used to perform gradual transition rendering from real-scene AR to VR virtual spacetime and from VR virtual spacetime back to real-scene AR, outputting a smooth switching image; the interactive response module receives user control commands, adjusts the fusion weight and viewpoint parameters in real time, and responds to user roaming interaction operations.

[0005] Beneficial technical effects: 1. Automatically generating scene scheduling instructions through semantic parsing of text eliminates the need for manual scene selection, enabling text-driven automatic loading of immersive scenes; 2. Setting up a unified spatial coordinate mapping module and constructing a virtual-real space transformation matrix solves the technical problem of misalignment between VR virtual scenes and AR real scenes; 3. The virtual-real layer fusion module supports dynamic transparency adjustment, achieving natural overlay of virtual objects and real scenes; 4. The spatiotemporal transition rendering module provides gradient transition effects, eliminating abrupt transitions between virtual and real scenes and enhancing immersion; 5. The entire architecture is modular and decoupled, requiring only expansion of the material library when adding new text entries or historical scene resources, without requiring system reconstruction, demonstrating strong scalability.

[0006] Detailed Implementation: A VR time-space travel interactive and AR real-scene fusion system with text-based linkage, in which each module operates collaboratively, is implemented in the following steps: Step 1: Import the text into the text semantic parsing module. The module performs text entity recognition, extracts regional, era, and landscape tags, and generates structured scene scheduling instructions; Step 2: The scene resource indexing module uses tags to search the material database and retrieves the corresponding historical time-space VR 3D scene data package; Step 3: The spatial coordinate unified mapping module reads the terminal's GPS and spatial positioning data and constructs a transformation matrix between AR real-scene coordinates and VR virtual 3D coordinates; Step 4: The VR virtual scene rendering module loads 3D scene resources and, based on the transformation matrix... Step 5: The AR real-scene acquisition module continuously acquires real-scene images from the terminal camera and outputs a real-scene image stream; Step 6: The virtual-real layer fusion module configures the layer overlay transparency parameters, overlays the virtual image layer on top of the real-scene layer, and generates a virtual-real fusion image; Step 7: When the time-space travel interaction command is triggered, the time-space transition rendering module starts the gradient rendering algorithm, gradually adjusts the layer transparency to complete the smooth transition between AR real-scene and VR virtual time-space scene; Step 8: The interaction response module continuously acquires user viewpoint movement and roaming operation commands, updates rendering parameters in real time, dynamically adjusts the virtual-real fusion image, and outputs it to the display terminal.

[0007] Figure 1 is a block diagram of the overall system architecture of the present invention; labels: 1-Text semantic parsing module; 2-Scene resource indexing module; 3-Spatial coordinate unified mapping module; 4-VR virtual scene rendering module; 5-AR real scene acquisition module; 6-Virtual-real layer fusion module; 7-Spatiotemporal transition rendering module; 8-Interactive response module. Figure 2 is a schematic diagram of the workflow of text-driven virtual-real scene linkage of the present invention; labels: 1-Import text; 2-Semantic parsing to extract scene tags; 3-Matching VR scene resources; 4-Virtual-real coordinate mapping; 5-Virtual-real image fusion; 6-Smooth switching between interaction and scene.

Claims

1. A VR time-space travel interactive and AR real-scene fusion system with text-linked interaction, characterized in that, The system includes: a semantic analysis module, a scene resource indexing module, a unified spatial coordinate mapping module, a VR virtual scene rendering module, an AR real-scene acquisition module, a virtual-real layer fusion module, a spatiotemporal transition rendering module, and an interactive response module. The semantic analysis module performs word segmentation and entity recognition on the input text, extracts geographical location, time sequence, and scene object tags, and outputs structured scene scheduling instructions. The scene resource indexing module receives scene scheduling instructions and retrieves pre-stored corresponding historical spatiotemporal VR 3D scene resources based on tag matching. The unified spatial coordinate mapping module establishes a transformation matrix between real-world AR real-scene geographical coordinates and VR virtual historical scene 3D coordinates to achieve virtual-real spatial alignment. The VR virtual scene rendering module loads successfully matched 3D scene resources, completes spatial positioning according to the transformation matrix, and outputs a VR virtual image data stream. The AR real-scene acquisition module calls the terminal camera device to acquire real-time images of the scene and outputs the original real-scene image. The virtual-real layer fusion module overlays the VR virtual image layer onto the AR real-scene image layer according to transparency weight parameters to generate a virtual-real fused image. The spatiotemporal transition rendering module is used to perform gradual transition rendering from real-world AR to VR virtual space and back from VR virtual space to real-world AR, outputting smooth switching screens; the interactive response module receives user control commands, adjusts the fusion weights and viewpoint parameters in real time, and responds to user roaming interaction operations.

2. The VR time-space travel interactive and AR real-scene fusion system with text-linked interaction as described in claim 1, characterized in that, The scene tags extracted by the semantic parsing module include time-series tags, geographic tags, and landscape entity tags, enabling multi-tag joint retrieval of scene resources.

3. The VR time-space travel interactive and AR real-scene fusion system with text-linked interaction as described in claim 1, characterized in that, The unified spatial coordinate mapping module dynamically updates the virtual-real coordinate transformation matrix based on the terminal positioning data, and corrects the positional deviation between the virtual and real scenes in real time.

4. The VR time-space travel interactive and AR real-scene fusion system with text-linked interaction as described in claim 1, characterized in that, The spatiotemporal transition rendering module uses a linear transparency interpolation algorithm to achieve a smooth transition between virtual and real images.

5. The VR time-space travel interactive and AR real-scene fusion system with text-linked interaction as described in claim 1, characterized in that, The virtual-real layer fusion module supports dynamically modifying the layer transparency weight, enabling semi-transparent overlay of virtual objects or switching between a fully real scene and a fully virtual scene.

Images