Virtuality and reality combined multi-dimensional display interaction method based on digital technology

Abstract

The invention relates to the technical field of virtual interaction, in particular to a virtuality and reality combined multi-dimensional display interaction method based on a digital technology, which comprises the steps of S1, virtual fusion scene construction, S2, user behavior capture and data fusion, S3, dynamic instruction generation and response, and S4, multi-modal feedback and dynamic adjustment. According to the method, holographic projection, atomization imaging and AR / VR are fused to construct a virtual-reality seamless combined multi-dimensional interaction scene, and a user is supported to freely pass through a naked-eye 3D space; personalized content is dynamically generated, and somatosensory and acousto-optic multi-mode interaction is realized in combination with a sensor network; and meanwhile, through a block chain reward mechanism and distributed intelligent control, the immersion, expansibility and commercialization potential are remarkably improved, and the core problems of single interaction, scene splitting, response lag and the like in the traditional technology are solved.

Description

Technical Field

[0001] The present invention relates to the field of virtual interaction technology, and more specifically, to a virtual-real multi-dimensional interactive display method based on digital technology. Background Art

[0002] AR scene interaction is a new technology that seamlessly integrates real-world and virtual world information. Using computers and other scientific technologies, it simulates and adds physical information (visual information, sound, smell, touch, etc.) that is difficult to experience within a certain time and space in the real world. This virtual information is then applied to the real world and perceived by human senses, achieving a sensory experience beyond reality. The real environment and virtual objects are visually and physically added to the same screen or space, coexisting and interacting simultaneously.

[0003] The existing book display technology has the following problems:

[0004] Traditional displays often rely on touchscreens or standalone VR devices, lacking multimodal interaction (such as somatosensory, audio, and visual interaction). The virtual and real worlds are fragmented, preventing users from freely navigating between virtual and real spaces. Display content is fixed, with no real-time adjustment of difficulty or feedback based on user behavior. For example, existing sandbox models can only be displayed statically, unable to dynamically present details through virtual-reality interaction. Summary of the Invention

[0005] The purpose of the present invention is to provide a multi-dimensional display interaction method combining virtuality and reality based on digital technology to solve the above technical problems.

[0006] The present invention provides the following technical solutions: In a first aspect, the present invention provides a multi-dimensional interactive display method combining virtuality and reality based on digital technology, comprising:

[0007] Step S1, construction of virtual fusion scene:

[0008] Use 3D modeling technology to generate three-dimensional spatial images of traditional cultural scenes to form virtual images;

[0009] Projecting virtual images onto a real-life model or a flat fog screen generated by an ultrasonic atomizer to form a dynamic image with virtual and real superposition;

[0010] Step S2: User behavior capture and data fusion:

[0011] Real-time capture of user motion data, including gestures, bow pulling force, and balance posture;

[0012] Match user action data with the spatial coordinates of the virtual fusion scene to generate interactive behavior signals;

[0013] Step S3, dynamic instruction generation and response:

[0014] Generate control instructions based on the interactive behavior signal, the control instructions including light brightness adjustment, virtual content update and sound and light effect triggering;

[0015] Execute control instructions to control the brightness of the physical model's light source, the image changes of the virtual scene, and the linkage between ambient sound and light;

[0016] Step S4: Multimodal feedback and dynamic adjustment:

[0017] Dynamically adjust the complexity of virtual scene display content based on user operation results;

[0018] Through AR / VR devices, users are given virtual rewards or hidden scenes are unlocked to complete closed-loop interaction.

[0019] Preferably, in step S1, the construction of the virtual-reality fusion scene further includes:

[0020] A flat fog screen is generated by an ultrasonic atomization generator, and dynamic 3D images are projected by a high-resolution projector, forming a naked-eye 3D virtual space that supports users to move freely.

[0021] Preferably, in step S2, capturing user behavior further includes:

[0022] Identify the spatial identification points of real props and generate a virtual model corresponding to the physical model based on a neural network algorithm to ensure real-time fit between the virtual and real scenes.

[0023] Preferably, the dynamic instruction generation in step S3 specifically includes:

[0024] Obtain the user's simulated bow drawing action, generate a virtual arrow trajectory based on the bow drawing data, and control the atomized screen to display the bull's-eye hit effect;

[0025] The user selects the interaction point of the virtual model according to the touch screen, which triggers the corresponding position of the physical model.

[0026] Preferably, the sound and light linkage control in step S3 includes:

[0027] Synchronize the lighting color, water fountain dynamics and ambient sound effects according to user operations.

[0028] Preferably, the dynamic adjustment in step S4 specifically includes:

[0029] If the user completes a preset number of tasks consecutively, the project speed and complexity of subsequent tasks will be automatically increased;

[0030] Set the user interaction threshold to trigger AR to overlay prompt information on the surface of real props.

[0031] Preferably, the method further comprises:

[0032] Capture user body movements and generate dynamic images to achieve scene interaction;

[0033] Historical scenes are reproduced using VR headsets, and users select and confirm historical scenes and interact with virtual characters.

[0034] Preferably, the method further comprises:

[0035] Identify the spatial identification points of real props, generate a virtual model corresponding to the physical model, and superimpose the historical scene into the user's field of view.

[0036] Preferably, the user operates and selects an interaction point of the virtual model according to the touch screen, which is triggered at a corresponding position of the physical model, and further includes:

[0037] The user clicks on the interaction point;

[0038] Generate a corresponding signal source according to the operation;

[0039] According to the signal source, the physical model is triggered to match the position luminous source image of the interaction point corresponding to the virtual model.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention integrates holographic projection, atomized imaging, and AR / VR to construct a multi-dimensional interactive scene that seamlessly combines the virtual and the real, supporting users to freely move through naked-eye 3D space; dynamically generates personalized content, and combines sensor networks to achieve somatosensory, sound and light multimodal interaction; at the same time, through the blockchain reward mechanism and distributed intelligent control, it significantly improves immersion, scalability and commercial potential, solving the core problems of traditional technologies such as single interaction, scene fragmentation and response lag. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Flow chart of the method of the present invention;

[0043] Figure 2 This is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] The present invention provides a multi-dimensional display interaction method combining virtuality and reality based on digital technology.

[0046] The hardware of the present invention includes a holographic projector, a 3D modeling workstation, an ultrasonic atomizer, an infrared / pressure sensor array, a VR / AR head-mounted display, a wearable somatosensory device, a distributed control host and sub-control box, and an interactive wall projection system. The software includes a 3D scene modeling engine, an AR / VR content generation platform, a multi-sensor data fusion module, a dynamic feedback algorithm, a distributed control protocol, and a user behavior analysis database.

[0047] The technical solution of the present invention is briefly explained using an example: optical imaging projects virtual characters, clothing, and artifacts onto a real-life model (such as a depiction of Confucius' holy relics), dynamically demonstrating historical scenes. An ultrasonic atomizer generates a flat mist screen, combined with a high-resolution projector to project dynamic 3D images (such as a chime bell performance). Users can freely navigate the virtual space created by the mist screen. A camera identifies real-life props (such as ritual utensils) and overlays virtual information (ritual explanations, interactive Q&A) onto the user's field of view. Wearable devices (such as pressure-sensing gloves) capture user movements (bow drawing force, balance posture) and map them to the virtual scene in real time (arrow trajectory, driving simulation). User operations (such as guqin sliding tones) trigger synchronized changes in lighting color, waterscape dynamics, and ambient sound effects. Infrared sensors capture user movements, and the projection system generates dynamic images (such as a spreading sea of ​​flowers). Combined with a touchscreen, users can copy calligraphy and paintings or play Go. This invention also includes distributed intelligent control, which uses a host computer to control lighting, projection, audio, and sensor equipment, supporting remote program transmission and scene combination. Independent areas (such as the Virtue Theater and the Six Arts Experience Zone) support manual operation, voice commands (such as the "Analects of Confucius" Q&A), and preset script modes. A sensor network collects device parameters (such as fog screen humidity and projection brightness), triggering warnings and automatic adjustments when abnormalities occur, enabling real-time status monitoring. Task difficulty (such as the complexity of martial arts moves) is dynamically adjusted based on interactive data (such as archery hit rate and answer accuracy). Completing tasks can award virtual badges or unlock hidden scenes (such as the complete process of an ancient coming-of-age ceremony).

[0048] The specific practical steps of the present invention are as follows:

[0049] Step S1, construction of a virtual fusion scene: using 3D modeling technology to generate a three-dimensional spatial stereoscopic image of a traditional cultural scene to form a virtual image; projecting the virtual image onto a real-scene model or a flat fog screen generated by an ultrasonic atomizer to form a dynamic picture of virtual and real superposition; the construction of the virtual and real fusion scene includes:

[0050] A flat fog screen is generated by an ultrasonic atomization generator, and dynamic 3D images are projected by a high-resolution projector, forming a naked-eye 3D virtual space that supports users to move freely.

[0051] Step S2, user behavior capture and data fusion: real-time capture of user action data, including gestures, bow strength, and balance posture; matching the user action data with the spatial coordinates of the virtual fusion scene to generate interactive behavior signals; user behavior capture also includes:

[0052] The camera identifies the spatial identification points of real props and generates a virtual model corresponding to the physical model based on the neural network algorithm to ensure real-time fit between the virtual and real scenes.

[0053] Step S3, dynamic instruction generation and response: Generate control instructions based on the interactive behavior signal, including light brightness adjustment, virtual content update, and sound and light effect triggering; execute the control instructions to control the brightness of the physical model's light source, the image changes of the virtual scene, and the ambient sound and light linkage; dynamic instruction generation specifically includes: obtaining the user's simulated bow action, generating a virtual arrow trajectory based on the bow data, and controlling the atomized screen to display the bull's-eye hit effect; the user operates and selects the virtual model interaction point through the touch screen, triggering the corresponding position on the physical model. The user clicks the interaction point; a corresponding signal source is generated based on the operation; based on the signal source, the physical model is triggered to match the light source at the interaction point corresponding to the virtual model.

[0054] The above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit the

[0055] Specifically, when the user wears pressure-sensing gloves to simulate the action of drawing a bow, a virtual arrow trajectory is generated based on the force data, and the fogged screen is controlled to display the bull's-eye hit effect. When the user selects the interaction point of the virtual model through touch screen operation, the brightness of the light source at the corresponding position of the physical model is triggered to change.

[0056] Sound and light linkage control includes:

[0057] The lighting color, water fountain dynamics, and ambient sound effects are synchronized based on user actions. Specifically, based on user actions (such as the guqin slide tone), the lighting color, water fountain dynamics, and ambient sound effects are synchronized to create an immersive experience that integrates sound, light, and shadow.

[0058] Step S4, multimodal feedback and dynamic adjustment: Dynamically adjust the complexity of the virtual scene's display content based on user operation results (such as archery hit rate, answer accuracy rate); feedback virtual rewards to users or unlock hidden scenes through AR / VR devices to complete closed-loop interaction. Dynamic adjustment specifically includes:

[0059] If the user completes a preset number of tasks consecutively, the project speed and complexity of subsequent tasks will be automatically increased;

[0060] Set a user interaction threshold to trigger AR overlay prompts onto the surface of real props. Specifically, if the user completes a preset number of martial arts moves in a row, the speed and complexity of subsequent moves will be automatically increased. If the user's error rate exceeds the threshold, AR overlay prompts will be triggered onto the surface of the real props.

[0061] In one embodiment, the method further includes: capturing user body movements to generate dynamic images to enable scene interaction; using a VR headset to recreate historical scenes, allowing the user to select and confirm the historical scenes and interact with them as a virtual character. For example, a wall-mounted interactive projection system can capture user body movements to generate dynamic images (such as a sea of ​​flowers expanding with a wave of the hand), and combined with a touch screen to enable calligraphy and painting copying or Go game playing.

[0062] In one embodiment, the method further includes identifying spatial landmarks of real props, generating a virtual model corresponding to the physical model, and overlaying historical scenes onto the user's field of view. For example, a historical teaching scene can be recreated through a VR headset, with the user selecting plot branches and interacting with virtual characters through gestures. The AR camera recognizes real props (such as bamboo slips) and overlays historical background explanations onto the user's field of view.

[0063] The present invention should also include a distributed control system, which consists of a central control room and sub-control boxes, wherein the central control room controls global equipment (lighting, projection, audio) through host linkage, and the sub-control boxes support multi-mode control of independent areas (such as the Six Arts Experience Area), including voice command response and preset script execution.

[0064] The present invention forms a complete "perception-response-adjustment" closed loop from scene construction (step 1) to user behavior capture (step 2), instruction generation (step 3) and then to dynamic feedback (step 4). Atomized imaging and sensor data (claim 3) jointly support the real-time fitting of virtual and real scenes, and sound and light linkage and dynamic adjustment ensure the immersive and adaptable user experience.

[0065] The basic principles, main features, and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the above-described embodiments. The above-described embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention claimed.

Claims

1. A multi-dimensional interactive display method combining virtuality and reality based on digital technology, characterized by: include: Step S1, construction of virtual fusion scene: Use 3D modeling technology to generate three-dimensional spatial images of traditional cultural scenes to form virtual images; Projecting virtual images onto a real-life model or a flat fog screen generated by an ultrasonic atomizer to form a dynamic image with virtual and real superposition; Step S2: User behavior capture and data fusion: Real-time capture of user motion data, including gestures, bow pulling force, and balance posture; Match user action data with the spatial coordinates of the virtual fusion scene to generate interactive behavior signals; Step S3, dynamic instruction generation and response: Generate control instructions based on the interactive behavior signal, the control instructions including light brightness adjustment, virtual content update and sound and light effect triggering; Execute control instructions to control the brightness of the physical model's light source, the image changes of the virtual scene, and the linkage between ambient sound and light; Step S4: Multimodal feedback and dynamic adjustment: Dynamically adjust the complexity of virtual scene display content based on user operation results; Through AR / VR devices, users are given virtual rewards or hidden scenes are unlocked to complete closed-loop interaction.

2. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: In step S1, the construction of the virtual-reality fusion scene further includes: A flat fog screen is generated by an ultrasonic atomization generator, and dynamic 3D images are projected by a high-resolution projector, forming a naked-eye 3D virtual space that supports users to move freely.

3. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: In step S2, capturing user behavior further includes: Identify the spatial identification points of real props and generate a virtual model corresponding to the physical model based on a neural network algorithm to ensure real-time fit between the virtual and real scenes.

4. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: The dynamic instruction generation in step S3 specifically includes: Obtain the user's simulated bow drawing action, generate a virtual arrow trajectory based on the bow drawing data, and control the atomized screen to display the bull's-eye hit effect; The user selects the interaction point of the virtual model according to the touch screen, which triggers the corresponding position of the physical model.

5. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: The sound and light linkage control in step S3 includes: Synchronize the lighting color, water fountain dynamics and ambient sound effects according to user operations.

6. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: The dynamic adjustment in step S4 specifically includes: If the user completes a preset number of tasks consecutively, the project speed and complexity of subsequent tasks will be automatically increased; Set the user interaction threshold to trigger AR to overlay prompt information on the surface of real props.

7. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 1 is characterized in that: The method further comprises: Capture user body movements and generate dynamic images to achieve scene interaction; Historical scenes are reproduced using VR headsets, and users select and confirm historical scenes and interact with virtual characters.

8. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 7 is characterized in that: The method further comprises: Identify the spatial identification points of real props, generate a virtual model corresponding to the physical model, and superimpose the historical scene into the user's field of view.

9. The virtual-real multi-dimensional interactive display method based on digital technology according to claim 4 is characterized in that: The user operates and selects an interaction point on the virtual model according to the touch screen, triggering the corresponding position on the physical model, and further comprising: The user clicks on the interaction point; Generate a corresponding signal source according to the operation; According to the signal source, the physical model is triggered to match the position light source of the interaction point corresponding to the virtual model.

Images