Deep-sea science popularization interaction system based on mixed reality technology

Abstract

The invention discloses a deep-sea science popularization interaction system based on a mixed reality technology, and particularly relates to the technical field of mixed reality experience science popularization education, and the system comprises a simulation manned cabin, the interior of the simulation manned cabin is provided with head-mounted MR glasses, the bottom of the simulation manned cabin is provided with a high-precision dynamic platform, and the interior of the simulation manned cabin is provided with a storage module. The storage module is connected with a playing module which is a projection lamp. A sensing module is arranged in the simulated manned cabin, the sensing module is connected with a simulated driving device, the simulated driving device is connected with a mechanical arm, the sensing module is connected with a feedback module, and the feedback module is connected with the playing module; an identification module is arranged on the outer side of the simulated manned cabin, and the identification module is an imaging sonar; an analysis module, a foldover module and an output module are arranged in the head-mounted MR glasses. According to the invention, an experiencer can be personally on the scene in a mode of combining reality and virtuality, the user experience and the detection interest are improved, and the science popularization effect of deep sea knowledge is improved.

Description

Technical Field

[0001] The present invention relates to the field of mixed reality experience science popularization education technology, and more specifically, the present invention relates to a deep sea science popularization interactive system based on mixed reality technology. Background Art

[0002] Mixed reality technology (MR) is a further development of virtual reality (VR) and augmented reality (AR). VR is a purely virtual closed environment with narrow application scenarios, AR is a simple virtual superposition of reality, and MR allows the real space and the virtual world to be perfectly integrated, realizing the coexistence and real-time interaction of physical entities and digital objects. The new visualization environment created builds an interactive feedback information loop between the real world, the virtual world and the user to enhance the realism of the user experience.

[0003] Existing forms of deep-sea science popularization are mainly based on traditional graphics, physical objects, and mechanical interactive displays, with a few using virtual reality (VR) or augmented reality (AR); however, the use of virtual reality (VR) is usually an isolated, personal experience, while the use of augmented reality (AR) has only a limited field range, and gesture controls are somewhat difficult to use, lacking content and excitement, and it is difficult to arouse the public's curiosity. Therefore, a deep-sea science popularization interactive system using mixed reality (MR) technology that integrates virtual reality (VR) and augmented reality (AR) is needed. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a deep-sea science popularization interactive system based on mixed reality technology to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: a deep sea science popularization interactive system based on mixed reality technology, comprising a simulated manned cabin, wherein head-mounted MR glasses are arranged in the simulated manned cabin, a high-precision dynamic platform is installed at the bottom of the simulated manned cabin, a storage module is arranged in the simulated manned cabin, the storage module is connected to a playback module, the playback module is a projection lamp, and the projection lamp is used to create an ocean effect; The simulated manned cabin is provided with a sensing module, the sensing module is connected to a simulated driving device, the simulated driving device is connected to a mechanical arm, the sensing module is used to sense the operation of the simulated driving device, the sensing module is connected to a feedback module, the feedback module is used to feed back the operation signal of the simulated driving device, and the feedback module is connected to the playback module; An identification module is arranged outside the simulated manned cabin, and the identification module is an imaging sonar, and the identification module is used to capture the real-life action of the experiencer operating the robotic arm; The head-mounted MR glasses are provided with an analysis module, a superposition module and an output module, the analysis module is connected to the recognition module, the analysis module is connected to the superposition module, the superposition module is connected to the output module, the analysis module is used to analyze the virtual image information broadcast by the playback module and the real scene action of the robot arm recorded by the recognition module, the superposition module uses a deep image analysis algorithm to identify surrounding objects, so that the virtual image and the real object can be perfectly superimposed together, and the output module is used to display the mixed image; The head-mounted MR glasses are connected to an audio device via Bluetooth.

[0006] Preferably, the simulated manned cabin comprises an outer shell, on which is provided a cabin door, an observation window, a wall sonar and an imaging sonar; inside of the outer shell are provided an audio device, a lighting lamp, a seat, a display screen, a simulated driving device, head-mounted MR glasses and a locator; outside of the outer shell are provided a projection lamp, a robotic arm and a sampling basket.

[0007] Preferably, the shell includes a ceiling, walls and a floor, and the ceiling, walls and floor are spray-painted with images of an ocean-themed background.

[0008] Preferably, the simulated driving device includes an operating table, two groups of operating handles and a plurality of buttons, the two groups of operating handles are perpendicular to the operating table, the operating table is provided with a plurality of buttons, and the mechanical arms are in two groups and are installed on the front side of the exterior of the cabin.

[0009] Preferably, the wall-impact sonar is provided in plurality and is respectively installed at the upper front, front, lower front, lower, lower rear, left and right positions of the shell, and the imaging sonar is installed at the front end of the outer side of the shell.

[0010] Preferably, the display screen is a three-fold connected screen, and the display screen is installed at the front end inside the shell.

[0011] Preferably, the head-mounted MR glasses integrate a display, a speaker, an image tracker and identifier, an accelerometer, a gyroscope, a magnetometer, a proximity sensor, an ambient light sensor, two computer vision cameras, and a 5-megapixel camera, and the head-mounted MR glasses have a built-in Android system.

[0012] Preferably, two groups of the positioners are provided, and both groups of the positioners are laser positioners.

[0013] Preferably, three observation windows are provided at the outer end of the shell, and a light box is provided at the glass of each observation window.

[0014] Preferably, the audio device is installed at the top of the inner part of the shell, and the speaker in the audio device is connected to the head-mounted MR glasses via Bluetooth.

[0015] Technical effects and advantages of the present invention: Through the superposition of virtual and real environment with MR glasses, the experiencer can watch the seabed topography and marine environment through the observation window. As the diving depth increases, the experiencer can observe and recognize marine life at different depths, achieving the effect of truly restoring the marine ecological environment. MR glasses are equivalent to the host of the head-mounted display. They are small and portable. By carrying two spatial computing cameras, they can sense and adapt to the environment, creating an immersive feeling for the experiencer and improving the user experience. When the user operates the joystick to control the robotic arm to grab rocks, the operation screen is displayed in real time in the cabin, achieving the interaction between the real world and the virtual environment through mechanical interactive experience, thereby increasing the interest of the user and improving the popular science effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0017] Figure 2 It is a schematic diagram of the top plan structure of the present invention.

[0018] Figure 3 It is a schematic diagram of the right-side planar structure of the present invention.

[0019] Figure 4 It is a schematic diagram of the display structure of the modules of the interactive system of the present invention.

[0020] Figure 5 It is a schematic diagram of the coordination structure of the robot arm operation and the playback module of the present invention.

[0021] The accompanying drawings are marked as follows: 1. outer shell; 2. hatch; 3. robotic arm; 4. sampling basket; 5. wall sonar; 6. lighting; 7. display screen; 8. driving simulation device; 9. seat; 10. head-mounted MR glasses; 11. locator; 12. observation window; 13. audio device; 14. projection lamp. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] As attached Figure 1-5A deep-sea science popularization interactive system based on mixed reality technology is shown, comprising a simulated manned cabin, wherein a head-mounted MR glasses 10 are arranged in the simulated manned cabin, a high-precision dynamic platform is installed at the bottom of the simulated manned cabin, a storage module is arranged in the simulated manned cabin, and the storage module is connected to a playback module, wherein the playback module is a projection lamp 14, and the projection lamp 14 is used to create an ocean effect; The simulated manned cabin is provided with a sensing module, the sensing module is connected to a simulated driving device 8, the simulated driving device 8 is connected to a mechanical arm 3, the sensing module is used to sense the operation of the simulated driving device 8, the sensing module is connected to a feedback module, the feedback module is used to feed back the operation signal of the simulated driving device 8, and the feedback module is connected to the playback module; The outer side of the simulated manned cabin is provided with a recognition module, which is an imaging sonar and is used to capture the real-life action of the experiencer operating the robotic arm 3; The head-mounted MR glasses 10 are provided with an analysis module, a superposition module and an output module. The analysis module is connected to the recognition module, the analysis module is connected to the superposition module, and the superposition module is connected to the output module. The analysis module is used to analyze the virtual image information broadcast by the playback module and the real scene action of the robot arm 3 recorded by the recognition module. The superposition module uses a deep image analysis algorithm to identify surrounding objects so that virtual images and real objects can be perfectly superimposed together. The output module is used to display mixed images. The head-mounted MR glasses 10 are connected to an audio device 13 via Bluetooth.

[0024] In specific implementation, after the user enters the simulated manned cabin, he puts on the head-mounted MR glasses 10 and starts the system. First, he dives. The playback module projects the diving scene stored in the storage module onto the front end of the simulated manned cabin through the projection lamp 14. At the same time, the high-precision dynamic platform cooperates to operate, so that the user can feel the diving situation with the cooperation of vision and body floating feeling. The head-mounted MR glasses 10 will display the scene changes during the deep diving process. Marine creatures and marine terrains at different depths appear in front of the user in turn. The user can stop diving at any time to carefully observe the creatures and environment of the ocean at that place, and cooperate with the sound device 13 to play the sound of the diving scene, thereby improving the sense of reality. When the user dives to the deepest part of the ocean, the user can operate the robotic arm 3 by operating the simulated driving device 8. The sensing module can sense the operation signal of the simulated driving device 8, cooperate with the recognition module to recognize the action information of the robotic arm 3, and the playback module cooperates to play the position scene of the object clamped by the robotic arm 3 as the robotic arm 3 moves. The analysis module analyzes the virtual image information broadcast by the playback module and the real-scene action of the robotic arm 3 recorded by the recognition module, and feeds the analysis data back to the overlay module. The overlay module uses a deep image analysis algorithm to recognize surrounding objects, so that the virtual image and the real object can be perfectly superimposed together, and then the output module is used to output the virtual-real mixed image and display it in the head-mounted MR glasses 10, so that the user can feel the real scene of his own operation and clamping, so that he can control the movement and grabbing action of the robotic arm 3 by controlling the simulated driving device 8, and display it in real time in front of the user, thereby enhancing the user's sense of experience and interest.

[0025] The simulated manned cabin includes an outer shell 1, on which a cabin door 2, an observation window 12, a wall sonar 5 and an imaging sonar are arranged; inside the outer shell 1, an audio device 13, a lighting lamp 6, a seat 9, a display screen 7, a simulated driving device 8, head-mounted MR glasses 10 and a locator 11 are arranged; outside the outer shell 1, a projection lamp 14, a robotic arm 3 and a sampling basket 4 are arranged.

[0026] During the specific implementation, the user opens the cabin door 2, and the lighting lamp 6 provides lighting for the cabin. The user takes the head-mounted MR glasses 10 from the seat 9 and wears them on the head. The user sits on the seat 9, and the projection lamp 14 plays the deep-sea image projection in front of the shell 1. The user watches from the observation window 12. With the operation of the high-precision dynamic platform at the bottom, the user can feel the process of diving in the deep sea in the submarine. During the diving process, the deep-sea scene can be watched through the head-mounted MR glasses 10. The audio device 13 provides scene sound effects. When diving to the bottom of the sea, the user can control the movement of the mechanical arm 3 by operating the simulation driving device 8. With the image projected by the projection lamp 14, the scene of the user clamping stones or shells in the deep sea can be simulated, and placed in the sampling basket 4 according to the user's operation, thereby realizing the combination of real and virtual scenes, increasing user interest, and thus improving the deep-sea science popularization effect.

[0027] The shell 1 includes a ceiling, walls and a floor, and the ceiling, walls and floor are spray-painted with images of ocean-themed backgrounds, which are convenient for users to watch, increase users' interest in exploring the unknown deep sea, and further increase users' interest in learning.

[0028] The simulated driving device 8 includes an operating table, two groups of operating handles and a plurality of buttons. The two groups of operating handles are perpendicular to the operating table. The operating table is provided with a plurality of buttons. The mechanical arms 3 are in two groups and are installed on the front side of the exterior of the cabin.

[0029] During specific implementation, two sets of robotic arms 3 are installed on the outside of the front side of the outer shell 1, controlled by a simulated driving device 8 and manually controlled by two operating levers. The upper arm can be operated in four directions: front, back, left, and right. The middle arm, forearm, and front claw can be operated in two directions: front, back, and right. This simulates grabbing rocks, marine life, etc., and placing them in a sampling basket 4.

[0030] The wall-impacting sonar 5 is provided in plurality and is respectively installed at the upper front, front, lower front, lower, lower rear, left and right positions of the housing 1 , and the imaging sonar is installed at the front end of the outer side of the housing 1 .

[0031] During specific implementation, the wall-impact sonar 5 simulates the measurement of the distances to obstacles in all directions of the submersible, providing high-speed distance data for navigation control. At the same time, the imaging sonar that films the experiencer operating the robotic arm 3 will transmit the captured operation scene back to the display screen 7 in the cabin in real time.

[0032] The display screen 7 is a three-fold screen connection screen, and the display screen 7 is installed at the front end inside the housing 1.

[0033] During the specific implementation, the experiencer can watch the corresponding split screen in the corresponding display screen 7 to observe the diving depth, wall sonar 5 data and other data information of the experience cabin, understand the diving situation of the experience cabin, touch control to open the interactive software, and watch the real-time grasping picture of the robotic arm 3 transmitted back, adjust the operating lever, and control the grasping activity of the robotic arm 3.

[0034] The head-mounted MR glasses 10 integrate a display, a speaker, an image tracker and identifier, an accelerometer, a gyroscope, a magnetometer, a proximity sensor, an ambient light sensor, two computer vision cameras, and a 5-megapixel camera. The head-mounted MR glasses have a built-in Android system.

[0035] Specifically, the head-mounted MR glasses 10 adopt the most cutting-edge MR device with fluid science fiction aesthetic design. The device senses and adapts to the environment by carrying two spatial computing cameras, creating an immersive experience for the audience. The device is placed on the armrest of seat 9, and the user only needs to sit on seat 9 to easily pick up and wear the device.

[0036] The positioners 11 are provided in two groups, and both groups of the positioners 11 are laser positioners.

[0037] During specific implementation, two groups of locators 11 are respectively installed at the corners of the inner top cover of the outer shell 1 in front of the seat 9. The positioning of the audience is achieved through the cooperation between the locator 11 and the mixed reality head-mounted MR glasses 10 and the simulated driving device 8, so as to better reflect the position of the audience during the interaction process and better realize the superposition and mixing of the audience's operation video and the virtual scene to form a mixed reality video.

[0038] The outer end of the housing 1 is provided with three observation windows 12 , and a light box is provided at the glass of each observation window 12 .

[0039] In specific implementation, the three observation windows 12 are like three eyes, paying attention to the movement outside the window at any time. The light box arranged at the glass of the observation window 12 can simulate the deep sea environment or the marine life scene seen.

[0040] The audio device 13 is installed at the top of the housing 1 , and the speaker in the audio device 13 is connected to the head-mounted MR glasses 10 via Bluetooth.

[0041] During specific implementation, the audio device 13 cooperates with the scene image sound to play, and can also play prompt voice and the wonderful sounds of the ocean. Through the speaker and the MR glasses connected via Bluetooth, the dubbing of the virtual content in AR can be displayed, bringing a very shocking spatial stereo sound effect to the audience.

[0042] Finally, a few points should be explained: First, in the description of the present invention, it should be noted that, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, which may refer to mechanical connection or electrical connection, or internal communication between two components, or direct connection. "upper", "lower", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may change; Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other; Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep-sea science popularization interactive system based on mixed reality technology, comprising a simulated manned cabin, wherein a head-mounted MR glasses (10) are arranged in the simulated manned cabin, and a high-precision dynamic platform is installed at the bottom of the simulated manned cabin, characterized in that: A storage module is provided in the simulated manned cabin, the storage module is connected to a playback module, the playback module is a projection lamp (14), and the projection lamp (14) is used to create an ocean effect; A sensing module is provided in the simulated manned cabin, the sensing module is connected to a simulated driving device (8), the simulated driving device (8) is connected to a mechanical arm (3), the sensing module is used to sense the operation of the simulated driving device (8), the sensing module is connected to a feedback module, the feedback module is used to feed back an operation signal of the simulated driving device (8), and the feedback module is connected to the playback module; An identification module is arranged outside the simulated manned cabin, the identification module is an imaging sonar, and the identification module is used to capture the real-life action of the experiencer operating the robotic arm (3); The head-mounted MR glasses (10) are provided with an analysis module, a superposition module and an output module, the analysis module is connected to the recognition module, the analysis module is connected to the superposition module, the superposition module is connected to the output module, the analysis module is used to analyze the virtual image information broadcast by the playback module and the real scene action of the mechanical arm (3) recorded by the recognition module, the superposition module uses a deep image analysis algorithm to identify surrounding objects, so that the virtual image and the real object can be perfectly superimposed together, and the output module is used to display the mixed image; The head-mounted MR glasses (10) are connected to an audio device (13) via Bluetooth.

2. The deep sea science popularization interactive system based on mixed reality technology according to claim 1, characterized in that: The simulated manned cabin comprises an outer shell (1), on which a cabin door (2), an observation window (12), a wall-impact sonar (5) and an imaging sonar are arranged; inside the outer shell (1), an audio device (13), an illuminating lamp (6), a seat (9), a display screen (7), a simulated driving device (8), head-mounted MR glasses (10) and a locator (11) are arranged; outside the outer shell (1), a projection lamp (14), a mechanical arm (3) and a sampling basket (4) are arranged.

3. The deep sea science popularization interactive system based on mixed reality technology according to claim 2, characterized in that: The housing (1) comprises a ceiling, walls and a floor, and the ceiling, walls and floor are spray-painted with images of an ocean-themed background.

4. The deep sea science popularization interactive system based on mixed reality technology according to claim 3 is characterized by: The simulated driving device (8) comprises an operating table, two sets of operating handles and a plurality of buttons, wherein the two sets of operating handles are perpendicular to the operating table, the operating table is provided with a plurality of buttons, and the mechanical arms (3) are in two sets and are mounted on the front side of the exterior of the cabin.

5. The deep sea science popularization interactive system based on mixed reality technology according to claim 4, characterized in that: The wall-impact sonar (5) is provided in plurality and is mounted respectively at the upper front, front, lower front, lower rear, left, and right positions of the housing (1), and the imaging sonar is mounted at the front end of the outer side of the housing (1).

6. The deep sea science popularization interactive system based on mixed reality technology according to claim 5, characterized in that: The display screen (7) is a three-fold screen connection screen, and the display screen (7) is installed at the front end inside the housing (1).

7. The deep sea science popularization interactive system based on mixed reality technology according to claim 6, characterized in that: The head-mounted MR glasses (10) integrate a display, a speaker, an image tracking and recognition device, an accelerometer, a gyroscope, a magnetometer, a proximity sensor, an ambient light sensor, two computer vision cameras, and a 5-megapixel camera. The head-mounted MR glasses have a built-in Android system.

8. The deep sea science popularization interactive system based on mixed reality technology according to claim 7, characterized in that: Two groups of the positioners (11) are provided, and both groups of the positioners (11) are laser positioners.

9. The deep sea science popularization interactive system based on mixed reality technology according to claim 8, characterized in that: Three observation windows (12) are arranged at the outer end of the housing (1), and a light box is arranged at the glass of each observation window (12).

10. The deep sea science popularization interactive system based on mixed reality technology according to claim 9, characterized in that: The audio device (13) is installed at the top end inside the housing (1), and a speaker inside the audio device (13) is connected to the head-mounted MR glasses (10) via Bluetooth.

Images