Immersive CAVE cinema interaction system and interaction method

Abstract

The invention relates to the technical field of virtual reality interaction based on graphic image display, and discloses an immersive CAVE cinema interaction system and method, the interaction system comprises a first depth camera and a visual angle processor, the first depth camera tracks and analyzes the head position of a user, and the visual angle processor extracts and analyzes the visual angle of the user through a first algorithm; controlling the visual angle of the display screen to change along with the user visual angle according to the user visual angle; the system further comprises a second depth camera and a visual content processor, the second depth camera tracks and analyzes user skeleton key points and recognizes user posture actions, and the visual content processor extracts and analyzes user actions through a first algorithm and adjusts a display picture of the display screen to change along with the user actions according to the user actions. And the environment control assembly is operated according to the head position of the user tracked by the first depth camera and the content of the display picture identified by the display system. According to the invention, the interactivity with the user can be effectively enhanced, and the immersive experience feeling of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual reality interaction based on graphic image display, and in particular to an immersive CAVE theater interaction system and interaction method. Background Art

[0002] With the rapid development of human-computer interaction technology, people's demand for interactive and immersive displays is increasing. Immersive CAVE theaters are display products that organically integrate high-pixel stereoscopic projection technology, 3D computer graphics, and sound design. They deliver ultra-realistic stereoscopic images, giving users a powerful, immersive experience. They also support interaction with virtual scenes, enhancing the user's immersive experience.

[0003] However, the current immersive CAVE theaters are not widely used. Existing ones focus too much on visual effects and ignore the interactivity with users, or the interaction methods are not targeted enough, which reduces the user experience.

[0004] Therefore, how to effectively enhance the interactivity between immersive theaters such as immersive CAVE theaters and users and improve the user's immersive experience is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides an immersive CAVE theater interaction system and interaction method to solve the problem that the immersive CAVE theater interaction method in the existing technology focuses too much on visual effects and ignores the interactivity with users, resulting in the interaction method being not natural enough and the user experience being poor.

[0006] The present disclosure provides an immersive CAVE theater interactive system, including a display system, wherein the display system includes a display screen;

[0007] The device further includes a first depth camera and a perspective processor electrically connected to each other, wherein the first depth camera is configured to track and analyze the position of the user's head; the perspective processor is configured to extract and analyze the user's perspective using a first algorithm, and control the perspective of the display screen to follow the changes in the user's perspective according to the user's perspective;

[0008] The device further includes a second depth camera and a visual content processor electrically connected thereto, wherein the second depth camera is configured to track and analyze key points of a user's skeleton and recognize user gestures; the visual content processor is configured to extract and analyze user gestures using a first algorithm and adjust the display screen to follow the user gestures according to the user gestures;

[0009] It also includes an environmental control component, which is electrically connected to the display system and the first depth camera respectively. The display system is configured to analyze and identify the display screen of the display screen through a second algorithm, and the environmental control component is configured to operate according to the user's head position obtained by tracking the first depth camera and the content of the display screen identified by the display system.

[0010] Optionally, the display system further includes a display angle controller and a display picture controller;

[0011] The viewing angle processor is electrically connected to the display screen through a display angle controller, and the visual content processor is electrically connected to the display screen through a display picture controller.

[0012] Optionally, the first algorithm is a human motion capture algorithm, and the first algorithm includes at least one of a recurrent neural network algorithm and a K-nearest neighbor algorithm.

[0013] Optionally, the second algorithm is a visual image recognition algorithm, and the second algorithm includes any one of the YOLO algorithm and the residual neural network algorithm.

[0014] Optionally, the environmental control component includes at least a platform control component, a blower control component and a sprayer control component.

[0015] Further optionally, the blower control component, the sprayer control component, and the platform control component are configured to operate according to the content of the display screen recognized by the display system;

[0016] The blower control component and the sprayer control component are configured to operate at a fixed point according to the user's head position obtained by tracking with the first depth camera.

[0017] Further optionally, the blower control assembly includes a blower controller and a blower electrically connected;

[0018] The sprayer control assembly includes an electrically connected sprayer controller and a sprayer;

[0019] The platform control assembly includes an electrically connected platform controller and a platform.

[0020] Further optionally, the blower is connected to an electric heating pipe, and the electric heating pipe is electrically connected to at least one of the heating wire or the heat pump;

[0021] The sprayer is connected to a plurality of liquid storages, each of which stores liquids with different smells;

[0022] The platform controller includes any one of an electromagnetic drive, a hydraulic drive, or a pneumatic drive.

[0023] Optionally, the first depth camera includes any one of a TOF depth camera and a structured light depth camera; and the second depth camera includes an RGB binocular stereo depth camera.

[0024] Based on the same inventive concept, the present disclosure further provides an immersive CAVE theater interaction method, which is applied to the above-mentioned immersive CAVE theater interaction system; the interaction method includes:

[0025] The first depth camera tracks and analyzes the user's head position in real time;

[0026] The viewing angle processor extracts and analyzes the user viewing angle through a first algorithm, and controls the viewing angle of the display screen in the display system to follow the user viewing angle according to the user viewing angle;

[0027] The second depth camera tracks and analyzes the user's skeleton key points in real time and recognizes the user's posture and movements;

[0028] The visual content processor extracts and analyzes the user action through a first algorithm, and adjusts the display screen according to the user action to follow the user action;

[0029] The display system analyzes and identifies the display screen of the display screen through the second algorithm, and the environmental control component operates according to the user's head position obtained by tracking the first depth camera and the content of the display screen identified by the display system.

[0030] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0031] The immersive CAVE theater interactive system provided by the present disclosure can not only capture and track the user's head position through the first depth camera to realize that the perspective processor obtains the user's optimal perspective based on the analysis of the first algorithm, and then controls the perspective of the display screen presented by the display screen, which automatically changes in real time with the change of the optimal perspective that can be seen by the user's head position, so that the display screen of the display screen can be seen by the user in the largest range, but also can capture the posture and movement of the human body through the second depth camera to realize that the visual content processor obtains the extraction result of the user's movement based on the analysis of the first algorithm, and controls the real-time display screen of the display screen to change with the user's movement, so that the interactivity between the user and the immersive CAVE theater is greatly enhanced, and according to the display screen obtained by the display system based on the second algorithm analysis, the environmental control component can also operate according to the display screen presented by the current display screen, which can effectively enhance the user's immersive experience in visual experience and environmental experience, enhance user interactivity, and improve user experience satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0033] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 This is a schematic structural block diagram of an immersive CAVE theater interactive system provided by an embodiment of the present disclosure;

[0035] Figure 2 This is another schematic block diagram of the structure of the immersive CAVE theater interactive system provided by an embodiment of the present disclosure;

[0036] Figure 3 This is another schematic block diagram of the structure of the immersive CAVE theater interactive system provided by an embodiment of the present disclosure;

[0037] Figure 4 This is a flowchart of the immersive CAVE theater interaction method provided by the embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0040] Please refer to Figure 1 , Figure 1 1 is a schematic structural block diagram of an immersive CAVE theater interactive system provided by an embodiment of the present disclosure. An immersive CAVE theater interactive system 000 provided by this embodiment includes a display system 30, and the display system 30 includes a display screen 301;

[0041] The device further includes a first depth camera 101 and a perspective processor 102 electrically connected to each other, wherein the first depth camera 101 is configured to track and analyze the position of the user's head; the perspective processor 102 is configured to extract and analyze the user's perspective through a first algorithm, and control the perspective of the display screen 301 to follow the changes of the user's perspective according to the user's perspective;

[0042] The second depth camera 201 and the visual content processor 202 are electrically connected. The second depth camera 201 is configured to track and analyze the key points of the user skeleton and recognize the user's gestures. The visual content processor 202 is configured to extract and analyze the user's actions through a first algorithm and adjust the display screen 301 according to the user's actions to follow the changes in the user's actions.

[0043] It also includes an environmental control component 203, which is electrically connected to the display system 30 and the first depth camera 101 respectively. The display system 30 is configured to analyze and identify the display screen 301 through a second algorithm, and the environmental control component 203 is configured to operate according to the user's head position tracked by the first depth camera 101 and the content of the display screen identified by the display system 30.

[0044] Specifically, the immersive CAVE theater interactive system 000 provided in this embodiment can realize that the playback perspective of the display screen 301 of the display system 30 and the visual image displayed by the display screen 301 are all carried out through the interaction of the user, so as to enhance the user's immersive interactive experience. Specifically, the immersive CAVE theater interactive system 000 includes a first depth camera 101 and a perspective processor 102. The first depth camera 101 and the perspective processor 102 are electrically connected. Optionally, the signal output end of the first depth camera 101 is electrically connected to the perspective processor 102. When the user uses the immersive CAVE theater interactive system, the first depth camera 101 is configured to track and analyze the user's head position in real time. After the first depth camera 101 captures and tracks the user's head position, the perspective processor 102 is configured to extract and analyze the user's perspective based on the first algorithm integrated and written therein. This user perspective can be understood as the optimal perspective for viewing the display screen 301 from the current user's head position. Based on the analyzed user perspective, the perspective of the display screen 301 is controlled to change in accordance with the user's perspective. That is, after the perspective processor 102 obtains the optimal perspective for viewing the display screen 301 through the first algorithm analysis, the perspective processor 102 can control the display screen 301 of the display system 30 to rotate until the majority of the display image of the display screen 301 is facing the user's head position, so that the display image of the display screen 301 can be viewed by the user to the greatest extent possible, thereby ensuring the integrity of the user's immersive viewing experience. In this embodiment, the perspective processor 102 uses the first depth camera 101 to capture and track the user's head position, thereby enabling the perspective processor 102 to obtain the optimal perspective for the user based on the analysis of the first algorithm, and then controlling the perspective of the display image presented by the display screen 301 to automatically change in real time in accordance with changes in the optimal perspective viewable from the user's head position, thereby ensuring that the display image of the display screen 301 can be viewed by the user to the greatest extent possible, thereby enhancing the user's immersive viewing experience.

[0045] Optionally, in this embodiment, the perspective processor 102 extracts and analyzes human motion based on a first algorithm. The first algorithm may be a human motion capture algorithm, such as the first algorithm includes at least any one of a recurrent neural network algorithm (RNN) and a K-nearest neighbor algorithm (KNN), or may be other human motion capture algorithms. This embodiment does not limit this, and only needs to satisfy the user's head position that can be captured by the first depth camera, and analyze to obtain the user's optimal perspective.

[0046] The immersive CAVE theater interactive system of this embodiment also includes a second depth camera 201 and a visual content processor 202. The second depth camera 201 and the visual content processor 202 are electrically connected. Optionally, the signal output end of the second depth camera 201 is electrically connected to the visual content processor 202. When the user uses the immersive CAVE theater interactive system, the second depth camera 201 is configured to track and analyze the user's skeleton key points in real time and identify the user's posture and movements. After the second depth camera 201 captures and tracks the user's posture and movements, the visual content processor 202 is configured to extract and analyze the user's movements based on the first algorithm written into its internal integration. This user action can be understood as the current user watching the display screen 301 and needing to perform an operation on the interface of the display screen, such as fast forward, exit, slow motion, zoom in, etc. The user can wave his hands or shake his head to perform the above operations. The visual content processor 202 extracts and analyzes the above user actions based on the first algorithm written into its internal integration, each corresponding to the operation action required by the current user. The visual content processor 202 obtains the display screen of the user watching the display screen 301 through the first algorithm analysis. After the user needs to perform an operation on the interface of the display screen, the visual content processor 202 continues to analyze the obtained user action and controls and adjusts the display screen of the display screen 301 to follow the user action. That is, when the visual content processor 202 obtains the display screen of the user watching the display screen 301 through the first algorithm analysis and the user needs to perform an operation on the interface of the display screen, the visual content processor 202 can continue to control the display screen 301 of the display system 30 to follow the user action to perform operations such as fast forward, exit, slow play, and zoom in, which is conducive to improving the user's immersive interactivity. For example, the user's hand can click the display screen in the air to pause, open, fast forward, etc. These operations in the air can actually operate the display screen, increase the interactive intelligent experience between the user and the theater system, and at the same time, can give full play to the advantages of the immersive CAVE theater. In this embodiment, the second depth camera 201 is used to capture the posture and movements of the human body, so that the visual content processor 202 obtains the extraction results of the user's movements based on the analysis of the first algorithm, and controls the real-time display screen of the display screen 301 to change with the user's movements, thereby greatly enhancing the interactivity between the user and the immersive CAVE theater, thereby further improving the user's immersive viewing experience.

[0047] The immersive CAVE theater interactive system of this embodiment also includes an environment control component 203, which is electrically connected to the display system 30 and the first depth camera 101, respectively. Optionally, the signal input end of the environment control component 203 is electrically connected to the first depth camera 101 and the display system 30, respectively. When the viewing angle of the display screen 301 is automatically adjusted to follow the user's head position through the control of the first depth camera 101 and the viewing angle processor 102, the content of the display screen of the display screen 301 is also controlled by the second depth camera 201 and the visual content processor 202 to complete the display screen following the user's movement posture. When the user immersively watches the display screen content required by the user at the best viewing angle presented by the display screen 301, the display system 30 is configured to analyze and identify the display screen of the display screen 301 through the second algorithm, that is, the second algorithm is integrated and written into the display system 30 to capture and analyze the current display screen 301. The real-time display screen, the environmental control component 203 is configured to operate according to the content of the display screen identified by the display system 30. For example, if the screen 301 currently displays a picture of a protagonist running in the rain, then the display system 30 can analyze and identify the immersive environment at this time based on the analysis and recognition of the second algorithm as user immersive sensing water spraying and vibration of the platform on which the user stands. Therefore, the display system 30 can transmit this analysis result to the environmental control component 203. The environmental control component 203 can use its set structure to control the water spraying, control the platform shaking, etc., thereby enhancing the user's immersive experience in the current viewing screen. In addition, the environmental control component 203 is further configured to control the immersive environment of the user's head position based on the position tracked by the first depth camera 101, so as to avoid the implementation direction of the components such as the water spraying structure set by the environmental control component 203 being incorrect. This embodiment can also capture the user's position through the first depth camera 101 and the environment of the display screen obtained by the display system 30 based on the second algorithm analysis, so as to enable the environment control component 203 to operate according to the display screen presented by the current display screen 301, thereby further enhancing the user's immersive experience through vision and environment.

[0048] The immersive CAVE theater interactive system provided in this embodiment can not only capture and track the user's head position through the first depth camera 101, so that the perspective processor 102 can obtain the user's optimal perspective based on the analysis of the first algorithm, and then control the perspective of the display screen 301, which automatically changes in real time with the change of the optimal perspective that can be seen by the user's head position, so that the display screen 301 can be seen by the user in the largest range, but also can capture the posture and movement of the human body through the second depth camera 201, so that the visual content processor 202 can obtain the extraction result of the user's movement based on the analysis of the first algorithm, and control the real-time display screen of the display screen 301 to change with the user's movement, so that the interactivity between the user and the immersive CAVE theater is greatly enhanced. In addition, according to the environment of the display screen obtained by the display system 30 based on the second algorithm analysis, the environment control component 203 can also operate according to the display screen presented by the current display screen 301, which can effectively enhance the user's immersive experience in terms of visual experience and environmental experience, enhance user interactivity, and improve user experience satisfaction.

[0049] Optionally, in this embodiment, the display system 30 analyzes and identifies the display screen of the display screen 301 through a second algorithm. The second algorithm can be a visual recognition algorithm, such as the second algorithm includes at least any one of the YOLO algorithm (You Only Look Once, YOLO) and the residual neural network algorithm (Residual Neural Network, Res NET), or it can also be other visual image recognition algorithms. This embodiment does not limit this, and it only needs to satisfy the display screen of the display screen 301 that can be analyzed by the display system 30.

[0050] It should be noted that this embodiment does not elaborate on the application principles of the first algorithm and the second algorithm. During specific implementation, reference can be made to the application principles of the first algorithm for human motion capture and the second algorithm for visual recognition in related technologies for understanding.

[0051] Optionally, in this embodiment, the type of the second depth camera 201 can be different from the type of the first depth camera 101. For example, the first depth camera 101 can include any of a time-of-flight depth camera (TOF depth camera) and a structured light depth camera; the second depth camera 201 can include an RGB binocular stereo depth camera. TOF depth cameras and structured light depth cameras recognize users or objects by emitting pulse signals, while RGB binocular stereo depth cameras recognize users or objects by sensing light. In this embodiment, the first depth camera 101 for identifying the user's head position and the second depth camera 201 for identifying the user's gestures are of different types. This avoids the situation where only the same depth camera is used or two depth cameras are used that both achieve recognition performance by emitting pulse signals, which may cause the inability to distinguish the pulse signals and make decoupling difficult, resulting in depth map errors. Therefore, in this embodiment, the first depth camera 101 that achieves recognition performance by emitting pulse signals is combined with the second depth camera 201 that achieves recognition performance by sensing light, which can effectively improve the signal decoupling efficiency and accuracy in the perspective processor 102 and the visual content processor 202, thereby facilitating the effectiveness of the system.

[0052] It will be understood that this embodiment is only an example of the type of the second depth camera 201 and the type of the first depth camera 101. In specific implementations, the types of the first depth camera 101 and the second depth camera 201 include but are not limited to these. The first depth camera 101 may include other depth cameras that achieve recognition performance by emitting pulse signals, and the second depth camera 201 may also include other depth cameras that achieve recognition performance by light sensing. This embodiment will not be described in detail here.

[0053] In some alternative embodiments, please refer to Figure 2 , Figure 2 is another structural schematic block diagram of the immersive CAVE theater interactive system provided by an embodiment of the present disclosure. In the immersive CAVE theater interactive system provided by this embodiment, the display system 30 further includes a display angle controller 302 and a display image controller 303;

[0054] The viewing angle processor 102 is electrically connected to the display screen 301 through the display angle controller 302 , and the visual content processor 202 is electrically connected to the display screen 301 through the display image controller 303 .

[0055] This embodiment explains that the signal output end of the first depth camera 101 is electrically connected to the perspective processor 102, and the perspective processor 102 is electrically connected to the display screen 301 through the display angle controller 302. When the user uses the immersive CAVE theater interactive system, the perspective processor 102 obtains the best perspective of the user viewing the display screen 301 through the first algorithm analysis, and then the perspective processor 102 is electrically connected to the display angle controller 302. Optionally, the display angle controller 302 can be a universal joint bracket connected to the back of the display screen 301. According to the user perspective obtained by the first depth camera 101 analysis , controls and adjusts the universal joint bracket of the display angle controller 302, so that the viewing angle of the display screen 301 follows the user's viewing angle. That is, when the first depth camera 101 tracks the user's head position, the viewing angle processor 102 obtains the optimal viewing angle of the user for viewing the display screen 301 through the first algorithm analysis, and the viewing angle processor 102 can control the rotation angle of the universal joint bracket of the display angle controller 302 to change, so that the display screen 301 can be seen by the user in the largest range under the rotation adjustment of the display angle controller 302, thereby ensuring the integrity of the user's immersive viewing of the screen.

[0056] The signal output end of the second depth camera 201 is electrically connected to the visual content processor 202, and the visual content processor 202 is electrically connected to the display screen 301 through the display screen controller 303. Optionally, the display screen controller 303 can be a background computer that controls the display screen 301. When a user uses the immersive CAVE cinema interactive system, the visual content processor 202 obtains the display screen of the user watching the display screen 301 through the first algorithm analysis. After the user needs to perform an operation on the interface of the display screen, the visual content processor 202 controls the background computer of the display screen controller 303 to adjust the display screen content according to the user action tracked by the second depth camera 201, so that the display screen of the display screen 301 changes with the user action. That is, when the second depth camera 201 tracks the user's posture action, the visual content processor 202 obtains the display screen of the user watching the display screen 301 through the first algorithm analysis. After the user needs to perform an operation on the interface of the display screen, the visual content processor 202 can control the display screen controller 303 of the display system 30 to change and adjust the content of the real screen. The display screen presented on the display screen 301 follows the user's action to fast forward, exit, slow down, zoom in, etc., thereby improving the user's immersive interactivity.

[0057] Optionally, the display screen 301 of the display system 30 in this embodiment may include any one of an annular screen, a spherical screen, a circular screen, and a topological screen. By changing the shape of the display screen 301, the user's viewing experience of the picture content can be further improved.

[0058] In some alternative embodiments, please refer to Figure 3 , Figure 3 This is another structural schematic block diagram of the immersive CAVE theater interactive system provided by an embodiment of the present disclosure. In the immersive CAVE theater interactive system provided by this embodiment, the environmental control component 203 includes at least a platform control component 203A, a blower control component 203B and a sprayer control component 203C.

[0059] Optionally, the blower control component 203B, the sprayer control component 203C, and the platform control component 203A are configured to operate according to the content of the display screen recognized by the display system 30;

[0060] The blower control component 203B and the sprayer control component 203C are configured to operate at a fixed point according to the user's head position tracked by the first depth camera 101 .

[0061] The blower control assembly 203B includes an electrically connected blower controller 203B1 and a blower 203B2; the sprayer control assembly 203C includes an electrically connected sprayer controller 203C1 and a sprayer 203C2; and the platform control assembly 203A includes an electrically connected platform controller 203A1 and a platform 203A2.

[0062] Optionally, the blower 203B2 is connected to an electric heating tube, which is electrically connected to at least one of an electric heating wire or a heat pump, or the electric heating tube can also be heated by other commonly used heating methods, which is not limited in this embodiment; the sprayer 203C2 is connected to multiple liquid storages, and the multiple liquid storages respectively store liquids with different smells; the platform controller 203A1 includes any one of an electromagnetic drive, a hydraulic drive or a pneumatic drive, or the platform controller 203A1 can also use other commonly used control methods to control the operation of the platform 203A2, which is not limited in this embodiment.

[0063] This embodiment explains that the immersive CAVE theater interactive system 000 also includes an environmental control component 203, which is electrically connected to the display system 30 and the first depth camera 101 respectively. When the viewing angle of the display screen 301 is automatically adjusted following the user's head position through the control of the first depth camera 101 and the viewing angle processor 102, the content of the display screen of the display screen 301 is also controlled by the second depth camera 201 and the visual content processor 202. After the display screen follows the user's movement posture changes, the user immersively watches the display screen content required by the user at the best viewing angle presented by the display screen 301. The second algorithm is integrated and written into the display system 30 to capture and analyze the real-time display screen of the current display screen 301, and the environmental control component 203 operates according to the content of the display screen recognized by the display system 30. Specifically, the environment control component 203 includes at least a platform control component 203A, a blower control component 203B and a sprayer control component 203C, wherein the blower control component 203B, the sprayer control component 203C and the platform control component 203A operate according to the content of the display screen recognized by the display system 30, and the blower control component 203B and the sprayer control component 203C are configured to operate at a fixed point according to the user head position tracked by the first depth camera 101, that is, the platform control component 203A controls the entire platform range where the user is located without the need for fixed-point positioning.

[0064] The blower control component 203B of this embodiment may include an electrically connected blower controller 203B1 and a blower 203B2. Optionally, the blower 203B2 is connected to an electric heating tube, which is electrically connected to at least one of an electric heating wire or a heat pump. The air delivered to the electric heating tube is heated by the electric heating wire or the heat pump to form a hot air flow. The blower controller 203B1 can control the blower 203B2 to blow the hot air flow of the electric heating tube toward the user who is watching the display screen. The blower controller 203B1 can control the blower 203B2 so that the wind speed, wind direction and wind temperature of the user's environment change with the changes in the real-time display screen of the current display screen 301.

[0065] The sprayer control component 203C of this embodiment includes an electrically connected sprayer controller 203C1 and a sprayer 203C2; optionally, the sprayer 203C2 is connected to multiple liquid storage devices, each of which stores liquids with different smells. The sprayer controller 203C1 can control the sprayer 203C2 to spray the liquids in the multiple liquid storage devices and integrate them into the hot air flow of the blower 203B2, so that the humidity of the user's environment changes with the changes in the real-time display screen of the current display screen 301.

[0066] The platform control component 203A of this embodiment includes an electrically connected platform controller 203A1 and a platform 203A2. Optionally, the platform controller 203A1 includes any one of an electromagnetic drive, a hydraulic drive, or a pneumatic drive. The platform 203A2 is the area where the user can stand. The platform controller 203A1 is connected to the platform 203A2 to control platform changes, such as controlling the platform 203A2 where the user is located to vibrate or tilt according to the content of the display screen, so as to enhance the user's immersive experience in the current environment.

[0067] When the environmental control component 203 of this embodiment is in use, for example, the screen 301 currently presents a picture of a protagonist running in a summer rainstorm. At this time, the display system 30 can analyze and identify based on the second algorithm that the immersive environment at this time is the user immersively sensing water spraying and the vibration of the platform on which the user stands. Therefore, the display system 30 can transmit this analysis result to the platform control component 203A, the blower control component 203B and the sprayer control component 203C of the environmental control component 203. The blower controller 203B1 of the blower control component 203B can control the blower 203B2 to blow the hot air flow from the electric heating tube to the user who is watching the display screen, so that the user can immersively feel the summer temperature in the current display screen. At the same time, the sprayer controller 203C1 can control the sprayer 203C2 to spray rain-like liquid in multiple liquid storage devices, and integrate it into the hot air flow of the blower 203B2, so that the user can immersively feel the summer rainstorm in the current display screen. The platform controller 203A1 can control the platform 203A2 where the user is located to vibrate or tilt following the running of the protagonist in the display screen, so as to enhance the user's immersive experience in the current environment.

[0068] It can be understood that this embodiment does not limit the shape structure of the platform 203A2 in the environmental control component 203, the shape structure of the blower 203B2, the structure of the electric heating pipe connected to the blower 203B2, and the shape structure of the liquid storage device connected to the sprayer 203C2. During specific implementation, the design can refer to the actual requirements of the immersive CAVE theater.

[0069] In some optional embodiments, please refer to Figure 1-Figure 3 and Figure 4 , Figure 4 This is a flowchart of an immersive CAVE theater interaction method provided by an embodiment of the present disclosure. The immersive CAVE theater interaction method provided by this embodiment can be applied to the above-mentioned immersive CAVE theater interaction system 000 to achieve an interactive experience between the user and the immersive CAVE theater;

[0070] The interaction method provided in this embodiment includes:

[0071] S11: The first depth camera 101 tracks and analyzes the user's head position in real time;

[0072] S12: The perspective processor 102 extracts and analyzes the user's perspective using a first algorithm;

[0073] S13: The viewing angle processor 102 controls the viewing angle of the display screen 301 in the display system 30 to follow the user's viewing angle according to the user's viewing angle;

[0074] S14: The second depth camera 201 tracks and analyzes the key points of the user's skeleton in real time and recognizes the user's posture and movements;

[0075] S15: The visual content processor 202 extracts and analyzes user actions using a first algorithm;

[0076] S16: The visual content processor 202 adjusts the display screen 301 according to the user's action so that the display screen follows the user's action;

[0077] S17: The display system 30 analyzes and identifies the display screen 301 through the second algorithm;

[0078] S18: The environment control component 203 operates according to the user's head position tracked by the first depth camera 101 and the content of the display screen recognized by the display system 30.

[0079] The interactive method provided in this embodiment for realizing the interactive experience between the user and the immersive CAVE theater by the immersive CAVE theater interactive system 000 may specifically include: the user uses Figure 1-Figure 3In any embodiment of the immersive CAVE theater interactive system 000, the first depth camera 101 first tracks and analyzes the user's head position in real time, and the perspective processor 102 extracts and analyzes the user's perspective based on the first algorithm integrated and written therein. This user perspective can be understood as the optimal perspective for the current user's head position to view the display screen 301. After the perspective processor 102 obtains the optimal perspective for the user to view the display screen 301 through the first algorithm analysis, the perspective processor 102 can control the perspective of the display screen 301 to follow the changes in the user's perspective based on the user's perspective obtained by the first depth camera 101. The perspective processor 102 can control the display screen 301 of the display system 30 to rotate to an angle so that most of the display screen of the display screen 301 is facing the user's head position, so that the display screen of the display screen 301 can be seen by the user to the greatest extent, which is conducive to ensuring the integrity of the user's immersive viewing image. While performing the above-mentioned interactive operations, the second depth camera 201 tracks and analyzes the key points of the user's skeleton in real time, identifies the user's posture and movements, and the visual content processor 202 extracts and analyzes the user's movements based on the first algorithm written into its internal integration. This user movement can be understood as the current user viewing the display screen 301 and needing to perform operations on the interface of the display screen, such as fast forward, exit, slow motion, zoom in, etc. The user can wave his hands or shake his head to perform the above-mentioned operations. The visual content processor 202 extracts and analyzes the above-mentioned user movements based on the first algorithm written into its internal integration, each corresponding to the operation required by the current user. Then the visual content processor 202 can continue to control and adjust the display screen 301 to follow the changes of the user movement according to the user movement obtained by analysis, that is, the visual content processor 202 can control the display screen 301 of the display system 30 to follow the user movement to perform operations such as fast forward, exit, slow motion, zoom in, etc., which is conducive to improving the user's immersive interactivity. When the above-mentioned interactive operation is completed and the user is immersively viewing the display screen 301 at the best viewing angle and the display screen content required by the user, the display system 30 analyzes and identifies the display screen of the display screen 301 through the second algorithm, that is, the display system 30 can internally integrate and write the second algorithm to capture and analyze the real-time display screen of the current display screen 301, and the environmental control component 203 operates according to the content of the display screen identified by the display system 30. For example, the picture currently presented on the display screen 301 is a picture of the protagonist running in the rain. At this time, the display system 30 can analyze and identify the immersive environment at this time based on the analysis and identification of the second algorithm as the user immersively sensing water spraying and the vibration of the platform on which the user stands. Therefore, the display system 30 can transmit this analysis result to the environmental control component 203, and the environmental control component 203 can use its set structure to control water spraying, control platform shaking, etc., thereby enhancing the user's immersive experience in the current viewing picture.In addition, the environment control component 203 can also control the immersive environment at a fixed point based on the user's head position tracked by the first depth camera 101, thereby avoiding errors in the implementation direction of components such as the water spray structure set by the environment control component 203.

[0080] The immersive CAVE theater interaction method provided in this embodiment can not only capture and track the user's head position through the first depth camera 101, so that the perspective processor 102 can obtain the user's optimal perspective based on the analysis of the first algorithm, and then control the perspective of the display screen 301, which automatically changes in real time with the change of the optimal perspective that can be seen at the user's head position, so that the display screen 301 can be seen by the user in the largest range, but also can capture the posture and movement of the human body through the second depth camera 201, so that the visual content processor 202 can obtain the extraction result of the user's movement based on the analysis of the first algorithm, and control the real-time display screen of the display screen 301 to change with the user's movement, so that the interactivity between the user and the immersive CAVE theater is greatly enhanced. In addition, according to the environment of the display screen obtained by the display system 30 based on the second algorithm analysis, the environment control component 203 can also operate according to the display screen presented by the current display screen 301, which can effectively enhance the user's immersive experience in terms of visual experience and environmental experience, enhance user interactivity, and improve user experience satisfaction.

[0081] It can be understood that the immersive CAVE theater interaction system and interaction method of this embodiment can be used in the field of new energy. For example, when the visual content of the display screen 301 presents various new energy formats, the user can view the details through upper limbs or gestures. At this time, the second depth camera 201 can capture the user's posture and movement to realize the visual content processor 202 to obtain the extraction result of the user's posture and movement based on the analysis of the first algorithm, and control the real-time display screen of the display screen 301 to change with the user's movement. For example, the user can view different details through changes in upper limbs or gestures: view the rotating structure of the solar thermal column, the structure of the pumped storage reservoir, the foundation structure of the offshore wind power pile, etc.

[0082] When the visual content of the display screen 301 is presented as different visual content, the display system 30 can analyze the environment of the current display screen based on the second algorithm, and further enable the environmental control component 203 to operate according to the display screen currently presented by the display screen 301; for example, when the visual content of the display screen 301 is presented as a display of a wind turbine, the blower control component 203B of the environmental control component 203 can control the output of normal temperature wind and adjust the wind speed according to the wind turbine speed; when the visual content of the display screen 301 is presented as displaying photovoltaic or solar thermal, the blower control component 203B of the environmental control component 203 can control the output of hot wind; when the visual content of the display screen 301 is presented as displaying offshore wind power, The sprayer control component 203C of the environmental control component 203 can control the spraying of seawater scent spray; for example, when the visual content of the display screen 301 is presented as displaying a complementary photovoltaic project, the sprayer control component 203C of the environmental control component 203 can control the spraying of grass scent spray; for example, when the visual content of the display screen 301 is presented as displaying a cross-photovoltaic park, the platform control component 203A of the environmental control component 203 can control the platform on which the user stands to tilt according to the visual content; for example, when the visual content of the display screen 301 is presented as displaying the rotation of wind turbine blades observed at high altitude, the platform control component 203A of the environmental control component 203 can control the platform on which the user stands to vibrate according to the visual content. It should be noted that the working principles of the first and second algorithms are not described in detail in this embodiment. For details, please refer to the explanation of the first and second algorithms in the above-mentioned immersive CAVE theater interactive system for understanding. The structure and working principle of the environmental control component 203 are not described in detail in this embodiment. For details, please refer to the explanation of the environmental control component 203 in the above-mentioned immersive CAVE theater interactive system for understanding.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0084] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. An immersive CAVE theater interactive system, characterized in that: A display system is included, wherein the display system includes a display screen; Also included is a first depth camera and a perspective processor electrically connected, wherein the first depth camera is configured to track and analyze a user's head position; The viewing angle processor is configured to extract and analyze the user viewing angle through a first algorithm, and control the viewing angle of the display screen to follow the change of the user viewing angle according to the user viewing angle; The system further includes a second depth camera and a visual content processor electrically connected thereto, wherein the second depth camera is configured to track and analyze key points of a user's skeleton and identify user gestures; the visual content processor is configured to extract and analyze user gestures using the first algorithm, and adjust the display screen to follow the user gestures according to the user gestures; It also includes an environmental control component, which is electrically connected to the display system and the first depth camera respectively. The display system is configured to analyze and identify the display screen of the display screen through a second algorithm, and the environmental control component is configured to operate according to the user's head position obtained by tracking the first depth camera and the content of the display screen identified by the display system.

2. The immersive CAVE theater interactive system according to claim 1, characterized in that: The display system also includes a display angle controller and a display picture controller; The viewing angle processor is electrically connected to the display screen through the display angle controller, and the visual content processor is electrically connected to the display screen through the display image controller.

3. The immersive CAVE theater interactive system according to claim 1, characterized in that: The first algorithm is a human motion capture algorithm, and the first algorithm includes at least one of a recurrent neural network algorithm and a K-nearest neighbor algorithm.

4. The immersive CAVE theater interactive system according to claim 1, characterized in that: The second algorithm is a visual picture recognition algorithm, and the second algorithm includes any one of the YOLO algorithm and the residual neural network algorithm.

5. The immersive CAVE theater interactive system according to claim 1, characterized in that: The environmental control component includes at least a platform control component, a blower control component and a sprayer control component.

6. The immersive CAVE theater interactive system according to claim 5, characterized in that: The blower control component, the sprayer control component, and the platform control component are configured to operate according to the content of the display screen recognized by the display system; The blower control component and the sprayer control component are configured to operate at a fixed point according to the position of the user's head tracked by the first depth camera.

7. The immersive CAVE theater interactive system according to claim 5, characterized in that: The blower control assembly includes a blower controller and a blower electrically connected; The sprayer control assembly includes an electrically connected sprayer controller and a sprayer; The platform control assembly includes a platform controller and a platform electrically connected.

8. The immersive CAVE theater interactive system according to claim 7, characterized in that: The blower is connected to an electric heating pipe, and the electric heating pipe is electrically connected to at least one of an electric heating wire or a heat pump; The sprayer is connected to a plurality of liquid storages, wherein the plurality of liquid storages respectively store liquids with different smells; The platform controller includes any one of an electromagnetic drive, a hydraulic drive, or a pneumatic drive.

9. The immersive CAVE theater interactive system according to claim 1, characterized in that: The first depth camera includes any one of a TOF depth camera and a structured light depth camera; the second depth camera includes an RGB binocular stereo depth camera.

10. An immersive CAVE theater interaction method, characterized in that: An immersive CAVE theater interactive system as described in any one of claims 1 to 9; The interaction method includes: The first depth camera tracks and analyzes the user's head position in real time; The viewing angle processor extracts and analyzes the user viewing angle through a first algorithm, and controls the viewing angle of a display screen in the display system to follow the user viewing angle according to the user viewing angle; The second depth camera tracks and analyzes the user's skeleton key points in real time and recognizes the user's posture and movements; The visual content processor extracts and analyzes the user action using the first algorithm, and adjusts the display screen according to the user action to follow the change of the user action; The display system analyzes and identifies the display screen of the display screen through a second algorithm, and the environment control component operates according to the user head position obtained by tracking the first depth camera and the content of the display screen identified by the display system.

Images