A holographic naked-eye teaching experiment cabin body sense interaction method and system

Abstract

The present application relates to somatosensory interaction technical field, the present application discloses a kind of holographic naked eye's teaching experiment cabin somatosensory interaction method and system, comprising: virtual environment scene construction module, holographic naked eye scene display module, somatosensory behavior feature identification module, somatosensory interaction behavior identification module, interaction command conversion module and multi-modal feedback module, establish depth coordinate system by somatosensory camera to carry out node identification to user and obtain user behavior depth image, and based on user behavior depth image carries out user behavior feature extraction, user behavior feature is tracked in real time using tracking algorithm, analysis obtains somatosensory interaction behavior identification result, somatosensory interaction behavior is converted into corresponding operation command based on somatosensory interaction behavior identification result, the accurate identification and quick response of user behavior are realized, effectively improve the accuracy and efficiency of somatosensory interaction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of somatosensory interaction, and more particularly to a holographic naked-eye teaching experiment cabin somatosensory interaction method and system. BACKGROUND

[0002] Holographic naked-eye technology is a technology that can achieve three-dimensional stereoscopic vision without the aid of auxiliary equipment such as glasses and headgear. Holographic technology records the amplitude and phase information of the reflected or transmitted light waves of an object to generate a hologram. When a specific light source illuminates the hologram, a three-dimensional image of the object can be reproduced. Naked-eye display technology utilizes the parallax characteristics of the human eye to project different perspective images onto the left and right eyes through optical elements such as microlens arrays, gratings, or spatial light modulators, forming stereoscopic vision.

[0003] Traditional experimental teaching methods are often limited by physical space and experimental equipment, making it difficult to meet the needs of large-scale and diversified teaching. Holographic naked-eye technology has revolutionized teaching experiment cabins. With holographic naked-eye technology, students can directly observe three-dimensional virtual environment scenes without wearing any auxiliary equipment, greatly enhancing the immersion and realism of experiments. Meanwhile, the application of somatosensory interaction technology allows students to interact with virtual environments through natural body movements, further enhancing the interactivity and interest of experiments.

[0004] However, combining holographic naked-eye technology and somatosensory interaction technology for use in teaching experiment cabins still faces some technical challenges, such as insufficient realism and interactivity of virtual environment scenes, and inaccurate somatosensory behavior recognition and real-time interaction command conversion. These issues limit the application effectiveness and user experience of holographic naked-eye teaching experiment cabins. SUMMARY

[0005] To overcome the above-mentioned defects of the prior art, the present application provides a holographic naked-eye teaching experiment cabin somatosensory interaction system to solve the problems existing in the background art.

[0006] The present application provides the following technical solutions: a holographic naked-eye teaching experiment cabin somatosensory interaction system, comprising: a virtual scene device deployment module, a holographic naked-eye scene display module, a somatosensory behavior feature recognition module, a somatosensory interaction behavior recognition module, an interaction command conversion module, and a multi-modal feedback module.

[0007] The virtual scene device deployment module constructs a three-dimensional virtual environment scene using a somatosensory camera and a high-definition camera according to the teaching content.

[0008] The holographic naked-eye scene display module presents the stereoscopic visual effect of the three-dimensional virtual environment scene using light field display technology.

[0009] The somatosensory behavior feature recognition module includes an image acquisition unit and a feature extraction unit, acquires a user behavior depth image through a somatosensory camera, and extracts a user behavior feature based on the user behavior depth image;

[0010] The somatosensory interaction behavior recognition module uses a tracking algorithm to track the user behavior feature in real time, and analyzes to obtain a somatosensory interaction behavior recognition result;

[0011] The interaction command conversion module converts the somatosensory interaction behavior into a corresponding operation command based on the somatosensory interaction behavior recognition result, to realize scene control of a three-dimensional virtual environment;

[0012] The multi-modal feedback module provides real-time feedback to a terminal according to the somatosensory interaction behavior recognition result and the interaction command operation result.

[0013] Preferably, the virtual scene environment construction module deploys 6 high-definition cameras and 4 somatosensory cameras in a 50 m2 space, supports simultaneous interaction of 30 students, and arranges the somatosensory cameras in a diagonal distribution, and arranges the high-definition cameras in a triangular layout to point to a teaching area for environment background acquisition.

[0014] The somatosensory camera includes three lenses, the middle lens is an RGB color camera, and the left and right lenses are respectively an infrared laser emitter and a 3D depth sensor composed of an infrared camera.

[0015] Preferably, the holographic naked-eye scene display module controls the light exit direction by using a speckle hologram based on a nanometer photonic crystal structure through a holographic functional screen, and accurately encodes and projects the light distribution of a three-dimensional scene through a spatial light modulator, the light distribution including light direction, light intensity, and light color.

[0016] Preferably, the somatosensory behavior feature recognition module extracts behavior features by real-time capture of human posture and motion trajectory combined with an AI algorithm, and includes an image acquisition unit and a feature extraction unit, and the specific content is as follows:

[0017] The image acquisition unit establishes a depth coordinate system through a somatosensory camera to identify a user joint node and acquire a user behavior depth image;

[0018] The feature extraction unit extracts a user behavior feature value based on a user behavior depth image by using a preset pixel value for user behavior feature extraction calculation.

[0019] Preferably, the image acquisition unit acquires a user behavior depth image through a somatosensory camera, and the specific content is as follows:

[0020] Step S1: recognize the joint points of the user through the body-sensing camera, obtain N joint points of the user, i=1, 2, 3, …, N, wherein i represents the number of each joint point, N represents the total number of the obtained joint points of the user, and a depth coordinate system is established with the perspective center of the infrared camera as the origin of the depth coordinate system, wherein the X axis is the direction of the connecting line between the infrared camera and the infrared laser emitter;

[0021] Step S2: when the user is in the initial position, the infrared camera captures and records the light spot reflected by the user, when the user moves, the infrared camera captures the light spot reflected by the user again, and the distance of the user in the space target from the origin of the depth coordinate system is calculated based on the offset distance of the target reflected light spot captured by the infrared camera when the user moves, and the calculation formula is: , wherein represents the distance of the user in the space target from the origin of the depth coordinate system, represents the distance of the user from the body-sensing camera in the plane, represents the focal length of the infrared camera, represents the length between the infrared camera and the infrared laser emitter, represents the offset distance of the target reflected light spot captured by the infrared camera when the user moves;

[0022] Step S3: the coordinates of each joint point in the user image obtained by the body-sensing camera are , the origin coordinates of the depth coordinate system are , and the plane coordinates of each joint point in the user image in the depth coordinate system are calculated based on the distance of the user in the space target from the origin of the depth coordinate system, and the calculation formula is:

[0023] , , wherein the plane coordinates of the joint point in the user image in the depth coordinate system, represents the lateral distortion coefficient of the lens, represents the longitudinal distortion coefficient of the lens;

[0024] Step S4: the plane coordinates of each joint point in the user image in the depth coordinate system are sequentially connected to obtain a user behavior depth image.

[0025] Preferably, the calculation formula for calculating the user behavior feature value based on the user behavior depth image is: , wherein represents the user behavior feature value, represents a preset pixel value, represents the depth value of the user behavior depth image at the preset pixel value, , and A feature parameter representing a user behavior depth image.

[0026] Preferably, the somatosensory interaction behavior recognition module uses a tracking algorithm to track the user behavior features in real time, and the specific content of the somatosensory interaction behavior recognition result is as follows:

[0027] The somatosensory interaction behavior frame images of the user are obtained by a somatosensory camera, and the user behavior feature value of each frame image is , m = 1, 2, 3,..., M, wherein M represents the total number of frame images of the somatosensory interaction behavior of the user, and m represents the number of each frame image;

[0028] The change value of the user behavior feature value of each frame image is calculated by a tracking algorithm to track the user behavior features in real time, and the calculation formula is: , m = 2, 3,..., M, wherein represents the change value of the user behavior feature value of each frame image;

[0029] According to the preset multiple somatosensory interaction behavior recognition threshold range, the somatosensory interaction behavior recognition is performed based on the change value of the user behavior feature value of each frame image: the change value of the user behavior feature value of the frame image is matched with the threshold range, and the matched threshold range result is transmitted to the interaction command conversion module.

[0030] Preferably, the interaction command conversion module receives the matched threshold range result transmitted by the somatosensory interaction behavior recognition module, and converts the somatosensory interaction behavior into a corresponding operation command according to the preset somatosensory interaction behavior of each threshold range.

[0031] Preferably, the multi-modal feedback module monitors whether the somatosensory interaction behavior recognition result and the interaction command operation result match in real time through a high-definition camera, and provides real-time feedback to the terminal.

[0032] A somatosensory interaction method for a holographic naked-eye teaching experiment cabin, comprising the following steps:

[0033] Step S01: constructing a three-dimensional virtual environment scene by using a somatosensory camera and a high-definition camera according to teaching content;

[0034] Step S02: presenting a stereoscopic visual effect of the three-dimensional virtual environment scene by using a light field display technology;

[0035] Step S03: obtaining a user behavior depth image by a somatosensory camera, and extracting user behavior features based on the user behavior depth image;

[0036] Step S04: tracking the user behavior features in real time by using a tracking algorithm, and analyzing a somatosensory interaction behavior recognition result.

[0037] Step S05: converting the somatosensory interaction behavior into corresponding operation commands based on the somatosensory interaction behavior recognition result, and realizing scene control of the three-dimensional virtual environment;

[0038] Step S06: providing real-time feedback to the terminal according to the somatosensory interaction behavior recognition result and the interaction command operation result of the user.

[0039] Technical effects and advantages of the present application:

[0040] The present application is provided with a virtual scene device deployment module, a holographic naked-eye scene display module, a somatosensory behavior feature recognition module, a somatosensory interaction behavior recognition module, an interaction command conversion module and a multi-modal feedback module, a depth coordinate system is established by a somatosensory camera to recognize the joint nodes of the user and obtain the depth image of the user behavior, the two-dimensional coordinates in the depth image are directly used for three-dimensional pose estimation, complex actions are recognized, and the calculation amount is simplified while the accuracy of action recognition is improved.

[0041] Based on the depth image of the user behavior, the user behavior features are extracted, the tracking algorithm is used to track the user behavior features in real time, the somatosensory interaction behavior recognition result is analyzed, the dynamic behavior mode is captured through the joint node motion trajectory of the continuous frames, the feature continuity is ensured, and the stability and reliability of the behavior recognition are improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 It is a structural schematic diagram of a holographic naked-eye teaching experiment cabin somatosensory interaction system.

[0043] Figure 2 It is a flowchart of a holographic naked-eye teaching experiment cabin somatosensory interaction method. DETAILED DESCRIPTION

[0044] The technical solutions in the present application will be described clearly and completely in combination with the drawings in the present application, and the forms of each structure described in the following embodiments are only examples, and the holographic naked-eye teaching experiment cabin somatosensory interaction method and system involved in the present application are not limited to each structure described in the following embodiments. All other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.

[0045] As shown in Figure 1 The present application provides a holographic naked-eye teaching experiment cabin somatosensory interaction system, which comprises a virtual scene device deployment module, a holographic naked-eye scene display module, a somatosensory behavior feature recognition module, a somatosensory interaction behavior recognition module, an interaction command conversion module and a multi-modal feedback module.

[0046] The virtual scene environment construction module constructs a three-dimensional virtual environment scene according to teaching content by using somatosensory cameras and high-definition cameras.

[0047] The holographic naked-eye scene display module presents a stereoscopic visual effect of the three-dimensional virtual environment scene by using light field display technology.

[0048] The somatosensory behavior feature recognition module includes an image acquisition unit and a feature extraction unit, acquires a user behavior depth image by using a somatosensory camera, and extracts a user behavior feature based on the user behavior depth image.

[0049] The somatosensory interaction behavior recognition module tracks a user behavior feature in real time by using a tracking algorithm, and analyzes to obtain a somatosensory interaction behavior recognition result.

[0050] The interaction command conversion module converts a somatosensory interaction behavior into a corresponding operation command based on the somatosensory interaction behavior recognition result, and realizes scene control of the three-dimensional virtual environment.

[0051] The multi-modal feedback module provides real-time feedback to a terminal according to a somatosensory interaction behavior recognition result and an interaction command operation result.

[0052] In this embodiment, it needs to be specifically explained that the virtual scene environment construction module deploys 6 high-definition cameras and 4 somatosensory cameras in a 50 square meter space, supports simultaneous interaction of 30 students, the somatosensory cameras are diagonally distributed, and the high-definition cameras are deployed in a triangular layout and aligned with the teaching area for environment background acquisition. The number of high-definition cameras and somatosensory cameras changes with the size of the space. The larger the space, the more high-definition cameras and somatosensory cameras are deployed. The smaller the space, the fewer high-definition cameras and somatosensory cameras are deployed.

[0053] The somatosensory camera includes three lenses. The middle lens is an RGB color camera, and the left and right lenses are respectively an infrared laser emitter and a 3D depth sensor composed of an infrared camera. The somatosensory camera can acquire a depth map of a current image and calculate depth information of feature points in the image, and can generate high-density point clouds, thereby greatly improving the running time of the system. The somatosensory camera has functions of human body dynamic capture, human body recognition, face recognition, voice recognition, etc., and can realize natural and efficient human-computer interaction.

[0054] In this embodiment, it needs to be specifically explained that the holographic naked-eye scene display module controls the light exit direction by using a speckle hologram based on a nano photonic crystal structure by installing a holographic function screen, and accurately encodes and projects light distribution of a three-dimensional scene by using a spatial light modulator. The light distribution includes light direction, light intensity, and light color.

[0055] Single display can provide tens to hundreds of view, the user in different angles, the eyes receive different perspective of light, form continuous stereoscopic perception.

[0056] In this embodiment, it needs to be specifically pointed out that the somatosensory behavior feature recognition module extracts behavior features by capturing human posture and motion trajectory in real time combined with AI algorithm, including image acquisition unit and feature extraction unit, the specific content is as follows:

[0057] Image acquisition unit: through the somatosensory camera, the joint node of the user is recognized and the user behavior depth image is acquired;

[0058] Feature extraction unit: based on the user behavior depth image, the user behavior feature value is calculated by using the preset pixel value to extract the user behavior feature.

[0059] In this embodiment, it needs to be specifically pointed out that the specific content of the image acquisition unit for acquiring the user behavior depth image is as follows:

[0060] Step S1: the joint node of the user is recognized by the somatosensory camera, and N joint nodes of the user are acquired, i=1, 2, 3,..., N, wherein i represents the number of each joint node, N represents the total number of acquired user joint nodes, the depth coordinate system is established with the perspective center of the infrared camera as the origin, wherein the X axis is the direction of the connecting line between the infrared camera and the infrared laser emitter;

[0061] Step S2: when the user is in the initial position, the infrared camera captures and records the light spot reflected by the user, when the user moves, the infrared camera captures the light spot reflected by the user again, the distance of the user in the space target from the origin of the depth coordinate system is calculated based on the offset distance of the target reflected light spot captured by the infrared camera when the user moves, and the calculation formula is: , wherein represents the distance of the user in the space target from the origin of the depth coordinate system, represents the distance of the user from the somatosensory camera in the plane, represents the focal length of the infrared camera, represents the length between the infrared camera and the infrared laser emitter, represents the offset distance of the target reflected light spot captured by the infrared camera when the user moves, when the user is in the initial position, the infrared camera captures and records the light spot reflected by the user, the coordinates of the center point of the light spot are , when the user moves, the infrared camera captures the light spot reflected by the user again, the coordinates of the center point of the light spot are , then the offset distance of the target reflected light spot captured by the infrared camera when the user moves is ;

[0062] Step S3: coordinates of each joint in the user image obtained by the body sense camera are , and the origin coordinates of the depth coordinate system are , and the plane coordinates of each joint in the user image in the depth coordinate system are calculated in combination with the distance of the user in the space target from the origin of the depth coordinate system, and the calculation formula is:

[0063] , wherein the plane coordinates of the joint in the user image in the depth coordinate system, represents a lateral distortion coefficient of the lens, represents a longitudinal distortion coefficient of the lens;

[0064] Step S4: the plane coordinates of each joint in the user image in the depth coordinate system are sequentially connected to obtain a user behavior depth image.

[0065] In the embodiment, it needs to be specifically explained that the calculation formula for calculating the user behavior feature value based on the user behavior depth image for user behavior feature extraction is: , wherein represents the user behavior feature value, represents a preset pixel value, represents a depth value of the user behavior depth image at the preset pixel value, and represent a feature parameter of the user behavior depth image;

[0066] The feature parameter is: , , and the depth value function is: when , ;

[0067] then .

[0068] In the embodiment, it needs to be specifically explained that the body sense interaction behavior recognition module uses a tracking algorithm to track the user behavior feature in real time, and the specific content of the body sense interaction behavior recognition result obtained by analysis is as follows:

[0069] The body sense interaction behavior frame image of the user is obtained by the body sense camera, and the user behavior feature value of each frame image is , m=1, 2, 3,..., M, wherein M represents the total number of frames of the body sense interaction behavior frame image of the user, and m represents the number of each frame image;

[0070] ​The change value of the user behavior feature value of each sub-frame image is calculated by using a tracking algorithm to track the user behavior feature in real time, and the calculation formula is: , m = 2, 3,..., M, wherein represents the change value of the user behavior feature value of each sub-frame image;

[0071] According to the preset multiple somatosensory interaction behavior recognition threshold range, the somatosensory interaction behavior recognition is performed based on the change value of the user behavior feature value of each sub-frame image: the change value of the user behavior feature value of the sub-frame image is matched with the threshold range, and the matched threshold range result is transmitted to the interaction command conversion module.

[0072] In this embodiment, it needs to be specifically pointed out that the interaction command conversion module receives the matched threshold range result transmitted by the somatosensory interaction behavior recognition module, and converts the somatosensory interaction behavior into a corresponding operation command according to the preset somatosensory interaction behavior of each threshold range.

[0073] In this embodiment, it needs to be specifically pointed out that the multi-modal feedback module monitors whether the somatosensory interaction behavior recognition result and the interaction command operation result of the user match in real time through the high-definition camera, and provides real-time feedback to the terminal. When it is monitored that the somatosensory interaction behavior recognition result of the user does not match the interaction command operation result, an error operation area is displayed in the three-dimensional virtual environment scene through the holographic naked-eye scene display module, and at the same time, an error prompt sound is emitted through the sound prompt device to remind the user to correct the operation. In addition, the multi-modal feedback module gives corresponding scores and feedback suggestions according to the operation performance of the user, helping the user to improve the operation skill and the experimental efficiency.

[0074] As shown in Figure 2 , in this embodiment, it needs to be specifically pointed out that a somatosensory interaction method of a holographic naked-eye teaching experiment cabin includes the following steps:

[0075] Step S01: constructing a three-dimensional virtual environment scene according to teaching content by using a somatosensory camera and a high-definition camera;

[0076] Step S02: presenting a stereoscopic visual effect of the three-dimensional virtual environment scene by using light field display technology;

[0077] Step S03: acquiring a user behavior depth image by using the somatosensory camera, and performing user behavior feature extraction based on the user behavior depth image;

[0078] Step S04: tracking the user behavior feature in real time by using a tracking algorithm, and analyzing to obtain a somatosensory interaction behavior recognition result;

[0079] Step S05: converting the somatosensory interaction behavior into corresponding operation commands based on the somatosensory interaction behavior recognition result, to realize scene control of the three-dimensional virtual environment;

[0080] Step S06: providing real-time feedback to the terminal according to the somatosensory interaction behavior recognition result and the interaction command operation result.

[0081] In the embodiment, it needs to be specifically pointed out that the difference between the embodiment and the prior art mainly lies in that the embodiment is provided with a virtual scene device deployment module, a holographic naked-eye scene display module, a somatosensory behavior feature recognition module, a somatosensory interaction behavior recognition module, an interaction command conversion module and a multi-modal feedback module, a depth coordinate system is established by a somatosensory camera to recognize the joint nodes of the user and obtain a user behavior depth image, the two-dimensional coordinates in the depth image are directly used for three-dimensional pose estimation, and complex actions are recognized, so that the calculation amount is simplified and the accuracy of action recognition is improved.

[0082] Based on the user behavior depth image, user behavior features are extracted, a tracking algorithm is used to track the user behavior features in real time, a somatosensory interaction behavior recognition result is obtained through analysis, a dynamic behavior mode is captured through the joint node motion trajectory of the continuous frames, the feature continuity is ensured, and thus the stability and reliability of the behavior recognition are improved.

[0083] Finally, the above is only the preferred embodiment of the present application and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

[0084] The above is only the specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A holographic naked-eye teaching experiment cabin body sense interaction system, characterized in that: The application relates to a virtual scene device deployment module, a holographic naked-eye scene display module, a somatosensory behavior feature recognition module, a somatosensory interaction behavior recognition module, an interaction command conversion module and a multi-modal feedback module. The virtual scene device deployment module constructs a three-dimensional virtual environment scene according to teaching content by using a somatosensory camera and a high-definition camera. The holographic naked-eye scene display module presents a stereoscopic visual effect of the three-dimensional virtual environment scene by using light field display technology. The somatosensory behavior feature recognition module comprises an image acquisition unit and a feature extraction unit, acquires a user behavior depth image by using the somatosensory camera, and extracts user behavior features based on the user behavior depth image. The somatosensory behavior feature recognition module extracts behavior features by capturing human postures and motion trajectories in real time and combining AI algorithms, and the specific content of the image acquisition unit and the feature extraction unit is as follows. The image acquisition unit establishes a depth coordinate system by using the somatosensory camera, recognizes joints of a user and acquires a user behavior depth image. The feature extraction unit extracts user behavior features by using preset pixel values based on the user behavior depth image, and calculates user behavior feature values. The somatosensory interaction behavior recognition module tracks user behavior features in real time by using a tracking algorithm, and analyzes to obtain somatosensory interaction behavior recognition results. The calculation formula of the user behavior feature value based on the user behavior depth image is: wherein represents the user behavior feature value, represents a preset pixel value, represents a depth value of the user behavior depth image at the preset pixel value, and represents a feature parameter of the user behavior depth image. The interaction command conversion module converts somatosensory interaction behaviors into corresponding operation commands based on the somatosensory interaction behavior recognition results, and realizes scene control of the three-dimensional virtual environment. The multi-modal feedback module provides real-time feedback to a terminal according to the somatosensory interaction behavior recognition results and the interaction command operation results. The virtual scene environment construction module deploys six high-definition cameras and four somatosensory cameras in a 50m2 space, supports simultaneous interaction of 30 students, and arranges the somatosensory cameras in a diagonal distribution and arranges the high-definition cameras in a triangular layout to face a teaching area for environment background acquisition.

2. The holographic naked-eye teaching experiment cabin body interaction system according to claim 1, characterized in that: The somatosensory camera comprises three lenses, the middle lens is an RGB color camera, and the left and right lenses are respectively an infrared laser emitter and a 3D depth sensor composed of an infrared camera. The holographic naked-eye scene display module controls light emission directions by using speckle holograms based on a nano photonic crystal structure, and accurately encodes and projects light distribution of a three-dimensional scene by using a spatial light modulator, wherein the light distribution comprises light direction, light intensity and light color. 3.The holographic naked-eye teaching experiment cabin body sense interaction system according to claim 1, characterized in that: The image acquisition unit acquires a user behavior depth image by using the somatosensory camera, and the specific content is as follows.

4. The holographic naked-eye teaching experiment cabin body interaction system according to claim 1, characterized in that: Step S1: recognizing joints of a user by using the somatosensory camera, acquiring N joints of the user, i=1, 2, 3,..., N, wherein i represents the number of each joint, N represents the total number of acquired joints of the user, taking a perspective center of the infrared camera as an origin of a depth coordinate system, establishing the depth coordinate system, wherein an X axis is a direction of a connecting line between the infrared camera and the infrared laser emitter; Step S4: connecting plane coordinates of each joint in the user image in the depth coordinate system in sequence to acquire a user behavior depth image. Step S2: When the user is at the initial position, the infrared camera captures and records the light spot reflected by the user, and when the user moves, the infrared camera captures the light spot reflected by the user again, the distance of the user in the space target from the origin of the depth coordinate system is calculated based on the offset distance of the target reflected light spot captured by the infrared camera when the user moves, and the calculation formula is: wherein represents the distance of the user in the space target from the origin of the depth coordinate system, represents the distance of the user from the body camera in the plane, represents the focal length of the infrared camera, represents the length between the infrared camera and the infrared laser emitter, represents the offset distance of the target reflected light spot captured by the infrared camera when the user moves. Step S3: Obtain the coordinates of each joint in the user image by the body-sensing camera as , the origin coordinates of the depth coordinate system as , and calculate the plane coordinates of each joint in the depth coordinate system in the user image in combination with the distance of the user in the space target from the origin of the depth coordinate system, the calculation formula being: , wherein a planar coordinate of the junction point in the depth coordinate system in the user image, denotes a lateral lens distortion coefficient, denotes a longitudinal lens distortion coefficient; ​ 5. The holographic naked-eye teaching experiment cabin body interaction system according to claim 1, characterized in that: The somatosensory interaction behavior recognition module utilizes a tracking algorithm to track user behavior features in real time, and the specific content of the somatosensory interaction behavior recognition result is as follows: The body sense interaction behavior of the user is acquired by a body sense camera to obtain a frame image, and a user behavior feature value of each frame image is , m = 1, 2, 3,..., M, wherein M represents a total frame number of the frame images of the user's body sense interaction behavior, and m represents a number of each frame image. The user behavior feature is tracked in real time by calculating the change value of the user behavior feature value of each sub-frame image by using a tracking algorithm, and the calculation formula is: , m = 2, 3,..., M, wherein represents the change value of the user behavior feature value of each sub-frame image. According to the preset somatosensory interaction behavior recognition threshold range, the somatosensory interaction behavior recognition is performed based on the change value of the user behavior feature value of each frame image: the change value of the user behavior feature value of the frame image is matched with the threshold range, and the matched threshold range result is transmitted to the interaction command conversion module.

6. The holographic naked-eye teaching experiment cabin body interaction system according to claim 1, characterized in that: The interaction command conversion module receives the matched threshold range result transmitted by the somatosensory interaction behavior recognition module, and converts the somatosensory interaction behavior into a corresponding operation command according to the preset somatosensory interaction behavior of each threshold range.

7. The holographic naked-eye teaching experiment cabin body interaction system according to claim 1, characterized in that: The multi-modal feedback module monitors whether the somatosensory interaction behavior recognition result and the interaction command operation result of the user match in real time through the high-definition camera, and provides real-time feedback to the terminal.

8. A holographic naked-eye teaching experiment cabin body sense interaction method, used for a holographic naked-eye teaching experiment cabin body sense interaction system according to any one of claims 1-7. The method comprises the following steps: Step S01: constructing a three-dimensional virtual environment scene according to the teaching content by using a somatosensory camera and a high-definition camera; Step S02: presenting a stereoscopic visual effect of the three-dimensional virtual environment scene by using a light field display technology; Step S03: acquiring a user behavior depth image by using the somatosensory camera, and extracting user behavior features based on the user behavior depth image; Step S04: tracking user behavior features in real time by using a tracking algorithm, and analyzing a somatosensory interaction behavior recognition result; Step S05: converting a somatosensory interaction behavior into a corresponding operation command based on the somatosensory interaction behavior recognition result, and realizing scene control of the three-dimensional virtual environment; Step S06: providing real-time feedback to the terminal according to the somatosensory interaction behavior recognition result and the interaction command operation result of the user.

Images