Multi-modal teaching interaction method based on virtual reality

Abstract

The invention belongs to the technical field of virtual teaching application, and relates to a virtual reality-based multi-modal teaching interaction method, which comprises the following steps of: calling a virtual teaching scene loaded with current classroom teaching resource content, determining display parameters of the virtual teaching scene according to student position positioning and putting environment perception, and putting the display parameters. According to the method, better virtual learning experience is provided for students, the effectiveness of the touch operation of the students is judged in the interaction process of the students and the virtual teaching scene so as to carry out instant response, and a visual guidance mechanism and a concentration prompting mechanism are started in the interaction event execution process, so that the learning concentration of the students on the virtual teaching scene is improved; and finally, interaction data of each student and the virtual teaching scene in the preset time length of the current classroom are collected, the participation performance of each student in the virtual teaching environment is exactly known, teaching management is promoted to be optimized, multi-mode virtual teaching interaction is developed from the perspective of students, and a new way is developed for future virtual teaching development.

Description

Technical Field

[0001] The invention belongs to the technical field of virtual teaching application, and relates to a multi-modal teaching interaction method based on virtual reality. Background Art

[0002] With the rapid development of emerging technologies such as virtual reality, augmented reality and mixed reality, their applications in the field of education are becoming more and more extensive. They can not only simulate various complex teaching scenarios and help students experience the situations described in textbooks, but also provide rich interactive means to enrich students' learning experience. However, how to deeply explore virtual teaching applications to achieve the best teaching interaction effect is an urgent problem to be solved.

[0003] In the prior art, there are also some solutions related to virtual teaching applications. For example, the invention patent application with patent number 2021114758344 is a method for generating a sense of presence in an enhanced teaching environment based on holographic imaging. It provides a new and systematic generation path and method for creating a sense of presence in an enhanced teaching environment by enhancing the generation of the teaching environment, multimodal interactive control and sense of presence. Although it helps teachers and students to have a good sense of presence experience in the enhanced teaching environment, it still faces the common limitations of the prior art in practical applications, which are specifically manifested as follows: 1. The prior art mostly focuses on the teaching display and regulation capabilities of virtual teaching scenes from the teacher's perspective, and ignores the in-depth optimization of the display effects of virtual teaching scenes from the student's perspective, resulting in a series of key display parameters, such as projection brightness, projection contrast, projection ratio and projection angle, etc., lacking a precise and detailed analysis process based on students' experience and cognitive characteristics, which affects students' immersion in the virtual teaching scene.

[0004] 2. The existing technology not only fails to fully realize the instant response and effectiveness judgment of students' touch operations in the interaction process between students and virtual teaching scenes, resulting in delays, no response or misjudgment when students try to interact with virtual teaching scenes, seriously affecting the fluency and accuracy of learning. It also lacks visual guidance function and focus prompt function in the execution process of interactive events, which may make students feel confused or distracted due to the lack of clear visual guidance and attention concentration prompts, reducing learning efficiency and quality.

[0005] 3. The existing technology lacks an effective feedback mechanism for students' immediate classroom participation during the virtual teaching process, and is unable to timely understand students' understanding of the current teaching content and their interests, which affects the effectiveness of virtual teaching guidance. Summary of the invention

[0006] In view of this, in order to solve the problems raised in the above background technology, a multimodal teaching interaction method based on virtual reality is proposed.

[0007] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides a multimodal teaching interaction method based on virtual reality, including: S1. According to the subject type and teaching chapter of the current class, a virtual teaching scene loaded with the current classroom teaching resource content is retrieved from a preset virtual teaching library.

[0008] S2. By locating the student positions in the current classroom and sensing the delivery environment, the display parameters of the current classroom virtual teaching scene are determined for delivery, where the display parameters include display delivery ratio, display delivery angle, display delivery brightness and display delivery contrast.

[0009] S3. Through the touch screen device configured by the students in the current classroom, the touch operation of the students on the foreground elements in the virtual teaching scene is perceived in real time, and it is determined whether the touch operation of the students on the foreground elements in the virtual teaching scene is valid. If it is determined to be valid, step S4 is executed.

[0010] S4. Identify the student's touch type for the current effective touch foreground element and immediately trigger the corresponding bound interaction event, where the touch types include double-click, long press, drag and zoom, and enable the visual guidance mechanism and focus prompt mechanism during the execution of the bound interaction event.

[0011] S5. Continue to collect the interaction data between each student and the virtual teaching scene within the preset duration of the current class, evaluate and provide feedback on each student's participation in the current classroom virtual teaching environment.

[0012] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention determines the display ratio and display angle of the virtual teaching scene by accurately locating the student's position, ensuring that each student in the current classroom can obtain the best possible visual experience, thereby improving the students' viewing comfort and learning immersion.

[0013] (2) The present invention conducts a trial projection of a virtual teaching scene to perceive the brightness of the projection environment, and determines the display projection brightness and display projection contrast of the current classroom virtual teaching scene. The display projection contrast can achieve local differentiation and regulation, helping to optimize the display projection effect of the virtual teaching scene, thereby enhancing the attractiveness and expressiveness of the teaching content.

[0014] (3) During the interaction between students and the virtual teaching scene, the present invention effectively realizes the instant response and effectiveness judgment of students' touch operations, accurately identifies students' touch intentions to avoid time waste and confusion caused by misoperation or invalid operation, thereby improving the learning efficiency and fluency of the virtual teaching scene.

[0015] (4) The present invention enables a visual guidance mechanism and a focus prompt mechanism during the execution process of the current effective touch foreground element binding interaction event, helping students focus on the current interaction task and reducing distractions, thereby significantly improving learning concentration and efficiency.

[0016] (5) The present invention evaluates each student's participation in the current classroom virtual teaching environment to accurately understand each student's performance and level of involvement in the virtual teaching environment, thereby optimizing teaching management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 The present invention is a flowchart of the steps for implementing the method.

[0019] Figure 2 This is a schematic diagram of the optimal visual field space area for students in the current classroom of the present invention.

[0020] Figure 3 A schematic diagram of the steps for enabling the visual guidance mechanism of the binding interaction event execution process of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 As shown, the present invention provides a multimodal teaching interaction method based on virtual reality, including: S1. According to the subject type and teaching chapter of the current class, a virtual teaching scene loaded with the current classroom teaching resource content is retrieved from a preset virtual teaching library.

[0023] S2. By locating the student positions in the current classroom and sensing the delivery environment, the display parameters of the current classroom virtual teaching scene are determined for delivery, where the display parameters include display delivery ratio, display delivery angle, display delivery brightness and display delivery contrast.

[0024] Specifically, determining the display parameters of the current classroom virtual teaching scene includes: determining the classroom position of the students through the positioning signal transmitted by the 3D viewing device worn by the students, thereby obtaining the classroom position of each student in the current classroom and the straight-line distance relative to the position of the virtual teaching scene projection device.

[0025] Plan the optimal field of view space area for each student in the current classroom, intercept the optimal field of view space sub-area corresponding to the straight-line distance relative to the virtual teaching scene projection device position, and screen the overlapping space area of ​​the optimal field of view space sub-area corresponding to the straight-line distance of each student in the current classroom relative to the virtual teaching scene projection device position as the reference projection space area for the current classroom virtual teaching scene, and determine the direction angle of its center of gravity relative to the virtual teaching scene projection device position as the display projection angle of the current classroom virtual teaching scene.

[0026] See also Figure 2 As shown, it should be noted that the specific planning process of the optimal field of view space area for each student in the current classroom is as follows: with the classroom position of each student in the current classroom as the center of the circle, the preset human field of view limit distance as the radius, and the preset human field of view limit angle as the center angle, the field of view fan-shaped plane area of ​​each student in the current classroom is constructed, and it is used as the cross-sectional fan-shaped area. Combined with the fixed standard release height of the virtual teaching scene stored in the WEB cloud, the field of view cone space area of ​​each student in the current classroom is further constructed, which is used as the optimal field of view space area for each student in the current classroom, wherein the preset human field of view limit distance and the preset human field of view limit angle are derived from the basis of human physiology research. The research determines the natural focusing distance range and eye movement angle range of the human eye, and selects the median from the range as the preset human field of view limit distance and the preset human field of view limit angle, respectively.

[0027] Get the volume of the optimal visual field space sub-area corresponding to the straight-line distance of each student in the current classroom relative to the location of the virtual teaching scene delivery device , is the number of each student in the current class. , according to the formula Determine the display ratio of the current classroom virtual teaching scene, where Preset the benchmark delivery volume for the virtual teaching scene. The number of students in the current class.

[0028] The embodiment of the present invention determines the display delivery ratio and display delivery angle of the virtual teaching scene by accurately locating the student's position, ensuring that each student in the current classroom obtains the best possible visual experience, thereby improving the students' viewing comfort and learning immersion.

[0029] Specifically, the determination of the display parameters of the current classroom virtual teaching scene also includes: the virtual teaching scene delivery device carries out a trial delivery of the virtual teaching scene with the above-mentioned display delivery ratio and display delivery angle, marks the smallest circumscribed cuboid of the virtual teaching scene delivery space as a virtual delivery peripheral box, divides the virtual delivery peripheral box into equal volume and equal shape spaces according to each preset orientation, perceives the ambient brightness of the equally divided space in each preset orientation, obtains the average ambient brightness of the virtual delivery peripheral box by mean calculation, multiplies it by the preset optimal visual adaptation ratio of the virtual teaching scene display brightness and the real environment brightness stored in the WEB cloud, and uses the product as the display delivery brightness of the current classroom virtual teaching scene.

[0030] It should be noted that the perception of the ambient brightness of the above-mentioned preset orientation-divided space is obtained through a light sensor detector integrated in the virtual teaching scene projection device.

[0031] Each display brightness range of the virtual teaching scene based on WEB cloud storage corresponds to the preset adaptive display contrast, and the display brightness range of the current classroom virtual teaching scene corresponds to the preset adaptive display contrast as the reference display contrast of the current classroom virtual teaching scene. .

[0032] According to the average ambient brightness of the virtual peripheral box , filter each high brightness position, each moderate brightness position and each low brightness position in the virtual delivery peripheral box, and adjust the display delivery contrast of each high brightness position, each moderate brightness position and each low brightness position corresponding to the average space to be , and , The contrast preset benchmark adjustment value is stored in the WEB cloud, so as to determine the display projection contrast of the current classroom virtual teaching scene.

[0033] Specifically, the screening of each high brightness position, each moderate brightness position and each low brightness position in the virtual delivery peripheral box includes: dividing the ambient brightness of the space by each preset position in the virtual delivery peripheral box; , is the number of each preset position, , calculate the standard deviation of the azimuth environment brightness of the virtual delivery peripheral box , Plan the average ambient brightness reference range of the virtual delivery peripheral box , filter the ambient brightness of the evenly divided space to be Each preset position in the filter is a moderate brightness position, and the ambient brightness of the evenly divided space is greater than or equal to Each preset position is a high brightness position, and the ambient brightness of the evenly divided space is less than or equal to The preset positions are low brightness positions.

[0034] It should be noted that the calculation formula for the standard deviation of the azimuth environment brightness of the above virtual placement peripheral box is: ,in The number of preset directions.

[0035] The embodiment of the present invention conducts trial projection of a virtual teaching scene to perceive the brightness of the projection environment, and determines the display projection brightness and display projection contrast of the current classroom virtual teaching scene, wherein the display projection contrast can achieve local differentiation and regulation, helping to optimize the display projection effect of the virtual teaching scene, thereby enhancing the attractiveness and expressiveness of the teaching content.

[0036] S3. Through the touch screen device configured by the students in the current classroom, the touch operation of the students on the foreground elements in the virtual teaching scene is perceived in real time, and it is determined whether the touch operation of the students on the foreground elements in the virtual teaching scene is valid. If it is determined to be valid, step S4 is executed.

[0037] Specifically, the method of judging whether the student's touch operation on the foreground element in the virtual teaching scene is valid includes: sensing the student's touch position on the touch screen through the touch screen device controller, mapping it into coordinates in the virtual teaching scene, and comparing it one by one with the preset coordinate sequence corresponding to the interactive activation item of each foreground element in the virtual teaching scene. If there is a coordinate in the preset coordinate sequence corresponding to the interactive activation item of any foreground element in the virtual teaching scene that is consistent with the coordinate in the virtual teaching scene mapped by the current student for the touch position on the touch screen, then it is judged that the student's touch operation on the foreground element in the virtual teaching scene is valid; otherwise, it is judged to be invalid.

[0038] It should be noted that the above-mentioned sensing of the student's touch position on the touch screen by the touch screen device controller specifically refers to the touch screen function type.

[0039] Exemplarily, if the touch screen device is a capacitive touch screen, a conductive layer is coated on its glass panel or film to form a preset capacitance matrix. When the student's finger touches the screen, the capacitance of the human body will change the capacitance value of the touch screen corresponding to the touch position. The touch screen controller continuously scans the capacitance matrix, detects the change in capacitance value and its corresponding change position, and senses the change position of the capacitance value as the student's touch position on the touch screen.

[0040] If the touch screen device is a resistive touch screen, it is composed of two conductive layers separated by a tiny isolation point in the middle. The touch screen controller applies voltage to one of the conductive layers and detects the voltage changes on the other conductive layer in real time. When the student touches the screen with his finger, the two conductive layers come into contact to form a circuit, and the current passes through the touch point, causing the voltage to change. The location of the induced voltage change is the location where the student touches the touch screen.

[0041] S4. Identify the student's touch type for the current effective touch foreground element and immediately trigger the corresponding bound interaction event, where the touch types include double-click, long press, drag and zoom, and enable the visual guidance mechanism and focus prompt mechanism during the execution of the bound interaction event.

[0042] Specifically, the identification of the student's touch type for the current valid touch foreground element includes: if the number of touch operations of the student for the interactive activation item of the current valid touch foreground element is greater than the preset number and the time interval between each touch operation is less than or equal to the double-click effective preset duration threshold, then the touch type is identified as a double-click.

[0043] If the student's single touch operation on the interactive activation item of the current valid touch foreground element lasts for a long time to meet the preset time threshold, the touch type is identified as a long press.

[0044] If the coordinates of the interactive activation item continue to change during the student's single touch operation on the interactive activation item of the current effective touch foreground element, the touch type is identified as dragging.

[0045] If the student's touch points for the currently valid touch foreground element interactive activation item are greater than or equal to a preset number, the touch type is identified as zooming.

[0046] It should be noted that, for the double-click, long-press, and dragging of the above touch types, the touch point for the interactive activation item of the current valid touch foreground element is 1, while for the zooming of the touch type, the corresponding touch point is greater than or equal to 2.

[0047] The embodiments of the present invention effectively achieve instant response and effectiveness judgment of students' touch operations during the interaction between students and virtual teaching scenes, accurately identify students' touch intentions to avoid time waste and confusion caused by misoperation or invalid operations, thereby improving the learning efficiency and fluency of virtual teaching scenes.

[0048] See also Figure 3 As shown, specifically, the visual guidance mechanism is enabled during the binding interaction event execution process, including: A1. maintaining the transparency of the current effective touch foreground element and reducing the transparency of each adjacent foreground element.

[0049] It should be noted that the transparency reduction values ​​of the foreground elements adjacent to the currently valid touch foreground element mentioned above are mainly based on the preset benchmark adjustment values ​​of the transparency of the virtual teaching scene implanted in the system development. The data comes from the professional developers of the virtual teaching scene based on industry standards and specifications, and are obtained through experimental testing to effectively highlight the currently valid touch foreground elements without excessively interfering with the user's line of sight.

[0050] A2. Randomly adjust the positions of the middle-ground elements in the virtual teaching scene so that they are arranged around the current effective touch foreground element.

[0051] A3. Blur the background of the virtual teaching scene.

[0052] Specifically, the focus prompt mechanism is enabled during the execution process of the binding interaction event, including: sensing the student's line of sight focus position in real time through the 3D viewing device worn by the student, and comparing it with the execution area scope of the current effective touch foreground element binding interaction event, so as to obtain the student's line of sight offset distance in real time, if the student's line of sight offset distance is detected to be greater than or equal to the preset focus discrete distance warning threshold, or the student's line of sight deviation distance is detected to be greater than 0 and the duration is greater than or equal to the preset focus discrete duration warning threshold, then the preset prompt sound effect is played through the student-configured touch screen device to prompt the student to focus on the current effective touch foreground element binding interaction event.

[0053] It should be noted that the above-mentioned student line of sight offset distance refers to the minimum distance between the student line of sight focus position and the positions of the boundary points of the current effective touch foreground element binding interaction event execution area. Specifically, the minimum difference is screened by subtracting the coordinates of the student line of sight focus position from the positions of the boundary points of the current effective touch foreground element binding interaction event execution area. If the minimum difference is less than or equal to 0, it means that the student line of sight focus position is within the scope of the current effective touch foreground element binding interaction event execution area, and the student line of sight offset distance is automatically set to 0. Conversely, if the minimum difference is greater than 0, it means that the student line of sight focus position is outside the scope of the current effective touch foreground element binding interaction event execution area, and the minimum difference is used as the student line of sight offset distance.

[0054] The embodiment of the present invention enables a visual guidance mechanism and a focus prompt mechanism during the execution process of the current effective touch foreground element binding interaction event, helping students focus on the current interaction task and reduce distractions, thereby significantly improving learning concentration and efficiency.

[0055] S5. Continue to collect the interaction data between each student and the virtual teaching scene within the preset duration of the current class, evaluate and provide feedback on each student's participation in the current classroom virtual teaching environment.

[0056] Specifically, the interaction data includes the number of valid touches, the timestamp corresponding to each valid touch, the foreground element, the touch type and the proportion of the line of sight focus time during the execution process of the bound interaction event.

[0057] It should be noted that the above-mentioned interaction data between each student and the virtual teaching scene within the preset duration of the current class is a summary of the touch screen devices and 3D viewing devices set by the students.

[0058] Specifically, the evaluation of each student's participation in the current classroom virtual teaching environment includes: extracting the number of effective touches between each student and the virtual teaching scene within the preset duration of the current classroom And the corresponding timestamp of each effective touch, calculate the average effective touch interval duration of each student for the virtual teaching scene within the preset duration of the current class , according to the formula Further obtain the touch activity of each student on the virtual teaching scene within the preset duration of the current class .

[0059] It should be noted that the above The method is obtained by performing difference analysis between the timestamps corresponding to each valid touch of each student and the virtual teaching scene within the preset duration of the current class and the timestamp corresponding to the previous valid touch, summarizing the differences and further calculating the mean.

[0060] Extract the foreground elements and touch types corresponding to each effective touch between each student and the virtual teaching scene within the preset duration of the current class, and count the number of effective touch foreground elements between each student and the virtual teaching scene within the preset duration of the current class And the number of touch types for each valid touch foreground element , is the number of each valid touch foreground element, , calculate the interactive diversity of each student on the virtual teaching scene within the current class preset duration , .

[0061] Extract the proportion of the time of eye focus during the execution of each effective touch corresponding to the touch type binding interaction event of each student and the virtual teaching scene within the preset duration of the current class , is the number of each effective touch, , calculate the interactive concentration of each student on the virtual teaching scene within the preset duration of the current class , , is the number of effective touches.

[0062] Will , , The products of the corresponding preset weight proportions are accumulated to obtain the participation of each student in the current classroom virtual teaching environment.

[0063] The embodiment of the present invention evaluates the participation of each student in the current classroom virtual teaching environment, accurately understands the performance and involvement of each student in the virtual teaching environment, and optimizes teaching management.

[0064] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A multimodal teaching interaction method based on virtual reality, characterized in that: The method includes: S1. According to the subject type and teaching chapter of the current class, retrieve the virtual teaching scene loaded with the current classroom teaching resource content from the preset virtual teaching library; S2. By locating the student positions in the current classroom and sensing the delivery environment, the display parameters of the current classroom virtual teaching scene are determined for delivery, where the display parameters include display delivery ratio, display delivery angle, display delivery brightness and display delivery contrast; S3. Through the touch screen device configured by the current classroom students, the students are aware of the touch operation of each foreground element in the virtual teaching scene in real time, and it is determined whether the touch operation of the foreground element in the virtual teaching scene is valid. If the judgment is valid, the step S4 is executed; S4. Identify the student's touch type for the current effective touch foreground element and immediately trigger its corresponding bound interaction event, where the touch type includes double-click, long press, drag and zoom, and enable the visual guidance mechanism and focus prompt mechanism during the execution of the bound interaction event; S5. Continue to collect the interaction data between each student and the virtual teaching scene within the preset duration of the current class, evaluate and provide feedback on each student's participation in the current classroom virtual teaching environment.

2. The multimodal teaching interaction method based on virtual reality according to claim 1, characterized in that: Determining the display parameters of the current classroom virtual teaching scene includes: determining the classroom position of the students through the positioning signal transmitted by the 3D viewing device worn by the students, thereby obtaining the classroom position of each student in the current classroom and the straight-line distance relative to the position of the virtual teaching scene projection device; Plan the optimal visual field space area for each student in the current classroom, intercept the optimal visual field space sub-area corresponding to the straight-line distance of the student relative to the virtual teaching scene delivery device position, select the overlapping space area of ​​the optimal visual field space sub-area corresponding to the straight-line distance of each student in the current classroom relative to the virtual teaching scene delivery device position as the reference delivery space area for the current classroom virtual teaching scene, and determine the direction angle of its center of gravity relative to the virtual teaching scene delivery device position as the display delivery angle of the current classroom virtual teaching scene; Get the volume of the optimal visual field sub-area corresponding to the straight-line distance of each student in the current classroom relative to the location of the virtual teaching scene delivery device , is the number of each student in the current class. , according to the formula Determine the display ratio of the current classroom virtual teaching scene, where Preset the benchmark delivery volume for the virtual teaching scene. The number of students in the current class.

3. The multimodal teaching interaction method based on virtual reality according to claim 2 is characterized in that: The method of determining the display parameters of the current classroom virtual teaching scene also includes: the virtual teaching scene delivery device carries out a virtual teaching scene trial delivery at the above display delivery ratio and display delivery angle, marks the smallest circumscribed cuboid of the virtual teaching scene delivery space as a virtual delivery peripheral box, divides the virtual delivery peripheral box into equal volume and equal form spaces according to each preset orientation, senses the ambient brightness of the equally divided space at each preset orientation, obtains the average ambient brightness of the virtual delivery peripheral box by average calculation, multiplies the average ambient brightness of the virtual delivery peripheral box by a preset optimal visual adaptation ratio of the virtual teaching scene display brightness and the real environment brightness stored in the WEB cloud, and uses the product as the display delivery brightness of the current classroom virtual teaching scene; Based on the WEB cloud storage, each display brightness range of the virtual teaching scene corresponds to the preset adaptive display contrast, and obtains the reference display contrast of the current classroom virtual teaching scene. ; According to the average ambient brightness of the virtual peripheral box , filter each high brightness position, each moderate brightness position and each low brightness position in the virtual delivery peripheral box, and adjust the display delivery contrast of each high brightness position, each moderate brightness position and each low brightness position corresponding to the average space to be , and , The contrast preset benchmark adjustment value is stored in the WEB cloud, so as to determine the display projection contrast of the current classroom virtual teaching scene.

4. The multimodal teaching interaction method based on virtual reality according to claim 3 is characterized in that: The screening of each high brightness position, each moderate brightness position and each low brightness position in the virtual delivery peripheral box includes: calculating the standard deviation of the position ambient brightness of the virtual delivery peripheral box by averaging the ambient brightness of the space at each preset position in the virtual delivery peripheral box , Plan the average ambient brightness reference range of the virtual delivery peripheral box , filter the ambient brightness of the evenly divided space to be Each preset position in the filter is a moderate brightness position, and the ambient brightness of the evenly divided space is greater than or equal to Each preset position is a high brightness position, and the ambient brightness of the evenly divided space is less than or equal to The preset positions are low brightness positions.

5. The multimodal teaching interaction method based on virtual reality according to claim 1, characterized in that: The method for judging whether the student's touch operation on the foreground element in the virtual teaching scene is valid includes: sensing the touch position of the touch screen by the touch screen device controller, mapping it into coordinates in the virtual teaching scene, and comparing it one by one with the preset coordinate sequence corresponding to the interactive activation item of each foreground element in the virtual teaching scene; if there is a coordinate in the preset coordinate sequence corresponding to the interactive activation item of any foreground element in the virtual teaching scene that is consistent with the coordinate in the virtual teaching scene mapped by the current student for the touch position of the touch screen, then it is judged that the student's touch operation on the foreground element in the virtual teaching scene is valid; otherwise, it is judged to be invalid.

6. The multimodal teaching interaction method based on virtual reality according to claim 1, characterized in that: The identifying the touch type of the student for the current effective touch foreground element includes: if the number of touch operations of the student for the interactive activation item of the current effective touch foreground element is greater than a preset number and the time interval between each touch operation is less than or equal to a double-click effective preset time threshold, then identifying the touch type as a double-click; If the student's single touch operation on the interactive activation item of the current effective touch foreground element continues to press and hold for a preset time threshold, the touch type is identified as a long press; If the coordinates of the interactive activation item continue to change during the student's single touch operation on the interactive activation item of the current effective touch foreground element, the touch type is identified as dragging; If the student's touch points for the currently valid touch foreground element interactive activation item are greater than or equal to a preset number, the touch type is identified as zooming.

7. The multimodal teaching interaction method based on virtual reality according to claim 1, characterized in that: The visual guidance mechanism is enabled during the binding interaction event execution process, including: A1. maintaining the transparency of the current effective touch foreground element and reducing the transparency of each adjacent foreground element; A2. Randomly adjust the positions of the middle-ground elements in the virtual teaching scene so that they are arranged around the current effective touch foreground element; A3. Blur the background of the virtual teaching scene.

8. The multimodal teaching interaction method based on virtual reality according to claim 1, characterized in that: The focus prompt mechanism is enabled during the execution process of the binding interaction event, including: sensing the student's line of sight focus position in real time through the 3D viewing device worn by the student, and comparing it with the execution area scope of the current effective touch foreground element binding interaction event to obtain the student's line of sight offset distance in real time, if the student's line of sight offset distance is detected to be greater than or equal to a preset focus discrete distance warning threshold, or the student's line of sight deviation distance is detected to be greater than 0 and the duration is greater than or equal to a preset focus discrete duration warning threshold, then the preset prompt sound effect is played through the student-configured touch screen device to prompt the student to focus on the current effective touch foreground element binding interaction event.

9. The multimodal teaching interaction method based on virtual reality according to claim 2, characterized in that: The interaction data includes the number of valid touches, the timestamp corresponding to each valid touch, the foreground element, the touch type and the proportion of the line of sight focus time during the execution process of the bound interaction event.

10. The multimodal teaching interaction method based on virtual reality according to claim 9, characterized in that: The evaluation of each student's participation in the current classroom virtual teaching environment includes: extracting the number of effective touches between each student and the virtual teaching scene within the preset duration of the current classroom And the corresponding timestamp of each effective touch, calculate the average effective touch interval duration of each student for the virtual teaching scene within the preset duration of the current class , according to the formula Further obtain the touch activity of each student on the virtual teaching scene within the preset duration of the current class ; Extract the foreground elements and touch types corresponding to each effective touch between each student and the virtual teaching scene within the preset duration of the current class, count the number of effective touch foreground elements between each student and the virtual teaching scene within the preset duration of the current class and the number of touch types of each effective touch foreground element, and calculate the interaction diversity of each student with the virtual teaching scene within the preset duration of the current class ; Extract the proportion of the time of each effective touch of each student and the virtual teaching scene during the execution of the touch type binding interaction event in the previous class preset time, and calculate the interactive concentration of each student on the virtual teaching scene in the current class preset time. ; Will , , The products of the corresponding preset weight proportions are accumulated to obtain the participation of each student in the current classroom virtual teaching environment.

Images