Intelligent VR teaching method, system and device for enhancing video teaching effect

Abstract

The invention relates to an intelligent VR teaching method, system and device for enhancing a video teaching effect. The method comprises the following steps: initializing a VR video playing scene structure and a VR model display scene structure; obtaining user behavior data and target teaching video stream data in the VR video playing scene structure, and loading a VR interaction model component group corresponding to the target teaching video stream data; generating a VR interaction model component control instruction based on the user behavior data and / or currently played target teaching video stream data; and generating and displaying a real-time VR interaction result model in the VR model display scene structure according to the VR interaction model component group in combination with the VR interaction model component control instruction. By adopting the method, one-way propagation of traditional video teaching can be converted into dynamic exploration, through dynamic adjustment and display of the VR interaction model, the learning efficiency and immersion of a learner can be remarkably improved, and a brand new normal form is provided for education digitization.

Description

Technical Field

[0001] The present invention belongs to the field of virtual teaching technology, and in particular relates to an intelligent VR teaching method, system and device for enhancing video teaching effects. Background Art

[0002] With the rapid development of information technology, virtual reality (VR) technology, as an emerging digital interactive tool, has demonstrated tremendous potential in education. VR technology uses computers to generate three-dimensional virtual environments, allowing users to immerse themselves in and interact with the virtual world. It offers immersive, interactive, and imaginative features. The application of virtual simulation technology in video teaching can enhance the fun of instruction and increase student enthusiasm for learning.

[0003] The adjustment of video teaching content in existing technologies requires teacher participation, which cannot enhance the teaching effect of video teaching. In addition, it mainly relies on two-dimensional video content, and learners can only watch videos passively, lacking interactivity and immersion. This one-way information transmission method is difficult to meet the needs of modern education for personalized, interactive and efficient learning. There are problems such as insufficient immersion, difficult to understand content and lack of interactive ability.

[0004] In the existing technology with authorization announcement number CN111064999B, VR embedded widgets can be drawn on the VR host application environment while the VR host application is active, and various interactive tutorial interfaces can be provided directly within the VR host application environment. However, this method mainly relies on pre-generated log data and video analysis, lacks the ability to dynamically respond to users' real-time behavior, and has shortcomings such as poor real-time interactivity, lack of personalized adjustment mechanism and high hardware dependence.

[0005] In the prior art application with publication number CN119155514A, a target teaching video is selected and the theme sub-video within the target teaching video is played. The user's facial expression data is collected within the theme sub-video playback time range. The user's understanding level within the theme adaptive playback time range and the granular sub-video time range are then calculated. The user's understanding level within the theme sub-video is then determined to determine whether the teaching content of the theme sub-video needs to be adjusted. Based on the user's course understanding data from the video teaching feedback, the teaching method is adaptively adjusted to enhance the video teaching effect. However, there are still areas that need improvement, such as a lack of real-time interactivity and the monotony of VR video content. Summary of the Invention

[0006] Based on this, it is necessary to provide an intelligent VR teaching method, system and device that can dynamically load interactive models, provide real-time feedback on learner behavior, and support personalized teaching to enhance video teaching effects in response to the above technical problems.

[0007] In a first aspect, the present application provides an intelligent VR teaching method for enhancing video teaching effects, comprising:

[0008] Initialize the VR video playback scene structure and VR model display scene structure;

[0009] Obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data;

[0010] Generate VR interactive model component control instructions based on user behavior data and / or currently played target teaching video stream data;

[0011] According to the VR interactive model component group and the VR interactive model component control instructions, a real-time VR interactive result model is generated and displayed in the VR model display scene structure.

[0012] In one embodiment, the intelligent VR teaching method for enhancing video teaching effects further includes:

[0013] Input the target teaching video stream data into the image segmentation model to perform feature recognition and generate a feature image set;

[0014] Based on the feature image set and the target teaching video stream data, combined with the image screening model, image screening is performed to generate a target image set. The target images in the target image set include semantic label data. The semantic label data is used to characterize the interactive characteristics of the VR interactive model corresponding to the target image.

[0015] Input the target image set into the VR interactive model generation model to generate a three-dimensional VR interactive model to obtain a VR interactive model;

[0016] Based on the VR interaction model and semantic label data, VR interaction model components are generated and a VR interaction model component group is constructed.

[0017] In one embodiment, generating a VR interaction model component based on the VR interaction model and semantic tag data includes:

[0018] Locating the connection of the VR interactive model based on the semantic label data, and identifying the connection of the VR interactive model, wherein the semantic label data includes connection type label data and connection position label data;

[0019] According to the semantic label data, the motion constraints of the connection part are applied to generate the degree of freedom allocation parameters;

[0020] Assign parameters based on the connection parts and degrees of freedom to build interactive deformation components of VR interactive models;

[0021] The transformation matrix of the VR interactive model is configured based on the VR interactive model joint components to obtain the VR interactive model parts.

[0022] In one embodiment, the image segmentation model is an improved YOLO model, which includes a feature image output detection head and a semantic label output detection head;

[0023] The image screening model is an improved converter model, which includes a linear converter module and a sliding window converter module;

[0024] The VR interactive model generation model includes a VR interactive model point cloud generation sub-model, a VR interactive model voxel modeling sub-model, and a VR engine integration sub-model;

[0025] The VR interactive model point cloud generation sub-model is an improved 3D point cloud generation adversarial neural network model.

[0026] In one embodiment, the VR interactive model component interaction control instructions include VR interactive model component selection control instructions and VR interactive model component interaction control instructions, the target teaching video stream data includes model display video stream data, and a real-time VR interaction result model is generated and displayed in a VR model display scene according to the VR interactive model component group and the VR interactive model component control instructions, including:

[0027] Selecting a VR interactive model component for interactive display from a VR interactive model component group according to a VR interactive model selection control instruction;

[0028] Generate a real-time VR interaction result model based on the VR interaction model components and the VR interaction model component interaction control instructions, and display the real-time VR interaction result model in the VR model display scene structure;

[0029] Obtain model display video stream data of the real-time VR interactive result model;

[0030] Play the model in the VR video playback scene structure to display the video stream data.

[0031] In one embodiment, generating VR interactive model component control instructions based on user behavior data and / or currently played target teaching video stream data includes:

[0032] Generate VR interactive model component default selection control instructions and VR interactive model component default display control instructions based on the currently playing target teaching video stream data, where the VR interactive model component default display control instructions are used to represent the basic display mode of the VR interactive model component corresponding to the currently playing target teaching video stream data in the initial state;

[0033] Identifying control instructions from user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions, where the VR interactive model component user interaction control instructions are used to characterize the interactive display mode of the VR interactive model component based on the user operation behavior;

[0034] updating the default selection control instruction of the VR interactive model component according to the user selection control instruction of the VR interactive model component to obtain the VR interactive model component selection control instruction;

[0035] The VR interactive model component real-time interactive control instructions and the VR interactive model component real-time interactive control instructions are integrated to generate the VR interactive model component interactive control instructions.

[0036] In one embodiment, the user behavior data includes eye movement behavior data and gesture behavior data, and control instruction recognition is performed on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions, including:

[0037] Use the timed aiming algorithm to identify control commands from eye movement behavior data and generate user-selected control commands for VR interactive model components;

[0038] The machine learning gesture recognition algorithm is used to identify control instructions of gesture behavior data and generate user interaction control instructions for VR interactive model components.

[0039] In a second aspect, the present application also provides an intelligent VR teaching system for enhancing video teaching effects, including:

[0040] VR scene initialization module, used to initialize the VR video playback scene structure and VR model display scene structure;

[0041] The basic data acquisition module is used to obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data;

[0042] A control instruction generation module is used to generate VR interactive model component control instructions based on user behavior data and / or currently played target teaching video stream data;

[0043] The interactive result generation module is used to generate and display a real-time VR interactive result model in the VR model display scene structure according to the VR interactive model component group and the VR interactive model component control instructions.

[0044] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any method of the first aspect of the present application are implemented.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any method of the first aspect of the present application.

[0046] The above-mentioned intelligent VR teaching methods, systems and devices that enhance the effectiveness of video teaching, through the organic combination of scene decoupling, intelligent perception and dynamic feedback, can effectively solve the core pain points of traditional virtual teaching such as distraction, blind operation and low knowledge conversion rate while maintaining the immersive advantages of VR technology, thereby achieving accurate matching of teaching content with learners' needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 A flowchart of an intelligent VR teaching method for enhancing video teaching effects provided in one embodiment of the present application;

[0049] Figure 2 A flowchart of a method for constructing a VR interactive model component group provided in one embodiment of the present application;

[0050] Figure 3 A flowchart of a method for constructing a VR interactive model component provided in one embodiment of the present application;

[0051] Figure 4 A flowchart of another intelligent VR teaching method for enhancing video teaching effects provided in one embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of an intelligent VR teaching system for enhancing video teaching effects provided in one embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0054] In one embodiment, Figure 1As shown, an intelligent VR teaching method for enhancing the effect of video teaching is provided. This embodiment uses the method applied to a VR terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a VR system including a VR terminal and a server, and is implemented through the interaction between the VR terminal and the server. In this embodiment, the method includes the following steps:

[0055] Step S101: Initialize the VR video playback scene structure and the VR model display scene structure.

[0056] Optionally, the VR terminal can initialize the VR video playback scene structure and the VR model display scene structure. The VR terminal can play the target teaching video corresponding to the target teaching video stream data in the VR video playback scene structure. The VR terminal can display the teaching VR display model related to the target teaching video corresponding to the target teaching video stream data in the VR model display scene structure.

[0057] Optionally, the steps for the VR terminal to initialize the VR video playback scene structure may include, but are not limited to, hardware preparation, scene setup, video resource import, video player setup, and video player setup. Hardware preparation may include ensuring that the head-mounted display (HMD) and related devices are connected and functioning properly; scene setup may include creating a basic scene in the VR engine; video resource import may include importing video resources into the project and ensuring their format is compatible; and video player setup may include creating a video player object and placing it in an appropriate location in the scene.

[0058] Optionally, the steps for the VR terminal to initialize the VR model to display the scene structure may include but are not limited to hardware preparation, scene setting, model import, and model placement and configuration. Among them, hardware preparation may include ensuring that the head-mounted display (HMD) and related equipment are connected and working properly and checking whether parameters such as the display resolution and refresh rate of the equipment meet the requirements; scene setting may include creating a basic scene in the VR engine, configuring the lighting conditions of the scene and adding environmental effects. The scene may include basic elements such as the ground and the sky; model import may include importing 3D model resources into the project, ensuring that its format (such as FBX, OBJ, etc.) is compatible with the VR engine, performing necessary optimization on the model, and checking the integrity and correctness of the model; model placement and configuration may include placing the model in the appropriate position in the scene, performing necessary scaling and rotation adjustments, and configuring the model's physical properties, collision detection, and interactive behavior.

[0059] Step S102: Obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data.

[0060] Optionally, the VR terminal can obtain user behavior data and target teaching video stream data in the VR video playback scene structure based on the sensing device, and identify and load the VR interactive model component group corresponding to the target teaching video stream data based on the target teaching video stream data.

[0061] Optionally, the VR interactive model component group may include the VR interactive model component corresponding to the teaching video stream data and the original display timing information of the VR interactive model component.

[0062] Optionally, user behavior data may include, but is not limited to, eye movement data, gesture data, facial data, and brain wave data. The eye movement data may include, but is not limited to, gaze point location data, gaze point duration data, smooth pursuit data, and pupil change data.

[0063] Furthermore, the VR terminal can identify the user's emotional parameters based on facial data and brain wave data.

[0064] Step S103: Generate VR interactive model component control instructions based on user behavior data and / or the currently played target teaching video stream data.

[0065] Optionally, the VR terminal may generate initial VR interactive model component control instructions based on the currently playing target teaching video stream data. The VR terminal may generate real-time updated VR interactive model component control instructions based on user behavior data and update the initial VR interactive model component control instructions.

[0066] Step S104: Generate and display a real-time VR interaction result model in the VR model display scene structure according to the VR interaction model component group and the VR interaction model component control instruction.

[0067] Optionally, the VR terminal can select a VR interactive model component to be displayed from the VR interactive model component group based on the VR interactive model component control instruction, and interactively transform the VR interactive model component to be displayed based on the VR interactive model component control instruction to generate a VR interactive result model, and display the VR interactive result model as a teaching VR display model in the VR model display scene structure.

[0068] Optionally, interactive transformations may include, but are not limited to, annotation, deformation, translation, rotation, and scaling.

[0069] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, by real-time loading of the VR interactive model component group corresponding to the target teaching video stream data and generating control instructions based on user behavior data, learners can interact with the virtual model in real time. This interactivity not only makes learning more interesting, but also enables learners to deepen their understanding of knowledge points through practical operations.

[0070] Furthermore, the aforementioned intelligent VR teaching method for enhancing video instruction dynamically adjusts the content presented in the VR interactive model based on user behavior data, catering to diverse learners' learning styles and cognitive levels. The system automatically adjusts the difficulty and presentation of instructional content based on learner behavior, ensuring each learner receives the most personalized learning experience. This personalized instruction significantly improves learning outcomes and provides customized instruction tailored to each learner's needs.

[0071] In one of the optional embodiments, please refer to Figure 2 , intelligent VR teaching methods that enhance the effectiveness of video teaching also include:

[0072] Step S201: input the target teaching video stream data into the image segmentation model, perform feature recognition, and generate a feature image set.

[0073] Step S202 : Based on the feature image set and the target teaching video stream data, combined with the image screening model, perform image screening to generate a target image set.

[0074] Specifically, the target image in the target image set may include semantic label data, and the semantic label data may be used to characterize the interactive characteristics of the VR interactive model corresponding to the target image.

[0075] Step S203: input the target image set into the VR interactive model generation model to generate a three-dimensional VR interactive model to obtain a VR interactive model.

[0076] Step S204: Generate VR interaction model components based on the VR interaction model and the semantic tag data, and construct a VR interaction model component group.

[0077] In the above-mentioned intelligent VR teaching method for enhancing the video teaching effect, by inputting the target teaching video stream data into the image segmentation model for feature recognition, the key features in the video can be accurately extracted to generate a high-quality feature image set; by performing image screening based on the feature image set and the target teaching video stream data in combination with the image screening model, redundant information can be effectively removed and images with important semantic information can be retained; by inputting the target image set into the VR interactive model generation model, a high-quality three-dimensional VR interactive model can be efficiently generated; by generating three-dimensional VR interactive model components with clear interactive characteristics, a richer and more realistic interactive experience can be constructed in the VR model display scene, which can enable learners to intuitively understand complex concepts and dynamic processes by operating these model components, thereby significantly improving learning effects.

[0078] In one of the optional embodiments, Figure 3As shown, VR interaction model components are generated based on the VR interaction model combined with semantic label data, including:

[0079] Step S301: Positioning the connection parts of the VR interactive model based on the semantic tag data, and identifying the connection parts of the VR interactive model.

[0080] Specifically, the semantic tag data may include connection type tag data and connection position tag data. The connection type tag data is used to characterize the type of the connection, and the connection position tag data is used to characterize the relative position of the connection in the VR interactive model.

[0081] Step S302 : performing motion constraints on the connection part according to the semantic label data to generate degree of freedom allocation parameters.

[0082] Step S303: constructing a VR interactive model interactive deformation component according to the connection part and degree of freedom allocation parameters.

[0083] Step S304: configuring the transformation matrix of the VR interactive model based on the VR interactive model joint components to obtain VR interactive model parts.

[0084] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, the connection parts of the VR interactive model are located through semantic label data, which can accurately identify the connection parts of the model and their types and positions; by constraining the motion of the connection parts and generating degree of freedom allocation parameters, the model components can be given motion characteristics, and learners can simulate real physical behaviors by operating these components, thereby significantly improving the sense of participation and fun of learning; by configuring the transformation matrix of the VR interactive model, precise control of the model components can be achieved, and a richer and more realistic interactive experience can be constructed in the VR model display scene.

[0085] In one optional embodiment, the image segmentation model is an improved YOLO model, which includes a feature image output detection head and a semantic label output detection head.

[0086] Optionally, the improved YOLO model can be obtained by setting a feature image output detection head and a semantic label output detection head in the head network structure based on any YOLO model from YOLOV5 to YOLOV9.

[0087] The image screening model is an improved transformer model, which may include a linear transformer (linear-Transformer) module and a sliding window transformer (Swin-Transformer) module.

[0088] Optionally, according to the direction of data flow, the improved converter model may include multiple sliding window converter modules and multiple linear converters in sequence.

[0089] The VR interactive model generation model includes a VR interactive model point cloud generation sub-model, a VR interactive model voxel modeling sub-model and a VR engine integration sub-model.

[0090] The VR interactive model point cloud generation sub-model can be used to improve the adversarial neural network model for 3D point cloud generation.

[0091] Optionally, the improved 3D point cloud generative adversarial neural network model can be obtained by adding image semantic labels as conditional information and adding a residual structure based on the 3D generative adversarial neural network model.

[0092] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, by adopting the improved YOLO model as the image segmentation model, it is possible to increase the simultaneous output of feature images and semantic labels, significantly improving the accuracy and efficiency of feature extraction; through the linear converter module and the sliding window converter module, it is possible to enhance the flexibility and accuracy of image screening, ensuring that the screened images have important semantic information; through the improved 3D point cloud generation adversarial neural network model, high-precision 3D point cloud data can be generated, which can provide a more accurate foundation for subsequent voxel modeling and VR engine integration.

[0093] In one optional embodiment, the VR interactive model component interaction control instruction includes a VR interactive model component selection control instruction and a VR interactive model component interaction control instruction, and the target teaching video stream data includes a model display video stream data, please refer to Figure 4 , based on the VR interactive model component group and the VR interactive model component control instructions, a real-time VR interactive result model is generated and displayed in the VR model display scene, including:

[0094] Step S407: selecting a VR interactive model component for interactive display from the VR interactive model component group according to the VR interactive model selection control instruction.

[0095] Step S408: Generate a real-time VR interaction result model based on the VR interaction model component and the VR interaction model component interaction control instruction, and display the real-time VR interaction result model in the VR model display scene structure.

[0096] Step S409: Obtain model display video stream data of the real-time VR interaction result model.

[0097] Step S410: Play the model display video stream data in the VR video playback scene structure.

[0098] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, by selecting control instructions and interactive control instructions through VR interactive model components, learners can flexibly select and manipulate VR interactive model components, making the teaching process more flexible and personalized.

[0099] In one of the optional embodiments, please refer to Figure 4 , based on user behavior data and / or the currently played target teaching video stream data, generate VR interactive model component control instructions, including:

[0100] Step S403: Generate VR interactive model component default selection control instructions and VR interactive model component default display control instructions based on the target teaching video stream data currently being played.

[0101] Specifically, the VR interactive model component default display control instruction can be used to represent the basic display mode of the VR interactive model component corresponding to the target teaching video stream data currently being played in the initial state. The VR interactive model component default display control instruction can include a VR interactive model component default selection control instruction and a VR interactive model component default display control instruction.

[0102] In step S404, control instructions are identified on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions. The VR interactive model component user interaction control instructions are used to represent the interactive display method of the VR interactive model component based on the user operation behavior.

[0103] Step S405: updating the default selection control instruction of the VR interactive model component according to the user selection control instruction of the VR interactive model component to obtain the VR interactive model component selection control instruction.

[0104] Step S406: integrating the VR interactive model component real-time interactive control instruction and the VR interactive model component real-time interactive control instruction to generate a VR interactive model component interactive control instruction.

[0105] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, by generating default selection and display control instructions based on the currently playing target teaching video stream data, it can ensure that the VR interactive model components most relevant to the teaching content are displayed in the initial state; by allowing learners to adjust the display method of VR interactive model components in real time through operational behavior, it can enhance learners' sense of participation in the learning process and help learners better understand complex concepts and dynamic processes.

[0106] In one optional embodiment, the user behavior data includes eye movement behavior data and gesture behavior data, and control instruction recognition is performed on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions, including:

[0107] The timed aiming algorithm is used to identify control instructions from eye movement behavior data and generate user selection control instructions for VR interactive model components.

[0108] The machine learning gesture recognition algorithm is used to identify control instructions of gesture behavior data and generate user interaction control instructions for VR interactive model components.

[0109] In the above-mentioned intelligent VR teaching method that enhances the effectiveness of video teaching, the recognition of eye movement behavior data and gesture behavior data can provide learners with a more natural and intuitive interaction method, enhance learners' immersion and participation, and make the learning experience closer to the interaction habits of the real world; gesture recognition technology can support the fine operation of complex three-dimensional models, allowing learners to explore the structure and function of the model more carefully; by real-time analysis of user behavior data and dynamic adjustment of teaching content, an intelligent and adaptive teaching environment can be built to ensure that every learner can have an effective learning experience.

[0110] In an exemplary embodiment, if Figure 4 As shown in the figure, the intelligent VR teaching method for enhancing the video teaching effect includes:

[0111] Step S401: Initialize the VR video playback scene structure and the VR model display scene structure.

[0112] Step S402: Obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data.

[0113] Step S403, based on the target teaching video stream data currently being played.

[0114] Generate VR interactive model component default selection control instructions and VR interactive model component default display control instructions.

[0115] Step S404: performing control instruction recognition on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions.

[0116] Step S405: updating the default selection control instruction of the VR interactive model component according to the user selection control instruction of the VR interactive model component to obtain the VR interactive model component selection control instruction.

[0117] Step S406: integrating the VR interactive model component real-time interactive control instruction and the VR interactive model component real-time interactive control instruction to generate a VR interactive model component interactive control instruction.

[0118] Step S407: selecting a VR interactive model component for interactive display from the VR interactive model component group according to the VR interactive model selection control instruction.

[0119] Step S408: Generate a real-time VR interaction result model based on the VR interaction model component and the VR interaction model component interaction control instruction, and display the real-time VR interaction result model in the VR model display scene structure.

[0120] Step S409: Obtain model display video stream data of the real-time VR interaction result model.

[0121] Step S410: Play the model display video stream data in the VR video playback scene structure.

[0122] In the above-mentioned intelligent VR teaching method for enhancing the effect of video teaching, by automatically generating VR interactive model components with semantic labels and interactive characteristics, the system can significantly improve the efficiency of teaching content generation and reduce the workload of teachers and developers; by real-time analysis of user behavior data and dynamic adjustment of teaching content, the system can build an intelligent and adaptive teaching environment; through rich interactive displays and synchronous video playback, it can provide learners with a more attractive learning experience, stimulate learners' curiosity and desire to explore, and make the learning process more enjoyable and efficient.

[0123] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0124] Based on the same inventive concept, the embodiments of the present application also provide an intelligent VR teaching system for enhancing video teaching effects for implementing the intelligent VR teaching method for enhancing video teaching effects mentioned above. The implementation solution provided by this system is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more embodiments of the intelligent VR teaching system for enhancing video teaching effects provided below can be found in the limitations of the intelligent VR teaching method for enhancing video teaching effects above, and will not be repeated here.

[0125] In an exemplary embodiment, Figure 5 As shown, an intelligent VR teaching system 500 for enhancing video teaching effects is provided, comprising:

[0126] The VR scene initialization module 501 can be used to initialize the VR video playback scene structure and the VR model display scene structure.

[0127] The basic data acquisition module 502 can be used to obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data.

[0128] The control instruction generation module 503 can be used to generate VR interactive model component control instructions based on user behavior data and / or the currently played target teaching video stream data.

[0129] The interaction result generation module 504 may be configured to generate and display a real-time VR interaction result model in a VR model display scene structure according to the VR interaction model component group and the VR interaction model component control instructions.

[0130] In one optional embodiment, the intelligent VR teaching system 500 for enhancing video teaching effects may also be used for:

[0131] The target teaching video stream data is input into the image segmentation model for feature recognition to generate a feature image set; based on the feature image set and the target teaching video stream data, combined with the image screening model, image screening is performed to generate a target image set, and the target image in the target image set includes semantic label data, and the semantic label data is used to characterize the interactive characteristics of the VR interactive model corresponding to the target image; the target image set is input into the VR interactive model generation model to generate a three-dimensional VR interactive model to obtain a VR interactive model; based on the VR interactive model and combined with the semantic label data, VR interactive model components are generated, and a VR interactive model component group is constructed.

[0132] In one optional embodiment, the intelligent VR teaching system 500 for enhancing video teaching effects may also be used for:

[0133] Based on the semantic label data, the connection parts of the VR interactive model are located and the connection parts of the VR interactive model are identified. The semantic label data includes connection part type label data and connection part position label data; the connection parts are motion constrained according to the semantic label data, and degree of freedom allocation parameters are generated; according to the connection parts and degree of freedom allocation parameters, the interactive deformation components of the VR interactive model are constructed; based on the VR interactive model joint components, the transformation matrix of the VR interactive model is configured to obtain the VR interactive model components.

[0134] In one optional embodiment, the interaction result generating module 504 may also be used to:

[0135] Select a VR interactive model component for interactive display from the VR interactive model component group according to the VR interactive model selection control instruction; generate a real-time VR interactive result model based on the VR interactive model component and the VR interactive model component interaction control instruction, and display the real-time VR interactive result model in the VR model display scene structure; obtain model display video stream data of the real-time VR interactive result model; play the model display video stream data in the VR video playback scene structure.

[0136] In one optional embodiment, the control instruction generating module 503 may also be used to:

[0137] Based on the target teaching video stream data currently being played, a VR interactive model component default selection control instruction and a VR interactive model component default display control instruction are generated. The VR interactive model component default display control instruction is used to characterize the basic display method of the VR interactive model component corresponding to the target teaching video stream data currently being played in the initial state; control instructions are identified on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions. The VR interactive model component user interaction control instructions are used to characterize the interactive display method of the VR interactive model component based on the user operation behavior; the VR interactive model component default selection control instruction is updated according to the VR interactive model component user selection control instruction to obtain the VR interactive model component selection control instruction; the VR interactive model component real-time interaction control instruction and the VR interactive model component real-time interaction control instruction are integrated to generate the VR interactive model component interaction control instruction.

[0138] In one optional embodiment, the control instruction generating module 503 may also be used to:

[0139] The timed aiming algorithm is used to identify control commands from eye movement behavior data and generate user selection control commands for VR interactive model components. The machine learning gesture recognition algorithm is used to identify control commands from gesture behavior data and generate user interaction control commands for VR interactive model components.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the intelligent VR teaching method for enhancing video teaching effects as described above are implemented.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The system embodiment described above is only illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0143] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. An intelligent VR teaching method for enhancing video teaching effect, characterized in that: The method comprises: Initialize the VR video playback scene structure and VR model display scene structure; Obtaining user behavior data and target teaching video stream data in the VR video playback scene structure, and loading the VR interactive model component group corresponding to the target teaching video stream data; Generate VR interactive model component control instructions based on user behavior data and / or the target teaching video stream data currently being played; A real-time VR interaction result model is generated and displayed in the VR model display scene structure according to the VR interaction model component group and the VR interaction model component control instructions.

2. The method according to claim 1, characterized in that The method further comprises: Inputting the target teaching video stream data into an image segmentation model to perform feature recognition and generate a feature image set; Based on the feature image set and the target teaching video stream data, in combination with an image screening model, image screening is performed to generate a target image set, wherein the target images in the target image set include semantic label data, and the semantic label data is used to characterize the interactive characteristics of the VR interactive model corresponding to the target image; Inputting the target image set into a VR interactive model generation model to generate a three-dimensional VR interactive model to obtain a VR interactive model; The VR interaction model components are generated based on the VR interaction model and combined with the semantic tag data, and the VR interaction model component group is constructed.

3. The method according to claim 2, characterized in that The step of generating the VR interaction model component based on the VR interaction model and the semantic tag data includes: Positioning the connection of the VR interactive model based on the semantic label data to identify the connection of the VR interactive model, wherein the semantic label data includes connection type label data and connection position label data; Performing motion constraints on the connection part according to the semantic label data to generate degree of freedom allocation parameters; Constructing a VR interactive model interactive deformation component according to the connection part and the degree of freedom allocation parameters; The transformation matrix of the VR interactive model is configured based on the VR interactive model joint component to obtain the VR interactive model component.

4. The method according to claim 2, wherein: The image segmentation model is an improved YOLO model, which includes a feature image output detection head and a semantic label output detection head; The image screening model is an improved converter model, which includes a linear converter module and a sliding window converter module; The VR interaction model generation model includes a VR interaction model point cloud generation sub-model, a VR interaction model voxel modeling sub-model and a VR engine integration sub-model; The VR interactive model point cloud generation sub-model is an improved three-dimensional point cloud generation adversarial neural network model.

5. The method according to any one of claims 1 to 4, characterized in that The VR interactive model component interaction control instruction includes a VR interactive model component selection control instruction and a VR interactive model component interaction control instruction, the target teaching video stream data includes model display video stream data, and the real-time VR interaction result model is generated and displayed in the VR model display scene according to the VR interactive model component group and the VR interactive model component control instruction, including: Selecting a VR interactive model component for interactive display from the VR interactive model component group according to the VR interactive model selection control instruction; Generate a real-time VR interaction result model based on the VR interaction model component and the VR interaction model component interaction control instruction, and display the real-time VR interaction result model in the VR model display scene structure; Obtaining model display video stream data of the real-time VR interaction result model; The model display video stream data is played in the VR video playback scene structure.

6. The method according to claim 5, characterized in that The generating of VR interactive model component control instructions based on user behavior data and / or the currently played target teaching video stream data includes: Generate a VR interactive model component default selection control instruction and a VR interactive model component default display control instruction based on the target teaching video stream data currently being played, wherein the VR interactive model component default display control instruction is used to represent a basic display mode of the VR interactive model component corresponding to the target teaching video stream data currently being played in an initial state; Performing control instruction recognition on the user behavior data to obtain VR interactive model component user selection control instructions and VR interactive model component user interaction control instructions, wherein the VR interactive model component user interaction control instructions are used to represent an interactive display mode of the VR interactive model component based on the user operation behavior; updating the VR interactive model component default selection control instruction according to the VR interactive model component user selection control instruction to obtain the VR interactive model component selection control instruction; The VR interactive model component real-time interactive control instruction and the VR interactive model component real-time interactive control instruction are integrated to generate the VR interactive model component interactive control instruction.

7. The method according to claim 6, characterized in that The user behavior data includes eye movement behavior data and gesture behavior data, and the control instruction recognition of the user behavior data to obtain the VR interactive model component user selection control instruction and the VR interactive model component user interaction control instruction includes: Using a timed aiming algorithm to identify control instructions from the eye movement behavior data, and generating user selection control instructions for the VR interactive model component; A machine learning gesture recognition algorithm is used to identify control instructions for the gesture behavior data to generate user interaction control instructions for the VR interactive model component.

8. An intelligent VR teaching system for enhancing video teaching effects, characterized in that: The system comprises: VR scene initialization module, used to initialize the VR video playback scene structure and VR model display scene structure; A basic data acquisition module is used to obtain user behavior data and target teaching video stream data in the VR video playback scene structure, and load the VR interactive model component group corresponding to the target teaching video stream data; A control instruction generation module is used to generate VR interactive model component control instructions based on user behavior data and / or the target teaching video stream data currently being played; The interactive result generation module is used to generate and display a real-time VR interactive result model in the VR model display scene structure according to the VR interactive model component group and the VR interactive model component control instructions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images