An immersive flipped classroom teaching system based on multiple virtual avatars and its working method

By introducing multi-virtual avatars and intelligent detection technologies in flipped classroom teaching, providing personalized interactive feedback and teaching strategies, the problem that traditional teaching models cannot meet the needs of different students is solved, and a more efficient and attractive learning experience is achieved. .

CN117348730BActive Publication Date: 2024-12-03JIANGSU FOOD & PHARMA SCI COLLEGE
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311349364.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-18
Publication Date
2024-12-03
Estimated Expiration
2043-10-18

AI Technical Summary

Technical Problem

The traditional flipped classroom teaching model lacks targetedness and cannot meet the learning needs and preferences of different students, resulting in poor learning results and experience quality.

Method used

Adopt an immersive flip classroom teaching system based on multiple virtual avatars, create personalized learning by building virtual teachers and virtual classrooms, using intelligent sensors and recognition algorithms to detect user characteristics and learning status in real time, providing personalized interactive feedback and teaching strategies, and creating personalized learning environment.

Benefits of technology

It improves learners' concentration and sense of participation, provides a realistic learning environment and personalized learning experience, optimizes the learning process and feedback mechanism, and significantly improves the learning effect and experience quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117348730B_ABST
    Figure CN117348730B_ABST
Patent Text Reader

Abstract

The present invention discloses an immersive flipped classroom teaching system based on multiple virtual avatars and a working method thereof, comprising a first interactive system and a head-mounted display system, wherein the head-mounted display system provides a VR scene to a user, and the first interactive system monitors and provides feedback on the VR scene provided by the head-mounted display system; the VR scene comprises a virtual teacher and a virtual classroom, and the virtual teacher interactively explains the content of the virtual classroom, and the method for constructing the virtual teacher comprises: constructing a virtual avatar database; detecting the characteristics of the user, outputting the detection result, and constructing a virtual avatar from the virtual avatar database according to the detection result; the virtual avatar monitors the learning content and the learning status of the user; and the virtual avatar provides interactive feedback to the user according to the monitored content.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of teaching systems, and in particular to an immersive flipped classroom teaching system based on multiple virtual avatars and a working method thereof. Background Art

[0002] As an efficient, convenient and flexible classroom teaching mode, the immersive flipped classroom teaching system uses the immersive, interactive, imaginative and intelligent characteristics of VR technology to build a teaching space beyond the screen limitation and create a new situational teaching environment with high immersion, natural and efficient interaction methods and audio-visual effects. Using virtual avatars as representatives of teachers and students in the virtual teaching environment can improve learners' concentration and sense of participation, and help stimulate their interest in learning. With the help of the immersive flipped classroom teaching system, virtual avatars with different functions are introduced into virtual teaching resources. Teachers can create different virtual teaching situations and carry out teaching activities such as pre-class preparation, in-class explanation, Q&A and communication, and post-class assessment; learners can independently prepare before class, ask questions, display and discuss in class, and review after class. However, the traditional flipped classroom teaching model lacks pertinence and cannot meet the learning needs and preferences of different students. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an immersive flipped classroom teaching system based on multiple virtual avatars and a working method thereof. Compared with the prior art, the disclosed scheme in the present invention can provide personalized teaching for different types of users.

[0004] The present invention is implemented through the following technical solutions: The present invention discloses an immersive flipped classroom teaching system based on multiple virtual avatars, including a first interactive system and a head-mounted display system, the head-mounted display system provides a VR scene to a user, and the first interactive system monitors and provides feedback on the VR scene provided by the head-mounted display system;

[0005] The VR scene includes a virtual teacher and a virtual classroom. The virtual teacher interactively explains the content of the virtual classroom. The methods of constructing the virtual teacher include:

[0006] Build an avatar database;

[0007] Detect the user's features and output the detection results

[0008] constructing a virtual avatar from a virtual avatar database according to the detection result;

[0009] The avatar monitors the learning content and the user’s learning status;

[0010] Based on the monitored content, the virtual avatar provides interactive feedback to the user.

[0011] Specifically, VR scenes include virtual teachers and virtual classrooms. The virtual teacher is a virtual entity that can provide interactive explanations based on the user's learning preferences and provide a personalized learning experience. The virtual teacher can interact with the user through voice, text, images and other forms, and flexibly adjust the teaching strategy based on the user's learning status and feedback. The virtual teacher also has the ability to understand the learning content and can provide targeted explanations and guidance based on the characteristics of the learning content.

[0012] The construction process of the virtual teacher includes the following steps: constructing a virtual avatar database, detecting the user's features and outputting the detection results, and constructing a virtual avatar from the virtual avatar database based on the detection results.

[0013] The teaching system detects user characteristics mainly by analyzing their voice, text, facial expressions, body movements, etc. These characteristics can be detected and identified in real time through intelligent sensors and recognition algorithms, and the results can be output.

[0014] Finally, the virtual avatar monitors the learning content and the user's learning status, and provides interactive feedback to the user based on the monitored content. This feedback can be timely reminders, explanations, and answers to questions, or targeted guidance and assistance based on the characteristics of the learning content. This interactive feedback mechanism can effectively improve the user's learning effect and experience quality.

[0015] Preferably, the avatar database includes one or more of appearance features, personality features, clothing features, voice features, and knowledge reserve features;

[0016] The steps of user feature detection include:

[0017] S100: Real-time judgment of the user's voice, text, facial expressions, and body movements;

[0018] S200: The user's physical condition, knowledge level test results, and personality test results are uploaded to the cloud;

[0019] S300: Campus teachers upload the judgment results of relevant users’ gender, age, and learning style to the cloud

[0020] S400: Combining the contents of steps S100 and S300, outputting the user's features to match the learning preferences, and determining the user's features based on the learning preferences.

[0021] Specifically, the avatar database is a database containing a variety of avatars, each of which has different appearance characteristics, personality characteristics, clothing characteristics, voice characteristics, knowledge reserve characteristics, etc. These characteristics can be combined according to certain rules to form various types of avatars to meet the learning preferences of different users.

[0022] The user feature detection step includes real-time judgment of the user's voice, text, facial expressions, and body movements, as well as uploading the user's physical condition, knowledge level test results, and personality test results to the cloud. These data can be detected and identified in real time through smart sensors and recognition algorithms, and stored and analyzed through the cloud.

[0023] Next, campus teachers upload the judgment results of the relevant users' gender, age, and learning style to the cloud. These data can help the system more accurately match the user's learning preferences and select the most suitable virtual avatar for construction.

[0024] Finally, combined with the contents of steps S100 and S300, the user's features are output to match the learning preferences, and the user features are determined based on the learning preferences. This step is the key to the entire feature detection process. It can comprehensively analyze the various features of the user and determine the most appropriate learning content and strategy based on the learning preferences. The above scheme not only provides a realistic learning environment and personalized learning experience, but also optimizes the learning process and feedback mechanism, which can effectively improve the user's learning effect and experience quality.

[0025] Preferably, the user's learning state includes the user's facial expression, the user's body movements, the user's voice and the user's text input;

[0026] The steps of interactive feedback include:

[0027] Identify whether the user is in an abnormal state based on the monitored content;

[0028] When an abnormal state is identified, the virtual avatar reminds the user;

[0029] The abnormal state refers to the user's language, facial expressions or body movements that are not conducive to learning, as well as irregular body movements.

[0030] Specifically, users' facial expressions and body movements can reflect their emotional state and level of learning engagement. For example, positive expressions usually indicate that they are interested and excited about the learning content; while negative expressions may indicate that they are confused or bored.

[0031] The user's voice and text input can reflect their learning progress and understanding. For example, the system can analyze the user's pronunciation and intonation through voice recognition technology to evaluate their oral expression ability; and through text input, the user can communicate and interact with the virtual teacher, and the system can understand the user's learning needs and problems.

[0032] After understanding the user's learning status, the interactive feedback step will identify whether the user is in an abnormal state based on the monitored content. Abnormal states here include users using language, expressions or body movements that are not conducive to learning, or performing irregular body movements. For example, if the user is obviously sleepy or distracted, the system can identify these abnormal states and remind or correct them in time.

[0033] Preferably, the first interactive system comprises:

[0034] A first bracket, the first bracket is arranged with a support surface, and the support surface is arranged close to the user's head;

[0035] A display device, the display device is mounted on the supporting surface of the first bracket;

[0036] Lower limb sensing system: The lower limb sensing system detects the status of the user's lower limbs. This system can detect the user's walking status, posture, etc. in real time through sensors and algorithms, and feed this information back to other parts of the system to synchronize the gait of the virtual character;

[0037] An upper limb sensing system detects the status of the user's upper limbs;

[0038] Among them, the display device shows the VR scene provided by the head-mounted display system to the user. The user will be able to see the virtual teacher and virtual classroom through the head-mounted display, and through the feedback from the lower limb and upper limb sensing systems, the virtual teacher can interact and explain according to the user's status, and outsiders can understand the specific viewing content of the students from the display device.

[0039] Preferably, the first interactive system further comprises a physical feedback system, and the structure of the physical feedback system comprises:

[0040] a first support platform, the first support platform supporting the first bracket and the user;

[0041] A lifting member, wherein three lifting members are arranged, a first end of which is arranged on the surface of the first supporting platform, and when the length of the lifting member changes, the distance between the second end of the lifting member and the first supporting platform changes;

[0042] a second supporting platform, the second supporting platform contacts the second end of the lifting member, and the first supporting platform and the second supporting platform are arranged opposite to each other;

[0043] The three lifting components are arranged in a triangle shape, the second support platform is arranged at the position where the user's feet are placed, and two lifting components are arranged at the position where the user's feet are placed. At the same time, the second support platform is also arranged at the position where the user's feet are placed, which makes the user's movements in the virtual environment more stable. In addition, there are two lifting components located at the position of the user's feet, which can help the system detect and feedback the user's lower limb status information more accurately. Through the above-mentioned physical feedback system, the user can interact with the virtual scene.

[0044] Preferably, the physical feedback system provides feedback to the user based on the teaching content of the virtual classroom, and the feedback steps include:

[0045] The teaching content shows the scenario selected by the user, and the selected scenario includes two options: left and right;

[0046] The user tilts the body and selects left or right options based on the tilt direction;

[0047] When the user's body is sensed to be tilted, the height of the lifting component in the corresponding position changes;

[0048] For example, in a scenario, when users are learning English words, memorizing words only through text descriptions may be abstract and difficult to understand. If combined with pictures, the meaning and application scenarios of words are presented in both visual and textual ways, it can help users understand the words more intuitively. Adjusting the tilt level can help combine words with pictures and provide a more immersive learning experience.

[0049] For example, when a user is learning the word "dog", he or she can choose the left or right tilt direction by bending different legs. At this time, the height of the lifting component at the corresponding position of the user's feet changes, causing the user's body to tilt. When the user tilts the body to an angle, the animal in the picture will also face the user accordingly, allowing the user to feel the meaning of the word and the corresponding application scenario more realistically. When the user selects the correct picture, the lifting component returns to its initial state and the user's body is balanced again. This interactive method can increase the fun of learning and make the user's memory more profound.

[0050] In addition to the combination of pictures and words, adjusting the tilt level can also achieve other forms of interaction and experience. For example, when learning grammar and sentence patterns, you can combine grammar and sentence patterns with virtual scenes by adjusting the tilt level. For example, when users learn the sentence pattern of "interrogative sentence", they can present the dialogue between virtual characters by adjusting the tilt level, allowing users to feel the tone and usage of interrogative sentences more realistically.

[0051] and / or,

[0052] The feedback steps include:

[0053] Identify that the user is in an abnormal state;

[0054] The height of the lifting member changes, causing the user's body to tilt;

[0055] For example, when the system detects that a user has some behaviors that are not conducive to learning, such as dozing off or talking to outsiders for a long time, it defines it as an abnormal state. This recognition is based on real-time monitoring and analysis of various behaviors and states of users during the learning process.

[0056] Once the system recognizes that the user is in an abnormal state, the system will proceed to the next step, and the height of the lifting component will change, causing the user's body to tilt: this is part of the feedback mechanism, and its purpose is to adjust the user's body posture to a state that is more conducive to learning. Specifically, when the system detects that the user is in an abnormal state, it will adjust the user's body posture by changing the height of the lifting component. For example, if the user dozes off, the height of the lifting component may decrease, causing the second support platform to rise, thereby reminding the user to stay awake and return to the correct viewing angle. This feedback mechanism can greatly improve the user's learning efficiency while also avoiding harm to the user's body.

[0057] and / or,

[0058] The feedback steps include:

[0059] Identify the dangerous factors in the user's environment;

[0060] The height of the lifting member changes, causing the user's body to tilt;

[0061] Among them, the degree of inclination of identifying the dangerous factors is greater than the degree of inclination of identifying the abnormal state of the user.

[0062] When the system detects possible dangerous factors in the user's environment, such as sudden obstacles, sudden natural disasters or other emergencies, it defines them as dangerous factors.

[0063] Once the system identifies dangerous factors in the user's environment, it will proceed to the next step, changing the height of the lifting component to tilt the user's body: this is part of the feedback mechanism, the purpose of which is to adjust the user's body posture to a state that is more conducive to dealing with danger. Specifically, when the system detects dangerous factors in the user's environment, it will adjust the user's body posture by changing the height of the lifting component to alert the user to dangerous factors in the surrounding environment. The degree of tilt when the dangerous factor is identified is greater than the degree of tilt when the user's abnormal state is identified. The user cannot study normally in this tilted state, so the user is forced to leave the VR scene.

[0064] The present invention also discloses a working method of an immersive flipped classroom teaching system based on multiple virtual avatars, which is executed through a teaching system. The working method comprises the following steps:

[0065] Set teaching objectives, prepare teaching video materials and upload them according to the teaching objectives;

[0066] Students learn and communicate independently through the teaching system, and the teaching system provides feedback to campus teachers on users' learning status and results, such as students' learning progress, academic performance, etc., and feeds this information back to campus teachers;

[0067] Campus teachers can provide guidance on problems encountered during the learning process based on the feedback of learning status and results. Campus teachers can understand the learning situation of each student through the system and provide targeted personalized tutoring and guidance. For example, teachers can provide more learning resources and guidance to students with slow learning progress; teachers can provide higher-level learning resources and challenges to students with excellent academic performance.

[0068] Based on the guidance of virtual teachers and campus teachers, users can display the results of learning content. Teachers are an important role in the teaching system and can provide personalized guidance and feedback based on students' learning needs and status. At the same time, users can also display their own learning results through the system, such as completing specific exercises and conducting virtual experiments.

[0069] This working method can effectively improve students' learning effects and experience quality. It stimulates students' learning motivation and creative thinking by providing them with immersive learning experience and personalized guidance services. It can also effectively improve students' learning outcomes and abilities. Compared with traditional teaching models, it is more flexible and adaptable and can meet the learning needs and personality development of different students.

[0070] In addition, this method can also improve the work efficiency and teaching quality of campus teachers. Through the automatic analysis and feedback of the teaching system, teachers can more accurately understand the learning status and needs of students, and thus provide targeted and effective guidance and resource support. At the same time, teachers can also interact with students in real time through the system, solve students' learning problems in a timely manner, and give students more support and encouragement.

[0071] Preferably, the step of displaying the results of the learning content includes:

[0072] Users demonstrate their technical learning results in class. This step of learners' direct presentation of learning content can be understood as a process of self-expression and communication. Through this process, learners can demonstrate their learned technical knowledge and skills in a physical form, and teachers can understand their learning status by observing learners' presentations.

[0073] The virtual teacher gives feedback on the user's learning problems. The teacher finds learning problems based on the results of the demonstration. In this step, the virtual teacher can find out the learners' problems in technology learning by analyzing their demonstrations, such as insufficient grasp of knowledge points and lack of proficiency in skill application. The virtual teacher feeds these problems back to the campus teachers to help them understand the learners' learning status more comprehensively.

[0074] According to the feedback of the problem, the teacher creates the corresponding learning situation. On the one hand, the campus teacher finds the problems of the learners in technology learning according to the feedback of the virtual teacher and his own observation, and creates the corresponding learning situation in a targeted manner. These situations can be new learning tasks, challenging practical problems, etc., aiming to help learners solve the problems they encounter. The campus teacher in the present invention is a teacher in a real environment, which is different from the virtual teacher;

[0075] Under the guidance of teachers, learners need to conduct in-depth analysis of the problems they encounter in the learning situation created by teachers. This process can help learners understand the problems more deeply and find strategies and methods to solve them;

[0076] Users can work together in groups and independently conduct exploration exercises. Under the guidance of teachers, users can solve learning problems through group cooperation. This can give full play to the power of the group and find the best way to solve problems through discussion and communication. At the same time, learners can also deeply master knowledge and skills in the process of exploration exercises;

[0077] The teacher provides guidance in groups. During the group cooperation process, the teacher can conduct rounds of guidance between each group to provide personalized guidance and help to learners. This guidance method can better meet the individual needs of learners and help them better master knowledge and skills. At the same time, it can also promptly correct the mistakes or deviations that learners make when solving problems;

[0078] In general, this working method of immersive flipped classroom teaching system based on multiple virtual avatars can effectively promote learners' learning and application ability of technology through the implementation of steps such as displaying learning outcomes in class, virtual teachers' feedback on problems, teachers creating learning situations, analyzing problems, group cooperative exploration exercises, and teacher guidance.

[0079] Preferably, the risk factors include one or more of a sudden change in ambient temperature, a sudden change in ambient air pressure, and abnormal sounds detected in the environment.

[0080] The present invention discloses an immersive flipped classroom teaching system based on multiple virtual avatars and a working method thereof. Compared with the prior art:

[0081] The VR scene in the present invention includes a virtual teacher and a virtual classroom. The virtual teacher is a virtual body that can interactively explain according to the user's learning preferences and provide a personalized learning experience. The virtual teacher can interact with the user in various forms such as sound, display text, and images, and flexibly adjust the teaching strategy according to the user's learning status and feedback. The virtual teacher also has the ability to understand the learning content and can provide targeted explanations and guidance according to the characteristics of the learning content. The construction process of the virtual teacher includes the following steps: building a virtual avatar database, detecting the user's characteristics and outputting the detection results, and building a virtual avatar from the virtual avatar database based on the detection results. The teaching system detects the user's characteristics mainly including analyzing their voice, text, facial expressions, body movements, etc. These characteristics can be detected and recognized in real time through intelligent sensors and recognition algorithms, and the results are output. Finally, the virtual avatar monitors the learning content and the user's learning status, and interactively feedbacks the user according to the monitored content. This feedback can be timely reminders, explanations, and answering questions, or it can be targeted guidance and assistance according to the characteristics of the learning content. This interactive feedback mechanism can effectively improve the user's learning effect and experience quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a three-dimensional schematic diagram of the teaching system in the present invention;

[0083] Figure 2 is a side view of the teaching system of the present invention;

[0084] Figure 3A flowchart of a virtual teacher component method in an embodiment

[0085] Figure 4 A schematic diagram of a virtual avatar monitoring process in an embodiment;

[0086] Figure 5 It is a structural schematic diagram of a physical feedback system in an embodiment;

[0087] Figure 6 This is a schematic structural diagram of a lifting component in one embodiment;

[0088] Figure 7 This is a teaching architecture diagram of a teaching system in one embodiment. DETAILED DESCRIPTION

[0089] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various exemplary embodiments of the present disclosure as defined by the claims and their equivalents. The description includes various details to assist in understanding, but these details are to be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and brevity.

[0090] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0091] It will be understood that when an element or layer is referred to as being "on", "connected to" or "coupled to" another element or layer, it may be directly on, directly connected to or coupled to the other element or layer, or one or more intermediate elements or layers may also be present. When an element is referred to as being "directly on" another element or layer, "directly connected to" or "directly coupled to" another element or layer, there is no intermediate element or layer. For example, when a first element is described as being coupled to or coupled to a second element, the first element may be directly coupled to or coupled to the second element, or the first element may be indirectly coupled to or coupled to the second element through one or more intermediate elements. The same reference numerals refer to the same elements. As used herein, the term "and / or" includes any and all combinations of one or more of the related listed items. In addition, when describing embodiments of the present invention, the use of "may" relates to "one or more embodiments of the present invention". When an expression such as "at least one of..." is after a list of elements, the entire list of modified elements is modified, and a single element in the list is not modified. In addition, the term "exemplary" is intended to refer to an example or legend. As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively.

[0092] It will be understood that, although the terms first, second, third, etc. may be used in this article to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited to these terms. These terms are used to distinguish an element, component, region, layer or section from another element, component, region, layer or section. Therefore, without departing from the teachings of the exemplary embodiments, the first element, first component, first region, first layer or first section discussed below may be referred to as the second element, second component, second region, second layer or second section. In the accompanying drawings, for clarity of explanation, the sizes of various elements, layers, etc. may be exaggerated.

[0093] For ease of description, spatially relative terms such as "beneath", "below", "lower", "above", "upper", etc. may be used herein to describe the relationship of one element or feature to another element (or elements) or feature (or features) as shown in the drawings. It will be understood that in addition to the orientation depicted in the drawings, the spatially relative terms are intended to also include different orientations of the device in use or operation. For example, if the device in the drawings is flipped, the elements described as being "below" or "below" other elements or features will be oriented as being "above" or "above" the other elements or features. Thus, the term "below" may include both orientations of above and below. The device may be otherwise oriented (e.g., rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0094] The terms used herein are only for the purpose of describing specific exemplary embodiments of the present invention and are not intended to limit the exemplary embodiments of the present invention described. As used herein, the singular forms "a" and "an" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when used in this specification, the terms "include", "including", "comprise" and / or "comprising" specify the presence of features, wholes, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, parts and / or groups thereof.

[0095] The present invention is implemented through the following technical solutions: an immersive flipped classroom teaching system based on multiple virtual avatars, such as Figure 1 and Figure 2 As shown, it includes a first interactive system 1 and a head-mounted display system 2, the head-mounted display system 2 provides a VR scene to the user, and the first interactive system monitors and provides feedback on the VR scene provided by the head-mounted display system 2;

[0096] VR scenes include virtual teachers and virtual classrooms. The virtual teachers interactively explain the contents of the virtual classrooms. Figure 3 As shown, the method of the virtual teacher component includes:

[0097] Step 1: Build an avatar database;

[0098] Step 2: Detect the user's features and output the detection results;

[0099] Step 3: constructing a virtual avatar from the virtual avatar database according to the detection results;

[0100] Step 4: The virtual avatar monitors the learning content and the user’s learning status;

[0101] Step 5: Based on the monitored content, the virtual avatar provides interactive feedback to the user.

[0102] Specifically, VR scenes include virtual teachers and virtual classrooms. The virtual teacher is a virtual entity that can provide interactive explanations based on the user's learning preferences and provide a personalized learning experience. The virtual teacher can interact with the user through voice, text, images and other forms, and flexibly adjust the teaching strategy based on the user's learning status and feedback. The virtual teacher also has the ability to understand the learning content and can provide targeted explanations and guidance based on the characteristics of the learning content.

[0103] The construction process of the virtual teacher includes the following steps: constructing a virtual avatar database, detecting the user's features and outputting the detection results, and constructing a virtual avatar from the virtual avatar database based on the detection results. The present invention does not make any specific limitation on the construction of the virtual character.

[0104] In one embodiment, the step of creating a virtual character includes:

[0105] Prepare a 3D model: First, you need to prepare a 3D model of a virtual character. This model can be created in 3D modeling software such as Blender;

[0106] Import virtual character model: Import the created virtual character model into a VR editor, such as Unity or Unreal Engine.

[0107] The teaching system detects user characteristics mainly by analyzing their voice, text, facial expressions, body movements, etc. These characteristics can be detected and identified in real time through intelligent sensors and recognition algorithms, and the results can be output.

[0108] Finally, the virtual avatar monitors the learning content and the user's learning status, and provides interactive feedback to the user based on the monitored content. This feedback can be timely reminders, explanations, and answers to questions, or targeted guidance and assistance based on the characteristics of the learning content. This interactive feedback mechanism can effectively improve the user's learning effect and experience quality.

[0109] like Figure 4As shown, in step 4, the user learns in the virtual classroom and interacts with the virtual teacher. A first monitoring module is arranged, whose main function is to monitor the user's status. The user's status may include physical status (such as sitting posture, sight direction, etc.), emotional status (such as facial expression, voice tone, etc.) and other learning behavior status (such as learning time, progress, etc.). The first monitoring module obtains and processes this information in real time through various sensors and algorithms, and transmits this information to the virtual teacher.

[0110] At the same time, the system is also equipped with a second monitoring module, whose main function is to monitor the status of the virtual classroom. The status of the virtual classroom can include the utilization of learning resources (such as the reading progress of learning materials, the completion of online tests, etc.), interaction (such as participation in online discussions, the frequency of real-time interactions, etc.) and other environmental factors (such as classroom atmosphere, noise level, etc.). The second monitoring module also obtains and processes this information in real time through various sensors and algorithms, and transmits this information to the virtual teacher.

[0111] Finally, after receiving the information from the first monitoring module and the second monitoring module, the virtual teacher will interact with the user based on the information. For example, if the user appears to be sleepy or distracted, the virtual teacher can adjust the teaching content or remind the user to focus; if the learning resources of the virtual classroom are on the key content, the virtual teacher will interact with the user to ask questions and consolidate the knowledge reserve.

[0112] In one embodiment, the avatar database includes one or more of appearance features, personality features, clothing features, voice features, and knowledge reserve features;

[0113] The steps of user feature detection include:

[0114] S100: Real-time judgment of the user's voice, text, facial expressions, and body movements;

[0115] S200: The user's physical condition, knowledge level test results, and personality test results are uploaded to the cloud;

[0116] S300: Campus teachers upload the judgment results of relevant users’ gender, age, and learning style to the cloud

[0117] S400: Combining the contents of steps S100 and S300, outputting the user's features to match the learning preferences, and determining the user's features based on the learning preferences.

[0118] Specifically, the avatar database is a database containing a variety of avatars, each of which has different appearance characteristics, personality characteristics, clothing characteristics, voice characteristics, knowledge reserve characteristics, etc. These characteristics can be combined according to certain rules to form various types of avatars to meet the learning preferences of different users.

[0119] The user feature detection step includes real-time judgment of the user's voice, text, facial expressions, and body movements, as well as uploading the user's physical condition, knowledge level test results, and personality test results to the cloud. These data can be detected and identified in real time through smart sensors and recognition algorithms, and stored and analyzed through the cloud.

[0120] Campus teachers upload the judgment results of relevant users' gender, age, and learning style to the cloud. These data can help the system more accurately match users' learning preferences and select the most suitable virtual avatar for construction.

[0121] Combined with the contents of steps S100 and S300, the user's features are output to match the learning preferences, and the user features are determined based on the learning preferences. This step is the key to the entire feature detection process. It can comprehensively analyze the various features of the user and determine the most appropriate learning content and strategy based on the learning preferences. Through the above scheme, not only a realistic learning environment and personalized learning experience are provided, but also the learning process and feedback mechanism are optimized, which can effectively improve the user's learning effect and experience quality.

[0122] In one embodiment, the user's learning state includes the user's facial expressions, the user's body movements, the user's voice, and the user's text input;

[0123] The steps of interactive feedback include:

[0124] Identify whether the user is in an abnormal state based on the monitored content;

[0125] When an abnormal state is identified, the virtual avatar reminds the user;

[0126] The abnormal state refers to the user's language, facial expressions or body movements that are not conducive to learning, as well as irregular body movements.

[0127] Specifically, users' facial expressions and body movements can reflect their emotional state and level of learning engagement. For example, positive expressions usually indicate that they are interested and excited about the learning content; while negative expressions may indicate that they are confused or bored.

[0128] The user's voice and text input can reflect their learning progress and understanding. For example, the system can analyze the user's pronunciation and intonation through voice recognition technology to evaluate their oral expression ability; and through text input, the user can communicate and interact with the virtual teacher, and the system can understand the user's learning needs and problems.

[0129] After understanding the user's learning status, the interactive feedback step will identify whether the user is in an abnormal state based on the monitored content. Abnormal states here include users using language, expressions or body movements that are not conducive to learning, or performing irregular body movements. For example, if the user is obviously sleepy or distracted, the system can identify these abnormal states and remind or correct them in time.

[0130] In one embodiment, the first interactive system 1 includes:

[0131] A first bracket 11, wherein the first bracket 11 is provided with a support surface, and the support surface is arranged close to the user's head;

[0132] Display device 12, the display device 12 is installed on the supporting surface of the first bracket 11;

[0133] The lower limb sensing system 14 detects the state of the user's lower limbs. This system can detect the user's walking state, posture, etc. in real time through sensors and algorithms, and feed this information back to other parts of the system for synchronizing the gait of the virtual character;

[0134] An upper limb sensing system 15, which detects the upper limb status of the user;

[0135] Among them, the display device 12 displays the VR scene provided by the head-mounted display system 2 to the user. The user will be able to see the virtual teacher and the virtual classroom through the head-mounted display, and through the feedback from the lower limb and upper limb sensing systems, the virtual teacher can interact and explain according to the user's status, and outsiders can understand the students' specific viewing content from the display device 12.

[0136] Another example Figure 5 and Figure 6 As shown, the first interactive system 1 further includes a physical feedback system 13, and the structure of the physical feedback system 13 includes:

[0137] A first supporting platform 131, the first supporting platform supports the first bracket 11 and the user;

[0138] A lifting member 132, wherein three lifting members 132 are arranged, and a first end is arranged on the surface of the first supporting platform 131, and when the length of the lifting member 132 changes, the distance between the second end of the lifting member 132 and the first supporting platform 131 changes;

[0139] A second supporting platform, the second supporting platform contacts the second end of the lifting member 132, and the first supporting platform 131 and the second supporting platform are arranged opposite to each other;

[0140] Among them, the positions of the three lifting components 132 are arranged in a triangle shape, the second support platform is arranged at the position where the user's feet are placed, and two lifting components 132 are arranged at the position where the user's feet are placed. At the same time, the second support platform is also arranged at the position where the user's feet are placed, which makes the user's actions more stable in the virtual environment. In addition, there are two lifting components located at the position of the user's feet, which can help the system more accurately detect and feedback the user's lower limb status information. Through the above-mentioned physical feedback system 13, the user can interact with the virtual scene.

[0141] In one embodiment, the physical feedback system 13 provides feedback to the user based on the teaching content of the virtual classroom, and the feedback steps include:

[0142] The teaching content shows the scenario selected by the user, and the selected scenario includes two options: left and right;

[0143] The user tilts the body and selects left or right options based on the tilt direction;

[0144] When the user's body is sensed to be tilted, the height of the lifting member 132 in the corresponding position changes;

[0145] For example, in a scenario, when users are learning English words, memorizing words only through text descriptions may be abstract and difficult to understand. If combined with pictures, the meaning and application scenarios of words are presented in both visual and textual ways, it can help users understand the words more intuitively. Adjusting the tilt level can help combine words with pictures and provide a more immersive learning experience.

[0146] When the user is learning the word "dog", he can choose the left or right tilt direction by bending his legs in different ways. At this time, the height of the lifting component at the corresponding position of the user's feet changes, causing the user's body to tilt. When the user tilts his body to an angle, the animal in the picture will also face the user accordingly, allowing the user to feel the meaning of the word and the corresponding application scenario more realistically. When the user selects the correct picture, the lifting component returns to its initial state and the user's body is balanced again. This interactive method can increase the fun of learning and make the user's memory more profound.

[0147] In addition to the combination of pictures and words, adjusting the tilt level can also achieve other forms of interaction and experience. For example, when learning grammar and sentence patterns, you can combine grammar and sentence patterns with virtual scenes by adjusting the tilt level. For example, when users learn the sentence pattern of "interrogative sentence", they can present the dialogue between virtual characters by adjusting the tilt level, allowing users to feel the tone and usage of interrogative sentences more realistically.

[0148] In yet another embodiment, the step of providing feedback includes:

[0149] Identify that the user is in an abnormal state;

[0150] The height of the lifting member 132 changes, causing the user's body to tilt;

[0151] For example, when the system detects that a user has some behaviors that are not conducive to learning, such as dozing off or talking to outsiders for a long time, it defines it as an abnormal state. This recognition is based on real-time monitoring and analysis of various behaviors and states of users during the learning process.

[0152] Once the system recognizes that the user is in an abnormal state, the system will proceed to the next step, and the height of the lifting member 132 will change, causing the user's body to tilt: this is part of the feedback mechanism, the purpose of which is to adjust the user's body posture to a state that is more conducive to learning. Specifically, when the system detects that the user is in an abnormal state, the user's body posture will be adjusted by changing the height of the lifting member. For example, if the user dozes off, the height of the lifting member may decrease, causing the second support platform to rise, thereby reminding the user to stay awake and return to the correct viewing angle. This feedback mechanism can greatly improve the user's learning efficiency while also avoiding damage to the user's body.

[0153] In yet another embodiment, the step of providing feedback includes:

[0154] Identify the dangerous factors in the user's environment;

[0155] The height of the lifting member 132 changes, causing the user's body to tilt;

[0156] Among them, the degree of inclination of identifying the dangerous factors is greater than the degree of inclination of identifying the abnormal state of the user.

[0157] When the system detects possible dangerous factors in the user's environment, such as sudden obstacles, sudden natural disasters or other emergencies, it defines them as dangerous factors.

[0158] Once the dangerous factors in the user's environment are identified, the system will proceed to the next step, and the height of the lifting member 132 will change, causing the user's body to tilt: this is part of the feedback mechanism, the purpose of which is to adjust the user's body posture to a state that is more conducive to dealing with danger. Specifically, when the system detects dangerous factors in the user's environment, it will adjust the user's body posture by changing the height of the lifting member to alert the user to the dangerous factors in the surrounding environment. The tilt degree of the dangerous factor is greater than the tilt degree of the abnormal state of the user. The user cannot study normally in this tilted state, so as to force the user to leave the VR scene.

[0159] The present invention also discloses a working method of an immersive flipped classroom teaching system based on multiple virtual avatars, which is executed by the teaching system, such as Figure 7 As shown, the working method comprises the steps of:

[0160] Set teaching objectives, prepare teaching video materials and upload them according to the teaching objectives;

[0161] Students learn and communicate independently through the teaching system, and the teaching system provides feedback to campus teachers on users' learning status and results, such as students' learning progress, academic performance, etc., and feeds this information back to campus teachers;

[0162] Campus teachers can provide guidance on problems encountered during the learning process based on the feedback of learning status and results. Campus teachers can understand the learning situation of each student through the system and provide targeted personalized tutoring and guidance. For example, teachers can provide more learning resources and guidance to students with slow learning progress; teachers can provide higher-level learning resources and challenges to students with excellent academic performance.

[0163] Based on the guidance of virtual teachers and campus teachers, users can display the results of learning content. Teachers are an important role in the teaching system and can provide personalized guidance and feedback based on students' learning needs and status. At the same time, users can also display their own learning results through the system, such as completing specific exercises and conducting virtual experiments.

[0164] This working method can effectively improve students' learning effects and experience quality. It stimulates students' learning motivation and creative thinking by providing them with immersive learning experience and personalized guidance services. It can also effectively improve students' learning outcomes and abilities. Compared with traditional teaching models, it is more flexible and adaptable and can meet the learning needs and personality development of different students.

[0165] In addition, this method can also improve the work efficiency and teaching quality of campus teachers. Through the automatic analysis and feedback of the teaching system, teachers can more accurately understand the learning status and needs of students, and thus provide targeted and effective guidance and resource support. At the same time, teachers can also interact with students in real time through the system, solve students' learning problems in a timely manner, and give students more support and encouragement.

[0166] The steps for displaying the results of learning content include:

[0167] Users demonstrate their technical learning results in class. This step of learners' direct presentation of learning content can be understood as a process of self-expression and communication. Through this process, learners can demonstrate their learned technical knowledge and skills in a physical form, and teachers can understand their learning status by observing learners' presentations.

[0168] The virtual teacher gives feedback on the user's learning problems. The teacher finds learning problems based on the results of the demonstration. In this step, the virtual teacher can find out the learners' problems in technology learning by analyzing their demonstrations, such as insufficient grasp of knowledge points and lack of proficiency in skill application. The virtual teacher feeds these problems back to the campus teachers to help them understand the learners' learning status more comprehensively.

[0169] According to the feedback of the problem, the teacher creates the corresponding learning situation. On the one hand, the campus teacher finds the problems of the learners in technology learning according to the feedback of the virtual teacher and his own observation, and creates the corresponding learning situation in a targeted manner. These situations can be new learning tasks, challenging practical problems, etc., aiming to help learners solve the problems they encounter. The campus teacher in the present invention is a teacher in a real environment, which is different from the virtual teacher;

[0170] Under the guidance of teachers, learners need to conduct in-depth analysis of the problems they encounter in the learning situation created by teachers. This process can help learners understand the problems more deeply and find strategies and methods to solve them;

[0171] Users can work together in groups and independently conduct exploration exercises. Under the guidance of teachers, users can solve learning problems through group cooperation. This can give full play to the power of the group and find the best way to solve problems through discussion and communication. At the same time, learners can also deeply master knowledge and skills in the process of exploration exercises;

[0172] The teacher provides guidance in groups. During the group cooperation process, the teacher can conduct rounds of guidance between each group to provide personalized guidance and help to learners. This guidance method can better meet the individual needs of learners and help them better master knowledge and skills. At the same time, it can also promptly correct the mistakes or deviations that learners make when solving problems;

[0173] In general, this working method of immersive flipped classroom teaching system based on multiple virtual avatars can effectively promote learners' learning and application ability of technology through the implementation of steps such as displaying learning outcomes in class, virtual teachers' feedback on problems, teachers creating learning situations, analyzing problems, group cooperative exploration exercises, and teacher guidance.

[0174] In summary, the VR scene in the present invention includes a virtual teacher and a virtual classroom. The virtual teacher is a virtual entity that can interactively explain according to the user's learning preferences and provide a personalized learning experience. The virtual teacher can interact with the user through voice, display text, images and other forms, and flexibly adjust the teaching strategy according to the user's learning status and feedback. The virtual teacher also has the ability to understand the learning content and can provide targeted explanations and guidance according to the characteristics of the learning content. The construction process of the virtual teacher includes the following steps: constructing a virtual avatar database, detecting the user's characteristics and outputting the detection results, and constructing a virtual avatar from the virtual avatar database based on the detection results. The teaching system detects the user's characteristics mainly including analyzing their voice, text, facial expressions, body movements, etc. These characteristics can be detected and recognized in real time by intelligent sensors and recognition algorithms, and the results are output. Finally, the virtual avatar monitors the learning content and the user's learning status, and interactively feedbacks the user according to the monitored content. This feedback can be timely reminders, explanations, and answers to questions, or it can be targeted guidance and assistance according to the characteristics of the learning content. This interactive feedback mechanism can effectively improve the user's learning effect and experience quality.

[0175] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0176] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An immersive flipped classroom teaching system based on multiple virtual avatars, It is characterized in that The system comprises a first interactive system (1) and a head-mounted display system (2), wherein the head-mounted display system (2) provides a VR scene to a user, and the first interactive system monitors and provides feedback on the VR scene provided by the head-mounted display system (2); The VR scene includes a virtual teacher and a virtual classroom, the virtual teacher interactively explains the content of the virtual classroom, and the method of the virtual teacher component includes: Build an avatar database; Detect the user's features and output the detection results; constructing a virtual avatar from a virtual avatar database according to the detection result; The virtual avatar monitors the learning content and the user’s learning status; Based on the monitored content, the virtual avatar provides interactive feedback to the user; The first interactive system (1) comprises: A first bracket (11), the first bracket (11) being provided with a support surface, the support surface being arranged close to the user's head; A display device (12), the display device (12) being mounted on a supporting surface of the first bracket (11); A lower limb sensing system (14), wherein the lower limb sensing system detects the state of the user's lower limbs; An upper limb sensing system (15), the upper limb sensing system detecting the status of the upper limbs of the user; Wherein, the display device (12) displays the VR scene provided to the user by the head mounted display system (2); The first interactive system (1) further comprises a physical feedback system (13), the structure of the physical feedback system (13) comprising: A first support platform (131), the first support platform supporting the first bracket (11) and a user; A lifting component (132), wherein three lifting components (132) are arranged, and a first end is arranged on the surface of the first supporting platform (131), and when the length of the lifting component (132) changes, the distance between the second end of the lifting component (132) and the first supporting platform (131) changes; a second supporting platform, the second supporting platform contacting the second end of the lifting member (132), the first supporting platform (131) and the second supporting platform being arranged opposite to each other; The three lifting components (132) are arranged in a triangular shape, the second support platform is arranged at a position where the user's feet are placed, and two lifting components (132) are arranged at the position where the user's feet are placed, so that the stability of the system can be improved; The physical feedback system (13) provides feedback to the user based on the teaching content of the virtual classroom. The feedback steps include: Identify that the user is in an abnormal state; The height of the lifting member (132) changes, causing the user's body to tilt; and, The feedback steps include: Identify the dangerous factors in the user's environment; The height of the lifting member (132) changes, causing the user's body to tilt; Among them, the degree of inclination of identifying the dangerous factors is greater than the degree of inclination of identifying the abnormal state of the user; The virtual avatar database includes one or more of appearance features, personality features, clothing features, voice features, and knowledge reserve features; The step of detecting the user's features comprises: S100: Real-time judgment of the user's voice, text, facial expressions, and body movements; S200: The user's physical condition, knowledge level test results, and personality test results are uploaded to the cloud; S300: Campus teachers upload the judgment results of the gender, age, and learning style of relevant users to the cloud; S400: Combining the contents of steps S100 and S300, outputting the user's features to match the learning preferences, and determining the user's features according to the learning preferences; The user's learning state includes the user's facial expression, the user's body movement, the user's voice and the user's text input; The steps of interactive feedback include: Identify whether the user is in an abnormal state based on the monitored content; When an abnormal state is identified, the virtual avatar reminds the user; The abnormal state is the user's language, expression or body movements that are not conducive to learning, as well as irregular body movements; Also included is a working method, the working method comprising the steps of: Set teaching objectives, prepare teaching video materials and upload them according to the teaching objectives; Students learn and communicate independently through the teaching system, and the teaching system provides feedback to campus teachers on the user's learning status and results; Campus teachers provide guidance on problems encountered during the user's learning process based on the feedback on learning status and results; Users demonstrate the results of their learning content based on the guidance of virtual teachers and campus teachers; The steps to demonstrate the results of learning content include: Users demonstrate their technical learning achievements in class; The virtual teacher gives feedback to the user on their learning problems, and the teacher discovers the learning problems based on the displayed results; Based on the feedback from the questions, the teacher creates the corresponding learning situation; Under the guidance of teachers, conduct in-depth analysis of the problems that arise; Users work together in groups and conduct independent inquiry exercises; The teacher provides guidance according to the groups.

2. The immersive flipped classroom teaching system based on multiple virtual avatars as claimed in claim 1, It is characterized in that The risk factors include one or more of a sudden change in ambient temperature, a sudden change in ambient air pressure, and abnormal sounds detected in the environment.