Virtual teacher interaction method and system driven by big education model fused with knowledge graph

Abstract

The invention relates to a virtual teacher interaction method and system driven by an education large model fused with a knowledge graph, and belongs to the field of artificial intelligence and wisdom education. The method comprises the following steps: constructing a data set and a question and answer model; application of function interaction; and through virtual teacher and scene driving, natural interaction of a virtual learning environment is realized. Wherein the question and answer model not only needs a data set to provide corpus data support for the question and answer model, but also needs to convert voice into a text and perform intention classification and session management by a subject knowledge framework, then obtains optimal feedback through an education big model, a subject knowledge graph and the like, and finally outputs the optimal feedback in a voice form; the function interaction comprises communication connection, GUI (Graphical User Interface) middle station control and result interaction. And the virtual teacher is responsible for receiving feedback information of the question and answer model so as to drive the virtual teacher to generate human-like postures and actions and change of teaching scenes. According to the invention, immersive and personalized teaching experience and convenient online question answering service can be provided for the user, so that diversified learning requirements of the user are met.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and smart education, and relates to a virtual teacher interaction method and system driven by a large education model integrated with a knowledge graph. Background Art

[0002] As the concept of the metaverse is being sought after by the market, virtual reality technology has also developed rapidly. Virtual human technology, which is based on virtual reality and uses digital technology to build three-dimensional models, has entered a period of rapid development. In the field of education, as user needs and cognitive levels continue to improve, people hope that virtual teachers can provide a more interactive and immersive learning experience and personalized guidance, truly realizing the educational ideals of "teaching without distinction" and "teaching students in accordance with their aptitude."

[0003] However, the current traditional virtual teachers have almost no good perception, cognition and behavior abilities, and cannot make any achievements in the field of professional knowledge. Specifically, in the current smart education scenario, the application of virtual teachers still has the following prominent problems: (1) Lack of realistic human body geometry expression. The application of virtual human technology in the field of education is not mature enough, and there are not many virtual teacher systems with real human body geometry; (2) Lack of real human body movement control. In the existing virtual teacher systems, there are only head images, or although there is a whole human body, most of them are static postures, without human body movement control, and lack of rich body language communication; (3) Lack of diversified interaction methods. At present, almost all interactive learning assistants use text information for interaction, lack of voice and animation interaction methods, which increases the difficulty for students to obtain information; (4) Lack of personalization and intelligence. In the existing virtual teacher systems, there are few virtual teachers for specific tasks, and most of them cannot provide personalized answers to specific knowledge points, making it difficult to support intelligent teaching tasks in immersive teaching environments, and the real-time and accuracy of the answers need to be improved. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a virtual teacher interaction method and system driven by a large educational model integrated with a knowledge graph.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A virtual teacher interaction method driven by a large educational model integrated with a knowledge graph, the method comprising the following steps:

[0007] S1. Establish a data set serving the question-answering model module through the corpus construction module. It provides corpus data support for the question-answering model through large model data fine-tuning, knowledge graph data set construction, and subject dialogue framework data set training;

[0008] S2. Build a question-answering model module, in which the student's output speech is first converted into recognizable text and then classified and managed by the subject knowledge framework. Then, the optimal feedback is obtained through the locally deployed education model and subject knowledge graph, and finally converted into speech output;

[0009] S3, realize virtual teacher control and interaction through the functional interaction layer, which includes communication connection, GUI control middle platform and result interaction. The communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher. The GUI control middle platform and result interaction enrich the user experience of the system in human-computer interaction through multi-modal input and personalized teacher attribute editing.

[0010] S4. Build an application presentation layer, which receives input from the functional interaction layer to drive the virtual teacher to generate human-like facial gestures, body movements, and generate realistic teaching scenes.

[0011] Further, in step S1, the corpus construction module is used to construct a fine-tuning dataset of the education model, a corpus training dataset of the subject dialogue framework, and a knowledge graph dataset; wherein,

[0012] The multi-task fine-tuning learning method is used to fine-tune the educational model. The process is as follows: generate continuous prompt words Promp, and only train the parameters of the prompt word Prompt part. The parameters of the educational model are fixed and unchanged, which can be expressed as:

[0013] h0,…,hi,e(x),hi+1,…,hm,e(y)

[0014] Where h represents the generated continuous prompt representation, e represents a pre-trained educational model, x represents the original input of the data, and y represents the label of the data;

[0015] The corpus training data set for the subject dialogue framework includes Date and Domain files. The NLU training data in the Date file includes the text entered by the user, the corresponding intent and entity; the Domain file specifies the intent, entity, slot and action;

[0016] For the knowledge graph dataset, students' questions are collected from various angles to construct a question dataset; multiple types of entities and relationships are established based on the fact triples in the subject knowledge graph and the concept ontology layer, and multiple question intent categories are established based on the question intent. Each question intent category corresponds to the query requirements in the knowledge graph database; at the same time, data enhancement methods are used to expand the dataset, including synonym replacement, word order reversal, subject replacement, and establishment of multiple question templates.

[0017] Further, in step S2, a question-answering model is set in the question-answering model module. The question-answering model is composed of an education model, a subject knowledge graph, a subject dialogue framework and a Graph RAG module. The education model is implemented based on its autoencoding idea, autoregression idea, Span shuffling and two-dimensional position encoding technology. In the regression space filling task, given an input text x = [x1,…x n ], sampling to obtain multiple text spans{s1,…s m}; Randomly arrange the order of spans to obtain all possible arrangement sets Z m ,in: And train according to the following goals:

[0018]

[0019] In the formula, p θ represents the probability factor, Represents a token, x corrupt Indicates corrupted text, represents a continuation token;

[0020] In terms of position coding, the educational model adopts the rotation position embedding coding method. The rotation position coding realizes relative position coding through absolute position coding at the starting point, and adds absolute position information at its target point through the following operations:

[0021]

[0022] In the formula, represents a two-dimensional row vector, m and n are absolute positions, q and k are query vector and key vector respectively, and f(·) is a position operation;

[0023] The educational model uses a standard self-attention mechanism to calculate the dot product between the query and the key as the weight, and normalizes it through the softmax function:

[0024]

[0025] Where Q, K, and V are query vectors, key vectors, and value vectors, respectively. d represents the dimension of the key vector, and softmax(·) is the normalization function.

[0026] Furthermore, the subject knowledge graph of the constructed question-answering model includes the subject knowledge ontology constructed using the ontology construction tool, and subject knowledge extraction, knowledge fusion and storage are performed on the subject knowledge ontology. For subject knowledge extraction, various sequence annotation models that integrate domain vocabulary information are used to identify entities in subject textbooks and teaching outlines, and then deep learning models are combined to extract relationships to obtain triples between subjects and entities; for knowledge fusion and storage, cosine similarity is used to complete entity alignment, and finally the triples are stored in the graph database;

[0027] The subject dialogue framework includes the NLU module and the Core module. The NLU module is used to process the natural language input by the user and convert it into structured data. The NLU module includes intent classification, entity recognition and slot filling parts. The Core module includes a dialogue state tracker and a dialogue strategy selector, which are used to hold a dialogue and determine the next operation. The subject dialogue framework is used for dialogue management, intent recognition and entity extraction. The education model and subject knowledge graph are used as generative models.

[0028] The Graph RAG module combines the structured data and generation capabilities of the knowledge graph to process complex data set information. It processes data in the following steps:

[0029] (1) User query: First, receive the user's query;

[0030] (2) Search for similar entities: Identify a set of entities related to the user input semantics from the knowledge graph and use the searched entities as the entry point into the knowledge graph;

[0031] (3) Text unit mapping and relationship extraction: The extracted text units are mapped to the corresponding entities, and the original text information is removed;

[0032] (4) Understanding intent: Understanding the user’s intent and context based on the conversation history;

[0033] (5) Generate responses: Generate and respond to user queries based on the generated filtered and sorted data.

[0034] Furthermore, the question-answering model module uses automatic speech recognition ASR and text-to-speech TTS technology to transmit information between the system backend and frontend. ASR acts on the voice signal input by the user to convert the voice into text; TTS converts the feedback text into natural speech.

[0035] Further, in step S3, the communication connection module in the functional interaction layer adopts HTTP and WebSocket communication protocols, and different communication protocols are set according to the communication between different modules. Among them, when the front end communicates with the GUI control center, the WebSocket communication protocol that supports real-time two-way communication is selected, a WebSocket object is created in the front end, and a message receiving and replying module is defined in the back end to establish communication; in the communication between the back-end modules and between the back end and the GUI control center, the HTTP communication protocol is adopted to define the server and the client, and the client sends a request to the server and receives a response through the specified communication address, and the server also receives the request and processes the response through the address;

[0036] The GUI control center in the functional interaction layer uses CSS and HTML to build the user interface. Its interaction includes editing the teacher's personal information, including name, age, timbre, etc.; controlling the opening and closing of the front-end teacher service and displaying the teacher's connection status; supporting students to manually input text to interact with the virtual teacher, thereby realizing multimodal input. The result interaction module includes the center dialog output and the front-end control output; the center dialog output is for the case where the input is a text signal, and the front-end control output is for the case where the input is a voice signal.

[0037] Further, in step S4, the virtual teacher behavior module of the application presentation layer is first used to construct the virtual teacher image, which includes the following steps:

[0038] Firstly, multiple groups of facial images of people from multiple angles are obtained;

[0039] Import the photo into the 3D model making software Metashape, convert the sparse facial point cloud into a dense facial point cloud and generate a character mesh model, import the facial mesh into the UE5 platform, and wrap the MetaHuman model on the mesh surface through facial feature point tracking and body solving.

[0040] After secondary editing and reshaping of key points on the MetaHuman platform, a personalized virtual human image is generated;

[0041] Finally, the generated virtual teacher image is imported into the virtual teaching platform through Bridge.

[0042] Furthermore, the gestures and movements of the virtual teacher include facial gestures and body movements. The facial gestures of the virtual teacher include: blink control, expression control, and lip shape control; the body movements of the virtual teacher include: quiet state, teaching state, and listening state;

[0043] Among them, facial gesture control uses a visual script system, which uses nodes and connecting lines to represent functions, variables and logical processes in the program to achieve facial gesture control; for blink control, a modification curve is introduced in the script system, and a CTRL Expressions Eye Blink module is added. The Alpha value is used to control eye closure, where 1 represents completely closed eyes and 0 represents completely open eyes. At the same time, a time axis variable is introduced to make its value change periodically over time and pass it to Alpha, so as to control the periodic and natural changes of blinking expressions;

[0044] In expression control, select several basic expressions from the expression library of the script system and map the basic expressions to the facial expressions of the virtual teacher, thereby driving the changes in the teacher's expressions; the newly derived expression sequence can be applied to any state of the virtual teacher, and can also be mixed with each other to generate complex expressions, and then integrated into the basic expression module;

[0045] For lip shape control, the answer feedback from the question-and-answer model is passed downward in the corresponding format; the request containing the answer content is passed to the corresponding API, at which point the front end receives an audio file response containing the answer content, which is then processed through the pinyin analysis of the lip shape blueprint to classify the phonemes, each phoneme corresponding to a unique lip shape, and then the answer content is divided into different lip shape animations to drive the virtual teacher.

[0046] Furthermore, animation generation technology is used to control the body movements of the virtual teacher to ensure natural and smooth movements; through action blueprint technology, the three states of the virtual teacher, quiet, teaching, and listening, are reasonably converted to achieve internal logic control; when there is no interaction, the virtual teacher remains quiet; when talking with students, it switches to a listening state; when explaining teaching content, it is in a teaching state. In addition, the system uses the virtual teaching scene module of the presentation layer to build the teaching environment and venue of the subject virtual teacher based on the UE5 platform to realize the construction of the virtual teaching scene.

[0047] The present invention also proposes a virtual teacher interaction system based on the aforementioned education big model fused with the knowledge graph driven virtual teacher interaction method, the system includes a model data layer, a functional interaction layer and an application presentation layer.

[0048] Among them, the question-answering model construction method is specifically manifested as the model data layer, including the question-answering model module and the corpus construction module. The question-answering model module recognizes the voice signal input by the student through the ASR service, converts it into a text signal, and analyzes and feedbacks it through the constructed question-answering model to seek the optimal solution. Finally, the optimal solution text signal is converted into a voice signal output by TTS processing; the corpus construction module is used to build a data set serving the question-answering model module; the functional interaction application method is manifested as the functional interaction layer, including the communication connection module, the GUI control middle platform, and the result interaction module. The communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher. The GUI control middle platform and the result interaction enrich the user experience of the system in human-computer interaction through multimodal input and personalized teacher attribute editing; the teacher and scene driving method is manifested as the application presentation layer, including the virtual teacher behavior module and the virtual teaching scene module. The virtual teaching scene module is used to construct the teaching scene, and the virtual teacher behavior module is used to establish the virtual teacher model and adjust the facial posture and body movements according to the driving information transmitted by the functional interaction layer.

[0049] The beneficial effects of the present invention are:

[0050] The present invention has a lifelike and customizable virtual teacher, and a highly intelligent teaching platform with subject characteristics. The construction of the virtual teacher uses the 3D model making software Agisoft Metashape to build a personalized virtual teacher's facial image, and obtains a dense point cloud through multi-angle shooting, and finally exports the facial model, which is then solved and edited by Metahuman, a high-fidelity digital human production platform; the teaching platform is based on UE5, and through the animation generation technology Animation Sequence, expression generation technology Control Rig Pose, blueprint compilation technology, and the construction of a 3D teaching environment and scene, it gives users the richest sensory experience.

[0051] The question-answering model of the present invention is committed to integrating the educational model, subject knowledge graph, subject dialogue framework and GraphRAG, and based on this, trains a corpus that best meets user needs. Users can talk freely with virtual teachers on personal interests, professional knowledge, natural sciences, etc. High intelligence and personalization not only improve the efficiency of interaction, but also meet the needs of users.

[0052] The present invention aims to improve the immersion and diversity of the interaction between users and virtual teachers. The front-end and back-end programs are organically integrated through the GUI control center, and on this basis, a teacher personal information section, a teacher voice switching section, a question-and-answer model control section, and a text input section are added. In a non-interactive state, the virtual teacher will also perform random fragmented actions, which further reflects the autonomy and disorder of the virtual character's behavior.

[0053] The invention has certain social and economic value. The platform can be used in the live broadcast of educational technology, which helps to enhance the public's awareness and interest in educational technology and promote the popularization and development of educational technology. In addition, virtual teachers can replace the positions of real lecturers to a certain extent, which can not only reduce labor costs, but also reduce the costs of corporate recruitment and training.

[0054] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0056] Figure 1 The overall structure diagram of the virtual teacher interaction method and system driven by the educational big model fusion knowledge graph of the present invention;

[0057] Figure 2 A flowchart of a virtual teacher interaction method and system driven by integrating a knowledge graph into an educational macro model of the present invention;

[0058] Figure 3 It is a schematic diagram of the virtual teacher image generation process of the virtual teacher behavior module in the application presentation layer of the present invention;

[0059] Figure 4 A schematic diagram of the classification of virtual teacher gestures and movements of a virtual teacher behavior module in the application presentation layer of the present invention;

[0060] Figure 5 This is a lip synchronization flowchart of the virtual teacher interaction method and system driven by the educational big model integrated with the knowledge graph of the present invention. DETAILED DESCRIPTION

[0061] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0062] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0063] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0064] See also Figure 1 to Figure 5 , which is a virtual teacher interaction method and system driven by a large educational model integrated with knowledge graph.

[0065] Example

[0066] This embodiment proposes a virtual teacher interaction method driven by a large education model integrated with a knowledge graph. Figure 1 As shown, it at least includes:

[0067] S1. Establish a data set serving the question-answering model module through the corpus construction module. It provides corpus data support for the question-answering model through large model data fine-tuning, knowledge graph data set construction, and subject dialogue framework data set training;

[0068] S2. Build a question-answering model module, in which the student's output speech is first converted into recognizable text and then classified and managed by the subject knowledge framework. Then, the optimal feedback is obtained through the locally deployed education model and subject knowledge graph, and finally converted into speech output;

[0069] S3, realize virtual teacher control and interaction through the functional interaction layer, which includes communication connection, GUI control middle platform and result interaction. The communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher. The GUI control middle platform and result interaction enrich the user experience of the system in human-computer interaction through multi-modal input and personalized teacher attribute editing.

[0070] S4. Build an application presentation layer, which receives input from the functional interaction layer to drive the virtual teacher to generate human-like facial gestures, body movements, and generate realistic teaching scenes.

[0071] This embodiment also proposes a virtual teacher interaction system driven by a large education model integrated with a knowledge graph, which includes a model data layer, a functional interaction layer, and an application presentation layer, wherein:

[0072] The model data layer includes the question-answering model module and the corpus construction module. The question-answering model module uses the ASR service to identify the voice signal input by the student, converts it into a text signal, and analyzes and feedbacks it through the constructed question-answering model to seek the optimal solution. Finally, TTS processes the optimal solution text signal into a voice signal output; the corpus construction module is used to build a data set that serves the question-answering model module;

[0073] The functional interaction layer includes the communication connection module, the GUI control platform, and the result interaction module; the communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher, and the GUI control platform and result interaction enrich the user experience of the system in terms of human-computer interaction through multi-modal input and personalized teacher attribute editing.

[0074] The application presentation layer includes a virtual teacher behavior module and a virtual teaching scene module. The virtual teaching scene module is used to construct the teaching scene, and the virtual teacher behavior module is used to establish a virtual teacher model and adjust facial postures and body movements according to the driving information transmitted by the functional interaction layer.

[0075] In this embodiment, if Figure 2 As shown in the figure, the question-answering model module uses ASR (automatic speech recognition) and TTS (text-to-speech) technologies, which play a vital role in building a virtual teaching system. ASR acts on the voice signal input by the user and converts the voice into text to facilitate further analysis by the question-answering model; TTS converts the feedback text into natural and fluent speech, making the speech sound more natural and humane.

[0076] In more detail, Alibaba Cloud ASR is currently the most advanced voice processing technology in China. This technology is widely used in fields such as voice transcription and voice search, and can help developers convert voice into text. After logging in to the Alibaba Cloud console, enter "ASR" in the search box and follow the prompts to activate the service. In order to call the API of the ASR service, you need to obtain the API access credentials, mainly AccessKey ID and AccessKey Secret. Check and use the SDK provided by Alibaba Cloud, which encapsulates the API call details. By introducing the necessary modules in the project and configuring the corresponding data, you can debug and test.

[0077] In addition, Azure's TTS service is part of Microsoft's cognitive services, which uses deep neural network technology to convert text into natural and fluent speech. This service not only supports multiple languages ​​and dialects, but also provides a rich selection of timbre, making the converted speech sound more natural and humane. Similarly, log in to your Azure account, search for "cognitive services", and then create a new "speech service" resource. After creating the resource, you will get an endpoint and a subscription key. These two pieces of information are required when calling the TTS service. Consult the official API call guide, use Python to introduce the necessary libraries in the project, deploy related modules, and finally test it.

[0078] The question-answering model module of this embodiment is provided with a question-answering model, which is composed of an education model, a subject knowledge graph, a subject dialogue framework, and Graph RAG. The education model uses ChatGLM-6B, which is based on the General Language Model (GLM) architecture and has 6.2 billion parameters. It has the following characteristics: (1) self-encoding idea; (2) autoregressive idea; (3) span shuffling + two-dimensional position encoding technology. Specifically, in the regression space filling task, given an input text x = [x1,…x n ], multiple text spans {s1,…s m In order to fully capture the interdependencies between spans, the order of spans can be randomly arranged to obtain all possible arrangement sets Z m ,in: So the pre-training goal is clear:

[0079]

[0080] In the formula, p θ represents the probability factor, Represents a token, x corrupt Indicates corrupted text, represents a continuation token;

[0081] In terms of position encoding, ChatGLM uses Rotary Positional Embeddings (RoPE) instead of the original absolute position encoding. RPE uses the idea of ​​complex numbers and its starting point is to achieve relative position encoding through absolute position encoding. Its goal is to add absolute position information to q, k through the following operations:

[0082]

[0083] In the formula, represents a two-dimensional row vector, m and n are absolute positions, q and k are query vector and key vector respectively, and f(·) is a position operation;

[0084] ChatGLM uses a standard self-attention mechanism, calculates the dot product between the query and the key as the weight, and normalizes it through the softmax function:

[0085]

[0086] Where Q, K, and V are query vectors, key vectors, and value vectors, respectively. d represents the dimension of the key vector, and softmax(·) is the normalization function.

[0087] The construction of subject knowledge graph involves multiple fields such as cognitive computing, knowledge representation and reasoning. It combines BERT-BiLSTM-CRF, BERT-wwm-BiGRU-attention and other models that integrate domain vocabulary information, focusing on the research of subject knowledge extraction and knowledge fusion tasks. Its structure can be divided into ontology construction, subject knowledge extraction, knowledge fusion and storage. Combining subject knowledge graph with education big model will achieve full coverage of subject knowledge and general knowledge, and will more comprehensively identify personalized problems.

[0088] The construction and improvement of the subject knowledge graph not only enables the question-answering system to more accurately analyze the semantics of user questions, but also accurately retrieve and match answers related to questions in the huge knowledge graph. The knowledge graph combines models such as BERT-BiLSTM-CRF and BERT-wwm-BiGRU-attention that integrate domain vocabulary information, focusing on the study of subject knowledge extraction and knowledge fusion tasks, which can be divided into ontology construction, subject knowledge extraction, knowledge fusion and storage. (1) Ontology construction. Taking into account the characteristics of multi-source data, the conceptual framework of the subject ontology is defined by deeply analyzing the core concepts and patterns of the automatic control discipline. On this basis, this embodiment uses the ontology construction tool protégé to construct the knowledge ontology of the automatic control discipline. (2) Subject knowledge extraction. First, the processed automatic control subject textbooks and teaching outlines are used to identify entities using the BERT-BiLSTM-CRF model that integrates domain vocabulary information; secondly, the BERT-BiGRU-attention model is combined to extract relationships to obtain the triple form between subject entities. (3) Knowledge fusion and storage. Entity alignment is completed using cosine similarity, and the triples are finally stored in the Neo4j graph database. After completing the automatic control subject knowledge extraction and knowledge fusion, the triple PDF format is selected as the sequence representation method of the knowledge graph, and the Neo4j graph database is used for efficient and intuitive storage and display.

[0089] The subject dialogue framework is based on Rasa, which is an open source machine learning framework that focuses on building conversational artificial intelligence. Rasa allows developers to build their own chatbots to achieve natural and intelligent conversational interactions. It includes two main modules: Rasa NLU and Rasa Core. Rasa NLU processes the natural language input by the user and converts it into structured data. It includes parts such as intent classification, entity recognition, and slot filling; at the same time, the model can be trained based on training data, and parameters can be adjusted and optimized to improve the accuracy and performance of the model. Rasa Core is a dialogue management platform for holding conversations and deciding what to do next. It includes components such as dialogue state trackers and dialogue strategy selectors. In the present invention, Rasa is used for dialogue management, intent recognition, and entity extraction, while the education big model and subject knowledge graph are used as generative models to generate natural language responses. The combined method makes full use of Rasa's dialogue management and natural language generation capabilities of the generative model, making the virtual teacher's responses more flexible and natural.

[0090] Graph RAG (Retrieval-Augmented Generation) provides a powerful method to process and understand complex data sets by combining the structured data and generation capabilities of knowledge graphs, achieving more efficient and intelligent information processing, and significantly improving the performance of question-answering systems. In particular, it shows its unique advantages when dealing with tasks that require complex reasoning and large amounts of information integration, enabling the system to more deeply understand the user's query and provide accurate answers based on the knowledge graph. The main steps include but are not limited to (1) User query: The system first receives the user's query, which may be a simple question or a more complex query. (2) Search for similar entities: The system identifies a set of entities from the knowledge graph that are semantically related to the user's input. These entities serve as entry points into the knowledge graph. (3) Text unit mapping and relationship extraction: The extracted text units are mapped to the corresponding entities, removing the original text information. (4) Utilizing conversation history: The system uses conversation history to better understand the user's intentions and context. (5) Generate responses: The system generates and responds to user queries based on the generated filtered and sorted data.

[0091] For the corpus construction module in the model data layer, it is mainly used to build the ChatGLM-6B fine-tuning dataset, Rasa's corpus training dataset, and the knowledge graph dataset.

[0092] For the ChatGLM-6B model, P-Tuning v1 is a technique for fine-tuning a small number of parameters of a pre-trained language model. Compared with retraining a new model, fine-tuning can greatly save computing resources and also achieve better performance.

[0093] In more detail, the so-called pre-trained language model refers to a model that has been trained on a large-scale language dataset and can understand natural language expressions and learn language knowledge from them. What P-Tuning v1 does is to fine-tune the pre-trained model according to specific tasks so that it can better adapt to specific tasks. Compared with retraining a new model, fine-tuning can greatly save computing resources and also achieve better performance. The fine-tuning method of P-Tuning v1 is to add Prompt to the fine-tuning process, and only train the parameters of the Prompt part, while the parameters of the language model remain fixed. P-Tuning v1 designs a method to automatically generate continuous prompts to improve the fine-tuning effect of the model. The template of P-Tuning v1 can be expressed by the following formula:

[0094] h0,…,hi,e(x),hi+1,…,hm,e(y)

[0095] Where h represents the continuous prompt representation of P-Tuning v1, e represents a pre-trained language model, x represents the original input of the data, and y represents the label of the data.

[0096] The core elements of Rasa's corpus training are Date and Domain files. The NLU training data in Date includes the text entered by the user, the corresponding intent, and the entity. The Domain file defines all the information that the conversational robot should know, which is equivalent to the brain framework, specifying intents, entities, slots, and actions. Rasa finally saves the trained model by extracting features, training conversation models, policy training, model evaluation, and selection. Date stores training data, including NLU training data (training intent recognition models) and Stories conversation flow data (training conversation policy selection models). Domain file: defines the domain of the conversation, including entities, slots, intents, actions, templates, etc. Among them, NLU training data includes the text entered by the user, the corresponding intent, and entities. Stories includes conversation states and conversation behaviors, and learns from real conversation data in the form of stories. Rasa finally saves the trained model by extracting features, training conversation models, policy training, model evaluation, and selection. After preparing the training data, you can train the model by running the Rasa train command. After the model training is completed, Rasa will save the final model and the corresponding configuration file.

[0097] In the construction of the knowledge graph dataset, the present invention collects student questions from relevant websites, online course discussion sections, and after-class exercises in textbooks, and manually constructs a small-scale automatic control subject question dataset (ACQDS) containing a total of 602 data items; at the same time, in order to increase the scale of the dataset, by observing the fact triples in the automatic control subject knowledge graph, and the three types of entities and six types of relationships constructed according to the automatic control domain concept ontology layer, this embodiment divides the question intent into 9 categories. Each intent category corresponds to 9 query requirements in the knowledge graph database. These query requirements are all implemented through the Cypher query language of the graph database Neo4j. In order to increase the scale of the dataset, this embodiment uses a data enhancement method to expand the dataset, mainly in the following ways: (1) Synonym replacement. Randomly select n words and replace them with their synonyms. This increases the diversity of the data by changing the expression of the words. (2) Word order reversal. Randomly select n words and change their positions. (3) Subject replacement. Replace the subject with other entities. (4) Design multiple question templates. Due to the diversity of natural language expressions, multiple question templates are designed for each intent category. According to the above method of this embodiment, the initial dataset is enhanced and expanded to more than 10 times and can be used for model training.

[0098] The functional interaction layer of this embodiment is equipped with a communication connection module, a GUI control middle platform, and a result interaction module, which is the hub of the entire virtual teaching system. It not only receives key information sent by the model data layer, but also connects to the application presentation layer to pass correct feedback to the system front end to drive the virtual teacher.

[0099] For the communication connection module, HTTP and WebSocket are used to facilitate the communication between the front-end, GUI, and back-end. Specifically, in the front-end, this embodiment selects WebSocket, which supports real-time two-way communication, which allows a persistent connection to be established between the front-end and the GUI control center. In the connection between the back-end modules and between the back-end and the GUI control center, this embodiment uses the more flexible and simple HTTP communication protocol.

[0100] More specifically, in the construction of the virtual teacher system, a total of 2 types of communication protocols are used: HTTP and WebSocket; when the front end communicates with the GUI control center, this embodiment selects WebSocket that supports real-time two-way communication, which allows a persistent connection to be established between the front end and the GUI control center so that when either party has data updated, the data can be immediately passed to the other party. Specifically, at the front end, this embodiment uses blueprint compilation technology to create a WebSocket object, define the back-end message receiving and reply modules, and create event monitoring, including back-end connection success, exceptions, and closure. At the back end, this embodiment uses the WebSockets library in Python to call functions, uses WebSockets.connect to establish a connection to the front end, sends messages through await WebSocket.send, and receives responses from the front end through await WebSocket.recv. The front-end and back-end communication addresses: ws: / / 127.0.0.1:10002 establish the details of the connection. When the client initiates a WebSocket connection request, it will use this address to send a handshake request to the back end and start two-way communication after the handshake is successful. In the connection between the back-end modules and between the back-end and the GUI control center, this embodiment uses a more flexible and simple HTTP communication protocol. Only a server and a client need to be defined. The client sends requests to the server and receives responses through the specified communication address, and the server also receives requests and processes responses through the address. Through the mutual cooperation of multiple communication protocols and multiple communication addresses, the information exchange of the entire virtual teacher system is realized.

[0101] For the GUI control middle platform, CSS and HTML are used to build a beautiful and rich user interface for the GUI control middle platform; and the result interaction module includes the middle platform dialog box output and the front-end control output; the middle platform dialog box output is for the case where the input is text signals, and the front-end control output is for the case where the input is voice signals.

[0102] More specifically, a graphical user interface (GUI) is an interface that allows users to interact with a system or application through graphical elements. In the virtual teacher system, a GUI control center is built so that students can directly control and monitor the virtual teacher's attributes and teaching status in real time through the user interface, which not only improves students' learning experience, but also enhances the interactivity between teachers and students. When developing a GUI (graphical user interface), CSS (cascading style sheets) and HTML (hypertext markup language) play a vital role. They each have unique functions and uses, and work together to build a user interface that is both beautiful and rich in functions. HTML defines the basic content and structure of a web page, and CSS is responsible for the style design of a web page. By specifying CSS style rules for HTML elements, the look and feel of the web page can be controlled, thereby creating a GUI that is both beautiful and in line with a specific image. The GUI structure is as follows: the system title name "Online Education Virtual Teacher" is centered at the top of the interface; the virtual teacher information bar is in the upper left corner, which contains the teacher's personal information and the teacher's personalized attribute settings; the system microphone settings and the teacher's connection status display are displayed in the lower left corner; the chat dialog box is on the right, which can support students to manually enter text to communicate and interact with the virtual teacher, thereby realizing multimodal input.

[0103] For the result interaction module, when the signals of two different modes, voice and text, are used as input, the output logic of the results is different; including the middle stage dialog box output and the front-end control output. Middle stage dialog box output: The dialog box output builds an interactive environment through the cooperation of CSS, HTML, and JS. Based on the GUI control, a chat dialog box with text signals as input is added to the middle stage. The interactive objects only include teachers and students, and a one-question-one-answer chat mode is adopted. The students on the left side of the dialog box can enter questions in the chat window. After analysis, feedback, and communication transmission by the question-and-answer model, the teacher on the right side of the dialog box directly responds to the questions through the chat window, thus completing a round of interaction. The front-end control output is mainly aimed at the case where the input is a voice signal. The chat window control is designed based on UE5. The underlying interaction logic of the control is controlled by the control blueprint. The top center title is "Dialog Box", and the light-colored rectangular area directly below the dialog box is the interaction "venue". The control will only be displayed in the front-end virtual teaching environment after the action of the virtual teacher's reply is triggered. Set the result output logic in the control blueprint. The structure includes two objects, teacher and student. The voice signals read by "Teacher" and "Student" are displayed as the student's question text, and the text signals fed back by the question-and-answer model are displayed as the teacher's answer text. When the number of inquiries is too many and the text exceeds the light-colored rectangular boundary, it will automatically slide to the area of ​​the latest text reply so that users can obtain information.

[0104] The system front end of this embodiment is the application presentation layer, which includes a virtual teacher behavior module and a virtual teaching scene module; from the structure, design, and construction of the teaching scene to the personalized modeling, facial posture, and body movements of the virtual teacher, through the integration of the two modules, a highly realistic and immersive virtual teaching scene environment is presented to the user.

[0105] like Figure 3 The flowchart of constructing the virtual teacher image is shown in the figure; first, take multiple sets of multi-angle facial image photos of people in a relatively well-lit indoor environment through a camera of any brand and model, the number needs to be greater than 30, and convert them into a fixed format. Import the photos into the 3D model making software Metashape, convert the sparse facial point cloud into a dense facial point cloud and generate a character mesh model, import the facial mesh into the UE5 platform, and successfully wrap the metahuman model on the mesh surface through the tracking of the facial feature points and the ontology solution of the character. After the secondary editing and key point reshaping of the MetaHuman platform, a personalized virtual human image is generated, and finally the generated virtual teacher image is imported into the virtual teaching platform through Bridge.

[0106] like Figure 4 The diagram of the virtual teacher's gesture action types in the virtual teacher behavior module in the application presentation layer is shown, where the virtual teacher's gesture actions include facial gestures and body movements, and the virtual teacher's facial gestures include: blink control, expression control, and lip shape control; among them, facial gesture control mainly relies on "blueprint"; it is a visual script system unique to UE5, which represents the functions, variables and logical processes in the program through nodes and connecting lines, allowing developers to create and edit level logic in a graphical way without writing traditional codes, making the programming process more intuitive, easy to understand, efficient and flexible. In the blink control, the facial blueprint is used. In the blueprint, the modification curve is introduced, the CTRLExpressions Eye Blink module is added, and the Alpha value is used to control the eye closure, where 1 represents completely closed eyes and 0 represents completely open eyes. At the same time, the timeline variable is introduced to make its value change periodically over time and pass it to Alpha, so as to control the periodic and natural changes of the blink expression. In expression control, first enter the Facial Pose library and select 3-5 basic expression templates. These expressions are implemented as Control Rig poses. This embodiment can directly map the expressions to the facial expressions of the virtual teacher through the built-in facial Rig, thereby driving the teacher's expression changes. In the facial blueprint, the newly exported expression sequence can be applied to any state of the virtual teacher, such as teaching state, quiet state, etc. At the same time, they can also be mixed with each other, and after generating appropriate expressions, they can be integrated into the basic expression module.

[0107] like Figure 5 The figure below shows the basic flow chart of lip shape control: the answer from the question-answer model is passed down in JSON format. The format varies depending on the speech synthesis service, but generally uses fields such as text, name, and language. The request containing the answer content is passed to the corresponding API. At this time, the front end receives an audio file response containing the answer content. After the pinyin parsing of the lip shape blueprint, the phonemes are classified into more than a dozen types such as PP, FF, DD, etc. Each phoneme corresponds to a unique lip shape, so that the answer content can be divided into different lip shape animations to drive the virtual teacher.

[0108] The body movements of the virtual teacher include: quiet state, teaching state, and listening state. The animation generation technology integrated in the UE5 platform, Animation Sequence, enables the virtual teacher to produce rich and vivid body movements. Animation Sequence is an animation asset that can be played on the skeletal mesh. It contains many keyframes. Each keyframe specifies the position, rotation, and scaling information of the skeleton at a specific time point. Playing the Animation Sequence can make the skeleton move, thereby driving the skeletal mesh to change together and produce animation. In the action blueprint, the three states of quiet, teaching, and listening are reasonably connected to realize the internal logic. When not interacting with the virtual teacher, the teacher is in a quiet state; when students talk to the virtual teacher, the teacher is in a listening state; when the teacher explains the teaching content, it is in a teaching state. Each state is the optimal solution for the teacher in the current teaching environment, which conforms to the teaching scenario of real teachers in reality.

[0109] Construction of virtual teaching scene. The environment and venue for the subject virtual teacher to teach are built based on the UE5 platform. According to the teaching content and application scenarios, stickers of different colors are selected from different virtual material libraries as the external decoration of objects in the studio; display screens, tables, blackboards, stationery, plants, etc. are selected as decorations of the studio; famous sayings with educational colors and world-famous paintings with literary atmosphere are posted on the walls of the studio. The virtual teacher will be located in the center of the studio, and she will bring real teaching experience to students within the designated scope.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A virtual teacher interaction method driven by a large educational model and knowledge graph, characterized by: The method comprises the following steps: S1. Establish a data set serving the question-answering model module through the corpus construction module. It provides corpus data support for the question-answering model through large model data fine-tuning, knowledge graph data set construction, and subject dialogue framework data set training; S2. Build a question-answering model module, in which the student's output speech is first converted into recognizable text and then classified and managed by the subject knowledge framework. Then, the optimal feedback is obtained through the locally deployed education model and subject knowledge graph, and finally converted into speech output; S3, realize virtual teacher control and interaction through the functional interaction layer, which includes communication connection, GUI control middle platform and result interaction. The communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher. The GUI control middle platform and result interaction support multi-modal input and personalized teacher attribute editing; S4. Build an application presentation layer, which receives input from the functional interaction layer to drive the virtual teacher to generate human-like facial gestures, body movements, and generate realistic teaching scenes.

2. According to claim 1, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: In step S1, the corpus construction module is used to construct a fine-tuning dataset for the education model, a corpus training dataset for the subject dialogue framework, and a knowledge graph dataset; wherein, The multi-task fine-tuning learning method is used to fine-tune the educational model. The process is as follows: generate continuous prompt words Promp, and only train the parameters of the prompt word Prompt part. The parameters of the educational model are fixed and unchanged, which can be expressed as: h0,…,hi,e(x),hi+1,…,hm,e(y) Where h represents the generated continuous prompt representation, e represents a pre-trained educational model, x represents the original input of the data, and y represents the label of the data; The corpus training data set for the subject dialogue framework includes Date and Domain files. The NLU training data in the Date file includes the text entered by the user, the corresponding intent and entity; the Domain file specifies the intent, entity, slot and action; For the knowledge graph dataset, students' questions are collected from various angles to construct a question dataset; multiple types of entities and relationships are established based on the fact triples in the subject knowledge graph and the concept ontology layer, and multiple question intent categories are established based on the question intent. Each question intent category corresponds to the query requirements in the knowledge graph database; at the same time, data enhancement methods are used to expand the dataset, including synonym replacement, word order reversal, subject replacement, and establishment of multiple question templates.

3. According to claim 2, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: In step S2, a question-answering model is set in the question-answering model module. The question-answering model is composed of an education model, a subject knowledge graph, a subject dialogue framework, and a Graph RAG module. The education model is implemented based on its autoencoding idea, autoregressive idea, Span shuffling, and two-dimensional position encoding technology. In the regression space filling task, given an input text x = [x1,…x n ], sampling to obtain multiple text spans{s1,…s m }; Randomly arrange the order of spans to obtain all possible arrangement sets Z m ,in: And train according to the following goals: In the formula, p θ represents the probability factor, Represents a token, x corrupt Indicates corrupted text, represents a continuation token; In terms of position coding, the educational model adopts the rotation position embedding coding method. The rotation position coding realizes relative position coding through absolute position coding at the starting point, and adds absolute position information at its target point through the following operations: In the formula, represents a two-dimensional row vector, m and n are absolute positions, q and k are query vector and key vector respectively, and f(·) is a position operation; The educational model uses a standard self-attention mechanism to calculate the dot product between the query and the key as the weight, and normalizes it through the softmax function: Where Q, K, and V are query vectors, key vectors, and value vectors, respectively. d represents the dimension of the key vector, and softmax(·) is the normalization function.

4. According to claim 3, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: The subject knowledge graph of the constructed question-answering model includes the subject knowledge ontology constructed using the ontology construction tool, and subject knowledge extraction, knowledge fusion and storage are performed on the subject knowledge ontology. For subject knowledge extraction, various sequence annotation models that integrate domain vocabulary information are used to identify entities in subject textbooks and teaching outlines, and then deep learning models are combined to extract relationships to obtain triples between subjects and entities; for knowledge fusion and storage, cosine similarity is used to complete entity alignment, and finally the triples are stored in the graph database; The subject dialogue framework includes the NLU module and the Core module. The NLU module is used to process the natural language input by the user and convert it into structured data. The NLU module includes intent classification, entity recognition and slot filling parts. The Core module includes a dialogue state tracker and a dialogue strategy selector, which are used to hold a dialogue and determine the next operation. The subject dialogue framework is used for dialogue management, intent recognition and entity extraction. The education model and subject knowledge graph are used as generative models. The Graph RAG module combines the structured data and generation capabilities of the knowledge graph to process complex data set information. It processes data in the following steps: (1) User query: First, receive the user's query; (2) Search for similar entities: Identify a set of entities related to the user input semantics from the knowledge graph and use the searched entities as the entry point into the knowledge graph; (3) Text unit mapping and relationship extraction: The extracted text units are mapped to the corresponding entities, and the original text information is removed; (4) Understanding intent: Understanding the user’s intent and context based on the conversation history; (5) Generate responses: Generate and respond to user queries based on the generated filtered and sorted data.

5. According to claim 4, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: The question-answering model module uses automatic speech recognition ASR and text-to-speech TTS technology to transmit information between the system backend and frontend. ASR acts on the voice signal input by the user to convert the voice into text; TTS converts the feedback text into natural speech.

6. According to claim 1, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: In step S3, the communication connection module in the functional interaction layer adopts HTTP and WebSocket communication protocols, and different communication protocols are set according to the communication between different modules. When the front end communicates with the GUI control center, the WebSocket communication protocol that supports real-time two-way communication is selected, a WebSocket object is created at the front end, and a message receiving and replying module is defined at the back end to establish communication; in the communication between back-end modules and between the back end and the GUI control center, the HTTP communication protocol is adopted to define the server and the client, and the client sends requests to the server and receives responses through the specified communication address, and the server also receives requests and processes responses through the address; The GUI control center in the functional interaction layer uses CSS and HTML to jointly construct the user interface. Its interaction includes editing of teacher personal information, including: name, age, voice, etc.; controlling the opening and closing of the front-end teacher service and displaying the teacher connection status; supporting students to manually input text to communicate and interact with the virtual teacher, thereby realizing multimodal input; the result interaction module includes the center dialog box output and the front-end control output; the center dialog box output is for the case where the input is text signal, and the front-end control output is for the case where the input is voice signal.

7. According to claim 1, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: In step S4, the virtual teacher behavior module of the application presentation layer is first used to construct a virtual teacher image, which includes the following steps: Firstly, multiple groups of facial images of people from multiple angles are obtained; Import the photo into the 3D model making software Metashape, convert the sparse facial point cloud into a dense facial point cloud and generate a character mesh model, import the facial mesh into the UE5 platform, and wrap the MetaHuman model on the mesh surface through facial feature point tracking and body solving. After secondary editing and reshaping of key points on the MetaHuman platform, a personalized virtual human image is generated; Finally, the generated virtual teacher image is imported into the virtual teaching platform through Bridge.

8. According to claim 7, a virtual teacher interaction method driven by a large educational model integrated with a knowledge graph is characterized by: The gestures of the virtual teacher include facial gestures and body movements. The facial gestures of the virtual teacher include: blink control, expression control, and lip shape control; the body movements of the virtual teacher include: quiet state, teaching state, and listening state; Among them, facial gesture control uses a visual script system, which uses nodes and connecting lines to represent functions, variables and logical processes in the program to achieve facial gesture control; for blink control, a modification curve is introduced in the script system, and a CTRL Expressions Eye Blink module is added. The Alpha value is used to control eye closure, where 1 represents completely closed eyes and 0 represents completely open eyes. At the same time, a time axis variable is introduced to make its value change periodically over time and pass it to Alpha, so as to control the periodic and natural changes of blinking expressions; In expression control, select several basic expressions from the expression library of the script system, map the basic expressions to the facial expressions of the virtual teacher, and drive the teacher's expression changes; the newly derived expression sequence can be applied to any state of the virtual teacher, and can also be mixed with each other to generate complex expressions, and then integrated into the basic expression module; For lip shape control, the answer feedback from the question-and-answer model is passed downward in the corresponding format; the request containing the answer content is passed to the corresponding API, at which point the front end receives an audio file response containing the answer content, which is then processed through the pinyin analysis of the lip shape blueprint to classify the phonemes, each phoneme corresponding to a unique lip shape, and then the answer content is divided into different lip shape animations to drive the virtual teacher.

9. The method for virtual teacher interaction driven by a large educational model integrated with a knowledge graph according to claim 8 is characterized by: Animation generation technology is used to control the body movements of virtual teachers to ensure natural and smooth movements; through action blueprint technology, the three states of virtual teachers, namely quiet, teaching, and listening, are reasonably converted to achieve internal logic control; when there is no interaction, the virtual teacher remains quiet; when talking with students, it switches to a listening state; when explaining teaching content, it is in a teaching state; in addition, the system applies the virtual teaching scene module of the presentation layer to build the teaching environment and venue of subject virtual teachers based on the UE5 platform to realize the construction of virtual teaching scenes.

10. A virtual teacher interaction system based on the virtual teacher interaction method driven by the educational big model integrated with the knowledge graph as described in any one of claims 1 to 9, characterized in that: The system includes a model data layer, a functional interaction layer, and an application presentation layer. Among them, the question-answering model construction method is specifically manifested as a model data layer, including a question-answering model module and a corpus construction module; the question-answering model module recognizes the voice signal input by the student through the ASR service, converts it into a text signal, and analyzes and feeds back through the constructed question-answering model to seek the optimal solution, and finally converts the optimal solution text signal into a voice signal output by TTS processing; the corpus construction module is used to construct a data set serving the question-answering model module; the functional interaction application method is manifested as a functional interaction layer, including a communication connection module, a GUI control middle platform, and a result interaction module; the communication connection is responsible for feeding back the output information of the received question-answering model to the front end to drive the virtual teacher, and the GUI control middle platform and result interaction support multimodal input and personalized teacher attribute editing; the teacher and scene driven method is manifested as an application presentation layer, including a virtual teacher behavior module and a virtual teaching scene module. The virtual teaching scene module is used to construct a teaching scene, and the virtual teacher behavior module is used to establish a virtual teacher model, and adjust facial posture and body movements according to the driving information transmitted by the functional interaction layer.

Images