An age-based virtual child behavior modeling and real-time interaction system for music teaching of normal school students

Abstract

The application relates to the technical field of artificial intelligence, and discloses an age-based virtual child behavior modeling and real-time interaction system for music teaching of normal school students, which comprises an age-based virtual child music behavior modeling engine, a normal school student music teaching multi-modal data acquisition module, a music teaching behavior-child response mapping engine, an immersive virtual music teaching interaction environment, a training parameter configuration module, a dynamic adjustment module, a teaching ability evaluation module, a teaching emotion-child emotion resonance feedback engine and a normal school student teaching ability longitudinal tracking and personalized training path recommendation module. The age-based virtual behavior model conforming to the music cognitive development law of children is constructed for a music teaching scene, real-time mapping and presentation of a music teaching specific mode to a virtual child response are realized, the authenticity and pertinence of music teaching training of normal school students are improved, and the simulation reality is improved and the classroom emotion management ability of normal school students is cultivated.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees. Background Technology

[0002] With the deep integration of artificial intelligence technology and education, the training model for teacher trainees is undergoing profound changes. Virtual simulation technology provides teacher trainees with a low-cost, repeatable, safe, and controllable teaching simulation environment, and has become an important research direction in the field of teacher education. Existing virtual teaching systems for teacher trainees mainly include the following categories:

[0003] The first category is general classroom management simulation systems, such as microteaching virtual simulation platforms built using Unity3D and SteamVR. This system presents real classroom problems and behaviors through virtual students, which are identified and handled by student teachers. The system records the operations and generates teaching reports. However, this system is a general classroom management simulation and does not involve music subjects or children's music learning behaviors.

[0004] The second category is general teaching skills training systems, such as virtual simulation experiments that train teaching language, blackboard writing, and teaching demeanor. These systems use an intelligent and objective evaluation system to collect teaching behavior information and quantify it in real time. However, these systems focus on general basic skills and lack music teaching scenarios and children's interaction models.

[0005] The third category is a comprehensive skills training system, which uses U3D technology and voice recognition technology, supports wearable devices, motion capture systems and physiological instruments to achieve classroom management skills training. Although it has rich hardware interfaces, it does not have a dedicated interactive protocol designed for music teaching.

[0006] The fourth category is the digital student micro-teaching classroom system, where novice teachers conduct segmented teaching with different types of digital students. AI monitors the teacher's speaking speed, question distribution, and student expressions in real time, and automatically generates an improvement list and comparison video after class. The system has proposed the concept of "digital student", but it does not distinguish the music cognition characteristics of different age groups.

[0007] The fifth category is music virtual simulation technology. Existing music virtual simulation is concentrated in the scenario of "students receiving music education", such as the virtual simulation experiment of Nanyin "Shang Si Guan" ensemble, the virtual simulation experiment of Confucius Temple Yayue, and the experiment of Zeng Hou Yi Bianzhong performance and composition. These are all intangible cultural heritage inheritance or instrument performance skills training. AI music classrooms and virtual music tutors mainly serve the music learning guidance of students, and there is a lack of a system for cultivating the music teaching ability of teacher trainees.

[0008] Current technologies suffer from the following common deficiencies: existing virtual teaching systems lack a hierarchical parameterized behavioral model based on the developmental patterns of children's musical cognition, and also lack a real-time causal mapping mechanism from music teaching-specific modalities to virtual children's learning responses. This results in a lack of targeted and real-time feedback in music teaching training for teacher trainees. Furthermore, existing systems lack a closed-loop mechanism to dynamically adjust virtual children's behavioral parameters based on teacher trainees' real-time teaching performance, and also lack a quantitative evaluation system for music teaching abilities. This limits both training effectiveness and evaluation capabilities. In addition, existing systems only collect general teaching behaviors and do not establish a real-time response mapping for the music discipline-specific "singing-conducting-rhythmic" modalities. Moreover, existing systems only evaluate teaching skills and do not establish an independent mapping channel from teachers' emotional states to students' emotional resonance. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides an age-group virtual child behavior modeling and real-time interaction system for music teaching for teacher trainees. This system solves the problems mentioned in the background, such as the lack of a real-time causal mapping mechanism between music teaching-specific modalities and virtual child learning responses in existing virtual teaching systems, and the lack of an emotional resonance channel independent of teaching behavior, which leads to a lack of realism and emotional interaction in music teaching and training.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees, comprising:

[0011] The age-group virtual children's music behavior modeling engine is configured to build virtual children's behavior models based on the music cognitive characteristic parameters of children corresponding to the target age group.

[0012] The multimodal data acquisition module for music teaching of teacher trainees is configured to collect singing demonstration data, conducting gesture data, body rhythm data and teaching language data of teacher trainees, and integrate them to generate multimodal teaching behavior feature vectors.

[0013] A music teaching behavior-child response mapping engine is configured to convert the multimodal teaching behavior feature vectors into learning response data of virtual children and generate a comprehensive response trend Rebehavior.

[0014] The teaching emotion-child emotional resonance feedback engine is configured to perceive the emotional state and empathy engagement of teacher trainees in real time, calculate the teaching empathy index (TEI), and calculate the emotion modulation coefficient based on the TEI. The comprehensive response trend Rbehavior is weighted a second time by the emotional resonance response increment ΔRemo.

[0015] An immersive virtual music teaching interactive environment is configured to drive virtual child characters to present corresponding behavioral animations and sing-along audio based on the second-weighted response data;

[0016] The module for longitudinal tracking of teaching abilities and personalized training path recommendation for teacher trainees is configured to persistently store training session data. Based on a deep knowledge tracking model, it performs time-series modeling of teacher trainees' mastery of music teaching knowledge points and recommends configuration parameters for the next training session based on the zone of proximal development theory.

[0017] As a preferred technical solution of the present invention, the age-segmented virtual children's music behavior modeling engine includes a preschool children's music perception behavior sub-model and a primary school students' music cognitive behavior sub-model.

[0018] The preschool children's music perception behavior sub-model is configured to encode the vocal range, rhythmic cognitive characteristics, attention span, body rhythm response patterns, and group imitation preferences of children aged 3-6.

[0019] The sub-model of primary school students' music cognitive behavior is configured to encode the pitch discrimination ability, symbol recognition ability, choral singing ability, music preference differentiation characteristics and cognitive strategy characteristics of children aged 6-12.

[0020] The virtual child behavior model includes a two-layer structure of global class parameters and individual offset parameters. The global class parameters determine the overall difficulty level of the training scenario, and the individual offset parameters are superimposed with random offsets on the global class parameters to reflect the behavioral differences between different virtual children in the same virtual class.

[0021] The global class parameters include class cooperation benchmark, class basic music level benchmark, and class discipline level benchmark.

[0022] The individual offset parameter is used to superimpose a random offset following a normal or Beta distribution onto each virtual child based on the global class parameter, so as to reflect the behavioral differences between different virtual children in the same virtual class.

[0023] As a preferred technical solution of the present invention, the multimodal data acquisition module for music teaching of teacher trainees is fused according to a unified time benchmark to generate multimodal teaching behavior feature vectors;

[0024] The multimodal data acquisition module for music teaching for teacher trainees includes a singing demonstration acquisition unit, a conducting gesture acquisition unit, a body rhythm acquisition unit, a teaching language acquisition unit, and a multimodal data fusion unit;

[0025] The singing demonstration acquisition unit collects audio of student teachers singing demonstrations through a microphone array;

[0026] The command gesture acquisition unit captures the trajectory of the teacher trainees' command gestures through a gesture recognition sensor;

[0027] The body rhythm acquisition unit acquires the key point coordinate sequence of the teacher trainees through a depth camera or visual pose estimation algorithm;

[0028] The teaching language acquisition unit transcribes teaching language text using speech recognition technology.

[0029] As a preferred technical solution of the present invention, the music teaching behavior-child response mapping engine includes singing teaching mapping rules, conducting teaching mapping rules, rhythm teaching mapping rules, and rule conflict fusion strategy. When multiple types of teaching behavior mapping rules are triggered simultaneously, a weighted fusion is performed to generate a comprehensive response trend Rbehavior.

[0030] The singing instruction mapping rule is used to generate virtual children's pitch deviation, rhythmic synchronization status, and changes in participation enthusiasm for singing along, based on the pitch deviation, rhythmic stability, and acoustic expressiveness characteristics of the teacher trainees' demonstration singing.

[0031] The conducting teaching mapping rules are used to generate the virtual children's choir's start time dispersion, speed maintenance ability, and vocal part balance state based on the clarity of the beat points, the standardization of the predicted movements, and the standardization of the beat diagram of the teacher trainees' conducting gestures.

[0032] The rhythmic teaching mapping rules are used to generate the synchronization rate and imitation accuracy of virtual children's body rhythm based on the synchronicity, amplitude, and complexity of the teacher's demonstration movements and the music beat.

[0033] The rule conflict fusion strategy is used to assign weights to singing teaching mapping rules, conducting teaching mapping rules, and rhythm teaching mapping rules based on the current teaching behavior type identification results, and then to perform a weighted summation after normalization to synthesize the response trend Rbehavior.

[0034] As a preferred technical solution of the present invention, the music teaching behavior-child response mapping engine further includes a comprehensive response generator, which further includes a virtual child singing voice synthesis submodule;

[0035] The integrated response generator combines the integrated response trend Rebehavior with individual offset parameters to generate a unique response sequence for each virtual child.

[0036] The virtual children's singing synthesis submodule renders virtual children's singing audio in real time based on a pre-recorded sample library of children's solo or choral singing or by using a parametric singing synthesis model.

[0037] As a preferred technical solution of the present invention, the immersive virtual music teaching interactive environment includes a virtual music classroom scene, a virtual child character rendering unit, and a music teaching-specific interactive protocol;

[0038] The training parameter configuration module is used to configure the target age group, global class parameters, virtual class size, and distribution of children's behavioral parameters.

[0039] The dynamic adjustment module is used to dynamically fine-tune the global class parameters based on the real-time performance of teacher trainees, the stability of their teaching behavior, and the improvement rate of virtual children's responses, using an adaptive difficulty feedback adjustment algorithm.

[0040] The teaching ability assessment module is used to record and statistically analyze the pitch stability, conducting clarity, rhythm synchronization rate, and language-child adaptability of teacher trainees, and generate a comprehensive teaching ability score and improvement suggestion report.

[0041] As a preferred technical solution of the present invention, the teaching emotion-child emotion resonance feedback engine includes a teacher trainee teaching emotion multimodal recognition unit, a teaching empathy engagement assessment unit, an emotion-child emotion resonance mapping rule, an emotion modulation coefficient calculator, and an emotion state visualization and child emotion state generation unit;

[0042] The multimodal recognition unit for teacher trainees' teaching emotions is configured to recognize the four-dimensional emotional characteristics of teacher trainees in the teaching process in parallel.

[0043] The teaching empathy engagement assessment unit is configured to weight and fuse four-dimensional emotional features according to predetermined weights to obtain a teaching empathy index (TEI) with a value range of [0,1].

[0044] As a preferred technical solution of the present invention, the emotion-child emotional resonance mapping rule is configured to trigger at least one type of resonance rule among enthusiastic resonance, anxiety contagion, frustration inhibition, and calming soothing based on the teaching empathy index TEI and four-dimensional emotional features, and outputs the emotional resonance response increment ΔRemo;

[0045] The emotion modulation coefficient calculator is configured to calculate the emotion modulation coefficient based on the Teaching Empathy Index (TEI) and the Emotion-Behavior Matching Degree (Ematch). And the comprehensive response trend Rbehavior is weighted twice;

[0046] The emotional state visualization and child emotional state generation unit is configured to generate a state vector of being emotionally infected by the teacher for each virtual child based on the modulated comprehensive response Rfinal and the emotional resonance response increment ΔRemo, and drive the immersive virtual music teaching interactive environment to perform real-time rendering.

[0047] As a preferred technical solution of the present invention, the longitudinal tracking and personalized training path recommendation module for teacher trainees’ teaching ability includes a persistent storage unit for training session data, a music teaching knowledge point graph construction unit, a deep knowledge tracker, a teaching ability growth curve generation unit, and a personalized training path recommendation engine.

[0048] The training session data persistent storage unit is configured to persistently store the data of each training session using the teacher trainee ID and the training session ID as primary keys.

[0049] The music teaching knowledge point map construction unit is configured to construct a music teaching knowledge point map MPKG containing four categories of music teaching ability points, conducting ability points, rhythm teaching ability points, and teaching language ability points, totaling 20-40 fine-grained knowledge points, and to clarify the prerequisite-successor relationships between each knowledge point.

[0050] The deep knowledge tracker is configured to use a deep knowledge tracking model (DKT) based on LSTM or Transformer to perform time-series modeling of the knowledge point interaction sequences of teacher trainees in each training session, output the mastery prediction of each knowledge point at the next moment, and perform online fine-tuning after each new training session.

[0051] As a preferred technical solution of the present invention, the teaching ability growth curve generation unit is configured to generate a single knowledge point growth curve and four major categories of ability radar charts based on the time series data of knowledge point mastery, and to perform plateau and regression anomaly detection on the growth curve.

[0052] The personalized training path recommendation engine is configured to select target knowledge points based on the Zone of Proximal Development (ZPD) theory, and combine knowledge point graph association expansion, age group and scenario matching, learning rate adaptive difficulty level and class parameter recommendation to output the next training recommendation package.

[0053] The next training recommendation package shall include at least: a list of target knowledge points, a list of related knowledge points, a recommended age group, a recommended scenario, a recommended difficulty level, initial global class parameters, a range of recommended teaching materials, an expected training duration, key skills to be focused on in this training, and an expected improvement in mastery.

[0054] The recommended package for the next training session is presented to the student teachers as a pre-filled configuration of the training parameter configuration module when the next training session starts, and serves as the benchmark parameter for the dynamic adjustment module to fine-tune in real time during this training session.

[0055] Compared with existing technologies, this invention provides an age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees, which has the following beneficial effects:

[0056] 1. Construct age-appropriate virtual behavior models for music teaching scenarios that conform to the cognitive development patterns of children in music, and realize real-time mapping and presentation of music teaching-specific modalities to virtual children's responses, thereby enhancing the authenticity and relevance of music teaching training for teacher trainees. Based on the main mapping channel of teacher trainees' teaching behavior → virtual children's learning response, further establish an independent mapping channel of teacher's emotions → children's emotional resonance, so that virtual children can generate emotional infection and emotional resonance response to the emotional state of teacher trainees, thereby enhancing the simulation realism and cultivating teacher trainees' classroom emotional management ability;

[0057] Furthermore, based on the snapshot mode of single training → single evaluation, a longitudinal tracking and personalized training path recommendation mechanism across training sessions is established. This enables the system to automatically recommend target knowledge points, age groups, class parameter configurations, and difficulty levels for the next training session based on the historical training data of teacher trainees. This achieves a long-term closed loop of training-evaluation-tracking-recommendation-retraining. By organically integrating the above-mentioned behavioral channels, emotional channels, and longitudinal tracking channels under a unified system architecture, redundancy between modules, temporal misalignment, and data inconsistency are avoided.

[0058] 2. For the first time, the musical cognitive characteristics of preschool children and primary school students are encoded into a computable hierarchical parameterized virtual behavior model. The model adopts a two-layer structure of global class parameters and individual offset parameters, which makes the virtual class have both an overall difficulty tone and retain the individual differences of children. This significantly improves the realism and configurability of the simulation. It also builds a multimodal data acquisition system covering the unique modalities of music teaching, such as singing demonstration, conducting gestures, and body rhythm, which breaks through the technical limitations of existing systems that only collect general teaching behaviors.

[0059] Establish a real-time causal mapping engine for teacher trainees' music teaching behavior and virtual children's music learning response to achieve immediate feedback on teaching behavior. Introduce rule conflict fusion strategies and quantifiable acoustic performance characteristics to change the existing one-way evaluation mode of post-class report generation.

[0060] 3. For the first time, an independent mapping channel was established between the emotional state of teacher trainees and the emotional resonance response of virtual children, through the Teaching Empathy Index (TEI) and Emotional Modulation Coefficient. Key technologies such as four types of emotional resonance rules enable virtual children to respond to the emotional state of teacher trainees with emotional infection and emotional resonance, significantly improving the simulation realism. At the same time, it enables teacher trainees to cultivate two key teaching abilities in classroom emotional management and emotional influence in virtual training. The Teaching Empathy Index (TEI) is constructed by late-stage fusion of four-dimensional emotional features of face, voice, body and eye contact, which is significantly better than single-modal emotion recognition schemes. It has stronger robustness in real training scenarios where teacher trainees wear VR headsets, classroom lighting is complex, and teacher trainees have large range of motion.

[0061] For the first time, Deep Knowledge Tracking (DKT) and Zone of Proximal Development (ZPD) theory have been introduced into the field of virtual simulation training for teacher trainees. This enables longitudinal tracking across training sessions and personalized training path recommendations, allowing the system to break through the snapshot mode of single training and single reporting. It has a long-term closed loop of training, evaluation, tracking, recommendation, and retraining. Furthermore, it achieves structured recommendations down to specific knowledge points through MPKG knowledge graphs, avoiding teacher trainees' frustration due to overly difficult training and ineffective training due to overly easy training.

[0062] The system employs a dual-layer closed-loop control structure, consisting of a long-term planning module for longitudinal tracking of teacher trainees' teaching abilities and a short-term fine-tuning module for personalized training path recommendation, enabling the system to possess both long-term growth planning capabilities and retain real-time adaptive capabilities within a single training session. This aligns with the hierarchical control principle in automatic control theory.

[0063] 4. By designing a VR interactive protocol specifically for music teaching and realizing real-time singing audio rendering through a virtual children's singing synthesis submodule, teacher trainees can engage in immersive music teaching interaction with virtual children through natural gestures and voice, enhancing the realism and immediacy of training.

[0064] Through the training parameter configuration module, dynamic adjustment module, and teaching ability assessment module, the system achieves precise matching and adaptive adjustment of training difficulty, as well as quantitative assessment of the music teaching ability of teacher trainees, forming a complete closed loop of training-feedback-assessment-improvement.

[0065] By constraining real-time performance metrics such as end-to-end latency, animation frame rate, and audio buffer length, the real-time performance and smoothness of immersive interaction are ensured.

[0066] By persistently storing training sessions, a dataset of music teaching abilities for teacher trainees is created, providing a data foundation for subsequent teaching and research analysis, AI model iteration, and education policy formulation. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention.

[0068] Figure 2 This is a structural diagram of the age-group virtual children's music behavior modeling engine of this invention.

[0069] Figure 3 This is a schematic diagram of the multimodal data acquisition module for music teaching for teacher trainees in this invention.

[0070] Figure 4 This is a flowchart of the music teaching behavior-child response mapping engine of the present invention.

[0071] Figure 5 This is a schematic diagram of the immersive virtual music teaching interactive environment interface of the present invention.

[0072] Figure 6 This is a schematic diagram of the interactive interface of the training parameter configuration and difficulty adjustment module of the present invention.

[0073] Figure 7 This is a schematic diagram of the multimodal data time alignment and feature vector construction of the present invention.

[0074] Figure 8 This is a flowchart of the virtual children's singing audio synthesis and animation rendering of the present invention.

[0075] Figure 9 This is the flowchart of the dynamic difficulty closed-loop control of this invention.

[0076] The diagram shows: 1. Age-group virtual children's music behavior modeling engine; 2. Multimodal data collection module for music teaching by teacher trainees; 3. Music teaching behavior-child response mapping engine; 4. Immersive virtual music teaching interactive environment; 5. Training parameter configuration module; 6. Dynamic adjustment module; 7. Teaching ability assessment module; 8. Teaching emotion-child emotional resonance feedback engine; 9. Vertical tracking of teacher trainees' teaching ability and personalized training path recommendation module. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0078] In the description of this invention, it should be noted that when an element is referred to as being "fixed to" or "set on" another element, it can be directly on or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to or indirectly connected to the other element.

[0079] In the description of this invention, it should be noted that the terms "center," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. "Several" means one or more, unless otherwise explicitly specified.

[0080] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0081] Example: Please refer to Figures 1-9 This invention provides the following technical solution: an age-segmented virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees, comprising an age-segmented virtual children's music behavior modeling engine 1, a multimodal data acquisition module for music teaching for teacher trainees 2, a music teaching behavior-child response mapping engine 3, an immersive virtual music teaching interactive environment 4, a training parameter configuration module 5, a dynamic adjustment module 6, a teaching ability assessment module 7, a teaching emotion-child emotional resonance feedback engine 8, and a longitudinal tracking and personalized training path recommendation module for teacher trainees' teaching ability 9. This system achieves real-time causal mapping from multimodal music teaching behavior to virtual children's responses, and superimposes an independent emotional resonance channel on this behavior channel to enhance the realism and immersion of virtual music teaching. Furthermore, the music teaching behavior-child response mapping engine generates a comprehensive response trend Rbehavior, and the teaching emotion-child emotional resonance feedback engine is introduced to perform a secondary weighting of this trend, constructing a dual closed-loop interaction mechanism with parallel behavior and emotional channels, thus solving the problem of missing emotional interaction in existing systems.

[0082] The age-group virtual children's music behavior modeling engine 1 is configured to build a virtual children's behavior model based on the music cognitive feature parameters of children corresponding to the target age group. The virtual children's behavior model includes a two-layer structure of global class parameters and individual offset parameters. The global class parameters determine the overall difficulty tone of the training scene, and the individual offset parameters are based on the global class parameters with random offsets to reflect the behavioral differences between different virtual children in the same virtual class.

[0083] The target age range includes the preschool stage and the primary school stage, with the primary school stage further subdivided into lower primary school grades and upper primary school grades;

[0084] The multimodal data acquisition module 2 for music teaching of teacher trainees is configured to collect singing demonstration data, conducting gesture data, body rhythm data and teaching language data of teacher trainees in virtual music teaching scenarios, and integrate them according to a unified time benchmark to generate multimodal teaching behavior feature vectors.

[0085] The Music Teaching Behavior-Child Response Mapping Engine 3 is configured to convert multimodal teaching behavior feature vectors into virtual children's learning response data according to preset mapping rules, and to perform weighted fusion to generate a comprehensive response trend Rbehavior when multiple teaching behavior mapping rules are triggered simultaneously.

[0086] The immersive virtual music teaching interactive environment 4 is configured to present a virtual music classroom scene and virtual child characters based on AR technology, and drive the virtual child characters to present corresponding behavioral animations and singing audio based on learning response data;

[0087] The Teaching Emotion - Child Emotional Resonance Feedback Engine 8 is configured to use multimodal emotion recognition technology to perceive the emotional state and empathy level of student teachers in virtual music teaching scenarios in real time. This is then used as an independent dimension to modulate the output of the Music Teaching Behavior - Child Response Mapping Engine 3, enabling virtual children to experience emotional infection and resonance in response to the student teachers' emotional expressions. The engine calculates the Teaching Empathy Index (TEI) and, based on the emotion modulation coefficient... The comprehensive response trend Rbehavior is weighted twice by the emotional resonance response increment ΔRemo, enabling virtual children to generate emotional infection and emotional resonance response to the emotional expression of teacher trainees;

[0088] The module 9 for longitudinal tracking of teaching ability and personalized training path recommendation of teacher trainees is configured to persistently store multimodal data of teacher trainees in multiple training sessions, perform time-series modeling of teacher trainees’ mastery of music teaching knowledge point graph based on deep knowledge tracking model, generate teaching ability growth curve, and adaptively recommend target knowledge points, age group, class parameter configuration and difficulty level for the next training based on the zone of proximal development theory.

[0089] The system meets at least one of the following real-time performance metrics:

[0090] End-to-end response latency does not exceed 150 milliseconds;

[0091] The frame rate for virtual children's lip-syncing and body animation is no less than 60 frames per second;

[0092] The length of the audio buffer for singing along should not exceed 20 milliseconds;

[0093] The Teaching Empathy Index (TEI) is updated at a frequency of no less than 5 Hz.

[0094] Emotion Modulation Coefficient The injection delay is no more than 30 milliseconds;

[0095] The prediction and reasoning time for knowledge point mastery is no more than 200 milliseconds;

[0096] The generation time for personalized training path recommendation packages is no more than 3 seconds.

[0097] The write latency for persistent storage of training session data does not exceed 100 milliseconds.

[0098] The typical hardware environment of the system includes: a host running virtual simulation software, a microphone array connected to the host, motion capture equipment, VR headset and controllers, and speakers for outputting sing-along audio. The host runs an age-group virtual children's music behavior modeling engine 1, a multimodal data acquisition module for music teaching of teacher trainees 2, a music teaching behavior-child response mapping engine 3, an immersive virtual music teaching interactive environment 4, a training parameter configuration module 5, a dynamic adjustment module 6, a teaching ability assessment module 7, a teaching emotion-child emotional resonance feedback engine 8, and a module for longitudinal tracking of teaching ability of teacher trainees and personalized training path recommendation 9.

[0099] The age-group virtual children's music behavior modeling engine 1 includes a preschool children's music perception behavior sub-model 1a and a primary school students' music cognitive behavior sub-model 1b;

[0100] The preschool children's music perception behavior sub-model 1a is configured to encode the vocal range, rhythm cognitive characteristics, attention duration, body rhythm response pattern and group imitation preference of children aged 3-6. Among them, the vocal range is c1-a1, the rhythm cognitive characteristics are imitation over symbols, the attention duration is 5-10 minutes of periodic fluctuation, and the body rhythm response pattern is mainly body rhythm.

[0101] The primary school students' music cognitive behavior sub-model 1b is configured to encode the pitch discrimination ability, symbol recognition ability, choral singing ability, music preference differentiation characteristics, and cognitive strategy characteristics of children aged 6-12. Among them, pitch discrimination ability is relative pitch perception, symbol recognition ability is reading simple musical scores, choral singing ability is two-part coordination, music preference differentiation characteristics are obvious individual differentiation, and cognitive strategy characteristics are the use of memory strategies.

[0102] The primary school students' music cognitive behavior sub-model 1b sets the vocal range to d1-c2 in the lower primary school model, the symbol recognition ability to read simplified musical notation, the choral singing ability to sing in unison and simple rounds, and the attention span to 12-15 minutes. In the upper primary school model, the vocal range is expanded to c1-d2 in the lower primary school model, the symbol recognition ability to read staff notation and simplified musical notation, the choral singing ability to sing in two parts, and the attention span to 18-22 minutes.

[0103] Global class parameters include class cooperation benchmark, class basic music level benchmark, and class discipline level benchmark.

[0104] The individual offset parameter is used to superimpose a normally distributed random offset onto each virtual child based on the global class parameter, in order to reflect the behavioral differences between different virtual children within the same virtual class.

[0105] The multimodal data acquisition module 2 for music teaching for teacher trainees includes a singing demonstration acquisition unit 2a, a conducting gesture acquisition unit 2b, a body rhythm acquisition unit 2c, a teaching language acquisition unit 2d, and a multimodal data fusion unit 2e.

[0106] The singing demonstration acquisition unit 2a acquires audio of teacher trainees singing through a microphone array and uses the YIN pitch detection algorithm to extract pitch curves, rhythmic stability and acoustic performance characteristics. The acoustic performance characteristics include dynamic range, fundamental frequency perturbation, vibrato energy and loudness change rate.

[0107] The command gesture acquisition unit 2b captures the command gesture trajectory of the teacher trainees through a gesture recognition sensor. Specifically, it captures the command gesture trajectory of the teacher trainees through a depth camera combined with an inertial sensor, and extracts the beat point position, the predicted movement amplitude, the standardization of the beat diagram, and the clarity of the speed prompt.

[0108] The body movement acquisition unit 2c acquires the key point coordinate sequence of the teacher trainees through a depth camera and evaluates the synchronization of the movements with the music beat and the adaptability of the movement amplitude and complexity.

[0109] The 2D teaching language acquisition unit uses speech recognition technology to transcribe teaching language texts and analyzes vocabulary difficulty, sentence length, instruction clarity, and abstract concept density.

[0110] The multimodal data fusion unit 2e aligns the data collected by each unit with a unified system clock, performs interpolation and resampling on low sampling rate data, and uses a sliding window mechanism to construct a unified multimodal teaching behavior feature vector.

[0111] The Music Teaching Behavior-Child Response Mapping Engine 3 includes singing teaching mapping rule 3a, conducting teaching mapping rule 3b, rhythm teaching mapping rule 3c, and rule conflict fusion strategy 3d. When multiple types of teaching behavior mapping rules are triggered simultaneously, a weighted fusion is performed to generate a comprehensive response trend Rbehavior.

[0112] Singing teaching mapping rule 3a is used to generate virtual children's pitch deviation, rhythmic synchronization status, and changes in participation enthusiasm based on the pitch deviation, rhythmic stability, and acoustic expressiveness characteristics of the teacher trainees' demonstration singing;

[0113] Conducting instruction mapping rule 3b is used to generate the virtual children's choir's start time dispersion, tempo maintenance ability, and vocal part balance based on the clarity of the beat points, the standardization of the predictive movements, and the standardization of the beat diagram of the teacher trainees' conducting gestures.

[0114] The rhythm teaching mapping rule 3c is used to generate the synchronization rate and imitation accuracy of virtual children's body rhythm based on the synchronicity, amplitude and complexity of the teacher's demonstration of the movements and the music beat.

[0115] The rule conflict fusion strategy 3d is used to assign weights to singing teaching mapping rule 3a, conducting teaching mapping rule 3b and rhythm teaching mapping rule 3c based on the current teaching behavior type identification results, and then generate a comprehensive response trend Rbehavior by weighted summation after normalization.

[0116] Music teaching behavior - child response mapping engine 3 also includes integrated response generator 3e, which in turn includes virtual child singing synthesis submodule 3f;

[0117] The comprehensive response generator 3e combines individual offset parameters with the comprehensive response trend Rbehavior to generate a unique response sequence for each virtual child. The unique response sequence includes lip-syncing animation, body rhythm animation, singing audio, facial expression changes, and attention state changes.

[0118] The virtual children's singing synthesis submodule 3f renders virtual children's singing audio in real time based on a pre-recorded sample library of children's solo singing;

[0119] Based on a pre-recorded sample library of children's solo singing, and combined with the target pitch sequence, tempo parameters and acoustic performance parameters, the samples are adjusted for pitch shift, time stretching and amplitude envelope.

[0120] The immersive virtual music teaching interactive environment 4 includes a virtual music classroom scene 4a, a virtual child character rendering unit 4b, and a music teaching-specific interactive protocol 4c.

[0121] Virtual Music Classroom Scene 4a is used to present kindergarten music activity areas, primary school music classrooms, choir rehearsal halls, and ethnic music-themed scenes, and is equipped with virtual musical instruments, sound effects equipment, and teaching props;

[0122] The virtual child character rendering unit 4b configures each virtual child with a unique appearance and clothing, and presents music learning behavior through lip-syncing animation, body animation and facial expression animation;

[0123] The dedicated interactive protocol 4c for music teaching includes a gesture virtual tapping protocol, a voice singing protocol, and a rhythm guidance protocol, which are used to receive natural interactive input from teacher trainees and trigger corresponding virtual child responses.

[0124] Training parameter configuration module 5 is used to configure the target age group, global class parameters, virtual class size and child behavior parameter distribution, and provides scene configuration unit and difficulty level unit;

[0125] The scene configuration unit is used to set the preset class size, age group, and behavioral parameter distribution. Among them, the class size is 5-40 students, the age group is preschool and primary school, and the preset behavioral parameter distribution is high cooperation class, inattentive class, and class with varying musical abilities.

[0126] The difficulty level unit is used to set up beginner, intermediate and advanced modes. Beginner mode: high cooperation, good foundation, stable attention, basic skills training; Intermediate mode: normal cooperation, normal foundation, occasional distraction, comprehensive ability training; Advanced mode: low cooperation, uneven foundation, frequent problems, emergency management training.

[0127] The dynamic adjustment module 6 is used to dynamically fine-tune the global class parameters based on the real-time performance of teacher trainees, the stability of their teaching behavior, and the improvement rate of virtual children's responses, using an adaptive difficulty feedback adjustment algorithm.

[0128] The adaptive difficulty feedback adjustment algorithm can be implemented based on the proportional-integral control logic of the response improvement rate:

[0129] When the response improvement rate R > 1.2, the difficulty level is considered too low, and the benchmark for class cooperation is lowered by 10%, while the benchmark for discipline level is raised by 5%.

[0130] When the response improvement rate R < 0.8, the difficulty level is considered too high, and the benchmark for class cooperation is raised by 10%, while the benchmark for discipline level is lowered by 5%.

[0131] The teaching ability assessment module 7 is used to record and statistically analyze the four dimensions of teacher trainees: pitch stability, conducting clarity, rhythm synchronization rate, and language-child adaptability. It also generates a comprehensive teaching ability score and improvement suggestion report according to predetermined weights: 25%, 25%, 20%, and 30%.

[0132] The Teaching Emotion-Child Emotion Resonance Feedback Engine 8 includes a multimodal identification unit for teacher trainees' teaching emotions 8a, a teaching empathy engagement assessment unit 8b, an emotion-child emotional resonance mapping rule 8c, an emotion modulation coefficient calculator 8d, and an emotion state visualization and child emotional state generation unit 8e;

[0133] The multimodal recognition unit 8a for teaching emotions of teacher trainees is configured to recognize the four-dimensional emotional characteristics of teacher trainees in the teaching process in parallel through the facial expression emotion recognition subunit 8a-1, the voice emotion analysis subunit 8a-2, the body tension analysis subunit 8a-3, and the eye attention analysis subunit 8a-4.

[0134] The facial expression emotion recognition subunit 8a-1 collects the facial video stream of teacher trainees at 30fps through the built-in camera of the VR headset. It uses a deep learning-based facial expression recognition model to identify 7 basic expressions frame by frame. The basic expressions include happiness, calmness, surprise, sadness, anger, fear, and disgust. The expression probability distribution pface∈R^7 is output in a 0.2-second window.

[0135] The voice emotion analysis subunit 8a-2 jointly analyzes the teaching language and model singing audio of teacher trainees, extracts emotion-related acoustic features including speech rate, pitch range, mean and variance of energy, prosodic fluctuation, and zero crossing rate, and uses a support vector machine one-dimensional convolutional neural network to classify five teaching emotion states pvoice∈R^5: enthusiasm, calmness, anxiety, fatigue, and frustration.

[0136] The limb tension analysis subunit 8a-3 calculates the posture stability index, gesture amplitude variance, and hand asymmetry of the joint coordinate sequence output by the body rhythm acquisition unit 2c in a 2-second window. The posture stability index is the ratio of joint velocity variance to baseline. It outputs four levels of tension: relaxed, normal, tense, and stiff, Lbody∈{0,1,2,3}.

[0137] The eye-attention analysis subunit 8a-4 uses the built-in eye-tracking module of the VR headset to sample at 120Hz to obtain the gaze direction and gaze duration of the teacher trainees. It calculates the proportion of gaze time for virtual child characters, musical scores, and the classroom as a whole in a 5-second window, and outputs three levels of eye state: focused, distracted, and avoidant (Egaze∈{0,1,2}).

[0138] The teaching empathy engagement assessment unit 8b is configured to weight and fuse four-dimensional emotional characteristics according to predetermined weights to obtain the teaching empathy index TEI with a value range of [0,1].

[0139] The formula for calculating the Teaching Empathy Index (TEI) is:

[0140] ;

[0141] in, Scoring facial expressions Score the emotion in the voice. Scoring of limb tension Eye attention scores were assigned, and all scores were normalized to the [0,1] interval.

[0142] Specifically, the teaching empathy engagement assessment unit 8b converts the four-dimensional emotional features output by the teacher trainees' teaching emotion multimodal recognition unit 8a into standardized scores according to the following rules:

[0143] =1-( [Sadness]+ [Anger]+ [fear]);

[0144] = [Passion] ×1.0+ [Calm] ×0.8- [Anxiety] × 0.5- [Frustration] × 0.7 (truncate to [0,1]);

[0145] =1- / 3;

[0146] =1- / 2;

[0147] Then, the Teaching Empathy Index (TEI) with values ​​ranging from [0,1] is obtained by weighted summation using the following formula:

[0148] ;

[0149] Based on the Teaching Empathy Index (TEI), the real-time empathy status of teacher trainees was divided into four levels: low empathy, low-to-medium empathy, neutral empathy, and high empathy. Low empathy: TEI < 0.3, low-to-medium empathy: 0.3 ≤ TEI < 0.5, neutral empathy: 0.5 ≤ TEI < 0.7, and high empathy: TEI ≥ 0.7.

[0150] The emotion-child emotional resonance mapping rule 8c is configured to trigger at least one type of resonance rule among enthusiastic resonance, anxiety contagion, frustration inhibition, and calming soothing based on the teaching empathy index TEI and four-dimensional emotional features, and outputs the emotional resonance response increment ΔRemo;

[0151] Emotion-Children's Emotional Resonance Mapping Rule 8c, based on the developmental psychology theory of emotional contagion and empirical research in music teaching, defines the following four categories of emotional resonance mapping rules:

[0152] The 8c-1 rule for enthusiastic resonance states that when the emotional tone of the voice is enthusiastic and the TEI is ≥0.7, the virtual child's participation enthusiasm increases by 15%-25%, the facial animation leans towards excitement, and the mouth opening and closing range is amplified by 10%-20%.

[0153] Anxiety Contagion Rule 8c-2: When the vocal emotion is anxiety and the physical tension is tension and stiffness, the attention stability of virtual children decreases by 20%-35%, and 3-5 virtual children show signs of tension such as looking around and leaning back.

[0154] Frustration suppression rule 8c-3: When the vocal emotion is frustrated and the expression is sad for 5 consecutive seconds, the overall volume of the virtual children's choir decreases by 5-10 dB, and the singing speed slows down slightly by 0.95-0.98 times, simulating the children's empathetic response to the teacher's negative emotions.

[0155] Calm and Soothing Rule 8c-4: When the Teaching Empathy Index (TEI) is in neutral empathy and the body tension is relaxed, the virtual child's attention recovery speed increases by 30%, and the discipline level benchmark increases by 0.05-0.10, simulating the effect of children being soothed by the teacher in a calm state;

[0156] The Emotion Modulation Coefficient Calculator 8d is configured to calculate the emotion modulation coefficient based on the Teaching Empathy Index (TEI) and the Emotion-Behavior Matching Degree (Ematch). And perform a second weighting on the overall response trend Rbehavior;

[0157] Specifically, based on the comprehensive response trend Rbehavior output by the Music Teaching Behavior-Child Response Mapping Engine 3, an emotion modulation coefficient is introduced. The response magnitude is then weighted twice.

[0158] Emotion Modulation Coefficient The calculation formula is:

[0159] ;

[0160] in, = 0.5 is the neutral empathy reference value, α=0.6 is the affective intensity coefficient, which controls the amplification or inhibition of the response caused by the deviation of the Teaching Empathy Index (TEI) from the neutral value, and β=0.3 is the affective matching coefficient, which controls the additional effect of whether the teacher's affective type and teaching behavior type are consistent on the response. The emotion-behavior matching degree is set to 1 when the vocal emotion is enthusiastic and the current teaching behavior type is singing instruction, 1 when the vocal emotion is calm and the current teaching behavior type is conducting instruction, and 0 in other cases.

[0161] With this modulator, enthusiastic teachers can amplify children's responses by up to 1.18 times when TEI=0.8 and Ematch=1, while anxious teachers can suppress children's responses by up to 0.76 times when TEI=0.3, achieving a refined superposition of the emotional channel on the behavioral channel;

[0162] The emotional state visualization and child emotional state generation unit 8e is configured to generate a state vector of being emotionally infected by the teacher for each virtual child based on the modulated comprehensive response Rfinal and the emotional resonance response increment ΔRemo, and drive the immersive virtual music teaching interactive environment 4 to perform real-time rendering.

[0163] Specifically, the emotional resonance response increment ΔRemo, output by the emotion-child emotional resonance mapping rule 8c, and the modulated combined response output by the emotional modulation coefficient calculator 8d are calculated using the following formula:

[0164] ;

[0165] Based on this, a state vector containing the following five components, representing the emotional impact of the teacher, is generated for each virtual child:

[0166] The facial animation offset Eanim∈[0,1]^7 is the facial offset determined by the emotional resonance rule, which is superimposed on the neutral facial expression output by the music teaching behavior-child response mapping engine 3.

[0167] The body rhythm is accompanied by additional movements Aadd∈{0,1,2}. When there is enthusiastic resonance, jumping and clapping are added. When anxiety is contagious, restless micro-movements are added. When there is frustration and inhibition, bowing the head and silence are added.

[0168] The dynamic adjustment of the singing audio is Ladj∈[-10,+5]dB;

[0169] The attention focusing radius is Fradius∈[0.3,1.5] meters;

[0170] The emotion label Clabel ∈ {excitement / calm / nervous / depressed / focused / distracted} six categories;

[0171] The state vector is used by the immersive virtual music teaching interactive environment 4 to drive the synchronous rendering of the lip-sync animation, body animation, facial expression animation and singing audio of the virtual child character in real time.

[0172] The module 9 for longitudinal tracking of teaching ability and personalized training path recommendation for teacher trainees includes a persistent storage unit for training session data 9a, a music teaching knowledge point graph construction unit 9b, a deep knowledge tracker 9c, a teaching ability growth curve generation unit 9d, and a personalized training path recommendation engine 9e.

[0173] The training session data persistent storage unit 9a is configured to persistently store at least the following data for each training session, with the teacher trainee ID and training session ID as the primary keys: training configuration, time-series data of multimodal teaching behavior feature vectors throughout the training process, time-series data of virtual child response data throughout the training process, comprehensive teaching ability score for a single training session, emotional resonance feedback data, training timestamp, and training duration.

[0174] Training configuration: age group, class size, global class parameters, difficulty level, training repertoire; comprehensive teaching ability score for a single training session: including four dimensions: pitch stability, conducting clarity, rhythm synchronization rate, and language suitability for children; emotional resonance feedback data: including the Teaching Empathy Index (TEI) time series and emotional resonance response time series; data storage adopts a time series database + partitioned table; time series database: InfluxDB, partitioned by teacher trainee ID, with a retention period of no less than 3 years;

[0175] The Music Teaching Knowledge Graph Construction Unit 9b is configured to construct a Music Teaching Knowledge Graph (MPKG) containing 30 fine-grained knowledge points across four categories: singing teaching ability points, conducting teaching ability points, rhythm teaching ability points, and teaching language ability points. MPKG clarifies the prerequisite-successor relationships between each knowledge point, including beat clarity → predictive action → beat diagram standardization, pitch demonstration → expressive singing, rhythm synchronization → movement amplitude adaptation → ethnic rhythm demonstration. The graph is persistently stored in JSON-LD format.

[0176] MPKG was constructed by a group of experts in the field of music education based on annotations of current classic literature, and the knowledge point embedding vectors were pre-trained by Word2Vec on music teaching corpus.

[0177] The Deep Knowledge Tracer 9c is configured to use the LSTM-based Deep Knowledge Tracing model DKT. LSTM stands for 128-dimensional hidden units with 2 layers, and DKT stands for Deep Knowledge Tracing. The input is the knowledge point interaction sequence from each training session of the teacher trainees, defined as follows:

[0178] ;

[0179] in, Let be the set of knowledge points involved in the t-th training session. The scores for each knowledge point in this training session are determined by the normalized mapping from Module 7 of the Teaching Ability Assessment. Embed vectors for knowledge points;

[0180] The model outputs the mastery prediction of all knowledge points at the next time step through the Embedding layer, the fully connected layer and the Sigmoid, p{t+1}∈[0,1]^N, where N is the total number of knowledge points;

[0181] The model is trained using a binary cross-entropy loss function. After each new training iteration, the training data is added to the sequence, and the Adam optimizer is used to fine-tune the model online for one epoch. The learning rate is 1e-4, and the batch size is 8, so that the model can gradually adapt to the learning pace and knowledge structure of specific teacher trainees.

[0182] The teaching ability growth curve generation unit 9d is configured to generate two types of visualization curves based on the time-series data of knowledge point mastery output by the deep knowledge tracker 9c:

[0183] (i) Growth curve for a single knowledge point, with the horizontal axis representing the training session number and the vertical axis representing the mastery of the knowledge point, superimposed with the LOESS smoothing trend line and the 95% confidence interval;

[0184] (ii) Major Category Ability Radar Chart: The average current mastery level of knowledge points under the four major categories of singing, conducting, rhythm, and language is mapped onto the radar. Figure 4 The dimension shows the evolution of the capability profile with the number of training sessions;

[0185] Anomaly detection is performed on the growth curve. If the mastery improvement of a knowledge point after three consecutive training sessions is Δp < 0.02, it is marked as a plateau knowledge point. If Δp < 0, it is marked as a regressing knowledge point. The marking results are added to the list of key knowledge points.

[0186] The personalized training path recommendation engine 9e is configured based on the Zone of Proximal Development (ZPD) theory. ZPD outputs the recommended training path for the next training iteration according to the following steps:

[0187] Step e-1, Target knowledge point selection: Take the current mastery vector pt, and divide the mastery into three levels: weak points, growth zone, and strong points. Weak points: <0.6, growth zone: [0.6, 0.85), strong points: ≥0.85;

[0188] Prioritize weak and growth areas as training targets for the next session. Review strong areas every 5 training sessions to prevent forgetting. The specific allocation is: 2 high-priority weak areas, 1-2 medium-priority growth areas, and 0-1 low-priority strong areas.

[0189] Step e-2, Knowledge Point Association Expansion: Using the selected main knowledge point as the anchor point, find 1-2 direct successor or predecessor knowledge points in MPKG as related knowledge points;

[0190] Step e-3, Age Group and Scene Recommendations: Singing and rhythm-related knowledge points are recommended for preschool, lower elementary grades, and elementary school music classrooms; conducting and teaching language-related knowledge points are recommended for upper elementary grades and choral rehearsal halls; ethnic music-related knowledge points are recommended for ethnic music-themed scenes.

[0191] Step e-4, Recommended difficulty level and class parameters: Calculate the learning rate v = (pt - p{t-3}) / 3 for the last three training sessions, and compare it with the historical average learning rate. Compare:

[0192] like The difficulty level will be increased by one level, adjusted from beginner to intermediate to advanced, with the cooperation benchmark lowered by 5-10% and the discipline benchmark increased by 5%.

[0193] like If the adjustment is lowered by one level, the compliance benchmark will be raised by 5-10%; otherwise, the current level will be maintained.

[0194] Step e-5, Recommendation Package Output: Output the recommendation package for the next training session, including the target knowledge point list, related knowledge point list, recommended age group, recommended scenario, recommended difficulty level, initial global class parameters, recommended teaching track range, expected training duration, key skills to focus on in this training session, and expected improvement in mastery. Target knowledge point list: up to 3 main knowledge points + 2 related knowledge points. Initial global class parameters: cooperation baseline, music baseline, discipline baseline, individual variance.

[0195] Step e-6, linked with the training parameter configuration module and dynamic adjustment module: the recommended package for the next training session is presented to the teacher trainees as a pre-filled configuration of the training parameter configuration module 5 when the next training session starts. The teacher trainees can apply it with one click or apply it after fine-tuning.

[0196] During this training process, the dynamic adjustment module 6 uses the initial global class parameters in the recommended package as a benchmark for real-time fine-tuning, rather than starting from the default values, thus achieving a two-layer closed loop of long-term planning → short-term fine-tuning.

[0197] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A virtual children's behavior modeling and real-time interactive system for music teaching for teacher trainees, characterized in that, include: The age-group virtual children's music behavior modeling engine (1) is configured to construct a virtual children's behavior model based on the children's music cognitive characteristic parameters corresponding to the target age group; The multimodal data acquisition module for music teaching of teacher trainees (2) is configured to collect singing demonstration data, conducting gesture data, body rhythm data and teaching language data of teacher trainees, and integrate them to generate multimodal teaching behavior feature vectors; Music teaching behavior-child response mapping engine (3) is configured to convert the multimodal teaching behavior feature vectors into virtual children's learning response data and generate a comprehensive response trend Rbehavior; The teaching emotion-child emotional resonance feedback engine (8) is configured to perceive the emotional state and empathy engagement of teacher trainees in real time, calculate the teaching empathy index (TEI), and calculate the emotion modulation coefficient based on the TEI. The comprehensive response trend Rbehavior is weighted a second time by the emotional resonance response increment ΔRemo. An immersive virtual music teaching interactive environment (4) is configured to drive virtual children's characters to present corresponding behavioral animations and singing audio based on the second-weighted response data; The module for longitudinal tracking of teaching ability and personalized training path recommendation for teacher trainees (9) is configured to persistently store training session data. Based on the deep knowledge tracking model, the module performs time-series modeling of the mastery of music teaching knowledge points in the graph and recommends the next training configuration parameters based on the zone of proximal development theory.

2. The age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees according to claim 1, characterized in that, The age-segmented virtual children's music behavior modeling engine (1) includes a preschool children's music perception behavior sub-model (1a) and a primary school students' music cognitive behavior sub-model (1b). The preschool children's music perception behavior sub-model (1a) is configured to encode the vocal range, rhythmic cognitive characteristics, attention duration, body rhythm response patterns and group imitation preferences of children aged 3-6. The primary school students' music cognitive behavior sub-model (1b) is configured to encode the pitch discrimination ability, symbol recognition ability, choral singing ability, music preference differentiation characteristics and cognitive strategy characteristics of children aged 6-12. The virtual child behavior model includes a two-layer structure of global class parameters and individual offset parameters. The global class parameters determine the overall difficulty level of the training scenario, and the individual offset parameters are superimposed with random offsets on the global class parameters to reflect the behavioral differences between different virtual children in the same virtual class. The global class parameters include class cooperation benchmark, class basic music level benchmark, and class discipline level benchmark. The individual offset parameter is used to superimpose a random offset following a normal or Beta distribution onto each virtual child based on the global class parameter, so as to reflect the behavioral differences between different virtual children in the same virtual class.

3. The age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees according to claim 1, characterized in that, The multimodal data acquisition module (2) for music teaching of teacher trainees generates multimodal teaching behavior feature vectors by fusing data according to a unified time reference. The multimodal data acquisition module for music teaching for teacher trainees (2) includes a singing demonstration acquisition unit (2a), a conducting gesture acquisition unit (2b), a body rhythm acquisition unit (2c), a teaching language acquisition unit (2d), and a multimodal data fusion unit (2e). The singing demonstration acquisition unit (2a) acquires audio of student teachers singing demonstrations through a microphone array; The command gesture acquisition unit (2b) captures the trajectory of the teacher trainee's command gestures through a gesture recognition sensor; The body rhythm acquisition unit (2c) acquires the key point coordinate sequence of the teacher trainee through a depth camera or visual pose estimation algorithm; The teaching language acquisition unit (2d) transcribes teaching language text using speech recognition technology.

4. The age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees according to claim 1, characterized in that, The music teaching behavior-child response mapping engine (3) includes singing teaching mapping rules (3a), conducting teaching mapping rules (3b), rhythm teaching mapping rules (3c), and rule conflict fusion strategy (3d). When multiple types of teaching behavior mapping rules are triggered simultaneously, a weighted fusion is performed to generate a comprehensive response trend Rbehavior. The singing teaching mapping rule (3a) is used to generate virtual children's pitch deviation, rhythm synchronization status and changes in participation enthusiasm based on the pitch deviation, rhythm stability and acoustic performance characteristics of the teacher trainees' singing demonstration; The conducting teaching mapping rule (3b) is used to generate the virtual children's chorus starting time dispersion, speed maintenance ability and vocal part balance state based on the clarity of the beat points, the standardization of the predicted actions and the standardization of the beat diagram of the teacher trainees' conducting gestures. The rhythmic teaching mapping rule (3c) is used to generate the synchronization rate and imitation accuracy of virtual children's body rhythm based on the synchronicity, amplitude and complexity of the teacher's demonstration of the movements and the music beat. The rule conflict fusion strategy (3d) is used to assign weights to the singing teaching mapping rule (3a), conducting teaching mapping rule (3b) and rhythm teaching mapping rule (3c) based on the current teaching behavior type identification results, and then generate a comprehensive response trend Rbehavior by weighted summation after normalization.

5. A virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees of claim 4, characterized in that, The music teaching behavior-child response mapping engine (3) also includes a comprehensive response generator (3e), which further includes a virtual children's singing voice synthesis submodule (3f). The integrated response generator (3e) combines individual offset parameters with the integrated response trend Rbehavior to generate a unique response sequence for each virtual child; The virtual children's singing synthesis submodule (3f) renders virtual children's singing audio in real time based on a pre-recorded sample library of children's solo or choral singing or using a parametric singing synthesis model.

6. The age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees according to claim 1, characterized in that, The immersive virtual music teaching interactive environment (4) includes a virtual music classroom scene (4a), a virtual child character rendering unit (4b), and a music teaching-specific interactive protocol (4c). The training parameter configuration module (5) is used to configure the target age group, global class parameters, and the distribution of virtual class size and children's behavior parameters; The dynamic adjustment module (6) is used to dynamically fine-tune the global class parameters based on the real-time performance of the teacher trainees, the stability of their teaching behavior, and the improvement rate of virtual children's responses through an adaptive difficulty feedback adjustment algorithm. The teaching ability assessment module (7) is used to record and statistically analyze the pitch stability, command clarity, rhythm synchronization rate and language-child adaptability of teacher trainees, and generate a comprehensive teaching ability score and improvement suggestion report.

7. The age-group virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees according to claim 1, characterized in that, The teaching emotion-child emotion resonance feedback engine (8) includes a teacher trainee teaching emotion multimodal recognition unit (8a), a teaching empathy engagement assessment unit (8b), an emotion-child emotion resonance mapping rule (8c), an emotion modulation coefficient calculator (8d), and an emotion state visualization and child emotion state generation unit (8e). The multimodal recognition unit (8a) for teacher trainees' teaching emotions is configured to recognize the four-dimensional emotional characteristics of teacher trainees in the teaching process in parallel. The teaching empathy engagement assessment unit (8b) is configured to weight and fuse four-dimensional emotional features according to predetermined weights to obtain a teaching empathy index (TEI) with a value range of [0,1].

8. A virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees of claim 7, characterized in that, The emotion-child emotional resonance mapping rule (8c) is configured to trigger at least one type of resonance rule among enthusiastic resonance, anxiety contagion, frustration inhibition, and calming soothing based on the teaching empathy index TEI and four-dimensional emotional features, and outputs the emotional resonance response increment ΔRemo; The emotion modulation coefficient calculator (8d) is configured to calculate the emotion modulation coefficient based on the Teaching Empathy Index (TEI) and the Emotion-Behavior Matching Degree (Ematch). And the comprehensive response trend Rbehavior is weighted twice; The emotional state visualization and child emotional state generation unit (8e) is configured to generate a state vector of being emotionally infected by the teacher for each virtual child based on the modulated comprehensive response Rfinal and the emotional resonance response increment ΔRemo, and drive the immersive virtual music teaching interactive environment (4) to perform real-time rendering.

9. A virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees of claim 1, characterized in that, The longitudinal tracking and personalized training path recommendation module (9) for teacher trainees' teaching abilities includes a persistent storage unit for training session data (9a), a music teaching knowledge point graph construction unit (9b), a deep knowledge tracker (9c), a teaching ability growth curve generation unit (9d), and a personalized training path recommendation engine (9e). The training session data persistent storage unit (9a) is configured to persistently store the data of each training session using the teacher trainee ID and the training session ID as primary keys; The music teaching knowledge point map construction unit (9b) is configured to construct a music teaching knowledge point map MPKG containing four categories of music teaching ability points, conducting ability points, rhythm teaching ability points, and teaching language ability points, totaling 20-40 fine-grained knowledge points, and to clarify the prerequisite-successor relationships between each knowledge point; The deep knowledge tracker (9c) is configured to use a deep knowledge tracking model DKT based on LSTM or Transformer to perform time-series modeling of the knowledge point interaction sequence of each training session for teacher trainees, output the mastery prediction of each knowledge point at the next moment, and perform online fine-tuning after each new training session.

10. A virtual children's behavior modeling and real-time interaction system for music teaching for teacher trainees of claim 9, characterized in that, The teaching ability growth curve generation unit (9d) is configured to generate a single knowledge point growth curve and four major categories of ability radar charts based on the time series data of knowledge point mastery, and to perform plateau and regression anomaly detection on the growth curve. The personalized training path recommendation engine (9e) is configured to select target knowledge points based on the Zone of Proximal Development (ZPD) theory, and combine knowledge point graph association expansion, age group and scene matching, learning rate adaptive difficulty level and class parameter recommendation to output the next training recommendation package; The next training recommendation package shall include at least: a list of target knowledge points, a list of related knowledge points, a recommended age group, a recommended scenario, a recommended difficulty level, initial global class parameters, a range of recommended teaching materials, an expected training duration, key skills to be focused on in this training, and an expected improvement in mastery. The recommended package for the next training session is presented to the student teacher as a pre-filled configuration of the training parameter configuration module (5) when the next training session starts, and serves as the benchmark parameter for the dynamic adjustment module (6) to fine-tune in real time during this training session.

Images