Virtual simulation teaching adaptive optimization system and method based on multi-modal behavior recognition

Abstract

The invention discloses a virtual simulation teaching adaptive optimization system and method based on multi-modal behavior recognition, and relates to the technical field of virtual simulation teaching. Comprising a data acquisition module which is responsible for collecting multi-modal data of students in a virtual simulation teaching process; and the data processing and analysis module is used for carrying out preprocessing, feature extraction and comprehensive analysis on the collected multi-modal data so as to obtain behavior features and knowledge mastering conditions of the students in the learning process. According to the system, knowledge points can be intelligently divided into three levels of mastered, partially mastered and unmastered, and resource information in a teaching resource library is combined, so that a personalized learning path is planned for students, the learning efficiency is improved, repeated learning is avoided, and the students are ensured to effectively consolidate the unmastered knowledge points at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual simulation teaching, and in particular to a self-adaptive optimization system and method for virtual simulation teaching based on multimodal behavior recognition. Background Art

[0002] With the rapid development of information technology and the in-depth advancement of educational informatization, virtual simulation teaching, as an emerging teaching method, is gradually gaining favor among educators. By simulating real-life scenarios and operational processes, virtual simulation teaching provides students with a safe, controllable, and efficient learning environment. However, traditional virtual simulation teaching often uses fixed teaching models and paths, making it difficult to adapt to the learning needs and progress of different students, resulting in uneven teaching results.

[0003] After searching, the application scheme of Chinese patent application number CN202210196436.7 discloses a virtual simulation training teaching management system, including an authority management subsystem, a teaching management subsystem, a practical operation management subsystem and a simulation application subsystem. The simulation application subsystem includes several virtual simulation projects; the authority management subsystem is used to verify the legitimacy of the login identity and stipulate its usage rights; the teaching management subsystem is used to provide teachers with online course scheduling services for the selected virtual simulation projects, and then generate practical operation request information to feed back to the simulation application subsystem. The teaching management system in the above patent has the following shortcomings: although it provides the function of virtual simulation teaching, it is managed uniformly through the system, lacks independent analysis of students, is difficult to adapt to the learning needs and progress of different students, and is less targeted. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a self-adaptive optimization system and method for virtual simulation teaching based on multimodal behavior recognition.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition includes:

[0007] A data acquisition module is responsible for collecting multimodal data of students during the virtual simulation teaching process;

[0008] A data processing and analysis module, which performs preprocessing, feature extraction, and comprehensive analysis on the collected multimodal data to obtain students' behavioral characteristics and knowledge mastery during the learning process;

[0009] A learning path generation module, which generates a personalized learning path for students based on the students' learning behavior characteristics and knowledge mastery obtained by the data processing and analysis module, combined with teaching objectives and course requirements;

[0010] A teaching resource library module, wherein the teaching resource library module stores virtual simulation teaching resources;

[0011] An interactive interface module is used for students to interact with the system.

[0012] Preferably, the data acquisition module specifically acquires data in the following manner:

[0013] Operation trajectory data: By embedding tracking scripts in the virtual simulation software, every operation action of the students in the virtual environment is recorded;

[0014] Dwell time data: Using the scene switching and element interaction events of the virtual simulation software, record the time students spend in each virtual scene or teaching element;

[0015] Error frequency data: In virtual simulation teaching, an error detection mechanism is set up to record the error type and time when the student's operation makes an error.

[0016] Preferably, the data processing and analysis module processes data in the following manner:

[0017] Data preprocessing: Clean and normalize the collected operation trajectory data, dwell time data, and error frequency data to remove noise data and outliers;

[0018] Feature extraction: Extract useful features from preprocessed data to describe students' learning behavior and status.

[0019] Preferably, the data processing and analysis module performs analysis in the following specific manner:

[0020] Machine learning algorithms and data analysis methods are used to conduct a comprehensive analysis of the extracted feature vectors to establish a mapping relationship between students' learning behavior and their knowledge mastery. A neural network model is used to train students' feature vectors to obtain a prediction model, which predicts the students' mastery of each knowledge point based on their current characteristics.

[0021] Preferably, when performing feature extraction, the data processing and analysis module specifically includes:

[0022] Operation trajectory feature extraction: Calculate the operation speed feature. The operation speed is measured by the number of operations per unit time. Let the number of operations within a time interval Δt be N Δt , operating speed Extract the operation frequency feature, which is the number of specific types of operations per unit time; the operation path complexity feature is measured by calculating the total length of the operation trajectory and the number of turning points. The Euclidean distance is used to calculate the distance between adjacent operation points, and the total length of the operation trajectory L is accumulated. traj , the number of turning points N turn Determine by judging the change of operation direction; these operation trajectory features are combined into a feature vector F traj ={V op ,N Δt ,L traj ,N turn};

[0023] Residence time feature extraction: Extract the average residence time, the longest residence time, and the shortest residence time features from the residence time data. Suppose the residence time of students in the jth virtual scene or teaching element is The average residence time Where m is the total number of virtual scenes or teaching elements, and the maximum stay time is Minimum stay time These residence time features are combined into a feature vector

[0024] Error frequency feature extraction: Extract error rate and error concentration features from error frequency data. The error rate is calculated by the ratio of the number of errors to the total number of operations. Let the total number of operations be N. total , the number of errors is N err , then the error rate Error concentration is measured by calculating the frequency of errors occurring in certain key knowledge points or operation links. Let the number of errors related to key knowledge points be Error concentration These error frequency features are combined into a feature vector F err ={R err ,C err}.

[0025] Preferably: when the data processing and analysis module performs comprehensive analysis, the extracted operation trajectory feature vector F traj , residence time feature vector F stay and the error frequency eigenvector F err Merge into a comprehensive feature vector F = {F traj ,F stay ,F err}; Use machine learning algorithms to train comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery.

[0026] Preferably: when the data processing and analysis module performs comprehensive analysis, when using a neural network model, the comprehensive feature vector is used as the input of the input layer, and through the nonlinear transformation and weight adjustment of the hidden layer, the predicted value of the student's mastery of each knowledge point is finally obtained in the output layer.

[0027] Preferably, the learning path generation process of the learning path generation module includes knowledge point evaluation and learning path planning, wherein the knowledge point evaluation is to classify the knowledge points into three levels: mastered, partially mastered, and not mastered according to the mastery level Kl of each knowledge point obtained by the prediction model M; assuming that the total number of knowledge points is L, for the lth knowledge point, if K l ≥θ1, it is considered that the student has mastered the knowledge point; if θ2≤K l <θ1, then the student is considered to have partially mastered the knowledge point; if K l <θ2, it is considered that the student has not mastered the knowledge point; among them, θ1 and θ2 are preset thresholds.

[0028] Preferably, the learning path planning of the learning path generation module is as follows: according to the evaluation results of the knowledge points, combined with the correlation and difficulty of the teaching resources, a personalized learning path is planned for the students; assuming that the total number of resources in the teaching resource library is R, and the correlation between the rth resource and the lth knowledge point is A rl , the difficulty of the resource is D r , then the learning path planning is expressed as an optimization problem:

[0029]

[0030] x r ∈{0,1}

[0031]

[0032] Among them, x r is the decision variable, indicating whether to select the rth teaching resource; is the target mastery level, that is, the mastery level of the lth knowledge point that students need to achieve; the constraint condition indicates that the selected teaching resources should be able to meet the students' learning needs for the knowledge points that have not been mastered; by solving this optimization problem, the optimal learning path for students is obtained.

[0033] Preferably, the optimization method of the optimization system comprises the following steps:

[0034] S1: Collect operation trajectory data, dwell time data and error frequency data;

[0035] S2: Clean and normalize the collected operation trajectory data, dwell time data, and error frequency data;

[0036] S3: Extract features from the operation trajectory data (such as calculation operation speed features, operation frequency features, and operation path complexity features) to describe the student's operation behavior, and combine these operation trajectory features into an operation trajectory feature vector;

[0037] S4: extract the average residence time, the longest residence time, and the shortest residence time features from the residence time data, and form these residence time features into a residence time feature vector;

[0038] S5: extracting error rate and error concentration features from the error frequency data, and combining these error frequency features into an error frequency feature vector. When performing comprehensive analysis, the data processing and analysis module combines the extracted operation trajectory feature vector, dwell time feature vector, and error frequency feature vector into a comprehensive feature vector.

[0039] S6: Use machine learning algorithms to train comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery. This is used to analyze students' learning behavior data in real time and predict their mastery of knowledge points.

[0040] S7: Based on the predicted value of the knowledge point mastery obtained by the comprehensive analysis module, the knowledge points are divided into three levels: mastered, partially mastered, and not mastered;

[0041] S8: Based on the evaluation results of knowledge points and combined with the resource information in the teaching resource library, a personalized learning path is planned for students;

[0042] S9: According to the learning path determined by the learning path generation module, the corresponding teaching resources are matched from the teaching resource library.

[0043] The beneficial effects of the present invention are:

[0044] 1. The system of the present invention can comprehensively capture students' learning behaviors and status through the comprehensive collection of various data such as operation trajectory, dwell time and error frequency, providing a rich information basis for accurate analysis and personalized teaching.

[0045] 2. Through in-depth analysis of different types of data, the present invention extracts key features such as operation speed, operation frequency, path complexity, dwell time statistics and error rate, which can carefully reflect students' learning habits, understanding level and operation proficiency; based on the results of data processing and analysis, the system can intelligently divide knowledge points into three levels: mastered, partially mastered and not mastered, and combine the resource information in the teaching resource library to plan personalized learning paths for students, which helps to improve learning efficiency, avoid repetitive learning, and ensure that students effectively consolidate the knowledge points that have not been mastered.

[0046] 3. The present invention has adaptive optimization capabilities and can adjust teaching strategies and resource recommendations in real time based on students' learning progress and feedback, so that the system can better meet students' personalized needs and provide a more practical learning experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the optimization method of the multimodal behavior recognition virtual simulation teaching adaptive optimization system proposed by the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention will be further described in detail below in conjunction with specific implementation methods.

[0049] Example 1:

[0050] The adaptive optimization system of virtual simulation teaching based on multimodal behavior recognition includes: data acquisition module, data processing and analysis module, learning path generation module, teaching resource library module, and interactive interface module;

[0051] in:

[0052] The data acquisition module is responsible for collecting multimodal data of students during the virtual simulation teaching process, including operation trajectory data, dwell time data, error frequency data, and other related data. The specific collection method is as follows:

[0053] ① Operation trajectory data: By embedding a tracking script in the virtual simulation software, each operation action of the student in the virtual environment is recorded, such as the timestamp and position information of mouse clicks, keyboard input, dragging, rotation, etc.; the operation trajectory of the student is a time series T = {t1, t2, ..., t n}, where t i Indicates the time point of the i-th operation, and the corresponding operation position information is P i =(x i ,y i ,z i ), the operation trajectory data is expressed as D traj ={(t i ,P i )|i=1,2,…,n};

[0054] ② Dwell time data: Using the scene switching and element interaction events of the virtual simulation software, record the time students spend in each virtual scene or teaching element; for the jth virtual scene or teaching element, the dwell time is for: in, represents the time when the student enters the jth scene or element, Indicates the time when the student leaves the scene or element; the residence time data of all scenes or elements constitute a set Where m is the total number of virtual scenes or teaching elements;

[0055] ③ Error frequency data: In virtual simulation teaching, an error detection mechanism is set up. When students make mistakes in their operations, the error type and the time when the error occurs are recorded. Let the number of occurrences of the kth error type be E k , then the error frequency data is expressed as D err ={E k |k=1,2,…,p}, where p is the total number of error types. At the same time, in order to analyze the severity of the error, a weight W is assigned to each error type k ,The weight is set according to professional knowledge. For example, for incorrect operations on key knowledge points, the weight is set higher.

[0056] The data processing and analysis module pre-processes, extracts features, and conducts comprehensive analysis on the collected multimodal data to obtain students' behavioral characteristics and knowledge mastery during the learning process. The specific methods are as follows:

[0057] ① Data preprocessing: The collected operation trajectory data, residence time data, and error frequency data are cleaned and normalized to remove noise data and outliers. For example, for operation trajectory data, a sliding window method is used for smoothing to reduce the impact of random fluctuations. For residence time data and error frequency data, they are standardized to conform to a certain numerical range to facilitate subsequent analysis and comparison.

[0058] ② Feature extraction: Extract useful features from the preprocessed data to describe students’ learning behavior and status; for example, extract features such as operation speed, operation frequency, and operation path complexity from the operation trajectory data; extract features such as average stay time, maximum stay time, and minimum stay time from the stay time data; extract features such as error rate and error concentration from the error frequency data; let the extracted feature vector be F = {f1, f2, ..., f q}, where q is the total number of features.

[0059] ③ Comprehensive analysis: Using machine learning algorithms and data analysis methods, we conduct a comprehensive analysis of the extracted feature vectors and establish a mapping relationship between students’ learning behavior and their knowledge mastery. We use a neural network model to train the student’s feature vectors and obtain a prediction model M. This model predicts the student’s mastery of each knowledge point based on their current characteristics. Let the student’s mastery of the lth knowledge point be K. l , then the prediction model is expressed as: K l=M(F) where the input of M is the feature vector F and the output is the student’s mastery of each knowledge point K l ; Improve the accuracy and reliability of predictions by training and optimizing model parameters with historical data.

[0060] The learning path generation module generates personalized learning paths for students based on the learning behavior characteristics and knowledge mastery of students obtained by the data processing and analysis module, combined with teaching objectives and course requirements. The learning path generation process is as follows:

[0061] ① Knowledge point evaluation: Based on the students’ mastery of each knowledge point Kl obtained by the prediction model M, the knowledge points are divided into three levels: mastered, partially mastered, and not mastered; let the total number of knowledge points be L, for the lth knowledge point, if K l ≥θ1, it is considered that the student has mastered the knowledge point; if θ2≤K l <θ1, then the student is considered to have partially mastered the knowledge point; if K l <θ2, it is considered that the student has not mastered the knowledge point; θ1 and θ2 are preset thresholds, which are adjusted according to teaching requirements and actual conditions;

[0062] ② Learning path planning: Based on the evaluation results of knowledge points, combined with the correlation and difficulty of teaching resources, a personalized learning path is planned for students. The goal of learning path planning is to enable students to complete the learning and consolidation of unmastered knowledge points in the shortest time and in the most effective way, while avoiding repeated learning of mastered knowledge points. Suppose the total number of resources in the teaching resource library is R, and the correlation between the rth resource and the lth knowledge point is A rl , the difficulty of the resource is D r , then the learning path planning is expressed as an optimization problem:

[0063]

[0064] x r ∈{0,1}

[0065]

[0066] Among them, x r is the decision variable, indicating whether to select the rth teaching resource; is the target mastery level, that is, the mastery level of the lth knowledge point that students need to achieve; the constraint condition indicates that the selected teaching resources should be able to meet the students' learning needs for the knowledge points that have not been mastered; by solving this optimization problem, the optimal learning path for students is obtained, that is, the order of teaching content and resources that need to be learned in sequence.

[0067] The teaching resource library module stores virtual simulation teaching resources, including virtual experiment scenarios, teaching courseware, video tutorials, exercises, tests, etc. Teaching resources are categorized and labeled according to knowledge points and difficulty levels, so that they can be accurately matched and recommended according to students' needs in the learning path generation module.

[0068] Interactive interface module,The interactive interface module is used for students to interact with the system.

[0069] The data processing and analysis module, when performing feature extraction, specifically includes:

[0070] Operation trajectory feature extraction: Extract multiple features from the operation trajectory data to describe the student's operation behavior. For example, calculate the operation speed feature, which is measured by the number of operations per unit time. Let the number of operations within a time interval Δt be N Δt , then the operating speed In addition, the operation frequency feature is extracted, that is, the number of specific types of operations per unit time; the operation path complexity feature is measured by calculating the total length of the operation trajectory, the number of turning points, and other indicators. For example, the Euclidean distance is used to calculate the distance between adjacent operation points, and the total length of the operation trajectory L is accumulated. traj , the number of turning points N turn Determine by judging the change of operation direction; these operation trajectory features are combined into a feature vector F traj ={V op ,N Δt ,L traj ,N turn}.

[0071] Stay time feature extraction: Extract features such as average stay time, maximum stay time, and minimum stay time from the stay time data. Suppose the stay time of students in the jth virtual scene or teaching element is The average residence time Where m is the total number of virtual scenes or teaching elements. Minimum stay time These residence time features are combined into a feature vector

[0072] Error frequency feature extraction: Extract features such as error rate and error concentration from error frequency data. The error rate is calculated by the ratio of the number of errors to the total number of operations. Let the total number of operations be N. total , the number of errors is N err , then the error rate Error concentration is measured by calculating the frequency of errors occurring in certain key knowledge points or operation links. For example, let the number of errors related to key knowledge points be Error concentration These error frequency features are combined into a feature vector F err ={R err ,C err}.

[0073] The data processing and analysis module extracts the operation trajectory feature vector F traj , residence time feature vector F stay and the error frequency eigenvector F err Merge into a comprehensive feature vector F = {F traj ,F stay ,F err Machine learning algorithms, such as neural networks or support vector machines, are used to train the comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery. For example, when using a neural network model, the comprehensive feature vector is used as the input of the input layer, and through nonlinear transformation and weight adjustment of the hidden layer, the predicted value of the student's mastery of each knowledge point is finally obtained in the output layer. During the training process, a large amount of historical data is used as training samples, and the parameters of the model are continuously optimized through the backpropagation algorithm until the model's prediction error reaches a satisfactory level. The trained model is used to analyze students' learning behavior data in real time and predict students' mastery of knowledge points.

[0074] Example 2:

[0075] The optimization method of the adaptive optimization system of virtual simulation teaching based on multimodal behavior recognition includes the following steps:

[0076] S1: Collect operation trajectory data, dwell time data and error frequency data;

[0077] S2: Clean and normalize the collected operation trajectory data, dwell time data, and error frequency data;

[0078] S3: Extract features from the operation trajectory data (such as calculation operation speed features, operation frequency features, and operation path complexity features) to describe the student's operation behavior, and combine these operation trajectory features into an operation trajectory feature vector;

[0079] S4: extract the average residence time, the longest residence time, and the shortest residence time features from the residence time data, and form these residence time features into a residence time feature vector;

[0080] S5: extracting features such as error rate and error concentration from the error frequency data, and combining these error frequency features into an error frequency feature vector. When performing comprehensive analysis, the data processing and analysis module combines the extracted operation trajectory feature vector, dwell time feature vector, and error frequency feature vector into a comprehensive feature vector.

[0081] S6: Use machine learning algorithms to train comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery. This is used to analyze students' learning behavior data in real time and predict their mastery of knowledge points.

[0082] S7: Based on the predicted value of the knowledge point mastery obtained by the comprehensive analysis module, the knowledge points are divided into three levels: mastered, partially mastered, and not mastered;

[0083] S8: Based on the evaluation results of knowledge points and combined with the resource information in the teaching resource library, a personalized learning path is planned for students;

[0084] S9: According to the learning path determined by the learning path generation module, the corresponding teaching resources are matched from the teaching resource library.

[0085] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. Based on multimodal behavior recognition virtual simulation teaching adaptive optimization system, it is characterized by: include: A data acquisition module is responsible for collecting multimodal data of students during the virtual simulation teaching process; A data processing and analysis module, which performs preprocessing, feature extraction, and comprehensive analysis on the collected multimodal data to obtain students' behavioral characteristics and knowledge mastery during the learning process; A learning path generation module, which generates a personalized learning path for students based on the students' learning behavior characteristics and knowledge mastery obtained by the data processing and analysis module, combined with teaching objectives and course requirements; A teaching resource library module, wherein the teaching resource library module stores virtual simulation teaching resources; An interactive interface module is used for students to interact with the system.

2. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 1 is characterized in that: The specific collection method of the data collection module is as follows: Operation trajectory data: By embedding tracking scripts in the virtual simulation software, every operation action of the students in the virtual environment is recorded; Dwell time data: Using the scene switching and element interaction events of the virtual simulation software, record the time students spend in each virtual scene or teaching element; Error frequency data: In virtual simulation teaching, an error detection mechanism is set up to record the error type and time when the student's operation makes an error.

3. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 1 is characterized in that: The data processing and analysis module specifically processes data as follows: Data preprocessing: Clean and normalize the collected operation trajectory data, dwell time data, and error frequency data to remove noise data and outliers; Feature extraction: Extract useful features from preprocessed data to describe students' learning behavior and status.

4. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 3 is characterized in that: The data processing and analysis module performs analysis in the following specific manner: Machine learning algorithms and data analysis methods are used to conduct a comprehensive analysis of the extracted feature vectors to establish a mapping relationship between students' learning behavior and their knowledge mastery. A neural network model is used to train students' feature vectors to obtain a prediction model, which predicts the students' mastery of each knowledge point based on their current characteristics.

5. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 3 is characterized in that: When performing feature extraction, the data processing and analysis module specifically includes: Operation trajectory feature extraction: Calculate the operation speed feature. The operation speed is measured by the number of operations per unit time. Let the number of operations within a time interval Δt be N Δt , operating speed Extract the operation frequency feature, which is the number of specific types of operations per unit time; the operation path complexity feature is measured by calculating the total length of the operation trajectory and the number of turning points. The Euclidean distance is used to calculate the distance between adjacent operation points, and the total length of the operation trajectory L is accumulated. traj , the number of turning points N turn Determine by judging the change of operation direction; these operation trajectory features are combined into a feature vector F traj ={V op ,N Δt ,L traj ,N turn }; Residence time feature extraction: Extract the average residence time, the longest residence time, and the shortest residence time features from the residence time data. Suppose the residence time of students in the jth virtual scene or teaching element is The average residence time Where m is the total number of virtual scenes or teaching elements, and the maximum stay time is Minimum stay time These residence time features are combined into a feature vector Error frequency feature extraction: Extract error rate and error concentration features from error frequency data. The error rate is calculated by the ratio of the number of errors to the total number of operations. Let the total number of operations be N. total , the number of errors is N err , then the error rate Error concentration is measured by calculating the frequency of errors occurring in certain key knowledge points or operation links. Let the number of errors related to key knowledge points be Error concentration These error frequency features are combined into a feature vector F err ={R err ,C err }.

6. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 5 is characterized in that: The data processing and analysis module extracts the operation trajectory feature vector F traj , residence time feature vector F stay and the error frequency eigenvector F err Merge into a comprehensive feature vector F = {F traj ,F stay ,F err }; Use machine learning algorithms to train comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery.

7. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 6 is characterized in that: During the comprehensive analysis process, the data processing and analysis module uses the neural network model to take the comprehensive feature vector as the input of the input layer, and finally obtains the predicted value of the student's mastery of each knowledge point in the output layer through nonlinear transformation and weight adjustment of the hidden layer.

8. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 7 is characterized in that: The learning path generation process of the learning path generation module includes knowledge point evaluation and learning path planning, wherein the knowledge point evaluation is: based on the student's mastery level Kl of each knowledge point obtained by the prediction model M, the knowledge points are divided into three levels: mastered, partially mastered, and not mastered; let the total number of knowledge points be L, for the lth knowledge point, if K l ≥θ1, it is considered that the student has mastered the knowledge point; if θ2≤K l <θ1, then the student is considered to have partially mastered the knowledge point; if K l <θ2, it is considered that the student has not mastered the knowledge point; among them, θ1 and θ2 are preset thresholds.

9. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to claim 8 is characterized in that: The learning path planning of the learning path generation module is as follows: according to the evaluation results of the knowledge points, combined with the correlation and difficulty of the teaching resources, a personalized learning path is planned for the students; let the total number of resources in the teaching resource library be R, and the correlation between the rth resource and the lth knowledge point be A rl , the difficulty of the resource is D r , then the learning path planning is expressed as an optimization problem: x r ∈{0,1} Among them, x r is the decision variable, indicating whether to select the rth teaching resource; is the target mastery level, that is, the mastery level of the lth knowledge point that students need to achieve; the constraint condition indicates that the selected teaching resources should be able to meet the students' learning needs for the knowledge points that have not been mastered; by solving this optimization problem, the optimal learning path for students is obtained.

10. The adaptive optimization system for virtual simulation teaching based on multimodal behavior recognition according to any one of claims 1 to 9, characterized in that: The optimization method of the optimization system comprises the following steps: S1: Collect operation trajectory data, dwell time data and error frequency data; S2: Clean and normalize the collected operation trajectory data, dwell time data, and error frequency data; S3: Extract features from the operation trajectory data (such as calculation operation speed features, operation frequency features, and operation path complexity features) to describe the student's operation behavior, and combine these operation trajectory features into an operation trajectory feature vector; S4: extract the average residence time, the longest residence time, and the shortest residence time features from the residence time data, and form these residence time features into a residence time feature vector; S5: extracting error rate and error concentration features from the error frequency data, and combining these error frequency features into an error frequency feature vector. When performing comprehensive analysis, the data processing and analysis module combines the extracted operation trajectory feature vector, dwell time feature vector, and error frequency feature vector into a comprehensive feature vector. S6: Use machine learning algorithms to train comprehensive feature vectors and establish a mapping relationship model between students' learning behavior and knowledge mastery. This is used to analyze students' learning behavior data in real time and predict their mastery of knowledge points. S7: Based on the predicted value of the knowledge point mastery obtained by the comprehensive analysis module, the knowledge points are divided into three levels: mastered, partially mastered, and not mastered; S8: Based on the evaluation results of knowledge points and combined with the resource information in the teaching resource library, a personalized learning path is planned for students; S9: According to the learning path determined by the learning path generation module, the corresponding teaching resources are matched from the teaching resource library.

Images