Intelligent education teaching evaluation method, system and equipment

Abstract

The invention discloses an intelligent education teaching evaluation method, system and device, and the method comprises the steps: obtaining first teaching data and first test data, analyzing the first teaching data and the first test data, and obtaining the first mapping of the first test data and the first teaching data; obtaining first monitoring data corresponding to the first teaching data according to the first teaching data; obtaining a second mapping of the first test data and the first monitoring data according to the first teaching data, the first monitoring data and the first mapping; obtaining a first test result according to the first test data; according to the score condition of each question in the first test result, obtaining teaching evaluation of each monitoring data slice in the first monitoring data; establishing a teaching evaluation model; and inputting to-be-evaluated monitoring data according to the teaching evaluation model, and obtaining an evaluation result corresponding to the to-be-evaluated monitoring data. The problems that an existing teaching evaluation scheme is prone to over-fitting, and output evaluation is prone to being unbiased are at least solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an intelligent education and teaching evaluation method, system and equipment. Background Art

[0002] With the rapid development of information technology, the application of artificial intelligence (AI) in various fields has gradually deepened, especially in the field of education. The widespread application of artificial intelligence technology has provided new opportunities for the innovation of education models and the improvement of education quality. In recent years, teaching evaluation, as an important part of the education process, has received more and more attention. Traditional teaching evaluation mainly relies on manual scoring and subjective judgment. The evaluation process is inefficient and it is difficult to fully and accurately reflect the learning status of students and the teaching quality of teachers. Especially in a large-scale education environment, the limitations of manual evaluation are more prominent.

[0003] Traditional teaching evaluation methods usually include written tests, homework, class participation and other indicators. The evaluation subjects are mostly teachers, and the evaluation standards and methods are often subjective and inconsistent. With the continuous expansion of the scale of education, the diversification of evaluation objects and the complexity of evaluation content, traditional evaluation methods face huge challenges. At present, the application of artificial intelligence has gradually penetrated into the field of classroom teaching quality evaluation, including the use of computer vision technology, which can conduct real-time analysis of teachers' teaching process through video monitoring, automatically identify teachers' teaching behavior, the communication of teaching content, and students' classroom participation and other indicators, and conduct teaching quality evaluation.

[0004] At present, the schemes for teaching evaluation using image recognition generally rely on posture recognition and expression recognition to evaluate the status of teachers and / or students in class, and evaluate education and teaching based on the status of teachers and / or students in class. However, the models used in the above schemes are prone to overfitting, and the teacher's teaching style is also likely to affect the final result. For example, teachers with a serious classroom atmosphere generally receive higher evaluations than teachers with an active classroom atmosphere. But in fact, teachers with an active classroom atmosphere may have better teaching effects. Evaluations based on the output of AI models are prone to bias. Summary of the invention

[0005] The present invention provides an intelligent education and teaching evaluation method, system and equipment, and provides a teaching effect evaluation scheme based on artificial intelligence, which at least solves the current scheme of using image recognition for teaching evaluation, which generally relies on posture recognition and expression recognition to evaluate the status of teachers and / or students in class, and evaluates education and teaching according to the status of teachers and / or students in class. However, the model used in the above scheme is prone to overfitting, and the evaluation based on the output of the AI ​​model is prone to bias.

[0006] This application provides an intelligent education and teaching evaluation method, including: Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; According to the first teaching data, obtaining first monitoring data corresponding to the first teaching data; According to the first teaching data, the first monitoring data and the first mapping, obtaining a second mapping between the first test data and the first monitoring data, wherein the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data; Conducting a test on the target to be tested according to the first test data to obtain a first test result; According to the score of each question in the first test result, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data; Establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

[0007] Optionally, the acquiring the first teaching data and the first test data, parsing the first teaching data and the first test data, and obtaining a first mapping between the first test data and the first teaching data includes: Acquire first test data, parse the first test data to obtain a first question, the first question belongs to the first test data; According to the first question, obtaining a first analysis corresponding to the first question; According to the first analysis, obtaining at least one first embedding vector of the first analysis; Acquire first teaching data, the first teaching data including at least one of text data, voice data and image data, parse the first teaching data, obtain a first teaching period, the first teaching period belongs to the first teaching data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector; Obtaining a mapping relationship between the first topic and the first teaching period according to the at least one first embedding vector and the second embedding vector; According to the mapping relationship between the first question and the first teaching period, a first mapping between the first test data and the first teaching data is obtained.

[0008] Optionally, the first teaching data at least includes voice data; The step of parsing the first teaching data to obtain a first teaching period includes: According to the voice data, obtaining ASR data corresponding to the voice data; According to the ASR data, performing semantic analysis on the ASR data, and slicing the ASR data according to the semantic analysis result to obtain ASR slice data; According to the ASR slice data, obtaining slice data of the first teaching data; A first teaching period is obtained according to the slice data of the first teaching data.

[0009] Optionally, the first teaching data includes text data, voice data and image data; The extracting, according to the first teaching period, features of the data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector includes: According to the text data, extract context features of the text data using a pre-trained BERT model to obtain text features; According to the speech data, obtaining ASR data corresponding to the speech data, extracting context features of the ASR data using a pre-trained BERT model, and obtaining speech features; Extracting image features using an image recognition model according to the image data to obtain a first image feature; According to the image data, extract text in the image using OCR, and extract context features of the text in the image using a pre-trained BERT model to obtain a second image feature; According to the text features, the voice features, the first image features and the second image features, a multimodal fusion model is used to map the text features, the voice features, the first image features and the second image features to a unified embedding space to obtain a second embedding vector.

[0010] Optionally, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data according to the score of each question in the first test result includes: According to the second mapping, obtaining questions in the first test data corresponding to each monitoring data slice; Obtaining scoring results for each monitoring data slice in the first monitoring data according to the scores of each question in the first test data corresponding to each monitoring data slice and the proportion of the content involved in each monitoring data slice in each question; The scoring results are standardized to obtain teaching evaluations of each monitoring data slice in the first monitoring data.

[0011] Optionally, the step of establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set includes: Establishing an initial fusion model, wherein the initial fusion model is configured to include a posture recognition sub-model, an expression analysis sub-model, and a temporal fusion sub-model, and wherein the initial fusion model is configured to take video data as input and take evaluation data as output; According to the data set, a training set and a test set are obtained; The initial fusion model is trained according to the training set and the test set to obtain a teaching evaluation model.

[0012] Optionally, inputting the monitoring data to be evaluated according to the teaching evaluation model to obtain the evaluation result corresponding to the monitoring data to be evaluated includes: According to the monitoring data to be evaluated, the slice data to be evaluated is obtained, wherein the slice data to be evaluated belongs to the monitoring data to be evaluated; According to the teaching evaluation model, the slice data to be evaluated is input to obtain the evaluation of each slice data to be evaluated; According to the evaluation of each slice data to be evaluated, an evaluation result corresponding to the evaluated monitoring data is obtained.

[0013] In another aspect, an intelligent education and teaching evaluation system includes a model management platform and an evaluation platform; The model management platform is configured to: Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; According to the first teaching data, obtaining first monitoring data corresponding to the first teaching data; According to the first teaching data, the first monitoring data and the first mapping, obtaining a second mapping between the first test data and the first monitoring data, wherein the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data; Conducting a test on the target to be tested according to the first test data to obtain a first test result; According to the score of each question in the first test result, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data; Establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; The model evaluation platform is configured to: According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

[0014] Optionally, a data collection platform is also included; The data collection platform is configured to collect and store at least one of the first teaching data, the first test data, the first monitoring data and the monitoring data to be evaluated.

[0015] On the other hand, an embodiment of the present application further provides a device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above method.

[0016] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the above method.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention provides an intelligent education and teaching evaluation method, system and device, comprising obtaining first teaching data and first test data, parsing the first teaching data and the first test data, obtaining a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; obtaining first monitoring data corresponding to the first teaching data according to the first teaching data; obtaining a second mapping between the first test data and the first monitoring data according to the first teaching data, the first monitoring data and the first mapping, wherein the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data; testing a target to be tested according to the first test data to obtain a first test result; obtaining a teaching evaluation of each monitoring data slice in the first monitoring data according to the score of each question in the first test result; establishing a teaching evaluation model according to the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; inputting the monitoring data to be evaluated according to the teaching evaluation model to obtain the evaluation result corresponding to the monitoring data to be evaluated. At least it solves the current solution of using image recognition for teaching evaluation, which generally relies on posture recognition and expression recognition to evaluate the status of teachers and / or students in class, and evaluates education and teaching based on the status of teachers and / or students in class. However, the model used in the above solution is prone to overfitting, and the evaluation based on the output of the AI ​​model is prone to bias. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation of the present application or the technical solution in the prior art, the following is a brief introduction to the drawings required for the specific implementation or the prior art description. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0019] Figure 1 A flowchart of an intelligent education and teaching evaluation method in this application; Figure 2 It is a structural schematic diagram of a device in this application; Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Example 1 like Figure 1 As shown, an intelligent education and teaching evaluation method includes: S1. Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data.

[0024] Specifically, the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data.

[0025] Optionally, the first teaching data is history classroom data, including one or more of lesson plans, audio data and video data corresponding to each lesson, wherein the audio data at least includes audio data corresponding to the teacher, and the video data at least includes video data corresponding to the blackboard writing.

[0026] Optionally, the first test data may include test questions, explanations of test questions, or test questions and explanations of test questions specifically used for evaluating teaching quality.

[0027] The purpose of this step is mainly to associate the test results with the teaching data, so as to use the test results to quantitatively evaluate the teaching quality and make the final education and teaching evaluation results more in line with objective results.

[0028] S2. Obtain first monitoring data corresponding to the first teaching data based on the first teaching data.

[0029] Optionally, the first monitoring data may be video data acquired by a camera, or may include physiological data such as a user's heart rate acquired by a wearable device such as a children's watch.

[0030] S3. Obtain a second mapping of the first test data and the first monitoring data based on the first teaching data, the first monitoring data and the first mapping.

[0031] Specifically, the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data.

[0032] Optionally, the monitoring data slice is data obtained by slicing the first monitoring data according to time, and each monitoring data slice corresponds to a time period in the first monitoring data.

[0033] By using the above steps, we can establish the relationship between the knowledge points of each question and the monitoring data slices. The purpose is to make the correspondence between the characteristics of the monitoring data and the evaluation results clearer and improve the interpretability of the model.

[0034] S4. Test the target to be tested according to the first test data to obtain a first test result.

[0035] S5. According to the score of each question in the first test result, obtain the teaching evaluation of each monitoring data slice in the first monitoring data.

[0036] Optionally, according to the scores of the questions corresponding to each monitoring data slice, a teaching evaluation of the corresponding monitoring data slice is obtained. The higher the score rate of the corresponding question, the higher the teaching evaluation of the monitoring data slice.

[0037] S6. Establish a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set.

[0038] Specifically, the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output.

[0039] Optionally, the base of the teaching evaluation model may be selected to include a base including a posture recognition sub-model, an expression analysis sub-model and a timing fusion sub-model.

[0040] S7. According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

[0041] By adopting the above scheme, the association between the test or examination and the monitoring data slice can be established, which can more accurately locate the behavior or other data that affects education and teaching. The evaluation results are more objective and more interpretable than the existing scheme that relies on the indicator system or expert evaluation results to establish the evaluation model, and can accurately evaluate the effect of education and teaching. At least it solves the current scheme of using image recognition for teaching evaluation, which generally relies on posture recognition and expression recognition to evaluate the status of teachers and / or students in class, and evaluates education and teaching based on the status of teachers and / or students in class, but the model used in the above scheme is prone to overfitting, and the evaluation based on the output of the AI ​​model is prone to bias.

[0042] Example 2 This embodiment is based on the first embodiment, and is an intelligent education and teaching evaluation method, including: S1. Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data.

[0043] Specifically, the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data.

[0044] Optionally, acquiring the first teaching data and the first test data, parsing the first teaching data and the first test data, and obtaining a first mapping between the first test data and the first teaching data includes: Acquire first test data, parse the first test data to obtain a first question, the first question belongs to the first test data; According to the first question, a first analysis corresponding to the first question is obtained; According to the first analysis, obtaining at least one first embedding vector of the first analysis; Acquire first teaching data, the first teaching data including at least one of text data, voice data and image data, parse the first teaching data, obtain a first teaching period, the first teaching period belongs to the first teaching data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector; Obtaining a mapping relationship between a first topic and a first teaching period according to at least one first embedding vector and a second embedding vector; According to the mapping relationship between the first question and the first teaching period, a first mapping between the first test data and the first teaching data is obtained.

[0045] Optionally, according to the first analysis, based on methods such as named entity recognition and keyword extraction, at least one first embedding vector of the first analysis is obtained, that is, each knowledge point corresponds to a first embedding vector. Similarly, according to the first teaching period, based on methods such as named entity recognition and keyword extraction, features of data corresponding to the first teaching period are extracted and feature fusion is performed to obtain at least one second embedding vector. The method for obtaining a mapping relationship between a first topic and a first teaching period according to at least one first embedding vector and a second embedding vector includes: It is determined whether the at least one first embedding vector and the at least one second embedding vector include an embedding vector group whose similarity is greater than a threshold; if included, the first topic is associated with the first teaching period.

[0046] Optionally, the first teaching data at least includes voice data; Parsing the first teaching data to obtain the first teaching period includes: According to the voice data, obtaining ASR data corresponding to the voice data; According to the ASR data, semantic analysis is performed on the ASR data, and the ASR data is sliced ​​according to the semantic analysis result to obtain ASR slice data; According to the ASR slice data, slice data of the first teaching data is obtained; A first teaching period is obtained according to the sliced ​​data of the first teaching data.

[0047] Optionally, the above solution can be implemented by the following pseudo code: S101. Install necessary libraries, such as SpeechRecognition for speech recognition, pyaudio for audio input, and transformers for the BERT model. S102. Import necessary libraries, such as the speech_recognition library for speech recognition, BertTokenizer and BertModel in the transformers library for text feature extraction, and torch for tensor operations. S103, initializing a speech recognizer, such as creating a Recognizer object to perform a speech recognition task; S104, loading an audio file and converting it to text, including defining a path of the audio file, opening the audio file using AudioFile, reading audio data through a speech recognizer, and converting the audio to text using Google's ASR engine; S105, extracting text features using BERT, including loading the BERT pre-trained model and vocabulary (BertTokenizer and BertModel), segmenting and encoding the text obtained by audio conversion, extracting text features using the BERT model, and using the output of the first token of the last hidden layer of the model as the text embedding vector; S106: Output text embedding features, including the extracted text embedding features and printing out the shape of the embedding vector.

[0048] Optionally, the first teaching data includes text data, voice data and image data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector includes: Based on the text data, use the pre-trained BERT model to extract the contextual features of the text data and obtain text features; According to the speech data, ASR data corresponding to the speech data is obtained, and the context features of the ASR data are extracted using the pre-trained BERT model to obtain speech features; Extracting image features using an image recognition model according to the image data to obtain a first image feature; According to the image data, use OCR to extract the text in the image, use the pre-trained BERT model to extract the context features of the text in the image, and obtain the second image features; According to the text features, the speech features, the first image features and the second image features, a multimodal fusion model is used to map the text features, the speech features, the first image features and the second image features to a unified embedding space to obtain a second embedding vector.

[0049] Optionally, the multimodal fusion model adopts the CLIP model, and after mapping the features from text, audio and image to a unified embedding space, splicing fusion or weighted fusion can be used for fusion. In this embodiment, weighted fusion is preferably used for fusion, wherein the weight of the language feature is greater than or equal to the weight of the text feature is greater than or equal to the weight of the first image feature is greater than or equal to the weight of the second image feature.

[0050] S2. Obtain first monitoring data corresponding to the first teaching data based on the first teaching data.

[0051] S3. Obtain a second mapping of the first test data and the first monitoring data based on the first teaching data, the first monitoring data and the first mapping.

[0052] Specifically, the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data.

[0053] S4. Test the target to be tested according to the first test data to obtain a first test result.

[0054] S5. According to the score of each question in the first test result, obtain the teaching evaluation of each monitoring data slice in the first monitoring data.

[0055] Optionally, obtaining teaching evaluations of each monitoring data slice in the first monitoring data according to the score of each question in the first test result includes: According to the second mapping, obtaining the questions in the first test data corresponding to each monitoring data slice; Obtaining scoring results for each monitoring data slice in the first monitoring data according to the scores of each question in the first test data corresponding to each monitoring data slice and the proportion of the content involved in each monitoring data slice in each question; The scoring results are standardized to obtain the teaching evaluation of each monitoring data slice in the first monitoring data.

[0056] Optionally, obtaining the scoring results of each monitoring data slice in the first monitoring data according to the scores of each question in the first test data corresponding to each monitoring data slice and the proportion of the content involved in each monitoring data slice in each question includes: Obtain at least one first slice topic corresponding to the first monitoring data slice, the first monitoring data slice belonging to the monitoring data slice, and obtain knowledge points corresponding to each first slice topic according to the first slice topic; Obtaining the importance of the first monitoring data slice relative to the first slice title according to the total amount of knowledge points corresponding to the first slice title and the number of knowledge points corresponding to the first monitoring data slice; According to the importance of the first monitoring data slice relative to all the first slice questions, the weight of each first slice question is obtained; the score rate of the question is used to obtain the scoring result of the first monitoring data slice; According to the scoring result of the first monitoring data slice, the scoring results of each monitoring data slice in the first monitoring data are obtained.

[0057] Optionally, the scoring result of the first monitoring data slice is calculated using the following formula:

[0058] in, P 1 represents the scoring result of the first monitoring data slice, N i Representative i The weight of the first slice question, D i Representative i The score rate of the first slice questions,n is the total number of questions in the first slice.

[0059] Optional, i The weight of the first slice question is equal to the i The number of knowledge points associated with the first slice of questions and the first monitoring data slice divided by the i The total number of knowledge points in the first slice of questions.

[0060] The above scheme provides a solution to improve the teaching evaluation accuracy of each monitoring data slice. The core of this education and teaching evaluation method lies in a more accurate and objective evaluation of each monitoring data slice. The above method can greatly improve the accuracy of this scheme and solve the problem of poor model interpretability and robustness caused by directly training the model through test results and a whole section of monitoring data.

[0061] S6. Establish a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set.

[0062] Specifically, the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output.

[0063] Optionally, a teaching evaluation model is established based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, including: Establishing an initial fusion model, the initial fusion model is configured to include a posture recognition sub-model, an expression analysis sub-model and a time series fusion sub-model, and the initial fusion model is configured to take video data as input and take evaluation data as output; According to the data set, obtain the training set and test set; The initial fusion model is trained according to the training set and the test set to obtain the teaching evaluation model.

[0064] Optionally, the overall architecture of the initial fusion model is configured to include a video frame extraction and preprocessing module, a gesture recognition sub-model, an expression analysis sub-model, and a temporal fusion sub-model; The video frame extraction and preprocessing module is configured to: decompose the video into each frame image and perform necessary preprocessing on each frame, such as resizing, normalization, etc.; The posture recognition sub-model is configured to use OpenPose or HRNet for posture estimation, identify the body key points of the characters in the video frames, and extract the posture information of the characters from each frame.

[0065] The expression analysis submodel is configured to perform facial expression analysis using VGG-Face or FACS, extract the facial expression of the character from each frame, and infer its emotional state.

[0066] The time series fusion sub-model is configured as follows: LSTM or convolutional neural network is used for time series modeling to fuse the spatial features (posture, expression) and temporal features (behavior and emotional state that changes over time) of video frames, and then the outputs of each sub-model are fused through weighted averaging, fusion network or multi-layer perceptron (MLP), and the results of posture estimation and expression analysis are combined with the output of the time series model to generate the final score.

[0067] Optionally, the initial fusion model is trained according to the training set and the test set to obtain a teaching evaluation model, including: S601, data preprocessing and data enhancement; S602, training the pose estimation sub-model, using the video data with the key point positions marked to train OpenPose or HRNet, performing regression training on the key points in each frame, and optimizing the model to minimize the key point prediction error; S603, training a facial expression analysis sub-model, using a data set containing facial expression labels to train a facial expression recognition model, extracting facial features from each frame, and analyzing the emotion category through CNN; S604, training of the time series fusion sub-model, inputting the features extracted from the pose estimation and facial expression analysis into LSTM or CNN, training the network to capture the temporal dynamic changes in the video, performing time series training on each video, predicting the emotional state at each moment, fusing the output results of each sub-model, and training a simple fully connected network to generate the final score; S605, loss function and optimization, use MSE loss function or cross entropy loss function, use Adam optimizer to optimize model training, and dynamically adjust the learning rate.

[0068] S7. According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

[0069] Optionally, according to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated, including: According to the monitoring data to be evaluated, slice data to be evaluated is obtained, and the slice data to be evaluated belongs to the monitoring data to be evaluated; According to the teaching evaluation model, the slice data to be evaluated is input to obtain the evaluation of each slice data to be evaluated; According to the evaluation of each slice data to be evaluated, an evaluation result corresponding to the evaluated monitoring data is obtained.

[0070] By adopting the above scheme, on the basis of the scheme in Example 1, each step is optimized to further improve the accuracy and interpretability of the evaluation model. This further solves the problem that the current scheme of using image recognition for teaching evaluation generally relies on posture recognition and expression recognition to evaluate the status of teachers and / or students in class, and evaluates education and teaching according to the status of teachers and / or students in class, but the model used in the above scheme is prone to overfitting, and the evaluation based on the output of the AI ​​model is prone to bias.

[0071] Example 3 An intelligent education and teaching evaluation system, including a model management platform and an evaluation platform; The model management platform is configured to: Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; According to the first teaching data, obtaining first monitoring data corresponding to the first teaching data; According to the first teaching data, the first monitoring data and the first mapping, a second mapping of the first test data and the first monitoring data is obtained, wherein the second mapping is configured as a mapping of each question in the first test data and each monitoring data slice in the first monitoring data; According to the first test data, the target to be tested is tested to obtain a first test result; According to the score of each question in the first test result, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data; Establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; The model evaluation platform is configured as follows: According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

[0072] Optionally, a data collection platform is also included; The data collection platform is configured to collect and store at least one of the first teaching data, the first test data, the first monitoring data and the monitoring data to be evaluated.

[0073] Optionally, acquiring the first teaching data and the first test data, parsing the first teaching data and the first test data, and obtaining a first mapping between the first test data and the first teaching data includes: Acquire first test data, parse the first test data to obtain a first question, the first question belongs to the first test data; According to the first question, a first analysis corresponding to the first question is obtained; According to the first analysis, obtaining at least one first embedding vector of the first analysis; Acquire first teaching data, the first teaching data including at least one of text data, voice data and image data, parse the first teaching data, obtain a first teaching period, the first teaching period belongs to the first teaching data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector; Obtaining a mapping relationship between a first topic and a first teaching period according to at least one first embedding vector and a second embedding vector; According to the mapping relationship between the first question and the first teaching period, a first mapping between the first test data and the first teaching data is obtained.

[0074] Optionally, the first teaching data at least includes voice data; Parsing the first teaching data to obtain the first teaching period includes: According to the voice data, obtaining ASR data corresponding to the voice data; According to the ASR data, semantic analysis is performed on the ASR data, and the ASR data is sliced ​​according to the semantic analysis result to obtain ASR slice data; According to the ASR slice data, slice data of the first teaching data is obtained; A first teaching period is obtained according to the sliced ​​data of the first teaching data.

[0075] Optionally, the first teaching data includes text data, voice data and image data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector includes: Based on the text data, use the pre-trained BERT model to extract the contextual features of the text data and obtain text features; According to the speech data, ASR data corresponding to the speech data is obtained, and the context features of the ASR data are extracted using the pre-trained BERT model to obtain speech features; Extracting image features using an image recognition model according to the image data to obtain a first image feature; According to the image data, use OCR to extract the text in the image, use the pre-trained BERT model to extract the context features of the text in the image, and obtain the second image features; According to the text features, the speech features, the first image features and the second image features, a multimodal fusion model is used to map the text features, the speech features, the first image features and the second image features to a unified embedding space to obtain a second embedding vector.

[0076] Optionally, obtaining teaching evaluations of each monitoring data slice in the first monitoring data according to the score of each question in the first test result includes: According to the second mapping, obtaining the questions in the first test data corresponding to each monitoring data slice; Obtaining scoring results for each monitoring data slice in the first monitoring data according to the scores of each question in the first test data corresponding to each monitoring data slice and the proportion of the content involved in each monitoring data slice in each question; The scoring results are standardized to obtain the teaching evaluation of each monitoring data slice in the first monitoring data.

[0077] Optionally, a teaching evaluation model is established based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, including: Establishing an initial fusion model, the initial fusion model is configured to include a posture recognition sub-model, an expression analysis sub-model and a time series fusion sub-model, and the initial fusion model is configured to take video data as input and take evaluation data as output; According to the data set, obtain the training set and test set; The initial fusion model is trained according to the training set and the test set to obtain the teaching evaluation model.

[0078] Optionally, according to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated, including: According to the monitoring data to be evaluated, slice data to be evaluated is obtained, and the slice data to be evaluated belongs to the monitoring data to be evaluated; According to the teaching evaluation model, the slice data to be evaluated is input to obtain the evaluation of each slice data to be evaluated; According to the evaluation of each slice data to be evaluated, an evaluation result corresponding to the evaluated monitoring data is obtained.

[0079] Example 4 This embodiment provides a device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0080] Specifically, Figure 2 As shown, Figure 2This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application. The device is an electronic device and may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0081] Those skilled in the art will appreciate that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0082] like Figure 2 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an application program for implementing an intelligent education and teaching evaluation method.

[0083] exist Figure 2 In the electronic device shown, the network interface 103 is mainly used for data communication with a network server; the user interface 104 is mainly used for data interaction with a user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the application program stored in the memory 105 for implementing an intelligent education and teaching evaluation method through the processor 101 to implement the above method.

[0084] Example 5 This embodiment provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement any of the above methods.

[0085] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0086] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0089] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions to enable a device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0091] The above are only preferred embodiments of the present disclosure. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the present disclosure.

Claims

1. An intelligent education and teaching evaluation method, characterized in that: include: Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; According to the first teaching data, obtaining first monitoring data corresponding to the first teaching data; According to the first teaching data, the first monitoring data and the first mapping, obtaining a second mapping between the first test data and the first monitoring data, wherein the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data; Conducting a test on the target to be tested according to the first test data to obtain a first test result; According to the score of each question in the first test result, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data; Establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

2. An intelligent education and teaching evaluation method according to claim 1, characterized in that: The acquiring of the first teaching data and the first test data, parsing the first teaching data and the first test data, and obtaining a first mapping between the first test data and the first teaching data includes: Acquire first test data, parse the first test data to obtain a first question, the first question belongs to the first test data; According to the first question, obtaining a first analysis corresponding to the first question; According to the first analysis, obtaining at least one first embedding vector of the first analysis; Acquire first teaching data, the first teaching data including at least one of text data, voice data and image data, parse the first teaching data, obtain a first teaching period, the first teaching period belongs to the first teaching data; According to the first teaching period, extracting features of data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector; Obtaining a mapping relationship between the first topic and the first teaching period according to the at least one first embedding vector and the second embedding vector; According to the mapping relationship between the first question and the first teaching period, a first mapping between the first test data and the first teaching data is obtained.

3. An intelligent education and teaching evaluation method according to claim 2, characterized in that: The first teaching data at least includes voice data; The step of parsing the first teaching data to obtain a first teaching period includes: According to the voice data, obtaining ASR data corresponding to the voice data; According to the ASR data, performing semantic analysis on the ASR data, and slicing the ASR data according to the semantic analysis result to obtain ASR slice data; According to the ASR slice data, obtaining slice data of the first teaching data; A first teaching period is obtained according to the slice data of the first teaching data.

4. The intelligent education and teaching evaluation method according to claim 2 is characterized in that: The first teaching data includes text data, voice data and image data; The extracting, according to the first teaching period, features of the data corresponding to the first teaching period and performing feature fusion to obtain a second embedding vector includes: According to the text data, extract context features of the text data using a pre-trained BERT model to obtain text features; According to the speech data, obtaining ASR data corresponding to the speech data, extracting context features of the ASR data using a pre-trained BERT model, and obtaining speech features; Extracting image features using an image recognition model according to the image data to obtain a first image feature; According to the image data, extract text in the image using OCR, and extract context features of the text in the image using a pre-trained BERT model to obtain a second image feature; According to the text features, the voice features, the first image features and the second image features, a multimodal fusion model is used to map the text features, the voice features, the first image features and the second image features to a unified embedding space to obtain a second embedding vector.

5. The intelligent education and teaching evaluation method according to claim 1 is characterized in that: The step of obtaining the teaching evaluation of each monitoring data slice in the first monitoring data according to the score of each question in the first test result includes: According to the second mapping, obtaining questions in the first test data corresponding to each monitoring data slice; Obtaining scoring results for each monitoring data slice in the first monitoring data according to the scores of each question in the first test data corresponding to each monitoring data slice and the proportion of the content involved in each monitoring data slice in each question; The scoring results are standardized to obtain teaching evaluations of each monitoring data slice in the first monitoring data.

6. The intelligent education and teaching evaluation method according to claim 5, characterized in that: The step of establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set includes: Establishing an initial fusion model, wherein the initial fusion model is configured to include a posture recognition sub-model, an expression analysis sub-model, and a temporal fusion sub-model, and wherein the initial fusion model is configured to take video data as input and take evaluation data as output; According to the data set, a training set and a test set are obtained; The initial fusion model is trained according to the training set and the test set to obtain a teaching evaluation model.

7. The intelligent education and teaching evaluation method according to claim 5 is characterized in that: According to the teaching evaluation model, inputting the monitoring data to be evaluated, and obtaining the evaluation results corresponding to the monitoring data to be evaluated, comprises: According to the monitoring data to be evaluated, the slice data to be evaluated is obtained, wherein the slice data to be evaluated belongs to the monitoring data to be evaluated; According to the teaching evaluation model, the slice data to be evaluated is input to obtain the evaluation of each slice data to be evaluated; According to the evaluation of each slice data to be evaluated, an evaluation result corresponding to the evaluated monitoring data is obtained.

8. An intelligent education and teaching evaluation system, characterized in that: Includes model management platform and evaluation platform; The model management platform is configured to: Acquire first teaching data and first test data, parse the first teaching data and the first test data, and obtain a first mapping between the first test data and the first teaching data, wherein the first mapping is configured as a mapping between each question in the first test data and each teaching period in the first teaching data; According to the first teaching data, obtaining first monitoring data corresponding to the first teaching data; According to the first teaching data, the first monitoring data and the first mapping, obtaining a second mapping between the first test data and the first monitoring data, wherein the second mapping is configured as a mapping between each question in the first test data and each monitoring data slice in the first monitoring data; Conducting a test on the target to be tested according to the first test data to obtain a first test result; According to the score of each question in the first test result, obtaining the teaching evaluation of each monitoring data slice in the first monitoring data; Establishing a teaching evaluation model based on the teaching evaluation of each monitoring data slice in the first monitoring data as a data set, wherein the teaching evaluation model is configured to take the monitoring data slice as input and take the teaching evaluation corresponding to the monitoring data slice as output; The model evaluation platform is configured to: According to the teaching evaluation model, the monitoring data to be evaluated is input to obtain the evaluation results corresponding to the monitoring data to be evaluated.

9. An intelligent education and teaching evaluation system according to claim 8, characterized in that: It also includes a data collection platform; The data collection platform is configured to collect and store at least one of the first teaching data, the first test data, the first monitoring data and the monitoring data to be evaluated.

10. A device, characterized in that: The device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images