Classroom training learning quality evaluation system based on intelligent AI
By adopting intelligent AI systems in classroom training and combining multimodal behavior recognition technology, the problem of insufficient accuracy and robustness of classroom behavior recognition in the existing technology is solved, and the accuracy and robustness of student behavior analysis in multi-person classroom teaching scenarios is achieved, and teaching optimization is supported.
Patent Information
- Application Number
- CN202411593399.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art has problems with insufficient recognition accuracy and robustness in classroom behavior recognition, especially in multi-person classroom teaching scenarios. Traditional methods require special sensors and have large calculations, which are not suitable for general applications.
The classroom training learning quality evaluation system based on intelligent AI is adopted, combined with the data acquisition module, multimodal behavior recognition module, training closed-loop learning behavior analysis model module and data core processing module, and the multimodal behavior recognition algorithm that integrates human posture estimation and RGB characteristics, accurately analyzes and evaluates students' classroom behavior.
It realizes the accuracy and robustness of student behavior analysis in multi-person classroom teaching scenarios, can more comprehensively evaluate students' classroom learning quality, provide interpretable quantitative indicators, and support teachers to optimize teaching design and implement teaching interventions.
Smart Images
Figure CN119539582A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of learning management technology, and in particular to a classroom training learning quality assessment system based on intelligent AI. Background Art
[0002] Speech-based classroom behavior recognition method: refers to the technology that recognizes and classifies classroom behavior and emotional states by identifying and analyzing the sound features and patterns in speech signals. This method usually uses signal processing, machine learning, and deep learning technologies, combined with sound features in speech signals, such as sound frequency, rhythm, pitch, etc., to infer human classroom behavior. This type of method first needs to detect the active speech in the audio clip and then cluster it with the voice of people in the database to identify the identity of the speaker. However, due to technical limitations, the recognition accuracy and robustness still need to be improved. In addition, trainees do not talk continuously during the training process, so it has great limitations;
[0003] Visual classroom behavior recognition method: refers to the method of using classroom teaching videos to identify and analyze students' behaviors in the classroom through computer vision technology. This type of method can identify students' classroom behaviors, including raising hands, writing, student interaction, listening attentively, making phone calls, leaving seats, etc., and can comprehensively analyze the quality of students' classroom learning. According to the extracted features, the visual classroom behavior analysis can be classified into behavior recognition methods based on skeleton features, behavior recognition methods based on RGB features, and behavior recognition methods based on depth features. According to the input data classification, there are image-based, video-based, and depth sensor-based behavior recognition methods. Behavior recognition methods based on depth features and some skeleton features need to use special sensor technologies (such as depth cameras, inertial measurement units, etc.) to capture human distance and skeleton joint information, and then process and analyze these data through algorithms to identify different postures and actions, and then recognize behaviors through time series postures. These methods not only require special sensors, but also the amount of calculation increases exponentially with the increase of people in the scene, and are not suitable for multi-person classroom teaching scenarios; student classroom learning behavior analysis technology mainly uses modern technical means to monitor, record and analyze students' learning behaviors in the classroom. The main technologies include online learning platform software, sensor technology, audio and video analysis technology, etc. The online learning platform is not suitable for the current classroom training of this project, and sensor technology requires the use of wearable devices, which are not suitable for this project. In recent years, artificial intelligence technology has developed rapidly, making it possible to analyze classroom teaching behaviors through audio and video technology. Therefore, the development of a student training learning behavior analysis system that incorporates machine learning, computer vision and other algorithms and technologies will enable learning analysis to achieve higher analysis accuracy, and provide data support for teachers to grasp students' student engagement status, optimize teaching design, and implement teaching interventions. Based on the above problems, a classroom training learning quality assessment system based on intelligent AI is proposed to solve such problems. Summary of the invention
[0004] The purpose of the present invention is to solve the problems existing in the prior art and to propose a classroom training learning quality assessment system based on intelligent AI.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] The classroom training learning quality assessment system based on intelligent AI includes a data acquisition module, a multimodal behavior recognition module, a training closed-loop learning behavior analysis model module, a data core processing module and a learning quality evaluation standard. The data acquisition module includes an activity video monitoring unit and an activity audio recognition monitoring unit. The multimodal behavior recognition module establishes a multimodal student classroom interactive behavior analysis AI model to analyze and study student classroom concentration, making and receiving phone calls and other interactive behavior information.
[0007] The training closed-loop learning behavior analysis model module is a multimodal behavior recognition that integrates human posture estimation and RGB features. It estimates human posture by analyzing key points of the human body in an image or video, and integrates the key point information obtained by human posture estimation with RGB features.
[0008] The data core processing module specifically includes an inference engine module, a message middleware module, and a WEB management module;
[0009] The learning quality evaluation standard is based on the learning behavior obtained by the data acquisition module, the multimodal behavior recognition module and the training closed-loop learning behavior analysis model module to comprehensively evaluate the students' classroom learning quality.
[0010] Preferably, the data acquisition module is based on one or more of online learning platform software, sensor technology, and audio and video analysis technology.
[0011] Preferably, the behavior recognition method based on RGB features refers to a method for analyzing and identifying human posture using RGB color information in an image. This method uses computer vision and deep learning technology to identify and understand the target's behavior by extracting features such as color, texture and shape in the video, thereby achieving accurate recognition of human posture and movement.
[0012] Preferably, the features obtained by fusing the key point information obtained by the human posture estimation with the RGB features contain both the key information of the human posture and rich visual information such as color and texture, making the behavior recognition more accurate and robust.
[0013] Preferably, the human posture estimation method is established based on a skeleton-based model, a contour-based model and a volume-based model, and a multimodal classroom student behavior recognition algorithm that integrates human posture estimation and RGB features is used to achieve student behavior analysis for multi-person training and teaching classrooms.
[0014] Preferably, each of the inference engine module, the message middleware module, and the WEB management module is separately packaged into a container image and runs independently.
[0015] Preferably, the inference engine module is responsible for loading the model file and converting it into a processor-specific engine file, inferring the video stream, and pushing the detected behavior results to the message middleware module in real time. The inference results are encapsulated in JSON format and pushed to the MQ module.
[0016] Preferably, the WEB management module is planned to adopt a front-end and back-end separation architecture design.
[0017] Compared with the prior art, the present invention has the following beneficial effects:
[0018] 1. The present invention realizes student behavior analysis for multi-person training and teaching classrooms by integrating human posture estimation and RGB features into a multimodal classroom student behavior recognition algorithm. The integrated features contain both key information of human posture and rich visual information such as color and texture, making behavior recognition more accurate and robust.
[0019] 2. The present invention is based on the learning behavior obtained by the data acquisition module, the multimodal behavior recognition module and the training closed-loop learning behavior analysis model module, comprehensively evaluates the students' classroom learning quality, and counts the closed-loop learning quality of trainees within a training cycle. Through interpretable quantitative indicators, it provides data support for teachers to grasp the students' student engagement status and optimize teaching design. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a diagram of the classroom training learning quality assessment system based on intelligent AI proposed by the present invention;
[0021] Figure 2 This is a module diagram of the data core processing module in the intelligent AI-based classroom training learning quality assessment system proposed by the present invention. DETAILED DESCRIPTION
[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0023] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "top / bottom" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0024] Example, see Figure 1-2 , the classroom training learning quality assessment system based on intelligent AI includes a data acquisition module, a multimodal behavior recognition module, a training closed-loop learning behavior analysis model module, a data core processing module and a learning quality evaluation standard. The data acquisition module includes an activity video monitoring unit and an activity audio recognition monitoring unit. The multimodal behavior recognition module establishes a multimodal student classroom interactive behavior analysis AI model to analyze and study the student's classroom concentration, answering and making phone calls and other interactive behavior information;
[0025] Collect audio and video data of students’ classroom activities during a training cycle;
[0026] Establish a multi-modal AI model for analyzing students’ classroom interaction behaviors, and analyze and study students’ classroom concentration, making phone calls, and other interactive behaviors;
[0027] Establish a set of interactive behavior evaluation indicators, score the students' interactive behavior through platform indicators, and finally give comprehensive evaluation indicators of students' classroom interactive behavior during the entire training period, providing an effective way to improve students' classroom learning and teachers' classroom teaching quality.
[0028] The training closed-loop learning behavior analysis model module is a multimodal behavior recognition that integrates human posture estimation and RGB features. It estimates human posture by analyzing the key points of the human body in the image or video, and fuses the key point information obtained by human posture estimation with the RGB features.
[0029] The data core processing module specifically includes the inference engine module, the message middleware module, and the WEB management module;
[0030] The learning quality evaluation standard is based on the learning behavior obtained by the data acquisition module, multimodal behavior recognition module and training closed-loop learning behavior analysis model module to comprehensively evaluate the students' classroom learning quality.
[0031] The data acquisition module is based on one or more of online learning platform software, sensor technology, and audio and video analysis technology.
[0032] The behavior recognition method based on RGB features refers to a method of analyzing and identifying human postures using the RGB color information in an image. This method uses computer vision and deep learning technology to extract features such as color, texture, and shape in the video to identify and understand the target's behavior, thereby achieving accurate recognition of human postures and movements.
[0033] The features after the key point information obtained by human posture estimation is fused with the RGB features not only contain the key information of human posture, but also contain rich visual information such as color and texture, making behavior recognition more accurate and robust.
[0034] The human posture estimation method is established based on skeleton-based models, contour-based models and volume-based models. Through the multi-modal classroom student behavior recognition algorithm that integrates human posture estimation and RGB features, student behavior analysis for multi-person training teaching classrooms is realized.
[0035] Each module in the inference engine module, message middleware module, and web management module is packaged into a container image and runs independently;
[0036] The inference engine module is responsible for loading the model file and converting it into a processor-specific engine file, inferring the video stream, and pushing the detected behavior results to the message middleware module in real time. The inference results are encapsulated in JSON format and pushed to the MQ module.
[0037] The WEB management module is planned to adopt a front-end and back-end separation architecture design. The trainees' positions are fixed during the entire training period. It is necessary to bind the trainees and behaviors by selecting and setting the personnel positions in the interface. It is necessary to edit the scoring indicators of different behaviors on the page to then calculate the quality of trainees' learning in class.
[0038] The present invention realizes student behavior analysis for multi-person training and teaching classrooms through a multimodal classroom student behavior recognition algorithm that integrates human posture estimation and RGB features. The fused features contain both key information on human posture and rich visual information such as color and texture, making behavior recognition more accurate and robust. Computer vision and deep learning technology are usually used to identify and understand the target's behavior by extracting features such as color, texture and shape in the video, thereby achieving accurate recognition of human posture and action. Based on the learning behavior obtained by the data acquisition module, the multimodal behavior recognition module and the training closed-loop learning behavior analysis model module, the classroom learning quality of students is comprehensively evaluated, and the closed-loop learning quality of students within a training cycle is counted. Through interpretable quantitative indicators, data support is provided for teachers to grasp the student engagement status and optimize teaching design.
[0039] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. Classroom training learning quality assessment system based on intelligent AI, characterized by: It includes a data acquisition module, a multimodal behavior recognition module, a training closed-loop learning behavior analysis model module, a data core processing module and a learning quality evaluation standard. The data acquisition module includes an activity video monitoring unit and an activity audio recognition monitoring unit. The multimodal behavior recognition module establishes a multimodal student classroom interactive behavior analysis AI model to analyze and study student classroom concentration, answering and making phone calls and other interactive behavior information. The training closed-loop learning behavior analysis model module is a multimodal behavior recognition that integrates human posture estimation and RGB features. It estimates human posture by analyzing key points of the human body in an image or video, and integrates the key point information obtained by human posture estimation with RGB features. The data core processing module specifically includes an inference engine module, a message middleware module, and a WEB management module; The learning quality evaluation standard is based on the learning behavior obtained by the data acquisition module, the multimodal behavior recognition module and the training closed-loop learning behavior analysis model module to comprehensively evaluate the students' classroom learning quality.
2. The classroom training learning quality assessment system based on intelligent AI according to claim 1 is characterized in that: The data acquisition module is based on one or more of online learning platform software, sensor technology, and audio and video analysis technology.
3. The classroom training learning quality assessment system based on intelligent AI according to claim 1 is characterized in that: The RGB feature-based behavior recognition method refers to a method for analyzing and identifying human postures using RGB color information in an image. This method uses computer vision and deep learning technology to extract features such as color, texture, and shape in the video to identify and understand the target's behavior, thereby achieving accurate recognition of human postures and movements.
4. The classroom training learning quality assessment system based on intelligent AI according to claim 1 is characterized in that: The features obtained by fusing the key point information obtained by the human posture estimation with the RGB features contain not only the key information of the human posture, but also rich visual information such as color and texture, making the behavior recognition more accurate and robust.
5. The classroom training learning quality assessment system based on intelligent AI according to claim 1 is characterized in that: The human posture estimation method is established by using a skeleton-based model, a contour-based model and a volume-based model, and realizes student behavior analysis for multi-person training teaching classrooms through a multi-modal classroom student behavior recognition algorithm that integrates human posture estimation and RGB features.
6. The classroom training learning quality assessment system based on intelligent AI according to claim 1 is characterized in that: Each of the inference engine module, the message middleware module, and the WEB management module is separately packaged into a container image and runs independently.
7. The classroom training learning quality assessment system based on intelligent AI according to claim 6 is characterized in that: The inference engine module is responsible for loading the model file and converting it into a processor-specific engine file, inferring the video stream, and pushing the detected behavior results to the message middleware module in real time. The inference results are encapsulated in JSON format and pushed to the MQ module.
8. The classroom training learning quality assessment system based on intelligent AI according to claim 7 is characterized in that: The WEB management module is planned to adopt a front-end and back-end separation architecture design.