Classroom state detection system based on image processing

Abstract

The invention provides a classroom state detection system based on image processing, and the system comprises a video collection module which is used for installing and controlling a high-resolution camera in a classroom, and achieves the collection of a classroom panoramic video; the image processing module is used for carrying out preprocessing and feature extraction on the collected video, and particularly extracting a limb movement image of the student; the data analysis module is used for comparing the extracted body movement images with a negative movement image library, performing learning state judgment, analyzing facial expressions of the students by utilizing a fatigue detection algorithm, and recording and counting negative movement information of the students; and the result display module is used for displaying a learning behavior report of the student, a curve graph including the negative action frequency information and a detailed learning behavior report. According to the invention, intelligent upgrading of classroom teaching management is realized, teachers can find and intervene learning problems of students in time, and improvement of teaching quality and improvement of learning effects of the students are promoted.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of image processing, and in particular to a classroom status detection system based on image processing. Background Art

[0002] In the modern education system, monitoring and analysis of classroom status is of great significance for improving teaching quality, optimizing teaching strategies and promoting individual development of students.

[0003] Although existing research has made some progress in image processing, it still faces many challenges. First, the complexity of the classroom environment requires that the image processing algorithm must have high robustness and accuracy. For example, different lighting conditions, student density, and classroom layout may affect the accuracy of detection. Secondly, the issue of privacy protection is also an issue that cannot be ignored. How to reasonably use image data without infringing on students' privacy is an important factor that must be considered in technology development and practical applications. In addition, most of the existing research focuses on algorithm development and software implementation, and there is relatively little research on the overall architecture of the system, performance optimization, and user experience. Summary of the invention

[0004] The present invention mainly provides a classroom status detection system based on image processing to solve the technical problems raised in the above background technology.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A classroom status detection system based on image processing, comprising:

[0007] Video acquisition module, used to install and control high-resolution cameras in the classroom to acquire panoramic videos of the classroom;

[0008] The image processing module is used to pre-process and extract features from the collected videos, especially to extract images of students’ body movements;

[0009] The data analysis module is used to compare the extracted body movement images with the negative movement image library to judge the learning status, analyze the students' facial expressions using fatigue detection algorithms, and record and count the students' negative movement information;

[0010] The result display module is used to display the students' learning behavior report, including a curve chart of the number of negative action information and a detailed learning behavior report.

[0011] Furthermore, the image processing module preprocesses and extracts features from the collected video, specifically including the following steps:

[0012] Step 1: obtain the original image from the video stream captured by the camera, and perform pre-processing operations such as grayscale, denoising, and noise reduction on the image to improve the image quality;

[0013] Step 2: Input the preprocessed image obtained in step 1 into the deep learning model to extract useful features;

[0014] Step 3: Use the extracted features, combined with computer vision and machine learning technology, to build a feature matching and classification model, compare the body movement images with the images in the negative action image library, and thus judge the student's behavior status;

[0015] Step 4: Use deep learning algorithms to analyze students’ facial expressions to determine their fatigue levels and emotional states;

[0016] Step five, record and count the students' negative action information throughout the process so that the data analysis module can further analyze and evaluate the students' learning behavior.

[0017] Furthermore, the data analysis module compares the extracted body movement images with the negative movement image library to judge the learning status, and uses the fatigue detection algorithm to analyze the student's facial expression, while recording and counting the student's negative movement information. Specifically, the following steps are included:

[0018] Step 1: Based on the comparison results of the body movement images and the negative movement image library, the student's current learning behavior status is judged. The learning behavior status includes whether the student is focused, whether he is tired, and whether he has negative emotions;

[0019] Step 2: Use fatigue detection algorithms and emotion recognition technology to continuously monitor students’ fatigue levels and emotional changes so that timely measures can be taken to improve students’ learning experience;

[0020] Step 3: Analyze students’ learning behaviors, discover potential learning habits and problems, and provide students with personalized learning suggestions;

[0021] Step 4: Generate a detailed learning behavior report, including a graph of the number of negative actions, a score for the learning behavior, and possible problems and suggestions.

[0022] Furthermore, the high-resolution camera includes a camera body, a bracket connected to the top of the camera body and connected to the wall, the bottom end of the bracket is L-shaped, and the bottom end of the bracket is connected to a cleaning mechanism. In the present invention, the bracket provides support for the camera body, and the camera body is used to monitor students in the classroom.

[0023] Furthermore, the cleaning mechanism includes a support block connected to one side of the bottom of the bracket frame, the support block is provided with a heat dissipation hole, and a heat dissipation fan is connected to the hole body of the heat dissipation hole. In the present invention, the heat generated by the camera body is guided to be discharged through the heat dissipation fan, the heat dissipation fan is accommodated through the heat dissipation hole, and the hot air generated by the heat dissipation fan is guided to be discharged.

[0024] Furthermore, a heat dissipation metal block is connected to one side surface of the support block, and a cleaning cloth is connected to one side surface of the heat dissipation metal block. In the present invention, as the camera body rotates, the camera body is cleaned by the cleaning cloth, and the heat generated by the camera body is discharged through the heat dissipation metal block.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The classroom status detection method and system based on image processing proposed in the present invention realize accurate and automated monitoring and evaluation of students' classroom behavior and status, and significantly improve the intelligent level of teaching management. By adopting high-resolution cameras and advanced image processing technology, the system can accurately capture students' body movements and facial expressions, and then effectively identify negative learning behaviors and fatigue states, providing teachers with real-time and detailed student learning behavior data. At the same time, the system supports multi-camera collaborative work and distributed computing technology to ensure the comprehensiveness of monitoring and the efficiency of data processing. In addition, the self-learning ability of the system enables it to continuously optimize the recognition algorithm, improve the accuracy and stability of detection, and further enhance its practicality and reliability. In summary, the technical effect of the present invention is to realize the intelligent upgrade of classroom teaching management, help teachers to promptly discover and intervene in students' learning problems, and promote the improvement of teaching quality and the improvement of students' learning effects.

[0027] The present invention will be explained in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a system block diagram of the present invention;

[0029] Figure 2 It is a structural schematic diagram of the camera of the present invention;

[0030] Figure 3 It is a side view of the camera of the present invention.

[0031] In the figure: 1. Video acquisition module; 2. Image processing module; 3. Data analysis module; 4. Result display module; 5. High-resolution camera; 51. Camera body; 52. Bracket; 53. Cleaning mechanism; 531. Support block; 532. Heat dissipation hole; 533. Heat dissipation fan; 534. Heat dissipation metal block; 535. Cleaning cloth. DETAILED DESCRIPTION

[0032] To facilitate understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings, but the present invention can be implemented in different forms and is not limited to the embodiments described in the text. On the contrary, these embodiments are provided to make the content disclosed in the present invention more thorough and comprehensive.

[0033] It should be noted that when an element is referred to as being "fixed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this article are for illustrative purposes only.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0035] For examples, please refer to the attached Figure 1-3 , a classroom status detection system based on image processing, comprising:

[0036] The video acquisition module 1 is used to install and control a high-resolution camera 5 in the classroom to acquire a panoramic video of the classroom;

[0037] Image processing module 2, used for preprocessing and feature extraction of the collected video, especially extracting the students' body movement images;

[0038] Data analysis module 3, used to compare the extracted body movement images with the negative movement image library, judge the learning status, analyze the students' facial expressions using fatigue detection algorithms, and record and count the students' negative movement information;

[0039] The result display module 4 is used to display the student's learning behavior report, including a curve chart of the number of negative action information and a detailed learning behavior report.

[0040] Furthermore, the image processing module 2 performs preprocessing and feature extraction on the collected video, specifically comprising the following steps:

[0041] Step 1: obtain the original image from the video stream captured by the camera, and perform pre-processing operations such as grayscale, denoising, and noise reduction on the image to improve the image quality;

[0042] Step 2: Input the preprocessed image obtained in step 1 into the deep learning model to extract useful features;

[0043] Step 3: Use the extracted features, combined with computer vision and machine learning technology, to build a feature matching and classification model, compare the body movement images with the images in the negative action image library, and thus judge the student's behavior status;

[0044] Step 4: Use deep learning algorithms to analyze students’ facial expressions to determine their fatigue levels and emotional states;

[0045] Step 5: Record and count the students' negative action information throughout the process so that the data analysis module 3 can further analyze and evaluate the students' learning behavior.

[0046] Furthermore, the data analysis module 3 compares the extracted body movement images with the negative movement image library to judge the learning state, and uses the fatigue detection algorithm to analyze the student's facial expression, while recording and counting the student's negative movement information. Specifically, the following steps are included:

[0047] Step 1: Based on the comparison results of the body movement images and the negative movement image library, the student's current learning behavior status is judged. The learning behavior status includes whether the student is focused, whether he is tired, and whether he has negative emotions;

[0048] Step 2: Use fatigue detection algorithms and emotion recognition technology to continuously monitor students’ fatigue levels and emotional changes so that timely measures can be taken to improve students’ learning experience;

[0049] Step 3: Analyze students’ learning behaviors, discover potential learning habits and problems, and provide students with personalized learning suggestions;

[0050] Step 4: Generate a detailed learning behavior report, including a graph of the number of negative actions, a score for the learning behavior, and possible problems and suggestions.

[0051] Furthermore, the high-resolution camera 5 includes a camera body 51, a bracket 52 connected to the top of the camera body 51 and connected to the wall, the bottom end of the bracket 52 is L-shaped, and the bottom end of the bracket 52 is connected to a cleaning mechanism 53. In the present invention, the bracket 52 provides support for the camera body 51, and the camera body 51 is used to monitor students in the classroom.

[0052] Furthermore, the cleaning mechanism 53 includes a support block 531 connected to one side of the bottom of the bracket 52, and a heat dissipation hole 532 is provided on the block body of the support block 531. A heat dissipation fan 533 is connected to the hole body of the heat dissipation hole 532. In the present invention, the heat generated by the camera body 51 is guided to be discharged through the heat dissipation fan 533, and the heat dissipation fan 533 is accommodated by the heat dissipation hole 532, and the hot air generated by the heat dissipation fan 533 is guided to be discharged.

[0053] Furthermore, a heat dissipation metal block 534 is connected to one side surface of the support block 531, and a cleaning cloth 535 is connected to one side surface of the heat dissipation metal block 534. In the present invention, as the camera body 51 rotates, the camera body 51 is cleaned by the cleaning cloth 535, and the heat generated by the camera body 51 is discharged through the heat dissipation metal block 534.

[0054] During the preset time period, the camera sends shooting instructions to the shooting terminal at intervals to shoot a panoramic view of the classroom. The captured video is pre-processed, including denoising and contrast enhancement, to improve image quality.

[0055] Image processing technology is used to extract the body movement images of each student in the classroom panorama (i.e., region capture). These images can include posture information of key parts such as the head and hands.

[0056] The extracted body movement images are compared with the pre-stored negative movement image library to determine whether the student's movement is a negative movement (such as dozing off, playing with mobile phones, etc.).

[0057] Using specially developed fatigue detection algorithms (such as YOLOv3 combined with Dropblock strategy), students’ facial expressions, especially the eye and mouth areas, are analyzed to determine whether students show signs of fatigue. In the eye area, the frame difference method is used to detect pixel changes in moving targets to identify blinking movements. The advantage of the frame difference method is that it selects the previous frame or the previous few frames as the reference background and performs a differential operation on the current frame and the reference background, thus effectively solving the problem of real-time updating and reconstruction of the standard background image.

[0058] If the student status is negative, based on the coordinate information, the seat number information pre-associated with the coordinate information is identified, and the student information pre-associated with the seat number is obtained through the seat number information, including name, student number, etc.

[0059] Record the time point of each negative action image acquisition of each student within the preset time period, and count the number of negative actions. Generate a curve chart representing the number of negative actions of students based on the continuous preset time period. At the same time, combined with course information, count the number of negative actions of students in different courses, and generate a detailed learning behavior report.

[0060] Advanced image processing algorithms, such as optical flow method and deep learning, are used to improve the accuracy and real-time performance of body movement recognition.

[0061] The system supports the coordinated work of multiple cameras to achieve all-round monitoring of the classroom without blind spots. At the same time, it uses distributed computing technology to improve the efficiency of image processing and data analysis.

[0062] The system has self-learning capabilities and can continuously optimize the negative action image library and recognition algorithm based on historical data to improve detection accuracy and stability.

[0063] The above is an exemplary description of the present invention in combination with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as such non-substantial improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A classroom status detection system based on image processing, characterized in that: include: A video acquisition module (1) is used to install and control a high-resolution camera (5) in a classroom to acquire panoramic videos of the classroom; An image processing module (2) is used to pre-process and extract features from the collected video, especially to extract images of students’ body movements; A data analysis module (3) is used to compare the extracted body movement images with the negative movement image library to determine the learning status, and to analyze the student's facial expression using a fatigue detection algorithm, while recording and counting the student's negative movement information; The result display module (4) is used to display the student's learning behavior report, including a curve chart of the number of negative action information and a detailed learning behavior report.

2. A classroom status detection system based on image processing according to claim 1, characterized in that: The image processing module (2) performs preprocessing and feature extraction on the collected video, specifically comprising the following steps: Step 1: obtain the original image from the video stream captured by the camera, and perform pre-processing operations such as grayscale, denoising, and noise reduction on the image to improve the image quality; Step 2: Input the preprocessed image obtained in step 1 into the deep learning model to extract useful features; Step 3: Use the extracted features, combined with computer vision and machine learning technology, to build a feature matching and classification model, compare the body movement images with the images in the negative action image library, and thus judge the student's behavior status; Step 4: Use deep learning algorithms to analyze students’ facial expressions to determine their fatigue levels and emotional states; Step 5: Record and count the students' negative action information throughout the process so that the data analysis module (3) can further analyze and evaluate the students' learning behavior.

3. The classroom status detection system based on image processing according to claim 1 is characterized in that: The data analysis module (3) compares the extracted body movement images with the negative movement image library to judge the learning state, and uses the fatigue detection algorithm to analyze the student's facial expression, while recording and counting the student's negative movement information. Specifically, the following steps are included: Step 1: Based on the comparison results of the body movement images and the negative movement image library, the student's current learning behavior status is judged. The learning behavior status includes whether the student is focused, whether he is tired, and whether he has negative emotions; Step 2: Use fatigue detection algorithms and emotion recognition technology to continuously monitor students’ fatigue levels and emotional changes; Step 3: Analyze students’ learning behaviors and discover potential learning habits and problems; Step 4: Generate a detailed learning behavior report, including a graph of the number of negative actions, a score for the learning behavior, and possible problems and suggestions.

4. The classroom status detection system based on image processing according to claim 1 is characterized in that: The high-resolution camera (5) comprises a camera body (51), a bracket (52) connected to the top of the camera body (51) and connected to the wall, the bottom end of the bracket (52) is L-shaped, and the bottom end of the bracket (52) is connected to a cleaning mechanism (53).

5. The classroom status detection system based on image processing according to claim 1 is characterized in that: The cleaning mechanism (53) comprises a support block (531) connected to one side of the bottom of the support (52) frame body, a heat dissipation hole (532) is provided on the block body of the support block (531), and a heat dissipation fan (533) is connected inside the hole body of the heat dissipation hole (532).

6. The classroom status detection system based on image processing according to claim 1 is characterized in that: A heat dissipation metal block (534) is connected to one side surface of the support block (531), and a cleaning cloth (535) is connected to one side surface of the heat dissipation metal block (534).

Images