English education supervision method and system based on artificial intelligence

Abstract

The invention discloses an English education supervision method and system based on artificial intelligence, and relates to the technical field of English education supervision. According to the English education supervision method based on artificial intelligence, through collecting classroom teaching video stream data of a set English classroom and carrying out target detection processing, key feature vectors of a plurality of subjects in each frame of classroom teaching image of the set English classroom are obtained and are matched with a student status face database; obtaining class attending state time sequence data of a plurality of students; based on a pre-trained classroom behavior recognition model, extracting a class attending state evaluation set of each student, inputting the class attending state evaluation set into a pre-established classroom attention comprehensive evaluation model, and extracting a classroom concentration evaluation value of each student. And corresponding education supervision suggestions and measures are taken and fed back to the preset supervision subject, so that the accuracy of education supervision is improved, and the supervision efficiency is effectively improved.

Description

An AI-based method and system for supervising English education Technical Field

[0001] This invention relates to the field of English education supervision technology, specifically to an English education supervision method and system based on artificial intelligence. Background Technology

[0002] In the field of English teaching, especially in the classroom, effectively monitoring students' learning status and improving the quality of classroom teaching has always been a key focus for educators. Traditional classroom supervision relies heavily on teachers' on-site observation and subjective judgment, making it difficult to achieve comprehensive, objective, and real-time monitoring of learning status. With the development of artificial intelligence and computer vision technology, student classroom behavior recognition methods based on video analysis are gradually being applied to teaching supervision. By capturing features such as students' facial expressions, gaze direction, and body movements, these methods can help assess students' classroom focus and participation.

[0003] The limitations of existing technologies include at least the following problems: existing technologies have difficulty capturing the nonlinear collaborative relationships between features, resulting in a high misjudgment rate for complex states such as active responses but slight limb movements. They fail to cover implicit distraction scenarios where the face is calm but the limbs move frequently, and they also fail to capture the temporal dynamic changes and continuous dependencies of students' listening states. They are unable to reflect the complete evolution process of students from a calm expression of focused listening to a frowning and thinking state. In addition, existing technologies do not accurately bind classroom behavior characteristics with student identities, making it difficult to clearly grasp the specific listening state of each student, thus making it difficult to formulate targeted educational intervention management. Furthermore, the screening and aggregation processing of limb movements is insufficient, which can easily misjudge effective learning actions as violations, affecting the accuracy of supervision. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an artificial intelligence-based method and system for supervising English education, which solves the problem that existing technologies struggle to accurately link identity and dynamic behavioral coordination, easily leading to inaccurate supervision.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an artificial intelligence-based English education supervision method, comprising the following steps: During a set English classroom supervision period, collecting classroom teaching video stream data of a set English classroom, wherein the classroom teaching video stream data includes several frames of classroom teaching image data, and performing target detection processing to obtain key feature vectors of several subjects in each frame of the set English classroom teaching image; matching the key feature vectors of each subject in each frame of the set English classroom teaching image with a student's student ID face database to obtain time-series data of the listening status of several students in the set English classroom; performing multi-dimensional supervision analysis on the time-series data of the listening status of each student in the set English classroom based on a pre-trained classroom behavior recognition model, extracting a listening status evaluation set for each student in the set English classroom, including teaching response evaluation value and limb entropy evaluation value; performing collaborative evaluation processing on the listening status evaluation set of each student in the set English classroom based on a pre-established comprehensive classroom attention evaluation model to obtain a classroom focus evaluation value for each student in the set English classroom; assigning a corresponding supervision level to each student in the set English classroom based on the classroom focus evaluation value, taking corresponding educational supervision suggestions, and feeding back to a pre-set supervision subject.

[0006] Furthermore, the classroom teaching image data specifically refers to the pixel value and two-dimensional coordinates of each pixel point in the classroom teaching image. The specific steps for obtaining the key feature vector of each subject in each frame of the classroom teaching image of the English classroom are as follows: preprocess the classroom teaching video stream data of the English classroom; based on the object detection algorithm, perform subject recognition on the preprocessed classroom teaching video stream data of the English classroom to obtain several subjects and corresponding subject regions in each frame of the classroom teaching image of the English classroom; perform targeted encryption extraction processing on the subject region of each subject in each frame of the classroom teaching image of the English classroom to generate the key feature vector of its corresponding subject.

[0007] Furthermore, the specific steps of the targeted encryption extraction process are as follows: The main body region of each subject in each frame of the English classroom teaching image is segmented to obtain the corresponding facial region; feature extraction is performed on the facial region of each subject in each frame of the English classroom teaching image to generate the corresponding facial feature vector; and lightweight hash encryption is performed on the facial feature vector of each subject in each frame of the English classroom teaching image to generate the corresponding key feature vector.

[0008] Furthermore, the specific steps to obtain the time-series data of the listening status of several students in the English classroom are as follows: The student registration face database includes facial image data of several students, and the standard key feature vector of each student in the student registration face database is extracted; the key feature vector of each subject in each frame of the English classroom teaching image is matched with the standard key feature vector of each student in the student registration face database to generate the time-series data of the listening status of several students in the English classroom.

[0009] Furthermore, the classroom behavior recognition model includes a visual response subnetwork and a motion analysis subnetwork for extracting the teaching response evaluation value and limb entropy evaluation value of each student in a set English classroom. The visual response subnetwork includes a convolutional extraction layer, a temporal fusion layer, and a fully connected output layer. The motion analysis subnetwork includes a limb recognition layer, a differential filtering layer, a temporal convolutional layer, and a motion evaluation output layer.

[0010] Furthermore, the specific steps for extracting the teaching response evaluation value of each student in the set English class are as follows: In the convolutional extraction layer, the temporal data of the listening status of each student in the set English class is subjected to feature encoding and fusion processing to generate the corresponding student's fused feature tensor; in the temporal fusion layer, the fused feature tensor of each student in the set English class is subjected to temporal dynamic feature capture and attention weighting processing to extract the corresponding student's temporal fused feature tensor; in the fully connected output layer, based on the temporal fused feature tensor of each student in the set English class, the teaching response evaluation value of the corresponding student is output.

[0011] Furthermore, the specific steps for generating the fusion feature tensor for each student in the English classroom are as follows: Based on the temporal data of the listening status of each student in the English classroom, extract the facial key point set of the corresponding student; perform hierarchical convolution processing on the facial key point set of each student in the English classroom to obtain the temporal feature map set of the corresponding student; and concatenate the temporal feature map sets of each student in the English classroom to form the fusion feature tensor of the corresponding student.

[0012] Furthermore, the specific steps for extracting the limb entropy evaluation value of each student in the English classroom are as follows: In the limb recognition layer, the limb key point recognition processing is performed on the temporal data of the listening state of each student in the English classroom to obtain the limb key point set of the corresponding student; in the differential filtering layer, the limb key point set of each student in the English classroom is differentially filtered to obtain the micro-movement sequence of the corresponding student; in the temporal convolutional layer, the multi-scale spatiotemporal feature extraction is performed on the micro-movement sequence of each student in the English classroom to obtain the micro-movement temporal feature vector of the corresponding student; in the action evaluation output layer, the micro-movement temporal feature vector of each student in the English classroom is subjected to sliding window aggregation processing to output the limb entropy evaluation value of the corresponding student.

[0013] Furthermore, the specific steps to obtain the classroom attention assessment value of each student in the English classroom are as follows: decouple the contribution of the listening status assessment set of each student in the English classroom to obtain the contribution coefficient set of the corresponding student; input the contribution coefficient set and listening status assessment set of each student in the English classroom into the pre-established comprehensive classroom attention assessment model to extract the classroom attention assessment value of the corresponding student.

[0014] An AI-based English education monitoring system includes: a data acquisition and subject detection module, used to collect classroom teaching video stream data of a set English classroom during a set monitoring period. The classroom teaching video stream data includes several frames of classroom teaching image data, and performs object detection processing to obtain key feature vectors of several subjects in each frame of the set English classroom teaching image; an identity matching module, used to match the key feature vectors of each subject in each frame of the set English classroom teaching image with a student registration face database to obtain time-series data of the listening status of several students in the set English classroom; and a behavior analysis module, used to analyze pre-trained lesson data... The classroom behavior recognition model performs multi-dimensional monitoring and analysis on the time-series data of each student's listening status in a designated English classroom, extracting an evaluation set of each student's listening status, including teaching response evaluation value and body entropy evaluation value; the classroom focus evaluation module is used to collaboratively evaluate the listening status evaluation set of each student in the designated English classroom based on a pre-established comprehensive classroom attention evaluation model, obtaining a classroom focus evaluation value for each student in the designated English classroom; the monitoring feedback management module is used to assign a corresponding monitoring level to each student in the designated English classroom based on the classroom focus evaluation value, take corresponding educational monitoring suggestions and measures, and provide feedback to the preset monitoring body.

[0015] The present invention has the following beneficial effects: (1) The English education supervision method based on artificial intelligence extracts the key feature vector of the subject through target detection and matches it with the student's face database to generate individual listening status time series data, thereby realizing the accurate association and dynamic tracking of student identity and classroom behavior, ensuring that each subject can be accurately matched with a specific student. Based on this, the listening status time series data is generated, and the teaching response evaluation value and limb entropy evaluation value are extracted by the classroom behavior recognition model, and the two are deeply integrated by the attention comprehensive evaluation model, so as to capture the complete behavioral evolution process of students from listening, thinking to interaction, making the supervision direction clearer. For example, it can identify the teaching response of students who, although their bodies move slightly, always follow the teacher's eyes, avoiding misjudging them as distracted behavior, thereby significantly improving the reliability of individual behavior analysis and thus carrying out precise supervision.

[0016] (2) This AI-based English education supervision method comprehensively and accurately captures students' listening status. Relying on the dual-network structure of the classroom behavior recognition model, the visual response sub-network extracts multi-scale features of facial key points through hierarchical convolution and captures dynamic changes in facial expressions through temporal fusion layer, accurately outputting teaching response evaluation values. The action analysis sub-network identifies limb key points, filters effective micro-movements through differential filtering, and then extracts micro-movement features through multi-scale temporal convolution, finally outputting limb entropy evaluation values ​​that reflect the stability of limb states. Thus, it accurately captures explicit response features such as facial expressions and identifies implicit state features such as limb micro-movements. At the same time, it eliminates invalid action interference through differential filtering, avoids misjudging effective learning actions such as note-taking, and ensures the comprehensiveness of the listening status evaluation set.

[0017] (3) This AI-based English education supervision method decouples the contribution of the listening status assessment set to obtain the contribution coefficient that fits each student. Then, it inputs the coefficient and the assessment set into the classroom attention comprehensive assessment model to realize the personalized extraction of the focus assessment value. This allows the contribution weight of the assessment value to be dynamically adjusted according to the teaching response and physical state characteristics of each student, so that the output classroom focus assessment value is more in line with the actual listening status of the students. This makes the supervision level assigned based on the assessment value more reasonable, and the educational intervention suggestions fed back to the supervision body are more targeted. This can avoid excessive supervision from affecting students' normal learning, and can also intervene in real distraction behavior in a timely manner, effectively balancing the intensity of supervision.

[0018] (4) This AI-based English education supervision system completes the acquisition of classroom teaching video streams and the extraction of key feature vectors of subjects through the data collection and subject detection module; the identity matching module matches the key feature vectors with the student's student registration face database to generate time-series data of the student's exclusive listening status, connecting individual and behavioral data; the behavior analysis module relies on the pre-trained classroom behavior recognition model to extract teaching response evaluation values ​​and limb entropy evaluation values; the classroom focus evaluation module outputs classroom focus evaluation values ​​through collaborative evaluation to clarify the student's true listening status; the supervision feedback management module finally assigns a supervision level to the student and provides feedback on education supervision suggestions, so that the supervision subject can quickly take educational suggestions that fit the actual situation of the student based on the analysis results, and effectively improve the implementation effect of classroom supervision.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] Figure 1 is a flowchart of an English education supervision method based on artificial intelligence according to the present invention.

[0021] Figure 2 is a flowchart illustrating the specific steps involved in obtaining the key feature vectors of each subject in each frame of a classroom teaching image in an English teaching setting, according to an artificial intelligence-based English education supervision method of the present invention.

[0022] Figure 3 is a block diagram of an English education supervision system based on artificial intelligence according to the present invention. Detailed Implementation

[0023] Please refer to Figure 1. This embodiment of the invention provides a technical solution: an English education supervision method based on artificial intelligence, comprising the following steps: during a set English classroom supervision period (e.g., 5 minutes), collecting classroom teaching video stream data of the set English classroom, the classroom teaching video stream data including several frames of classroom teaching image data, and performing target detection processing to obtain key feature vectors of several subjects in each frame of the set English classroom teaching image; matching the key feature vector of each subject in each frame of the set English classroom teaching image with the student's student ID face database to obtain the time sequence data of the listening status of several students (IDs) in the set English classroom; Based on a pre-trained classroom behavior recognition model, multi-dimensional monitoring analysis is performed on the time-series data of each student's listening status in a designated English class. This extracts an evaluation set of each student's listening status, including teaching response evaluation values ​​and body entropy evaluation values. Based on a pre-established comprehensive classroom attention evaluation model, the evaluation set of each student's listening status is collaboratively evaluated to obtain a classroom focus evaluation value. Based on the classroom focus evaluation value, each student is assigned a corresponding monitoring level, and appropriate educational monitoring recommendations are implemented and fed back to the pre-designated monitoring entity (such as the English teacher). Specifically, the process involves comparing each student's classroom focus assessment score in an English class with a preset threshold range. If a student's score exceeds the upper limit of the threshold range, their monitoring level is designated as Level 1 (Highly Focused); if their score falls within the threshold range, they are designated as Level 2 (Moderately Focused); and if their score is below the lower limit, they are designated as Level 3 (Less Focused). For students at Level 1 (Highly Focused), the teacher... The teacher's terminal pushes intelligent learning enhancement suggestions, such as sending extended English learning materials to the student's personal learning space after class; for students under Level 2 supervision (generally focused): suggestions for mild teaching intervention are pushed to the teacher's terminal, such as prompting the teacher to increase eye contact with the student when explaining key knowledge points in class, or suggesting the teacher approach the student's group and participate in the discussion during group discussions; for students under Level 3 supervision (lower attention): suggestions for immediate intervention are pushed to the teacher's terminal, such as issuing a real-time pop-up alert during class and recommending a simple interactive command that can be initiated immediately, such as: Please [Student Name] read the current paragraph, etc.

[0024] Specifically, as shown in Figure 2, the classroom teaching image data specifically refers to the pixel value and two-dimensional coordinates of each pixel in the classroom teaching image. The specific steps to obtain the key feature vector of each subject in each frame of the classroom teaching image of the English classroom are as follows: Preprocessing the classroom teaching video stream data of the English classroom is performed as follows: Gaussian filtering algorithm can be used to denoise each frame of the classroom teaching video stream. The Gaussian filter kernel size can be set to 3×3, and the standard deviation to 0.8. Gaussian noise caused by changes in classroom lighting, camera sensor noise, etc., is eliminated by weighted averaging of each pixel and its neighboring pixels in each frame. To address potential salt-and-pepper noise in the video stream, a median filtering algorithm is used for secondary denoising. The median filtering window size can be set to 3×3. Pixels with abrupt grayscale changes in each frame are replaced to ensure clear image texture and prevent noise interference with subsequent subject recognition accuracy. Based on a target detection algorithm, subject recognition is performed on the preprocessed English classroom teaching video stream data to obtain several subjects and their corresponding subject regions in each frame of the English classroom teaching image. Specifically, in this implementation example, the target detection algorithm can use the YOLOv8 target detection model, which utilizes a feature extraction network (CSPDarknet). 53) Extract depth features from each frame of the image; fuse multi-scale features using a Feature Pyramid Network (FPN), then classify and regress each candidate region in the image, outputting the bounding box coordinates (with the top left corner of the image as the origin, and the top left corner x-coordinate, y-coordinate, width, and height of the bounding box) for all student-type targets with a confidence level higher than 0.7 in each frame; classify and filter all student-type targets output by the model, distinguishing between student subjects and teacher subjects. Based on prior knowledge of the classroom scene, the teacher subject is usually located in the podium area (the upper 1 / 3 of the image), and the student subject is usually located in the desk area (the lower 2 / 3 of the image). By calculating the center coordinates of the bounding box of each student target, the region where it is located is determined, and targets located in the desk area are selected as candidate student subjects. Teacher subjects located in the podium area and other possible interfering targets (such as staff entering the classroom) are eliminated. For each candidate student subject after selection, the corresponding subject region (including face and posture, etc.) is extracted from each frame image according to the bounding box coordinates output by the model. The pixel range of the subject region is all the pixels enclosed by the bounding box. For the subject region of each subject in each frame of classroom teaching image of the English classroom, targeted encryption extraction processing is performed to generate the key feature vector of the corresponding subject.

[0025] The specific steps of targeted encryption extraction are as follows: For each subject in each frame of the English classroom teaching image, the main body region is segmented to obtain the corresponding facial region. Specifically, a pre-trained MaskR-CNN face semantic segmentation model is loaded. This model is based on the Wideface face dataset for basic pre-training. Fine-tuning is performed using 5000 classroom occlusion images to address common classroom scenarios such as students looking down, books obscuring faces, and partial occlusion between classmates. Model parameters are configured, with a pixel classification confidence threshold of 0.85 and a mask output threshold of 0.8. Low-confidence non-face segmentation results are filtered out. The main body region image is input into the model, which then performs ResNet50 deep feature learning, Region Proposal Network (RPN) to generate face candidate boxes, and a mask branch pixel semantic classification to segment facial sub-regions, removing torsos, clothing, and desks. Non-facial background pixels; for partially occluded facial areas, morphological closing operations are performed using 3×3 rectangular structural elements; micro-holes are filled by first dilation and then erosion to ensure the integrity of the facial area, and finally, the facial area image of each subject is output, recording the two-dimensional coordinates and pixel values ​​of all pixels; feature extraction processing is performed on the facial area of ​​each subject in each frame of the English classroom teaching image to generate the corresponding facial feature vector of the subject. Specifically, facial recognition algorithms (such as HRNet, RetinaFace) are used to extract key points of the facial area of ​​each subject, which can accurately locate and select multiple key points such as the corners of the eyes (such as the inner and outer corners of the eyes), the alar of the nose (such as the left and right alar points and the tip of the nose), the corners of the mouth (such as the left and right corners of the mouth), the brow ridge (such as the starting point / midpoint / end point of the left and right brow ridges), and the jawline (such as the jaw angle point and the midpoint of the jaw). First, the Mask The facial regions obtained from R-CNN segmentation are cropped and normalized to a standard size (e.g., 112×112 pixels) to eliminate size and local offset interference. Then, the high-resolution feature preservation network of HRNet (or the multi-scale convolutional network of RetinaFace) extracts shallow edge features (e.g., eye corner contours) and deep texture features (e.g., nose wing details), fusing features from different levels to enhance keypoint feature representation. Subsequently, the algorithm's built-in keypoint regression head (containing fully connected layers + mean squared error loss function) performs coordinate regression on the fused features, outputting a coordinate set containing common keypoints for the entire face. Finally, based on preset point category labels, keypoints such as the corners of the eyes, nose wings, and mouth are selected from the common coordinate set, and their two-dimensional coordinates within the facial region are output. Based on the extracted keypoints, the inner corners of the eyes and the tip of the nose are selected as core anchor points to construct a standard facial pose template (112×112 pixels) and preset its standard coordinates. Using the original two-dimensional coordinates of the core anchor points and the preset coordinates of the standard template as constraints, the rotation and scaling parameters of the affine transformation matrix are solved. , , , Translation parameters , The formula for affine transformation is: ;in, The two-dimensional coordinates of the (original pixel) core anchor point. Given the preset coordinates of the core anchor point in the standard template, substitute the (original pixel) 2D coordinates of the core anchor point and the preset coordinates of the core anchor point in the standard template into the affine transformation formula to obtain an overdetermined linear equation system. Solve the overdetermined linear equation system using the least squares method to obtain the unique transformation matrix parameters, namely the rotation and scaling parameters. , , , Translation parameters , The two-dimensional coordinates of the aforementioned key points are transformed into standard two-dimensional coordinates in a standard template. Based on the standard two-dimensional coordinates of the key points, topological constraint feature vectors are extracted, including: Euclidean distances between key point pairs (such as the distance from the inner corner of the eye to the tip of the nose, the distance between the left and right corners of the mouth, etc.), used to characterize the proportional features of the overall facial contour; angular features of key points (such as the angle formed by the midpoint of the brow bone, the tip of the nose, and the midpoint of the chin, the angle formed by the inner corner of the eye, the ala of the nose, and the corner of the mouth, etc.), used to characterize the morphological features of local facial regions; and neighborhood multi-scale texture features of each key point are extracted, with 3×3 and 5×5 scale neighborhood windows selected as the center of each key point. (In a classroom setting, a small window captures fine textures such as wrinkles around the eyes and the contour of the nose, while a large window captures the contour textures of the brow bone and jawline.) For each neighboring window, Local Binary Pattern (LBP) texture features and Histogram of Gradient Orientation (HOG) features are extracted. LBP features are used to characterize the microstructure of the texture, and HOG features are used to characterize the gradient changes of the texture. The features of the two scales and two textures are fused to generate texture feature sub-vectors, and the topological constraint feature vectors are concatenated to form a facial feature vector. Lightweight hash encryption is then applied to the facial feature vectors of each subject in each frame of the English classroom teaching image to generate the corresponding key feature vectors. Specifically, the process involves: L2 normalization of the facial feature vectors to eliminate the impact of feature vector amplitude differences on encryption effectiveness and subsequent matching accuracy, ensuring the consistency of the encrypted feature vectors; and using a lightweight optimized version of the Locality Sensitive Hash (LSH) algorithm with the following configuration parameters: the number of hash functions can be set to 16, each hash function uses a random linear projection method, and the output dimension is 32-dimensional, resulting in a lightweight hash feature vector. The normalized facial feature vector is then converted into a continuous floating-point data stream and input into the configured lightweight LSH algorithm. The algorithm uses 16 hash functions to perform random linear projection on the data stream, converting the feature values ​​of each dimension... The data is mapped to a preset hash bucket, and then concatenated and quantized using bucket indexes to generate a hash feature vector that preserves similarity. This means that for subjects with similar original facial features, the encrypted hash feature vector also has high similarity, ensuring accuracy when matching with the student registration face database. The encrypted hash feature vector is then formatted to match the data type (e.g., 32-bit floating-point) of the feature vectors stored in the student registration face database. Simultaneously, the vector dimension and numerical range are checked to ensure they meet the matching requirements of the student registration database. If a dimension mismatch exists, it is corrected through linear interpolation or dimension compression (preserving the core hash features). Finally, the key feature vector for each subject is generated.

[0026] In this implementation scheme, multi-stage collaborative processing ensures both the accuracy of key feature vectors and adapts to the actual needs of classroom scenarios. Preprocessing employs Gaussian and median filtering to effectively eliminate interference from changes in classroom lighting and sensor noise. Secondly, YOLOv8 combined with region prior knowledge is used to screen subjects, accurately distinguishing between teachers and students and eliminating irrelevant interfering targets. When segmenting faces, MaskR-CNN corrects for occlusion issues to ensure the integrity of facial regions. Key points are standardized through affine transformation, and topological constraints and multi-scale texture feature fusion are combined to make the extracted features more comprehensive. Finally, lightweight hash encryption preserves feature similarity while adapting to the matching requirements of the student database, improving matching accuracy.

[0027] Specifically, the steps to obtain the time-series data of the listening status of several students in a set English class are as follows: The student student registration face database includes facial image data of several students, and the standard key feature vector of each student in the student registration face database is extracted, with the extraction logic consistent with that of the key feature vector. The key feature vector of each subject in each frame of the set English class teaching image is matched with the standard key feature vector of each student in the student registration face database to generate the time-series data of the listening status of several students in the set English class. Specifically, cosine similarity is used to extract the facial similarity value between the key feature vector of each subject in each frame of the set English class teaching image and the standard key feature vector of each student in the student registration face database, and this value is compared with a preset facial similarity threshold (e.g., 0.85). If the facial similarity between a subject and a student in the student registration database is ≥ the facial similarity threshold, then the subject is anchored to the corresponding student (if multiple students meet the threshold, the facial similarity is selected). The student with the highest similarity value is selected as the matching result. A one-to-one identity matching is performed between the subject in each frame and the student in the student database, and the student is assigned an ID in the student database as a temporary ID (which is decoupled after class). A frame-level record is created for each successfully matched student, using the frame timestamp of the classroom teaching video as the axis. The record includes: student temporary ID, frame timestamp, similarity value of the corresponding subject in the frame, and the subject's region in the frame. If the student's corresponding subject is not detected in the frame (facial similarity value < 0.85), it is marked as absent. All frame-level records for each student are sorted in ascending order of timestamp. The subject region corresponding to the student in consecutive frames is marked as the time-series data of the student's listening status, including the subject region corresponding to the student in each frame of the classroom teaching image and various key points in the facial region. For missing frames (e.g., no matching record for a student at a certain timestamp), interpolation is performed (filling in the mainstream status of the preceding and following 3 frames). For duplicate frames (multiple frames with the same timestamp are repeatedly matched), the record with the highest similarity value is retained.

[0028] The classroom behavior recognition model includes a visual response subnetwork for extracting the teaching response evaluation value and limb entropy evaluation value of each student in a set English classroom, and a motion analysis subnetwork. The visual response subnetwork includes a convolutional extraction layer, a temporal fusion layer, and a fully connected output layer. The motion analysis subnetwork includes a limb recognition layer, a differential filtering layer, a temporal convolutional layer, and a motion evaluation output layer.

[0029] The pre-training steps of the classroom behavior recognition model are as follows: Construct an English classroom student behavior annotation dataset, including an English classroom student facial expression annotation dataset and a body micro-movement annotation dataset. Collect several frames of time-series data on student listening status. For the English classroom student facial expression annotation dataset, annotate the teaching response level (0~1 range) for each frame / continuous frame, and simultaneously annotate the coordinates of facial key points and facial expression semantic labels (such as attentive listening, daydreaming, frowning while thinking, etc.). For the body micro-movement annotation dataset, annotate the two-dimensional coordinates of each frame's body key points (wrist, fingertips, elbow, etc.), the type of micro-movement (leg shaking, pen spinning, finger tapping, etc.), and the corresponding sample entropy label (based on manually judged action regularity annotation, entropy value range 0~1). Perform preprocessing, and divide the corresponding annotation datasets into training, validation, and test sets in a 7:2:1 ratio.

[0030] The classroom behavior recognition model is trained by taking the temporal fusion layer of the visual response subnetwork as an example. The student facial feature tensor of each frame is input into the bidirectional GRU (gated recurrent unit) to extract temporal features. The bidirectional GRU layer controls the flow of information through its internal gating mechanism (reset gate, update gate, and candidate hidden state) and learns the long-term dependencies between each time step (frame). The bidirectional GRU can effectively capture the dynamic changes of the temporal data of the listening state (such as the trend of the student's facial expression changes from distraction to focus). The bidirectional GRU network optimizes the weights through the backpropagation algorithm to minimize the prediction error (such as mean squared error MSE). The bidirectional GRU will gradually learn the long-term dependencies and short-term fluctuations of the student's facial expression features and capture the dynamic process of teaching response, such as the temporal changes of expressions such as focused listening, frowning while thinking, and distraction.

[0031] During training, optimization algorithms (such as the AdamW optimizer) are used to minimize prediction errors, and hyperparameters (such as learning rate, number of GRU hidden layer units, and convolutional kernel size) are adjusted to improve model performance. The model is evaluated using a validation set, and an optimal model is finally optimized. After training, the generalization ability of the model is evaluated using a test set to ensure that the model can accurately process unseen classroom listening time-series data and extract effective features from it. Finally, the trained classroom behavior recognition model is saved, and the model parameters (convolutional layer weights, GRU layer weights, attention module weights, etc.) are used in subsequent practical application stages to ensure that it can be deployed online for real-time evaluation of classroom student listening status.

[0032] In this implementation plan, identity matching uses cosine similarity combined with a 0.85 threshold for screening, which can accurately match classroom subjects with students in the student database. The temporary ID design takes into account both matching traceability and privacy protection. The timestamp axis constructs frame-level records and fills in missing frames and removes duplicate frames to ensure the integrity of the time-series data of the class status. The model pre-training constructs a dedicated labeled dataset for classroom scenarios. The bidirectional GRU effectively captures dynamic changes in facial expressions. Combined with AdamW optimization and multi-set evaluation, the optimal model is optimized to ensure accurate processing of unseen time-series data. It can be directly deployed online to achieve real-time evaluation and extract accurate evaluation values ​​for effective educational supervision.

[0033] Specifically, the steps for extracting the teaching response evaluation value of each student in a given English class are as follows: In the convolutional extraction layer (of the visual response sub-network), the temporal data of each student's listening status in the given English class is processed by feature encoding and fusion to generate the corresponding student's fused feature tensor; in the temporal fusion layer (of the visual response sub-network), the fused feature tensor of each student in the given English class is processed by temporal dynamic feature capture and attention weighting to extract the corresponding student's temporal fused feature tensor. Specifically, the fused feature tensor is input into the bidirectional GRU network in the temporal order of the frames. Through the coordinated interaction of the reset gate, update gate, and candidate hidden state built into the bidirectional GRU network, feature fusion is achieved: the reset gate filters and retains key historical frame feature information, the update gate dynamically adjusts the fusion ratio of historical frame and current frame features, and the candidate hidden state generates core feature representations based on the filtered historical information and current frame features. Finally, the current frame and the corresponding historical / future frame features are fused to output the result. The hidden states contain temporal relationships. The forward GRU traverses from frame 1 to the last frame of the current period, accurately capturing the trend of facial expressions from the past to the present, such as the gradual shift from student distraction to focus. The backward GRU traverses from the last frame to frame 1 of the current period, capturing the continuity of the current state into the future, such as whether the focused state can continue into subsequent teaching segments. Finally, the two sets of hidden state sequences output by the forward and backward GRUs are concatenated along the channel dimension to obtain the GRU temporal feature tensor. The value at each position in this tensor is the feature value of the corresponding frame and channel. This tensor completely encodes the dynamic changes in facial expressions between consecutive frames. Based on SE (Squeeze-and-Excitation) channel attention, weight redistribution is performed on multiple channels of the GRU temporal feature tensor. That is, global average pooling is performed along the frame dimension of the GRU temporal feature tensor to perform a squeezing operation, compressing the corresponding frame features of each channel into global statistical values ​​to aggregate channel information. Specifically: ;in, For the first Global statistics for each channel The first one in the GRU temporal feature tensor Frame, First The eigenvalues ​​of the channel, 1, 2, 3, ... , The total number of frames. 1, 2, 3, ... , The total number of channels is given; then the global statistics are input into two cascaded fully connected layers, and the channel weights are adaptively generated through ReLU and Sigmoid nonlinear transformations, as shown in the formula: ;in, , For the pre-trained weight matrix, , For bias terms, The activation function is Sigmoid; finally, the channel weights are multiplied element-wise by the GRU temporal feature tensor to complete the weighted selection, as shown in the formula: Output the channel-weighted feature tensor; based on this, temporal attention is introduced to process the channel-weighted feature tensor. Frame features are weighted and scored at the frame level, focusing on key time frames such as teacher questions and key points of explanation, while weakening background frame features such as those of students looking down to take notes. First, each frame feature is input into a single fully connected layer, and an initial frame weight is generated through a Sigmoid nonlinear transformation to evaluate the matching degree between each frame and the teaching response. Then, Softmax normalization is performed on the initial weights to calibrate the weight distribution, ensuring that the sum of the weights is 1. Finally, the normalized frame weights are summed frame-by-frame with the channel-weighted feature tensor to eliminate frame dimensionality and generate the final temporal fusion feature tensor. In the fully connected output layer (of the visual response subnetwork), based on the temporal fusion feature tensor of each student in the English classroom, the corresponding student's teaching response evaluation value is output. Specifically, this layer includes two fully connected layers and a nonlinear activation. The temporal fusion feature tensor is input into the first fully connected layer, and through the synergistic effect of the pre-trained weight matrix and bias term, high-dimensional features are mapped to low-dimensional dense features, i.e.: ;in, These are the low-dimensional features output by the first fully connected layer. This is the pre-trained weight matrix for the first fully connected layer. For temporal fusion feature tensors, The first fully connected layer is used as the pre-trained bias term. The low-dimensional features output from the first fully connected layer are input into the second fully connected layer (output layer). Through the linear mapping between the pre-trained weight matrix and the bias term, combined with the Sigmoid activation function, the output value is normalized to the [0,1] interval to obtain the teaching response evaluation value corresponding to each student. That is, the formula for outputting the teaching response evaluation value of a certain student is: ;in, This represents a student's evaluation score for their teaching response. This is the pre-trained weight matrix for the second fully connected layer. This is the pre-trained bias term for the second fully connected layer.

[0034] The aforementioned weight matrix and bias terms are all parameters obtained by pre-training on a dataset of English classroom student facial expression annotations (the annotation information includes teaching response levels and corresponding temporal features). The training process uses the mean squared error loss function (MSE) to minimize the deviation between the predicted evaluation value and the actual annotation value, ensuring the accuracy of the mapping relationship.

[0035] The specific steps for generating the fusion feature tensor for each student in the set English classroom are as follows: Based on the temporal data of each student's listening status in the set English classroom, extract the corresponding student's facial keypoint set, that is, extract the pixel values ​​and two-dimensional coordinates of all pixels (including multiple keypoints) in the facial region of each student's listening status temporal data in the English classroom, and use this as the facial keypoint set; perform hierarchical convolution processing on the facial keypoint set of each student in the set English classroom to obtain the corresponding student's temporal feature map set, that is, combine the sliding receptive field of the convolution kernel and the channel feature response mechanism to sequentially... Each frame independently undergoes hierarchical convolution operations, sequentially generating shallow texture feature maps such as the corners of the eyes and mouth, mid-level local expression feature maps such as brow ridges and eyelid opening and closing, deep muscle movement feature maps such as forehead texture when frowning and cheekbone bulge when smiling, and high-level semantic feature maps such as frowning when listening attentively and frowning when thinking. Through sliding traversal and response value filtering of each convolutional kernel, combined with feature combination and residual connection mechanisms, feature mapping from low-level pixel changes to high-level semantics is achieved. Finally, the high-level semantic feature maps of each frame in the time-series classroom teaching images are stacked according to the frame dimension to form a fused feature tensor. This method is used to characterize multi-dimensional changes in students' facial expressions, reflecting their immediate response to teaching content. For example, it extracts shallow texture features such as eye corner contours and mouth lines. A 3×3 convolution kernel is used to perform a sliding traversal (stride 1) on facial pixels, with a 3×3 pixel range as the sliding receptive field. The convolution response value at each pixel location is obtained by calculating the dot product of the pixels within the receptive field and the kernel weights, and then adding a bias term. A response value threshold of 0.5 is set, and pixels higher than the threshold are selected as edge texture pixels. Adjacent edge texture pixels are then connected based on the eight-neighbor connectivity principle. Continuous texture contours such as the corners of the eyes and the lines of the mouth are formed, ultimately generating a single-frame shallow texture feature map, which carries the basic geometric contour information of the face. Deep muscle movement features such as forehead texture when frowning and cheek bulge when smiling are extracted: using the single-frame shallow texture feature map as input, a sliding traversal is performed using a 3×3 convolution kernel (stride 2). Downsampling is achieved through a stride of 2, expanding the receptive field of the convolution to 5×5 pixels to ensure complete coverage of local expression areas such as the brow bone and eyelids. The combined response value of the shallow texture contours within the receptive field (representing the correlation strength of multiple textures) is calculated and set to 0.A response value threshold of 6 is used to select high-response-value local regions as mid-level local expression feature regions. For each region, the variance of the density distribution of internal edge texture pixels is calculated (the larger the variance, the more pronounced the local expression), thus characterizing the change in local expression intensity. Finally, a single-frame mid-level local expression feature map is generated, carrying information about changes in facial local expressions and reflecting the student's initial emotional response. High-level semantic features such as focused listening and frowning while thinking are extracted: For the single-frame deep muscle motion feature map, a 2-layer 3×3 convolutional kernel with a residual block structure is used (this convolutional kernel has been trained in the model pre-training stage using a facial expression dataset labeled with semantic tags such as focused listening and frowning while thinking, learning the correspondence between different feature combinations and semantic tags) for sliding traversal processing (step size 1). The receptive field of the convolutional kernel is 3×3 pixels, which can cover multi-dimensional combinations of shallow texture, mid-level local expressions, and deep muscle motion features. The three types of features within the receptive field are calculated... The combined response value (i.e., the matching degree between the feature combination within the receptive field and the target semantic label based on the correspondence learned during pre-training) is used to calculate the response value threshold (e.g., 0.8). Regions with response values ​​higher than the threshold are selected as high-level semantic feature regions. For each high-level semantic feature region, the weighted average of the features of each channel within it is calculated (the weights are the semantic importance weights learned during the training of the convolutional kernel) to obtain the semantic response value, which is used to accurately represent high-level semantics such as attentive listening and frowning while thinking. Finally, a single-frame high-level semantic feature map is generated. This feature map carries high-level semantic information of facial expressions and directly reflects the student's immediate response to the teaching content, resulting in multiple single-frame high-level semantic feature maps, which are labeled as a temporal feature map set. The temporal feature map sets of each student in the English class are then concatenated to form the fusion feature tensor of the corresponding student. That is, multiple single-frame high-level semantic feature maps are stacked in frame-time order to form a fusion feature tensor.

[0036] The specific steps for extracting the limb entropy evaluation value of each student in a set English class are as follows: In the limb recognition layer (of the motion analysis sub-network), the limb keypoint recognition processing is performed on the time-series data of each student's listening state in the set English class to obtain the corresponding student's limb keypoint set. Specifically, the limb sub-region is separated from the main region of each frame of classroom teaching image to eliminate the interference of the facial region on the limb keypoint recognition; the HRNet pose estimation model pre-trained based on the classroom student limb pose annotation dataset is loaded, and the single-frame limb sub-region image is input into the model. Through a multi-scale feature fusion mechanism, the wrist, fingertip, elbow, shoulder, knee, and foot are output. The system first identifies the two-dimensional coordinates and recognition confidence of key limb points such as the ankle. Then, a confidence threshold (e.g., 0.7) is set to remove low-confidence key points. Based on anatomical constraints (such as the relative position range of the wrist and elbow, and the angle range of the shoulder and hip), coordinate deviations caused by limb occlusion and posture shifts are corrected. Finally, the single-frame limb key points of all time-series frames are integrated in frame-time order to form a limb key point set, including the pixel values ​​and two-dimensional coordinates of several limb key points in each frame of the classroom teaching image. In the differential filtering layer (of the motion analysis sub-network), the limb key point set of each student in the English classroom is differentially filtered to obtain its... The corresponding micro-movement sequence for each student is as follows: The difference between the two-dimensional coordinates of the same limb key points of each student in adjacent frames of classroom teaching images is calculated, divided by the time interval between adjacent frames to obtain the velocity components in the x and y directions. The resultant velocity is then extracted from these components. The difference between the resultant velocities of adjacent frames is calculated, divided by the time interval to obtain the resultant acceleration. This yields the resultant velocity and resultant acceleration corresponding to each limb key point of each student in each frame of classroom teaching images (the first and second frames are defaulted to 0). The resultant velocity and resultant acceleration corresponding to each limb key point of each student in each frame of classroom teaching images are then compared with preset resultant velocity thresholds and resultant acceleration thresholds, respectively. For processing, if the resultant velocity of a certain limb key point in a certain frame is greater than a preset resultant velocity threshold (e.g., 0.04) and the resultant acceleration is greater than a preset resultant acceleration threshold (e.g., 0.02), then the velocity and acceleration features of the corresponding limb key point in that frame are removed; if the resultant velocity is less than or equal to the preset resultant velocity threshold and the resultant acceleration is less than or equal to the preset resultant acceleration threshold (including the default 0 value in the first and second frames), then it is determined to be a micro-movement such as leg shaking, pen spinning, or finger tapping, and the velocity and acceleration features of the corresponding limb key point in that frame are retained. All retained key points are combined in each frame of classroom teaching image with their resultant velocity and resultant acceleration, and integrated in frame time sequence to generate a micro-movement sequence.In the temporal convolutional layer (of the motion analysis subnetwork), multi-scale spatiotemporal feature extraction is performed on the micro-motion sequence of each student in the English classroom setting to obtain the corresponding micro-motion temporal feature vector. Specifically, the feature dimension of the micro-motion sequence is flattened, and the motion features (resultant velocity, resultant acceleration) of all key points in each frame are concatenated in key point order to adapt to the format of the one-dimensional temporal convolutional layer. Three sets of parallel one-dimensional temporal convolutional branches are constructed to capture the core motion patterns of micro-motions at different time scales through convolutional kernels of different sizes. That is, the short temporal convolutional branch performs convolution operation on the micro-motion features of each consecutive 3 frames, using a one-dimensional convolutional kernel of size 3 along the frame temporal sequence. The feature extraction process uses a sliding window with a stride of 1. Based on the linear combination principle of convolutional kernel weights and bias terms, the feature vectors of the three frames within the sliding window are weighted, summed, and the bias term is added to obtain the feature response value corresponding to that window. For the feature response value output by each sliding window, a nonlinear transformation is performed using the ReLU activation function (setting negative response values ​​to 0 and retaining only positive response values ​​to focus on effective micro-motion change information). Then, the activated features are standardized through a batch normalization layer. The feature response values ​​of five consecutive sliding windows are used to form several sub-sequence analysis windows. The variance of the feature response values ​​within the sub-sequence windows is calculated to extract short-term fluctuation features of high-frequency micro-motions, which are used to characterize finger tapping and fingertip micro-movements. The micro-motions of high-frequency actions are analyzed to recover short-term temporal variation patterns, reflecting the instantaneous amplitude and frequency of these micro-motions. The mid-temporal convolution branch performs convolution operations on the micro-motion features of each consecutive 5 frames, using a 5-dimensional convolution kernel to slide along the temporal dimension of the frames with a stride of 1 to extract features. Based on the same logic as the short-temporal convolution branch, the feature response value corresponding to that window is obtained. For the feature response value output by each sliding window, a ReLU activation function (setting negative response values ​​to 0) is sequentially concatenated with a batch normalization layer for nonlinear transformation and standardization. Several sub-sequence analysis windows are formed using the feature response values ​​of three consecutive sliding windows, and the mean and sum of the feature response values ​​within each sub-sequence window are calculated. The range is used to extract the amplitude fluctuation features of medium-scale micro-movements, which is used to characterize the amplitude fluctuation pattern of the wrist's small-scale rotation from acceleration to constant speed and then to deceleration during pen spinning. It can reflect the duration and amplitude change trend of medium-scale micro-movements. The long temporal convolution branch performs convolution operation on the micro-movement features of each continuous 7 frames. It uses a one-dimensional convolution kernel of size 7 to slide along the temporal dimension of the frame to extract features. Based on the same logic as the analysis of the short temporal convolution branch, the feature response value corresponding to the window is obtained. For the feature response value output by each sliding window, the ReLU activation function (negative response value is set to 0) and the batch normalization layer (parameters are the same as the previous branch) are sequentially connected for nonlinear transformation and standardization.Several subsequence analysis windows are formed by using the feature response values ​​of four consecutive sliding windows. The periodic correlation coefficient of the feature response values ​​within the subsequence windows is calculated to extract long-term micro-motion periodic trend features, which can reflect the period length and overall motion law of long-term micro-motions. The output features of three sets of parallel convolutional branches are concatenated along the channel dimension to obtain a high-dimensional temporal feature sequence that integrates multi-scale motion information. The concatenated feature sequence is then dimensionality-optimized by a one-dimensional average pooling layer. Global average pooling is then performed to calculate the average value of each feature channel along the frame temporal dimension to obtain the micro-motion temporal feature vector. In the motion evaluation output layer of the motion analysis subnetwork, the micro-motion temporal feature vectors of each student in the English classroom are aggregated using a sliding window method to output the corresponding student's limb entropy evaluation value. Specifically, the micro-motion temporal feature vectors are used as input, first reconstructed into a temporal sequence, and then the sliding window is divided into 20 frames with a step size of 10 frames. Each sliding window extracts a sub-feature sequence, resulting in several independent sub-feature sequences. The sample entropy of each sub-sequence is calculated, and then the sample entropy of all sliding windows is averaged to obtain the limb entropy evaluation value for each student. The smaller the value, the more regular and stable the student's micro-motions are.

[0037] In this implementation scheme, a dual-network approach combining visual response and motion analysis is used to accurately extract two core evaluation values: teaching response and limb entropy. The visual response sub-network extracts facial features from shallow texture to high-level semantics through hierarchical convolution. A bidirectional GRU combined with channel and temporal attention fully captures the dynamic changes in facial expressions over time, ensuring that the teaching response evaluation value closely matches students' actual classroom responses. Meanwhile, the motion analysis sub-network uses HRNet combined with anatomical constraints to accurately identify key points of the limbs, differential filtering eliminates large movements, multi-scale temporal convolution captures micro-movement features of different durations, and sliding window calculation of sample entropy ensures that the limb entropy evaluation value accurately reflects the movement patterns, avoiding misjudgment of effective movements such as note-taking, thereby improving the accuracy of evaluation value extraction.

[0038] Specifically, the steps to obtain the classroom attention assessment value for each student in the designated English class are as follows: The contribution decoupling process is performed on the listening status assessment set of each student in the designated English class to obtain the corresponding student's contribution coefficient set (including teaching response contribution coefficient and body state contribution coefficient). Specifically, the teaching response assessment value and body entropy assessment value of each student in the designated English class are read, and the body entropy assessment value is positively processed to obtain body attention (i.e., 1 - body entropy assessment value). The corresponding information entropy value is extracted, and then the corresponding information entropy value is transformed using the reciprocal inhibition mapping function f(x) = 1 / (1+x), such as 1 / (1 + information entropy value of teaching response assessment value), and summed to obtain the information entropy sum value. The transformed information entropy value is then compared with the information entropy sum value to obtain the teaching response contribution coefficient and body state contribution coefficient corresponding to the parameters. The contribution coefficient set and listening status assessment set of each student in the designated English class are input into a pre-established comprehensive classroom attention assessment model to extract the corresponding student's classroom attention assessment value. The comprehensive classroom attention assessment model is as follows: ;in, To set the first English class Student's classroom attention assessment score To set the first English class The teaching response evaluation value of each student To set the first English class The physical entropy assessment value of each student To determine the contribution coefficient of teaching response in English classrooms, To determine the contribution coefficient of body language in English classrooms, These are the collaborative gain coefficients stored in the database.

[0039] It should be noted that in this model The acquisition steps are as follows: Collect historical listening status sample data in English classrooms, which should include at least 500 valid samples: each sample corresponds to one student's teaching response evaluation value, limb entropy evaluation value, and the teacher's manually labeled classroom focus label for that period (label value range is [0,1], the larger the value, the higher the focus); Optimization objective setting: Substitute the teaching response evaluation value and limb entropy evaluation value from the historical samples into the comprehensive classroom attention evaluation model, with the minimum mean square error between the model's output classroom focus evaluation value and the manually labeled focus label as the optimization objective, and use the gradient descent algorithm to optimize the model. Iterative optimization is performed, adjusting the value of λ and calculating the mean square error in each iteration, until the mean square error converges to a preset threshold (e.g., mean square error ≤ 0.05). The finally converged value is then... The value is stored in the database as a synergistic gain coefficient for subsequent classroom focus assessments.

[0040] In the model This method integrates teaching response assessment values ​​with positively transformed body focus, using teaching response contribution coefficients and body state contribution coefficients to strengthen the basic influence weights of these two characteristics on classroom focus (aligning with the classroom assessment logic that teaching response is the core of active participation and body state is a passive focus aid); simultaneously, the exponential form can amplify the positive effect of high-contribution characteristics (e.g., the higher the teaching response assessment value, the better). The more significant the gain, the more fundamental the nonlinear weighting of the two features is realized.

[0041] In the model This is used to introduce the synergistic gain effect between instructional response and physical focus: it corresponds to the cosine similarity between instructional response and physical focus (quantifying the degree of matching between the two assessment values). The more consistent the two assessment values ​​are (e.g., high instructional response and high physical focus), the closer this fractional value is to 1; combined with the synergistic gain coefficient... This can amplify the focus assessment value in this scenario.

[0042] This implementation plan combines contribution decoupling with a synergistic gain model to ensure accurate classroom focus assessment. Contribution decoupling dynamically allocates the weights of teaching response and body state based on information entropy, which aligns with the assessment logic of teaching response as the core and body state as an auxiliary factor. The synergistic gain coefficient is optimized through multiple sets of historical samples, which can amplify the gain when the assessment values ​​are consistent, making the results more in line with the true focus. The accurately extracted classroom focus assessment values ​​can help teachers quickly classify students, guide students with low teaching response but high body focus to actively interact, and intervene in a timely manner for students with low scores in both areas, thereby improving the targeting and efficiency of classroom management.

[0043] Please refer to Figure 3. This embodiment of the invention provides a technical solution: an artificial intelligence-based English education monitoring system, comprising: a data acquisition and subject detection module, used to acquire classroom teaching video stream data of a set English classroom during a set English classroom monitoring period. The classroom teaching video stream data includes several frames of classroom teaching image data, and performs object detection processing to obtain key feature vectors of several subjects in each frame of the set English classroom teaching image; an identity matching module, used to match the key feature vectors of each subject in each frame of the set English classroom teaching image with a student registration face database to obtain time-series data of the listening status of several students in the set English classroom; and a behavior analysis module. The system includes: a classroom behavior recognition model for multi-dimensional monitoring and analysis of the listening status time-series data of each student in a designated English class; an evaluation set of listening status for each student in the designated English class, including teaching response evaluation value and body entropy evaluation value; a classroom focus evaluation module for collaborative evaluation processing of the listening status evaluation set of each student in the designated English class based on a pre-established comprehensive classroom attention evaluation model, to obtain the classroom focus evaluation value of each student in the designated English class; and a monitoring feedback management module for assigning a corresponding monitoring level to each student in the designated English class based on the classroom focus evaluation value, taking corresponding educational monitoring suggestions and measures, and feeding back to the preset monitoring subject.

[0044] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0045] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An artificial intelligence-based method for supervising English education, characterized in that, Includes the following steps: During a designated English classroom monitoring period, video stream data of designated English classrooms is collected. This video stream data includes several frames of classroom teaching images, which are then processed for target detection to obtain key feature vectors of several subjects in each frame of the designated English classroom teaching image. The key feature vectors of each subject in each frame of the designated English classroom teaching image are matched with a student's facial database to obtain time-series data of the listening status of several students in the designated English classroom. Based on a pre-trained classroom behavior recognition model, multi-dimensional monitoring analysis is performed on the time-series data of each student's listening status in the designated English classroom, extracting a listening status evaluation set for each student, including teaching response evaluation values ​​and limb entropy evaluation values. Based on a pre-established comprehensive classroom attention evaluation model, the listening status evaluation set of each student in the designated English classroom is collaboratively evaluated to obtain a classroom focus evaluation value for each student in the designated English classroom. Based on the classroom focus evaluation value, each student in the designated English classroom is assigned a corresponding monitoring level, and corresponding educational monitoring recommendations are implemented and fed back to the pre-designated monitoring body.

2. The method for supervising English education based on artificial intelligence according to claim 1, characterized in that, The classroom teaching image data specifically refers to the pixel value and two-dimensional coordinates of each pixel in the classroom teaching image. The specific steps to obtain the key feature vector of each subject in each frame of the classroom teaching image of the English classroom are as follows: preprocess the classroom teaching video stream data of the English classroom; based on the target detection algorithm, perform subject recognition on the preprocessed classroom teaching video stream data of the English classroom to obtain several subjects and corresponding subject regions in each frame of the classroom teaching image of the English classroom. For each subject in each frame of the English classroom teaching image, targeted encryption extraction is performed on the subject region to generate the corresponding key feature vector of the subject.

3. The method for supervising English education based on artificial intelligence according to claim 2, characterized in that, The specific steps of the targeted encryption extraction process are as follows: The main body region of each subject in each frame of the English classroom teaching image is segmented to obtain the corresponding facial region; feature extraction is performed on the facial region of each subject in each frame of the English classroom teaching image to generate the corresponding facial feature vector; and lightweight hash encryption is performed on the facial feature vector of each subject in each frame of the English classroom teaching image to generate the corresponding key feature vector.

4. The method for supervising English education based on artificial intelligence according to claim 1, characterized in that, The specific steps to obtain the time-series data of the listening status of several students in a set English class are as follows: The student student facial database includes facial image data of several students, and the standard key feature vector of each student in the student student facial database is extracted; the key feature vector of each subject in each frame of the set English class teaching image is matched with the standard key feature vector of each student in the student student facial database to generate the time-series data of the listening status of several students in the set English class.

5. The method for supervising English education based on artificial intelligence according to claim 1, characterized in that, The classroom behavior recognition model includes a visual response subnetwork and a motion analysis subnetwork, which are used to extract the teaching response evaluation value and limb entropy evaluation value of each student in a set English class. The visual response subnetwork includes a convolutional extraction layer, a temporal fusion layer, and a fully connected output layer. The motion analysis subnetwork includes a limb recognition layer, a differential filtering layer, a temporal convolutional layer, and a motion evaluation output layer.

6. The method for supervising English education based on artificial intelligence according to claim 5, characterized in that, The specific steps for extracting the teaching response evaluation value of each student in a set English class are as follows: In the convolutional extraction layer, the temporal data of each student's listening status in the set English class are processed by feature encoding and fusion to generate the corresponding student's fused feature tensor; in the temporal fusion layer, the fused feature tensor of each student in the set English class is processed by temporal dynamic feature capture and attention weighting to extract the corresponding student's temporal fused feature tensor; in the fully connected output layer, based on the temporal fused feature tensor of each student in the set English class, the teaching response evaluation value of the corresponding student is output.

7. The method for supervising English education based on artificial intelligence according to claim 6, characterized in that, The specific steps for generating the fusion feature tensor for each student in the English classroom are as follows: Based on the temporal data of the listening status of each student in the English classroom, extract the facial key point set of the corresponding student; perform hierarchical convolution processing on the facial key point set of each student in the English classroom to obtain the temporal feature map set of the corresponding student; and concatenate the temporal feature map sets of each student in the English classroom to form the fusion feature tensor of the corresponding student.

8. The method for supervising English education based on artificial intelligence according to claim 5, characterized in that, The specific steps for extracting the limb entropy evaluation value of each student in a set English class are as follows: In the limb recognition layer, the limb key point recognition processing is performed on the temporal data of the listening state of each student in the set English class to obtain the limb key point set of the corresponding student; in the differential filtering layer, the limb key point set of each student in the set English class is differentially filtered to obtain the micro-movement sequence of the corresponding student; in the temporal convolutional layer, multi-scale spatiotemporal feature extraction is performed on the micro-movement sequence of each student in the set English class to obtain the micro-movement temporal feature vector of the corresponding student; in the action evaluation output layer, the micro-movement temporal feature vector of each student in the set English class is processed by sliding window aggregation to output the limb entropy evaluation value of the corresponding student.

9. The method for supervising English education based on artificial intelligence according to claim 1, characterized in that, The specific steps to obtain the classroom focus assessment value of each student in the set English class are as follows: Decouple the contribution of the listening status assessment set of each student in the set English class to obtain the contribution coefficient set of the corresponding student; input the contribution coefficient set and listening status assessment set of each student in the set English class into the pre-established comprehensive classroom attention assessment model to extract the classroom focus assessment value of the corresponding student.

10. An AI-based English education supervision system, employing the AI-based English education supervision method according to any one of claims 1-9, characterized in that, include: The data acquisition and subject detection module is used to collect classroom teaching video stream data of a set English classroom during a set English classroom monitoring period. The classroom teaching video stream data includes several frames of classroom teaching image data, and subject detection processing is performed to obtain key feature vectors of several subjects in each frame of classroom teaching image of the set English classroom. The identity matching module matches the key feature vectors of each subject in each frame of the English classroom teaching image with the student's face database to obtain the time-series data of the listening status of several students in the English classroom. The behavior analysis module performs multi-dimensional monitoring analysis on the time-series data of the listening status of each student in the English classroom based on a pre-trained classroom behavior recognition model, and extracts the listening status evaluation set of each student in the English classroom, including teaching response evaluation value and body entropy evaluation value. The classroom focus evaluation module performs collaborative evaluation processing on the listening status evaluation set of each student in the English classroom based on a pre-established comprehensive classroom attention evaluation model to obtain the classroom focus evaluation value of each student in the English classroom. The monitoring feedback management module assigns a corresponding monitoring level to each student in the English classroom based on the classroom focus evaluation value, takes corresponding educational monitoring suggestions, and feeds back to the preset monitoring subject.

Images