Natural language assisted teacher gesture recognition method
Through the natural language-assisted teacher gesture recognition method, combined with video feature extraction and text description, the context dependence and background interference problems of gesture recognition in the existing technology are solved, high-accuracy gesture recognition in teaching scenarios is achieved, and the quality of teaching interaction and teacher teaching ability training are improved.
Patent Information
- Application Number
- CN202510909006.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-02
- Publication Date
- 2025-10-21
AI Technical Summary
Existing teacher gesture recognition technology has context dependence, background interference and modeling architecture defects, resulting in insufficient gesture recognition accuracy. In particular, visual feature similarity and semantic association are difficult to effectively capture in teaching scenarios.
A natural language-assisted method is adopted to extract video features through a contrastive learning model, combine one-dimensional temporal convolution and gated recurrent units to capture gesture details, generate text descriptions and use Sentence-BERT to encode category names, fuse visual and text features, and apply self-attention mechanism and cross-entropy loss function for gesture classification.
It significantly improves the accuracy of gesture recognition and can accurately distinguish gestures in scenes with changing lighting, occlusion, and similar backgrounds. It enhances the ability to recognize gestures in teaching scenarios and improves the effectiveness of teaching interaction quality assessment and teacher teaching ability training.
Smart Images

Figure CN120823646A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of artificial intelligence and relates to a natural language-assisted teacher gesture recognition method. Background Art
[0002] Teacher gestures, as an important part of teaching behavior, are a concrete expression of the teacher's thinking process and play multiple key functions in teaching. Through gestures, teachers can reinforce the key points of teaching content, provide students with more intuitive demonstrations, promote students' in-depth understanding of abstract concepts and enhance knowledge retention. For example, in team sports teaching, gestures can quickly convey tactical intentions and position adjustment information, significantly improving the efficiency of teacher-student collaboration. Studies have shown that diverse teacher gestures can provide learners with multi-dimensional information input, build a bridge for understanding knowledge, and serve as an extension of language expression, effectively deepening the quality of classroom interaction. In addition, referential gestures can also serve as clues for students to recall knowledge, and have important feedback value for teaching reflection and teacher professional development.
[0003] Existing teacher gesture recognition technology has significant limitations. While methods based on skeletal point tracking can stably analyze hand motion trajectories and resist light interference, their underlying algorithm design suffers from the defect of disembedding teaching semantics: when such methods use skeletal key point heat maps and motion trajectory modeling, they separate gestures from teaching behaviors such as blackboard writing, teaching aid interaction, and teacher-student gaze, ignoring the contextual dependence of teaching scenarios. This is specifically manifested in three core issues:
[0004] (1) Context-dependent challenges: The same hand gesture may belong to different semantic categories depending on the interaction object, while gestures with significant visual differences may be classified into the same category because they express the same teaching meaning;
[0005] (2) Background interference problem: The backgrounds of teaching scenes are highly similar, which increases the difficulty of distinguishing visual features;
[0006] (3) Modeling architecture defects: The existing framework adopts a cascade architecture of "spatial feature extraction + temporal information modeling", which leads to the isolation of the relationship between gesture categories and makes it difficult to jointly model complex semantic associations.
[0007] While temporal modeling techniques such as 3D-CNN and temporal segmentation networks (TSNs) have demonstrated excellent performance in general action recognition tasks, their reliance on low-level visual features makes it difficult to capture the semantic information of the teaching scene, limiting the accuracy of teacher gesture recognition. Therefore, a new approach is urgently needed that can integrate gesture visual features, dynamic motion information, and teaching context, addressing background similarity, intra-class differences, and inter-class similarity through a cross-modal complementary mechanism. Summary of the Invention
[0008] In view of this, the purpose of the present invention is to provide a natural language-assisted teacher gesture recognition method, so as to make up for the gesture misjudgment problem caused by relying solely on visual data in current teacher gesture recognition, and improve the accuracy of teacher gesture recognition.
[0009] In order to achieve the above object, the present invention provides the following technical solutions:
[0010] A natural language-assisted teacher gesture recognition method comprises the following steps:
[0011] S1: Video feature extraction stage: Data cleaning and preprocessing of the teacher gesture video dataset, and the contrastive language-image pre-training (CLIP) model based on contrastive learning pre-training is used to extract spatial features F video ∈R T×N1 , where T is the number of video frames and N1 is the spatial feature dimension;
[0012] S2: Video feature temporal mining stage: The spatial features extracted in S1 are processed using a one-dimensional temporal convolution operation to capture short-term temporal information and local temporal associations, and the spatial features are modeled using a gated recurrent unit (GRU) to extract long-term temporal dependencies and cross-period contextual information.
[0013] S3: Text feature extraction stage: Generative Pre-trained Transformer 4 Optimized (GPT-4o) is used to generate text descriptions of the teacher's hand posture frame by frame based on the prompt word instructions, and Sentence-Bidirectional Encoder Representations from Transformers (Sentence-BERT) is used to encode the text descriptions to obtain text features. Where N2 is the text feature dimension;
[0014] S4: Video-text fusion stage: The spatial feature F extracted by S1 video and the text features F extracted by S3 text Splicing to form fusion features
[0015] S5: Fusion feature temporal mining stage: The self-attention mechanism is used to perform temporal modeling on the fusion features generated in S4 to fuse long-distance temporal information, and position embedding is integrated to compensate for the lack of position information;
[0016] S6: Classification head setting stage: Use the Sentence-BERT to encode the interpreted gesture category name to obtain category text features, calculate the similarity between the video features output by S5 and the category text features, apply the softmax function to obtain the gesture category probability distribution, and use the cross entropy loss function for model training.
[0017] Furthermore, the output of the one-dimensional temporal convolution and the GRU is integrated by weighted summation in S2 to combine short-term temporal information and long-term temporal dependencies, which can be expressed as follows:
[0018] Y time =α(t)·GRU(X)+β(t)·Conv1D(X)+γ·R(X)
[0019] Where X is the spatial feature input extracted by S1, Conv1D represents a one-dimensional temporal convolution operation, GRU represents a gated recurrent unit operation, R(X) is a regularization term used to control model complexity and prevent overfitting; γ is a coefficient that controls the influence of regularization; α(t) and β(t) are time-dependent dynamic weight functions, and t is a time point in the sequence.
[0020] Furthermore, the weight function and parameters in the formula are defined as:
[0021]
[0022] Among them, λ1 and λ2 are learnable parameters used to adaptively adjust the contribution weights of GRU and one-dimensional temporal convolution.
[0023] Furthermore, the prompt word instruction in S3 is "describe the hand posture of the teacher in the image".
[0024] Furthermore, the self-attention mechanism in S5 is a multi-head attention mechanism (Multi-HeadAttention, MHA), and the position embedding integration expression is:
[0025]
[0026] Among them, F fusion (i) represents the fused feature vector of the i-th frame, PE(i,P) represents the position encoding based on the frame position i, and P is the position parameter.
[0027] Furthermore, the similarity s is calculated in S6 i The formula is:
[0028]
[0029] Among them, Yvideo is the video feature vector output by S5, is the text feature vector of the i-th gesture category, N is the total number of categories, and ‖·‖ represents the vector norm.
[0030] Furthermore, the softmax function and the cross entropy loss function are applied in S6, and the expression is:
[0031]
[0032] Among them, L is the cross entropy loss, y i is the true label indicator function, when the sample belongs to the i-th category y i =1, otherwise 0.
[0033] Furthermore, the pre-processing in S1 includes adjusting the video spatial resolution to H K ×W K =224×224 pixels.
[0034] Furthermore, the method further includes performing data cleaning on the video data before S1, including removing invalid frames and filtering noise.
[0035] Furthermore, the method is applied to online teaching quality assessment, smart classroom behavior modeling, or teacher teaching ability training scenarios, analyzing the depth of teacher interaction or optimizing teaching expressiveness through gesture recognition.
[0036] The beneficial effects of the present invention are:
[0037] (1) A generative model generates text descriptions of gestures frame by frame, deeply binding visual features with natural language semantics and leveraging the contextual reasoning capabilities of natural language to accurately capture the interaction intent in teaching scenarios. For example, for gestures like “pointing to the blackboard” and “pointing to the students,” which are visually similar but semantically different, the text descriptions supplement key contextual information such as “the teacher’s finger is pointing toward the formula on the blackboard” or “the teacher’s palm is pointing toward the students.” This allows the model to break through the limitations of low-level vision and significantly improve the accuracy of distinguishing ambiguous gestures.
[0038] (2) Combining one-dimensional temporal convolution with gated recurrent units to simultaneously capture the instantaneous details and long-range semantic associations of gestures:
[0039] Short-term temporal convolution extracts inter-frame micro-motions (such as the emphasis signal formed by rapid finger shaking);
[0040] The long-term gated recurrent unit models cross-period action logic (such as a complete command gesture consisting of "raise hand → pause → press down").
[0041] The two are adaptively fused through dynamic weights, enabling the model to not only recognize transient gesture changes but also understand the deep intentions of continuous teaching actions.
[0042] (3) Breaking through the limitations of isolated labels in traditional one-hot classification, an innovative semantic embedding classification head is adopted:
[0043] Using Sentence-BERT to encode category name text (e.g., "Invite to raise your hand" and "Request to be quiet"), the model learns the semantic similarity between "invite" and "request" and avoids incorrectly separating related gestures.
[0044] By measuring the distance between video features and categorical text features using cosine similarity, the feature space distribution of similar gestures is forced to be more compact. This effectively reduces the intra-class variation in "praise" gestures caused by differences in teacher style, while completely resolving the problem of misclassification uniformity caused by traditional methods that ignore semantic associations.
[0045] (4) Video-text features are spliced together to form a fusion representation and build a bidirectional enhancement mechanism:
[0046] When visual features are ambiguous (e.g., hands are occluded), text descriptions provide semantic supplements for key actions;
[0047] When the text description is ambiguous (e.g., “raising your hand” may correspond to multiple intentions), visual dynamic features constrain the semantic space.
[0048] The synergistic effect of the two significantly improves the model's generalization ability in scenes with changing lighting, occlusion interference, and similar backgrounds.
[0049] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:
[0051] Figure 1 This is a schematic diagram of the present invention;
[0052] Figure 2 This is the confusion matrix diagram of the common one-hot classification method.
[0053] Figure 3 This is the confusion matrix diagram of the method in this paper. DETAILED DESCRIPTION
[0054] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0055] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.
[0056] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0057] See also Figure 1 This is a natural language-assisted teacher gesture recognition method, characterized by the following steps: This method is suitable for scenarios such as online teaching quality assessment (analyzing the depth of interaction in remote teaching through gesture recognition, such as whether gestures enhance knowledge transfer), smart classroom behavior modeling (in smart classrooms, real-time analysis of the relationship between teacher gestures and teaching links, such as "emphasis gestures" corresponding to key knowledge explanations), and teacher teaching ability training (based on gesture recognition feedback, guiding teachers to optimize the standardization and expressiveness of gestures and enhance the teaching appeal). This method focuses on natural language-assisted teacher gesture recognition and solves technical problems: high similarity in teacher gesture backgrounds, intra-class differences in teacher gestures, and inter-class similarity.
[0058] Step 1: The teacher gesture recognition task uses video data as the research object and uses the CLIP model based on contrastive learning to extract the spatial features of the teacher gesture. Given a teacher gesture video, where T is the number of frames and the spatial resolution is H K =WK =224. The extracted spatial feature is F video ∈R T×N1 First, T represents the number of frames and N1 represents the feature dimension.
[0059] Step 2: Use one-dimensional temporal convolution to capture short-term temporal information. Use GRU to capture long-range dependencies in sequence data, which can retain and extract long-term contextual information. Through weighted summation, the integration of these two methods combines temporal features of different scales, enabling the model to fully understand time series data. Formally, the mathematical expression of this time mining is:
[0060] Y time =α(t)·GRU(X)+β(t)·Conv1D(X)+γ·R(X)
[0061] Where, and is a time-dependent dynamic weight function, and λ1 and λ2 are learned parameters that enable the model to adaptively adjust the contribution of the GRU and the one-dimensional temporal convolution based on the time point t in the sequence. γ is a coefficient that controls the influence of regularization. R(x) is a regularization term used to control model complexity and prevent overfitting.
[0062] Step 3: Using GPT-4o’s image-to-text generation capabilities, we input the prompt “Describe the teacher’s hand gesture in the image” and generate T descriptions of the teacher’s hand gestures corresponding to the frame. We then use the sentence-level embedding model Sentence-BERT to encode the text to obtain high-dimensional features describing the text.
[0063] Step 4: Obtain the teacher's gesture spatial features and frame-level teacher gesture text description features The final fusion feature is formed by splicing
[0064] Step 5: The multi-head attention mechanism calculates attention weights from multiple perspectives in parallel, effectively fusing long-range temporal information. Position embedding is used to integrate it into the fusion feature matrix to compensate for the lack of inherent position differentiation ability of the self-attention mechanism. Formally, the calculation of the fused and position-encoded video features can be enhanced by the expression:
[0065]
[0066] Where MHA stands for multi-head attention mechanism. F represents the feature vector of the i-th frame in the fusion feature matrix. fusion (i) represents the position encoding based on the position P of frame i in the sequence.
[0067] Step 6: First, for each category name definition, use Sentence-BERT to map the category name definition text into a high-dimensional feature matrix Y class ∈R N×D Among them, Y class The nth row of n , represents the text feature of the category name of the nth category. Finally, the cross entropy loss function is used to convert the final video feature Y video and N category name text features Y class ∈R N×D , calculate the cosine similarity, apply Softmax, and then calculate the cross entropy loss, which is mathematically expressed as:
[0068]
[0069] Among them, y i is the true label and L is the cross entropy loss.
[0070] like Figure 2 and Figure 3 The confusion matrix diagram of Figure 2 As shown in , label misclassification presents an approximately uniform distribution. This can be attributed to the high similarity of background features between different gesture categories. In addition, one-hot labels ignore the potential relationships between categories, making it difficult for the model to effectively extract the key differences between categories. Therefore, the misclassification of the model is approximately randomly distributed. Taking the "inviting students to raise their hands gesture" as an example, the traditional one-hot method ignores the intrinsic relationship between categories, which often causes it to be misclassified into significantly different gesture categories. Categories with low classification accuracy, such as "praise" and "explanation" gestures, show significant intra-class variation due to individual differences in teachers' expression styles. This feature poses a major challenge to traditional models in accurately classifying complex and dynamic features. As Figure 3 As shown, the accuracy of these two gesture categories is significantly improved by the method of the present invention. This improvement can be attributed to the head network design proposed in this chapter, which can effectively learn and generalize the individual variations in these gestures, thereby significantly improving classification accuracy and overall performance. In addition, the integration of a multi-level temporal decoder into the backbone network significantly improves the recognition accuracy of gestures that rely on temporal context, such as "praise" and "raise hand." These findings further demonstrate the effectiveness of the proposed invention.
[0071] Example 1: Online teaching quality assessment application
[0072] 1.1 Data collection and preprocessing
[0073] Obtain the teacher's lecture video through the classroom recording system, cut out a 30-second segment, and adjust the resolution to 224×224 pixels;
[0074] Automatically filter invalid frames caused by lens occlusion and dark lighting, and retain valid gesture segments;
[0075] 1.2 Multimodal Feature Extraction
[0076] Use the CLIP model to extract visual features frame by frame to capture the hand's spatial posture;
[0077] Generate frame-level text descriptions through GPT-4o: "Left hand with five fingers open pointing to the projection screen", "Right hand with a fist clenched to emphasize key concepts";
[0078] Sentence-BERT encodes text into 768-dimensional semantic vectors;
[0079] 1.3 Spatiotemporal Fusion Analysis
[0080] Splicing visual and text features to form a fusion matrix;
[0081] One-dimensional temporal convolution captures the instantaneous emphasis action of “clicking the screen three times in a row”;
[0082] GRU models the complete questioning process of "raising hand to signal → pausing and waiting → pointing at students";
[0083] 1.4 Teaching Effect Mapping
[0084] Calculate the semantic similarity between video features and teaching behavior categories such as "questioning", "emphasizing", and "feedback";
[0085] Output gesture-teaching behavior correlation heat map to identify the period when gesture enhances knowledge transfer;
[0086] Automatically generate assessment reports: emphasize that gestures cover 80% of key knowledge points, and questioning gestures lead to increased student response rates.
[0087] Example 2: Smart Classroom Behavior Modeling Application
[0088] 2.1 Real-time video stream processing
[0089] Deploy edge computing devices to capture teacher gestures at a rate of 5 frames per second;
[0090] Dynamic segmentation gesture unit: from hand raising to action reset;
[0091] 2.2 Context-Adaptive Recognition
[0092] When detecting interaction with teaching aids, GPT-4o generates the text "Holding a set square to demonstrate geometric theorem";
[0093] For visually similar gestures, text descriptions are used to distinguish between “tapping on the blackboard to emphasize” and “lightly touching the electronic screen to turn the page”;
[0094] Position coding relates the spatial correspondence between gestures and the writing area on the blackboard;
[0095] 2.3 Analysis of teaching stages
[0096] The self-attention mechanism identifies the transition features of the "concept explanation-example demonstration-classroom practice" stage;
[0097] The dynamic weight function α(t)) strengthens the long-term dependency modeling during the introduction period of the new course;
[0098] Output teaching behavior map: the concept explanation stage mainly uses pointing gestures, and the classroom practice stage mainly uses patrolling gestures;
[0099] 2.4 Visualization of inter-class relationships
[0100] The semantic embedding classification head establishes the correlation between "guiding thinking" and "inspiring discussion".
[0101] Example 3: Teacher teaching ability training application
[0102] 3.1 Multi-dimensional feature extraction
[0103] The CLIP model captures visual features such as gesture amplitude, direction, and stability;
[0104] GPT-4o generates description text: "Insufficient arm swing" and "Finger tremors affect the clarity of pointing."
[0105] 3.2 Dual-Scale Motion Evaluation
[0106] Temporal convolution detects micro-movement defects: blurred instructions caused by finger tremors;
[0107] GRU analysis of macro-motion coherence: the decomposition of demonstration movements is not clear;
[0108] The regularization term R(X) suppresses overfitting and avoids misjudging personal habits as errors;
[0109] 3.3 Semantic Feedback Generation
[0110] Compare the similarity between the actual gesture features and the text embedding of the "standard demonstration gesture";
[0111] Output improvement suggestions: Increase the expansion range of the invitation gesture to enhance interactivity;
[0112] Visualize the distribution of intra-class features and compress the range of variation caused by teachers' personal styles;
[0113] 3.4 Training Effect Verification
[0114] Continuously collect data on teachers’ improved gestures;
[0115] Verify the improved expressiveness through the diagonal enhancement of the confusion matrix;
[0116] Generate training report: Emphasize the improvement of gesture intention clarity.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.
Claims
1. A natural language-assisted teacher gesture recognition method, characterized by: The following steps are involved: S1: Video feature extraction stage: Data cleaning and preprocessing of the teacher gesture video dataset, and spatial feature extraction using the Contrastive Language–Image Pre-training (CLIP) model based on contrastive learning pre-training Where T is the number of video frames and N1 is the spatial feature dimension; S2: Video feature temporal mining stage: The spatial features extracted in S1 are processed using a one-dimensional temporal convolution operation to capture short-term temporal information and local temporal associations, and the spatial features are modeled using a gated recurrent unit (GRU) to extract long-term temporal dependencies and cross-period contextual information. S3: Text feature extraction stage: Generative Pre-trained Transformer 4 Optimized (GPT-4o) is used to generate text descriptions of the teacher's hand posture frame by frame based on the prompt word instructions, and Sentence-Bidirectional Encoder Representations from Transformers (Sentence-BERT) is used to encode the text descriptions to obtain text features. Where N2 is the text feature dimension; S4: Video-text fusion stage: The spatial feature F extracted by S1 video and the text features F extracted by S3 text Splicing to form fusion features S5: Fusion feature temporal mining stage: The self-attention mechanism is used to perform temporal modeling on the fusion features generated in S4 to fuse long-distance temporal information, and position embedding is integrated to compensate for the lack of position information; S6: Classification head setting stage: Use the Sentence-BERT to encode the interpreted gesture category name to obtain category text features, calculate the similarity between the video features output by S5 and the category text features, apply the softmax function to obtain the gesture category probability distribution, and use the cross entropy loss function for model training.
2. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: In S2, the output of one-dimensional temporal convolution and GRU is integrated by weighted summation to combine short-term temporal information and long-term temporal dependencies. The formula is expressed as: Y time =α(t)·GRU(X)+β(t)·Conv1D(X)+γ·R(X) Where X is the spatial feature input extracted by S1, Conv1D represents a one-dimensional temporal convolution operation, GRU represents a gated recurrent unit operation, R(X) is a regularization term used to control model complexity and prevent overfitting; γ is a coefficient that controls the influence of regularization; α(t) and β(t) are time-dependent dynamic weight functions, and t is a time point in the sequence.
3. The natural language-assisted teacher gesture recognition method according to claim 2, characterized in that: The weight function and parameters in the formula are defined as: Among them, λ1 and λ2 are learnable parameters used to adaptively adjust the contribution weights of GRU and one-dimensional temporal convolution.
4. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The prompt word instruction in S3 is "describe the hand posture of the teacher in the image".
5. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The self-attention mechanism in S5 is a multi-head attention mechanism (Multi-HeadAttention, MHA), and the position embedding integration expression is: Among them, F fusion (i) represents the fused feature vector of the i-th frame, PE(i,P) represents the position encoding based on the frame position i, and P is the position parameter.
6. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The similarity s is calculated in S6 i The formula is: Among them, Y video is the video feature vector output by S5, is the text feature vector of the i-th gesture category, N is the total number of categories, and ‖·‖ represents the vector norm.
7. The natural language-assisted teacher gesture recognition method according to claim 6, characterized in that: In S6, the softmax function and the cross entropy loss function are applied, and the expression is: Among them, L is the cross entropy loss, y i is the true label indicator function, when the sample belongs to the i-th category y i =1, otherwise 0.
8. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The preprocessing in S1 includes adjusting the video spatial resolution to H K ×W K =224×224 pixels.
9. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The method further includes performing data cleaning on the video data before S1, including removing invalid frames and filtering noise.
10. The natural language-assisted teacher gesture recognition method according to claim 1, characterized in that: The method is applied to online teaching quality assessment, smart classroom behavior modeling, or teacher teaching ability training scenarios, and can analyze the depth of teacher interaction or optimize teaching expressiveness through gesture recognition.