Intelligent classroom student posture key point detection method based on improved YOLOv8-Pose

By introducing CBAM attention module, multi-scale detection head and SIoU loss function into the motion key point detection algorithm, the detection problems in small targets and complex environments are solved, the detection accuracy and robustness are improved, and it is suitable for real-time use in smart classrooms Behavior monitoring.

CN118918341APending Publication Date: 2024-11-08CHONGQING NORMAL UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410890965.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing motion key point detection algorithm is insufficient in accuracy and robustness in small object detection and complex environments, especially in highly dynamic and rapidly changing scenarios, making it difficult to effectively identify and track subtle motions and multiple objects. Interaction.

Method used

The CBAM lightweight attention module, multiple scale detection heads, cross-level cascade fusion and improved SIoU loss function are introduced, combined with new iterative training strategies, to enhance the network's detection ability of small targets and perception ability of complex scenarios.

Benefits of technology

It improves the performance of motion key points detection in small goals and complex scenarios, enhances the recognition rate of students and teachers' behavior, improves detection accuracy and robustness, and is suitable for real-time monitoring and analysis in smart classrooms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918341A_ABST
    Figure CN118918341A_ABST
Patent Text Reader

Abstract

The invention discloses a smart classroom student posture key point detection method based on improved YOLOv8-Pose, and relates to the technical field of smart education, and the method comprises the following steps: S1, introducing a CBAM lightweight attention module which is used for small target detection and complex scene perception; the sensitivity to small target individuals is improved; a main mathematical derivation process of the CBAM is provided; according to the invention, a CBAM lightweight attention module is introduced to enhance the attention of the network on a detection target, so that the sensitivity to individuals is improved, and a multi-scale detection head is adopted, so that the algorithm can comprehensively detect a plurality of individuals with different sizes in a complex image, especially a human body in the image; according to the method, the recognition rate of students and teachers in class in the smart classroom can be enhanced, recognition perception is enhanced, the method can better cooperate with modern science and technology education to understand the classroom learning state of the students and the class behaviors of the teachers, application and development in classrooms of middle and primary schools in various provinces can be promoted, and the method has practical and theoretical significance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart education technology, and in particular to a method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose. Background Art

[0002] In today's era of rapid technological development, the research on smart classrooms has increasingly become a focus in the academic and industrial fields. This research field is committed to enabling smart classrooms to more comprehensively and accurately identify and perceive the surrounding environment through multiple sensors from the front, back and top of the classroom to obtain more information. As a prominent and key issue in the research of smart classrooms, motion key point detection directly involves the key nodes of perceiving human motion and the environment. Motion key point detection technology can be widely used in smart classrooms to help educators monitor and analyze students' behaviors in real time, thereby improving teaching effectiveness. Through motion key point detection technology, smart classrooms can identify students' postures and movements, such as whether they are listening carefully, actively answering questions or dozing off. This technology can capture students' images through cameras and use deep learning algorithms to identify and track human key points in the image. Through motion key point detection, we can monitor students' head, eyes, arms and body postures to determine whether students are actively participating in classroom activities. For example, a student's head down may mean that he is reading or taking notes, while a student's eyes closed and body relaxed may mean that he is dozing off. Real-time monitoring of these behaviors can help teachers intervene in time to improve classroom interaction and attention. The application of motion key point detection technology in smart classrooms can provide educators with powerful tools to understand and analyze students' behaviors and states in real time, thereby improving teaching effectiveness and students' learning experience. With the continuous advancement of technology, the application prospects of motion key point detection in smart classrooms will be broader, bringing more innovation and changes to the field of education.

[0003] However, there are challenges in small object detection, which limits the performance of small people with denser perception. This includes the capture and analysis of subtle movements. When dealing with these small objects, current methods have not yet reached the ideal level, and this limitation may have a certain impact on the precision analysis. Secondly, when faced with the challenges of complex environments, the detection of motion key points becomes more challenging in highly dynamic and rapidly changing scenes. This may involve complex situations such as interactions between multiple objects, changes in lighting conditions, and occlusions, making it difficult for traditional motion key point detection algorithms to achieve satisfactory results in such situations.

[0004] In these scenarios, current algorithms may show low accuracy and robustness. Therefore, the goal of this study is to solve the problem of small target detection and motion keypoint detection in complex environments through innovative methods. By overcoming these limitations, we aim to enhance the detection and analysis of student behavior and teacher behavior in smart classrooms, and be better able to adapt to complex environments and detect small targets. In the past, there were two main approaches to pose estimation: top-down and bottom-up. The top-down method is a keypoint detection paradigm that starts with the overall target and gradually refines the keypoint localization. The whole is detected first, and then the keypoints in the target area are precisely located. This method performs well in scenarios involving multiple targets, especially in applications such as human pose estimation. Initially, the top-down method uses advanced object detectors such as Faster R-CNN or YOLO to identify the target area in the image. This provides basic information for subsequent keypoint localization, including the location and confidence score of the target. Next, in the keypoint localization stage, a specific keypoint localization network (such as the hourglass network) is used to predict the keypoints in the target area with high accuracy. Finally, post-processing and optimization steps are used to ensure the accuracy and robustness of the detection results. The advantage of the top-down approach is that it can adapt to situations where multiple goals need to be processed simultaneously, providing strong support for the execution of complex tasks.

[0005] In contrast, the bottom-up approach represents another paradigm in keypoint detection. Its notable feature is to directly detect all possible keypoints in the entire image and then associate these points to form a complete object. The bottom-up approach directly processes the entire image and is suitable for scenes that require dense keypoint detection, providing an effective solution for complex and dense detection tasks. First, in the keypoint detection stage, a dense keypoint detector like OpenPose is used to directly process the entire image to provide a keypoint estimate for each pixel. Subsequently, through the association and merging stages, adjacent keypoints are connected to form parts of the human body or other objects, resulting in a complete set of keypoints. Finally, the pose estimation and filtering steps evaluate the various poses formed and select the most suitable one. The advantage of the bottom-up approach is that it directly processes the entire image, making it very suitable for dense and complex scenes, thus providing an effective solution for systems that require dense keypoint detection tasks. However, both top-down and bottom-up methods have common limitations in the field of motion keypoint detection. First, they face challenges in small object detection because both methods may be limited in perceiving small-sized objects. It can be relatively difficult for these methods to capture and analyze fine-grained motion, and they also face challenges in handling complex scenes. In highly dynamic and rapidly changing environments, bottom-up approaches can be disrupted by factors such as interactions between multiple objects, changes in lighting conditions, and occlusions. These complexities make it difficult for traditional motion keypoint detection algorithms to achieve ideal performance in such scenes, which can lead to reduced accuracy and robustness of detection methods in these challenging environments.

[0006] In order to solve the above limitations, a smart classroom student posture key point detection method based on improved YOLOv8-Pose is proposed to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose to solve the problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention provides a method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose, comprising the following steps:

[0009] S1. Introduce the CBAM lightweight attention module: used for small target detection and complex scene perception; improve sensitivity to small target individuals; and provide the main mathematical derivation process of CBAM: the channel attention calculation formula used to obtain the channel feature weight distribution is as follows:

[0010] M c=σ(FC(AvgPool(X))+FC(MaxPool(X)))

[0011] Among them, M c Represents the output of channel attention, which is used to adjust the weight of channel features. This formula calculates the feature representation of channel attention; the spatial attention calculation formula used to obtain the weight distribution of spatial positions is as follows:

[0012] M s =σ(FC(AvgPool(X))+FC(MaxPool(X)))

[0013] Among them, M s Represents the output of spatial attention, which is used to adjust the weight of spatial position; this formula calculates the feature representation of spatial attention; the formula for combining channel and spatial attention through element multiplication is as follows:

[0014]

[0015] This formula combines channel attention and spatial attention using element-by-element multiplication to produce a comprehensive attention feature map; the attention feature map generation formula is as follows:

[0016] A=Conv(M)

[0017] This formula generates the final attention feature map through convolution operation, which is used to adjust the channel and spatial information in the input feature map; the formula for generating the feature map after applying attention is as follows:

[0018]

[0019] This formula multiplies the input feature map by the attention feature map element to obtain the feature map after attention is applied;

[0020] The formula used to generate the final output feature map is as follows:

[0021] Y = Conv(ReLU(BN(S)))

[0022] This formula generates the final output feature map through convolution, rectified linear unit (ReLU) and batch normalization (BN) operations for subsequent tasks;

[0023] S1.1. Use multiple scale detection heads: This enables the algorithm to comprehensively detect multiple individuals of different sizes in complex images, especially human bodies in images;

[0024] S2. Add cross-layer cascade fusion: It is used to further enhance the feature fusion between shallow and deep networks and reduce the missed detection rate of small target individuals;

[0025] S3. Improvement of loss function: SIoU loss function: The loss function CIOU is modified and replaced, and the bounding box regression positioning loss function of the algorithm is redefined with the modified SIOU;

[0026] S3.1. Through a series of experiments, we verify the excellent performance of YOLOv8-PoseRoom in motion key point detection in small objects and complex scenes.

[0027] S4. New iterative training strategy: In the first stage, the learning rate is initialized with a specific value (1e-4), and then the learning rate is reduced when each milestone is reached; next, the best model obtained in the previous stage is used as the pre-trained model to initialize the parameters, reset the learning rate, and restart the training from a specific epoch, and then run the rest of the process.

[0028] Furthermore, in S2, specifically in YOLOv8 PoseRoom, two cross-layer communication channels are introduced between the backbone network and the neck network to enhance the feature fusion capability between the shallow and deep networks; the feature map fusion formula with different channel numbers is as follows:

[0029] M i =Concat(B i ,C i ,A i ).

[0030] Furthermore, in S3, the CIoU loss is replaced by the SIoU loss, which includes four components: angle cost, distance cost, shape cost, and IoU cost;

[0031] The formula for angle cost calculation is:

[0032]

[0033] The height difference between the centers of the two boxes is:

[0034]

[0035] The width difference is:

[0036]

[0037] If the angle α is greater than 45°, the remaining angles α are used, considering the angle with the y-axis instead of the x-axis. Next is the calculation of the distance cost, which represents the distance between the center points of the predicted bounding box and the ground truth bounding box. Combined with the angle cost defined above, SIoU redefines the distance cost as shown in formula (A):

[0038]

[0039] γ=2-Λ(D);

[0040] Formulas (B), (C), and (D) involve C W and C H The term represents the width and height of the minimum bounding box of the ground truth and prediction box; when α is close to 0, the contribution of the distance cost is significantly reduced. On the contrary, when α is close to π / 4, the contribution of the distance cost increases. As the angle increases, γ is assigned a time-first distance value;

[0041] The definition of shape cost is shown in formula (E):

[0042]

[0043] (w,h) and (W gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ controls the degree of attention paid to shape loss;

[0044] The SIoU loss function is shown in formula (H):

[0045]

[0046] Furthermore, in S3.1, the experiments use two widely used datasets to perform the model's motion keypoint detection and pose estimation tasks. These two datasets provide different scenes and image samples; the two widely used datasets include the COCO dataset and the MPII human pose dataset.

[0047] Furthermore, in S3.1, the operating system of the experimental device used for the experiment is based on Ubuntu 20.04.3 LTS, and the laboratory used for the experiment includes multiple A6000 and 3090 card sets; Python 3.8.8 is selected as the main programming language, and most importantly, PyTorch 1.10.0 is used as the deep learning framework for implementing and training the smart classroom motion key point detection and posture estimation model.

[0048] Furthermore, in S3.1, the state-of-the-art pose estimation model is selected as the baseline model for performance comparison and evaluation.

[0049] Furthermore, the COCO dataset has a total of 50,125 samples, of which the validation set consists of 5,815 samples and the test set contains 5,815 samples; the MPII human pose dataset includes 30,225 samples, of which 4,728 samples are in the validation set and 4,218 samples are in the test set; for image data, normalization is used to standardize its size to the same resolution; the key points in the dataset are annotated in detail, including the key points of the body, hands, and face; data augmentation techniques for increasing data diversity are used, including random rotation, mirror flipping, scaling, and translation; the dataset is divided into training, validation, and test sets for model validation and performance evaluation; dedicated data loading tools and libraries are used, including PyTorch's DataLoader.

[0050] Furthermore, in S4, the model performs approximately 18.7 GFLOPS of floating-point operations per second; regarding the hyperparameter settings for model training, the following configurations are adopted: the initial learning rate (lr0) is 0.01, which starts with a relatively small learning rate; the final learning rate is set to 0.2, and the learning rate is gradually increased; the momentum of the stochastic gradient descent is set to 0.927; the weight decay is configured as weight_decay=0.0005 to control model complexity and reduce overfitting; a warm-up strategy of 3 epochs is also used to gradually increase the learning rate, with a total of 300 training epochs; in addition, the input image size is set to 640×640.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The CBAM lightweight attention module is introduced to enhance the network's attention to the detection target, thereby improving the sensitivity to individuals. Multiple scale detection heads are used to enable the algorithm to comprehensively detect multiple individuals of different sizes in complex images, especially the human body in the image. Cross-level cascade fusion is also added to further enhance the feature fusion between shallow and deep networks and reduce the missed detection rate of target individuals. Finally, the loss function CIOU is modified, and the bounding box regression positioning loss function of the algorithm is redefined with the modified SIOU, which accelerates the convergence of model training and improves the detection accuracy. At the same time, in order to overcome the problem of long training time for lightweight models, a new iterative training strategy is proposed to give full play to the potential of YOLOv8-PoseRoom. Through a series of experiments, the excellent performance of YOLOv8-PoseRoom in motion key point detection in small targets and complex scenes is verified. This method can enhance the recognition rate of students and teachers in smart classrooms, enhance the perception of recognition, and better cooperate with modern scientific and technological education to understand students' classroom learning status and teachers' classroom behavior. It can promote the application and development in primary and secondary school classrooms in various provinces and regions, and has practical and theoretical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a calculation diagram of the SIoU loss function in a smart classroom student posture key point detection method based on improved YOLOv8-Pose in the present invention;

[0054] Figure 2 This is a performance comparison chart of the methods on COCO and MPII datasets in a smart classroom student posture key point detection method based on improved YOLOv8-Pose of the present invention;

[0055] Figure 3 This is a model performance comparison diagram on the MPII dataset in a smart classroom student posture key point detection method based on improved YOLOv8-Pose of the present invention;

[0056] Figure 4 This is a comparison chart of model performance on the COCO dataset in a smart classroom student posture key point detection method based on improved YOLOv8-Pose of the present invention;

[0057] Figure 5 A comparison chart of model parameters and floating-point operations on COCO and MPII datasets in a method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose of the present invention;

[0058] Figure 6 This is a diagram of ablation experiment results on COCO and MPII datasets in a smart classroom student posture key point detection method based on improved YOLOv8-Pose of the present invention;

[0059] Figure 7 This is an ablation experiment diagram on COCO and MPII datasets in a smart classroom student posture key point detection method based on improved YOLOv8-Pose in the present invention;

[0060] Figure 8 A method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose is presented in the present invention to verify the YOLOv8-PoseRoom diagram in a real scene;

[0061] Fig. 9 The present invention provides a smart classroom student posture key point detection method based on improved YOLOv8-Pose to verify the YOLOv8-PoseRoom diagram in the classroom scene. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Top-down pose estimation methods are a widely used method for motion keypoint detection. In this field, there are several well-known top-down methods, including CPN (Cascaded Pyramid Network), Hourglass Network, CPM (Convolutional Pose Machine) and Alpha Pose. These methods have achieved remarkable achievements in motion keypoint detection. First, CPN adopts a cascaded pyramid structure to gradually refine the position information of motion keypoints through hierarchical pyramid feature extraction to improve accuracy. This method has strong robustness and is suitable for complex scenes and small target detection. Secondly, the hourglass network is a classic top-down method characterized by its symmetrical encoding and decoding structure, which can accurately locate motion keypoints at different scales. The hourglass network is often used for human pose estimation, and its multi-level feature extraction helps to handle complex motion keypoint detection tasks. CPM is another notable top-down method that utilizes a multi-level convolutional network, with each stage focusing on refining the localization of motion keypoints. This hierarchical structure helps to improve accuracy and the ability to handle complex pose detection. Finally, AlphaPose is a top-down method based on deep learning, which is known for its high accuracy and robustness. It adopts a bottom-up strategy to first detect candidate key points of body parts, and then generate the final pose through association and filtering. This method has achieved outstanding results in the field of human pose estimation.

[0064] These top-down methods play an important role in motion key point detection in smart classrooms, provide strong support for complex image detection, and show excellent performance in small object detection and complex scenes.

[0065] Research based on bottom-up pose estimation methods also plays a vital role in keypoint detection. Below, I will introduce five bottom-up models widely used in this field: First, OpenPose is a well-known bottom-up method that uses convolutional neural networks to directly detect all possible keypoints in an image, and then forms a complete pose by associating these points. The advantage of OpenPose lies in its dense keypoint detection, making it suitable for applications that require high-density pose information. Second, associative embedding is a bottom-up method that combines scattered keypoint information into a complete pose by learning the relationship between keypoints based on the bottom-up concept. This method is good at handling occlusion and interaction between multiple targets. CPN-CRF is a method that combines cascaded pose networks and conditional random fields (CRF) to achieve motion keypoint detection through cascaded networks and graph models. This combination enhances the accuracy and robustness of the model, especially for complex scenes. DeepPose is a deep learning method that directly regresses the positions of motion keypoints by training a neural network. Although it is a direct regression method, it has achieved significant success in motion keypoint detection, especially when dealing with small targets. SimpleBaseline is a bottom-up approach that uses a deep neural network that can predict the positions of all key points simultaneously. This approach simplifies the model structure while maintaining high accuracy and efficiency. These bottom-up methods provide powerful motion key point detection tools for the application and development of smart classrooms, allowing us to perform well in complex scenes and scenes involving occluded objects.

[0066] Pose estimation based on YOLO (You Only Look Once) is an emerging direction in this field and has attracted widespread research interest. With the rapid development of deep learning, significant progress has been made in the field of pose estimation, while traditional methods rely heavily on manually designed features and models. However, the pose estimation method based on YOLO introduces real-time performance, enabling the detection and tracking of motion keypoints at high frame rates, which provides us with higher efficiency and accuracy in tasks that require fast response. In addition, YOLO's end-to-end design makes it directly applicable to multimodal sensor data such as images, depth maps, and infrared images, thereby enhancing the perception capabilities of the device. Its innovative technologies, including multi-scale detection heads and attention mechanisms, enhance the performance of small target detection, making it more suitable for complex scenes. In summary, the pose estimation method based on YOLO represents the latest progress in motion keypoint detection in smart classrooms, providing a powerful solution for a variety of application scenarios. Future research directions may focus on further improving accuracy, enhancing robustness, and extending its applicability to a wider range of smart classroom promotion tasks, promoting the application of smart classrooms under the modern education system.

[0067] The YOLO-Pose model is built on top of the popular YOLOv8

[17] object detection algorithm, leveraging its efficient object detection capabilities. Specifically, YOLO-Pose uses CSP-darknet 53 as the backbone network for feature extraction. This network architecture performs well in image feature extraction and helps capture rich semantic information. In addition, to better handle multi-scale information, YOLO-Pose introduces PANet (Path Aggregation Network) as the neck to fuse features of different scales. This multi-scale feature fusion helps the model fully understand the image content, thereby improving the accuracy and robustness of pose estimation. The network structure of YOLO-Pose also includes four decoupled heads of different scales for simultaneously predicting candidate boxes and key points. This decoupled head design allows the model to perform motion key point detection at different scales to accommodate objects of various sizes. This is crucial for our smart classrooms because we may need to handle targets of different distances and sizes, such as students of different sizes in front and behind. In summary, YOLO-Pose combines the object detection capabilities of YOLOv 8, the feature extraction capabilities of CSP-darknet 53, the multi-scale feature fusion of PANet, and the design of the multi-scale decoupled head to build a powerful pose estimation model. This model has a wide range of application potential in motion key point detection in smart classrooms and can solve the challenges in complex scenes and small object detection.

[0068] See also Figure 1-Figure 9 , the present invention provides a technical solution:

[0069] See also Figure 1-Figure 9 As shown, a method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose includes the following steps:

[0070] S1. Introducing the CBAM lightweight attention module: used for small target detection and complex scene perception; improving sensitivity to small target individuals; small target detection is a challenging problem, especially in applications such as smart classrooms that need to identify student behavior in large classrooms, which may be a limiting factor. CBAM enhances the network's attention to small targets, making it more sensitive to subtle motion details, thereby enhancing small target detection performance. In addition, complex scene perception involves challenges such as interactions between multiple objects, changes in lighting conditions, and occlusions, which traditional methods may have difficulty handling in these scenarios. CBAM helps the model better understand complex scenes by improving feature focus, thereby enhancing the model's motion key point detection performance in complex environments; and provides the main mathematical derivation process of CBAM: The channel attention calculation formula used to obtain the channel feature weight distribution is as follows:

[0071] M c=σ(FC(AvgPool(X))+FC(MaxPool(X)))

[0072] Among them, M c Represents the output of channel attention, which is used to adjust the weight of channel features. This formula calculates the feature representation of channel attention; the spatial attention calculation formula used to obtain the weight distribution of spatial positions is as follows:

[0073] M s =σ(FC(AvgPool(X))+FC(MaxPool(X)))

[0074] Among them, M s Represents the output of spatial attention, which is used to adjust the weight of spatial position; this formula calculates the feature representation of spatial attention; the formula for combining channel and spatial attention through element multiplication is as follows:

[0075]

[0076] This formula combines channel attention and spatial attention using element-by-element multiplication to produce a comprehensive attention feature map; the attention feature map generation formula is as follows:

[0077] A=Conv(M)

[0078] This formula generates the final attention feature map through convolution operation, which is used to adjust the channel and spatial information in the input feature map; the formula for generating the feature map after applying attention is as follows:

[0079]

[0080] This formula multiplies the input feature map by the attention feature map element to obtain the feature map after attention is applied;

[0081] The formula used to generate the final output feature map is as follows:

[0082] Y = Conv(ReLU(BN(S)))

[0083] This formula generates the final output feature map through convolution, rectified linear unit (ReLU) and batch normalization (BN) operations for subsequent tasks;

[0084] S1.1. Use multiple scale detection heads: This enables the algorithm to comprehensively detect multiple individuals of different sizes in complex images, especially human bodies in images;

[0085] S2. Add cross-level cascade fusion: It is used to further enhance the feature fusion between shallow and deep networks and reduce the missed detection rate of small target individuals. Cross-level cascade fusion aims to overcome the limitations of feature information flow in traditional models. By establishing connections between features at different layers, more comprehensive and effective information exchange can be achieved.

[0086] S3. Improvement of loss function: SIoU loss function: The loss function CIOU was modified and replaced, and the bounding box regression positioning loss function of the algorithm was redefined with the modified SIOU, which accelerated the convergence of model training and improved the detection accuracy;

[0087] S3.1. Through a series of experiments, we verify the excellent performance of YOLOv8-PoseRoom in motion key point detection in small objects and complex scenes.

[0088] S4. New iterative training strategy: In the first stage, the learning rate is initialized with a specific value (1e-4), and then the learning rate is reduced when each milestone is reached; next, the best model obtained in the previous stage is used as the pre-trained model to initialize the parameters, reset the learning rate, and restart the training from a specific epoch, and then run the rest of the process.

[0089] Furthermore, in S2, specifically in YOLOv8 PoseRoom, two cross-layer communication channels are introduced between the backbone network and the neck network to enhance the feature fusion capability between the shallow and deep networks; this innovative approach strengthens information exchange, ensures the effective extraction of motion key point layer features, and helps prevent information loss; the feature map fusion formula with different channel numbers is as follows:

[0090] M i =Concat(B i ,C i ,A i );

[0091] By cross-layer fusing the original player silhouette features extracted from the shallow network and the refined silhouette features extracted from the deep network, the information exchange between the shallow and deep features is enhanced, which allows the network to selectively extract feature information and solve problems such as missed detection and false detection caused by the original network's reliance on a single source of fused features; therefore, this enhancement leads to improved prediction accuracy.

[0092] Furthermore, in S3, SIoU loss is used instead of CIoU loss. SIoU loss includes four components: angle cost, distance cost, shape cost and IoU cost. The parameters used in SIoU loss function are as follows: Figure 1 As shown;

[0093] The formula for angle cost calculation is:

[0094]

[0095] The height difference between the centers of the two boxes is:

[0096]

[0097] The width difference is:

[0098]

[0099] If the angle α is greater than 45°, the remaining angles α are used, considering the angle with the y-axis instead of the x-axis. Next is the calculation of the distance cost, which represents the distance between the center points of the predicted bounding box and the ground truth bounding box. Combined with the angle cost defined above, SIoU redefines the distance cost as shown in formula (A):

[0100]

[0101] γ=2-Λ(D);

[0102] Formulas (B), (C), and (D) involve C W and C H The term represents the width and height of the minimum bounding box of the ground truth and prediction box; when α is close to 0, the contribution of the distance cost is significantly reduced. On the contrary, when α is close to π / 4, the contribution of the distance cost increases. As the angle increases, γ is assigned a time-first distance value;

[0103] The definition of shape cost is shown in formula (E):

[0104]

[0105] (w,h) and (W gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ controls the degree of attention paid to shape loss;

[0106] The SIoU loss function is shown in formula (H):

[0107]

[0108] Furthermore, in S3.1, the experiment uses two widely used datasets to perform the model's motion keypoint detection and pose estimation tasks. These two datasets provide different scenes and image samples, providing a rich source of data for evaluating the algorithm; the two widely used datasets include the COCO dataset and the MPII human pose dataset:

[0109] COCO Dataset (Common Objects in Context): COCO is a large multimodal dataset widely used for computer vision tasks; it contains more than one million images covering a variety of scenes and environments; the COCO dataset is known for its diversity and richness, including annotations for tasks such as image description, object detection, and human keypoint estimation; in the experiments, a portion of the COCO dataset was used, focusing mainly on the task of human pose estimation; these images include samples of human subjects in different poses, along with corresponding annotations for human keypoints; this makes the COCO dataset ideal for evaluating motion keypoint detection and pose estimation algorithms;

[0110] MPII Human Pose Dataset: The MPII Human Pose Dataset is specially designed for human pose estimation research; it contains more than 25,000 images, including images of single or multiple people captured in various indoor and outdoor scenes; each image is annotated with key points corresponding to body parts (such as head, shoulders, elbows, wrists, hips, knees, and ankles); the MPII Human Pose Dataset is widely used in the field of human pose estimation; in the experiments, this dataset is used to verify the performance of the algorithm in specific scenarios and compare its results with those of other datasets; the two datasets provide rich image samples and related annotations, covering different scenarios and poses; they provide strong support for the experimental evaluation of pose estimation algorithms, and can fully understand their performance in different environments.

[0111] Furthermore, in S3.1, the operating system of the experimental device used for the experiment is based on Ubuntu 20.04.3 LTS, which is a wide choice for deep learning and computer vision research and provides a powerful development and execution environment. In order to accelerate the training and reasoning of deep learning models, the laboratory used for the experiment includes multiple A6000 and 3090 card groups, which constitute powerful computing power and large memory capacity; and Python 3.8.8 is selected as the main programming language, benefiting from its rich deep learning library and experimental development tool ecosystem. Most importantly, PyTorch 1.10.0 is used as a deep learning framework for implementing and training smart classroom motion key point detection and posture estimation models; the comprehensive setting of the experimental environment provides strong computing support for this method, ensuring the credibility of the experiment and the accuracy of the results.

[0112] Furthermore, in S3.1, several classic and state-of-the-art pose estimation models are selected as baseline models for performance comparison and evaluation; these baseline models include:

[0113] OpenPose (2017): OpenPose is a classic multi-person pose estimation model that can detect key points of multiple people, including body, hand, and facial key points; it uses a multi-level convolutional neural network architecture with high accuracy and robustness;

[0114] AlphaPose (2017): AlphaPose is an advanced multi-person pose estimation model that uses a joint optimization method to estimate the poses of multiple people simultaneously; it performs well in complex scenes and provides strong multi-person pose estimation performance;

[0115] SPPE (Single-Person Pose Estimation) (2017): SPPE is a model specifically designed for single-person pose estimation, which achieves high-precision pose estimation by predicting the coordinates of each key point; it is suitable for application scenarios that require high-precision single-person pose estimation, such as motion analysis and human behavior recognition;

[0116] DensePose (2018): DensePose is an advanced pose estimation model that not only detects key points of the human body, but also densely annotates the human body surface; it maps the human body surface onto a two-dimensional image so that each pixel corresponds to a specific point on the human body surface, providing more detailed pose information;

[0117] SimpleBaseline (2018): SimpleBaseline is a concise and efficient pose estimation model that improves spatial resolution by adding deconvolution layers based on ResNet. Despite its simple structure, it performs well in multi-person pose estimation tasks and has high detection accuracy.

[0118] HigherHRNet-W32 (2019): HigherHRNet is a high-resolution pose estimation model with higher spatial resolution and improved keypoint detection performance; the W32 version was selected for performance comparison;

[0119] HRNet (2019): HRNet (High-Resolution Network) is a high-resolution pose estimation model that improves the accuracy and robustness of key point detection by maintaining high-resolution representation and continuously performing multi-scale fusion. HRNet is suitable for pose estimation tasks in complex scenes.

[0120] DarkPose (2020): DarkPose is a pose estimation model based on high-resolution feature maps. It improves the accuracy and robustness of key point detection by using high-resolution feature maps and non-maximum suppression algorithms. It is suitable for multi-person pose estimation tasks in complex scenes.

[0121] MMPose (2020): MMPose is a deep learning-based pose estimation framework that integrates multiple advanced pose estimation algorithms; it provides a flexible modular design, supports multiple pose estimation tasks, and has good scalability and ease of use;

[0122] YOLO-Pose-640 (2021): This is a multi-person pose estimation model based on YOLOv3, known for its real-time performance;

[0123] YOLO-Pose-960 (2021): This is an upgraded version of YOLO-Pose-640 with a higher input resolution to improve keypoint detection accuracy;

[0124] Lite-HRNet (2021): Lite-HRNet is a lightweight high-resolution pose estimation model designed for mobile devices and embedded systems; it significantly reduces the computation and model size while maintaining high-resolution features, making it ideal for real-time applications;

[0125] PRN (Pose Residual Network) (2021): PRN is a pose estimation model that improves the accuracy of key point detection through residual learning and multi-scale feature fusion; it performs well in pose estimation in complex scenes and is suitable for a variety of application scenarios;

[0126] YOLOv7-w6-pose (2022): This is a pose estimation model based on YOLOv7 with smaller model size and faster inference speed, suitable for real-time applications;

[0127] RTMpose (2022): RTMpose is a novel pose estimation model that integrates the Recursive Temporal Module (RTM) into the pose estimation framework, enabling the model to capture temporal dependencies and improve the accuracy of pose estimation over time.

[0128] DWpose (2022): DWpose is another advanced pose estimation model that uses a deep weighted network (DWN) to dynamically adjust the importance of different body regions during pose estimation; this adaptive weighting mechanism enhances the model’s ability to focus on relevant body parts and improves the overall pose estimation accuracy.

[0129] By selecting these different baseline models, the performance of the proposed YOLOv8-PoseRoom algorithm can be comprehensively compared and evaluated.

[0130] Furthermore, the COCO dataset has a total of 50,125 samples, of which the validation set consists of 5,815 samples and the test set contains 5,815 samples; the MPII human pose dataset includes 30,225 samples, of which 4,728 samples are in the validation set and 4,218 samples are in the test set; for image data, normalization is used to standardize its size to the same resolution to ensure the stability of the model under different input image sizes; in addition, the key points in the dataset are annotated in detail, including the key points of the body, hands and face; these annotations play a vital role in model training; data augmentation techniques for increasing data diversity are utilized, including random rotation, mirror flipping, scaling, and translation; the dataset is divided into training set, validation set, and test set for model verification and performance evaluation; dedicated data loading tools and libraries, including PyTorch's DataLoader, are used to efficiently load and process data, ensuring the high data quality and availability of the motion key point detection and pose estimation algorithms of this model; this rigorous data preprocessing environment provides a reliable data foundation for the experiments of this method and ensures the credibility and stability of the experimental results.

[0131] Furthermore, in S4, the model performs about 18.7 GFLOPS of floating-point operations per second, showing strong computing power; and 300 trainings were performed, resulting in a model weight of 30MB; regarding the hyperparameter settings for model training, the following configurations were adopted: the initial learning rate (lr0) was 0.01, which started with a relatively small learning rate to promote rapid convergence in the early stages of training; the final learning rate was set to 0.2, and the learning rate was gradually increased to fine-tune the model in the later stages; the momentum of stochastic gradient descent was set to 0.927, which helped to accelerate the convergence of the model; weight decay was configured as weight_decay=0.0005 to control model complexity and alleviate overfitting; a 3-epoch warm-up strategy was also used to gradually increase the learning rate, with a total of 300 training epochs; in addition, the input image size was set to 640×640 to accommodate the architecture and task requirements of the model; these hyperparameters were carefully adjusted to ensure that the model effectively learned the characteristics of the data and achieved good performance; the selection of these parameters was empirically optimized during the training process to obtain the best experimental results.

[0132] It should be noted that the classical evaluation indicators widely used in object detection tasks are mainly used to comprehensively evaluate the performance of the proposed smart classroom motion key point detection method. Specifically, the following key evaluation indicators are focused on:

[0133] AP^50 is an evaluation index in object detection and key point detection. Its full name is "Average Precision at IoU = 0.50". It represents the average precision when the intersection over union (IoU) is equal to 0.50. The specific explanation is as follows:

[0134] IoU (Intersection over Union): is an indicator to evaluate the degree of overlap between the detection box and the actual annotation box. The calculation method is to divide the intersection area of ​​the detection box and the actual annotation box by their union area;

[0135] IoU=0.50: indicates that the IoU value is 0.50, that is, the overlapping area of ​​the detection box and the annotation box accounts for at least half of the total area;

[0136] AP (Average Precision): is an indicator that comprehensively evaluates the performance of the model at different thresholds; AP50 represents the average precision when IoU is 0.50 at different confidence thresholds;

[0137] The formula is given by:

[0138]

[0139] |C| is the number of object classes, Ri is the set of detected bounding boxes for class i, Pi is the set of ground truth bounding boxes for class i, Precision(Ri,Pi,0.5) is the precision of class i at 50% IoU, Recall(Ri,Gi,0.5) is the recall of class i at 50% IoU;

[0140] AP^75 is another evaluation indicator in object detection and key point detection. Its full name is "AveragePrecision at IoU=0.75". It represents the average precision when the intersection over Union (IoU) is equal to 0.75. It provides a more rigorous evaluation of model performance. The formula is:

[0141]

[0142] The meanings of the variables are the same as those in AP50;

[0143] Objects are usually divided into three categories according to their size: small (S), medium (M), and large (L); Average AP^M: Average Precision (Medium) or model performance focusing on medium-sized objects; the formula is defined as:

[0144]

[0145] Where AP(Ri,Pi,Medium) represents the average precision of medium-sized objects of class i;

[0146] AP^L (Average Precision for Large objects) refers to the average accuracy of the model when detecting large objects. AP^L focuses on the detection performance of large objects. The formula is given by:

[0147]

[0148] Where AP(Ri,Pi,Large) represents the average precision of large-size objects of class i;

[0149] like Figure 2 As shown in the figure, the performance of this method is compared with multiple methods on the COCO and MPII datasets. On the COCO dataset, this method shows a significant advantage over OpenPose. The model in this method achieves 3.81, 6.93, 4.41, and 3.99 percentage point improvements in AP^50, AP^75, AP^M, and AP^L, respectively. Compared with other competing methods, this method also shows high performance. For example, relative to HigherHRNet-W32, the AP^50 in this method is improved by 1.51 percentage points, the AP^75 is improved by 5.43 percentage points, the AP^M is improved by 2.21 percentage points, and the AP^L is improved by 1.79 percentage points; relative to Lite-HRNet, the AP^50 in this method is improved by 1.06 percentage points, the AP^75 is improved by 3.83 percentage points, the AP^M is improved by 0.71 percentage points, and the AP^L is improved by 0.29 percentage points.

[0150] On the MPII dataset, this method also performs well; compared with OpenPose, the model of this method improves by 3.82, 6.91, 5.40, and 3.02 percentage points in AP^50, AP^75, AP^M, and AP^L, respectively. Compared with other methods, the performance of this method shows significant enhancement. For example, compared with HigherHRNet-W32, the AP^50 of this method is improved by 2.42 percentage points, the AP^75 is improved by 6.21 percentage points, the AP^M is improved by 3.60 percentage points, and the AP^L is improved by 1.72 percentage points; compared with Lite-HRNet, the AP^50 of this method is improved by 1.79 percentage points, the AP^75 is improved by 5.63 percentage points, the AP^M is improved by 4.12 percentage points, and the AP^L is improved by 2.72 percentage points. Figure 3 and Figure 4The contents of the table are shown, clearly demonstrating the performance advantages of the present method on the COCO and MPII datasets; these experimental results verify the effectiveness and robustness of the proposed motion key point detection method.

[0151] Figure 5 A comparison of parameter counts (PARAMS) and floating point operations (FLOPs) of different models on COCO and MPII datasets is given. The purpose of this table is to compare models in terms of model complexity and computational resource consumption. On COCO dataset, the OpenPose model has 7.11 million parameters and 10.08 billion FLOPs, the AlphaPose model has 6.92 million parameters and 9.98 billion FLOPs, the HigherHRNet-W32 model has 6.85 million parameters and 9.78 billion FLOPs, the YOLO-Pose-640 model has 7.45 million parameters and 10.38 billion FLOPs, the YOLO-Pose-960 model has 7.65 million parameters and 10.58 billion FLOPs, the YOLOv7-w6-pose model has 7.35 million parameters and 10.28 billion FLOPs, while the present model has a relatively low parameter count of 4.92 million but still maintains 9.10 billion FLOPs. The PARAMS and FLOPs of each model on the MPII dataset also vary. Compared with the COCO dataset, the PARAMS and FLOPs of the MPII dataset are generally lower, but the differences still exist. Our model has 4.75 million parameters and 8.90 billion FLOPs on the MPII dataset. Although its parameter count is lower compared to other models, it still maintains a relatively high floating point operation. Our model has a relatively low model complexity and computational resource consumption, but it is still competitive in performance. This shows that our model is effective and feasible for the task of motion keypoint detection. In practical applications, this will help reduce hardware resource requirements and improve the usability of the model.

[0152] like Figure 6Results of ablation experiments conducted on motion keypoint detection methods are presented, aiming to investigate the impact of different model components on the performance. Specifically, two model components, CBAM and CCFU, are introduced and combined with the baseline model to evaluate their performance on the COCO and MPII datasets. On the COCO dataset, the baseline model (Experiment 1) achieves a performance of 83.70 AP^50, 57.80 AP^50-95, 58.00 AP^M, and 72.60 AP^L. Subsequently, the CBAM component is introduced (Experiment 2), which slightly improves the performance, increasing the AP^50 to 84.70, the AP^50-95 to 58.40, the AP^M to 59.90, and the AP^L to 73.90. Next, by introducing the CCFU component (Experiment 3), the performance is further improved, reaching 85.20 AP^50, 59.00 AP^50-95, 60.30 AP^M, and 72.70 AP^L. Finally, the CBAM and CCFU components are introduced at the same time (Experiment 4), and the performance reaches the highest, with AP^50 of 85.41, AP^50-95 of 59.13, AP^M of 60.11, and AP^L of 74.19. On the MPII dataset, the performance of the baseline model (Experiment 1) is 81.52, 55.62, 56.82, and 70.42, respectively. With the introduction of CBAM (Experiment 2), CCFU (Experiment 3), and both CBAM and CCFU (Experiment 4), the performance gradually improves. AP^50 reaches 82.52, 83.02, and 83.22, respectively; AP^50-95 reaches 56.22, 56.82, and 58.62, respectively; AP^M reaches 57.72, 58.12, and 58.92, respectively; and AP^L reaches 71.72, 70.52, and 71.72, respectively. From the results of these ablation experiments, we can see the positive impact of CBAM and CCFU components on model performance. Especially on the COCO dataset, their introduction significantly improves the key point detection performance, indicating that these components play a vital role in enhancing the effectiveness of motion key point detection algorithms. These results further consolidate the advantages of this method and provide strong support for key point detection in applications.

[0153] like Figure 7We further explore the impact of different intersection-over-union (IoU) loss functions (CIoU, GIoU, DIoU, and SIoU) on the performance of motion keypoint detection methods. First, on the COCO dataset, the performance of each loss function can be observed. The CIoU loss function (Experiment 1) has an AP^50, AP^50-95, AP^M, and AP^L of 81.98, 56.08, 56.28, and 70.88, respectively. (Experiment 2) significantly improves the performance of all indicators, with values ​​of 82.98, 56.68, 58.18, and 72.18. (Experiment 3) continues to improve the performance, reaching AP^50, AP^50-95, AP^M, and AP^L of 83.48, 57.28, 58.58, and 71.98, respectively. (Experiment 4) shows the best performance, with AP^50 of 85.40, AP^50-95 of 58.48, APM of 60.20, and AP^L of 60.10. On the MPII dataset, a similar trend is observed. The AP^50, AP^50-95, AP^M, and AP^L of (Experiment 1) are 79.80, 53.90, 54.10, and 68.70, respectively. (Experiment 2) shows a significant improvement in performance, with AP^50, AP^50-95, APM, and APL values ​​of 80.80, 54.50, 55.00, and 69.50, respectively. DIoU (Experiment 3) continues to improve the performance, with AP^50, AP^50-95, AP^M, and AP^L of 81.30, 55.10, 55.40, and 69.30, respectively. The SIoU loss function (Experiment 4) once again achieved the best performance, with AP^50 of 83.24, AP^50-95 of 56.93, AP^M of 58.92, and AP^L of 71.04. Overall, the SIoU loss function shows the best performance on both COCO and MPII datasets. These experimental results emphasize the effectiveness of the SIoU loss function in model motion keypoint detection, providing strong support for this method, which is closely related to the topic of this study and highlights the importance of loss function selection in applications.

[0154] like Figure 8 As shown, it is clear that the model shows excellent performance in the various operational scenarios studied. The figure shows that the model successfully captures targets of different sizes and motion states in real motion scenes, demonstrating excellent recognition and tracking capabilities. Whether in densely populated environments or in complex motion backgrounds, the model shows excellent robustness and accuracy. This excellent performance is of great significance for practical applications and provides strong support for perception capabilities in complex scenes. Fig. 9 To verify the model performance in real classroom scenarios.

[0155] in conclusion:

[0156] In this study, the YOLOv8-PoseRoom model is introduced and a series of experiments are conducted to evaluate its performance on the motion keypoint detection task. The experimental results show that YOLOv8-PoseRoom significantly improves motion keypoint detection compared to traditional methods, especially in detecting small objects and complex scenes. The model enhances object sensitivity, accelerates model training convergence, and improves detection accuracy by introducing the CBAM attention mechanism module, multi-scale detection head, and SIoU bounding box regression with localization loss function, thus providing better perception and execution capabilities for practical applications. Although the YOLOv8-PoseRoom model achieves encouraging results in various aspects, it has limitations that need further exploration and improvement. Among them, its performance in detecting extremely small objects, although improved, still faces challenges, especially in capturing and analyzing the tiny details of motion. In addition, although the model shows robustness in complex scenes, its effectiveness may be reduced in scenes characterized by high-density interactions and overlapping objects, which highlights the need for enhanced detection capabilities in crowded environments. In addition, the model's adaptability to different and changing environments, such as fluctuations in lighting, weather conditions, and backgrounds, remains to be fully tested and optimized. Another key area is the model's real-time processing capabilities, especially on low-power devices, which is critical for applications that require immediate responses. To address the outstanding limitations, the goal is to introduce advanced attention mechanisms and feature fusion methods to improve the model's perception capabilities, especially in dealing with occluded and densely interacting objects. Through these focused efforts, it is expected that significant progress will be made in perfecting the YOLOv8-PoseRoom model, ensuring its higher effectiveness and applicability in the field of smart education.

Claims

1. A method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose, characterized in that: The following steps are involved: S1. Introduce the CBAM lightweight attention module: used for small target detection and complex scene perception; improve sensitivity to small target individuals; and provide the main mathematical derivation process of CBAM: the channel attention calculation formula used to obtain the channel feature weight distribution is as follows: M c =σ(FC(AvgPool(X))+FC(MaxPool(X))); Among them, M c Represents the output of channel attention, which is used to adjust the weight of channel features. This formula calculates the feature representation of channel attention; The spatial attention calculation formula used to obtain the weight distribution of spatial positions is as follows: M s =σ(FC(AvgPool(X))+FC(MaxPool(X))) Among them, M s Represents the output of spatial attention, which is used to adjust the weight of spatial position; this formula calculates the feature representation of spatial attention; the formula for combining channel and spatial attention through element multiplication is as follows: This formula combines channel attention and spatial attention using element-wise multiplication to produce a comprehensive attention feature map; The attention feature map generation formula is as follows: A=Conv(M) This formula generates the final attention feature map through convolution operation, which is used to adjust the channel and spatial information in the input feature map; the formula for generating the feature map after applying attention is as follows: This formula multiplies the input feature map by the attention feature map element to obtain the feature map after attention is applied; The formula used to generate the final output feature map is as follows: Y = Conv(ReLU(BN(S))) This formula generates the final output feature map through convolution, rectified linear unit (ReLU) and batch normalization (BN) operations for subsequent tasks; S1.

1. Use multiple scale detection heads: This enables the algorithm to comprehensively detect multiple individuals of different sizes in complex images, especially human bodies in images; S2. Add cross-layer cascade fusion: It is used to further enhance the feature fusion between shallow and deep networks and reduce the missed detection rate of small target individuals; S3. Improvement of loss function: SIoU loss function: The loss function CIOU is modified and replaced, and the bounding box regression positioning loss function of the algorithm is redefined with the modified SIOU; S3.

1. Through a series of experiments, we verify the excellent performance of YOLOv8-PoseRoom in motion key point detection in small objects and complex scenes. S4. New iterative training strategy: In the first stage, the learning rate is initialized with a specific value (1e-4), and then the learning rate is reduced when each milestone is reached; next, the best model obtained in the previous stage is used as the pre-trained model to initialize the parameters, reset the learning rate, and restart the training from a specific epoch, and then run the rest of the process.

2. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S2, specifically in YOLOv8PoseRoom, two cross-layer communication channels are introduced between the backbone network and the neck network to enhance the feature fusion capability between the shallow and deep networks; the feature map fusion formula with different channel numbers is as follows: M i =Concat(B i ,C i ,A i )。 3. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S3, the CIoU loss is replaced by the SIoU loss, which consists of four components: angle cost, distance cost, shape cost, and IoU cost; The formula for angle cost calculation is: The height difference between the centers of the two boxes is: The width difference is: If the angle α is greater than 45°, the remaining angles α are used, considering the angle with the y-axis instead of the x-axis. Next is the calculation of the distance cost, which represents the distance between the center points of the predicted bounding box and the ground truth bounding box. Combined with the angle cost defined above, SIoU redefines the distance cost as shown in formula (A): γ=2-Λ(D); Formulas (B), (C), and (D) involve C W and C H The entries of represent the width and height of the minimum bounding box of the ground truth and prediction box; When α is close to 0, the contribution of the distance cost decreases significantly. On the contrary, when α is close to π / 4, the contribution of the distance cost increases. As the angle increases, γ is assigned a time-preferred distance value. The definition of shape cost is shown in formula (E): (w,h) and (W gt ,h gt ) are the width and height of the predicted box and the real box respectively, and θ controls the degree of attention paid to shape loss; The SIoU loss function is shown in formula (H):

4. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S3.1, the experiments use two widely used datasets for the model's motion keypoint detection and pose estimation tasks, which provide different scenes and image samples; the two widely used datasets include the COCO dataset and the MPI I human pose dataset.

5. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S3.1, the operating system of the experimental device used for the experiment is based on Ubuntu 20.04.3LTS, and the laboratory used for the experiment includes multiple A6000 and 3090 card sets; Python 3.8.8 is selected as the main programming language, and most importantly, PyTorch 1.10.0 is used as the deep learning framework for implementing and training the smart classroom motion key point detection and posture estimation model.

6. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S3.1, the state-of-the-art pose estimation models are selected as baseline models for performance comparison and evaluation.

7. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 4, characterized in that: The COCO dataset has a total of 50,125 samples, of which the validation set consists of 5,815 samples and the test set contains 5,815 samples; the MPII human pose dataset includes 30,225 samples, of which 4,728 samples are in the validation set and 4,218 samples are in the test set; for image data, normalization is used to standardize its size to the same resolution; the key points in the dataset are annotated in detail, including the key points of the body, hands, and face; Use data augmentation techniques to increase data diversity, including random rotation, mirror flipping, scaling, and translation; Split the dataset into training, validation, and test sets for model validation and performance evaluation; use dedicated data loading tools and libraries, including PyTorch’s DataLoader.

8. The method for detecting key points of student postures in a smart classroom based on improved YOLOv8-Pose as claimed in claim 1, characterized in that: In S4, the model performs approximately 18.7 GFLOPS of floating-point operations per second; regarding the hyperparameter settings for model training, the following configurations are adopted: the initial learning rate (lr0) is 0.01, which starts with a relatively small learning rate; the final learning rate is set to 0.2, and the learning rate is gradually increased; the momentum of the stochastic gradient descent is set to 0.927; the weight decay is configured as weight_decay=0.0005 to control model complexity and alleviate overfitting; a warm-up strategy of 3 epochs is also used to gradually increase the learning rate, with a total of 300 training epochs; in addition, the input image size is set to 640×640.

Citation Information

Cited By

  • Physical education evaluation system based on artificial intelligence

    CN120339004A