A school classroom-based method and system for recognizing children's actions
By integrating the prediction algorithms and logical judgment models of GCN and TCN, the accuracy problem of children's action recognition in school classroom scenarios is solved, achieving high-precision classification and recognition of children's actions, which is suitable for occluded environments in school classrooms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINAN PRESCHOOL TEACHERS COLLEGE
- Filing Date
- 2025-01-16
- Publication Date
- 2026-05-19
AI Technical Summary
Existing action recognition methods exhibit poor generalization performance under specific surveillance camera conditions such as kindergartens and primary schools. In particular, in school classrooms, due to the large differences in children's height, severe occlusion, and the influence of multiple camera monitoring perspectives, conventional algorithms struggle to accurately identify children's fine movements.
A novel prediction algorithm integrating GCN and TCN is adopted. The human skeleton sequence data is processed by spatial logic judgment and temporal logic judgment models respectively, and then fused with the prediction results of GCN and TCN networks. The logic judgment guides the model output, thereby improving the accuracy of action recognition.
The model achieves high-precision classification and recognition of children's movements in a school classroom setting, effectively handling movement recognition under occlusion conditions and improving the model's recognition performance in a school classroom setting.
Smart Images
Figure CN119888860B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of action recognition technology, and in particular relates to a method and system for recognizing children's actions in a school classroom. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] Human motion recognition and prediction is an important research direction in the field of computer vision. Human skeleton-based motion recognition and prediction has broad application prospects in fields such as intelligent transportation, intelligent security, human-computer interaction, and motion analysis.
[0004] Currently, significant progress has been made in the research of human motion recognition algorithms under normal conditions. However, under specific surveillance camera conditions such as kindergartens and primary schools, there are significant obstructions in classrooms and other scenes from the perspective of campus surveillance. For example, when children are sitting in chairs, the desks may obstruct their lower bodies. In addition, there is a significant difference in height between children and adults. Conventional human motion recognition algorithms are usually trained based on adult data, so they cannot be well applied to special scenarios such as classrooms from the perspective of campus surveillance.
[0005] Meanwhile, conventional human motion recognition based on deep learning methods has become mainstream, with existing techniques mainly focusing on two directions: RGB images and human joint sequences. Compared to RGB image sequence-based methods, methods based on graph convolutional networks (GCNs) for human skeleton modeling exhibit stronger robustness to occlusion and complex scenes. Furthermore, processing joint data consumes fewer computational resources than video streams, resulting in better real-time performance when handling multi-channel video surveillance in practical applications.
[0006] The application of Generic Networks (GCNs) in action recognition has evolved from simple graph representations based on skeletal points to the introduction of Spatiotemporal Convolutional Networks (ST-GCNs). ST-GCNs treat skeleton sequences as graph structures with joints as nodes and connections between joints as edges. They learn features between nodes through spatial graph convolution and model temporal information through temporal graph convolution, thus extracting action features more comprehensively. With further research, explorations based on GCNs have focused on two main directions: first, more representative skeleton data construction methods; and second, more effective GCN model design.
[0007] From the first perspective, the AS-GCN network uses a multi-task learning strategy to output pose predictions, capturing the dependencies between joints through two modules (Actional Links and Structural Links). However, module collaboration leads to soaring computational costs, resulting in inefficiency when processing large-scale, long-sequence skeleton data and making it difficult to meet real-time requirements. On the other hand, it is overly dependent on the distribution of task data; if the features of the actual scene data are biased, the accuracy of pose prediction will be severely affected. 2s-AGCN combines an adaptive topology graph, using the backpropagation algorithm of the neural network for automatic updates to increase the representation of joint connection strength, thereby capturing and enhancing richer feature representations. However, it is sensitive to the initial value of backpropagation; poor initial values can easily lead to local optima, hindering the ideal construction of the topology graph. At the same time, when the training samples are limited, complex topology updates can easily cause overfitting, resulting in poor generalization performance. MS-G3D, on the other hand, constructs a unified spatiotemporal graph, which consists of multiple subgraphs, each representing the spatial relationship of joints in a certain frame, effectively simulating the relationship between different joints in different coordinate systems. However, in the process of constructing and processing spatiotemporal graphs, as the number of joints and frames increases, the complexity of the graph structure increases dramatically, and the storage and computational overhead increases exponentially. At the same time, the weight allocation strategy when different subgraphs are merged lacks an adaptive mechanism, making it difficult to fit diverse action patterns, resulting in inaccurate characterization of some key joint relationships.
[0008] From the second perspective, traditional GCNs, operating as feedforward networks, limit lower-level access to higher-level semantic information. To address this issue, Feedback Graph Convolutional Networks (FGCNs) emerged. Unlike directly applying complete skeleton sequences, FGCNs employ a multi-level temporal sampling strategy to sparsely extract input segment sequences from the skeleton data, aiming to gradually acquire global spatiotemporal features. However, in the sample selection stage, they lack rigorous screening criteria, easily introducing noisy segments and missing keyframes, interfering with feature learning. Furthermore, the feedback control parameters between layers are complex, making parameter tuning difficult and resulting in poor system stability. STSF-GCNs conceptualize skeleton data, resembling a unified spatiotemporal topology. While adequate for handling routine actions, they struggle to quickly adapt to the topology adjustments required by irregular or sudden actions due to their overly structured nature and lack of flexibility.
[0009] Transformer-based networks are commonly used for a wide range of language tasks, such as text classification, machine translation, or question answering, but they are now being extended to vision tasks, such as image classification, object detection, segmentation, image restoration, and point cloud registration.
[0010] Currently, significant progress has been made in the field of action recognition using the Transformer architecture. Transformer-based methods effectively alleviate the problem of focusing only on local information and excel at capturing extensive dependencies in long sequences. When applied to tasks involving skeleton-based human action recognition, the Transformer architecture demonstrates proficiency in capturing temporal relationships. For example, PRTR (Pose Recognition with Cascade Transformers) focuses on human pose recognition and introduces a cascaded Transformer architecture with two forms. Human action recognition methods based on the HRNet-dual-stream Transformer network, which fuse multiple features into a dual-stream network model, improve the accuracy of human action recognition.
[0011] However, the Transformer architecture also has some shortcomings in action recognition. Due to limitations in capturing and encoding the inherently complex high-dimensional semantic information in skeleton data, its effectiveness in modeling spatial relationships remains limited. Furthermore, when processing long sequences, the complexity of its self-attention mechanism leads to significant computational resource consumption, restricting its practical application.
[0012] Therefore, the main shortcomings of existing action recognition methods in specific surveillance camera conditions such as kindergartens and primary schools are: 1. The scarcity of children's training data and the significant obstruction in classrooms and other school scenes under surveillance view cause neural networks trained on conventional public datasets to exhibit poor generalization performance in this scenario, thus limiting the deployment and application of action recognition in school scenes.
[0013] 2. Currently, human motion recognition algorithms based on graph convolutional networks are mainly applied in conventional scenarios. However, children's heights differ significantly from adults, and there are also large height differences within a class. Furthermore, in places like schools, multiple cameras monitor the data source. Unlike ordinary ground cameras, these cameras often have a downward, oblique view, and people are generally densely packed within the monitored area. Additionally, obstructions from desks and other objects in classrooms often limit the view to the upper body of children. Therefore, current human motion recognition algorithms based on graph convolutional networks are essentially motion recognition models trained on adult eye-level data, exhibiting poor generalization ability in special scenarios like school campuses.
[0014] 3. In classroom settings, most movements are small, making it difficult for conventional action recognition models to make detailed distinctions. Summary of the Invention
[0015] To overcome the shortcomings of the prior art, this invention provides a method and system for recognizing children's actions in a school classroom. It is a fusion prediction algorithm model that can achieve high-precision classification and recognition of children's actions in a school classroom setting.
[0016] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0017] The first aspect of this invention provides a method for recognizing children's actions in a school classroom.
[0018] A method for recognizing children's actions in a school classroom, comprising:
[0019] Obtain human skeleton sequence data;
[0020] The human skeleton sequence data is input into the GCN network and the TCN network respectively, and prediction results based on the GCN network and the TCN network are obtained respectively.
[0021] Meanwhile, the spatial and temporal actions were derived from the human skeleton sequence data using spatial logic judgment models and temporal logic judgment models, respectively.
[0022] The derived spatial and temporal actions are fused with the prediction results based on GCN and TCN networks to obtain the final action pose.
[0023] A child motion recognition system based on school classrooms, comprising:
[0024] The data acquisition module is configured to acquire human skeleton sequence data;
[0025] The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively;
[0026] The logic judgment module is configured to simultaneously deduce spatial and temporal actions from the human skeleton sequence data through spatial logic judgment model and temporal logic judgment model, respectively.
[0027] The fusion output module is configured to fuse the derived spatial and temporal actions with the prediction results based on GCN and TCN networks to obtain the final action pose.
[0028] A third aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a method as described in the first aspect of the present invention.
[0029] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of a method as described in the first aspect of the present invention.
[0030] The fifth aspect of the present invention provides a computer program product containing instructions that, when run on a computer, cause the computer program to perform the steps of a method as described in the first aspect of the present invention when executed by a processor.
[0031] The above one or more technical solutions have the following beneficial effects:
[0032] This invention trains the model on a publicly available dataset and then fine-tunes it on a dataset of classroom-monitored children. The final model prediction results are combined with the logical reasoning results of skeletal joint points. The logical reasoning guides the derivation results of the GCN and TCN modules to achieve higher classification performance for more refined movements and meet the accuracy requirements of the task.
[0033] This invention applies traditional recognition algorithms to a school classroom setting, demonstrating that conventional algorithms suffer from performance degradation when handling fine-grained action classification in situations with occlusion. For example, GCN directly performs inference and prediction from a classroom monitoring perspective, but due to scene and other factors, it often fails to correctly infer the accurate actions of occluded parts, leading to subsequent prediction and inference errors. This invention, by adding spatial logic guidance, can adjust the prediction results of deep models like GCN based on the task scenario, thereby avoiding inference errors caused by occlusion in deep models.
[0034] This invention proposes a novel fusion prediction algorithm based on GCN and TCN. By fusing the spatial and temporal logic of the GCN module and the TCN module respectively, the action recognition module is enhanced, which can be applied to the fine action recognition of children in the school classroom scene where there is occlusion.
[0035] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0037] Figure 1 This is a flowchart of a child action recognition method based on a school classroom, according to Embodiment 1 of the present invention.
[0038] Figure 2 This is a schematic diagram of a child action recognition method based on a school classroom, according to Embodiment 1 of the present invention. Detailed Implementation
[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0040] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.
[0041] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.
[0042] Example 1
[0043] This embodiment discloses a method for recognizing children's actions in a school classroom, including:
[0044] Obtain human skeleton sequence data;
[0045] The human skeleton sequence data is input into the GCN network and the TCN network respectively, and prediction results based on the GCN network and the TCN network are obtained respectively.
[0046] Meanwhile, the spatial and temporal actions were derived from the human skeleton sequence data using spatial logic judgment models and temporal logic judgment models, respectively.
[0047] The derived spatial and temporal actions are fused with the prediction results based on GCN and TCN networks to obtain the final action pose.
[0048] To more clearly illustrate this embodiment, as follows: Figure 1 As shown, a children's action recognition method based on school classrooms mainly involves first training on a publicly available dataset, then fine-tuning it on a dataset of children monitored in classrooms. The final model inference results are combined with the logical inference results of skeletal joint points. Logical judgments guide the inference results of the GCN and TCN modules to achieve higher classification performance on more refined actions, meeting the accuracy requirements of the task. This can be described in detail as follows:
[0049] Step 1: Obtain human skeleton sequence data.
[0050] In this embodiment, obtaining human skeleton sequence data specifically involves obtaining the human joint positions in school classroom video frames and using AlphaPose to obtain the human skeleton sequence data contained in the video frame sequence as model input.
[0051] Step 2: Input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain the prediction results based on the GCN network and the TCN network respectively.
[0052] In this embodiment, the specific steps for constructing the GCN network and the TCN network include:
[0053] Step 2-1: First, pre-train the GCN network and the TCN network using a standard public dataset to obtain the pre-trained model parameters.
[0054] Specifically, the UAV-Human dataset, captured from the perspective of a drone, was selected, and the positions of human joints in the video frames contained in the dataset were obtained. The human skeleton sequence data contained in the video frame sequence was obtained through AlphaPose as the model input.
[0055] Because the UAV-Human dataset has similar angles to those seen from surveillance cameras, it is used as a pre-training dataset for model pre-training. The output is a predicted classification based on the UAV-Human dataset, and in subsequent processes, the classification weights in the prediction header are removed, and the previously used weight parameters are reused.
[0056] (1) Pre-training the GCN network, the specific process is as follows:
[0057] First, the graph data X from the video frame is input, and features are extracted through a graph convolutional layer (Gconv). Then, the graph is coarsened into subgraphs through a pooling layer. Next, the readout layer sums or averages the hidden representations of these subgraphs to summarize the final graph representation. Then, it is processed by a multilayer perceptron (MLP), and finally, the classification result is obtained through a softmax function, which yields the prediction result based on the GCN network.
[0058] (2) Pre-training the TCN network, the specific process is as follows:
[0059] TCN consists of dilated, causal 1D convolutional layers with the same input and output lengths. It includes an input layer, multiple hidden layers, and an output layer, each composed of residual blocks. Each residual block contains weight normalization, ReLU activation, Dropout, and causal convolution operations, and 1x1 convolution operations are used to adjust the dimensions in parts of the residual block.
[0060] Human skeleton sequence data is input into the input layer, and after passing through multiple hidden layers, the prediction result based on the TCN network is obtained through the output layer.
[0061] Step 2-2: Next, use the model parameters obtained after pre-training to construct GCN and TCN network models for classroom behavior prediction.
[0062] Steps 2-3: Obtain labeled data from campus surveillance cameras in kindergartens, primary schools, etc., and further train and fine-tune the model to enhance its recognition and generalization capabilities in this scenario.
[0063] In terms of model training, pre-training accelerates convergence because the pre-trained model has learned general features from large-scale datasets, allowing it to find optimal parameter values more quickly when training on specific downstream tasks. It also addresses the issue of insufficient data by using the general features from the pre-trained model as prior knowledge, reducing the risk of overfitting when training with limited data. Regarding model performance, pre-training improves the model's generalization ability. The general features learned by the pre-trained model on large-scale, diverse datasets help it adapt to new data distributions. Furthermore, it enables the model to learn better feature representations, allowing it to learn high-level abstract features from the data, providing more valuable input features for downstream tasks and thus improving their performance.
[0064] Step 3: Simultaneously, the spatial and temporal actions are derived from the human skeleton sequence data using spatial logic judgment models and temporal logic judgment models, respectively.
[0065] In this embodiment, the specific details of constructing the spatial logic judgment model and the temporal logic judgment model are as follows:
[0066] Step 3-1: First, obtain the results of human judgment in terms of space and time.
[0067] At the spatial level, this involves precise consideration of the relative positions of various joints in the human body. For example, when focusing on the hand joints, if their position is higher than the top of the head (a spatial reference point), this spatial state can be preliminarily interpreted as a tendency to raise the hand. This spatial judgment is not isolated but comprehensively considers the overall posture of the human body and the spatial relationships between different joints. The human body is an organic whole; the position of one joint is often interconnected with other joints, collectively forming specific spatial posture patterns. These patterns provide important clues for movement judgment.
[0068] In the time dimension, tracking the movement trajectory of joint points is crucial. Taking the judgment of a hand-raising action as an example, we not only focus on whether the hand's joint points ultimately reach a position higher than the head, but also need to observe its movement process. That is, the hand's joint points start from an initial position (e.g., below the head), gradually move upwards until they reach the top of the head, and remain in this position above the head for a period of time. This continuous change and sustained state in the time series is a key element in determining whether the action has occurred. By recording the positions of the joint points at different points in time, we can clearly depict the temporal context of the action.
[0069] Step 3-2: Next, the human judgment results in space and time are used as input and passed to a multilayer perceptron (MLP) layer for feature extraction.
[0070] MLP layers possess powerful nonlinear fitting capabilities, enabling deep learning and processing of the spatial and temporal features of the input. Within MLP layers, through multiple neurons and multi-layer network structures, the input features are continuously transformed and abstracted, uncovering potential action patterns and logical relationships.
[0071] Step 3-3: Finally, the features processed by the MLP layer are used to output the logical judgment result through the Softmax function.
[0072] The Softmax function converts the output of the MLP layer into a probability distribution, which can clearly represent the probability of each action category occurring.
[0073] For example, in a scenario involving the judgment of a hand-raising action, the Softmax output provides the probability values of the action belonging to "raising hand" and other possible actions. By comparing these probability values, we can accurately determine whether the action corresponding to the current joint movement is a hand-raising action. The entire model achieves accurate judgment of spatial and temporal action logic through this process from spatial and temporal judgment input to MLP processing and then to Softmax output.
[0074] In this embodiment, the derived spatial and temporal actions specifically include: raising a hand, reading, writing, talking, yawning, and small movements (such as twirling a pen or fiddling with stationery).
[0075] Step 4: Fuse the derived spatial and temporal actions with the prediction results based on GCN and TCN networks to obtain the final action pose.
[0076] In this embodiment, as Figure 2 As shown, the specific steps are as follows:
[0077] Step 4-1: The spatial actions derived from the spatial logic judgment model are fused with the prediction results based on the GCN network through the MOE module via the spatial Logistic function to obtain the spatial fusion result.
[0078] Specifically, let the Logistic function for spatial logical judgment be: Its output is the probability vector of each category to which a spatial action belongs. , represented as:
[0079] ;
[0080] in For the number of spatial action categories, This can be represented as a real vector space with dimensions based on the number of spatial action categories.
[0081] Suppose the node features obtained after GCN processing Represented as:
[0082] ;
[0083] in, It is represented as a 512-dimensional real vector space.
[0084] The output is obtained after passing through a fully connected layer. for:
[0085] ;
[0086] in For the number of spatial action categories, This can be represented as a real vector space with dimensions based on the number of spatial action categories.
[0087] The output value of GCN through the gated network is: After passing through the softmax function, the gated control is obtained. :
[0088] ;
[0089] The output evaluation value of the spatial Logistic function through the gating network is... Its gating weight for:
[0090] ;
[0091] Its fusion spatial output for:
[0092] ;
[0093] Step 4-2: Simultaneously, the time action derived from the time logic judgment model is fused with the prediction results based on the TCN network through the MOE module via the time Logistic function to obtain the time fusion result.
[0094] Specifically, let the Logistic function for time-based logical judgments be: Its output is a probability vector of the spatial action belonging to each category, expressed as:
[0095] ;
[0096] in For the number of time-based action categories, This can be represented as a real vector space with dimensions based on the number of time-based action categories.
[0097] Suppose the node features obtained after TCN processing Represented as:
[0098] ;
[0099] in, It is represented as a 512-dimensional real vector space.
[0100] The output is obtained after passing through a fully connected layer. for:
[0101] ;
[0102] in For the number of time-based action categories, This can be represented as a real vector space with dimensions based on the number of time-based action categories.
[0103] The output valuation of the gated network for the TCN is After passing through the softmax function, the gated control is obtained. :
[0104] ;
[0105] The output evaluation value of the gating network for the spatial logistic function is Its gating weight for:
[0106] ;
[0107] Its fused time output for:
[0108] ;
[0109] Step 4-3: After concatenating the spatial fusion result obtained in Step 4-1 with the temporal fusion result obtained in Step 4-2, input the result into the linear layer to obtain the output of the linear layer. Finally, output the prediction vector through the prediction head.
[0110] Specifically, spatial integration Results of time fusion After concatenation, the vector is input into the linear layer. Let the concatenated vector be: ;
[0111] The weight matrix of the linear layer is:
[0112] ;
[0113] in The bias term is for the final predicted number of action categories:
[0114] ;
[0115] The output of the linear layer is:
[0116] ;
[0117] Assume the output of the linear layer is The final prediction head outputs the prediction vector. for:
[0118] ;
[0119] The above formulas describe the integration of TCN network with temporal logic judgment and GCN network with spatial logic judgment, combining these disparate modules to form a fusion model.
[0120] Therefore, this invention proposes a novel fusion prediction algorithm based on GCN and TCN. By fusing spatial and temporal logic into the GCN and TCN modules respectively, logical judgments are used to compensate for the model's judgment and derivation in fine-grained action segmentation. That is, logical judgments affect the original GCN and TCN model outputs, resulting in superior performance in the special scenario of classroom monitoring. This strengthens the action recognition module and can be applied to the fine-grained action recognition of children in occluded situations in school classrooms.
[0121] Example 2
[0122] The purpose of this embodiment is to provide a child action recognition system based on a school classroom, including:
[0123] The data acquisition module is configured to acquire human skeleton sequence data;
[0124] The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively;
[0125] The logic judgment module is configured to simultaneously deduce spatial and temporal actions from the human skeleton sequence data through spatial logic judgment model and temporal logic judgment model, respectively.
[0126] The fusion output module is configured to fuse the derived spatial and temporal actions with the prediction results based on GCN and TCN networks to obtain the final action pose.
[0127] Example 3
[0128] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.
[0129] Example 4
[0130] The purpose of this embodiment is to provide a computer-readable storage medium.
[0131] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.
[0132] Example 5
[0133] The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, causes the computer to perform the methods and functions involved in any of the embodiments described above.
[0134] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.
[0135] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.
[0136] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A method for recognizing children's actions in a school classroom, characterized in that, include: Obtain human skeleton sequence data; The human skeleton sequence data is input into the GCN network and the TCN network respectively, and prediction results based on the GCN network and the TCN network are obtained respectively. Meanwhile, the human skeleton sequence data is used to deduce spatial and temporal actions through spatial logic judgment models and temporal logic judgment models, respectively. The construction of the spatial logic judgment model and the temporal logic judgment model is as follows: the human judgment results in space and time are obtained; the human judgment results in space and time are used as input and passed to a multilayer perceptron (MLP) layer for feature extraction; the features processed by the MLP layer are output as logical judgment results through the Softmax function. The derived spatial and temporal actions are fused with the prediction results based on GCN and TCN networks to obtain the final action pose, specifically: The spatial actions derived from the spatial logic judgment model are fused with the prediction results based on the GCN network through the MOE module via the spatial Logistic function to obtain the spatial fusion result. The time-based actions derived from the time logic judgment model are fused with the prediction results based on the TCN network through the MOE module via the time logistic function to obtain the time fusion result. The spatial fusion result and the temporal fusion result are concatenated and then input into the linear layer to obtain the output of the linear layer. Finally, the prediction vector is output through the prediction head.
2. The method for recognizing children's actions in a school classroom as described in claim 1, characterized in that, The acquisition of human skeleton sequence data specifically involves obtaining the positions of human joints in school classroom video frames and using AlphaPose to obtain the human skeleton sequence data contained in the video frame sequence as model input.
3. The method for recognizing children's actions in a school classroom as described in claim 1, characterized in that, The construction of GCN and TCN networks is as follows: The GCN and TCN networks were pre-trained using a standard public dataset to obtain the pre-trained model parameters. The pre-trained model parameters are used to construct GCN and TCN network models for classroom behavior prediction. We acquired and labeled data from campus surveillance cameras to further train and fine-tune the model, thereby enhancing its recognition and generalization capabilities in this scenario.
4. The method for recognizing children's actions in a school classroom as described in claim 3, characterized in that, The pre-trained GCN network is specifically designed as follows: First, the graph data in the video frame is input, and features are extracted through the graph convolutional layer. Then, the graph is coarsened into subgraphs through the pooling layer. Then, the readout layer sums or averages the hidden representations of these subgraphs to summarize the final graph representation. Then, it is processed by the multilayer perceptron and finally the classification result is obtained through the Softmax function, which is the prediction result based on the GCN network. The pre-trained TCN network is specifically as follows: The TCN network consists of extended, causal 1D convolutional layers with the same input and output lengths, including an input layer, multiple hidden layers, and an output layer, each consisting of residual blocks; each residual block contains weight normalization, ReLU activation function, Dropout, and causal convolution operations, and in some residual blocks, convolution operations are used to adjust the dimensions; Human skeleton sequence data is input into the input layer, and after passing through multiple hidden layers, the prediction result based on the TCN network is obtained through the output layer.
5. The method for recognizing children's actions in a school classroom as described in claim 1, characterized in that, The process of deriving spatial and temporal actions from human skeletal sequence data using spatial logic judgment models and temporal logic judgment models specifically includes: To obtain human judgment results in space and time; The human judgment results in space and time are used as input and passed to the multilayer perceptron layer for feature extraction. The features processed by the MLP layer are used to output the logical judgment result through the Softmax function.
6. A child motion recognition system based on school classrooms, characterized in that, include: The data acquisition module is configured to acquire human skeleton sequence data; The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively; The logic judgment module is configured to: simultaneously deduce spatial and temporal actions from the human skeleton sequence data through spatial logic judgment model and temporal logic judgment model respectively; the construction of the spatial logic judgment model and the temporal logic judgment model is as follows: obtain the human judgment results in space and time, take the human judgment results in space and time as input, pass them to a multilayer perceptron (MLP) layer for feature extraction, and output the logic judgment result through the Softmax function after the features are processed by the MLP layer. The fusion output module is configured to fuse the derived spatial and temporal actions with the prediction results based on GCN and TCN networks to obtain the final action pose, specifically: The spatial actions derived from the spatial logic judgment model are fused with the prediction results based on the GCN network through the MOE module via the spatial Logistic function to obtain the spatial fusion result. The time-based actions derived from the time logic judgment model are fused with the prediction results based on the TCN network through the MOE module via the time logistic function to obtain the time fusion result. The spatial fusion result and the temporal fusion result are concatenated and then input into the linear layer to obtain the output of the linear layer. Finally, the prediction vector is output through the prediction head.
7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1 to 5.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.