Children action recognition method and system based on school classroom

Abstract

The invention provides a school classroom-based child action recognition method and system, and relates to the technical field of action recognition, and the method comprises the steps: obtaining human skeleton sequence data; respectively inputting the human body skeleton sequence data into a GCN network and a TCN network, and respectively obtaining prediction results based on the GCN network and the TCN network; meanwhile, spatial actions and time actions are derived from the human skeleton sequence data through a spatial logic judgment model and a time logic judgment model respectively; and fusing the deduced space action and time action with a prediction result based on the GCN network and the TCN network to obtain a final action posture. According to the invention, high-precision classification and identification of children actions in a campus classroom scene can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of motion recognition, and in particular relates to a method and system for children's motion recognition based on school classrooms. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Human action recognition and prediction is an important research direction in the field of computer vision. Action recognition and prediction based on the human skeleton has broad application prospects in intelligent transportation, intelligent security, human-computer interaction, motion analysis and other fields.

[0004] At present, the research on human motion recognition algorithms in conventional environments has made great progress. However, under specific surveillance camera conditions such as in some kindergartens and primary schools, there is a lot of obstruction in scenes such as classrooms under the campus surveillance perspective. For example, when children sit on chairs, the desks will block the lower body of the children, and there is a big difference in height between children and adults. Conventional human motion recognition algorithms are usually trained based on adult data, so they cannot be well applied to special scenes such as classrooms under the campus surveillance perspective.

[0005] At the same time, conventional human action recognition based on deep learning methods has become mainstream. Existing technical methods mainly start from two mainstream directions: RGB images and human joint sequences. Compared with the method based on RGB image sequences, the method of modeling the human skeleton based on graph convolutional networks (GCN) shows stronger robustness in the case of occlusion and complex scenes. Processing joint point data at the same time consumes less computing resources than video streams, and has better real-time performance when processing multi-channel video surveillance in practical applications.

[0006] The application of GCN in the field of action recognition has evolved from a simple graph structure representation based on skeleton points to the introduction of the spatiotemporal convolutional network (ST-GCN). ST-GCN regards the skeleton sequence as a graph structure with joints as nodes and the connections between joints as edges. It learns the features between nodes through spatial graph convolution and models the temporal information through temporal graph convolution, thereby extracting action features more comprehensively. With the deepening of research, the exploration and research based on GCN mainly revolves around two directions: one is a more representative way to construct skeleton data; the other is a more effective GCN model design.

[0007] From the first perspective, the AS-GCN network uses a multi-task learning strategy to output posture predictions, and captures the dependencies between joints through two modules (Actional Links and Structural Links). However, the module collaboration causes the computational cost to soar, and it is inefficient when processing large-scale, long-sequence skeleton data, making it difficult to meet real-time requirements; on the other hand, it relies too much on the distribution of task data. If the characteristics of the actual scene data are biased, the accuracy of posture prediction will be seriously affected. 2s-AGCN combines an adaptive topological graph and uses the back-propagation algorithm of a neural network to automatically update to increase the representation of the strength of joint connections, thereby capturing and enhancing richer feature representations. However, it is sensitive to the initial value of back-propagation. Poor initial values ​​are prone to fall into local optimality, hindering the ideal construction of the topological graph. At the same time, when the training samples are limited, complex topological updates are prone to overfitting and poor generalization performance. MS-G3D constructs a unified spatiotemporal graph, which consists of multiple subgraphs, each of which represents the spatial relationship of joints on a certain frame, effectively simulating the relationship between different joints in different coordinate systems. However, in the process of constructing and processing the space-time graph, as the number of joints and frames increases, the complexity of the graph structure increases dramatically, and the storage and computing overhead increases exponentially. At the same time, the weight allocation strategy when merging different sub-graphs lacks an adaptive mechanism, making it difficult to adapt to diverse action patterns, resulting in inaccurate portrayal of some key joint relationships.

[0008] From the second perspective, the traditional GCN operates as a feedforward network, which limits the access of the lower level to the semantic information of the higher level. To solve this problem, the Feedback Graph Convolutional Network (FGCN) came into being. Unlike the direct application of the complete skeleton sequence, FGCN adopts a multi-level temporal sampling strategy to sparsely extract the input fragment sequence from the skeleton data, aiming to gradually obtain the global spatiotemporal features. However, in the sample selection process, it lacks rigorous screening criteria, and is prone to mixing in noise fragments and missing key frames, which interferes with feature learning; and the feedback control parameters between levels are complicated, the adjustment is difficult, and the system stability is poor. STSF-GCN conceptualizes the skeleton data as a unified spatiotemporal topological structure, which can cope with conventional actions. Once it encounters irregular and sudden actions, it is difficult to quickly adapt to the topological adjustment requirements caused by sudden action mutations due to its strong structure and insufficient flexibility.

[0009] Networks based on the Transformer architecture are often applied to a wide range of language tasks (such as text classification, machine translation, or question answering), but are now also extended to visual tasks such as image classification, object detection, segmentation, image restoration, and point cloud registration.

[0010] At present, in the field of action recognition, the Transformer architecture has made significant progress. Transformer-based methods effectively alleviate the problem of focusing only on local information and are good at capturing extensive dependencies in long sequences. When applied to tasks involving skeleton-based human action recognition, the Transformer architecture demonstrates proficiency in capturing temporal relationships. For example, PRTR (Pose Recognition with Cascade Transformers) revolves around human posture recognition and introduces two forms of cascaded Transformer architectures. The human action recognition method based on the HRNet-two-stream Transformer network integrates a two-stream network model with multiple features to improve the accuracy of human action recognition.

[0011] However, the Transformer architecture also has some shortcomings in the direction of action recognition. Due to its limitations in capturing and encoding the complex high-dimensional semantic information inherent in skeleton data, its effectiveness in modeling spatial relationships is still limited. At the same time, when processing long sequences, the complexity of the Transformer's self-attention mechanism leads to high consumption of computing resources, which limits its use in practical applications.

[0012] Therefore, the shortcomings of existing action recognition methods in specific surveillance camera conditions such as kindergartens and primary schools are mainly: 1. The scarcity of children's training data and the large occlusion of campus scenes such as classrooms from the surveillance perspective make the neural network fully trained based on conventional public data sets show poor generalization performance in this scenario, thus limiting the deployment and application of action recognition in campus scenarios.

[0013] 2. Currently, human action recognition algorithms based on graph convolutional networks are mainly used in conventional scenarios. However, children are much taller than adults, and there are large differences in height within a class. In addition, in places such as schools, multiple cameras are used as input data for analysis. Unlike ordinary ground cameras, this type of monitoring perspective is mostly from top to bottom. In addition, the number of people is generally dense, and is blocked by desks in the classroom. Most of the time, only the upper body of children can be observed by the camera. Therefore, the current human action recognition algorithms based on graph convolutional networks are all action recognition models trained based on adult head-on data, and have poor generalization for special scenarios such as campuses.

[0014] 3. In classroom scenarios, most movements are small in amplitude, and conventional movement recognition models find it difficult to make detailed distinctions. Summary of the invention

[0015] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for children's motion recognition based on school classrooms, which is a fusion prediction algorithm model that can achieve high-precision classification and recognition of children's motions in campus classroom scenarios.

[0016] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a method for identifying children's actions based on a school classroom.

[0017] A method for children's action recognition based on school classroom, comprising: Obtain human skeleton sequence data; Inputting the human skeleton sequence data into the GCN network and the TCN network respectively, and obtaining prediction results based on the GCN network and the TCN network respectively; At the same time, the human skeleton sequence data is used to derive spatial actions and temporal actions through spatial logic judgment models and temporal logic judgment models respectively; The derived spatial and temporal actions are fused with the prediction results based on the GCN network and TCN network to obtain the final action posture.

[0018] A children's action recognition system based on school classroom, comprising: The data acquisition module is configured to: acquire human skeleton sequence data; The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively; The logic judgment module is configured to: simultaneously, derive spatial actions and temporal actions from the human skeleton sequence data through a spatial logic judgment model and a temporal logic judgment model respectively; The fusion output module is configured to fuse the derived spatial action and temporal action with the prediction results based on the GCN network and the TCN network to obtain the final action posture.

[0019] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method as described in the first aspect of the present invention are implemented.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a method as described in the first aspect of the present invention.

[0021] A fifth aspect of the present invention provides a computer program product comprising instructions, which, when running on a computer, enables the computer program to implement the steps in a method as described in the first aspect of the present invention when executed by a processor.

[0022] One or more of the above technical solutions have the following beneficial effects: The present invention first conducts training based on a public dataset and then performs fine-tuning on a collected classroom monitoring children dataset. The final model prediction results are combined with the logical reasoning and judgment results of the skeletal joints. The logical judgment guides the GCN and TCN modules to derive the results, so as to obtain higher classification performance on finer movements and meet the accuracy requirements of the task.

[0023] The present invention is based on applying traditional recognition algorithms to campus classroom scenarios, and verifies that conventional algorithms have performance degradation when processing refined action classification in the presence of occlusion. For example, GCN directly performs inference prediction under the classroom monitoring perspective. Due to the influence of the scene, it often cannot correctly infer the accurate action of the occluded part, resulting in subsequent prediction and reasoning errors. The present invention, by adding spatial logic guidance, can adjust the prediction results of deep models such as GCN according to the task scenario, thereby avoiding the reasoning errors of the deep model caused by occlusion.

[0024] The present invention proposes a novel fusion prediction algorithm based on GCN and TCN. By fusing the GCN module and the TCN module with spatial and temporal logic respectively, the action recognition module is strengthened. It can be applied to the recognition of children's fine movements in the presence of occlusion in campus classroom scenes.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is a flow chart of a method for identifying children's movements based on a school classroom according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a method for identifying children's movements based on a school classroom according to a first embodiment of the present invention. DETAILED DESCRIPTION

[0028] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0030] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0031] Embodiment 1 This embodiment discloses a method for identifying children's actions based on a school classroom, including: Obtain human skeleton sequence data; Inputting the human skeleton sequence data into the GCN network and the TCN network respectively, and obtaining prediction results based on the GCN network and the TCN network respectively; At the same time, the human skeleton sequence data is used to derive spatial actions and temporal actions through spatial logic judgment models and temporal logic judgment models respectively; The derived spatial and temporal actions are fused with the prediction results based on the GCN network and TCN network to obtain the final action posture.

[0032] In order to explain this embodiment more clearly, Figure 1 As shown in the figure, a method for children's action recognition based on school classrooms is mainly to first train based on public data sets, and then fine-tune on the collected classroom monitoring children's data sets, combine the final model reasoning results with the logical reasoning results of the skeleton joints, and guide the GCN and TCN modules to derive the results by logical judgment, so as to obtain higher classification performance on more delicate actions and meet the accuracy requirements of the task. It can be specifically described as follows: Step 1: Obtain human skeleton sequence data.

[0033] In this embodiment, obtaining the human skeleton sequence data specifically involves obtaining the positions of human joints in school classroom video frames, and obtaining the human skeleton sequence data contained in the video frame sequence through AlphaPose as a model input.

[0034] Step 2: Input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively.

[0035] In this embodiment, the specific steps of constructing the GCN network and the TCN network include: Step 2-1: First, pre-train the GCN network and TCN network respectively using regular public data sets to obtain the pre-trained model parameters.

[0036] Specifically, we select the UAV-Human dataset shot from a drone perspective, and obtain the positions of human joints in the video frames contained in the dataset. We use AlphaPose to obtain the human skeleton sequence data contained in the video frame sequence as the model input.

[0037] Since the UAV-Human dataset is similar to the surveillance perspective, it is used as a pre-training dataset for model pre-training. The prediction classification based on the UAV-Human dataset is output, and in the subsequent process, the prediction head classification weight is deleted and the previous weight parameter is used.

[0038] (1) Pre-training the GCN network. The specific process is as follows: First, the graph data X in the video frame is input, and features are extracted through the graph convolution layer (Gconv). Then, the graph is coarsened into subgraphs through the pooling layer. Then, the readout layer sums or averages the hidden representations of these subgraphs to summarize the final graph representation, which is then processed by the multi-layer perceptron (MLP) and finally the classification result is obtained through the Softmax function, that is, the prediction result based on the GCN network is obtained.

[0039] (2) Pre-training TCN network. The specific process is as follows: TCN consists of dilated, causal 1D convolutional layers with the same input and output lengths. It includes an input layer, multiple hidden layers, and an output layer, each of which consists of residual blocks. Each residual block contains weight normalization, ReLU activation function, Dropout, and causal convolution operations, and 1x1 convolution operations are used to adjust the dimension in some residual blocks.

[0040] The human skeleton sequence data is input into the input layer, and after passing through multiple hidden layers, the prediction result based on the TCN network is obtained through the output layer.

[0041] Step 2-2. Secondly, the model parameters obtained after pre-training are used to construct the GCN network and TCN network models for classroom behavior prediction.

[0042] Step 2-3: Obtain labeled data collected by campus surveillance in kindergartens, primary schools, etc., and further train and fine-tune the model to enhance the model's recognition and generalization capabilities in this scenario.

[0043] In terms of the model training process, it can accelerate convergence, because the pre-trained model has learned common features on a large-scale data set, and can find better parameter values ​​more quickly when training for specific downstream tasks; at the same time, it can also solve the problem of insufficient data, using the common features in the pre-trained model as prior knowledge to reduce the risk of overfitting when training the model in a small amount of data. In terms of model performance, pre-training can improve the generalization ability of the model. The common features learned by the pre-trained model on a large-scale and diverse data set help it adapt to new data distributions; and it can enable the model to learn better feature representations. The pre-training process enables the model to learn high-level abstract features in the data, providing more valuable input features for downstream tasks, thereby improving downstream task performance.

[0044] Step 3: At the same time, the human skeleton sequence data is used to derive spatial actions and temporal actions through a spatial logic judgment model and a temporal logic judgment model respectively.

[0045] In this embodiment, the specific contents of constructing the spatial logic judgment model and the temporal logic judgment model are as follows: Step 3-1: First, obtain the human judgment results in space and time.

[0046] At the spatial level, it involves precise consideration of the relative positions of the joints of the human body. For example, when focusing on the joints of the hand, if its position is higher than the spatial reference point of the top of the head, then this spatial state can be preliminarily understood as a tendency to raise the hand. The spatial judgment here is not isolated, but takes into account the overall posture of the human body and the spatial relationship between different joints. The human body is an organic whole, and the position of a joint is often interrelated with other joints. Together, they constitute specific spatial posture patterns, and these patterns provide important clues for action judgment.

[0047] In the time dimension, it is crucial to track the movement trajectory of the joints. Taking the hand-raising action judgment as an example, we not only pay attention to whether the joints of the hand are finally above the head, but also need to observe its movement process. That is, the joints of the hand start from the initial position (such as under the head), gradually move upward until they reach the top of the head, and stay at this position above the head for a period of time. This continuous change and continuous state in the time series is the key factor in judging whether the action has occurred. By recording the position of the joints at different time points, we can clearly depict the time context of the action.

[0048] Step 3-2. Secondly, the human judgment results in space and time are passed as input to a multi-layer perceptron (MLP) layer for feature extraction.

[0049] The MLP layer has a powerful nonlinear fitting capability and can deeply learn and process the spatial and temporal features of the input. In the MLP layer, through multiple neurons and multi-layer network structures, the input features are continuously transformed and abstracted to mine the potential action patterns and logical relationships.

[0050] Step 3-3. Finally, the features processed by the MLP layer are used to output the logical judgment results through the Softmax function.

[0051] The Softmax function converts the output of the MLP layer into a probability distribution, which can clearly represent the possibility of each action category occurring.

[0052] For example, in the scenario of hand-raising action judgment, the Softmax output will give the probability value of the action belonging to "hand-raising" and other possible actions. By comparing these probability values, we can accurately judge whether the action corresponding to the current joint point movement is a hand-raising action. The entire model achieves accurate judgment of spatial and temporal action logic through this process from spatial and temporal judgment input to MLP processing and then to Softmax output.

[0053] In this embodiment, the derived spatial actions and temporal actions specifically include: raising hands, reading actions, writing, talking actions, yawning, small actions (such as spinning a pen, fiddling with stationery), etc.

[0054] Step 4: Fuse the derived spatial and temporal actions with the prediction results based on the GCN network and TCN network to obtain the final action posture.

[0055] In this embodiment, if Figure 2 As shown, the specific steps are as follows: Step 4-1: The spatial action derived based on the spatial logic judgment model is fused with the prediction result based on the GCN network through the spatial Logistic function through the MOE module to obtain the spatial fusion result.

[0056] Specifically, the Logistic function of spatial logical judgment is assumed to be , whose output is the probability vector of each category to which the spatial action belongs , expressed as: ; in is the number of spatial action categories, That is, it is represented as a real vector space with a dimension based on the number of spatial action categories.

[0057] Assume that the node features obtained after GCN processing It is expressed as: ; in, Represented as a 512-dimensional real vector space.

[0058] After the fully connected layer, the output is for: ; in is the number of spatial action categories, That is, it is represented as a real vector space with a dimension based on the number of spatial action categories.

[0059] The output of GCN is estimated by the gating network: , after the softmax function, the gate is obtained : ; The output evaluation value of spatial Logistic through the gating network is , whose gating weight for: ; The fused spatial output for: ; Step 4-2: At the same time, the time action derived based on the time logic judgment model is fused with the prediction result based on the TCN network through the MOE module through the time logistic function to obtain the time fusion result.

[0060] Specifically, let the Logistic function of time logic judgment be , its output is the probability vector of each category to which the spatial action belongs, expressed as: ; in is the number of temporal action categories, That is, it is represented as a real vector space with a dimension based on the number of time action categories.

[0061] Assume that the node features obtained after TCN processing It is expressed as: ; in, Represented as a 512-dimensional real vector space.

[0062] After the fully connected layer, the output is for: ; in is the number of temporal action categories, That is, it is represented as a real vector space with a dimension based on the number of time action categories.

[0063] The gating network estimates the output of TCN as , after the softmax function, the gate is obtained : ; The output evaluation value of the gating network for spatial Logistic is , whose gating weight for: ; The fused time output for: ; Step 4-3: splice the spatial fusion result obtained in step 4-1 and the temporal fusion result obtained in step 4-2 and input them into the linear layer to obtain the output of the linear layer, and finally output the prediction vector through the prediction head.

[0064] Specifically, the space is integrated Fusion results with time After concatenation, input it into the linear layer. Let the concatenated vector be: ; The weight matrix of the linear layer is: ; in To predict the final number of action categories, the bias term is: ; The output of the linear layer is: ; Suppose the linear layer output is After the final prediction head outputs the prediction vector for: ; The above formulas all describe the fusion of TCN network and temporal logic judgment, GCN network and spatial logic judgment, and combine these scattered modules to form a fusion model.

[0065] Therefore, the present invention actually proposes a new fusion prediction algorithm based on GCN and TCN, which fuses the GCN module and the TCN module with spatial and temporal logic respectively, and uses logical judgment to make up for the judgment and deduction of the model in fine-grained action segmentation, that is, the model output of the original GCN and TCN is affected by logical judgment, so that in this special scenario under the classroom monitoring perspective, the performance is more superior, the action recognition module is strengthened, and it can be applied to the recognition of fine movements of children in the presence of occlusion in campus classroom scenes.

[0066] Embodiment 2 The purpose of this embodiment is to provide a children's action recognition system based on school classroom, including: The data acquisition module is configured to: acquire human skeleton sequence data; The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively; The logic judgment module is configured to: simultaneously, derive spatial actions and temporal actions from the human skeleton sequence data through a spatial logic judgment model and a temporal logic judgment model respectively; The fusion output module is configured to fuse the derived spatial action and temporal action with the prediction results based on the GCN network and the TCN network to obtain the final action posture.

[0067] Embodiment 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0068] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0069] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0070] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0071] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0072] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for identifying children's actions based on school classrooms, characterized in that: include: Obtain human skeleton sequence data; Inputting the human skeleton sequence data into the GCN network and the TCN network respectively, and obtaining prediction results based on the GCN network and the TCN network respectively; At the same time, the human skeleton sequence data is used to derive spatial actions and temporal actions through spatial logic judgment models and temporal logic judgment models respectively; The derived spatial and temporal actions are fused with the prediction results based on the GCN network and TCN network to obtain the final action posture.

2. A method for identifying children's movements based on school classrooms as claimed in claim 1, characterized in that: The obtaining of human skeleton sequence data specifically involves obtaining human joint positions in school classroom video frames, and obtaining the human skeleton sequence data contained in the video frame sequence through AlphaPose as a model input.

3. A method for identifying children's movements based on school classrooms as claimed in claim 1, characterized in that: Construct the GCN network and TCN network, specifically: Using conventional public data sets, pre-train the GCN network and TCN network respectively to obtain the pre-trained model parameters; The model parameters obtained after pre-training are used to construct GCN network and TCN network models for classroom behavior prediction; Obtain labeled data collected by campus surveillance and further train and fine-tune the model to enhance the model's recognition and generalization capabilities in this scenario.

4. A method for identifying children's movements in a school classroom as claimed in claim 3, characterized in that: The pre-trained GCN network is specifically as follows: firstly, the graph data in the video frame is input, and features are extracted through the graph convolution layer, then the graph is coarsened into sub-graphs through the pooling layer, and then the hidden representations of these sub-graphs are summed or averaged by the readout layer to summarize the final graph representation, and then processed through the multi-layer perceptron, and finally the classification result is obtained through the Softmax function, that is, the prediction result based on the GCN network is obtained; The pre-trained TCN network is specifically: The TCN network consists of dilated, causal 1D convolutional layers with the same input and output lengths, including an input layer, multiple hidden layers, and an output layer. Each layer consists of residual blocks. Each residual block contains weight normalization, ReLU activation function, Dropout, and causal convolution operations, and convolution operations are used to adjust the dimension in some residual blocks. The human skeleton sequence data is input into the input layer, and after passing through multiple hidden layers, the prediction result based on the TCN network is obtained through the output layer.

5. The method for identifying children's actions in a school classroom as claimed in claim 1, characterized in that: The method of deriving spatial actions and temporal actions from the human skeleton sequence data through a spatial logic judgment model and a temporal logic judgment model respectively specifically includes: Obtain human judgment results in space and time; The human judgment results in space and time are passed as input to the multi-layer perceptron layer for feature extraction; The features processed by the MLP layer output logical judgment results through the Softmax function.

6. A method for identifying children's movements in a school classroom as claimed in claim 1, characterized in that: The derived spatial action and temporal action are fused with the prediction results based on the GCN network and the TCN network to obtain the final action posture, specifically: The spatial action derived from the spatial logic judgment model is fused with the prediction result based on the GCN network through the spatial Logistic function through the MOE module to obtain the spatial fusion result; The time action derived from the time logic judgment model is fused with the prediction result based on the TCN network through the MOE module through the time logistic function to obtain the time fusion result; The spatial fusion result and the temporal fusion result are concatenated and input into the linear layer to obtain the output of the linear layer, and finally the prediction vector is output through the prediction head.

7. A children's action recognition system based on school classroom, characterized in that: include: The data acquisition module is configured to: acquire human skeleton sequence data; The algorithm prediction module is configured to: input the human skeleton sequence data into the GCN network and the TCN network respectively, and obtain prediction results based on the GCN network and the TCN network respectively; The logic judgment module is configured to: simultaneously, derive spatial actions and temporal actions from the human skeleton sequence data through a spatial logic judgment model and a temporal logic judgment model respectively; The fusion output module is configured to fuse the derived spatial action and temporal action with the prediction results based on the GCN network and the TCN network to obtain the final action posture.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images