Robot infant teaching system and device based on somatosensory interaction

Abstract

The invention relates to the technical field of robot teaching, and discloses a robot infant teaching system and device based on somatosensory interaction. According to the method, a somatosensory data acquisition module acquires skeleton point motion time sequence data and an interactive behavior video stream of a target child through a depth camera and an inertial sensor; the multi-modal feature fusion module is used for extracting dynamic track features from the skeleton motion data, analyzing behavior semantic features of the video stream, and generating a fusion interaction feature set through cross-modal association fusion; the interaction sensitivity calculation module is used for calling a pre-training dynamic weighting network model to carry out attention distribution on the fusion features and generating a teaching interaction sensitivity coefficient set containing different teaching link response sensitivity distributions; the abnormal behavior tracing module is used for generating a teaching behavior abnormal tracing result for indicating an abnormal source link and a deviation type; and the interaction strategy generation module generates a self-adaptive teaching strategy according to the traceability result and sends the self-adaptive teaching strategy to the robot motion control unit.

Description

Technical Field

[0001] This invention relates to the field of robotics teaching technology, specifically to a robotic early childhood teaching system and equipment based on motion-sensing interaction. Background Technology

[0002] In the field of early childhood education, with the development of intelligent technology, robotic teaching systems are gradually becoming important tools to assist children's learning. Most existing early childhood teaching robots rely on pre-set programs to conduct teaching activities, lacking dynamic perception and flexible response to children's real-time behavior. While some systems have attempted to introduce interactive functions, they still have significant limitations in behavioral perception and teaching adaptation.

[0003] Most robotic teaching systems rely on limited sensing methods, often simply using ordinary cameras to capture images of children or simple touch sensors to obtain interaction signals. This single-modal sensing approach struggles to comprehensively capture a child's movement and intentions; for example, it cannot accurately identify details such as the range of motion, the continuity of movements, and changes in facial expressions. Due to the lack of precise tracking of skeletal motion trajectories, the system cannot determine whether a child is actively participating in the learning activities, nor can it perceive the proper form of the child's movements during operation.

[0004] In terms of feature processing, existing technologies lack effective fusion analysis of the collected raw data. Image information and simple motion signals are often processed separately, failing to establish correlations between different modalities. This results in generated feature vectors that cannot fully reflect the interactive state of young children. Consequently, the teaching system is biased in understanding the semantics of children's behavior, making it difficult to distinguish between unintentional actions and intentional interactions, thus affecting the accuracy of teaching responses.

[0005] Existing systems lack dynamic adaptability in adjusting interactive sensitivity. During the teaching process, different stages have varying requirements for interactive response. For example, games require higher sensitivity to enhance engagement, while knowledge explanations require appropriately lower sensitivity to avoid interference. However, existing systems often use fixed response thresholds, failing to adjust sensitivity distribution based on children's real-time behavioral characteristics, easily leading to over- or under-response issues.

[0006] When young children exhibit abnormal behaviors such as inattention or deviation from teaching requirements, existing systems struggle to quickly trace the source of the abnormality. Due to the lack of joint analysis of skeletal movement time-series data and behavioral video streams, it is impossible to pinpoint the specific teaching stage where the abnormal behavior occurred, or to determine the type of behavioral deviation, resulting in a lack of targeted teaching intervention. Regarding strategy generation, existing systems primarily use pre-set teaching strategies, which cannot be dynamically adjusted based on real-time behavioral abnormalities, failing to meet the individualized learning needs of different children and hindering the improvement of teaching effectiveness. Summary of the Invention

[0007] The purpose of this invention is to provide a robot-based early childhood education system based on motion-sensing interaction to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a robot-based early childhood education system based on motion-sensing interaction, the method comprising: The motion data acquisition module is used to acquire the time-series data of the skeletal movement of the target child and the video stream of the interactive behavior through a depth camera and an inertial sensor; The multimodal feature fusion module is used to perform dynamic trajectory feature extraction processing on the motion time series data of the skeleton points to generate skeleton motion feature vectors, perform action semantic parsing processing on the interactive behavior video stream to generate behavior semantic feature vectors, and perform cross-modal association and fusion of the skeleton motion feature vectors and the behavior semantic feature vectors to generate a fused interactive feature set. The interaction sensitivity calculation module is used to call a pre-trained dynamic weighted network model to perform attention allocation processing on the fused interaction feature set and generate a set of teaching interaction sensitivity coefficients, which includes the distribution of interaction response sensitivity corresponding to different teaching stages. An abnormal behavior tracing module is used to perform joint root cause analysis on the motion time series data of the skeletal points and the video stream of the interactive behavior based on the interaction response sensitivity distribution, and generate teaching behavior abnormality tracing results. The teaching behavior abnormality tracing results are used to indicate the teaching link and behavior deviation type from which the abnormality originates. The interaction strategy generation module is used to generate an adaptive teaching strategy based on the abnormal teaching behavior tracing results, and send the adaptive teaching strategy to the robot motion control unit.

[0009] Preferably, the step of performing action semantic parsing processing on the interactive behavior video stream to generate a behavior semantic feature vector includes: The interactive video stream is processed by action sequence segmentation to obtain multiple continuous action segments; A pre-trained 3D convolutional network model is invoked to perform spatiotemporal feature encoding on each of the continuous action segments to generate an initial action feature vector; The initial action feature vector is subjected to teaching behavior feature enhancement processing, including: matching a set of standard behavior templates associated with the current action segment from a preset children's behavior knowledge base; inputting the set of standard behavior templates into a feature encoder for template semantic embedding processing to generate a set of template feature vectors; and performing similarity weighted fusion between the set of template feature vectors and the initial action feature vector to generate an enhanced action feature vector. The enhanced action feature vectors of each continuous action segment are integrated by temporal position encoding to generate the behavioral semantic feature vector.

[0010] Preferably, the step of performing dynamic trajectory feature extraction processing on the skeletal point motion time-series data to generate a skeletal motion feature vector includes: The abnormal motion interval detection process is performed on the motion time sequence data of the skeletal points to identify abnormal fluctuation time segments in the joint angle change sequence; Within the abnormal fluctuation time segment, the original skeletal data is subjected to multi-scale sliding window sampling to obtain multiple local trajectory data segments; A pre-trained graph convolutional network model is invoked to perform joint association feature extraction processing on each of the local trajectory data segments, generating a local motion feature vector; Global trajectory aggregation processing is performed on the local motion feature vectors of each of the local trajectory data segments to generate the skeletal motion feature vector.

[0011] Preferably, the step of cross-modal association and fusion of the skeletal motion feature vector and the behavioral semantic feature vector to generate a fused interactive feature set includes: The behavioral semantic feature vector is aligned to the time dimension and mapped to the same timestamp sequence as the skeletal motion feature vector. A cross-attention mechanism for skeletal behavior is constructed, and the cross-modal correlation matrix between the trajectory features of each timestamp in the skeletal motion feature vector and the semantic features of the corresponding timestamp in the behavior semantic feature vector is calculated. Based on the cross-modal correlation matrix, the skeletal motion feature vector and behavioral semantic feature vector are subjected to bidirectional feature interaction processing to generate interactive trajectory feature vector and interactive semantic feature vector. The interaction trajectory feature vector and the interaction semantic feature vector are subjected to gated feature fusion processing to generate the fused interaction feature set.

[0012] Preferably, the step of calling a pre-trained dynamically weighted network model to perform attention allocation processing on the fused interactive feature set to generate a set of teaching interaction sensitivity coefficients includes: The fused interactive feature set is input into the feature filtering layer of the dynamic weighted network model for key feature extraction processing to obtain the filtered core interactive feature set. The cross-association layer of the dynamic weighted network model is invoked to perform high-order feature combination processing on the core interaction feature set, generating an association feature tensor; The sensitivity allocation layer of the dynamic weighted network model is used to calculate the weights of the associated feature tensors for different teaching stages, thereby generating the interactive response sensitivity distribution for different teaching stages. Based on the interaction response sensitivity distribution, the associated feature tensor is subjected to feature weighting and aggregation processing to generate the set of teaching interaction sensitivity coefficients.

[0013] Preferably, the step of performing joint root cause analysis on the skeletal point motion time-series data and interactive behavior video stream based on the interaction response sensitivity distribution to generate abnormal teaching behavior tracing results includes: Based on the interaction response sensitivity distribution, abnormal sensitivity items exceeding a preset threshold are filtered to generate a candidate abnormal feature set; Each abnormality sensitivity item in the candidate abnormality feature set is subjected to reverse feature parsing to determine its corresponding skeletal motion abnormality description and behavioral semantic deviation description. The pre-trained source tracing inference model is invoked to perform joint causal analysis on the description of abnormal skeletal movement and the description of deviation in behavioral semantics, generating a set of abnormal source tracing paths; The set of abnormal tracing paths is subjected to confidence evaluation, and the target tracing path with the highest confidence is selected as the abnormal tracing result of the teaching behavior.

[0014] Preferably, the step of generating an adaptive teaching strategy based on the source tracing results of the abnormal teaching behavior includes: The analysis reveals the abnormal teaching processes and behavioral deviation types in the results of the abnormal teaching behavior tracing. Match a set of historical adjustment strategies associated with the teaching segment from the history teaching strategy library that corresponds to the source of the anomaly; Based on the behavioral deviation type, the set of historical adjustment strategies is filtered for effectiveness to obtain a set of effective interaction strategies. The effective set of interaction strategies is subjected to multi-objective optimization to generate adaptive teaching strategies that meet the teaching constraints.

[0015] Preferably, the step of performing multi-objective optimization processing on the set of effective interaction strategies to generate adaptive teaching strategies that meet the teaching constraints includes: Construct a three-dimensional optimization target space that includes teaching progress completion rate, behavioral deviation correction rate, and interaction response timeliness; Map each effective interaction strategy to a coordinate point in the three-dimensional optimization target space; A set of candidate policies located on the Pareto front is selected using a multi-objective optimization algorithm; The strategy adaptation model is invoked to perform adaptive scoring on the candidate strategy set based on the real-time status of the current teaching scenario; The candidate strategy with the highest adaptability score is selected as the adaptive teaching strategy.

[0016] Preferably, the system further includes: The real-time feedback module is used to continuously collect physiological response data and behavioral feedback data of children during the process of the robot executing the adaptive teaching strategy. The strategy optimization module is used to input the physiological response data and behavioral feedback data into the incremental learning model for dynamic strategy optimization processing, and generate an optimized adaptive teaching strategy.

[0017] Preferably, when the strategy optimization module performs dynamic strategy optimization processing, it includes: The data on the effectiveness of the current adaptive teaching strategy are correlated with the physiological response data to generate a strategy correction coefficient. The interactive response parameters in the adaptive teaching strategy are dynamically adjusted based on the strategy correction coefficient. The adjusted interactive response parameters are matched and verified with the behavioral feedback data to generate the optimized adaptive teaching strategy.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This motion-sensing interactive robotic early childhood education system effectively compensates for the shortcomings of existing early childhood education robots in terms of behavior perception, feature processing, response adjustment, and strategy adaptation through the collaborative work of multiple modules.

[0019] At the data acquisition level, the system combines depth cameras and inertial sensors to simultaneously acquire temporal data of children's skeletal movements and video streams of their interactive behaviors, overcoming the limitations of single-sensor perception. This multi-source data acquisition mode can comprehensively capture children's limb movement trajectories, movement rhythms, and behavioral scene information, providing rich raw data support for subsequent feature analysis and enabling the system to perceive children's behavior more comprehensively and meticulously.

[0020] The multimodal feature fusion module performs targeted processing on skeletal motion data and behavioral video streams, and then generates a fused interactive feature set through cross-modal correlation fusion. This process overcomes the shortcomings of isolated feature processing in existing technologies, organically combining the dynamic trajectory features of skeletal motion with the semantic features of behavioral video, enabling a more accurate reflection of children's interactive intentions and behavioral states. For example, it can determine the standardization of children's actions through skeletal motion features, and understand the purpose of children's actions through behavioral semantic features, making the system's interpretation of children's behavior more in-depth and accurate.

[0021] The interaction sensitivity calculation module introduces a pre-trained dynamically weighted network model to allocate attention to the fused feature set, generating an interaction response sensitivity distribution corresponding to different teaching stages. This design changes the fixed sensitivity mode of existing systems, enabling the system to dynamically adjust its response strategy based on the real-time behavioral characteristics of children. In stages where children actively participate, the system can increase sensitivity to enhance interactivity; in stages requiring focused learning, it can appropriately decrease sensitivity to reduce interference, making the teaching interaction more aligned with the children's behavioral rhythm and the needs of the teaching scenario.

[0022] The abnormal behavior tracing module analyzes the root causes of abnormal behaviors by jointly analyzing skeletal motion time-series data and behavioral video streams, combined with interactive response sensitivity distribution. This function solves the problem of existing systems struggling to pinpoint the source of abnormalities, accurately identifying the teaching stage where the abnormal behavior occurred and the type of behavioral deviation, such as inattention or incorrect movements. By clearly identifying the root cause of the abnormality, it provides a clear direction for subsequent teaching adjustments, making teaching interventions more targeted.

[0023] The interaction strategy generation module generates adaptive teaching strategies based on anomaly tracing results and sends them to the robot's motion control unit, enabling the robot's teaching behavior to respond in real time to changes in the children's states. This dynamic adjustment mechanism changes the existing system's fixed teaching process model, allowing for timely adjustments to teaching content, pace, or interaction methods based on children's behavioral anomalies. This meets the learning characteristics and needs of different children, making the teaching process more flexible and efficient, and enhancing children's participation and experience in teaching activities. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating the working principle of the motion-sensing interactive robot-based early childhood education system described in this invention. Figure 2 A flowchart for semantic parsing and processing of interactive video stream actions; Figure 3 A diagram illustrating the semantic parsing and processing of interactive video stream actions; Figure 4 A flowchart for extracting dynamic trajectory features from skeletal point motion temporal data; Figure 5 A diagram illustrating the process of extracting dynamic trajectory features from temporal data of skeletal point motion. Figure 6 A flowchart for multi-objective optimization processing of an effective set of interaction strategies. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Please see Figure 1 This invention provides a robot-based early childhood education system and equipment based on motion-sensing interaction, wherein the method includes: The system uses a depth camera and inertial sensors to collect real-time temporal data of skeletal point motion and interactive behavior video streams of target children. The motion data acquisition module employs multi-sensor synchronization technology to ensure precise alignment of the timestamps of the skeletal point motion data and the video stream. A multimodal feature fusion module extracts dynamic trajectory features from the skeletal point motion temporal data to generate skeletal motion feature vectors; simultaneously, it performs action semantic analysis on the interactive behavior video stream to generate behavioral semantic feature vectors. Through cross-modal association fusion technology, the skeletal motion feature vectors and behavioral semantic feature vectors are fused into a unified fused interactive feature set. An interaction sensitivity calculation module uses a pre-trained dynamic weighted network model to allocate attention to the fused interactive feature set, generating a set of teaching interaction sensitivity coefficients that reflect the distribution of interaction response sensitivity across different teaching stages. An abnormal behavior tracing module performs joint root cause analysis on the skeletal point motion temporal data and interactive behavior video streams based on the interaction response sensitivity distribution to identify the teaching stage and behavioral deviation type from which the abnormality originates. An interaction strategy generation module generates adaptive teaching strategies based on the abnormality tracing results and executes these strategies through the robot's motion control unit, achieving dynamic optimization of the teaching process.

[0027] Example 1: See Figure 2 Action semantic parsing and processing of interactive behavior video streams is a key step in enabling the system to understand children's behavior. The acquired interactive behavior video streams are segmented into action sequences using a sliding window technique, dividing the continuous video stream into multiple action segments with fixed time spans. The duration of each action segment is adaptively adjusted according to the dynamic characteristics of the teaching scenario to accommodate changes in the rhythm of behavior in different teaching stages. The segmented action segments maintain temporal continuity and integrity, avoiding breaks in action semantics due to segmentation.

[0028] A pre-trained 3D convolutional network model encodes the spatiotemporal features of each action segment. This model employs a multi-layer 3D convolutional structure, capable of simultaneously capturing feature changes in video data across both spatial and temporal dimensions. Spatially, the model extracts local features of human posture and movement morphology through convolutional kernels; temporally, it uses temporal convolution to capture the dynamic evolution of the action. After multi-layer feature extraction, the model outputs an initial action feature vector, which comprehensively represents the spatiotemporal characteristics of the action segment. Teaching behavior feature enhancement further optimizes the semantic expressive power of the initial action feature vector. The system retrieves a set of standard behavior templates associated with the current action segment from a pre-defined early childhood behavior knowledge base. The knowledge base stores standard behavior sequences in typical teaching scenarios, including teacher demonstrations, children's imitations, and interactive behavior patterns. The feature encoder performs semantic embedding processing on the set of standard behavior templates, generating a set of template feature vectors. This encoder employs a bidirectional long short-term memory network structure, which can fully model the temporal dependencies of the standard behavior templates while preserving their semantic information.

[0029] During the feature fusion stage, the system calculates the similarity between the initial action feature vector and each template feature vector. The similarity calculation uses cosine similarity, reflecting the closeness of the current action segment to the standard behavior template in the feature space. Based on the similarity weights, the system performs a weighted summation of the template feature vectors to generate an enhanced action feature vector. This process enables the initial action feature vector to incorporate the semantic information of the standard behavior template, thereby improving the accuracy and robustness of action representation.

[0030] The system inputs the enhanced motion feature vectors of each action segment into a temporal position encoder for integration. The temporal position encoder employs a self-attention mechanism to dynamically model the temporal dependencies between action segments. Through positional encoding techniques, the system preserves the temporal order information of action segments while capturing long-distance dependencies. The integrated behavioral semantic feature vector not only contains local features of individual action segments but also encompasses global motion semantic information of the entire video stream.

[0031] Throughout the entire action semantic parsing process, the system progressively enhances the semantic richness of behavioral representations through multi-level feature extraction and fusion techniques. A 3D convolutional network model captures the underlying spatiotemporal features of actions, while the teaching behavior feature enhancement process introduces domain knowledge to optimize the semantic expression of features. A temporal position encoder further integrates global temporal information, ensuring that the behavioral semantic feature vector comprehensively reflects children's behavioral patterns in teaching interactions. The system's action semantic parsing processing exhibits strong adaptability, capable of handling the diversity of behaviors in different teaching scenarios. By dynamically adjusting the segmentation length of action fragments and the feature fusion weights, the system can flexibly adapt to fast-paced or slow-paced teaching activities.

[0032] See Figure 3 This chart demonstrates the entire process of action semantic parsing and processing in interactive video streams. The chart contains five sub-charts: Action Sequence Segmentation: This shows the process of dividing a continuous video stream into multiple action segments with fixed time spans; different grayscale bars represent the feature intensity of different segments. 3D Convolutional Spatiotemporal Feature Encoding: This uses a grayscale heatmap to show the feature distribution of each action segment in the spatiotemporal dimension; darker colors indicate higher feature values. Standard Behavior Template Features: This displays the feature vectors of standard behavior templates retrieved from a children's behavior knowledge base; different grayscale lines represent different templates. Action Segment and Template Similarity: This uses a grayscale bar chart to show the similarity between the current action segment and each standard template; darker bars indicate higher similarity. Enhanced Behavioral Semantic Feature Vector: This uses a grayscale heatmap to show the enhanced features after template-weighted fusion, reflecting the feature representation after fusing standard behavioral semantic information.

[0033] Example 2: See Figure 4 Dynamic trajectory feature extraction from skeletal point motion time-series data is a crucial step in the system's understanding of preschool children's motor behavior. This process starts with raw skeletal point data acquired from depth cameras and inertial sensors, and gradually constructs a high-dimensional representation that comprehensively reflects the characteristics of preschool children's movements through multi-stage analysis. The system first performs anomaly detection on continuous skeletal point motion time-series data, identifying atypical fluctuation segments in joint angle sequences. This detection employs a dynamic thresholding method based on statistical distribution, which can adapt to the different movement habits and amplitude differences among preschool children. After detecting anomalous fluctuation time segments, the system focuses on analyzing these intervals, using a multi-scale sliding window technique to extract local motion trajectory data. The window size automatically adjusts according to the joint movement amplitude; a wider window is used for large-scale movements to capture the overall trend, while a narrower window is used for fine movements to retain detailed features.

[0034] Feature extraction of local trajectory data segments is achieved through a pre-trained graph convolutional network model. This model models the human skeletal structure as graph data, where joints are the vertices and skeletal connections are the edges. Graph convolution operations effectively capture the motion relationships between joints by propagating information along the graph's topology. The model employs a multi-layer graph convolutional structure; shallow layers extract the cooperative motion patterns of local joint groups, while deep layers model the global coordination relationships of all joints. Each local trajectory data segment, after processing by the graph convolutional network, generates a corresponding local motion feature vector. These vectors contain both the motion characteristics of the joints themselves and reflect the dynamic relationships between joints.

[0035] The global trajectory aggregation stage integrates scattered local motion features into a unified skeletal motion feature vector. This process uses an attention mechanism to dynamically calculate the contribution weight of each local feature, with the weight value depending on the temporal position and motion amplitude of the local trajectory data segment. Local features at key time points or containing significant motion changes are assigned higher weights, thus being highlighted in the global features. Through this weighted fusion method, the skeletal motion feature vector generated by the system can take into account both the overall trend of the motion process and key details, forming a comprehensive characterization of children's motor behavior.

[0036] In the cross-modal association and fusion stage, the system needs to effectively combine skeletal motion features with behavioral semantic features. Since the two types of features may originate from devices with different sampling frequencies, the system first performs temporal alignment on the behavioral semantic feature vectors. This process employs a dynamic time warping algorithm, which maps the behavioral semantic features to the same temporal reference as the skeletal motion features through nonlinear stretching or compression, while maintaining the temporal logic of the actions. The aligned features are then fed into a skeletal-behavioral cross-attention mechanism, which establishes a mapping relationship between skeletal motion trajectories and behavioral semantics by calculating the correlation matrix between the features.

[0037] The cross-attention mechanism employs a bidirectional interactive architecture, simultaneously calculating the attention distribution of skeletal motion features on behavioral semantic features, and vice versa. This bidirectional design allows the two types of features to guide each other; feature extraction of skeletal motion trajectory references the contextual information of behavioral semantics, while the understanding of behavioral semantics incorporates the physical characteristics of skeletal motion. Based on the calculated cross-modal correlation matrix, the system interactively reconstructs the original features, generating interactive trajectory feature vectors and interactive semantic feature vectors that contain both skeletal motion details and behavioral semantics.

[0038] Gated feature fusion is the final step in cross-modal integration. This process dynamically adjusts the fusion ratio of the two types of interactive features through a learnable gating mechanism. The gating signal is generated based on the contextual information of the current teaching scenario. For tasks requiring precise motion analysis, it favors skeletal motion features, while for tasks requiring semantic understanding, it emphasizes behavioral semantic features. The fused feature set not only retains the core information of the two types of original data but also generates new joint representations through cross-modal interaction. These representations can more comprehensively reflect the overall state of children in teaching interactions.

[0039] At the technical implementation level, the system's dynamic trajectory feature extraction and cross-modal fusion modules employ a distributed computing architecture. The computational tasks of the graph convolutional network model and attention mechanism are distributed across multiple processing units for parallel execution, significantly improving processing efficiency. To address the high-dimensionality of skeletal point data, the system utilizes feature dimensionality reduction and sparsity techniques, reducing computational complexity while preserving key information. The temporal alignment and attention computation during cross-modal fusion have also undergone algorithmic optimization to ensure high analytical accuracy under real-time requirements.

[0040] The system's anti-interference capability is fully demonstrated in this embodiment. For abnormal skeletal point data caused by sensor noise or transient occlusion, the dynamic trajectory feature extraction module repairs the data through temporal context analysis and spatial consistency verification. During cross-modal fusion, when the data quality of one modality deteriorates, the system automatically adjusts the fusion weights, relying more on reliable information from another modality.

[0041] See Figure 5 This chart demonstrates the entire process of dynamic trajectory feature extraction and processing from skeletal motion time-series data. The chart contains six sub-charts: Original skeletal motion time-series data: showing the motion trajectories of multiple joints, with different grayscale lines representing different joint movement patterns; Abnormal motion interval detection: showing how the system identifies abnormal fluctuation time segments in joint angle sequences, with gray-filled areas indicating detected abnormal intervals; Multi-scale sliding window sampling processing: using different grayscale lines to show abnormal indicators after sliding averaging at different window sizes, used to capture motion features at different time scales; Joint association feature matrix: showing the motion correlation between joints through a grayscale matrix, with darker colors indicating a higher degree of correlation; Local motion feature vector: showing local motion features extracted from different time segments, with different grayscale lines representing feature vectors for different time periods; Global skeletal motion feature vector: using grayscale bar charts to show the global motion features aggregated through an attention mechanism, with darker bar colors indicating more significant features in that dimension.

[0042] Example 3: The processing of the fused interactive feature set by the dynamic weighted network model is the core component of the system's teaching interaction sensitivity analysis. This model employs a hierarchical processing architecture, progressively extracting key information from the original fused features and establishing the correlation between teaching segments and interactive responses. The system first inputs the fused interactive feature set into a feature filtering layer, which uses sparse coding principles to compress and select high-dimensional features. During sparse coding, each input feature is represented as a linear combination of a small number of basis vectors, obtained through pre-training, capable of capturing typical patterns in teaching interactions. The core interactive feature set output by the feature filtering layer retains the most discriminative information from the original data while significantly reducing feature dimensionality, providing a more compact data representation for subsequent processing.

[0043] The cross-association layer performs high-order feature combination analysis on the core interaction feature set. This layer mines the potential correlations between different teaching stages through tensor operations, expanding the one-dimensional feature vector into a multi-dimensional correlation feature tensor. The system employs the following computational method during tensor construction:

[0044] in: Represents the elements in the three-dimensional correlation feature tensor. , and These represent the core interactive feature matrices from different feature channels. This represents the tensor product operation. These are learnable weight coefficients. This refers to the number of feature channels. This calculation process can capture complex multi-factor relationships in teaching interactions, such as the non-linear relationship between changes in children's attention and the difficulty of teaching materials. The generated correlation feature tensor contains rich cross-feature information, providing a multi-dimensional data foundation for sensitivity analysis.

[0045] The sensitivity allocation layer calculates the interaction response weights for different teaching stages based on the associated feature tensor. This layer employs a multi-branch neural network structure, with each branch corresponding to a teaching stage type. The network first performs channel separation on the input tensor, mapping features of different dimensions to the corresponding teaching stage analysis space. Within each branch, stage-specific features are extracted through cascaded convolutions and nonlinear transformations. Finally, normalization is applied to generate the interaction response sensitivity value for that stage. The sensitivity value calculation comprehensively considers factors such as historical interaction patterns, current teaching status, and individual differences among children, resulting in a dynamically weighted evaluation result.

[0046] The feature weighted aggregation stage combines the associated feature tensor with the sensitivity distribution to generate the final set of teaching interaction sensitivity coefficients. The aggregation process employs an attention-based feature selection mechanism, assigning greater aggregation weights to feature dimensions with higher sensitivity. This dynamic weighting method ensures that the system can flexibly adjust the degree of attention given to different interaction features according to actual teaching needs. For example, in teaching segments requiring fine motor guidance, the system enhances the sensitivity to skeletal motion features; while in segments emphasizing social interaction, it increases the response weight to behavioral semantic features.

[0047] The training process of the dynamically weighted network model employs an end-to-end optimization strategy. The loss function is designed as a combination of feature reconstruction error and sensitivity prediction error, where feature reconstruction error measures the quality of reconstructing core interaction features, and sensitivity prediction error assesses the accuracy of classifying teaching segments. The training data includes interaction records from various typical teaching scenarios, covering behavioral patterns and teaching interaction types of children of different ages. Through large-scale data training, the model can learn feature representations and sensitivity evaluation criteria with generalization capabilities.

[0048] In terms of model architecture design, the dynamic weighted network adopts a modular component structure. The feature selection layer, cross-correlation layer, and sensitivity allocation layer can be updated and replaced independently, facilitating customized adjustments for specific teaching needs. The model also supports an incremental learning mechanism, enabling it to adapt to new teaching scenarios and child groups through local parameter updates without retraining the entire network. This flexibility allows the system to continuously optimize its interactive analysis capabilities and maintain long-term teaching adaptability.

[0049] In this embodiment, the system's real-time processing capabilities are significantly optimized. The dynamically weighted network model employs a lightweight design, reducing computational complexity through depthwise separable convolutions and parameter sharing techniques. The construction process of the associated feature tensor utilizes tensor decomposition, transforming high-dimensional operations into a combination of multiple low-dimensional operations, significantly improving processing efficiency. The multi-branch structure of the sensitivity allocation layer supports parallel computing, allowing analysis tasks from different teaching stages to be performed simultaneously. These optimizations ensure that the system can complete complex sensitivity analyses in real time during the teaching process, providing support for immediate policy adjustments.

[0050] The dynamic weighted network model exhibits strong robustness in handling noise and uncertainty. To address potential interference during sensor data acquisition, the model incorporates an outlier suppression mechanism in the feature selection layer, identifying and removing unreliable feature components through statistical testing. The tensor operations in the cross-correlation layer possess inherent noise smoothing properties, effectively mitigating the impact of random errors. The sensitivity allocation layer employs an integrated decision-making strategy, combining the analysis results from multiple feature channels to generate the final sensitivity assessment, reducing the interference of single feature fluctuations on the overall judgment.

[0051] The interpretability of the model is also a key feature of this embodiment. The dynamic weighted network generates sensitivity coefficients while simultaneously providing feature contribution analysis, intuitively demonstrating the influence of each factor on the final decision. This transparent analysis process helps teachers understand the system's decision-making logic and facilitates the smooth implementation of human-computer collaborative teaching. The system also supports the visualization of sensitivity analysis results, displaying changes in response patterns across different teaching stages through a graphical interface, providing an intuitive reference for teaching reflection and improvement.

[0052] The application scope of the dynamically weighted network model extends beyond basic instructional interaction analysis. By adjusting the input feature types and output sensitivity definitions, the model can be extended to a wider range of educational scenarios. For example, in special education, the model can be configured to focus on interaction features specific behavioral disorders; in language teaching, it can be adjusted to emphasize sensitivity analysis of speech and facial expressions. This scalability enables the system to adapt to diverse educational needs, providing customized interaction analysis solutions for different scenarios.

[0053] Example 4: In the actual operation of the robotic early childhood education system, the root cause analysis module is responsible for accurately tracing the source of abnormal teaching behaviors. Taking a specific block-building teaching scenario as an example, if the system detects that a child fails multiple times while imitating building a tower-like structure, the system will initiate the abnormal source tracing process. First, the system filters out abnormal items that exceed a preset dynamic threshold based on the distribution of interaction response sensitivity, generating a candidate abnormal feature set. This set includes abnormal indicators in multiple dimensions, such as abnormal hand joint movement amplitude and insufficient attention duration.

[0054] Table 1: Set of candidate anomaly features in this scenario.

[0055]

[0056] In the reverse feature parsing phase, the system maps the abnormal features in the table to specific descriptions of movement abnormalities and behavioral deviations. For the AF-102 feature, the system identifies that the child has difficulty in fine motor control, manifested as inaccurate finger pinching angle when grasping blocks; the AF-203 feature reflects the child's insufficient attention span, with frequent head turning causing the gaze to deviate from the teaching area; and the AF-305 feature shows that the child has a bias in judging spatial position, resulting in inaccurate placement of blocks.

[0057] The source-tracing inference model uses a graph structure to analyze the causal relationships between these anomalous features. The model constructs a three-layered inference graph: bottom-layer nodes represent specific anomalous features, middle-layer nodes represent potential problems in teaching processes, and top-layer nodes correspond to root cause classifications. Taking feature AF-102 as an example, the model, through analysis of historical data, found a strong correlation between this feature and two middle-layer nodes: "insufficient fine motor skill instruction" and "excessively fast teaching pace." Combined with the feature AF-203 (attention deficit), the model further infers that the root cause may lie in "the teaching pace not matching the children's learning abilities."

[0058] During the confidence assessment phase, the system calculates the support index for each source tracing path. Support calculation considers three dimensions: historical case matching degree, logical coherence between features, and real-time contextual fit. In the aforementioned block-building case, the "teaching rhythm mismatch" path received the highest comprehensive score because it simultaneously explained both abnormal hand movements and distraction, and highly matched the current teaching progress data. The system ultimately identified this path as the target source tracing result, clarifying that the anomaly stemmed from rhythm control issues in the group teaching process, with the behavioral deviation type being difficulty in motor execution accompanied by distraction.

[0059] The adaptive teaching strategy generation process makes targeted adjustments based on the source tracing results. The system first queries the historical strategy database, retrieving a set of historical strategies related to "adjusting the teaching pace." This set includes various intervention methods, such as segmented demonstration, slowing down the pace, and individual guidance. In the effectiveness screening stage, the system prioritizes strategies that have simultaneously improved motor execution and attention maintenance in similar past cases. For example, the combined strategy of "step-by-step breakdown + voice prompts" in the strategy database has shown good overall results in historical records.

[0060] During the strategy optimization phase, the system conducts a multi-dimensional evaluation of the selected effective strategies. Taking a block-building scenario as an example, the system weighs three key indicators: task completion progress, improvement in action accuracy, and attention maintenance duration. While a step-by-step demonstration strategy can improve action accuracy, it may prolong task time; while adding voice prompts helps maintain attention, it may increase information load. By balancing these factors, the system ultimately generates an adaptive strategy that divides the building steps into three stages, each accompanied by concise voice prompts.

[0061] During strategy implementation, the system continuously monitors changes in children's responses. For the block-building example, the adjusted teaching strategy is as follows: the robot first demonstrates the base construction, waits for the children to complete it before demonstrating the middle layer, and finally guides the children in building the top. A short voice prompt is played at the beginning of each stage, such as "Now please place the blocks in the marked positions." The system verifies the effectiveness of the strategy by collecting real-time data on children's joint movements and gaze direction. If attention is detected to be diverted again or the error rate increases, the system will trigger a secondary adjustment of the strategy.

[0062] The closed-loop process of anomaly tracing and strategy adjustment demonstrates the system's dynamic adaptability. In the later stages of the block-building case, the system discovered new difficulties encountered by children after completing the base. Source analysis revealed that this was due to insufficient spatial positioning ability leading to alignment problems with the middle-layer blocks. The system then, while maintaining the step-by-step teaching framework, added visual markers to assist in the learning process, displaying alignment reference lines on the contact surfaces of the blocks. This layered and in-depth adjustment approach allows the teaching strategies to dynamically evolve along with the children's learning progress.

[0063] The knowledge update mechanism of the causal reasoning model is also a key feature of this embodiment. The system periodically incorporates new teaching cases and strategy effectiveness feedback into the analysis model, optimizing the reasoning rules through incremental learning. For example, when a correlation is found between a certain type of attention deficit and changes in the classroom environment, the model automatically adjusts the weights of the relevant features. This continuous evolution capability allows the system's analytical accuracy to improve over time, gradually forming customized analysis models for specific teaching environments and preschool groups.

[0064] Example 5: See Figure 6 The process of generating adaptive teaching strategies through multi-objective optimization is a key step in the system's realization of personalized teaching. This process, based on the effective set of interaction strategies selected in the previous stage, uses multi-dimensional comprehensive evaluation and optimization to ultimately determine the most suitable adjustment plan for the current teaching scenario and the child's state. The system first constructs a three-dimensional optimization objective space, with each dimension representing a core indicator that the teaching strategy needs to consider. The first dimension is the teaching progress completion rate, reflecting the strategy's effectiveness in advancing the established teaching objectives; the second dimension is the behavior deviation correction rate, measuring the degree to which the strategy improves abnormal behaviors; and the third dimension is the timeliness of interaction response, assessing the timeliness and fluency of strategy execution. These three dimensions together constitute a three-dimensional framework for strategy evaluation, ensuring that the generated strategies balance teaching effectiveness and interactive experience.

[0065] In the target space mapping phase, the system transforms each candidate strategy into coordinate points in three-dimensional space. The coordinate values ​​are calculated based on the strategy's historical execution data and the real-time characteristics of the current scenario. The teaching progress completion rate is derived by analyzing the task progression speed of the strategy in similar scenarios, while also considering the remaining task load in the current teaching phase. The behavioral deviation correction rate is comprehensively evaluated based on children's real-time feedback data and the strategy's past correction effects. The timeliness of the interactive response is calculated based on the preparation and implementation time required for strategy execution. This mapping method transforms abstract teaching strategies into quantifiable and comparable spatial locations, providing an intuitive basis for subsequent optimization and selection.

[0066] Multi-objective optimization algorithms seek the optimal set of strategies in a three-dimensional objective space. The system employs an improved Pareto optimization method to select candidate strategies that are not completely dominated by other strategies in any of the three dimensions. These strategies are distributed across a Pareto front, representing strategy choices with different emphases: some prioritize rapid advancement of teaching progress, some focus on thoroughly correcting behavioral deviations, and others emphasize maintaining smooth interaction. The algorithm ensures the representativeness of the strategy distribution on the front by calculating crowding density, avoiding over-concentration of selection in a particular local area. This approach preserves strategy diversity, providing ample selection space for subsequent context adaptation.

[0067] The strategy adaptation model adaptively scores candidate strategies based on the real-time state of the current teaching scenario. The scoring process considers three core factors: the child's immediate state characteristics, environmental conditions, and the requirements of the teaching stage. For the child's state, the model analyzes their current attention level, emotional state, and fatigue level; environmental conditions include classroom noise levels, light intensity, and the presence of distractions; and the requirements of the teaching stage involve the course schedule and the core objectives of the current lesson. The model dynamically adjusts the weighting of each factor through a hierarchical weighting mechanism. For example, at the end of the lesson, more emphasis may be placed on completing the teaching schedule, while when children show obvious signs of fatigue, the comfort of the interaction will be prioritized.

[0068] In the final strategy selection phase, the system comprehensively compares the adaptability scores of each candidate strategy. The strategy with the highest score is not necessarily the best in all dimensions, but rather the solution that achieves the optimal balance in the current specific context. Taking block sorting instruction as an example, the system might choose a strategy that has moderate progress but significantly improves children's operational accuracy, rather than simply pursuing a solution that aims to complete the task quickly. This selection mechanism reflects the system's understanding of the essence of teaching, focusing not only on the completion of the surface task but also on the children's genuine gains in the process.

[0069] Fine-tuning the strategy parameters is the final step in the optimization process. Based on the scoring results, the system adjusts the execution details of the selected strategy, including the speed of movement demonstration, the frequency of verbal prompts, and the waiting time for responses. These parameter adjustments are based on real-time feedback data from children and are gradually optimized in small steps. For example, when the system detects that a child has difficulty understanding the current demonstration speed, it automatically slows down the robot's movements and adds periodic pauses. This refined parameter adjustment allows general strategies to adapt to individual differences, achieving truly personalized teaching.

[0070] The dynamism of multi-objective optimization is reflected in the continuous strategy evaluation mechanism. While executing the selected strategy, the system continuously monitors the achievement of the three objective dimensions. When the actual results deviate from expectations, the system triggers a strategy re-optimization process. This closed-loop optimization mechanism ensures that teaching strategies can be dynamically adjusted according to the teaching progress and changes in the children's state, avoiding the rigidity that may result from fixed strategies. In actual teaching, this dynamism manifests as a gradual evolution of strategies, transitioning from initial behavior correction to ability development, and finally leading to independent exploration.

[0071] The system optimization process also considers the long-term impact of teaching strategies. In addition to the three immediate target dimensions, the model implicitly assesses the potential impact of strategies on children's learning interest and self-confidence. By analyzing the long-term effects of similar strategies in historical data, the system prioritizes those solutions that address the current problem while also promoting future development. For example, in selecting motor instruction strategies, the system may favor guided discovery over direct demonstration, as the former, although slower initially, is more conducive to cultivating children's exploratory abilities and problem-solving skills.

[0072] The implementation of multi-objective optimization relies on a rich policy knowledge base built by the system. This knowledge base not only stores the basic descriptions and execution parameters of various policies, but also records a large amount of historical execution case performance data. Each policy entry is associated with multi-dimensional evaluation indicators and applicable conditions, providing sufficient decision-making basis for the optimization algorithm. The knowledge base uses a graph structure to organize policy data, facilitating the system's rapid retrieval of relevant policies and discovery of potential combinations. This knowledge representation method supports innovative combinations of policies, enabling the system to generate new solutions that surpass the original policy library.

[0073] The system's optimization process also reflects considerations of educational ethics. In strategy selection, the system avoids approaches that, while offering quick results, may harm children's autonomy. For example, it avoids strategies that involve excessive prompting or taking over tasks to improve short-term task completion rates. This design philosophy ensures that the system's teaching practices align with fundamental principles of child development, protecting children's learning autonomy and enjoyment of exploration while pursuing effective learning outcomes.

[0074] The implementation of multi-objective optimization also relies on efficient real-time computing capabilities. The system adopts a hierarchical computing architecture, distributing time-consuming policy evaluation tasks to multiple processing units for parallel execution. The core optimization algorithm has been specially optimized to complete the screening and comparison of a large-scale policy set within a finite time. This computational efficiency ensures the system's real-time response capability in actual teaching, preventing significant interactive delays in the complex multi-objective optimization process.

[0075] Example 6: The real-time feedback and strategy optimization module constitutes the core mechanism for the system to dynamically adjust teaching. This module runs continuously during the robot's execution of adaptive teaching strategies. Through multi-source data acquisition and incremental learning technology, the teaching strategies can continuously evolve in accordance with the children's real-time state. The system collects children's physiological response data through non-invasive sensing devices, including heart rate variability measured by wrist-worn sensors and emotional arousal data obtained by finger-contact skin conductivity sensors. These physiological signals are recorded with millisecond-level time resolution, forming a continuous data stream reflecting the children's internal state. Simultaneously, multi-view visual sensors deployed in the classroom environment capture children's behavioral feedback data, including changes in facial micro-expressions, the amplitude of limb movements, and interaction methods with teaching materials. These multimodal data streams are aligned through time synchronization technology to construct a multi-dimensional observation system that comprehensively reflects the children's acceptance state.

[0076] The analysis of physiological response data employed a time-series pattern-based approach. The system identified rhythmic variations in heart rate variability signals, distinguishing between different state patterns such as attention span, cognitive load, and emotional fluctuations. Skin conductance signals were analyzed for amplitude fluctuations and duration characteristics to assess children's level of tension and engagement when facing teaching tasks. The analysis results of these physiological indicators were correlated with the timing of the implementation of teaching strategies, forming an intrinsic basis for evaluating strategy effectiveness. For example, when the system detected a specific rhythmic enhancement in heart rate variability indicators after strategy adjustment, it might reflect an increase in children's cognitive engagement; while a steady decrease in skin conductance levels might indicate an improvement in emotional state.

[0077] The processing of behavioral feedback data integrates computer vision and motion analysis technologies. Facial expression recognition algorithms extract motion unit features from the eye and mouth regions of the video stream, distinguishing states such as confusion, understanding, and excitement. Body movement analysis focuses on changes in how children interact with teaching materials, such as adjustments in grip strength when building blocks or changes in brush pressure when drawing. These explicit behavioral features are cross-validated with internal physiological response data, collectively forming a multi-faceted evaluation of strategy implementation effectiveness. The system pays particular attention to subtle changes in behavioral feedback, such as brief shifts in eye gaze or slight hesitations in finger movements; these details often reflect key areas in the teaching strategy that require optimization.

[0078] The incremental learning model is responsible for transforming real-time collected feedback data into strategy optimization knowledge. The model employs an online learning architecture, using the currently implemented teaching strategy as a baseline. It calculates strategy correction coefficients by analyzing the direction and extent of deviation between the feedback data and the expected results. This coefficient is a multi-dimensional vector containing adjustment suggestions for various parameters in the strategy, such as the adjustment amount for the speed of action demonstrations, the magnitude of changes in the frequency of verbal cues, and the adjustment value for the waiting time for responses. The coefficient calculation process considers the temporal characteristics and multimodal consistency of the feedback data; when physiological responses and behavioral feedback point to the same conclusion, the correction magnitude is increased accordingly; when there are discrepancies between different data sources, a conservative adjustment approach is adopted.

[0079] The dynamic adjustment of strategy parameters follows the principle of gradual optimization. The system does not make drastic strategy changes based on a single set of feedback data. Instead, it establishes trends through continuous observation before implementing phased adjustments. This design avoids strategy oscillations caused by accidental factors, ensuring the stability of the teaching process. During the adjustment process, the system maintains a record of the strategy's evolution trajectory, tracking the historical changes and corresponding effects of various parameters, forming a traceable optimization process. For example, in adjusting the frequency of language prompts, the system might start with a prompt every two minutes, gradually fine-tuning it to the optimal interval based on the children's responses, rather than directly adopting a preset fixed frequency.

[0080] The adjusted strategy was validated using a closed-loop testing method. After parameter adjustments, the system maintained a set observation window, focusing on monitoring changes in feedback indicators related to the adjustment goals. For example, when adjusting the speed of action demonstrations, the system paid particular attention to the accuracy and completion time of children's imitation; when changing the language prompts, it focused on analyzing children's eye-following rate and response delay. This goal-oriented validation mechanism ensured that each adjustment yielded a clear effect assessment, providing a reliable basis for subsequent optimization. During the validation process, the system distinguished between short-term adaptive effects and long-term stable effects, avoiding misjudging children's temporary adaptation to the new strategy as its effectiveness.

[0081] The effectiveness of real-time feedback and strategy optimization is reflected in the dynamic adaptability of the teaching process. In the block sorting teaching case, the system initially adopted a standard strategy of sorting by shape. Through real-time monitoring, it was found that children exhibited increased skin conductance and hesitation when sorting round blocks. The system gradually adjusted its strategy: first, it added tactile cues for round blocks; then, it introduced size comparisons to assist in sorting; and finally, it adjusted to sorting by a combination of shape and size features. This progressive strategy evolution kept the teaching difficulty within the optimal learning range for children, preventing it from becoming too easy and losing its challenge, or too difficult and causing frustration.

[0082] The system's incremental learning mechanism features knowledge accumulation. Each optimization experience generated from a teaching interaction is abstracted into rule-based knowledge and stored in the strategy optimization knowledge base. This knowledge is categorized and indexed according to dimensions such as teaching scenario type, children's age stage, and behavioral characteristics, forming transferable optimization experiences. When encountering similar teaching situations, the system can quickly retrieve relevant optimization history, accelerating the strategy adjustment process in new scenarios. The knowledge base also records long-term effect data of different strategy adjustment schemes, enabling the system to evaluate the comprehensive benefits of strategy optimization from a more macro-timescale perspective, avoiding short-sighted local optimization.

[0083] The timeliness of real-time feedback processing is a key technical feature of this embodiment. The system employs a streaming computing architecture, keeping the latency of the entire process from the acquisition and parsing of physiological signals and behavioral data to strategy adjustment within the second range. This real-time capability ensures a natural and smooth teaching interaction, avoiding disconnections caused by system latency. The data processing pipeline uses a priority scheduling mechanism to ensure that critical feedback signals are responded to immediately, while secondary indicators can have more lenient time requirements. For example, a child's confused expression or sudden pause in movement will trigger an immediate interruption, while fluctuations in normal physiological indicators are incorporated into a periodic adjustment plan.

[0084] Multimodal fusion analysis of feedback data is another significant feature of this embodiment. The system does not rely on a single data source for judgment, but rather integrates intrinsic state indicators of physiological signals with explicit evidence of behavioral performance to form a comprehensive assessment conclusion. When facial expressions show confusion but body movements remain fluid, the system may determine that the child is on the verge of a cognitive breakthrough and choose to maintain the current strategy for a short period of observation; conversely, when physiological indicators show tension accompanied by behavioral withdrawal, supportive strategy adjustments will be immediately initiated. This multi-angle cross-validation analysis method significantly improves the accuracy of state judgment, making strategy adjustments more precise and effective.

[0085] The system's optimization process also considers individual differences and fluctuations in children's states. The same feedback data may correspond to different optimization directions for different children or at different times. By establishing individual baseline profiles for each child, the system distinguishes between their normal behavior and temporary changes, avoiding misjudging individual characteristics as problems requiring correction. For example, a high frequency of movement in a naturally active child may not require intervention; however, when the same behavior occurs in a normally quiet child, it may be necessary to investigate the underlying reasons. This personalized interpretation capability allows the system to transcend mechanical, standardized responses and achieve truly individualized teaching support.

[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0087] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A robot-based early childhood education system based on motion-sensing interaction, characterized in that, The system includes: The motion data acquisition module is used to acquire the time-series data of the skeletal movement of the target child and the video stream of the interactive behavior through a depth camera and an inertial sensor; The multimodal feature fusion module is used to perform dynamic trajectory feature extraction processing on the motion time series data of the skeleton points to generate skeleton motion feature vectors, perform action semantic parsing processing on the interactive behavior video stream to generate behavior semantic feature vectors, and perform cross-modal association and fusion of the skeleton motion feature vectors and the behavior semantic feature vectors to generate a fused interactive feature set. The interaction sensitivity calculation module is used to call a pre-trained dynamic weighted network model to perform attention allocation processing on the fused interaction feature set and generate a set of teaching interaction sensitivity coefficients, which includes the distribution of interaction response sensitivity corresponding to different teaching stages. An abnormal behavior tracing module is used to perform joint root cause analysis on the motion time series data of the skeletal points and the video stream of the interactive behavior based on the interaction response sensitivity distribution, and generate teaching behavior abnormality tracing results. The teaching behavior abnormality tracing results are used to indicate the teaching link and behavior deviation type from which the abnormality originates. The interaction strategy generation module is used to generate an adaptive teaching strategy based on the abnormal teaching behavior tracing results, and send the adaptive teaching strategy to the robot motion control unit.

2. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, The step of performing action semantic parsing processing on the interactive behavior video stream to generate a behavior semantic feature vector includes: The interactive video stream is processed by action sequence segmentation to obtain multiple continuous action segments; A pre-trained 3D convolutional network model is invoked to perform spatiotemporal feature encoding on each of the continuous action segments to generate an initial action feature vector; The initial action feature vector is subjected to teaching behavior feature enhancement processing, including: matching a set of standard behavior templates associated with the current action segment from a preset children's behavior knowledge base; inputting the set of standard behavior templates into a feature encoder for template semantic embedding processing to generate a set of template feature vectors; and performing similarity weighted fusion between the set of template feature vectors and the initial action feature vector to generate an enhanced action feature vector. The enhanced action feature vectors of each continuous action segment are integrated by temporal position encoding to generate the behavioral semantic feature vector.

3. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, The step of performing dynamic trajectory feature extraction processing on the temporal data of the skeletal point motion to generate a skeletal motion feature vector includes: The abnormal motion interval detection process is performed on the motion time sequence data of the skeletal points to identify abnormal fluctuation time segments in the joint angle change sequence; Within the abnormal fluctuation time segment, the original skeletal data is subjected to multi-scale sliding window sampling to obtain multiple local trajectory data segments; A pre-trained graph convolutional network model is invoked to perform joint association feature extraction processing on each of the local trajectory data segments, generating a local motion feature vector; Global trajectory aggregation processing is performed on the local motion feature vectors of each of the local trajectory data segments to generate the skeletal motion feature vector.

4. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, The step of cross-modal association and fusion of the skeletal motion feature vector and the behavioral semantic feature vector to generate a fused interactive feature set includes: The behavioral semantic feature vector is aligned to the time dimension and mapped to the same timestamp sequence as the skeletal motion feature vector. A cross-attention mechanism for skeletal behavior is constructed, and the cross-modal correlation matrix between the trajectory features of each timestamp in the skeletal motion feature vector and the semantic features of the corresponding timestamp in the behavior semantic feature vector is calculated. Based on the cross-modal correlation matrix, the skeletal motion feature vector and behavioral semantic feature vector are subjected to bidirectional feature interaction processing to generate interactive trajectory feature vector and interactive semantic feature vector. The interaction trajectory feature vector and the interaction semantic feature vector are subjected to gated feature fusion processing to generate the fused interaction feature set.

5. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, The process of calling a pre-trained dynamically weighted network model to perform attention allocation on the fused interactive feature set, generating a set of teaching interaction sensitivity coefficients, including: The fused interactive feature set is input into the feature filtering layer of the dynamic weighted network model for key feature extraction processing to obtain the filtered core interactive feature set. The cross-association layer of the dynamic weighted network model is invoked to perform high-order feature combination processing on the core interaction feature set, generating an association feature tensor; The sensitivity allocation layer of the dynamic weighted network model is used to calculate the weights of the associated feature tensors for different teaching stages, thereby generating the interactive response sensitivity distribution for different teaching stages. Based on the interaction response sensitivity distribution, the associated feature tensor is subjected to feature weighting and aggregation processing to generate the set of teaching interaction sensitivity coefficients.

6. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, The method of performing joint root cause analysis on the skeletal point motion time-series data and interactive behavior video stream based on the interactive response sensitivity distribution to generate abnormal teaching behavior tracing results includes: Based on the interaction response sensitivity distribution, abnormal sensitivity items exceeding a preset threshold are filtered to generate a candidate abnormal feature set; Each abnormality sensitivity item in the candidate abnormality feature set is subjected to reverse feature parsing to determine its corresponding skeletal motion abnormality description and behavioral semantic deviation description. The pre-trained source tracing inference model is invoked to perform joint causal analysis on the description of abnormal skeletal movement and the description of deviation in behavioral semantics, generating a set of abnormal source tracing paths; The set of abnormal tracing paths is subjected to confidence evaluation, and the target tracing path with the highest confidence is selected as the abnormal tracing result of the teaching behavior.

7. The robot-based early childhood education system based on motion-sensing interaction according to claim 6, characterized in that, The step of generating an adaptive teaching strategy based on the abnormal teaching behavior tracing results includes: The analysis reveals the abnormal teaching processes and behavioral deviation types in the results of the abnormal teaching behavior tracing. Match a set of historical adjustment strategies associated with the teaching segment from the history teaching strategy library that corresponds to the source of the anomaly; Based on the behavioral deviation type, the set of historical adjustment strategies is filtered for effectiveness to obtain a set of effective interaction strategies. The effective set of interaction strategies is subjected to multi-objective optimization to generate adaptive teaching strategies that meet the teaching constraints.

8. The robot-based early childhood education system based on motion-sensing interaction according to claim 7, characterized in that, The step of performing multi-objective optimization on the set of effective interaction strategies to generate adaptive teaching strategies that meet teaching constraints includes: Construct a three-dimensional optimization target space that includes teaching progress completion rate, behavioral deviation correction rate, and interaction response timeliness; Map each effective interaction strategy to a coordinate point in the three-dimensional optimization target space; A set of candidate policies located on the Pareto front is selected using a multi-objective optimization algorithm; The strategy adaptation model is invoked to perform adaptive scoring on the candidate strategy set based on the real-time status of the current teaching scenario; The candidate strategy with the highest adaptability score is selected as the adaptive teaching strategy.

9. The robot-based early childhood education system based on motion-sensing interaction according to claim 1, characterized in that, Also includes: The real-time feedback module is used to continuously collect physiological response data and behavioral feedback data of children during the process of the robot executing the adaptive teaching strategy. The strategy optimization module is used to input the physiological response data and behavioral feedback data into the incremental learning model for dynamic strategy optimization processing, and generate an optimized adaptive teaching strategy.

10. The robot-based early childhood education system based on motion-sensing interaction according to claim 9, characterized in that, When the strategy optimization module performs dynamic strategy optimization processing, it includes: The data on the effectiveness of the current adaptive teaching strategy are correlated with the physiological response data to generate a strategy correction coefficient. The interactive response parameters in the adaptive teaching strategy are dynamically adjusted based on the strategy correction coefficient. The adjusted interactive response parameters are matched and verified with the behavioral feedback data to generate the optimized adaptive teaching strategy.

Images