Intelligent children learning system based on carpet with replaceable carpet surface

Abstract

The invention relates to an intelligent child learning system based on a carpet with a replaceable carpet surface. The system comprises a replaceable carpet surface module, a multi-mode sensing module, a data preprocessing module, a task management module, an association recommendation module and an interaction feedback module. According to the invention, original data is obtained through data acquisition; a data preprocessing method of multi-source data synchronous alignment, abnormal data detection and restoration, multi-modal feature normalization and space coordinate unified mapping is adopted; a carpet with a replaceable carpet surface is used as a physical interaction medium, so that the learning immersion and the touch experience are enhanced while the learning environment is highly extensible; a multi-modal deep learning model is adopted as a task management model, various perception data can be fused, and historical memory information is adaptively combined; an improved graph convolution model is adopted as an association recommendation model, multi-dimensional association between learning tasks is dynamically modeled, the current learning state is considered, historical behavior data is fused, and a highly-adaptive recommendation list is generated.

Description

Technical Field

[0001] This invention relates to the field of children's intelligent learning technology, specifically to an intelligent children's learning system based on replaceable carpet surfaces. Background Technology

[0002] The intelligent children's learning system is an educational technology that combines children's actual actions with virtual tasks for interaction. Utilizing sensors and image recognition technology, it enables children to complete learning tasks in a real physical environment. By recognizing children's movements and task completion status through cameras, it adjusts the task difficulty in real time, providing a personalized learning experience and enhancing the interactivity and adaptability of children's education. This offers families and educational institutions a more flexible and intelligent learning tool.

[0003] However, traditional intelligent children's learning systems suffer from several technical problems. First, they rely on fixed or single interactive interfaces, limiting the diversity and adaptability of learning topics and leading to monotonous and repetitive learning content. Second, their systems have poor scalability, making it difficult to integrate new tasks or upgrade technology. Third, their model design depends on single or limited data sources, failing to effectively integrate spatiotemporal context information, thus easily leading to misjudgments or evaluation biases. Furthermore, they lack advanced feature fusion and memory mechanisms, making it difficult to dynamically adapt to individual differences, resulting in rigid and static learning task management. Fourth, their learning task recommendation methods are often based on simple rules or static task associations, ignoring the dynamic interactions and complex dependencies between learning tasks, leading to recommendations that are often out of sync with children's actual progress and interests. Summary of the Invention

[0004] To address the above issues and overcome the shortcomings of existing technologies, this invention provides an intelligent children's learning system based on replaceable carpet surfaces. Traditional intelligent children's learning systems suffer from limitations such as fixed or single interactive interfaces, restricting the diversity and adaptability of learning themes, leading to monotonous and repetitive learning content, poor system scalability, and difficulty in integrating new tasks or upgrading technologies. This solution creatively employs replaceable carpet surfaces as the physical interaction medium, allowing for flexible switching of learning themes through quick carpet replacement. The learning environment is highly scalable while enhancing immersion and tactile experience. Furthermore, traditional intelligent children's learning systems address issues such as model design relying on single or limited data sources, failing to effectively integrate spatiotemporal context information, easily leading to misjudgments or evaluation biases, and lacking advanced feature fusion and memory mechanisms, making it difficult to dynamically adapt to individual differences and manage learning tasks effectively. Addressing the rigid and static technical issues, this solution creatively employs a multimodal deep learning model as the task management model. By fusing various perceptual data and adaptively combining historical memory information, it can more accurately identify learning topics and assess task completion, thereby achieving refined management and personalized adjustments to the learning process. Furthermore, addressing the technical problem of traditional intelligent children's learning systems where task recommendation methods are typically based on simple rules or static task associations, ignoring the dynamic interactions and complex dependencies between learning tasks, leading to recommendations that often fail to reflect children's actual progress and interests, this solution creatively uses an improved graph convolutional model as the association recommendation model. By dynamically modeling the multidimensional relationships between learning tasks, considering the current learning state while incorporating historical behavioral data, it generates a highly adaptive recommendation list, ensuring that recommended tasks match the child's ability level.

[0005] The technical solution adopted by the present invention is as follows: The intelligent children's learning system based on replaceable carpet provided by the present invention includes a replaceable carpet module, a multimodal perception module, a data preprocessing module, a task management module, an association recommendation module, and an interactive feedback module;

[0006] The replaceable carpet module, based on the carpet base and the replaceable carpet assembly, adopts a quick-release structure to connect with the base layer, allowing for rapid replacement to adapt to different learning tasks.

[0007] The multimodal perception module collects data through the visual acquisition component and sensor component in the carpet base to obtain the task management raw dataset, and sends the task management raw dataset to the data preprocessing module;

[0008] The data preprocessing module employs data preprocessing methods such as multi-source data synchronization and alignment, abnormal data detection and repair, multi-modal feature normalization, and unified spatial coordinate mapping to obtain a preliminary task management dataset, and then sends the preliminary task management dataset to the task management module.

[0009] The task management module is used to build the model required for learner learning task management. By constructing a multimodal deep learning model as the task management model, it obtains the learner's learning topic and task completion evaluation value based on the input data, and sends the task completion evaluation value to the association recommendation module, and sends the learning topic and the task completion evaluation value to the interactive feedback module.

[0010] The associated recommendation module is used to recommend subsequent learning tasks. By constructing an improved graph convolutional model as the associated recommendation model, it obtains the recommendation score of subsequent learning tasks based on the input data, determines the associated recommended learning tasks for learners, and sends the associated recommended learning tasks to the interactive feedback module.

[0011] The interactive feedback module provides interactive feedback to learners through voice broadcasting, based on the learning topic, the task completion evaluation value, and the associated recommended learning tasks.

[0012] Furthermore, in the replaceable carpet module, the carpet base and the replaceable carpet assembly are connected to the base layer via a quick-release structure. The carpet base specifically includes a visual acquisition component and a sensor component for data acquisition. The replaceable carpet assembly specifically includes multiple carpets with different learning themes and replaceable task identification items for different learning tasks contained in each carpet. Each carpet has a unique identification mark embedded in it. The learning theme can be switched by installing the carpet with the corresponding learning theme on the carpet base, and the learning task can be switched by installing the replaceable task identification items for the corresponding learning task on the carpet.

[0013] Furthermore, in the multimodal perception module, the task management raw dataset specifically includes visual trajectory data, carpet pressure distribution data, carpet surface identification data, and environmental interaction data. The visual trajectory data specifically includes learner movement trajectory data, gesture data, body posture change data, carpet learning area interaction data, and dwell time data on the carpet surface, all collected by the visual acquisition component. The carpet pressure distribution data specifically includes time-series data of the learner's pressure heatmap on the carpet, collected by the sensor component. The carpet surface identification data specifically includes unique identification data of replaceable carpet surfaces, collected by the visual acquisition component. The environmental interaction data specifically includes physical interaction behavior data of the learner with the teaching aids at the edge of the carpet surface, collected by the sensor component.

[0014] Furthermore, in the data preprocessing module, the multi-source data synchronization and alignment is used to solve the problem of data timing misalignment caused by inconsistent acquisition rates of different sensors. Specifically, it obtains a multimodal data sequence with precise time axis alignment through unified timestamp allocation, data stream buffer management, and interpolation synchronization algorithm.

[0015] The abnormal data detection and repair is used to identify and correct noise and outliers in the sensor acquisition process. Specifically, it obtains a complete and reliable sensor data sequence through sliding window statistical detection, Kalman filter smoothing repair, and data continuity verification.

[0016] The multimodal feature normalization is used to unify the dimensions and numerical ranges of data from different sensors. Specifically, it obtains a scale-uniform feature representation through the min-max normalization method.

[0017] The unified spatial coordinate mapping is used to establish a unified reference coordinate system for the carpet surface. Specifically, it involves obtaining multi-source spatial data under a unified coordinate system through coordinate system calibration, coordinate transformation matrix calculation, and spatial consistency verification.

[0018] By performing multi-source data synchronization and alignment, abnormal data detection and repair, multimodal feature normalization, and unified spatial coordinate mapping, the original task management dataset is preprocessed to obtain a preliminary task management dataset.

[0019] Furthermore, in the task management module, a model is constructed to manage learner learning tasks. Specifically, a multimodal deep learning model is constructed as the task management model. The multimodal deep learning model outputs the learning topic and task completion evaluation value through multimodal feature extraction and fusion and spatiotemporal memory mechanism.

[0020] The task management module specifically includes a carpet identification unit, a multimodal feature encoding unit, a multimodal fusion unit, a spatiotemporal memory unit, and a model output unit;

[0021] The carpet identification unit is used to identify the unique identifier of the replaceable carpet to determine the current learning topic, and includes:

[0022] Dual-stream feature extraction is used to extract complementary visual features from carpet marking images. Specifically, it uses two parallel convolutional networks, Local Detail Stream and Global Structure Stream, to obtain local detail features and global structure features, respectively.

[0023] Cross-attention fusion is used to effectively integrate complementary information from two stream features. Specifically, it fuses local detail features and global structural features through a cross-attention mechanism, and performs residual connection with the concatenated features of the local detail features and global structural features after linear mapping to obtain fused enhanced features.

[0024] The topic classification output is used to output the final learning topic classification result. Specifically, it processes and enhances the features through a fully connected layer and a softmax activation function to obtain the probability distribution of each learning topic. The learning topic with the highest probability is selected as the current learning topic, and the learning task sequence of the current learning topic is obtained.

[0025] The multimodal feature encoding unit is used to encode different types of preprocessed data into high-level feature representations. Specifically, it uses 3D convolutional neural networks, graph convolutional networks, and long short-term memory networks to process visual trajectory data, carpet pressure distribution data, and environmental interaction data to obtain visual trajectory features, pressure distribution features, and environmental interaction features.

[0026] The multimodal fusion unit is used to adaptively fuse feature information from different modalities, and includes:

[0027] Modal importance weight calculation is used to dynamically evaluate the importance of each modal feature to the current task. Specifically, it calculates the weight coefficient of each modality through an attention mechanism to obtain the modal importance weight of each modal feature.

[0028] Gated feature fusion is used for controllable feature fusion based on weights. Specifically, it adjusts the degree of fusion of each modality feature through a gating signal to obtain multimodal fused features.

[0029] The spatiotemporal memory unit is used to perform task state reasoning by combining historical memory, and its content includes:

[0030] Short-term memory representation updates are used to maintain the temporary memory of the current task segment. Specifically, the hidden state is updated by using the long short-term memory network based on multimodal fusion features and the short-term memory representation of the previous time step to obtain the short-term memory representation of the current time step.

[0031] Long-term memory retrieval is used to retrieve relevant information from historical experience. Specifically, it obtains a long-term memory context representation related to the current state through similarity calculation and attention-weighted aggregation.

[0032] Memory-enhanced reasoning is used to combine short-term and long-term memory representations for comprehensive reasoning. Specifically, it involves splicing the short-term memory representation of the current time step with the long-term memory context representation related to the current state and performing a linear transformation to obtain memory-enhanced features.

[0033] The model output unit is used to evaluate the learner's completion quality of the learning task on the current learning topic, and output the final learning topic and task completion evaluation value. Specifically, it processes memory enhancement features through a regression layer to obtain the task completion evaluation value and outputs the learning topic and task completion evaluation value.

[0034] By integrating the carpet identification unit, the multimodal feature encoding unit, the multimodal fusion unit, the spatiotemporal memory unit, and the model output unit, a multimodal deep learning model is constructed and trained. The trained multimodal deep learning model serves as a task management model. The preliminary task management dataset is used as the input to the task management model to obtain the current learner's learning topic and task completion evaluation value.

[0035] Furthermore, in the association recommendation module, the following is used to recommend subsequent learning tasks based on the current learner's learning topic and task completion evaluation value: Specifically, an improved graph convolution model is constructed as the association recommendation model. The improved graph convolution model outputs the association recommendation learning task by combining task relevance calculation with task association graph convolution.

[0036] The association recommendation module specifically includes a task relevance calculation unit, a task association graph construction unit, a graph convolution unit, and an association recommendation output unit.

[0037] The task relevance calculation unit is used to quantify the degree of correlation between different learning tasks. Specifically, it obtains task relevance values ​​by modeling temporal relevance and fusing multi-dimensional relevance, and constructs a task relevance matrix.

[0038] The task association graph construction unit is used to represent the relationship between learning tasks as a graph structure. Specifically, it uses learning tasks as nodes, preset metadata of learning tasks based on prior knowledge as node features, and task relevance values ​​as edge weights to construct a fully connected graph and obtain the task association graph.

[0039] The graph convolution unit is used to propagate and update feature information on the task association graph. Specifically, it obtains the graph convolution output features of each node by performing multi-layer graph convolution operations on the task association graph.

[0040] The associated recommendation output unit is used to generate personalized associated recommendation learning tasks for learners. Specifically, it processes the graph convolution output features of each node through a regression layer to obtain the learning task recommendation score of each node, and selects the learning tasks corresponding to the top N nodes with the learning task recommendation scores sorted in descending order as associated recommendation learning tasks.

[0041] By integrating the task relevance calculation unit, the task association graph construction unit, the graph convolution unit, and the association recommendation output unit, an improved graph convolution model is constructed and trained. The trained improved graph convolution model serves as the association recommendation model. The current learner's learning topic and task completion evaluation value are used as inputs to the association recommendation model to obtain the current learner's association recommendation learning task.

[0042] Furthermore, in the interactive feedback module, specifically, feedback is provided to the learner via voice broadcast, based on the current learning topic output by the task management model and the corresponding learning task. Based on the current learner's task completion evaluation value output by the task management model, feedback is provided to the learner on the completion quality of the current learning task. If the task completion evaluation value of the current learning task reaches the preset target, the learner proceeds to the next learning task of the current learning topic. If the task completion evaluation value of the current learning task does not reach the preset target and exceeds the preset task completion time, the current learning task is restarted. If the preset target of the last learning task of the current learning topic is reached, personalized recommended learning tasks are provided to the learner based on the related recommended learning tasks output by the related recommendation model.

[0043] The beneficial effects achieved by the present invention using the above solution are as follows:

[0044] (1) In view of the technical problems of traditional intelligent children's learning systems, which use fixed or single interactive interfaces, which limit the diversity and adaptability of learning topics, easily lead to monotonous and repetitive learning content, and have poor system scalability, making it difficult to integrate new tasks or upgrade technology, this solution creatively adopts replaceable carpet as physical interaction medium. By quickly changing the carpet, the learning topics can be flexibly switched. The learning environment is highly scalable, while enhancing the immersion and tactile experience of learning.

[0045] (2) In view of the technical problems of traditional intelligent children's learning systems, such as model design relying on a single or limited data source, inability to effectively integrate spatiotemporal context information, which easily leads to misjudgment or evaluation bias, lack of advanced feature fusion and memory mechanism, difficulty in dynamically adapting to individual differences, and rigid and static learning task management, this solution creatively adopts a multimodal deep learning model as the task management model. By integrating multiple perceptual data and adaptively combining historical memory information, it can more accurately identify learning topics and evaluate task completion, thereby achieving refined management and personalized adjustment of the learning process.

[0046] (3) In view of the technical problem that traditional intelligent children's learning systems usually recommend learning tasks based on simple rules or static task associations, ignoring the dynamic interaction and complex dependencies between learning tasks, resulting in the recommendation results often being out of touch with the children's actual progress and interests, this solution creatively adopts an improved graph convolution model as the association recommendation model. By dynamically modeling the multidimensional associations between learning tasks, considering the current learning status, and incorporating historical behavioral data, a highly adapted recommendation list is generated to ensure that the recommended tasks are in line with the children's ability level. Attached Figure Description

[0047] Figure 1A schematic diagram of the modules of the intelligent children's learning system based on replaceable carpet provided by the present invention;

[0048] Figure 2 This is a flowchart illustrating the data preprocessing module.

[0049] Figure 3 This is a flowchart illustrating the task management module.

[0050] Figure 4 This is a flowchart illustrating the process of the recommendation module.

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0052] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0053] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0054] Example 1, see Figure 1 The present invention provides an intelligent children's learning system based on replaceable carpet, including a replaceable carpet module, a multimodal perception module, a data preprocessing module, a task management module, an association recommendation module, and an interactive feedback module;

[0055] The replaceable carpet module, based on the carpet base and the replaceable carpet assembly, adopts a quick-release structure to connect with the base layer, allowing for rapid replacement to adapt to different learning tasks.

[0056] The multimodal perception module collects data through the visual acquisition component and sensor component in the carpet base to obtain the task management raw dataset, and sends the task management raw dataset to the data preprocessing module;

[0057] The data preprocessing module employs data preprocessing methods such as multi-source data synchronization and alignment, abnormal data detection and repair, multi-modal feature normalization, and unified spatial coordinate mapping to obtain a preliminary task management dataset, and then sends the preliminary task management dataset to the task management module.

[0058] The task management module is used to build the model required for learner learning task management. By constructing a multimodal deep learning model as the task management model, it obtains the learner's learning topic and task completion evaluation value based on the input data, and sends the task completion evaluation value to the association recommendation module, and sends the learning topic and the task completion evaluation value to the interactive feedback module.

[0059] The associated recommendation module is used to recommend subsequent learning tasks. By constructing an improved graph convolutional model as the associated recommendation model, it obtains the recommendation score of subsequent learning tasks based on the input data, determines the associated recommended learning tasks for learners, and sends the associated recommended learning tasks to the interactive feedback module.

[0060] The interactive feedback module provides interactive feedback to learners through voice broadcasting, based on the learning topic, the task completion evaluation value, and the associated recommended learning tasks.

[0061] Example 2, see Figure 1 This embodiment is based on the above embodiment. In the replaceable carpet module, the carpet base and the replaceable carpet assembly are connected to the base layer using a quick-release structure. The carpet base specifically includes a visual acquisition component and a sensor component for data acquisition. The replaceable carpet assembly specifically includes multiple carpets with different learning themes and replaceable task identification items for different learning tasks contained in each carpet. Each carpet has a unique identification mark. The learning theme can be switched by installing a carpet with the corresponding learning theme on the carpet base, and the learning task can be switched by installing replaceable task identification items for the corresponding learning task on the carpet.

[0062] By performing the above operations, this solution addresses the technical problems of traditional intelligent children's learning systems, which use fixed or single interactive interfaces, limiting the diversity and adaptability of learning topics, easily leading to monotonous and repetitive learning content, and having poor system scalability, making it difficult to integrate new tasks or upgrade technology. This solution creatively adopts replaceable carpets as physical interaction media, allowing for flexible switching of learning topics by quickly changing the carpet surface. The learning environment is highly scalable while enhancing the immersive learning experience and tactile sensation.

[0063] Example 3, see Figure 1This embodiment is based on the above embodiment. In the multimodal perception module, the task management raw dataset specifically includes visual trajectory data, carpet pressure distribution data, carpet surface identification data, and environmental interaction data. The visual trajectory data specifically includes learner movement trajectory data, gesture data, body posture change data, carpet learning area interaction data, and dwell time data on the carpet surface, all collected by the visual acquisition component. The carpet pressure distribution data specifically includes time-series data of the learner's pressure heatmap on the carpet, collected by the sensor component. The carpet surface identification data specifically includes unique identification data of replaceable carpet surfaces, collected by the visual acquisition component. The environmental interaction data specifically includes physical interaction behavior data of the learner with the teaching aids at the edge of the carpet surface, collected by the sensor component.

[0064] Example 4, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In the data preprocessing module, the multi-source data synchronization and alignment is used to solve the problem of data timing misalignment caused by inconsistent acquisition rates of different sensors. Specifically, it obtains a multimodal data sequence with precise time axis alignment through unified timestamp allocation, data stream buffer management and interpolation synchronization algorithm.

[0065] The abnormal data detection and repair is used to identify and correct noise and outliers in the sensor acquisition process. Specifically, it obtains a complete and reliable sensor data sequence through sliding window statistical detection, Kalman filter smoothing repair, and data continuity verification.

[0066] The multimodal feature normalization is used to unify the dimensions and numerical ranges of data from different sensors. Specifically, it obtains a scale-uniform feature representation through the min-max normalization method.

[0067] The unified spatial coordinate mapping is used to establish a unified reference coordinate system for the carpet surface. Specifically, it involves obtaining multi-source spatial data under a unified coordinate system through coordinate system calibration, coordinate transformation matrix calculation, and spatial consistency verification.

[0068] By performing multi-source data synchronization and alignment, abnormal data detection and repair, multimodal feature normalization, and unified spatial coordinate mapping, the original task management dataset is preprocessed to obtain a preliminary task management dataset.

[0069] Example 5, see Figure 1 and Figure 3This embodiment is based on the above embodiment. In the task management module, a model is used to construct the model required for learner learning task management. Specifically, a multimodal deep learning model is constructed as the task management model. The multimodal deep learning model outputs the learning topic and task completion evaluation value through multimodal feature extraction and fusion and spatiotemporal memory mechanism.

[0070] The task management module specifically includes a carpet identification unit, a multimodal feature encoding unit, a multimodal fusion unit, a spatiotemporal memory unit, and a model output unit;

[0071] The carpet identification unit is used to identify the unique identifier of the replaceable carpet to determine the current learning topic, and includes:

[0072] Dual-stream feature extraction is used to extract complementary visual features from carpet marking images. Specifically, it uses two parallel convolutional networks, Local Detail Stream and Global Structure Stream, to obtain local detail features and global structure features, respectively. The formulas used are as follows:

[0073] ;

[0074] In the formula, This represents the input carpet identification data. This represents the output features of the first layer of the local detail flow. This represents the output features of the second layer of the Local Detail Flow. This represents the output features of the third layer of the Local Detail Flow. Indicates local details, Represents the ReLU activation function. This represents the batch normalization function. This represents a convolution function with a stride of 1, a kernel size of 3×3, and 32 output channels. This represents a convolution function with a stride of 1, a kernel size of 3×3, and 64 output channels. This represents a convolution function with a stride of 1, a kernel size of 3×3, and 128 output channels. This represents the average pooling function. This represents the output features of the first layer of the global structured flow. This represents the output features of the second layer of the global structure flow. This represents the output feature of the third layer of the global structure flow. This represents the output features of the fourth layer of the global structured flow. Represents the max pooling function. This represents a convolution function with a stride of 2, a kernel size of 7×7, and 64 output channels. This represents a convolution function with a stride of 2, a kernel size of 3×3, and 128 output channels. This represents a convolution function with a stride of 2, a kernel size of 3×3, and 256 output channels. This represents a dilated convolution function with a dilation rate of 2, a kernel size of 3×3, and 512 output channels. CV represents the global information vector, H represents the feature height, and W represents the feature width. Indicates in features The value at position (a, b), Represents global structural features. This represents the sigmoid activation function. , and Learnable weight matrices representing distinct global structural flows;

[0075] Cross-attention fusion is used to effectively integrate complementary information from two stream features. Specifically, it fuses local detail features and global structural features through a cross-attention mechanism, and performs residual connection with the concatenated features of the local detail features and global structural features after linear mapping to obtain fused enhanced features.

[0076] The topic classification output is used to output the final learning topic classification result. Specifically, it processes and enhances the features through a fully connected layer and a softmax activation function to obtain the probability distribution of each learning topic. The learning topic with the highest probability is selected as the current learning topic, and the learning task sequence of the current learning topic is obtained.

[0077] The multimodal feature encoding unit is used to encode different types of preprocessed data into high-level feature representations. Specifically, it uses 3D convolutional neural networks, graph convolutional networks, and long short-term memory networks to process visual trajectory data, carpet pressure distribution data, and environmental interaction data to obtain visual trajectory features, pressure distribution features, and environmental interaction features.

[0078] The multimodal fusion unit is used to adaptively fuse feature information from different modalities, and includes:

[0079] Modality importance weight calculation is used to dynamically evaluate the importance of each modality feature to the current task. Specifically, it calculates the weight coefficient of each modality through an attention mechanism to obtain the modality importance weight of each feature. The formula used is as follows:

[0080] ;

[0081] In the formula, This represents the modal importance weight of the c-th modal feature. Modal importance can be represented by a learnable weight matrix. Represents the hyperbolic tangent function. Represents the linear mapping weights of the c-th modal feature. The linear mapping bias term represents the feature of the c-th mode. Represents the c-th modal feature. Indicates the first Linear mapping weights of modal features Indicates the first The linear mapping bias term of the modal features, Indicates the first Modal characteristics;

[0082] Gated feature fusion is used for controllable feature fusion based on weights. Specifically, it adjusts the fusion degree of each modality feature through a gating signal to obtain multimodal fused features. The formula used is as follows:

[0083] ;

[0084] In the formula, Gw represents the multimodal fusion gating weight vector. This represents the softmax activation function. Indicates the gating mapping weights, Indicates the gating mapping bias term. Indicates visual trajectory features. Indicates pressure distribution characteristics, Indicates environmental interaction characteristics, Indicates multimodal fusion features, This represents the multimodal fusion gating weight value of the c-th modal feature;

[0085] The spatiotemporal memory unit is used to perform task state reasoning by combining historical memory, and its content includes:

[0086] Short-term memory (STM) representation updates are used to maintain temporary memory for the current task segment. Specifically, the hidden state is updated using a long short-term memory (LSTM) network based on multimodal fusion features and the STM representation from the previous time step to obtain the current STM representation. The formula used is as follows:

[0087] ;

[0088] In the formula, This represents the short-term memory representation of the current time step. This represents the short-term memory of the previous time step. This represents the multimodal fusion features at the current time step. This represents the function that runs the Long Short-Term Memory network;

[0089] Long-term memory retrieval is used to retrieve relevant information from historical experience. Specifically, it obtains a long-term memory context representation related to the current state through similarity calculation and attention-weighted aggregation, using the following formula:

[0090] ;

[0091] In the formula, The long-term memory representation represents the attention weight of the k-th item. This represents the function for calculating cosine similarity. Long-term memory represents the k-th term. Long-term memory represents the first Item, LC represents the long-term memory context representation associated with the current state;

[0092] Memory-enhanced reasoning is used to combine short-term and long-term memory representations for comprehensive reasoning. Specifically, it involves splicing the short-term memory representation of the current time step with the long-term memory context representation related to the current state and performing a linear transformation to obtain memory-enhanced features.

[0093] The model output unit is used to evaluate the learner's completion quality of the learning task on the current learning topic, and output the final learning topic and task completion evaluation value. Specifically, it processes memory enhancement features through a regression layer to obtain the task completion evaluation value and outputs the learning topic and task completion evaluation value.

[0094] By integrating the carpet identification unit, the multimodal feature encoding unit, the multimodal fusion unit, the spatiotemporal memory unit, and the model output unit, a multimodal deep learning model is constructed and trained. The trained multimodal deep learning model serves as a task management model. The preliminary task management dataset is used as the input to the task management model to obtain the current learner's learning topic and task completion evaluation value.

[0095] By performing the above operations, this solution addresses the technical problems of traditional intelligent children's learning systems, which rely on single or limited data sources in model design, cannot effectively integrate spatiotemporal context information, thus easily leading to misjudgments or evaluation biases, and lack advanced feature fusion and memory mechanisms, making it difficult to dynamically adapt to individual differences and resulting in rigid and static learning task management. This solution creatively adopts a multimodal deep learning model as the task management model. By fusing multiple perceptual data and adaptively combining historical memory information, it can more accurately identify learning topics and evaluate task completion, thereby achieving refined management and personalized adjustment of the learning process.

[0096] Example 6, see Figure 1 and Figure 4This embodiment is based on the above embodiment. In the association recommendation module, it is used to recommend subsequent learning tasks based on the current learner's learning topic and task completion evaluation value. Specifically, it constructs an improved graph convolution model as the association recommendation model. The improved graph convolution model outputs the association recommendation learning task by combining task relevance calculation with task association graph convolution.

[0097] The association recommendation module specifically includes a task relevance calculation unit, a task association graph construction unit, a graph convolution unit, and an association recommendation output unit.

[0098] The task relevance calculation unit is used to quantify the degree of correlation between different learning tasks. Specifically, it obtains task relevance values ​​by modeling temporal relevance and fusing multi-dimensional relevance, and constructs a task relevance matrix. The formula used is as follows:

[0099] ;

[0100] In the formula, This represents the temporal correlation value between the m-th learning task of the current learning topic and the n-th learning task of a non-current learning topic. This represents the skill relevance value between the m-th learning task of the current learning topic and the n-th learning task of a non-current learning topic. This represents the difficulty correlation value between the m-th learning task of the current learning topic and the n-th learning task of a non-current learning topic. This represents a dynamic time warping function. This indicates that the m-th learning task of the current learning topic is within the time window. The sequence of task completion evaluation values ​​within the scope. This indicates that the nth learning task, which is not the current learning topic, is within the time window. The sequence of task completion evaluation values ​​within the scope. Represents the timing decay coefficient. Indicates time interval, and These represent the task skill vectors for the m-th learning task of the current learning topic and the n-th learning task of a different topic, respectively. They are preset metadata based on prior knowledge. , and These represent the difficulty assessment values ​​of the m-th learning task in the current learning topic, the n-th learning task outside the current learning topic, and the maximum difficulty assessment value, respectively. They are preset metadata based on prior knowledge. This represents the task relevance value between the m-th learning task of the current learning topic and the n-th learning task of a non-current learning topic, and is an element of the task relevance matrix. Indicates the time-related weights, Indicates skill-related weights, Indicates the weight related to difficulty;

[0101] The task association graph construction unit is used to represent the relationship between learning tasks as a graph structure. Specifically, it uses learning tasks as nodes, preset metadata of learning tasks based on prior knowledge as node features, and task relevance values ​​as edge weights to construct a fully connected graph and obtain the task association graph.

[0102] The graph convolution unit is used to propagate and update feature information on the task association graph. Specifically, it obtains the graph convolution output features of each node by performing multi-layer graph convolution operations on the task association graph.

[0103] The associated recommendation output unit is used to generate personalized associated recommendation learning tasks for learners. Specifically, it processes the graph convolution output features of each node through a regression layer to obtain the learning task recommendation score of each node, and selects the learning tasks corresponding to the top N nodes with the learning task recommendation scores sorted in descending order as associated recommendation learning tasks.

[0104] By integrating the task relevance calculation unit, the task association graph construction unit, the graph convolution unit, and the association recommendation output unit, an improved graph convolution model is constructed and trained. The trained improved graph convolution model serves as the association recommendation model. The current learner's learning topic and task completion evaluation value are used as inputs to the association recommendation model to obtain the current learner's association recommendation learning task.

[0105] By performing the above operations, this solution addresses the technical problem that traditional intelligent children's learning systems often rely on simple rules or static task associations for learning task recommendation, neglecting the dynamic interactions and complex dependencies between learning tasks. This results in recommendation results that are often out of sync with children's actual progress and interests. This solution creatively adopts an improved graph convolutional model as the association recommendation model. By dynamically modeling the multidimensional relationships between learning tasks, considering the current learning state, and incorporating historical behavioral data, it generates a highly adaptive recommendation list, ensuring that the recommended tasks match the child's ability level.

[0106] Example 7, see Figure 1This embodiment, based on the above embodiment, specifically in the interactive feedback module, uses voice broadcasting to provide feedback to the learner on the current learning topic and corresponding learning task, based on the learning topic output by the task management model. It also provides feedback on the completion quality of the current learning task based on the task completion evaluation value output by the task management model. If the task completion evaluation value of the current learning task reaches a preset target, the learner proceeds to the next learning task of the current learning topic. If the task completion evaluation value of the current learning task does not reach the preset target and exceeds the preset task completion time, the current learning task is restarted. If the preset target of the last learning task of the current learning topic is reached, personalized recommended learning tasks are provided to the learner based on the associated recommended learning tasks output by the association recommendation model.

[0107] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0108] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

[0109] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An intelligent child learning system based on a changeable carpet surface carpet, characterized by: The system comprises a replaceable carpet surface module, a multi-modal perception module, a data preprocessing module, a task management module, an association recommendation module and an interaction feedback module. The replaceable carpet surface module is based on a carpet base and a replaceable carpet surface assembly, adopts a quick-release structure to connect with the base layer, and is quickly replaced to adapt to different learning tasks. The multi-modal perception module obtains a task management original data set by data acquisition. The data preprocessing module adopts a data preprocessing method of multi-source data synchronization alignment, abnormal data detection and repair, multi-modal feature normalization and spatial coordinate unified mapping to obtain a task management preliminary data set. The task management module is used to build a model required for learning task management of a learner, specifically by building a multi-modal deep learning model as a task management model. The association recommendation module is used to recommend subsequent learning tasks based on the learning theme and the task completion degree evaluation value of the current learner, specifically by building an improved graph convolution model as an association recommendation model. The interaction feedback module specifically feeds back the current learning theme and the corresponding learning task to the learner based on the learning theme of the current learner output by the task management model through voice broadcasting, feeds back the completion quality of the current learning task to the learner based on the task completion degree evaluation value of the current learner output by the task management model, enters the next learning task of the current learning theme if the task completion degree evaluation value of the current learning task reaches the preset target, re-performs the current learning task if the task completion degree evaluation value of the current learning task does not reach the preset target and exceeds the preset task completion time, and feeds back the personalized recommended learning task to the learner based on the association recommendation learning task of the current learner output by the association recommendation model if the preset target of the last learning task of the current learning theme is reached.

2. The changeable carpet face based smart child learning system of claim 1, wherein: The task management module specifically includes a carpet surface identification recognition unit, a multi-modal feature encoding unit, a multi-modal fusion unit, a space-time memory unit and a model output unit.

3. The changeable carpet face based smart child learning system of claim 2, wherein: The carpet surface identification recognition unit is used to identify the unique identification of the replaceable carpet surface to determine the current learning theme, and includes: Dual-stream feature extraction is used to extract complementary visual features from carpet surface identification images, specifically by using two parallel convolutional networks of local detail stream and global structure stream to obtain local detail features and global structure features, respectively; Cross-attention fusion is used to effectively integrate the complementary information of dual-stream features, specifically by using cross-attention mechanism to fuse local detail features and global structure features, and performing residual connection with the spliced features of linearly mapped local detail features and global structure features to obtain fusion enhanced features. The subject classification output is used to output a final learning subject classification result, specifically, the fusion enhanced features are processed by a full connection layer and a softmax activation function to obtain a probability distribution of each learning subject, a learning subject with the maximum probability is selected as a current learning subject, and a learning task sequence of the current learning subject is obtained; The multi-modal feature encoding unit is configured to encode different types of data after preprocessing into high-level feature representations, specifically, a 3D convolutional neural network, a graph convolution network and a long short-term memory network are respectively used to process visual trajectory data, carpet pressure distribution data and environmental interaction data to obtain visual trajectory features, pressure distribution features and environmental interaction features; The multi-modal fusion unit is configured to adaptively fuse feature information of different modalities, including: The modal importance weight calculation is configured to dynamically evaluate the importance of each modal feature to the current task, specifically, the weight coefficients of each modality are calculated by an attention mechanism to obtain the modal importance weights of each modal feature; The gated feature fusion is configured to perform controllable feature fusion based on the weights, specifically, the fusion degree of each modal feature is adjusted by a gating signal to obtain multi-modal fusion features; The spatio-temporal memory unit is configured to combine historical memory to perform task state reasoning, including: The short-term memory representation update is configured to maintain temporary memory of a current task segment, specifically, the current time step short-term memory representation is updated based on the multi-modal fusion features and the hidden state of the previous time step short-term memory representation by a long short-term memory network; The long-term memory retrieval is configured to retrieve relevant information from historical experience, specifically, the long-term memory context representation related to the current state is obtained by similarity calculation and attention weighted aggregation; The memory enhanced reasoning is configured to combine the long short-term memory representation for comprehensive reasoning, specifically, the memory enhanced features are obtained by concatenating the current time step short-term memory representation and the long-term memory context representation related to the current state and performing linear transformation; The model output unit is configured to evaluate the completion quality of the learning task of the learner on the current learning subject, and output the final learning subject and the task completion degree evaluation value, specifically, the memory enhanced features are processed by a regression layer to obtain the task completion degree evaluation value, and the learning subject and the task completion degree evaluation value are output; By integrating the carpet surface mark recognition unit, the multi-modal feature encoding unit, the multi-modal fusion unit, the spatio-temporal memory unit and the model output unit, a multi-modal deep learning model is constructed and trained, and the trained multi-modal deep learning model is used as a task management model, the task management preliminary data set is used as the input of the task management model, and the learning subject and the task completion degree evaluation value of the current learner are obtained.

4. The changeable skin based carpet smart child learning system of claim 1, wherein: The association recommendation module includes a task correlation calculation unit, a task association graph construction unit, a graph convolution unit and an association recommendation output unit.

5. The changeable carpet face based smart child learning system of claim 4, wherein: The task correlation calculation unit is configured to quantify the correlation degree between different learning tasks, specifically, the task correlation value is obtained by modeling the time series correlation and fusing multi-dimensional correlation, and a task correlation matrix is constructed; The task correlation graph construction unit is configured to represent the learning tasks and the relationships between the learning tasks as a graph structure, specifically, to construct a fully connected graph by taking the learning tasks as nodes, taking preset metadata of the learning tasks based on prior knowledge as node features, and taking task correlation values as edge weights, to obtain a task correlation graph; The graph convolution unit is configured to propagate and update feature information on the task correlation graph, specifically, to obtain graph convolution output features of each node by performing a multi-layer graph convolution operation on the task correlation graph; The correlation recommendation output unit is configured to generate personalized correlation recommendation learning tasks for a learner, specifically, to obtain learning task recommendation degree scores of each node by processing the graph convolution output features of each node through a regression layer, and to select learning tasks corresponding to the top N nodes in descending order of the learning task recommendation degree scores as the correlation recommendation learning tasks. By integrating the task correlation calculation unit, the task correlation graph construction unit, the graph convolution unit, and the correlation recommendation output unit, an improved graph convolution model is constructed and trained. The trained improved graph convolution model serves as a correlation recommendation model. The learning theme and the task completion degree evaluation value of a current learner are input into the correlation recommendation model to obtain correlation recommendation learning tasks for the current learner.

6. The changeable skin based carpet smart child learning system of claim 1, wherein: In the replaceable carpet surface module, the carpet base specifically includes a visual acquisition assembly and a sensor assembly for data acquisition, and the replaceable carpet surface assembly specifically includes a plurality of carpet surfaces of different learning themes and replaceable task identification items included in each carpet surface for performing different learning tasks. Each carpet surface is embedded with a unique identification mark. By installing a carpet surface of a corresponding learning theme on the carpet base to switch the learning theme, and installing a replaceable task identification item of a corresponding learning task on the carpet surface to switch the learning task.

7. The changeable skin based carpet smart child learning system of claim 1, wherein: The task management raw data set specifically includes visual trajectory data, carpet pressure distribution data, carpet surface identification data, and environmental interaction data. The visual trajectory data specifically includes motion trajectory data, gesture action data, body posture change data, carpet surface learning area interaction data, and carpet surface stay duration data of a learner on the carpet surface acquired by the visual acquisition assembly. The carpet pressure distribution data specifically includes pressure and thermal map time series data of a learner on the carpet acquired by the sensor assembly. The carpet surface identification data specifically includes unique identification mark data of the replaceable carpet surface acquired by the visual acquisition assembly. The environmental interaction data specifically includes physical interaction behavior data of a learner and a carpet edge teaching aid acquired by the sensor assembly.

8. The changeable skin based carpet smart child learning system of claim 1, wherein: In the data preprocessing module, the multi-source data synchronization alignment is configured to solve the data time sequence misalignment problem caused by inconsistent acquisition rates of different sensors, specifically, by using a unified timestamp allocation, a data stream buffer management, and an interpolation synchronization algorithm to obtain multi-modal data sequences with accurate time axis alignment. The abnormal data detection and repair is used for identifying and correcting noise and outliers in the sensor collection process, specifically through sliding window statistical detection, Kalman filter smoothing repair and data continuity verification, to obtain complete and reliable sensor data sequences. The multi-modal feature normalization is used for unifying the dimensions and numerical ranges of different sensor data, specifically through the minimum-maximum normalization method, to obtain scale-unified feature representation. The spatial coordinate unified mapping is used for establishing a unified reference coordinate system of the carpet surface, specifically through coordinate system calibration, coordinate transformation matrix calculation and spatial consistency verification, to obtain multi-source spatial data in a unified coordinate system. Through the multi-source data synchronization alignment, the abnormal data detection and repair, the multi-modal feature normalization and the spatial coordinate unified mapping, the task management original data set is preprocessed to obtain a task management preliminary data set.

Images