Children personalized education recommendation method based on multi-modal data analysis

Abstract

The invention discloses a children personalized education recommendation method based on multi-modal data analysis. The method comprises the following steps: step 1, data acquisition; step 2, a cross-modal attention mechanism; step 3, dynamic recommendation; based on multi-modal data fusion and an advanced algorithm model, the limitation that traditional education recommendation depends on subjective judgment is avoided, and a scientific and quantitative basis is provided for children education recommendation. Through a dynamic classification mechanism and a hierarchical recommendation strategy, individual differences and interest tendencies of children can be accurately matched, and waste of educational resources and depression of children's interests are effectively avoided. The characteristics of the children and the family environment factors are integrated, comprehensive consideration of the growth of the children is realized, and the applicability and the effectiveness of the recommendation scheme are improved.

Description

Technical Field

[0001] This invention relates to the field of educational technology, specifically to a personalized educational recommendation method for children based on multimodal data analysis. Background Technology

[0002] The education sector currently faces numerous pressing problems. The education market is rife with irregularities, with parents often blindly following trends and enrolling their children in classes, completely ignoring individual differences. This not only results in a severe waste of educational resources but also significantly suppresses children's interest in learning. Talent assessment also suffers from deficiencies; traditional talent testing largely relies on subjective judgment or uses controversial genetic testing methods, lacking scientifically sound quantitative evidence. Furthermore, the crucial influence of the family environment on child development is severely overlooked; current technologies fail to systematically integrate the roles of family atmosphere, educational methods, and other factors in a child's growth.

[0003] In existing technologies, child psychological assessment (such as CN113100767A) focuses on single behavioral data and cannot comprehensively reflect the child's psychological state; in terms of family environment analysis (such as "Application of Family Digital Environment in Primary School Science Education"), there is a lack of effective quantitative tools, making it difficult to accurately assess the family environment; educational recommendation algorithms (such as CN120087843A) have not yet achieved the integration of data from children and families, and the accuracy and applicability of the recommendation results need to be improved. Summary of the Invention

[0004] The purpose of this invention is to provide a personalized education recommendation method for children based on multimodal data analysis, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for personalized education recommendation for children based on multimodal data analysis, comprising the following steps: Step 1, Data Collection: Obtain body movement data through wearable devices (sampling rate ≥200Hz), micro-expression data through cameras (resolution ≥1080P), psychological trajectory data through sandplay games (including object placement frequency ≥5 times / minute), family interaction data through smart terminals (parent-child interaction frequency ≥1 time / week), and voice emotion data through speech-to-text modules. Step 2, Cross-modal attention mechanism: including input layer, attention calculation, and multi-head fusion; Input layer: mapping multimodal data into feature vectors Q, K, and V, where Q represents behavioral data, K represents psychological data, and V represents family environment data; Attention calculation: using the scaled dot product attention formula:

[0006] in, is a feature dimension; Multi-head fusion: generate a fusion feature vector through 8 parallel attention heads wherein is an output projection matrix; Step 3, dynamic recommendation: according to the fusion feature vector F and the family atmosphere index FAI, a recommendation scheme is generated through a multi-layer perceptron MLP, and the recommendation confidence C is greater than or equal to 0.7.

[0007] Preferably, in the cross-modal attention mechanism, the dimension of the behavior data feature vector Q is 128, the dimension of the psychological data feature vector K is 64, and the dimension of the family environment data feature vector V is 32.

[0008] Preferably, in the multi-head fusion process, the output of each attention head is According to the formula

[0009] is calculated, wherein , , is a linear projection matrix.

[0010] Preferably, the calculation formula of the recommendation confidence C is:

[0011] wherein , is a trainable parameter, is the first dimensional component of the fusion feature vector .

[0012] Preferably, the calculation of the family atmosphere index FAI comprehensively considers the family economic conditions, the education investment proportion, the parent-child communication time length and the interaction quality score, and the weight of each dimension is determined by the analytic hierarchy process.

[0013] Preferably, the dynamic and static classification model adopts a random forest algorithm, and when the child activity level is greater than a set threshold (action frequency is greater than 10 times per minute and focused time length proportion is less than 60%), it is determined as "dynamic", otherwise it is determined as "static".

[0014] Preferably, when the child screen cutting frequency is greater than 5 times per hour or the heart rate variability is greater than 20%, the difficulty of the recommended content is reduced by 1-2 levels, and the interaction link proportion is increased.

[0015] The system architecture of the application is: (1) Data acquisition layer Child end: Collect children's limb movement data through wearable devices, sampling rate ≥ 200Hz; Obtain micro-expression data through high-definition cameras, resolution ≥ 1080P; Equipped with voice-to-text module for collecting and analyzing children's voice emotional data. Family end: Use smart terminals to record family interaction data such as parent-child interaction duration and family activity frequency, requiring parent-child interaction frequency ≥ 1 time / week. Psychological assessment tool: Analyze children's psychological trajectory data through sand table game, drawing task and other modules, with object placement frequency ≥ 5 times / minute. (2) Algorithm analysis layer Preprocessing module: Clean the collected voice text data, remove neutral words and punctuation, and extract emotional intensity labels; Perform noise reduction, standardization and other preprocessing operations on other modal data. Feature fusion module: Use the optimized cross-modal attention mechanism to effectively integrate children's behavior, psychology, family and other multi-modal data to generate a fusion feature vector. Classification model: Based on the random forest algorithm, build a dynamic and static classification model, when the child's activity level exceeds the set threshold, it is determined as "dynamic" type, otherwise it is "static" type. Recommendation engine: Combine collaborative filtering and content recommendation algorithms to generate personalized education recommendation plans based on children's classification results, fusion feature vectors and family atmosphere index. (3) Feedback optimization layer Parental APP: Real-time display of recommendation results, children's behavior data and learning state analysis to parents, making it easy for parents to understand their children. Dynamic adjustment module: Regularly update children's growth records and dynamically adjust algorithm weights and recommendation strategies based on children's learning feedback and state changes.

[0016] Compared with the prior art, the beneficial effects of the present application are: The present application is based on multi-modal data fusion and advanced algorithm model, avoiding the limitations of traditional education recommendation relying on subjective judgment, providing scientific and quantitative basis for children's education recommendation. Through dynamic classification mechanism and hierarchical recommendation strategy, it can accurately match the individual differences and interest tendencies of children, effectively avoiding the waste of education resources and the suppression of children's interests. Integrating children's own characteristics and family environment factors, it realizes the all-round consideration of children's growth, improves the applicability and effectiveness of the recommendation scheme. Through real-time feedback adjustment mechanism, it can optimize the recommendation scheme according to the learning state of children and the changes of family environment, ensuring the sustainability and stability of the recommendation effect. The optimized cross-modal attention mechanism improves the efficiency and accuracy of data fusion, so that the recommendation scheme can be quickly generated and accurately pushed. BRIEF DESCRIPTION OF DRAWINGS

[0017] Fig. 1 is a system architecture diagram of the present application; Fig. 2 is a cross-modal attention mechanism fusion flowchart of the present application; Fig. 3 is a dynamic recommendation engine working principle schematic diagram of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] Please refer to Figs. 1-3 The present application provides a personalized education recommendation method for children based on multi-modal data analysis, comprising the following steps: Step 1, data acquisition: acquiring limb motion data (sampling rate ≥ 200 Hz) through wearable devices, acquiring micro-expression data (resolution ≥ 1080P) through cameras, acquiring psychological trajectory data (containing object placement frequency ≥ 5 times / minute) through sand table games, acquiring family interaction data (parent-child interaction frequency ≥ 1 time / week) through intelligent terminals, and acquiring voice emotion data through a voice-to-text module; Step 2, cross-modal attention mechanism: including input layer, attention calculation and multi-head fusion; input layer: mapping multi-modal data into feature vectors Q, K and V, wherein Q is behavior data, K is psychological data, and V is family environment data; attention calculation: using a scaled dot-product attention formula:

[0020] wherein, is the feature dimension; Multi-head fusion: generating a fusion feature vector F through 8 parallel attention heads wherein, is the output projection matrix; Step 3, dynamic recommendation: generating a recommendation scheme through a multi-layer perceptron (MLP) according to the fusion feature vector F and the family atmosphere index (FAI), with a recommendation confidence C ≥ 0.7.

[0021] In the cross-modal attention mechanism, the dimension of the behavior data feature vector Q is 128, the dimension of the psychological data feature vector K is 64, and the dimension of the family environment data feature vector V is 32.

[0022] In the multi-head fusion process, the output of each attention head is According to the formula

[0023] Compute, wherein, , , is a linear projection matrix.

[0024] The calculation formula of the recommendation confidence C is:

[0025] wherein, , is a trainable parameter, is the first dimensional component of the fusion feature vector .

[0026] The calculation of the family atmosphere index FAI comprehensively considers the family economic conditions, the education investment proportion, the parent-child communication time length and the interaction quality score, and the weight of each dimension is determined by the analytic hierarchy process.

[0027] The dynamic and static classification model adopts a random forest algorithm, and when the child activity level is greater than a set threshold (motion frequency is greater than 10 times / minute and focused time length proportion is less than 60%), it is determined as "dynamic", otherwise it is determined as "static".

[0028] When dynamically adjusting the difficulty of the recommended content, when the child screen cutting frequency is greater than 5 times / hour or the heart rate variability is greater than 20%, the recommended content difficulty is reduced by 1-2 levels, and the interaction link proportion is increased.

[0029] The system architecture of the application is: (1) Data acquisition layer Child end: collect the child's limb motion data by means of wearable devices, the sampling rate is greater than or equal to 200Hz; obtain micro-expression data through a high-definition camera, the resolution is greater than or equal to 1080P; equipped with a voice-to-text module for collecting and analyzing the child's voice emotion data. Family end: use a smart terminal to record family interaction data such as parent-child interaction time and family activity frequency, and the parent-child interaction frequency is required to be greater than or equal to 1 time / week. Psychological assessment tool: analyze the child's psychological trajectory data through sand table game, drawing task and other modules, and the object placement frequency is greater than or equal to 5 times / minute. (2) Algorithm analysis layer Preprocessing module: clean the collected voice text data, remove neutral words and punctuation marks, and extract emotion intensity labels; perform noise reduction, standardization and other preprocessing operations on other modal data. Feature fusion module: use the optimized cross-modal attention mechanism to effectively integrate the child's behavior, psychology, family and other multi-modal data to generate a fusion feature vector. Classification Model: Based on the Random Forest algorithm, a dynamic-static classification model is constructed. When the child's activity level exceeds the set threshold, it is determined as "dynamic" type, otherwise as "static" type. Recommendation Engine: Combining collaborative filtering and content recommendation algorithm, personalized education recommendation plan is generated according to the child's classification result, fusion feature vector and family atmosphere index. (3) Feedback Optimization Layer Parental APP: Real-time display of recommendation results, child behavior data and learning status analysis to parents, making it easy for parents to understand their children. Dynamic Adjustment Module: Regularly update the child's growth profile, dynamically adjust algorithm weights and recommendation strategies according to the child's learning feedback and state changes.

[0030] Example 1: Interest Recommendation for "Dynamic" Children Data Collection: 5-year-old children frequently move objects in sand table games (action frequency 12 times / minute), micro-expression recognition shows that the concentration duration accounts for 70%, the family records the frequency of parent-child interaction as 2 times / week, and voice emotion analysis shows that positive vocabulary accounts for 65%.

[0031] Algorithm Analysis: The dynamic-static classification model determines it as a "dynamic" child, and the family atmosphere index FAI=0.65 (medium interaction level). Through the cross-modal attention mechanism to fuse multi-modal feature vectors, the recommendation engine generates a plan: programming (creativity training) + calligraphy (patience training), with weights of 0.75 and 0.25 respectively, and a recommendation confidence C=0.82.

[0032] Result Output: Parental APP displays recommended courses and child behavior data visualization charts. After 1 month of retesting, the child's screen cutting frequency is reduced to 3 times / hour, and the concentration duration accounts for 75%.

[0033] Example 2: Family Environment Optimization Recommendation Data Collection: The family records the length of parent-child reading together <1 hour / week, the child's painting task appears "loneliness" theme multiple times, voice emotion analysis shows that negative vocabulary accounts for 30%, and body movement frequency is low (5 times / minute).

[0034] Algorithm Analysis: Family atmosphere index (FAI)=0.35 (low interaction level), dynamic-static classification model determines it as a "static" child. The recommendation engine generates a plan: increase family collaboration activities (such as parent-child handicraft, shared reading plan), with a weight of 0.9, and recommends a picture book reading course, with a recommendation confidence C=0.78.

[0035] Result Output: After the parents implement the recommended plan, after 2 months of retesting, the frequency of parent-child interaction increases to 3 times / week, the child's painting theme changes to "cooperation", and the proportion of negative voice vocabulary decreases to 15%.

[0036] The present application is based on multi-modal data fusion and advanced algorithm model, avoids the limitation of traditional education recommendation relying on subjective judgment, and provides scientific and quantitative basis for children education recommendation. Through dynamic classification mechanism and hierarchical recommendation strategy, the individual differences and interest tendency of children can be accurately matched, and the waste of education resources and the suppression of children's interest can be effectively avoided. The children's own characteristics and family environment factors are integrated, the all-round consideration of children's growth is realized, and the applicability and effectiveness of the recommendation scheme are improved. Through real-time feedback adjustment mechanism, the recommendation scheme can be optimized in time according to the learning state of children and the change of family environment, so as to ensure the continuity and stability of the recommendation effect. The optimized cross-modal attention mechanism improves the efficiency and accuracy of data fusion, so that the recommendation scheme can be quickly generated and accurately pushed.

[0037] Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions recorded in the foregoing embodiments or make equivalent replacements for part of the technical features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A personalized education recommendation method for children based on multimodal data analysis, characterized in that: Includes the following steps: Step 1, Data Collection: Obtain body movement data through wearable devices, micro-expression data through cameras, psychological trajectory data through sandplay games, family interaction data through smart terminals, and voice emotion data through voice-to-text modules; Step 2, Cross-modal attention mechanism: including input layer, attention computation, and multi-head fusion; Input layer: Maps multimodal data into feature vectors Q, K, and V, where Q represents behavioral data, K represents psychological data, and V represents family environment data; Attention calculation: Employs the scaled dot product attention formula: in, For feature dimensions; Multi-head fusion: Generating fused feature vectors through 8 parallel attention heads. ,in To output the projection matrix; Step 3, Dynamic Recommendation: Based on the fused feature vector F and the family atmosphere index FAI, a recommendation scheme is generated through a multilayer perceptron (MLP), with a recommendation confidence level C ≥ 0.

7.

2. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: In the aforementioned cross-modal attention mechanism, the behavioral data feature vector Q has a dimension of 128, the psychological data feature vector K has a dimension of 64, and the family environment data feature vector V has a dimension of 32.

3. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: In the multi-head fusion process, the output of each attention head According to the formula Calculate, where, , , It is a linear projection matrix.

4. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: The formula for calculating the recommendation confidence level C is as follows: in, , For trainable parameters, To fuse feature vectors The Dimensional components.

5. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: The Family Atmosphere Index (FAI) is calculated by comprehensively considering family economic conditions, the proportion of educational investment, the duration of parent-child communication, and the quality of interaction. The weights of each dimension are determined by the Analytic Hierarchy Process (AHP).

6. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: The dynamic / static classification model uses the random forest algorithm. When a child's activity level is greater than a set threshold, they are classified as "dynamic"; otherwise, they are classified as "static".

7. The personalized education recommendation method for children based on multimodal data analysis according to claim 1, characterized in that: When dynamically adjusting the difficulty of recommended content, if a child's screen-switching frequency is detected to be >5 times / hour or heart rate variability is detected to be >20%, the difficulty of the recommended content will be reduced by 1-2 levels, while the proportion of interactive elements will be increased.

Images