A method, apparatus, equipment, medium, and program product for personalized learning analysis.

By acquiring multimodal learning data and performing cross-modal fusion and multi-dimensional diagnosis, personalized learning suggestions are generated, which solves the problems of single dimension and insufficient dynamism in existing learning analysis methods, and realizes refined and hierarchical learning diagnosis and personalized suggestions.

CN123115371APending Publication Date: 2026-11-17BEIJING & AIWEIWU TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610386084.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-03-26
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing AI-based learning analysis methods provide results with limited dimensions and insufficient dynamism, failing to meet the needs of personalized path planning and resource matching.

Method used

By acquiring multimodal learning data (text, speech, vision, and behavioral data), preprocessing, labeling, temporal alignment, and feature extraction are performed. A unified learning fusion feature vector is generated using a cross-modal attention fusion mechanism. Combined with a multi-dimensional diagnostic model, a diagnosis is made, and personalized learning suggestions are output.

Benefits of technology

It enables a comprehensive perception and structured representation of students' learning process, improving the accuracy and comprehensiveness of learning diagnosis and providing a scientific basis for teachers' precise teaching, students' adaptive learning, and parents' collaborative education.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115371A_ABST
    Figure CN123115371A_ABST
Patent Text Reader

Abstract

This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and program product for personalized learning analysis. The method includes: acquiring multimodal learning data of a target object, including text data, voice data, visual data, and behavioral data; processing the multimodal learning data to obtain feature vectors corresponding to each modality; dynamically weighting and fusing the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector; based on the learning fusion feature vector, performing preset multi-dimensional diagnoses using a preset multi-dimensional diagnostic model, and outputting multi-dimensional diagnostic results; and generating an analysis report for the target object based on the multi-dimensional diagnostic results, including personalized learning suggestions. This method enables multi-dimensional diagnostic analysis, providing more accurate decision-making basis for personalized path planning and resource matching, and improving scenario adaptability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and program product for personalized learning analysis. Background Technology

[0002] Learning analysis, also known as learning progress analysis, refers to the systematic collection, processing, and interpretation of various data related to the learning process to comprehensively assess learners' knowledge acquisition, ability development, learning behavior characteristics, and emotional state. Traditional learning analysis mainly relies on subjective or semi-subjective methods such as teacher experience, classroom observation, standardized tests, and questionnaires, which have limitations such as incomplete data collection, delayed analysis, and highly subjective conclusions. With the accelerated pace of educational informatization, the scale and complexity of data generated during the learning process have significantly increased. How to conduct efficient, accurate, and personalized learning analysis has become a key issue in improving educational quality and implementing the concept of individualized instruction.

[0003] In recent years, artificial intelligence (AI) technologies, particularly machine learning, deep learning, natural language processing, and knowledge graphs, have made groundbreaking progress, providing powerful tools for processing high-dimensional, nonlinear, and multimodal data. AI technology has been widely applied in computer vision, speech recognition, recommender systems, and decision support, demonstrating its remarkable ability to automatically extract features from massive amounts of data, discover complex patterns, and predict future trends. Driven by increased computing power, algorithm optimization, and data accumulation, AI is continuously evolving from specialized intelligence to general-purpose intelligence, and its deep integration with various industries has become a significant trend in technological development.

[0004] In related technologies, the digital infrastructure for K-12 basic education has been initially improved, with hardware and software such as smart pens, learning machines, smart classroom cameras, and online question-answering platforms becoming widely available, providing hardware support for multimodal data collection. At the same time, the accuracy of AI technology in fields such as computer vision and speech recognition has reached the threshold for application in educational scenarios (e.g., facial expression recognition accuracy > 90%, speech emotion recognition accuracy > 85%), making it feasible to apply AI technology to educational scenarios for learning analysis.

[0005] However, current learning analysis methods have the following technical problems:

[0006] Existing AI-based learning analysis methods offer limited analytical results and lack dynamism, thus requiring optimization. Summary of the Invention

[0007] Based on this, it is necessary to provide a personalized learning analysis method, device, computer equipment, computer-readable storage medium, and computer program product that can perform multi-dimensional diagnostic analysis, provide more accurate decision-making basis for personalized path planning and resource matching, and adapt to multiple scenarios, in order to address the above-mentioned technical problems.

[0008] Firstly, this application provides a method for implementing personalized learning analysis. The method includes:

[0009] Acquire multimodal learning data of the target object, wherein the multimodal learning data includes text data, voice data, visual data, and behavioral data;

[0010] The multimodal learning data is processed to obtain feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction.

[0011] The feature vectors are dynamically weighted and fused using a pre-defined cross-modal attention fusion mechanism to generate a unified learning fusion feature vector.

[0012] Based on the learning fusion feature vector, preset multi-dimensional diagnoses are performed using a preset multi-dimensional diagnostic model, and multi-dimensional diagnostic results are output.

[0013] Based on the multi-dimensional diagnostic results, an analysis report is generated for the target object, and the analysis report includes personalized learning suggestions.

[0014] In one embodiment, the processing of the multimodal learning data to obtain feature vectors corresponding to each modality includes preprocessing, annotation, temporal alignment, and feature extraction, including:

[0015] The voice data is subjected to noise reduction and silence removal processing;

[0016] The visual data is then processed for face detection and expression classification.

[0017] The behavioral data is subjected to outlier detection and data cleaning.

[0018] In one embodiment, the step of dynamically weighting and fusing the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector includes:

[0019] Based on a preset attention configuration mechanism, the attention weight parameters of the feature vectors corresponding to each modality are calculated respectively;

[0020] The feature vector is weighted and fused based on the attention weight parameters to obtain the learning situation fusion feature vector.

[0021] In one embodiment, the preset diagnostic dimensions include knowledge mastery, core competencies, learning status, and learning emotions. Based on the learning situation fusion feature vector, the preset multi-dimensional diagnostic model is used to perform preset multi-dimensional diagnoses, and the output multi-dimensional diagnostic results include:

[0022] Extract text features and behavioral features, classify knowledge mastery levels based on the text features and behavioral features, and verify and correct the classification results based on the visual features and speech features;

[0023] And / or assess logical reasoning ability based on the text features and behavioral features, assess language expression ability based on the speech features and text features, and assess practical learning ability based on the visual features and behavioral features;

[0024] And / or perform attention classification based on the visual features, assess fatigue based on the visual features and the behavioral features, and analyze course participation based on the voice features and the behavioral features;

[0025] And / or perform emotion category classification based on the speech features, perform facial expression emotion recognition based on the visual features, and construct an emotion-behavior mapping relationship by combining the behavioral features.

[0026] In one embodiment, the method further includes:

[0027] Based on the multi-dimensional diagnostic results, the focus index of the target object is determined;

[0028] In response to the target object's focus index falling below a preset threshold, a real-time alert is triggered and an intervention plan matching the target object is generated.

[0029] In one embodiment, the method further includes:

[0030] Real-time data acquisition and lightweight processing are performed using pre-deployed edge devices;

[0031] Multimodal fusion and model inference are performed based on pre-deployed cloud servers.

[0032] Secondly, this application also provides a device for implementing personalized learning analysis. The device includes:

[0033] The data acquisition module is used to acquire multimodal learning data of the target object, including text data, voice data, visual data, and behavioral data.

[0034] The data processing module is used to process the multimodal learning data to obtain feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction.

[0035] The feature fusion module is used to dynamically weight and fuse the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector.

[0036] The multidimensional analysis module is used to perform preset multidimensional diagnoses based on the learning situation fusion feature vector and through a preset multidimensional diagnostic model, and output multidimensional diagnostic results.

[0037] The report generation module is used to generate an analysis report for the target object based on the multi-dimensional diagnostic results, and the analysis report includes personalized learning suggestions.

[0038] In one embodiment, the data processing module includes:

[0039] The voice data processing module is used to perform noise reduction and silence removal processing on the voice data;

[0040] A visual data processing module is used to perform face detection and expression classification processing on the visual data;

[0041] The behavior data processing module is used to perform outlier detection and data cleaning on the behavior data.

[0042] In one embodiment, the feature fusion module includes:

[0043] The attention configuration module is used to calculate the attention weight parameters of the feature vectors corresponding to each modality based on a preset attention configuration mechanism.

[0044] The weighted fusion module is used to perform weighted fusion of the feature vector based on the attention weight parameters to obtain the learning situation fusion feature vector.

[0045] In one embodiment, the preset diagnostic dimensions include knowledge acquisition, core competencies, learning status, and learning attitude, and the multidimensional analysis module includes:

[0046] The knowledge mastery analysis module is used to extract text features and behavioral features, classify knowledge mastery levels based on the text features and behavioral features, and verify and correct the classification results based on the visual features and speech features.

[0047] And / or a core competency analysis module, used to evaluate logical reasoning ability based on the text features and the behavioral features, to evaluate language expression ability based on the voice features and the text features, and to evaluate practical learning ability based on the visual features and the behavioral features;

[0048] And / or a learning status analysis module, used to classify attention based on the visual features, assess fatigue based on the visual features and the behavioral features, and analyze course participation based on the voice features and the behavioral features;

[0049] And / or a learning sentiment analysis module, used to classify sentiment categories based on the speech features, recognize facial expressions based on the visual features, and construct an emotion-behavior mapping relationship by combining the behavioral features.

[0050] In one embodiment, the device further includes:

[0051] The focus module is used to determine the focus index of the target object based on the multi-dimensional diagnostic results.

[0052] The focus intervention module is used to trigger a real-time warning and generate an intervention plan that matches the target object when the focus index of the target object is lower than a preset threshold.

[0053] In one embodiment, the device further includes:

[0054] The edge device module is used for real-time data acquisition and lightweight processing based on pre-deployed edge devices;

[0055] The cloud server module is used for multimodal fusion and model inference based on a pre-deployed cloud server.

[0056] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a personalized learning analysis method as described in any embodiment of the first aspect.

[0057] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of a personalized learning analysis method as described in any embodiment of the first aspect.

[0058] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of a personalized learning analysis method as described in any embodiment of the first aspect.

[0059] The aforementioned personalized learning analysis method, apparatus, computer equipment, storage medium, and computer program product, derived from the technical features in the embodiments, can achieve the following beneficial effects to address the technical problems in the background art:

[0060] This application provides a method for personalized learning analysis, comprising: acquiring multimodal learning data of a target object, the multimodal learning data including text data, speech data, visual data, and behavioral data; processing the multimodal learning data to obtain feature vectors corresponding to each modality, the processing including preprocessing, annotation, temporal alignment, and feature extraction; dynamically weighting and fusing the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector; based on the learning fusion feature vector, performing preset multi-dimensional diagnoses through a preset multi-dimensional diagnostic model, and outputting multi-dimensional diagnostic results; and generating an analysis report for the target object based on the multi-dimensional diagnostic results, the analysis report including personalized learning suggestions. In implementation, by acquiring four types of multimodal learning data (text, speech, vision, and behavior), a full-dimensional perception and structured representation of the student's learning process is achieved. Through preprocessing, annotation, temporal alignment, and feature extraction, heterogeneous data is effectively fused, overcoming the problems of single modality and fragmented information in traditional learning analysis. Further utilizing a cross-modal attention fusion mechanism, dynamic weighted fusion of feature vectors from various modalities is performed to generate a unified learning fusion feature vector, enhancing the complementarity and robustness of feature representation. Based on this, a pre-defined multi-dimensional diagnostic model is used to comprehensively diagnose knowledge mastery, core competencies, learning status, and emotional attitudes, outputting refined and hierarchical diagnostic results. This effectively addresses the pain points of traditional methods, such as their single diagnostic dimension and neglect of non-intellectual factors. Finally, an analysis report containing personalized learning suggestions is generated based on the multi-dimensional diagnostic results. This not only improves the accuracy and comprehensiveness of learning diagnosis but also provides a scientific basis for teachers' precise teaching, students' adaptive learning, and parents' collaborative education, demonstrating strong practicality, interpretability, and educational adaptability. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is an application environment diagram of a personalized learning analysis method in one embodiment;

[0063] Figure 2 This is a schematic diagram of the first process of a personalized learning analysis method in one embodiment;

[0064] Figure 3 This is a schematic diagram of the second process of a personalized learning analysis implementation method in another embodiment;

[0065] Figure 4 This is a schematic diagram of the third process of a personalized learning analysis implementation method in another embodiment;

[0066] Figure 5 This is a schematic diagram of the fourth process of a personalized learning analysis implementation method in another embodiment;

[0067] Figure 6 This is a schematic diagram of the fifth process of a personalized learning analysis implementation method in another embodiment;

[0068] Figure 7 This is a schematic diagram of the sixth process of a personalized learning analysis implementation method in another embodiment;

[0069] Figure 8 This is a schematic diagram of the data processing architecture for a personalized learning analysis method in a specific embodiment.

[0070] Figure 9 This is a flowchart illustrating a personalized learning analysis method in a specific embodiment.

[0071] Figure 10 This is a structural block diagram of a personalized learning analysis implementation device in one embodiment;

[0072] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] The personalized learning analysis method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0075] In one embodiment, such as Figure 2 As shown, a personalized learning analysis method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0076] Step 202: Obtain multimodal learning data of the target object, including text data, voice data, visual data, and behavioral data.

[0077] For example, text data may include homework answers, test paper writing, class notes, explanations of incorrect answers, learning logs, and the logical expression of essays / discussion questions. Voice data may include spoken expression, classroom speaking voice, reading fluency, questioning tone, group discussion voice, and emotional voice characteristics (such as fatigue or irritability). Visual data may include classroom attention expressions (such as focus, distraction, confusion, or boredom), body language (such as posture and frequency of raising hands), eye movements, writing posture, and experimental operation actions. Behavioral data may include answering time, speed of correcting incorrect answers, video viewing progress, number of times key content is replayed, frequency of note marking, accuracy rate of interactive quizzes, and resource browsing preferences.

[0078] Step 204: Process the multimodal learning data to obtain the feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction.

[0079] Step 206: The feature vectors are dynamically weighted and fused using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector.

[0080] Step 208: Based on the learning situation fusion feature vector, perform preset multi-dimensional diagnoses through the preset multi-dimensional diagnostic model and output multi-dimensional diagnostic results.

[0081] Step 2010: Based on the multi-dimensional diagnostic results, generate an analysis report for the target object, the analysis report including personalized learning suggestions.

[0082] In the above-mentioned personalized learning analysis method, by combining the technical features in the embodiments and making reasonable derivations, the beneficial effect of solving the technical problems raised in the background art is achieved:

[0083] This application provides a method for personalized learning analysis, comprising: acquiring multimodal learning data of a target object, the multimodal learning data including text data, speech data, visual data, and behavioral data; processing the multimodal learning data to obtain feature vectors corresponding to each modality, the processing including preprocessing, annotation, temporal alignment, and feature extraction; dynamically weighting and fusing the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector; based on the learning fusion feature vector, performing preset multi-dimensional diagnoses through a preset multi-dimensional diagnostic model, and outputting multi-dimensional diagnostic results; and generating an analysis report for the target object based on the multi-dimensional diagnostic results, the analysis report including personalized learning suggestions. In implementation, by acquiring four types of multimodal learning data (text, speech, vision, and behavior), a full-dimensional perception and structured representation of the student's learning process is achieved. Through preprocessing, annotation, temporal alignment, and feature extraction, heterogeneous data is effectively fused, overcoming the problems of single modality and fragmented information in traditional learning analysis. Further utilizing a cross-modal attention fusion mechanism, dynamic weighted fusion of feature vectors from various modalities is performed to generate a unified learning fusion feature vector, enhancing the complementarity and robustness of feature representation. Based on this, a pre-defined multi-dimensional diagnostic model is used to comprehensively diagnose knowledge mastery, core competencies, learning status, and emotional attitudes, outputting refined and hierarchical diagnostic results. This effectively addresses the pain points of traditional methods, such as their single diagnostic dimension and neglect of non-intellectual factors. Finally, an analysis report containing personalized learning suggestions is generated based on the multi-dimensional diagnostic results. This not only improves the accuracy and comprehensiveness of learning diagnosis but also provides a scientific basis for teachers' precise teaching, students' adaptive learning, and parents' collaborative education, demonstrating strong practicality, interpretability, and educational adaptability.

[0084] In one embodiment, such as Figure 3 As shown, step 202 includes:

[0085] Step 302: Perform noise reduction and silence removal processing on the voice data.

[0086] Step 304: Perform face detection and expression classification processing on the visual data.

[0087] Step 306: Perform outlier detection and data cleaning on the behavioral data.

[0088] For example, data cleaning can be tailored to different modal characteristics by formulating cleaning rules. Specifically, text modality can employ spell correction and semantic deduplication algorithms. Speech modality can be processed through noise reduction and silence removal to filter interference. The core of this noise suppression is achieved using the spectral subtraction formula, as shown in the following equation:

[0089]

[0090] in This is the speech spectrum after noise reduction. This is the spectrum of noisy speech. This is the noise suppression coefficient (values ​​range from 0.8 to 0.95). This is the estimated noise spectrum.

[0091] For example, the visual modality can use a face detection algorithm to remove non-student areas, while the behavioral modality can remove invalid data through outlier detection (such as extreme answering times) using the 3σ criterion: when a data point deviates from the mean by more than 3 times the standard deviation, it is judged as an outlier and removed.

[0092] For example, an education-specific data annotation system can be pre-built in the server, adopting an "AI pre-annotation + manual verification" model—text modality annotation includes "knowledge point mastery tags" and "logical error types"; speech modality annotation includes "pronunciation accuracy" and "emotional tags (calm / positive / irritable)"; visual modality annotation includes "facial expression tags" and "attention level (1-5)"; and behavioral modality annotation includes "learning behavior type" and "efficiency level". Annotation consistency verification uses the Kappa coefficient calculation, as shown in the following formula:

[0093]

[0094] in, This represents the actual consistency rate. To achieve the desired consistency rate, a Kappa value ≥ 0.8 is required to pass the validation.

[0095] For example, in the data alignment process, the server can use "timestamp + learning task ID" as a dual primary key to achieve spatiotemporal alignment of multimodal data. For instance, it can associate "frowning facial expressions (visual modality), whispered questions (vocal modality), and incorrect problem-solving steps (text modality)" when answering math problems (behavioral modality) with the learning task of "solving a quadratic equation." Timestamp synchronization uses linear interpolation to correct for time discrepancies between different devices, ensuring an alignment error ≤100ms.

[0096] For example, in terms of data fusion, a "modal feature extraction-cross-modal attention fusion" architecture can be adopted. The core algorithm and parameters are as follows: ① Feature extraction: For the visual modality, ResNet-50 (input size 224×224, batch size 32, learning rate 1e-4) is used to extract 2048-dimensional features; for the speech modality, Wav2Vec2-base (sampling rate 16kHz, Mel-spectral dimension 80) is used to extract 768-dimensional features; for the text modality, BERT-base-chinese (maximum sequence length 128, batch size 16) is used to extract 768-dimensional features; for the behavioral modality, MLP (3 fully connected layers, hidden layer dimension 512→256, activation function ReLU) is used to extract 256-dimensional features. ② Fusion strategy: A cross-modal attention mechanism is adopted. The core formula for calculating attention weights is as follows:

[0097]

[0098] in, To query the matrix (a specific modality feature). The key matrix (all modal features) is the key matrix. Value matrix (all modal features) The query matrix dimension is defined. Weights are dynamically assigned using an attention weight matrix (dimension = sum of the dimensions of each modality feature). In the knowledge diagnosis scenario, the initial weights are set to 0.4 for text, 0.3 for behavior, 0.15 for vision, and 0.15 for speech. The weights are iteratively optimized based on training with 100,000+ labeled samples per semester, and finally a 512-dimensional unified learning information feature vector is output.

[0099] Specifically, data cleaning involves customizing cleaning rules for different modal characteristics—text modality uses spell correction and semantic deduplication algorithms; speech modality uses noise reduction and silence removal to filter interference; visual modality uses face detection algorithms to remove non-student areas; and behavioral modality uses outlier detection (such as extreme answering times) to remove invalid data. Data annotation involves constructing an education-specific annotation system using an "AI pre-annotation + manual verification" model—text modality is annotated with "knowledge point mastery tags" and "logical error types"; speech modality is annotated with "pronunciation accuracy" and "emotional tags (calm / positive / irritable)"; visual modality is annotated with "facial expression tags" and "focus level (1-5)"; and behavioral modality is annotated with "learning behavior type" and "efficiency level". Data alignment uses "timestamp + learning task ID" as dual primary keys to achieve spatiotemporal alignment of multimodal data. For example, "frowning expression (visual modality), whispered questioning (speech modality), and incorrect problem-solving steps (text modality) when answering math problems (behavioral modality)" are associated with the learning task of "solving a quadratic equation". Data fusion: The architecture of "modal feature extraction - cross-modal attention fusion" is adopted. The core algorithms and parameters are as follows: ① Feature extraction: For visual modality, ResNet-50 (input size 224×224, batch size 32, learning rate 1e-4) is used to extract 2048-dimensional features; for speech modality, Wav2Vec2-base (sampling rate 16kHz, Mel spectrum dimension 80) is used to extract 768-dimensional features; for text modality, BERT-base-chinese (maximum sequence length 128, batch size 16) is used to extract 768-dimensional features; for behavioral modality, MLP (3 fully connected layers, hidden layer dimension 512→256, activation function ReLU) is used to extract 256-dimensional features. ② Fusion Strategy: A cross-modal attention mechanism is adopted, which dynamically allocates weights through an attention weight matrix (dimension = sum of the dimensions of each modality feature). In the knowledge diagnosis scenario, the initial weights are set to 0.4 for text, 0.3 for behavior, 0.15 for vision, and 0.15 for speech. The weights are iteratively optimized based on training with 100,000+ labeled samples per semester, and finally outputs a 512-dimensional unified learning feature vector.

[0100] In this embodiment, targeted preprocessing of speech, visual, and behavioral data effectively eliminates environmental noise, irrelevant areas, and abnormal behavioral interference, improving the quality and consistency of data across modalities. This lays a clean and reliable data foundation for subsequent feature extraction and cross-modal fusion, enhancing the accuracy and stability of learning diagnosis.

[0101] In one embodiment, such as Figure 4 As shown, step 206 includes:

[0102] Step 402: Based on the preset attention configuration mechanism, calculate the attention weight parameters of the feature vectors corresponding to each modality.

[0103] For example, the model can be built primarily with text modalities and secondarily with behavioral modalities. The key algorithm stack is as follows: ① Text processing: A BERT-base-chinese pre-trained model (batch size 32, learning rate 2e-5, 10 iterations during fine-tuning) is used to semantically encode the answer content and incorrect question analysis, outputting 768-dimensional text features; keyword features are extracted using TextCNN (convolution kernel size 3 / 4 / 5, number 128, pooling method max pooling), and concatenated with BERT features to form a 1024-dimensional comprehensive feature. ② Knowledge point classification: An XGBoost classifier (objective function multi:softprob, tree depth 6, learning rate 0.1, estimators=200, regularization parameters L1=0.1, L2=0.2) is used to map features to knowledge point mastery levels. ③ Behavior Modification: Introduce the LightGBM regression model (objective function reg: squarederror, tree depth 4, learning rate 0.05), with the answering time, number of corrections, and review frequency as inputs, and output "mastery confidence correction coefficient" (range 0.8-1.2) to weight and modify the XGBoost output results. For example, when "answering time is too long but correct", the correction coefficient is set to 0.9, and "mastery" is downgraded to "mastery but not proficiency".

[0104] Step 404: The feature vector is weighted and fused based on the attention weight parameters to obtain the learning situation fusion feature vector.

[0105] For example, in the optimization of the fusion layer, visual and speech modal verification can be introduced. The core algorithm is a multimodal decision fusion model: ① Visual verification: MobileNetV2 (lightweight adaptation for edge devices, input size 160×160, batch size 64) is used to classify facial expression images and output the probabilities of 5 states such as "confused / focused / relaxed"; combined with Dlib keypoint detection (68 facial keypoints) to calculate eye opening degree (threshold < 0.2 is judged as distraction) and frown coefficient (the change in distance between forehead keypoints > 0.3 is judged as confusion). ② Speech verification: MFCC (13-dimensional coefficients + 1st-order difference + 2nd-order difference, a total of 39 dimensions) is used to extract speech features, which are input into an LSTM model (2-layer bidirectional LSTM, hidden layer dimension 256, dropout=0.3) to classify into 3 emotion probabilities of "questioning / confident / irritable". ③ Decision Fusion: A weighted voting method is adopted, with a weight of 0.6 for the text and behavioral basis results, a weight of 0.25 for the visual state probability, and a weight of 0.15 for the voice emotion probability. When the overall probability is >0.7, the original conclusion is strengthened, and when it is <0.5, the label is adjusted (e.g., when “text error + visual focus + voice confidence”, the overall probability is <0.5, and it is judged as “careless mistake”).

[0106] In this embodiment, by dynamically calculating the attention weights of each modality feature vector, adaptive fusion of text, speech, vision and behavioral data is achieved, enhancing the complementarity and discriminative power of feature representation, improving the distinguishability and diagnostic effectiveness of learning and learning fusion features, and overcoming the problem of insufficient adaptability of traditional fixed weight fusion mechanisms in different scenarios.

[0107] In one embodiment, such as Figure 5 As shown, the preset diagnostic dimensions include knowledge mastery, core competencies, learning status, and learning attitude. Step 208 includes:

[0108] Step 502: Extract text features and behavioral features, classify knowledge mastery levels based on the text features and behavioral features, and verify and correct the classification results based on the visual features and speech features.

[0109] Step 504: Evaluate logical reasoning ability based on the text features and behavioral features, evaluate language expression ability based on the speech features and text features, and evaluate practical learning ability based on the visual features and behavioral features.

[0110] Step 506: Classify focus based on the visual features, assess fatigue based on the visual features and behavioral features, and analyze course participation based on the voice features and behavioral features.

[0111] Step 508: Classify emotions based on the speech features, recognize facial expressions based on the visual features, and construct an emotion-behavior mapping relationship by combining the behavioral features.

[0112] For example, emotion analysis can be achieved through speech emotion recognition and facial expression emotion recognition. Speech emotion recognition can employ a fusion algorithm of "traditional features + deep learning features": ① Feature extraction: Traditional features such as MFCC (13-dimensional coefficients + 1st-order difference + 2nd-order difference, totaling 39 dimensions), fundamental frequency (F0), and short-time energy (STE) are combined with the Wav2Vec2-small model (pre-trained on a Chinese emotional speech corpus, with a batch size of 32 and a learning rate of 1e-4) to extract 768-dimensional deep features. ② Feature fusion: Attention-weighted fusion is adopted, dynamically allocating the proportion of traditional features (weight 0.3) and deep features (weight 0.7) through a single fully connected network. ③ Emotion classification: A BiLSTM+Attention model (BiLSTM hidden layer dimension 256, attention head number 4, dropout=0.35) is used to output 5 emotion categories: "calm / positive / irritable / anxious / confident," with an accuracy of 89%. The anxiety emotion is verified by the fundamental frequency fluctuation amplitude (>50Hz).

[0113] For example, facial expression and emotion recognition can employ a dual-branch model of "keypoint features + image features": ① Keypoint features: MediaPipe Face Mesh extracts 468 3D facial keypoints, and calculates the Euclidean distance between keypoints (e.g., the distance between the brow bone and the corner of the mouth) to construct 128-dimensional geometric features. ② Image features: EfficientNet-B0 (input size 224×224, batch size 64, learning rate 3e-4, transfer learning from ImageNet) is used to extract 2048-dimensional image features, and an attention module is introduced to focus on key areas of the eyes, eyebrows, and mouth. ③ Classification model: The dual-branch features are concatenated and input into an MLP (hidden layer dimensions 1024→512→256, activation function GELU), outputting 7 emotion labels: "joy / anger / sorrow / happiness / confusion / boredom / excitement". In educational scenarios, the accuracy of recognizing confusion and boredom reaches 90.1%.

[0114] In this way, a behavioral-emotional association can be established. Specifically, an "emotion-behavior" mapping rule can be established, such as "repeatedly clicking on the same knowledge point + frowning expression + irritable voice" as "learning anxiety"; "actively expanding learning + excited expression + active participation" as "strong interest in learning".

[0115] In this embodiment, by integrating features from different modalities, a collaborative diagnosis of four dimensions—knowledge mastery, core competencies, learning status, and learning emotions—is achieved. This enhances the comprehensiveness and explanatory power of learning analysis, enabling a multi-faceted characterization of students' learning status from cognitive, behavioral, and emotional perspectives, and providing a more refined and comprehensive basis for personalized teaching decisions.

[0116] In one embodiment, it can be as follows Figure 6 As shown, the method further includes:

[0117] Step 602: Determine the focus index of the target object based on the multi-dimensional diagnostic results.

[0118] For example, a three-level algorithm of "face detection + expression classification + behavior correction" can be used to diagnose attention levels: ① Face detection: The MTCNN algorithm (minimum face size 20×20, confidence threshold 0.7, NMS threshold 0.3) quickly locates the student's facial region with a processing speed of up to 30fps, suitable for classroom scenarios with multiple students. ② Expression classification: Based on an improved CNN model (input 64×64 grayscale image, containing 3 convolutional layers + 2 fully connected layers, introducing batch normalization layers to prevent overfitting, learning rate 5e-4), expressions are classified into 5 categories: "focused / distracted / confused / bored / other", with an accuracy of 92.5%; combined with Dlib keypoints, the gaze direction (deviating from the center of the screen >30° is judged as distracted) and eye opening / closing degree (<0.2 is judged as drowsy) are calculated, and the initial attention level (level 1-5) is output. ③ Behavior correction: A logistic regression model is used (inputs are classroom interaction response speed and the rate of change of answer accuracy), and the output is a correction coefficient. For example, "initial level 3 + response speed < 1 second" is corrected to level 4, and "initial level 3 + response speed > 5 seconds" is corrected to level 2.

[0119] For example, a multi-feature fusion algorithm integrating visual and behavioral modalities can be used for fatigue diagnosis: ① Visual features: 68 facial key points are extracted using MediaPipe Face Mesh, and the frequency of eyelid drooping (>5 times per minute is abnormal) and the duration of yawning (>3 seconds per yawn is abnormal) are calculated to construct 2D visual fatigue features. ② Behavioral features: The sliding window method (window size 10 minutes) is used to statistically analyze the rate of change in answering speed (decrease >20% is abnormal) and the rate of change in error rate (increase >15% is abnormal) to construct 2D behavioral fatigue features. ③ Fusion diagnosis: An XGBoost classifier (objective function multi:softmax, tree depth 5, estimators=150, learning rate 0.08) is used as input, and the 4D fusion features are output as fatigue level (mild / moderate / severe), where the threshold for severe fatigue is a model output probability >0.7.

[0120] Step 604: In response to the target object's focus index falling below a preset threshold, a real-time alert is triggered and an intervention plan matching the target object is generated.

[0121] For example, a "multi-dimensional feature weighting + temporal smoothing" algorithm can be used to diagnose course participation: ① Feature quantification: For the speech modality, the VAD (voice activity detection) algorithm is used to statistically analyze the percentage of speaking time (as a percentage of total class time) and the number of times speaking; for the behavioral modality, the frequency of raising hands (number of times per class) and the interactive question-and-answer participation rate (percentage of answers to pushed questions) are statistically analyzed, totaling 4 core features, each normalized to a score of 0-10. ② Weight calculation: The entropy weight method (calculated based on the feature entropy values ​​of 1000 class samples) is used to determine the weights: speaking time percentage 0.3, number of speaking times 0.25, frequency of raising hands 0.2, and interactive question-and-answer participation rate 0.25. ③ Temporal smoothing: The participation score is processed every 5 minutes using an exponential moving average (EMA, smoothing coefficient 0.8) to avoid the influence of instantaneous fluctuations. A final score <60 indicates low participation.

[0122] In this embodiment, by monitoring students' attention levels in real time and triggering warnings and interventions when the levels fall below a threshold, dynamic perception and immediate response to students' learning status are achieved. This improves the timeliness and relevance of teaching interventions, helps to improve students' learning status, enhance classroom participation, and promote the effective implementation of personalized teaching loops.

[0123] In one embodiment, it can be as follows Figure 7 As shown, the method further includes:

[0124] Step 702: Real-time data acquisition and lightweight processing based on pre-deployed edge devices.

[0125] Step 704: Perform multimodal fusion and model inference based on a pre-deployed cloud server.

[0126] In this embodiment, through a cloud-edge collaborative architecture, edge devices perform real-time data acquisition and lightweight processing, reducing data transmission pressure and latency; the cloud is responsible for complex fusion and model inference, achieving a balance between high-performance computing and real-time response, and improving the deployment feasibility and service stability of the system in real educational scenarios.

[0127] In one specific embodiment, the personalized learning analysis implementation device provided in this application can be as follows: Figure 8 and Figure 9 As shown.

[0128] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0129] Based on the same inventive concept, this application also provides a personalized learning analysis implementation device for implementing the personalized learning analysis implementation method described above. The solution provided by this device is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the personalized learning analysis implementation device provided below can be found in the limitations of the personalized learning analysis implementation method described above, and will not be repeated here.

[0130] In one embodiment, such as Figure 10 As shown, a personalized learning analysis device is provided, including: a data acquisition module, a data processing module, a feature fusion module, a multidimensional analysis module, and a report generation module, wherein:

[0131] The data acquisition module is used to acquire multimodal learning data of the target object, including text data, voice data, visual data, and behavioral data.

[0132] The data processing module is used to process the multimodal learning data to obtain feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction.

[0133] The feature fusion module is used to dynamically weight and fuse the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector.

[0134] The multidimensional analysis module is used to perform preset multidimensional diagnoses based on the learning situation fusion feature vector and through a preset multidimensional diagnostic model, and output multidimensional diagnostic results.

[0135] The report generation module is used to generate an analysis report for the target object based on the multi-dimensional diagnostic results, and the analysis report includes personalized learning suggestions.

[0136] In one embodiment, the data processing module includes:

[0137] The voice data processing module is used to perform noise reduction and silence removal processing on the voice data;

[0138] A visual data processing module is used to perform face detection and expression classification processing on the visual data;

[0139] The behavior data processing module is used to perform outlier detection and data cleaning on the behavior data.

[0140] In one embodiment, the feature fusion module includes:

[0141] The attention configuration module is used to calculate the attention weight parameters of the feature vectors corresponding to each modality based on a preset attention configuration mechanism.

[0142] The weighted fusion module is used to perform weighted fusion of the feature vector based on the attention weight parameters to obtain the learning situation fusion feature vector.

[0143] In one embodiment, the preset diagnostic dimensions include knowledge acquisition, core competencies, learning status, and learning attitude, and the multidimensional analysis module includes:

[0144] The knowledge mastery analysis module is used to extract text features and behavioral features, classify knowledge mastery levels based on the text features and behavioral features, and verify and correct the classification results based on the visual features and speech features.

[0145] And / or a core competency analysis module, used to evaluate logical reasoning ability based on the text features and the behavioral features, to evaluate language expression ability based on the voice features and the text features, and to evaluate practical learning ability based on the visual features and the behavioral features;

[0146] And / or a learning status analysis module, used to classify attention based on the visual features, assess fatigue based on the visual features and the behavioral features, and analyze course participation based on the voice features and the behavioral features;

[0147] And / or a learning sentiment analysis module, used to classify sentiment categories based on the speech features, recognize facial expressions based on the visual features, and construct an emotion-behavior mapping relationship by combining the behavioral features.

[0148] In one embodiment, the device further includes:

[0149] The focus module is used to determine the focus index of the target object based on the multi-dimensional diagnostic results.

[0150] The focus intervention module is used to trigger a real-time warning and generate an intervention plan that matches the target object when the focus index of the target object is lower than a preset threshold.

[0151] In one embodiment, the device further includes:

[0152] The edge device module is used for real-time data acquisition and lightweight processing based on pre-deployed edge devices;

[0153] The cloud server module is used for multimodal fusion and model inference based on a pre-deployed cloud server.

[0154] The modules in the aforementioned personalized learning analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0155] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a personalized learning analysis method.

[0156] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0158] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0159] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0161] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for realizing individual chemical emotion analysis, characterized by, The method includes: Acquire multimodal learning data of the target object, wherein the multimodal learning data includes text data, voice data, visual data, and behavioral data; The multimodal learning data is processed to obtain feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction. The feature vectors are dynamically weighted and fused using a pre-defined cross-modal attention fusion mechanism to generate a unified learning fusion feature vector. Based on the learning fusion feature vector, preset multi-dimensional diagnoses are performed using a preset multi-dimensional diagnostic model, and multi-dimensional diagnostic results are output. Based on the multi-dimensional diagnostic results, an analysis report is generated for the target object, and the analysis report includes personalized learning suggestions.

2. The method of claim 1, wherein, The process of processing the multimodal learning data to obtain feature vectors corresponding to each modality includes preprocessing, annotation, temporal alignment, and feature extraction. The voice data is subjected to noise reduction and silence removal processing; The visual data is then processed for face detection and expression classification. The behavioral data is subjected to outlier detection and data cleaning.

3. The method of claim 1, wherein, The step of dynamically weighting and fusing the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector includes: Based on a preset attention configuration mechanism, the attention weight parameters of the feature vectors corresponding to each modality are calculated respectively; The feature vector is weighted and fused based on the attention weight parameters to obtain the learning situation fusion feature vector.

4. The method of claim 1, wherein, The preset diagnostic dimensions include knowledge mastery, core competencies, learning status, and learning attitude. Based on the learning situation fusion feature vector, the preset multi-dimensional diagnostic model is used to perform preset multi-dimensional diagnoses, and the output multi-dimensional diagnostic results include: Extract text features and behavioral features, classify knowledge mastery levels based on the text features and behavioral features, and verify and correct the classification results based on the visual features and speech features; And / or assess logical reasoning ability based on the text features and behavioral features, assess language expression ability based on the speech features and text features, and assess practical learning ability based on the visual features and behavioral features; And / or perform attention classification based on the visual features, assess fatigue based on the visual features and the behavioral features, and analyze course participation based on the voice features and the behavioral features; And / or perform emotion category classification based on the speech features, perform facial expression emotion recognition based on the visual features, and construct an emotion-behavior mapping relationship by combining the behavioral features.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the multi-dimensional diagnostic results, the focus index of the target object is determined; In response to the target object's focus index falling below a preset threshold, a real-time alert is triggered and an intervention plan matching the target object is generated.

6. The method of claim 1, wherein, The method further includes: Real-time data acquisition and lightweight processing are performed using pre-deployed edge devices; Multimodal fusion and model inference are performed based on pre-deployed cloud servers.

7. An individual chemical profile analysis implementation device, characterized by, The device includes: The data acquisition module is used to acquire multimodal learning data of the target object, including text data, voice data, visual data, and behavioral data. The data processing module is used to process the multimodal learning data to obtain feature vectors corresponding to each modality. The processing includes preprocessing, annotation, temporal alignment and feature extraction. The feature fusion module is used to dynamically weight and fuse the feature vectors using a preset cross-modal attention fusion mechanism to generate a unified learning fusion feature vector. The multidimensional analysis module is used to perform preset multidimensional diagnoses based on the learning situation fusion feature vector and through a preset multidimensional diagnostic model, and output multidimensional diagnostic results. The report generation module is used to generate an analysis report for the target object based on the multi-dimensional diagnostic results, and the analysis report includes personalized learning suggestions. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.