Student potential and deviation identification method and device and readable storage medium

Abstract

The invention provides a student potential and deviation identification method and device and a readable storage medium, and the method comprises the steps: collecting the multi-modal data of a student, and carrying out the feature extraction, and obtaining a multi-dimensional feature; performing feature fusion on the multi-dimensional features to obtain student feature vectors; performing potential and deviation identification based on the student feature vector to obtain a potential identification result and a deviation identification result of the student; and generating a visual report of the potential and the deviation of the student, wherein the visual report comprises a potential identification result and a deviation identification result of the student and corresponding personalized learning plans and teaching method suggestions. According to the method, through combination of multi-modal data fusion and a deep learning model, accurate identification of student potential and learning bias is realized, and an identification strategy is continuously optimized through a reinforcement learning mechanism, so that effective support is provided for personalized teaching.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device and readable storage medium for identifying student potential and preference. Background Art

[0002] With the rapid development of artificial intelligence, computer vision, natural language processing and other technologies, multimodal behavior analysis has gradually become a research hotspot in the field of education. This technology integrates information from multiple modalities such as video, audio, and text to achieve a comprehensive and in-depth analysis of student behavior, making it possible to accurately identify students' potential and preferences.

[0003] Multimodal behavior analysis technology refers to the use of multiple sensors and data sources (such as cameras, microphones, touch screens, etc.) to simultaneously collect and analyze data from different modalities to achieve a comprehensive understanding and judgment of complex behaviors. In the field of education, this technology can be applied to multiple scenarios such as classroom observation and student behavior analysis, providing teachers with more comprehensive and objective student behavior data support.

[0004] Therefore, based on multimodal behavior analysis technology, how to accurately identify students' potential and preferences in order to achieve personalized teaching for students has become a problem that needs to be solved. Summary of the invention

[0005] The technical problem to be solved by this application is to provide a student potential and preference identification method, device and readable storage medium to solve the problems existing in the prior art in view of the above-mentioned deficiencies in the prior art.

[0006] In the first aspect, the present application provides a method for identifying student potential and bias,

[0007] Methods include:

[0008] S1, collect students' multimodal data and perform feature extraction to obtain multidimensional features;

[0009] S2, performing feature fusion on the multi-dimensional features to obtain a student feature vector;

[0010] S3, performing potential and bias identification based on the student feature vector to obtain the student's potential identification result and bias identification result;

[0011] S4. Generate a visualization report of the student's potential and bias, wherein the visualization report includes the student's potential identification results, bias identification results, and corresponding personalized learning plans and teaching method recommendations.

[0012] In some embodiments, S3 includes:

[0013] Performing deep feature extraction on the student feature vector through a Transformer model to obtain deep-level features;

[0014] The dynamic change characteristics of the deep-level features are obtained through a long short-term memory network LSTM model;

[0015] The dynamic change features are subjected to potential classification and bias identification through the Softmax layer to obtain the potential identification results and bias identification results of the students.

[0016] In some embodiments, the potential identification results include at least logical reasoning, spatial imagination, and number sense;

[0017] The bias recognition results include at least visual, auditory, and kinesthetic types.

[0018] In some embodiments, the multimodal data includes text data, image data, and sound data;

[0019] S1, including:

[0020] The text data, image data and sound data of the students are collected, and the text features, image features and sound features of the students are obtained through feature extraction.

[0021] In some embodiments, the text features, image features, and sound features of the student are obtained by feature extraction, including:

[0022] Use the BERT model to extract features from text data and obtain corresponding text features;

[0023] The image data is subjected to feature extraction through the convolutional neural network CNN to obtain the corresponding image features;

[0024] The sound data is feature extracted through the recurrent neural network (RNN) to obtain the corresponding sound features.

[0025] In some embodiments, before extracting features, the method further includes: preprocessing the multimodal data;

[0026] Wherein, the preprocessing of the text data includes word segmentation and removal of stop words;

[0027] The preprocessing of the image data includes resizing, cropping and normalization;

[0028] The preprocessing of the sound data includes noise reduction, normalization processing and Mel frequency cepstral coefficient MFCC feature extraction.

[0029] In some embodiments, after S3, the method further includes:

[0030] The potential and bias identification process is optimized through the reinforcement learning Q-learning algorithm to obtain the optimized potential identification results and bias identification results.

[0031] In a second aspect, the present application provides a device for identifying student potential and preference, the device comprising:

[0032] A data collection module is configured to collect multimodal data of students and perform feature extraction to obtain multidimensional features;

[0033] A feature fusion module, which is configured to perform feature fusion on the multi-dimensional features to obtain a student feature vector;

[0034] A potential and bias identification module, which is configured to identify the potential and bias based on the student's feature vector to obtain the student's potential identification result and bias identification result;

[0035] A report generation module is configured to generate a visual report of a student's potential and bias, wherein the visual report includes the student's potential identification results, bias identification results, and corresponding personalized learning plans and teaching method recommendations.

[0036] In a third aspect, the present application provides a device for identifying student potential and bias, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the method for identifying student potential and bias described in the first aspect above.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for identifying student potential and preference described in the first aspect above is implemented.

[0038] The present application provides a method, device and readable storage medium for identifying student potential and bias, the method comprising: collecting multimodal data of students, and performing feature extraction to obtain multidimensional features; performing feature fusion on the multidimensional features to obtain student feature vectors; performing potential and bias identification based on the student feature vectors to obtain the student's potential identification results and bias identification results; generating a visualization report of the student's potential and bias, the visualization report containing the student's potential identification results, bias identification results and corresponding personalized learning plans and teaching method suggestions. The present application provides a method for identifying student potential and bias, which realizes comprehensive analysis and identification of student potential and bias by adopting multimodal data fusion combined with deep learning technology. The whole process forms a clear process from data collection to result output, realizes multi-level interpretation of data through feature extraction and fusion, and can continuously learn and optimize its own identification strategy. Through precise multimodal analysis, effective support is provided for personalized teaching, enabling educators to better understand students' learning needs. This application achieves accurate identification of students' potential and learning preferences by combining multimodal data fusion and deep learning models, and continuously optimizes the identification strategy through reinforcement learning mechanisms, providing effective support for personalized teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] Figure 1 A flow chart of a method for identifying student potential and bias provided in an embodiment of the present application;

[0041] Figure 2 A flowchart of another method for identifying student potential and bias provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the structure of a device for identifying student potential and preference provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of another device for identifying student potential and preference provided in an embodiment of the present application.

[0044] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the technical solution of the present application, the implementation mode of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] It should be understood that the specific embodiments and drawings described herein are only used to explain the present application, rather than to limit the present application.

[0047] It can be understood that, in the absence of conflict, the various embodiments in the present application and the various features in the embodiments can be combined with each other.

[0048] It can be understood that, for the convenience of description, the drawings of the present application only show the parts related to the present application, while the parts unrelated to the present application are not shown in the drawings.

[0049] It can be understood that each unit and module involved in the embodiments of the present application may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0050] It can be understood that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0051] It is understandable that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present application may occur in an order different from that marked in the drawings.

[0052] It is understood that the flowcharts and block diagrams of the present application illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present application. Among them, each box in the flowchart or block diagram may represent a unit, module, program segment, code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.

[0053] It can be understood that the units and modules involved in the embodiments of the present application can be implemented by software or hardware, for example, the units and modules can be located in a processor.

[0054] With the deepening of educational informatization, multimodal behavior analysis technology will play a more important role in the field of education and contribute to the cultivation of high-quality talents with innovative spirit and practical ability. At present, the importance of accurate identification of students' potential and preferences is self-evident, which is specifically reflected in the following aspects:

[0055] (1) Personalized teaching needs: Every student has unique potential and interests. Accurately identifying these characteristics helps to achieve personalized teaching and improve teaching effectiveness.

[0056] (2) Student development guidance: By identifying students’ potential and preferences, teachers can provide students with more targeted development guidance and suggestions to help them better plan for the future.

[0057] (3) Optimization of educational resources: Accurately identifying students’ needs will help schools and educational institutions optimize the allocation of educational resources and improve the efficiency of their use.

[0058] This application uses multimodal behavior analysis to accurately identify students' potential and preferences, and has the following advantages:

[0059] (1) Comprehensiveness: Integrating data from multiple modalities enables a comprehensive analysis of student behavior, avoiding the one-sidedness brought about by a single data source.

[0060] (2) Accuracy: Advanced algorithms and models are used to accurately identify and analyze student behaviors, improving the accuracy and reliability of identification.

[0061] (3) Real-time: The system can collect and process data in real time, providing instant feedback to teachers so that they can adjust their teaching strategies in a timely manner.

[0062] This application proposes a student potential and bias identification system based on multimodal data fusion and deep learning models. The system uses the BERT word vector model, Transformer model and LSTM model, comprehensively considers multiple learning expressions such as text, images and sounds, and accurately identifies student potential and learning bias. Through in-depth analysis of students' multi-source data, it helps teachers understand students' learning status and provide personalized teaching suggestions.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] The present application provides a method for identifying student potential and preferences. The working process of the method can be implemented by electronic devices, such as computers, handheld smart terminals, etc. For the convenience of explanation, the embodiments of the present application are described with the method execution subject being a computer.

[0065] Figure 1 A schematic diagram of a method for identifying student potential and bias provided in an embodiment of the present application, Figure 2Another schematic diagram of the method for identifying student potential and bias provided in the embodiment of the present application is as follows: Figure 1 as well as Figure 2 As shown, the present application provides a method for identifying student potential and preference, the method comprising S1-S4, specifically as follows:

[0066] S1, collect students' multimodal data and perform feature extraction to obtain multidimensional features;

[0067] In some embodiments, the multimodal data includes text data, image data, and sound data;

[0068] S1, including: collecting text data, image data and sound data of students, and obtaining text features, image features and sound features of the students through feature extraction.

[0069] Specifically, the multimodal data collection process includes:

[0070] (1) Text data: Collect text information from students’ homework, exams, and class notes. Perform word segmentation, remove stop words, and tag parts of speech, and convert them into word vector representations using the BERT model.

[0071] (2) Image data: Use a high-definition camera to record students’ facial expressions and body movements in class.

[0072] (3) Sound data: Use a high-sensitivity microphone to record students’ speeches and collect voice features.

[0073] In some embodiments, before extracting features, the method further includes: preprocessing the multimodal data;

[0074] Wherein, the preprocessing of the text data includes word segmentation and removal of stop words;

[0075] The preprocessing of the image data includes resizing, cropping and normalization;

[0076] The preprocessing of the sound data includes noise reduction, normalization processing and Mel Frequency Cepstrum Coefficient (MFCC) feature extraction.

[0077] Specifically, the data preprocessing process includes:

[0078] (1) Text data preprocessing:

[0079] Tokenization: Split text into processable words or phrases.

[0080] Stop word removal: Remove common meaningless words to improve the signal-to-noise ratio.

[0081] (2) Image data preprocessing:

[0082] Resizing: Resize images to a uniform size for subsequent processing.

[0083] Cropping and Normalization: Preserve important features and improve processing stability.

[0084] (3) Sound data preprocessing:

[0085] Noise reduction and normalization processing ensure recording quality and data comparability.

[0086] MFCC feature extraction: obtain the key features of audio through Mel frequency cepstral coefficients.

[0087] In some embodiments, the text features, image features, and sound features of the student are obtained by feature extraction, including:

[0088] Use the BERT model to extract features from text data and obtain corresponding text features;

[0089] The convolutional neural network (CNN) is used to extract features from image data and obtain corresponding image features, including visual features of students’ expressions, body movements, and math problems.

[0090] The sound data is extracted through the recurrent neural network (RNN) to obtain the corresponding sound features, including the student's speaking speed, pitch, pauses, etc.

[0091] Specifically, the feature extraction process includes:

[0092] (1) Text feature extraction:

[0093] Use BERT word vectors to convert text into dense vector representation. The mathematical formula is:

[0094] E=BERT(W(D))

[0095] The input is the text W(D) after word segmentation, and the output is the text feature vector E.

[0096] Working principle: The BERT model receives the text sequence W(D) after word segmentation, outputs the vector representation of each word through its multi-layer Transformer structure, and then average pools these vectors to obtain a 512-dimensional text feature vector E.

[0097] (2) Image feature extraction:

[0098] Use convolutional neural network (CNN) to extract image features:

[0099] F=CNN(I)

[0100] The input is image data I, and the output is image feature vector F.

[0101] Working principle: The CNN model receives preprocessed image data I, extracts image features through convolutional layers and pooling layers, and finally outputs a 2048-dimensional image feature vector F.

[0102] (3) Sound feature extraction:

[0103] Using recurrent neural network (RNN) to process sound features:

[0104] S = RNN (MFCC (A))

[0105] The input is the Mel-frequency cepstral coefficient MFCC; the output is the sound feature vector S.

[0106] Working principle: First, 13-dimensional features are extracted from audio data A through the MFCC algorithm, and then these features are processed using the RNN model to capture time series information and output a 256-dimensional sound feature vector S.

[0107] S2, performing feature fusion on the multi-dimensional features to obtain a student feature vector;

[0108] Specifically, the extracted text features, image features, and sound features are concatenated to form a unified student feature vector:

[0109] X=[E;F;S]

[0110] Among them, the output is the synthesized student feature vector X, and the input is the text feature vector E, the image feature vector F and the sound feature vector S.

[0111] Working principle: The text feature vector E, image feature vector F and sound feature vector S are concatenated to form a 3072-dimensional student feature vector X.

[0112] S3, performing potential and bias identification based on the student feature vector to obtain the student's potential identification result and bias identification result;

[0113] In some embodiments, S3 includes:

[0114] Performing deep feature extraction on the student feature vector through a Transformer model to obtain deep-level features;

[0115] The dynamic change characteristics of the deep-level features are obtained through a long short-term memory network LSTM model;

[0116] The dynamic change features are subjected to potential classification and bias identification through the Softmax layer to obtain the potential identification results and bias identification results of the students.

[0117] Specifically, the potential and bias identification process includes:

[0118] (1) Transformer model:

[0119] The Transformer model is used to extract deeper features and capture the inherent rules of students’ mathematics learning. Specifically, the student feature vector X is input into the Transformer to extract deeper features:

[0120] H = Transformer(X)

[0121] Among them, the output is the deep feature H.

[0122] Working principle: The Transformer model receives a 3072-dimensional student feature vector X, extracts deeper features through the self-attention mechanism and feedforward neural network, and outputs a 1024-dimensional deep feature H.

[0123] (2) LSTM hidden layer:

[0124] Use the LSTM model to capture dynamic changes, such as the thinking process of students when solving math problems. Specifically, input the deep feature H into the LSTM model to capture dynamic changes:

[0125] L=LSTM(H)

[0126] Among them, the output is the LSTM hidden layer output L.

[0127] Working principle: The LSTM model receives 1024-dimensional deep features H, captures the dynamic changes of sequence data through its gating mechanism, and outputs 128-dimensional dynamically changing features L.

[0128] (3) Potential Identification:

[0129] The potential is classified through the Softmax layer:

[0130] P = Softmax(LW+b)

[0131] Among them, the input is the LSTM output L; the output is the potential recognition result P, W is the weight matrix, and b is the bias vector.

[0132] In machine learning and deep learning, the Softmax function is often used in the output layer of multi-classification problems to convert a real vector into a probability distribution. W and b in the formula Softmax(LW+b) represent the following meanings:

[0133] W (weight matrix): In neural networks, W represents the weight matrix connecting each neuron in the previous layer to each neuron in the current layer. In the context of the Softmax function, L usually represents the unnormalized predictions (also called logits) from the previous layer, which are linear combinations of the neurons in the current layer. Each element wij of W defines the weight of the connection between the i-th neuron in the previous layer and the j-th neuron in the current layer.

[0134] b (bias vector): b is a vector containing the bias term for each output neuron. The bias allows the model to add a constant term to each output neuron before calculating the activation function. This is functionally equivalent to changing the activation threshold of the neuron.

[0135] Specifically, if L is an m-dimensional vector (the output from the previous layer), W is an n×m weight matrix (where n is the number of neurons in the current layer), and b is an n-dimensional bias vector, then LW+b is calculated as follows:

[0136] First, matrix multiplication LW is performed to obtain an n-dimensional vector, each element of which is the weighted sum of L. Then the bias vector b is added to each of the above elements. The final vector is processed by the Softmax function, each element is converted to a value between 0 and 1, and the sum of all elements is 1, thus forming a valid probability distribution.

[0137] Working principle: The Softmax layer receives the 128-dimensional LSTM output L, calculates the probability distribution of each potential category, and outputs the potential recognition result P.

[0138] In some embodiments, the potential identification results include at least logical reasoning, spatial imagination, and number sense.

[0139] Among them, logical reasoning potential: Logical reasoning refers to the ability to understand concepts, evaluate arguments, solve problems, and make decisions through analysis and reasoning. This potential involves identifying patterns, understanding relationships, applying rules and principles, and deriving new conclusions from known information. Students with strong logical reasoning potential usually have the following abilities: think about problems clearly and accurately, distinguish between facts and opinions, identify assumptions and premises, construct valid arguments, and identify logical fallacies.

[0140] Among them, spatial imagination potential: Spatial imagination ability refers to the ability to understand and manipulate spatial relationships without direct visual assistance. This potential is essential for understanding graphics, shapes, sizes, directions, positions, and how they change in three-dimensional space. Students with strong spatial imagination potential usually have the following abilities: forming a three-dimensional image of an object in their mind, understanding and manipulating complex geometric relationships, predicting the movement or change of objects in space, and designing and creating spatial layouts.

[0141] Among them, number sense (mathematical perception) potential: number sense refers to the intuitive understanding of numbers and the relationship between them, as well as the ability to make quantitative reasoning. This potential is not limited to mathematical calculations, but also includes the overall understanding of numerical patterns, proportions, estimation and mathematical problems. Students with strong number sense potential usually have the following abilities:

[0142] Quickly estimate values, identify patterns and relationships between numbers, understand mathematical concepts and principles, and apply mathematical knowledge to solve real-world problems.

[0143] (4) Bias identification:

[0144] Also use the Softmax layer to identify learning bias:

[0145] P = Softmax(LW'+b')

[0146] Among them, the input is the LSTM output L; the output is the bias recognition result B, W' is the weight matrix, and b' is the bias vector.

[0147] Working principle: The Softmax layer is also used to receive the 128-dimensional LSTM output L, calculate the probability distribution of each learning bias category, and output the bias recognition result B.

[0148] In some embodiments, the biased recognition results include at least visual, auditory, and kinesthetic types.

[0149] Among them, Visual Learning Style: Students with a visual learning style tend to receive and process information best through visual stimulation. They usually respond well to images, charts, color codes, presentations, and other visual aids. The following are the characteristics of visual learners: like to read and view pictures, charts, and presentations, easily remember what they see, may take notes or draw pictures to help memory when studying, and tend to look for visual clues to understand and remember information.

[0150] Among them, Auditory Learning Style: Students with Auditory Learning Style prefer to learn through auditory stimulation. They are usually sensitive to sound elements such as oral explanations, discussions, lectures, and music. The following are the characteristics of auditory learners: like to learn through listening, discussing, and asking questions, strengthen memory through repetition and verbal retelling, tend to use recording equipment to review course content, and may prefer a quiet environment when studying to better listen to information.

[0151] Among them, Kinesthetic Learning Style: Students with kinesthetic learning style learn best through physical movement and hands-on operation. They usually need to understand and remember information through practical activities, experiments, simulations and movements. The following are the characteristics of kinesthetic learners: like to learn through hands-on operation, experiments and role-playing, strengthen memory through physical movement and practical operation, may need to take frequent breaks when learning in order to regulate energy through activities, and tend to use models, tools and real objects to understand abstract concepts.

[0152] In some embodiments, after S3, the method further includes:

[0153] The potential and bias identification process is optimized through the reinforcement learning Q-learning algorithm to obtain the optimized potential identification results and bias identification results.

[0154] This step uses the Q-learning algorithm to continuously optimize the potential and bias recognition models based on students' feedback and learning results to improve recognition accuracy. Specifically, the affinity learning mechanism is used to improve the recognition accuracy of the potential recognition result P and the bias recognition result B:

[0155]

[0156] The input is state s, action a and reward r; the output is the updated state-action value function Q(s,a).

[0157] In the Q-learning algorithm in reinforcement learning, the relationship between state (s), action (a), reward (r), potential (P) and bias (B) can be understood as follows:

[0158] State s: In the context of reinforcement learning, state s represents the current situation of the environment. In the context of potential and bias identification, state s may include factors such as students’ personal information, historical learning behavior, test scores, learning materials, and learning environment. State s describes the student’s current learning status.

[0159] Action a: Action a is the action that the reinforcement learning agent (here, the potential and bias identification model) can take in a given state s. In the context of potential and bias identification, action a may include recommending specific learning resources, adjusting learning paths, providing customized learning plans, etc.

[0160] Reward r: Reward r is the feedback obtained from the environment after the agent takes action a. In the context of potential and bias identification, reward r can be defined based on student feedback (such as satisfaction surveys, self-assessment of learning effects) and learning effects (such as test scores, project completion). Reward r reflects whether the action taken helps improve the accuracy of recognition.

[0161] Potential P: Potential P is the potential ability or talent of the student identified by the model. This is usually assessed based on the student's performance, interests and previous learning record.

[0162] Bias B: Bias B is the student’s learning preference or tendency identified by the model, which may include interest in specific subjects, learning style (such as visual learning, auditory learning), study habits, etc.

[0163] The specific relationship is as follows:

[0164] State s and action a: In state s, the model chooses an action a to optimize the recognition of potential P and bias B.

[0165] Action a and reward r: After taking action a, the model receives a reward r based on the student's actual feedback and learning effect. If action a leads to more accurate recognition results, the reward r will be positive; if action a leads to worse recognition results, the reward r will be negative or zero.

[0166] Reward r and updated state-action value function Q(s,a): Reward r is used to update the Q function, which represents the expected return of taking action a in state s. The update formula of the Q function is usually:

[0167]

[0168] Potential P and bias B with state s, action a, reward r: Potential P and bias B are the outputs that the model attempts to optimize by taking action a in state s and based on reward r. The goal of the model is to continuously adjust the action selection strategy through the reinforcement learning process to improve the accuracy of potential and bias identification.

[0169] The Q-learning algorithm optimizes the potential and bias identification model through continuous exploration (trying different actions) and utilization (selecting actions based on the Q function), so that the model can better predict students' potential and bias, thereby providing more personalized learning support.

[0170] How it works: The Q-learning algorithm iteratively updates the state-action value function Q(s,a), where α is the learning rate, r is the reward, γ is the discount factor, s' is the next state, and a' is the next action. The algorithm aims to find the optimal strategy to maximize long-term rewards.

[0171] For example, if Student A achieves excellent results in a math competition, the probability of his / her "logical reasoning" potential is increased, and the teaching strategy is adjusted to further strengthen his / her logical reasoning ability.

[0172] S4. Generate a visualization report of the student's potential and bias, wherein the visualization report includes the student's potential identification results, bias identification results, and corresponding personalized learning plans and teaching method recommendations.

[0173] Specifically, a visualization report on the student's potential and bias is generated based on the student's potential identification results and bias identification results, including:

[0174] (1) Potential analysis: potential assessment of each subject.

[0175] (2) Bias analysis: learning style bias.

[0176] (3) Recommendations: Personalized learning plans and suggestions for adjustments to teaching methods.

[0177] For example, a visualization report could include the following:

[0178] Student A: has strong logical reasoning ability and spatial imagination, but relatively weak sense of numbers.

[0179] Student B: has a strong understanding of mathematical concepts, but relatively weak calculation ability.

[0180] Student C: is curious about math problems, but lacks the patience and perseverance to solve them.

[0181] Evaluation results:

[0182] Through the analysis of students' multimodal data, the following mathematical potential and learning biases were identified:

[0183] Student A: has strong logical reasoning ability and spatial imagination, but relatively weak sense of numbers.

[0184] Student B: has a strong understanding of mathematical concepts, but relatively weak calculation ability.

[0185] Student C: is curious about math problems, but lacks the patience and perseverance to solve them.

[0186] Based on the evaluation results, teachers develop personalized learning plans for each student and adjust teaching methods.

[0187] For Student A, the teacher recommended him to participate in the Mathematical Modeling Competition and strengthen his number sense training.

[0188] For Student B, the teacher uses a gamified teaching method to help the student improve his calculation ability.

[0189] For Student C, the teacher encourages him to study in groups and cultivates his patience and perseverance in solving problems.

[0190] Through multimodal behavior analysis technology, teachers can have a more comprehensive understanding of students' mathematics learning status and develop personalized teaching strategies based on students' potential and learning preferences, thereby improving teaching effectiveness, stimulating students' interest in learning mathematics, and cultivating their mathematical thinking ability.

[0191] This application provides a method for identifying student potential and bias, which realizes the comprehensive analysis and identification of student potential and bias by adopting multimodal data fusion combined with deep learning technology. The whole process forms a clear flow from data collection to result output, realizes multi-level interpretation of data through feature extraction and fusion, and can continuously learn and optimize its own identification strategy. Through precise multimodal analysis, it provides effective support for personalized teaching, enabling educators to better understand students' learning needs. This application realizes the accurate identification of student potential and learning bias by combining multimodal data fusion and deep learning models, and continuously optimizes the identification strategy through reinforcement learning mechanism, providing effective support for personalized teaching.

[0192] For ease of understanding, the following is a complete implementation process of student potential and preference identification based on the above technical solution, including the calculation steps of real data.

[0193] 1. Data Collection

[0194] Text Data

[0195] -A student's history exam essay was collected and the following text sequence was obtained after preprocessing:

[0196] - Input: ["student", "should", "diligent", "learning", "knowledge", "is", "power", "source"]

[0197] Image data

[0198] -Using a high-definition camera to record the student's facial expressions and body movements in class, a total of 10 images were collected. After preprocessing, the image size was uniformly adjusted to 256x256 pixels.

[0199] Sound data

[0200] - The 5-minute speech of the student in class was recorded using a high-sensitivity microphone. After noise reduction and normalization, 5 audio files were generated.

[0201] 2. Data Preprocessing

[0202] Text Data Preprocessing

[0203] - Input: ["student", "should", "be", "diligent", "in", "learning", "knowledge", "is", "the", "source", "of", "power"]

[0204] - Output: The text sequence after removing stop words (assuming "is" is a stop word).

[0205] Image Data Preprocessing

[0206] - Input: 10 images of 256x256 pixels.

[0207] - Output: The preprocessed image frames.

[0208] Sound Data Preprocessing

[0209] - Input: 5 audio files.

[0210] - Output: MFCC feature vectors, with 13-dimensional features extracted from each audio segment.

[0211] 3. Feature Extraction

[0212] Text Feature Extraction

[0213] - Input: The text sequence after removing stop words.

[0214] - Process: The BERT model converts the text into vector representations.

[0215] - Output: A 512-dimensional text feature vector E.

[0216] Image Feature Extraction

[0217] - Input: The preprocessed image frames.

[0218] - Process: The CNN model extracts features.

[0219] - Output: 2048-dimensional feature vectors F for each image, a total of 10 vectors.

[0220] Sound Feature Extraction

[0221] - Input: MFCC feature vectors.

[0222] - Process: The RNN model processes the sound features.

[0223] -Output: 256-dimensional feature vector S for each audio segment, 5 vectors in total.

[0224] 4. Feature Fusion

[0225] - Input: text feature vector E, image feature vector F, sound feature vector S. - Processing: concatenate E and the average of F and S.

[0226] - Output: 3072-dimensional student feature vector X.

[0227] 5. Potential and bias identification

[0228] Transformer model processing

[0229] - Input: Student feature vector X.

[0230] -Processing: Transformer model extracts deeper features.

[0231] -Output: 1024-dimensional deep features H.

[0232] LSTM model processing

[0233] -Input: deep features H.

[0234] -Processing: LSTM model captures dynamic changes.

[0235] -Output: 128-dimensional feature L.

[0236] Potential Identification

[0237] - Input: LSTM output L.

[0238] -Processing: Softmax layer classification.

[0239] -Output: potential identification result P.

[0240] Bias Identification

[0241] - Input: LSTM output L.

[0242] -Processing: Softmax layer classification.

[0243] -Output: biased recognition result B.

[0244] 6. Reinforcement Learning

[0245] Q-learning algorithm

[0246] - Input: current state s, action taken a, and reward obtained r.

[0247] -Processing: Update the state-action value function Q(s,a).

[0248] -Output: Updated Q(s,a).

[0249] 7. Result output

[0250] Visual Reports

[0251] -Generate a visualization report based on the potential identification results P and bias identification results B.

[0252] -Content includes potential analysis, bias analysis and personalized recommendations.

[0253] Through the above steps, we obtained the potential and bias identification results of the student, and made the following suggestions in the report:

[0254] - Potential analysis shows that the student has high potential in mathematical logic.

[0255] -Bias analysis shows that this student is more suitable for learning through a combination of visual and auditory methods.

[0256] -Personalized recommendations include recommending the student to participate in math and science competitions and using a combination of video and audio teaching resources.

[0257] It should be understood that, although the various steps in the flowcharts in the above-described embodiments are sequentially displayed according to the indications of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily to be carried out sequentially, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0258] Figure 3 A schematic diagram of a device for identifying student potential and preference provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the present application provides a device for identifying student potential and preference. In some embodiments, the device includes:

[0259] A data collection module 11 is configured to collect multimodal data of students and perform feature extraction to obtain multidimensional features;

[0260] A feature fusion module 12, which is configured to perform feature fusion on the multi-dimensional features to obtain a student feature vector;

[0261] A potential and bias identification module 13, which is configured to identify the potential and bias based on the student's feature vector, and obtain the potential identification result and bias identification result of the student;

[0262] The report generation module 14 is configured to generate a visual report of the student's potential and bias, wherein the visual report includes the student's potential identification result, bias identification result, and corresponding personalized learning plan and teaching method recommendations.

[0263] Regarding the limitation of the device for identifying student potential and preference, reference may be made to the limitation of the method for identifying student potential and preference in the above-mentioned embodiments of the present application, which will not be elaborated in this embodiment.

[0264] Figure 4 Another schematic diagram of the student potential and preference identification device provided in the embodiment of the present application is as follows Figure 4 As shown, in some embodiments, the present application provides a student potential and preference identification device, including a memory 22 and a processor 21, the memory storing a computer program, and the processor being configured to run the computer program to execute the student potential and preference identification method in the above-mentioned embodiments of the present application.

[0265] The memory is connected to the processor, the memory may be a flash memory or a read-only memory or other memory, and the processor may be a central processing unit or a single-chip microcomputer.

[0266] In some embodiments, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying student potential and preference in the above-mentioned embodiments of the present application is implemented.

[0267] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0268] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A method for identifying student potential and preference, characterized in that: The method comprises: S1, collect students' multimodal data and perform feature extraction to obtain multidimensional features; S2, performing feature fusion on the multi-dimensional features to obtain a student feature vector; S3, performing potential and bias identification based on the student feature vector to obtain the student's potential identification result and bias identification result; S4. Generate a visualization report of the student's potential and bias, wherein the visualization report includes the student's potential identification results, bias identification results, and corresponding personalized learning plans and teaching method recommendations.

2. The method for identifying student potential and tendency according to claim 1, characterized in that: S3, including: Performing deep feature extraction on the student feature vector through a Transformer model to obtain deep-level features; The dynamic change characteristics of the deep-level features are obtained through a long short-term memory network LSTM model; The dynamic change features are subjected to potential classification and bias identification through the Softmax layer to obtain the potential identification results and bias identification results of the students.

3. The method for identifying student potential and tendency according to claim 2, characterized in that: The potential identification results at least include logical reasoning, spatial imagination, and number sense; The bias recognition results include at least visual, auditory, and kinesthetic types.

4. The method for identifying student potential and preference according to any one of claims 1 to 3, characterized in that: The multimodal data includes text data, image data and sound data; S1, including: The text data, image data and sound data of the students are collected, and the text features, image features and sound features of the students are obtained through feature extraction.

5. The method for identifying student potential and tendency according to claim 4, characterized in that: The text features, image features and sound features of the student are obtained by feature extraction, including: Use the BERT model to extract features from text data and obtain corresponding text features; The image data is subjected to feature extraction through the convolutional neural network CNN to obtain the corresponding image features; The sound data is feature extracted through the recurrent neural network (RNN) to obtain the corresponding sound features.

6. The method for identifying student potential and tendency according to claim 4, characterized in that: Before feature extraction, it also includes: preprocessing the multimodal data; Wherein, the preprocessing of the text data includes word segmentation and removal of stop words; The preprocessing of the image data includes resizing, cropping and normalization; The preprocessing of the sound data includes noise reduction, normalization processing and Mel frequency cepstral coefficient MFCC feature extraction.

7. The method for identifying student potential and preference according to any one of claims 1 to 3, characterized in that: After S3, it also includes: The potential and bias identification process is optimized through the reinforcement learning Q-learning algorithm to obtain the optimized potential identification results and bias identification results.

8. A device for identifying student potential and tendency, characterized in that: The device comprises: A data collection module is configured to collect multimodal data of students and perform feature extraction to obtain multidimensional features; A feature fusion module, which is configured to perform feature fusion on the multi-dimensional features to obtain a student feature vector; A potential and bias identification module, which is configured to identify the potential and bias based on the student's feature vector to obtain the student's potential identification result and bias identification result; A report generation module is configured to generate a visual report of a student's potential and bias, wherein the visual report includes the student's potential identification results, bias identification results, and corresponding personalized learning plans and teaching method recommendations.

9. A device for identifying student potential and tendency, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the student potential and preference identification method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying student potential and preference as described in any one of claims 1 to 7 is implemented.

Images