Student user portrait construction method and terminal based on multi-modal data fusion

Abstract

The invention discloses a student user portrait construction method based on multi-modal data fusion and a terminal. The method comprises the following steps: acquiring learning behavior data, social behavior data and hobby and interest data of a user, and respectively extracting corresponding feature information; calculating the fusion weight of the feature information of each mode and generating a comprehensive feature vector; and on the basis of the comprehensive feature vector, performing student user portrait construction by using a clustering algorithm, an analytic hierarchy process and knowledge graph analysis. According to the method, data features of learning behaviors, social behaviors and hobbies and interests are extracted, modal information is synthesized by using a fusion weight mechanism, and finally a multi-dimensional and fine-grained student portrait is formed. Compared with a traditional single data source modeling method, the method has the advantages that the comprehensiveness, the accuracy and the personalized adaptive capacity of portraits are remarkably improved, and various application scenes such as personalized teaching recommendation, academic early warning and interest expansion activity matching can be effectively supported.

Description

Technical Field

[0001] The present invention relates to the field of digital education, and in particular to a method and terminal for constructing a student user portrait based on multimodal data fusion. Background Art

[0002] With the continuous development of educational informatization and artificial intelligence, student user profiling technology has gradually become an important support for personalized education recommendations, intelligent teaching assistance, and educational management decision-making. Existing student user profiling methods mainly collect student learning behavior data, such as course duration, homework completion, and test scores, and perform simple statistical analysis to infer students' learning abilities, interests, and potential problem areas. These methods can reflect students' learning status to a certain extent, but they also have significant limitations.

[0003] First, traditional student portraits often rely on single-modal data (such as learning behavior data) and fail to fully integrate multi-dimensional information about students' social behavior, interests, and hobbies. This results in one-sided portrait content and makes it difficult to fully portray students' personality traits and potential needs. Secondly, existing technologies often use static feature extraction and simple classification modeling, lacking a dynamic fusion mechanism for the importance of different modal features and data reliability. This can easily lead to information redundancy or missing key features, reducing the accuracy and interpretability of the portrait. In addition, in the process of student portrait modeling, existing methods generally ignore the in-depth analysis of students' knowledge mastery structure and learning path logic, and are unable to reveal potential loopholes and growth opportunities in students' learning process from the cognitive structure level, resulting in a large deviation between the portrait and the student's actual status. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and terminal for constructing a student user portrait based on multimodal data fusion, so as to solve the problem that a deep connection cannot be established with the user during the user portrait construction process.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for constructing a student user portrait based on multimodal data fusion, comprising the steps of: S1. Acquire multimodal data and extract corresponding feature information respectively; the multimodal data at least includes the user's learning behavior data, social behavior data, and interest and hobby data; S2. Calculating the fusion weight of the feature information of the multimodal data and generating a comprehensive feature vector; S3. Based on the comprehensive feature vector, student user portraits are constructed using clustering algorithms, hierarchical analysis methods, and knowledge graph analysis.

[0006] In order to solve the above technical problems, another technical solution adopted by the present invention is: A terminal for constructing a student user portrait based on multimodal data fusion includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a method for constructing a student user portrait based on multimodal data fusion are completed.

[0007] The present invention provides a method and terminal for constructing student user profiles based on multimodal data fusion. By extracting data features from three categories—learning behavior, social behavior, and hobbies—and integrating information from each modality using a fusion weighting mechanism, it ultimately forms a multidimensional, fine-grained student profile. Compared to traditional single-data-source modeling methods, this invention significantly improves the comprehensiveness, accuracy, and personalized adaptability of the profile, effectively supporting a variety of application scenarios such as personalized teaching recommendations, academic early warning, and matching interest-based activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 Schematic diagram of a flow chart of a method for constructing a student user portrait based on multimodal data fusion in an embodiment of the present invention; Figure 2 This is a flow chart of learning behavior data processing in an embodiment of the present invention; Figure 3 This is a schematic diagram of the knowledge gap feature extraction process in an embodiment of the present invention; Figure 4 This is a specific flow chart of a method for constructing a student user portrait based on multimodal data fusion in an embodiment of the present invention; Figure 5 A schematic diagram of a terminal for constructing a student user portrait based on multimodal data fusion in an embodiment of the present invention; Description of labels: 1. A terminal for constructing a student user profile based on multimodal data fusion; 2. Memory; 3. Processor. DETAILED DESCRIPTION

[0009] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0010] Please refer to Figure 1 and Figure 4 , a student user portrait construction method based on multimodal data fusion, comprising the steps of: S1. Acquire multimodal data and extract corresponding feature information respectively; the multimodal data at least includes the user's learning behavior data, social behavior data, and interest and hobby data; S2. Calculating the fusion weight of the feature information of the multimodal data and generating a comprehensive feature vector; S3. Based on the comprehensive feature vector, student user portraits are constructed using clustering algorithms, hierarchical analysis methods, and knowledge graph analysis.

[0011] As can be seen from the above description, the beneficial effects of this invention are: By extracting three types of data features—learning behavior, social behavior, and hobbies—and integrating these modal information using a fusion weighting mechanism, a multidimensional, fine-grained student portrait is ultimately formed. Compared to traditional single-data-source modeling methods, this invention significantly improves the comprehensiveness, accuracy, and personalized adaptability of the portrait, effectively supporting a variety of application scenarios such as personalized teaching recommendations, academic early warning, and matching interest-based activities.

[0012] In some embodiments, step S1 specifically includes: Based on learning behavior data, recurrent neural networks or long short-term memory networks are used to extract course completion progress and knowledge point mastery characteristics; For social behavior data, the PageRank algorithm is used to extract social activity and social influence features; For interest and hobby data, the latent Dirichlet allocation algorithm is used to extract the distribution characteristics of interest topics.

[0013] As can be seen from the preceding description, by selecting the optimal algorithms for different data types during the feature extraction phase, the accuracy and depth of feature extraction are improved. Recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) are used to capture time series information about learning behaviors, the PageRank algorithm mines social activity and influence, and the LDA algorithm refines potential topics of interest. This makes each modal feature more expressive, providing high-quality input for subsequent fusion and modeling, and avoiding the information loss problem associated with traditional statistical feature methods.

[0014] Specifically, the step S1 further includes: Combined with the subject knowledge graph, the knowledge gap features and learning logic deviation features in the learning behavior data are extracted.

[0015] As can be seen from the above description, introducing subject knowledge graphs into the process of extracting learning behavior data features can fully leverage the sequential dependencies and logical relationships between knowledge points to accurately depict the structure of students' knowledge mastery. Compared to methods based solely on surface behavior analysis, this invention, through graph mapping and structural feature extraction, can effectively identify students' deep cognitive vulnerabilities and learning path deviations, thereby accurately locating weak areas of learning.

[0016] Among them, the subject knowledge graph can be constructed using the following steps: Collect subject area data and use named entity recognition algorithms to extract subject-related entities; Extracting feature vectors from the entity and performing relationship sample representation based on the feature vectors; the feature vectors include at least one of text grammatical features, semantic features, entity structure features, and semantic similarity features; Construct a separator for the support vector machine and generate a linear decision function: ; Where x is the feature vector; w is the weight vector; b is the bias term; is a symbolic function that satisfies: When z>0, ;When z=0; , when z<0, ; The weight vector w and the bias term b are determined by minimizing the objective function, which is expressed as follows: ; Where k is the objective function; C is the penalty parameter; is a slack variable, indicating the tolerance for relationship misclassification; The feature vector is input into the linear decision function to generate deterministic relationship samples and uncertain relationship samples, and the association relationship between the deterministic relationship samples is output.

[0017] Using the uncertainty relationship sample as input to a graph convolutional neural network model; Node feature updates are performed based on the graph convolutional neural network model, as shown below: ; Where, is the feature representation of node i in layer l; is a node The set of neighbor nodes of and Node and neighbor nodes degree; It is The weight matrix of the layer, which is used to map neighbor node features to the new feature space; is the bias vector; is the activation function; Based on the updated node features, the potential relationship types between the uncertainty relationship samples are inferred, and the association relationships between the uncertainty relationship samples are output according to the potential relationship types.

[0018] Construct a subject knowledge graph with the entities as nodes and the association relationships as edges.

[0019] In the above steps, SVMs are first used to perform preliminary data classification. Based on rich, manually designed features, such as text grammatical and semantic features (lexical features, part-of-speech features, and syntactic structure features), knowledge graph structural features (entity degrees and shortest path features), and semantic similarity features (word vector similarity and topic model features), SVMs can quickly classify data and screen for relationships with high certainty. For samples that are difficult to identify, they are then input into the GCN model. GCNs automatically learn features from the data using the initial node features (entity attribute features and text description features) as well as graph structural features (adjacency matrix and degree matrix), enabling in-depth analysis of complex graph data and uncovering potential relationships.

[0020] Preferably, please refer to Figure 2 and Figure 3 , the combination of subject knowledge graph and extraction of knowledge gap features and learning logic deviation features in the learning behavior data specifically includes: Collect subject data, apply named entity recognition algorithms to identify subject entities, and use relationship extraction algorithms to construct a subject knowledge graph with entities as nodes and relationships as edges; Cleaning and structuring the learning behavior data, and establishing a correspondence between learning records and nodes in the subject knowledge graph; Analyze students' learning paths in the subject knowledge graph based on a graph traversal algorithm and extract sequential features of the learning paths; Based on the subject knowledge graph and the sequential characteristics of the learning path, the learning integrity of the students in each knowledge chain is determined, and the knowledge gap characteristics and learning logic deviation characteristics are identified.

[0021] As can be seen from the above description, a systematic process for constructing and extracting subject knowledge graphs has been proposed. This involves multiple steps, including entity recognition through NER, relationship extraction to establish a node relationship graph, behavioral data association, and learning path analysis, to ensure that learning behavior data is accurately mapped to the knowledge graph structure. Sequential feature extraction of learning paths and chain integrity verification can comprehensively quantify the cognitive structure of students' knowledge networks and identify potential learning breakpoints.

[0022] The process of acquiring the knowledge gap characteristics includes: Based on the subject knowledge graph and the sequential characteristics of the learning path, the accessibility score S(n) of each knowledge point n is calculated, where: ; Where N(n) is the set of all predecessor nodes of knowledge point n; W(n,m) is the dependency weight of predecessor node m to node n; A(m) is the mastery level of predecessor node m; D(n) is the out-degree of node n; When the reachability score S(n) is detected to be lower than the preset mastery threshold τ, node n is marked as a knowledge gap.

[0023] As can be seen from the above description, by defining the knowledge point accessibility score S(n) and combining it with the mastery level and dependency strength of the predecessor nodes, a quantitative assessment of students' knowledge mastery is achieved. By detecting whether S(n) is below the preset threshold τ, knowledge gaps can be automatically identified, significantly improving the granularity and automation of knowledge mastery assessment and avoiding the one-sided evaluation problems of traditional assessments based on single assignments or exam results.

[0024] The process of acquiring the learning logic deviation feature includes: Based on the subject knowledge graph and the learning path sequence characteristics, the learning logic deviation index LPI is calculated, where: ; Where T(n) is the timestamp when the student completes the learning of node n; I is an indicator function, which takes the value of 1 when the learning path sequence violates the logic in the subject knowledge graph, and 0 otherwise; V is the set of nodes in the knowledge graph.

[0025] As can be seen from the above description, by introducing the Learning Logic Deviation Index (LPI), the system can automatically identify whether the learning sequence is reasonable, quantitatively reflecting the degree to which learning behavior violates the knowledge graph logic. Compared to traditional methods that rely solely on manual interpretation of learning logs, the LPI metric enables automated and quantitative testing of the rationality of learning paths, greatly improving the interpretability of learning process data and the efficiency of problem discovery.

[0026] In some embodiments, step S2 further includes: Fusion weight W m Calculate according to the following formula: ; Where α is the scene sensitivity factor, I m is the modal importance coefficient, R m is the modal reliability coefficient.

[0027] It is understandable that in the multimodal data fusion module, in order to solve the problem that the traditional fixed fusion weight strategy cannot adapt to different scenarios and data fluctuations, the solution introduces a dynamic weight adaptive algorithm. The fusion mechanism is not only based on the intrinsic statistical characteristics of each modal data, but also takes into account the task importance of each modality in different application scenarios, thereby realizing the intelligent regulation of the multimodal feature fusion strategy. In the implementation process, the reliability of each type of modal data is first evaluated. The learning behavior data focuses on the stability of the time series, which is manifested in the stability of the frequency and order of students' learning of each knowledge point; the social behavior data evaluates the variance of its interaction frequency to measure its behavioral consistency and information integrity; the interest preference data can be measured by the activity participation distribution entropy to evaluate the consistency of its interest expression. The reliability coefficient of each modality obtained in the above way is recorded as R m , which can objectively reflect the availability and credibility of the data itself. At the same time, in the application process of the solution, the importance coefficient of the mode is preset according to the scenario requirements. m For example, in precision teaching tasks, the importance of learning behavior data is much higher than that of social behavior data, while in campus community recommendation tasks, the opposite may be true.

[0028] Among them, α∈[0,1] is the scene sensitivity factor, which determines the emphasis of the fusion strategy on the importance of the preset and the reliability of the data. When α approaches 1, the fusion weight is more biased towards the task prior, which is suitable for situations where there is a clear optimization goal for the scene; when α is small, the fusion is more dependent on the quality and characteristics of the data itself, which is suitable for data-driven tasks. The final generated weight W m Directly participating in the fusion of multimodal features, they are not only used as the linear combination coefficients of the feature vectors of each modality in the weighted sum fusion method, but also serve as the initial fusion weights in the multi-layer perceptron neural network. This enables the deep fusion model to have a good distribution of modal importance at the beginning of training, thereby improving the model's convergence speed and fusion effect. This weight generation mechanism not only strengthens the efficiency of utilizing the complementarity between modalities, but also significantly improves the discriminative ability of fused features in portrait construction, avoiding overall portrait distortion caused by fluctuations in the quality of a single modality, and overall ensuring the accuracy and robustness of student portraits.

[0029] From the above description, it can be seen that compared with traditional fixed weight fusion, this embodiment greatly improves the effectiveness and adaptability of fusion features in different educational application scenarios.

[0030] In some embodiments, step S3 specifically includes: Based on the fused comprehensive feature vector, the K-means algorithm is used to perform preliminary clustering of the student group to form basic portrait categories; For each basic portrait category, the importance weight of each portrait dimension is determined by combining the hierarchical analysis method, and multi-dimensional fine-grained portrait modeling is performed on the students in the group based on the importance weight; In the fine-grained portrait modeling process, graph neural networks are used to extract features from students' knowledge mastery maps, supplement cognitive structure features, and optimize the final student user portrait.

[0031] As can be seen from the above description, a modeling approach based on the integration of K-means clustering, the Analytic Hierarchy Process (AHP), and a Graph Neural Network (GNN) is proposed to form a multi-level, multi-dimensional student profiling system. Preliminary clustering ensures basic grouping accuracy, while AHP weight optimization enhances fine-grained differentiation, and the GNN structure extracts and refines cognitive features. This modeling approach balances group commonalities with individual differences, improving the explanatory power and applicability of the profiling.

[0032] Specifically, based on the fused comprehensive feature vector, a highly interpretable and high-resolution student user portrait system is constructed through a mechanism combining hierarchical clustering with the analytic hierarchy process (AHP). The specific steps are as follows: First, the comprehensive feature vector generated by the data fusion module is used as input for a first-level clustering operation. Using the K-means algorithm, the distribution density of the fused features in high-dimensional space is iteratively calculated and segmented into several basic student groups. Each basic group exhibits consistent characteristics across key dimensions such as learning behavior, social behavior, and interest preferences, thereby initially forming basic profile categories with common characteristics, such as "active learner" and "socially active."

[0033] After completing the preliminary clustering, the modal distribution characteristics of the cluster centroid vector are further analyzed and combined with the dynamic fusion weight W determined in the previous fusion step. m Alignment is performed to ensure that the preliminary clustering results are consistent with the importance distribution of different modal features, avoiding image distortion caused by fusion bias.

[0034] Then, for each basic portrait category, we enter the secondary weight modeling process. Using the analytic hierarchy process (AHP), we construct a pairwise judgment matrix: ; where element a ij represents the importance ratio of the i-th portrait dimension to the j-th portrait dimension; after consistency test, the weight vector W = [w1, w2, ..., w n ] and calibrate the weight vector. The calibration method is: ; in Represents the modality fusion weight corresponding to the i-th dimension feature, It is the final dimension weight after calibration, ensuring that the weight under the hierarchical structure can reflect the real data contribution and scene importance between modalities. In the three-level feature refinement stage, based on the association between students' learning behavior and knowledge graph, the "knowledge mastery graph" is further introduced as a structured feature source, and the mastery, dependency and path structure of each knowledge point node in the graph neural network (GNN) are encoded to generate a feature set. , the graph structure indicators such as the number of knowledge gaps, learning path deviations and cognitive jumps are integrated into the final portrait, further expanding the depth and structural complexity of the student portrait.

[0035] Through this three-layer nested mechanism, we can achieve a clear distinction between portrait categories while retaining the structured representation of individual differences. The constructed student portraits are both highly goal-oriented and have the ability to be continuously optimized in dynamic educational scenarios, thus providing a solid model foundation for subsequent portrait update modules.

[0036] In the profile construction step, a combination of hierarchical clustering and the analytic hierarchy process (AHP) is employed. First, a hierarchical clustering algorithm is used to initially categorize students into different hierarchical groups. Then, the AHP method is used to determine the weights of various profile dimensions (such as learning style, social orientation, and interests and specialties). By comprehensively considering the characteristics of each dimension, a student user profile that better meets actual needs is constructed. This method fully considers the differences in the importance of each dimension during the clustering process, making the constructed profile more targeted.

[0037] In some embodiments, step S3 further includes step S4: A reinforcement learning algorithm based on a deep Q network uses reward function feedback to dynamically optimize the portrait construction process and update student user portraits; the reward function is calculated based on the performance improvement rate, social activity change rate and prediction error.

[0038] As can be seen from the above description, by introducing a Deep Q-Network (DQN) reinforcement learning mechanism during the portrait optimization phase, the portrait modeling strategy is dynamically adjusted based on the reward function, enabling the adaptive evolution of the portrait as the student changes. Compared to traditional periodic manual updates, this invention can respond in real time to changes in learning outcomes, social activity, and prediction errors, ensuring simultaneous improvement in the timeliness and accuracy of the portrait.

[0039] Specifically, in the optimization and updating phase, in order to achieve adaptive adjustment of the portrait construction to the educational goal, a closed-loop feedback mechanism with deep reinforcement learning as the core is constructed, embedded in the aforementioned multimodal fusion strategy and hierarchical portrait modeling logic, forming a complete dynamic update system. This mechanism is based on the core dimensional state of the student portrait and defines the state space as ,in Represents the structural dimensions in the portrait, such as learning style, number of knowledge gaps, and social tendency index, which are derived from the knowledge structure and social influence characteristics modeled by the graph neural network. Represents educational effect indicators associated with the portrait, such as the rate of change of grades and activity participation. The action space is coupled with the aforementioned fusion and modeling modules, and includes the dynamic adjustment behavior of the model parameters. (such as K-means clustering threshold, GCN depth), fusion weight reset behavior , and strategic behaviors based on structural changes triggering knowledge graph reconstruction .

[0040] During the state transfer process, the deep Q network is used for strategy optimization. The core of this is to define a reward function that is highly sensitive to the educational goal. By mapping the direct impact of each update behavior on the educational results, a learnable feedback mechanism is formed. The reward function is defined as: ; Among them, ΔScore represents the student's performance improvement rate in phased courses or standardized assessments, directly quantifies the knowledge mastery effect after the learning path adjustment, and is mapped to the learning ability dimension in the portrait; ΔActivity represents the rate of change of students' interactive behavior in campus or virtual communities, reflecting the external behavioral response brought about by the social dimension after the strategy update; Loss is the error value of the current portrait in predicting learning status or interest behavior, and is calculated based on the sum of the squares of the difference between the portrait's true label and the model output.

[0041] The weight parameters β1, β2, and β3 are set according to different educational scenarios. For example, β1 is enhanced in personalized teaching dominated by academic improvement, while β2 is increased in activity recommendations aimed at quality development, so that the reinforcement learning strategy has a clear goal orientation in different scenarios.

[0042] By constructing a cumulative experience pool of state-action-reward triples (s, a, r) ​​and combining it with multimodal fusion weights and layered portrait weights The DQN training process updates the action value function Q(s,a) in each round of policy iteration and finally outputs the optimal action strategy: ; Its role is to drive the dynamic reconstruction of subsequent portraits. Reinforcement learning continuously drives the structural evolution of the portrait throughout its lifecycle, enabling it to maintain an optimal matching state in scenarios where multimodal data changes dynamically and educational goals shift continuously, thereby achieving the optimal balance between timeliness, accuracy, and goal consistency.

[0043] Specifically, during the optimization and update step, a reinforcement learning algorithm is used to optimize the profiling model. The profiling process is considered a decision-making process, with the model output (i.e., the constructed student profile) serving as the decision outcome. A reward function is established to assign rewards or penalties based on metrics such as the profile's accuracy and completeness. The model continuously adjusts its parameters based on reward feedback to improve its ability to construct high-quality student profiles. Furthermore, during the update process, data version management technology is employed to record the data and model status of each update, facilitating backtracking and comparative analysis, further optimizing the profiling process.

[0044] Please refer to Figure 5 A student user portrait construction terminal 1 based on multimodal data fusion includes a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, it completes the steps in a student user portrait construction method based on multimodal data fusion.

[0045] In some embodiments, see Figure 4 , the application process of the above method is as follows: In the personalized tutoring scenario, the solution begins with multimodal data fusion. This involves collecting student learning behavior data, such as online course duration, course completion progress, and homework completion; social behavior data, such as frequency of communication within study groups and active participation in discussions; and interest data, such as willingness to participate in enrichment activities across various disciplines. For example, student Xiao Li's learning behavior data reveals a high error rate in math courses, but a high error rate in homework assignments. Social behavior data indicates active participation in study groups, and interest data reveals a strong interest in math competitions. Next, feature extraction is performed. Using deep learning's RNN / LSTM algorithms, Xiao Li's learning behavior data reveals a lack of logical connection in his mathematical understanding of function concepts. Combined with knowledge graph technology, this analysis reveals a lack of understanding of the relationship between functions and equations within the functional knowledge system. Using the PageRank algorithm to analyze his social behavior data, Xiao Li demonstrates a strong influence by actively sharing ideas within the study group. The LDA algorithm is then used to extract potential interest in function optimization from the text describing his math competition participation. A profile was then constructed. Based on a clustering algorithm, Xiao Li was categorized as someone who "has learning difficulties in specific subjects, but is socially active and interested in expanding their discipline." Hierarchical clustering and the AHP method were used to determine the weights for dimensions such as learning style, social orientation, and interests and strengths. A detailed student profile was constructed, highlighting Xiao Li's weaknesses in function learning, his active participation in learning exchanges, and his preference for math competitions and function optimization. Finally, in the optimization and update phase, an online learning algorithm was used to adjust the profile in real time based on Xiao Li's subsequent learning data. If Xiao Li's accuracy in assignments related to functions and equations improves in the new math course, the knowledge mastery characteristics will be updated promptly. When there are new math competition participation records, the interests and strengths section of the profile will be optimized through reinforcement learning. Based on the profile, the teacher will develop a personalized learning plan for Xiao Li and regularly update the profile based on new data to continuously optimize the learning and tutoring strategy.

[0046] In the context of organizing campus club activities, the solution first integrates multimodal data, collecting data on students' interests and hobbies in various fields such as music, sports, and art; social behavior data such as their roles in club activities and the frequency of interaction with club members; and learning behavior data on the amount of time available for club activities after school. For example, on one campus, interest data for most students showed a strong passion for music performance, social behavior data indicated a desire to make like-minded friends in the club, and learning behavior data indicated a certain amount of free time on weekends and after school. Subsequently, in the feature extraction phase, the LDA algorithm was used to extract specific interest topics such as pop music singing and classical instrument playing from students' descriptions of music performances. The PageRank algorithm was then used to analyze the distribution of students' activity and influence in the club's social network, confirming that some students possessed strong organizational and coordination skills within the club organization. The learning behavior data was then combined with knowledge graph analysis to clarify the availability of different students' extracurricular time. Next, profiles were constructed. A clustering algorithm was used to categorize students into groups characterized by their interest in music performance, a desire for social interaction, and the availability of extracurricular time for club activities. Hierarchical clustering and AHP were used to determine the weights for each profile dimension, creating student profiles tailored to campus club activities, detailing their interests, preferences, social needs, and time availability. During the optimization and update phase, student feedback on past club activities was collected in real time. The profiles were updated using online learning algorithms. Through reinforcement learning, the profile models were optimized based on indicators such as participation rate and satisfaction. Based on these profiles, school clubs organized activities such as pop music competitions and classical instrument recitals, and included interactive sessions to promote student social interaction. Following the activities, data version management techniques were used for comparative analysis to further optimize the organizational structure and content of subsequent club activities.

[0047] In summary, the present invention provides a method and terminal for constructing a student user portrait based on multimodal data fusion, which uses a variety of advanced algorithms in the feature extraction stage, significantly improving the accuracy and depth of feature extraction. For learning behavior data, the use of RNN / LSTM in deep learning and combined with knowledge graph technology can accurately capture students' long-term mastery of knowledge points and the mastery of the knowledge system's structural characteristics, and effectively analyze learning paths and knowledge gaps. For social behavior data, the PageRank algorithm accurately extracts social influence features and quantifies students' status in social circles. Interest and hobby data uses the LDA algorithm to mine potential interest topics and enrich interest preference types. These sophisticated feature extraction methods provide a high-quality data foundation for subsequent data fusion and portrait construction, so that the constructed student user portraits can better reflect the true characteristics of students.

[0048] Using a variety of models, including clustering algorithms, neural networks, and generative adversarial networks (GANs), to construct student user profiles has significantly improved their quality and diversity. Clustering algorithms categorize students into different categories, forming a preliminary student profile framework, such as those with active learning or social activity. Neural networks can handle complex data relationships, further optimizing profiles. GANs, through adversarial training of generators and discriminators, make the generated student profiles more realistic and increase their diversity. Furthermore, a combination of hierarchical clustering and the analytic hierarchy process (AHP) takes into account the varying importance of each profile dimension, creating more targeted and realistic student user profiles, providing strong support for personalized education.

[0049] During the optimization and updating phase, we employ technologies such as transfer learning, online learning, reinforcement learning, and data version management to ensure the timeliness and accuracy of student user profiles. Transfer learning technology can quickly construct high-quality profiles when used with data from new student groups, reducing training time and data requirements. Online learning algorithms such as stochastic gradient descent (SGD) update the profile construction model in real time, ensuring that the profiles reflect students' current status. Reinforcement learning algorithms continuously optimize model parameters through a reward-feedback mechanism, improving the ability to construct high-quality student profiles. Data version management technology records the data and model status of each update, facilitating backtracking and comparative analysis, further improving the profile construction process and ensuring that student user profiles always accurately reflect students' dynamic changes.

[0050] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for constructing student user portraits based on multimodal data fusion, characterized by: Including steps: S1. Acquire multimodal data and extract corresponding feature information respectively; the multimodal data at least includes the user's learning behavior data, social behavior data, and interest and hobby data; S2. Calculating the fusion weight of the feature information of the multimodal data and generating a comprehensive feature vector; S3. Based on the comprehensive feature vector, student user portraits are constructed using clustering algorithms, hierarchical analysis methods, and knowledge graph analysis.

2. The method for constructing a student user profile based on multimodal data fusion according to claim 1, characterized in that: The step S1 specifically includes: Based on learning behavior data, recurrent neural networks or long short-term memory networks are used to extract course completion progress and knowledge point mastery characteristics; For social behavior data, the PageRank algorithm is used to extract social activity and social influence features; For interest and hobby data, the latent Dirichlet allocation algorithm is used to extract the distribution characteristics of interest topics.

3. The method for constructing a student user portrait based on multimodal data fusion according to claim 1 or 2, characterized in that: The step S1 further includes: Combined with the subject knowledge graph, the knowledge gap features and learning logic deviation features in the learning behavior data are extracted.

4. The method for constructing a student user profile based on multimodal data fusion according to claim 3, characterized in that: The extraction of knowledge gap features and learning logic deviation features from the learning behavior data by combining the subject knowledge graph specifically includes: Collect subject data, apply named entity recognition algorithms to identify subject entities, and use relationship extraction algorithms to construct a subject knowledge graph with entities as nodes and relationships as edges; Cleaning and structuring the learning behavior data, and establishing a correspondence between learning records and nodes in the subject knowledge graph; Analyze students' learning paths in the subject knowledge graph based on a graph traversal algorithm and extract sequential features of the learning paths; Based on the subject knowledge graph and the sequential characteristics of the learning path, the learning integrity of the students in each knowledge chain is determined, and the knowledge gap characteristics and learning logic deviation characteristics are identified.

5. The method for constructing a student user portrait based on multimodal data fusion according to claim 4, characterized in that: The process of obtaining the knowledge gap feature includes: Based on the subject knowledge graph and the sequential characteristics of the learning path, the accessibility score S(n) of each knowledge point n is calculated, where: ； Where N(n) is the set of all predecessor nodes of knowledge point n; W(n,m) is the dependency weight of predecessor node m to node n; A(m) is the mastery level of predecessor node m; D(n) is the out-degree of node n; When the reachability score S(n) is detected to be lower than the preset mastery threshold τ, node n is marked as a knowledge gap.

6. The method for constructing a student user profile based on multimodal data fusion according to claim 4, characterized in that: The process of acquiring the learning logic deviation feature includes: Based on the subject knowledge graph and the learning path sequence characteristics, the learning logic deviation index LPI is calculated, where: ； Where T(n) is the timestamp when the student completes the learning of node n; I is an indicator function, which takes the value of 1 when the learning path sequence violates the logic in the subject knowledge graph, and 0 otherwise; V is the set of nodes in the knowledge graph.

7. The method for constructing a student user profile based on multimodal data fusion according to claim 1, characterized in that: The step S2 further includes: Fusion weight W m Calculate according to the following formula: ； Where α is the scene sensitivity factor, I m is the modal importance coefficient, R m is the modal reliability coefficient.

8. The method for constructing a student user profile based on multimodal data fusion according to claim 1, characterized in that: The step S3 specifically includes: Based on the fused comprehensive feature vector, the K-means algorithm is used to perform preliminary clustering of the student group to form basic portrait categories; For each basic portrait category, the importance weight of each portrait dimension is determined by combining the hierarchical analysis method, and multi-dimensional fine-grained portrait modeling is performed on the students in the group based on the importance weight; In the fine-grained portrait modeling process, graph neural networks are used to extract features from students' knowledge mastery maps, supplement cognitive structure features, and optimize the final student user portrait.

9. The method for constructing a student user profile based on multimodal data fusion according to claim 1, characterized in that: The step S3 further includes a step S4: A reinforcement learning algorithm based on a deep Q network uses reward function feedback to dynamically optimize the portrait construction process and update student user portraits; the reward function is calculated based on the performance improvement rate, social activity change rate and prediction error.

10. A student user portrait construction terminal based on multimodal data fusion, characterized by: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it completes the steps in the method for constructing a student user portrait based on multimodal data fusion as described in any one of claims 1 to 9.

Images