Multi-modal learning resource intelligent recommendation method based on AI large model

Abstract

The invention discloses a multi-modal learning resource intelligent recommendation method based on an AI large model, and relates to the technical field of educational information, and the method comprises the steps: collecting multi-modal learning resources, carrying out the semantic feature extraction and vectorization representation, and obtaining a structured resource set; constructing a knowledge mastering state vector based on the resource set, and determining a knowledge hole set; virtual nodes are generated according to the knowledge holes, and an enhanced knowledge graph is constructed; calculating semantic path correlation and a path comprehensive score in the enhanced knowledge graph, and screening a candidate resource set; calculating a personalized score based on the candidate resource set, and generating a final recommendation list; and acquiring feedback data of a learner to obtain a learning income index, performing node and edge level attribution analysis, and performing enhanced updating and weight normalization processing on the knowledge graph according to an increment contribution weight to form a new knowledge graph. According to the invention, through knowledge mastering state modeling and map enhancement, automatic identification and dynamic completion of knowledge holes of learners are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of educational information technology, and particularly relates to a multi-modal learning resource intelligent recommendation method based on an AI large model. BACKGROUND

[0002] With the rapid development of educational informatization and artificial intelligence technology, the types and quantities of learning resources are exponentially increasing, covering various modalities such as text, image, audio and video. The fusion analysis of multi-modal content has become an important research direction of intelligent education recommendation systems. In recent years, semantic understanding and knowledge graph technology have been widely applied in educational resource management to describe the association between knowledge points and the relationship of learning paths.

[0003] However, the existing methods still have two limitations: first, the resource feature extraction method is still mainly based on local feature fusion or shallow semantic matching, lacking unified representation of cross-modal semantics, and unable to realize deep understanding and association expression of multi-source learning content, resulting in limited recommendation accuracy; second, the learner modeling is mainly based on static labels or historical behavior aggregation, and cannot realize the identification and structured completion of knowledge blind spots through dynamic modeling of knowledge state and graph enhancement, making it difficult to realize adaptive optimization of learning paths. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a multi-modal learning resource intelligent recommendation method based on an AI large model to solve the problems of non-uniform semantic feature extraction and inability to realize identification and structured completion of knowledge blind spots in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical solutions: The present application provides a multi-modal learning resource intelligent recommendation method based on an AI large model, which comprises: collecting multi-modal learning resources, performing semantic feature extraction and vectorization representation, and obtaining a structured resource set; Based on the structured resource set, a knowledge mastery state vector is constructed, and a knowledge hollow set is determined by the knowledge mastery state vector; According to the knowledge hollow set, an enhanced knowledge graph and a virtual node are obtained; According to the virtual node, the semantic path correlation between the enhanced knowledge graph and the structured resource set is calculated, the path comprehensive score is obtained, the structured resource set is sorted by the path comprehensive score, and a candidate resource set is obtained; Based on the candidate resource set, a personalized score is calculated, the candidate resource set is sorted by the personalized score, and a final recommendation list is obtained; Based on the final recommendation list, collect interactive feedback data, obtain learning benefit indicators, perform node and edge-level attribution, and calculate incremental contribution weights. Based on incremental contribution weights, the enhanced knowledge graph is strengthened and updated. The aggregated value and stability of virtual nodes are calculated, weight normalization and structural sparsity are performed, and a new knowledge graph and parameter configuration are obtained.

[0007] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on AI large models described in this invention, the specific steps for collecting multimodal learning resources, extracting semantic features and vectorizing them to obtain a structured resource set are as follows. Collect multimodal learning resources, extract semantic features and perform unified vectorization representation to obtain semantic vectors of learning resources and semantic vectors of knowledge points; Calculate the similarity between the semantic vectors of learning resources and the semantic vectors of knowledge points, establish a mapping relationship between learning resources and knowledge points based on the similarity, and obtain a structured resource set.

[0008] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on a large AI model described in this invention, the specific steps for constructing a knowledge mastery state vector based on a structured resource set and determining the knowledge gap set through the knowledge mastery state vector are as follows. Based on a structured resource set, learning behavior data is read, and a comprehensive learning performance score is calculated based on the learning behavior data. The learning performance scores of each knowledge point are weighted and summarized to obtain a knowledge mastery status vector. When the mastery of a knowledge point is lower than the preset mastery threshold, the knowledge point is identified as a knowledge gap. All knowledge gaps are combined to obtain a knowledge gap set.

[0009] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on a large AI model described in this invention, the specific steps for obtaining the enhanced knowledge graph and virtual nodes based on the knowledge gap set are as follows: Based on the set of knowledge gaps, each knowledge gap and its corresponding upstream and downstream knowledge nodes are semantically reasoned using the BERT pre-trained language model and CLIP model to obtain virtual nodes; Insert virtual nodes into the existing knowledge graph to obtain an enhanced knowledge graph.

[0010] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on AI large models described in this invention, the steps of calculating the semantic path relevance of virtual nodes to the structured resource set in the enhanced knowledge graph, obtaining a comprehensive path score, and ranking the structured resource set according to the comprehensive path score to obtain a candidate resource set are as follows: Calculate the path correlation between virtual nodes and structured resource sets, collect learners' dwell time, completion rate, assessment scores and feedback information, and calculate resource quality; The resource quality and path relevance are weighted and integrated to obtain a comprehensive path score. The structured resource set is then ranked according to the comprehensive path score, and a candidate resource set is selected from the top-ranked learning resources.

[0011] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on a large AI model described in this invention, the steps of calculating personalized scores based on a candidate resource set, sorting the candidate resource set according to the personalized scores, and obtaining the final recommendation list are as follows: Difficulty matching, modality preference and learning objective consistency analysis are performed on the candidate resource set to obtain analysis scores. Personalized scores are calculated by combining the analysis scores with the path comprehensive scores. The candidate resource set is sorted according to personalized scores, and the top-ranked candidate resources are selected to generate the final recommendation list.

[0012] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on a large AI model described in this invention, the specific steps for collecting interactive feedback data and obtaining learning benefit indicators based on the final recommendation list are as follows: Collect learners' clicks, dwell time, completion rate, and learning assessment scores on the final recommendation list to obtain interactive feedback data; The knowledge gain is calculated by analyzing the learning assessment scores before and after learning from the interactive feedback data, and the learning efficiency is calculated by combining the knowledge gain with the duration of study. By weighting and integrating knowledge gain and learning efficiency, a learning benefit metric is obtained.

[0013] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on a large AI model described in this invention, the specific steps for performing node and edge-level attribution and calculating incremental contribution weights are as follows: The learner's interactive feedback in the final recommended resource is traced along the path of the final recommended resource in the enhanced knowledge graph, mapped to the corresponding knowledge point nodes and the connection relationships between them, and the mapping of the path chain is obtained. Based on the mapping of learning benefit indicators and path chains, all knowledge point nodes and their associated edges in the enhanced knowledge graph are attributed, and the incremental contribution weights are calculated.

[0014] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on AI large models described in this invention, the steps of strengthening and updating the enhanced knowledge graph based on incremental contribution weights and calculating the aggregated revenue value and stability of virtual nodes are as follows. Based on the incremental contribution weights, edge weight reinforcement and update are performed on each edge in the enhanced knowledge graph to obtain the new edge weight matrix and the updated knowledge graph; Calculate the revenue aggregation value and stability index of each virtual node. When the revenue aggregation value and stability index meet the preset threshold requirements, the virtual node is solidified as a formal knowledge node; otherwise, it is marked as a reclaimed node and its weak connection edges are cleared.

[0015] As a preferred embodiment of the intelligent recommendation method for multimodal learning resources based on large AI models described in this invention, the specific steps for performing weight normalization and structure sparsification to obtain a new knowledge graph and parameter configuration are as follows: Perform weight normalization and structure sparsification on the updated knowledge graph; Based on the new edge weight matrix, learning benefit index and personalized score, the residual is minimized to obtain recommendation parameters, adaptive calibration is performed, and a new knowledge graph and parameter configuration are obtained.

[0016] The beneficial effects of this invention are as follows: by employing large AI models such as BERT and CLIP to extract semantic features and uniformly vectorize multimodal learning resources, the fusion and association of learning content from different modalities in the same semantic space are realized; by constructing learner knowledge mastery state vectors and generating enhanced knowledge graphs, the automatic identification of learners' knowledge blind spots and dynamic completion of knowledge structures are realized; by combining path relevance, resource quality, and learning feedback for adaptive weight optimization and knowledge graph reinforcement updates, the continuous evolution of the recommendation mechanism and the stable optimization of recommendation results are realized. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Fig. 1 This is a flowchart of an intelligent recommendation method for multimodal learning resources based on a large AI model.

[0019] Fig. 2 A flowchart for constructing knowledge mastery status and identifying knowledge gaps.

[0020] Fig. 3 A flowchart for building an enhanced knowledge graph.

[0021] Fig. 4 A flowchart for sorting candidate resources and generating a recommendation list. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figs. 1-4 This is one embodiment of the present invention, which provides an intelligent recommendation method for multimodal learning resources based on a large AI model, including the following steps: S1. Collect multimodal learning resources, extract semantic features and vectorize them to obtain a structured resource set.

[0026] We collect multimodal learning resources from learning platforms, including text, images, audio, and video, such as textbooks, question sets, video explanations, and audio explanations. During collection, we associate these resources with their metadata, including resource type (e.g., text, image, audio), resource ID, course / knowledge point tags, learning objectives, author information, and publication time.

[0027] For text resources, the BERT pre-trained language model is used to extract semantic features from each text segment. After word segmentation, the features are input into BERT to obtain the context-aware representation of each text segment. The context-aware representation is then used as the text semantic vector.

[0028] For image resources, the CLIP model is used. The images are input into the CLIP visual encoder to obtain the semantic representation of each image, which is used as the image semantic vector.

[0029] For audio resources, the audio resources are converted into text, and the text is input into the BERT pre-trained language model to obtain audio semantic vectors.

[0030] For video resources, keyframes are extracted from the video, and the CLIP model is applied to each keyframe for processing. The audio portion of the video is processed using the same steps as the audio resources to obtain a video semantic vector obtained by weighted combination of images and audio.

[0031] By projecting text semantic vectors, image semantic vectors, audio semantic vectors, and video semantic vectors into a shared vector space, resources of different modalities can interact, compare, and cluster in the same space.

[0032] Each multimodal learning resource is assigned a set of knowledge point tags, corresponding to the knowledge content covered by the learning resource. For each learning resource, a corresponding semantic vector is extracted. For each knowledge point related to the learning resource, a corresponding knowledge point semantic vector is extracted. To establish the relationship between knowledge points and learning resources, the cosine similarity between the semantic vectors of the learning resources and the knowledge points is calculated to obtain the similarity between them. By calculating the similarity of all learning resources and knowledge points, a structured resource set is obtained. The structured resource set includes detailed information for each learning resource, its corresponding knowledge point tags, learning objectives, and resource type. Each learning resource in the structured resource set is associated with multiple knowledge points, and each association has a similarity score, representing the strength of the relationship between the resource and the knowledge point.

[0033] The training steps for the BERT pre-trained language model are as follows: In the pre-training phase, BERT is trained using a large amount of unlabeled text data, employing two core tasks: Masked Language Model (MLM) and Next Sentence Prediction (NSP). In the MLM task, the model randomly masks some words in the input text and then predicts these masked words, aiming to teach the model bidirectional dependencies within the context. In the NSP task, the model learns to determine whether two sentences are adjacent in the original text, capturing the relationships between sentences. Through these two tasks, BERT learns rich contextual information and semantic representations. In the fine-tuning phase, BERT is trained on specific downstream tasks (such as text classification and question answering) to adapt it to the needs of specific tasks.

[0034] The CLIP model is trained using a large number of images and their corresponding text descriptions. The model consists of two main parts: an image encoder and a text encoder, which encode images and text using convolutional neural networks (CNNs) and Transformer architectures, respectively. The training objective is to maximize the similarity between matching images and text descriptions using a contrastive loss function, while minimizing the similarity between mismatched image-text pairs. Specifically, CLIP calculates the cosine similarity between images and text in a shared semantic space, and trains using positive and negative sample pairs of images and text, thereby aligning the semantic features of images and their descriptive texts within the same space.

[0035] S2. Based on the structured resource set, construct a knowledge mastery state vector, and determine the knowledge gap set through the knowledge mastery state vector.

[0036] Learner behavior data is obtained from the learning platform, including click data (which learning resources learners clicked on), dwell time (how long learners spent on each resource), completion data (whether learners watched the entire video or completed a learning module), and assessment scores (learners' scores on tests, quizzes, or assignments). A weighted average of click data, dwell time, completion data, and assessment scores is used to obtain a comprehensive learning performance score. A weighted average of the comprehensive learning performance scores for each knowledge point's related learning resources is then used to obtain the knowledge point mastery level. The mastery levels of all knowledge points are then aggregated to form a learner's knowledge mastery status vector.

[0037] By using a knowledge mastery state vector, knowledge gaps are identified. Specifically, if the mastery level of a certain knowledge point is lower than a preset mastery threshold, the learner is considered to have insufficient mastery of that knowledge point. Furthermore, by combining learning objectives and curriculum design, it is possible to identify which knowledge points are the foundation for subsequent learning. When learners have low mastery levels of certain key knowledge points, these knowledge points can be identified as knowledge gaps, i.e., knowledge points that learners need to focus on supplementing at the current learning stage. These identified unmastered knowledge points are then combined to obtain a set of knowledge gaps.

[0038] It should be noted that the mastery threshold is calculated by taking the mean and standard deviation to determine the mastery distribution of all learners across all knowledge points, and the mastery threshold is set at the 25th percentile of the mastery distribution to identify the knowledge points that have not been mastered.

[0039] S3. Based on the knowledge gap set, obtain the enhanced knowledge graph and virtual nodes.

[0040] Each knowledge gap in the knowledge gap set and its corresponding upstream and downstream knowledge nodes are taken as input and fed into the BERT pre-trained language model and CLIP model for semantic reasoning. This process helps to understand the semantic relationships between various knowledge points and generate virtual node information for these knowledge points. This includes node definition (providing a brief definition or description for each virtual node, explaining the core content or learning objective of the knowledge point it represents), contextual semantic description (providing a contextual description for each virtual node to help further understand the role, importance, and relevance of the knowledge point to other knowledge points), and knowledge connection relationship (generating the connection relationship between the virtual node and other related knowledge points based on the reasoning results, such as prerequisites, successors, etc.).

[0041] Each virtual node corresponds to a knowledge gap. These virtual nodes are inserted into the existing knowledge graph. Within the existing knowledge graph, a new node is inserted for each virtual node. Node attributes include the name, definition, description of the knowledge point, and its connections to other nodes. Based on the knowledge connections of the virtual nodes, the newly added edges describe the dependencies between the virtual nodes and existing knowledge points. By inserting virtual nodes and new edge relationships, a richer, enhanced knowledge graph is obtained. This enhanced knowledge graph reflects the knowledge points that learners have not yet mastered and clarifies the relationships between these knowledge points and other related knowledge points.

[0042] It should be noted that upstream and downstream knowledge nodes refer to other knowledge points related to the current knowledge point. Existing knowledge graphs are created by automatically extracting entities and relationships from large amounts of text data, and then combining this with machine learning models for reasoning and updating.

[0043] S4. Based on the virtual nodes, calculate the semantic path relevance with the structured resource set in the enhanced knowledge graph, obtain the comprehensive path score, rank the structured resource set according to the comprehensive path score, and obtain the candidate resource set.

[0044] The matching degree between virtual nodes and learning resources is evaluated by calculating the path relevance between virtual nodes and learning resources. Path relevance includes path reachability and resource semantic similarity. Path reachability is evaluated by calculating the shortest path between virtual nodes and learning resources, and resource semantic similarity is measured by calculating the similarity between the semantic vectors of virtual nodes and learning resources using cosine similarity. Path relevance is obtained by weighted sum of path reachability and resource semantic similarity.

[0045] To comprehensively evaluate the matching degree between each learning resource and virtual node, a comprehensive path score is calculated by combining path relevance and resource quality. Specifically, resource quality is assessed by calculating the proportion of learner dwell time on the resource to the total resource time, thus evaluating learner engagement. Secondly, the completion rate is calculated to reflect whether learners have fully learned the resource; a higher completion rate results in a higher quality score. Learner scores in resource-related assessments measure the resource's effectiveness in improving learners' knowledge acquisition. Finally, resource quality is further evaluated based on learner feedback, such as ratings and comments. The resource quality is obtained by weighted averaging of dwell time, completion rate, assessment scores, and learner feedback. The comprehensive path score is obtained by weighted averaging of resource quality and path relevance.

[0046] It should be noted that the weighting of path accessibility and resource semantic similarity in path relevance is based on learner behavioral data (such as learning outcomes, completion rate, and engagement). The weighting calculates the impact of path accessibility and semantic similarity on learning outcomes, and by comparing the effects of different weights, the weighting that optimizes learning effectiveness is selected. The weighting of dwell time, completion rate, assessment scores, and learner feedback in resource quality calculation is determined by analyzing the correlation between each indicator and learning outcomes in historical data, using correlation analysis methods, and assigning higher weights to the indicators most relevant to learning outcomes. The weighting of resource quality and path relevance in the overall path score calculation is determined by analyzing learner feedback on recommended resources in historical data, using statistical analysis to calculate the contribution of each indicator to learning outcomes, and thus determining its weight.

[0047] All learning resources are ranked based on their comprehensive path scores. Resources with higher scores indicate stronger semantic relevance to virtual nodes. These resources are then ranked from highest to lowest score, and a select set of top-ranked resources is chosen as a candidate resource set. For each candidate resource in the set, an interpretable knowledge path chain is generated, including the definition and contextual description of the starting virtual node, the semantic similarity and path reachability analysis results of the learning resource, and the relationship between the learning resource and related knowledge points (e.g., prerequisite and successor relationships).

[0048] S5. Based on the candidate resource set, perform difficulty matching, modality preference and learning objective consistency analysis, calculate personalized scores, sort the candidate resource set according to the personalized scores, and obtain the final recommendation list.

[0049] For each candidate resource in the candidate resource set, the difficulty of the content is matched with the learner's ability. The difficulty matching is calculated. Specifically, video tutorials, math problem sets, etc., can be assigned difficulty levels according to the complexity of their content. The learner's ability is usually measured by their historical learning records (such as test scores, mastered knowledge points, etc.). By comparing the learner's ability with the difficulty of the resource, a matching score can be obtained. The higher the matching score, the more suitable the difficulty of the resource is for the learner's ability. Conversely, the lower the matching score, the more suitable the difficulty is for the learner.

[0050] Modality preference matching is obtained by analyzing learners' historical learning data. Specifically, preference scores are calculated by weighting learners' interaction intensity with each modality resource (e.g., the time spent watching videos or reading text). Learners' modality preference scores are then matched with the modality types of candidate resources to obtain the modality preference matching degree for each resource. The higher the matching degree, the more consistent the resource is with the learner's modality preference.

[0051] Learning objective consistency is calculated by comparing the matching degree between learners' personalized learning objectives and the knowledge point tags of candidate resources. Specifically, learners' learning objectives are transformed into a set of specific knowledge points, representing the knowledge domains that learners expect to master. For each candidate resource, the similarity between the relevant knowledge points and the knowledge points in the learning objective is calculated using Jaccard similarity. Resources with higher similarity scores indicate that their content is more in line with learners' learning objectives, thus deriving a learning objective consistency score for that resource.

[0052] Calculate the personalized score for each candidate resource, as shown in the expression: ; in, Indicates candidate resources Personalized scores, Indicates candidate resources Difficulty matching, Indicates candidate resources Modal preference matching, Indicates candidate resources Consistency of learning objectives Indicates candidate resources The overall path score, This indicates the strength of the impact of the adjusted difficulty matching on the personalized score. This indicates the strength of the modal preference matching effect on personalized scores. This indicates the moderating effect of learning objective consistency on personalized scores. This indicates the strength of the influence of the overall score of the adjustment path on the personalized score.

[0053] It should be noted that, , , as well as It involves analyzing learners' historical behavioral data to calculate the correlation between different factors (e.g., difficulty matching, modality preference, learning goal consistency, path relevance) and learning outcomes (e.g., completion rate, learning progress). Then, based on the regression analysis results, it determines the degree of contribution of each factor to the final recommendation score, thereby obtaining their respective influence.

[0054] All candidate resources are sorted from highest to lowest based on their personalized scores, and the top-ranked candidate resources are selected as the final recommendation list.

[0055] S6. Based on the final recommendation list, collect interactive feedback data, obtain learning benefit indicators, perform node and edge-level attribution, and obtain incremental weight information.

[0056] The system collects interaction data for each learning resource in the final recommendation list, including click data, dwell time, completion rate, and assessment scores. The ratio of the assessment score after learning to the assessment score before learning is used as knowledge gain, and the ratio of knowledge gain to dwell time is used as learning efficiency. The learning benefit index is obtained by weighted averaging of knowledge gain and learning efficiency.

[0057] It should be noted that the weights of knowledge gain and learning efficiency in calculating learning benefits are determined by analyzing historical data and calculating the degree of influence of knowledge gain and learning efficiency on learning outcomes based on learners' feedback and interaction behavior, thereby determining the weight of each indicator.

[0058] Based on the mapping between learning benefit metrics and path chains, attribution analysis is performed at the node and edge levels. The contribution of each node (knowledge point) and each edge (dependency relationship between knowledge points) to the learning benefit is calculated, and the incremental contribution weight is obtained. The expression is as follows: ; in, Representing knowledge points Weighting of incremental contribution to learning gains Indicates in knowledge points Knowledge gain on Indicates in knowledge points Learning efficiency This represents a set of knowledge points.

[0059] S7. Based on incremental weight information, the enhanced knowledge graph is strengthened and updated, the revenue aggregation value and stability of virtual nodes are calculated, weight normalization and structural sparsity are performed, and a new knowledge graph and parameter configuration are obtained.

[0060] By using the incremental contribution weight information of each edge, edge weight reinforcement update is performed to obtain the updated edge weight matrix, expressed as: ; in, Indicates the first After the second iteration Update the weight values. Indicates the first During the next iteration, the edge The current weight value, This represents the incremental learning rate coefficient. Representing an edge The average incremental weight within the current time window, Represents the structural smoothness coefficient. Representing knowledge points The average of all outgoing edge weights. This means that the updated weights of each edge are pruned element-by-element to limit the result to a specific range. Within the range.

[0061] It should be noted that the incremental learning rate coefficient It is determined by analyzing the average convergence rate of edge weight changes in historical update records, selecting a fixed empirical value that strikes a balance between convergence stability and response sensitivity. Average incremental weight. It is achieved by counting edges within the current time window. The incremental contribution weights of the knowledge points at both ends are averaged. (Structural smoothness coefficient) It is determined by comparing the proportion of the fluctuation amplitude of the edge weights in each round with the average difference of the neighborhood during the historical evolution of the graph, and selecting an empirical value that can stabilize the convergence speed of the edge weight variance within a preset range. The preset range is the statistically stable interval of the edge weight variance during the historical evolution process, which is usually determined by the median ± standard deviation of the historical variance.

[0062] After the edge weights are updated, the aggregated revenue value of each virtual node on the connected edges is calculated. The aggregated revenue value is the sum of the incremental contribution weights of all connected edges. At the same time, the stability index of the virtual node is calculated, with the expression as follows: ; in, Represents virtual nodes Stability index This represents the average of all learning gain metrics associated with this virtual node. This represents the standard deviation of the learning gain metric associated with the virtual node. This represents the minimum value before dividing by zero.

[0063] A dual threshold judgment is performed based on the virtual node's revenue aggregation value and stability index. When the virtual node's revenue aggregation value is high and the stability index meets the threshold requirements, the virtual node is solidified as a formal knowledge node and its main connection edges are retained. When the virtual node's revenue aggregation value is too low or fluctuates greatly, it is marked as a reclaimed node and its weak connection edges are removed.

[0064] It should be noted that the threshold requirements include a cumulative return threshold and a stability threshold. The cumulative return threshold and stability threshold are obtained through statistical analysis of the distribution of node returns and learning stability in historical multi-round recommendations. First, a large amount of virtual node return and stability data in different learning cycles are collected, and the mean and standard deviation of their distribution are calculated. Then, quantiles that can effectively distinguish high-quality nodes from noisy nodes (e.g., the 75th percentile and the 25th percentile) are selected as the basis for threshold setting. Finally, the solidification threshold corresponding to the higher quantile and the recycling threshold corresponding to the lower quantile are determined.

[0065] To prevent the graph edge weights from becoming unbalanced after the update, the outgoing edge weights of each node are normalized. The temperature-controlled Softmax method is used for weight normalization, and the expression is: ; in, Representing knowledge points Points of knowledge The normalized weights, Representing knowledge points The set of all outgoing edges, Representing knowledge points Points of knowledge The unnormalized weights, This represents the temperature coefficient.

[0066] It should be noted that the temperature coefficient By multiple candidate groups Below, we statistically analyze knowledge path coverage and path consistency. Path coverage is defined as the ratio of the actual number of nodes involved in the recommended path to the total number of nodes in the graph, while path consistency is defined as the proportion of overlap between the recommended path and high-yield edges. Then, we plot the correlation between these two factors. The trend curve of change is used to select the point that balances coverage and consistency, which is then used as the final temperature coefficient. .

[0067] After normalization, the outgoing edge set is sparsified, retaining only the top few edges with the highest weights after normalization.

[0068] On the updated knowledge graph, the recommendation parameters (path comprehensive score fusion coefficient and personalized score fusion coefficient) are recalibrated based on the new edge weights, learning benefit indicators, and personalized scores. The expression is as follows: ; in, This represents the path comprehensive score fusion coefficient. This represents the personalized score fusion coefficient. Indicates learning benefits. This represents the overall score of the path. This represents the Huber loss function.

[0069] The updated edge weight matrix, node fixation results, and recommendation parameters are saved together as a new graph version, enabling the self-evolution of the knowledge graph.

[0070] This embodiment also provides a computer device applicable to the intelligent recommendation method for multimodal learning resources based on a large AI model, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent recommendation method for multimodal learning resources based on a large AI model as proposed in the above embodiment.

[0071] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0072] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the intelligent recommendation method for multimodal learning resources based on a large AI model as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0073] In summary, this invention achieves the fusion and association of learning content from different modalities in the same semantic space by employing large AI models such as BERT and CLIP to extract semantic features and uniformly vectorize multimodal learning resources; it achieves automatic identification of learners' knowledge blind spots and dynamic completion of knowledge structures by constructing learner knowledge mastery state vectors and generating enhanced knowledge graphs; and it achieves continuous evolution of the recommendation mechanism and stable optimization of recommendation results by combining path relevance, resource quality, and learning feedback for adaptive weight optimization and knowledge graph reinforcement updates.

[0074] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent recommendation of multimodal learning resources based on large AI models, characterized in that: include, Collect multimodal learning resources, extract semantic features and vectorize them to obtain a structured resource set; Based on the structured resource set, a knowledge mastery state vector is constructed, and the knowledge mastery state vector is used to determine the knowledge gap set; Based on the knowledge gap set, obtain an enhanced knowledge graph and virtual nodes; Based on virtual nodes, the semantic path relevance with the structured resource set is calculated in the enhanced knowledge graph to obtain the comprehensive path score. The structured resource set is then ranked according to the comprehensive path score to obtain the candidate resource set. Based on the candidate resource set, a personalized score is calculated, and the candidate resource set is sorted according to the personalized score to obtain the final recommendation list; Based on the final recommendation list, collect interactive feedback data, obtain learning benefit indicators, perform node and edge-level attribution, and calculate incremental contribution weights. Based on incremental contribution weights, the enhanced knowledge graph is strengthened and updated. The aggregated value and stability of virtual nodes are calculated, weight normalization and structural sparsity are performed, and a new knowledge graph and parameter configuration are obtained.

2. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 1, characterized in that: The specific steps for collecting multimodal learning resources, extracting semantic features and vectorizing them to obtain a structured resource set are as follows: Collect multimodal learning resources, extract semantic features and perform unified vectorization representation to obtain semantic vectors of learning resources and semantic vectors of knowledge points; Calculate the similarity between the semantic vectors of learning resources and the semantic vectors of knowledge points, establish a mapping relationship between learning resources and knowledge points based on the similarity, and obtain a structured resource set.

3. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 2, characterized in that: The process involves constructing a knowledge mastery state vector based on a structured resource set, and then using this vector to determine the set of knowledge gaps. The specific steps are as follows: Based on a structured resource set, learning behavior data is read, and a comprehensive learning performance score is calculated based on the learning behavior data. The learning performance scores of each knowledge point are weighted and summarized to obtain a knowledge mastery status vector. When the mastery of a knowledge point is lower than the preset mastery threshold, the knowledge point is identified as a knowledge gap. All knowledge gaps are combined to obtain a knowledge gap set.

4. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 3, characterized in that: The specific steps for obtaining the enhanced knowledge graph and virtual nodes based on the knowledge gap set are as follows: Based on the set of knowledge gaps, each knowledge gap and its corresponding upstream and downstream knowledge nodes are semantically reasoned using the BERT pre-trained language model and CLIP model to obtain virtual nodes; Insert virtual nodes into the existing knowledge graph to obtain an enhanced knowledge graph.

5. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 4, characterized in that: The steps involve calculating the semantic path relevance of virtual nodes to the structured resource set within the enhanced knowledge graph, obtaining a comprehensive path score, and then ranking the structured resource set based on the comprehensive path score to obtain a candidate resource set. Calculate the path correlation between virtual nodes and structured resource sets, collect learners' dwell time, completion rate, assessment scores and feedback information, and calculate resource quality; The resource quality and path relevance are weighted and integrated to obtain a comprehensive path score. The structured resource set is then ranked according to the comprehensive path score, and a candidate resource set is selected from the top-ranked learning resources.

6. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 5, characterized in that: The process involves calculating personalized scores based on the candidate resource set, sorting the candidate resource set using these personalized scores, and obtaining the final recommendation list. The specific steps are as follows: Difficulty matching, modality preference and learning objective consistency analysis are performed on the candidate resource set to obtain analysis scores. Personalized scores are calculated by combining the analysis scores with the path comprehensive scores. The candidate resource set is sorted according to personalized scores, and the top-ranked candidate resources are selected to generate the final recommendation list.

7. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 6, characterized in that: The specific steps for collecting interactive feedback data and obtaining learning benefit metrics based on the final recommendation list are as follows: Collect learners' clicks, dwell time, completion rate, and learning assessment scores on the final recommendation list to obtain interactive feedback data; The knowledge gain is calculated by analyzing the learning assessment scores before and after learning from the interactive feedback data, and the learning efficiency is calculated by combining the knowledge gain with the duration of study. By weighting and integrating knowledge gain and learning efficiency, a learning benefit metric is obtained.

8. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 7, characterized in that: The specific steps for performing node and edge-level attribution and calculating incremental contribution weights are as follows: The learner's interactive feedback in the final recommended resource is traced along the path of the final recommended resource in the enhanced knowledge graph, mapped to the corresponding knowledge point nodes and the connection relationships between them, and the mapping of the path chain is obtained. Based on the mapping of learning benefit indicators and path chains, all knowledge point nodes and their associated edges in the enhanced knowledge graph are attributed, and the incremental contribution weights are calculated.

9. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 8, characterized in that: The process of strengthening and updating the enhanced knowledge graph based on incremental contribution weights, and calculating the aggregated revenue and stability of virtual nodes, involves the following steps: Based on the incremental contribution weights, edge weight reinforcement and update are performed on each edge in the enhanced knowledge graph to obtain the new edge weight matrix and the updated knowledge graph; Calculate the revenue aggregation value and stability index of each virtual node. When the revenue aggregation value and stability index meet the preset threshold requirements, the virtual node is solidified as a formal knowledge node; otherwise, it is marked as a reclaimed node and its weak connection edges are cleared.

10. The intelligent recommendation method for multimodal learning resources based on a large AI model as described in claim 9, characterized in that: The specific steps for performing weight normalization and structure sparsification to obtain the new knowledge graph and parameter configuration are as follows: Perform weight normalization and structure sparsification on the updated knowledge graph; Based on the new edge weight matrix, learning benefit index and personalized score, the residual is minimized to obtain recommendation parameters, adaptive calibration is performed, and a new knowledge graph and parameter configuration are obtained.

Images