Learning resource recommendation method for intention-aware dual-contrast learning

Abstract

The invention provides a learning resource recommendation method for intention-aware double contrast learning, and relates to the technical field of sequence recommendation. The method comprises the following steps: introducing a mask language model (MLM) strategy, and using a historical learning resource sequence of a user to pre-train and finely adjust a Transform; a pre-trained Transform is used as a basic encoder, and long-term learning intention embedding of a user is captured in combination with an attention mechanism; constructing a learning resource map, and extracting short-term learning intention embedding of the user from pairwise learning resource conversion; a self-supervised double contrast learning strategy is designed, and modeling of long and short term learning intentions is enhanced; and adaptively fusing long and short term learning intentions, predicting a next learning resource, and constructing and optimizing an overall loss function. Through double-layer learning intention modeling and a self-supervised contrast learning strategy, long and short term learning intentions of a user are comprehensively captured, challenges of sparse learning behaviors and noise data are effectively coped, and personalized, accurate and robust learning resource recommendation is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of sequence recommendation, and in particular to a learning resource recommendation method, system, storage medium and electronic device for intention-aware double contrast learning. Background Art

[0002] With the rise of large-scale open online learning resource platforms, the demand for personalized learning resource recommendation systems is growing, in order to help users quickly locate the resources that best suit their learning progress and needs from massive learning resources.

[0003] In the related art, existing studies mainly regard learning resource recommendation as a sequential recommendation problem, aiming to predict the next learning resource target based on the historical learning resources registered by the user, and develop models based on recurrent neural networks (RNNs) or reinforcement learning (RL) to capture the user's learning interest. However, these methods do not fully explore the dependencies between learning resources in the modeling process, which is crucial for understanding user interests. The main difference between learning resource recommendation and traditional sequential recommendation lies in the importance of learning resource dependencies. This relationship is hidden in the user's behavior and has not been fully utilized to model their interest in the target learning resource. In fact, the user's motivation for choosing the next learning resource includes two aspects: one is the prerequisite relationship of learning resources, that is, a certain learning resource provides basic knowledge for another learning resource; the other is educational pursuit, that is, the user chooses the appropriate learning resource according to his or her goal. Although many sequential models can capture the user's interest, the model based on the next prediction paradigm fails to fully capture the learning resource dependencies existing in all learning sequences, resulting in suboptimal model learning parameters and poor recommendation effect.

[0004] In addition, it is not sufficient to rely solely on the user's learning sequence to infer learning interests, and it is necessary to mine more fine-grained learning resource dependencies to predict the next learning resource of interest. This omission makes the model unable to fully utilize the intrinsic connections between learning resources, affecting the accuracy and personalization of recommendations.

[0005] Finally, the interactions between users and learning resources are often sparse and noisy, which has a negative impact on the performance of learning resource recommendations. According to statistics, on online learning resource platforms such as Udemy, more than 80% of users have registered less than 10 learning resources, which shows that data sparsity is a major challenge for learning resource recommendation. In addition, users may mistakenly select learning resources that do not meet their learning goals, resulting in noise in the interaction data. This sparse and noisy characteristic poses a huge obstacle to accurately modeling user interest representations, and this factor is often overlooked in previous research on learning resource recommendation. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the deficiencies in the prior art, the present invention provides a learning resource recommendation method, system, storage medium and electronic device for intent-aware dual contrast learning, which solves the technical problem of providing users with personalized, biochemical, accurate and robust learning resource recommendations.

[0008] (II) Technical solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0010] A learning resource recommendation method for intention-aware double contrast learning is based on a preset learning resource recommendation network, and the method includes:

[0011] Get the historical learning resource sequence set of all users;

[0012] Based on the historical learning resource sequence set, a masked language model MLM strategy is used to pre-train the Transformer;

[0013] The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's long-term learning intention embedding from each historical learning resource sequence;

[0014] Based on the pairwise learning resource conversion of the historical learning resource sequence set, a learning resource graph is constructed, and the last historical learning resource of each historical learning resource sequence is used as a guide, combined with an attention mechanism, to extract the user's short-term learning intention embedding;

[0015] Randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss;

[0016] Retrieve similar historical learning resource sequences that have short-term similar learning intentions as each historical learning resource sequence, obtain the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss;

[0017] The long-term learning intention embedding and the short-term learning intention embedding are integrated to obtain the prediction result of the user's next learning resource to construct a cross entropy loss function of the main recommendation task;

[0018] Based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross-entropy loss functions, an overall loss function is constructed and optimized until the network converges, and the learning resource recommendation network with optimal parameters is used to predict the next learning resource of the target user.

[0019] Preferably, the method uses the pre-trained Transformer as a basic encoder and combines the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence; including:

[0020] Construct the inverse position matrix of each historical learning resource sequence;

[0021] Based on each historical learning resource sequence and its inverse position matrix, the pre-trained Transformer is used as the basic encoder to obtain the user's learning resource embedding;

[0022] Obtaining the user's session embedding by averaging the learning resource embeddings;

[0023] Based on the session embedding, an attention mechanism is used to assign a weight to each learning resource embedding;

[0024] Based on each learning resource embedding and its weight, the user's weighted long-term learning intention embedding is obtained.

[0025] Preferably, the pairwise learning resource conversion based on the historical learning resource sequence set constructs a learning resource graph, and uses the last historical learning resource of each historical learning resource sequence as a guide, combined with an attention mechanism, to extract the user's short-term learning intention embedding; including:

[0026] Based on the pairwise learning resource conversion of the historical learning resource sequence set, a learning resource graph is constructed, and the first-order Markov transition probability distribution is extracted from the learning resource transition probability matrix of the learning resource graph using the last historical learning resource of each historical learning resource sequence as a guide;

[0027] In combination with the attention mechanism, weighted calculation is performed on the importance of each historical learning resource in the first-order Markov transition probability distribution;

[0028] Based on the weighted calculation result of the importance of each historical learning resource and combined with the initial embedding of each historical learning resource, the user's weighted short-term learning intention embedding is obtained.

[0029] Preferably, the randomly enhancing each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss includes:

[0030] Randomly enhance each historical learning resource sequence to obtain a corresponding enhanced historical learning resource sequence;

[0031] The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's enhanced long-term learning intention embedding from each enhanced historical learning resource sequence;

[0032] Setting a learning batch to obtain an enhanced batch embedding; wherein the long-term learning intention embedding and the enhanced long-term learning intention embedding of the same user are used as the first positive sample pair, and the long-term learning intention embedding and the enhanced long-term learning intention embedding of other users are used as the first negative sample set;

[0033] Based on the first positive sample pair and the first negative sample set, a self-supervised contrastive learning loss is constructed.

[0034] Preferably, the retrieving similar historical learning resource sequences having short-term similar learning intentions with each historical learning resource sequence, obtaining the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss; comprising:

[0035] Retrieve similar historical learning resource sequences that have the same last historical learning resource and predicted next learning resource as each historical learning resource sequence, and regard them as positive sample sets of the user's historical learning resource sequences, and calculate the similarity between each historical learning resource sequence and any similar historical learning resource sequence in its positive sample set;

[0036] The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's long-term retrieval learning intention embedding from each similar historical learning resource sequence;

[0037] Set a learning batch and obtain a batch embedding with retrieval; wherein the long-term learning intention embedding and the retrieval long-term learning intention embedding of the same user are used as the second positive sample pair, and the long-term learning intention embedding and the retrieval long-term learning intention embedding of other users are used as the second negative sample set;

[0038] Based on the similarity between each historical learning resource sequence and any similar historical learning resource sequence in the positive sample set, and the corresponding second positive sample pairs and second negative sample sets, a supervised contrastive learning loss is constructed.

[0039] Preferably, the fusion of the long-term learning intention embedding and the short-term learning intention embedding to obtain the prediction result of the user's next learning resource to construct the cross entropy loss function of the main recommendation task includes:

[0040] Using a multi-layer perceptron MLP to assign weights to the long-term learning intention embedding and the short-term learning intention embedding;

[0041] Based on the long-term learning intention embedding and its weight, and the short-term learning intention embedding and its weight, obtaining the user's final learning intention embedding;

[0042] Obtain the prediction result of the user's next learning resource through the dot product operation of the user's final learning intention embedding and the initial embedding of the learning resource;

[0043] Based on the prediction results and label embedding of the user's next learning resource, the cross entropy loss function of the main recommendation task is constructed.

[0044] Preferably, the Adam optimization algorithm is used to optimize and solve the learning resource recommendation network, and the overall loss function is calculated to update the network parameters until convergence.

[0045] A learning resource recommendation system for intention-aware dual contrast learning, based on a preset learning resource recommendation network, includes:

[0046] The acquisition module is used to obtain the historical learning resource sequence set of all users;

[0047] A pre-training module, used for pre-training a Transformer using a masked language model (MLM) strategy based on the historical learning resource sequence set;

[0048] The capture module is used to use the pre-trained Transformer as the basic encoder and combine it with the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence;

[0049] An extraction module, configured to construct a learning resource graph based on the pairwise learning resource conversion of the historical learning resource sequence set, and extract the user's short-term learning intention embedding by taking the last historical learning resource of each historical learning resource sequence as a guide and combining an attention mechanism;

[0050] The contrastive learning module is used to randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss;

[0051] and for retrieving similar historical learning resource sequences with short-term similar learning intentions as each historical learning resource sequence, obtaining the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss;

[0052] A fusion module, used to fuse the long-term learning intention embedding and the short-term learning intention embedding, obtain the prediction result of the user's next learning resource, and construct a cross entropy loss function of the main recommendation task;

[0053] The prediction module is used to construct an overall loss function based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function and optimize until the network converges, and use the learning resource recommendation network with optimal parameters to predict the next learning resource of the target user.

[0054] A storage medium stores a computer program for recommending learning resources for intention-aware double contrastive learning, wherein the computer program enables a computer to execute the learning resource recommendation method for double contrastive learning as described above.

[0055] An electronic device, comprising:

[0056] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a learning resource recommendation method for performing double contrastive learning as described above.

[0057] (III) Beneficial effects

[0058] The present invention provides a learning resource recommendation method, system, storage medium and electronic device for intention-aware dual contrast learning. Compared with the prior art, it has the following beneficial effects:

[0059] The present invention aims to provide users with personalized, accurate and robust learning resource recommendations. First, the masked language model (MLM) strategy is introduced and the user's historical learning resource sequence is used to pre-train and fine-tune the Transformer; secondly, the pre-trained Transformer is used as the basic encoder, combined with the attention mechanism to capture the user's long-term learning intention embedding; thirdly, a learning resource graph is constructed to extract the user's short-term learning intention embedding from paired learning resource conversions; then, a self-supervised dual contrast learning strategy is designed to enhance the modeling of long-term and short-term learning intentions; finally, the long-term and short-term learning intentions are adaptively fused to predict the next learning resource, and the overall loss function is constructed and optimized. Through the two-layer learning intention modeling and the self-supervised contrast learning strategy, the user's long-term and short-term learning intentions are fully captured, and the challenges of sparse learning behavior and noisy data are effectively addressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A block diagram of a learning resource recommendation method for intention-aware dual contrast learning provided by an embodiment of the present invention;

[0062] Figure 2 A framework diagram of a learning resource recommendation method for intent-aware dual contrastive learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The embodiments of the present application solve the technical problem of providing personalized, accurate and robust learning resource recommendations to users by providing a learning resource recommendation method, system, storage medium and electronic device for intent-aware dual contrast learning.

[0065] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0066] The present invention involves Transformer, supervised and self-supervised contrastive learning, attention mechanism and other technologies, aiming to provide users with personalized, accurate and robust learning resource recommendations.

[0067] The technical concept of the scheme can be summarized as follows: first, introduce the masked language model (MLM) strategy and use the user's historical learning resource sequence to pre-train and fine-tune the Transformer; second, use the pre-trained Transformer as the basic encoder and combine it with the attention mechanism to capture the user's long-term learning intention embedding; third, construct a learning resource graph to extract the user's short-term learning intention embedding from paired learning resource conversions; then, design a self-supervised dual contrast learning strategy to enhance the modeling of long- and short-term learning intentions; finally, adaptively fuse long- and short-term learning intentions, predict the next learning resource, and construct and optimize the overall loss function.

[0068] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0069] Embodiment 1:

[0070] like Figure 1 As shown, an embodiment of the present invention provides a learning resource recommendation method for intention-aware dual contrast learning, based on a preset learning resource recommendation network, the method includes:

[0071] S 1. Obtain the historical learning resource sequence set of all users;

[0072] S2. Based on the historical learning resource sequence set, a masked language model (MLM) strategy is used to pre-train the Transformer.

[0073] S3, using the pre-trained Transformer as the basic encoder and combining it with the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence;

[0074] S4, based on the pairwise learning resource conversion of the historical learning resource sequence set, construct a learning resource graph, and use the last historical learning resource of each historical learning resource sequence as a guide, combined with the attention mechanism, to extract the user's short-term learning intention embedding;

[0075] S5, randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss;

[0076] S6, retrieve similar historical learning resource sequences that have short-term similar learning intentions with each historical learning resource sequence, obtain the user's retrieval long-term learning intention embedding, and construct a supervised contrastive learning loss;

[0077] S7, integrating the long-term learning intention embedding and the short-term learning intention embedding, obtaining the prediction result of the user's next learning resource, so as to construct a cross entropy loss function of the main recommendation task;

[0078] S 8. Based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function, an overall loss function is constructed and optimized until the network converges, and the learning resource recommendation network with the optimal parameters is used to predict the next learning resource of the target user.

[0079] The embodiments of the present invention comprehensively capture users' long-term and short-term learning intentions through double-layer learning intention modeling and a self-supervised comparative learning strategy, and effectively address the challenges of sparse learning behavior and noisy data.

[0080] like Figure 2 As shown, Figure 2 This paper provides a framework diagram of a learning resource recommendation method for intention-aware double contrast learning. Figure 2 , details the steps of the above scheme:

[0081] In step S1, the historical learning resource sequence sets of all users are obtained.

[0082] In this step: Let U be the user set, and Among them, u i represents the i-th user, |U| represents the number of users; let C be the learning resource set, and Among them, c j represents the jth learning resource, |C| represents the number of learning resources, and Indicates c j The initial embedding of , d represents the embedding dimension; before time t, ui The historical learning resource sequence is sorted in chronological order and is denoted as S i =[c i，1 , c i，2 , …, c i，j , …, c i，t ],make Indicates c i，j The initial embedding of , then the historical learning resource sequence set of all users is represented as S = [S1, S2, ..., S |U| ].

[0083] In step S2, based on the historical learning resource sequence set, a masked language model (MLM) strategy is used to pre-train the Transformer.

[0084] In an optional implementation, pre-training a Transformer using a masked language model (MLM) strategy includes:

[0085] S21. In order to better capture the contextual knowledge between courses, the Mask Language Model (MLM) is used to pre-train the Transformer, introducing more supervisory signals to guide the training of model parameters; the special tag “[Mask]” is used to pre-train the Transformer. i Randomly mask a part of the learning resources and move "[Mask]" to the end to get u i Masked historical learning resource sequence and u i Masked Learning Resource Set Use formula (1) to calculate the MLM pre-training loss Loss MLM , get u i The next predicted learning resource that may be interacted with;

[0086]

[0087] In formula (1), Indicates u i Masked learning resources, Indicated in The next possible interactive learning resource is predicted based on probability.

[0088] S22. Pre-train S to obtain the next predicted learning resource set that U may interact with. Fine-tune the Transformer.

[0089] In step S3, the pre-trained Transformer is used as the basic encoder and combined with the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence.

[0090] In an optional embodiment, the capture process of long-term learning intent embedding includes:

[0091] S31. Construct an inverse position matrix for each historical learning resource sequence.

[0092] S32. Based on each historical learning resource sequence and its inverse position matrix, the pre-trained Transformer is used as the basic encoder to obtain the user's learning resource embedding.

[0093] S33. Obtain the user's session embedding by averaging the learning resource embedding.

[0094] S34. Based on the session embedding, an attention mechanism is used to assign a weight to each learning resource embedding.

[0095] S35. Based on each learning resource embedding and its weight, obtain the user's weighted long-term learning intention embedding.

[0096] Specifically:

[0097] In step S31, according to S i Use formula (2) to construct the inverse position matrix P i ={p i，t , p i,t-1 , ..., p i,t-j , ..., p i，1}, to describe the learning order of different courses and highlight the importance of recent learning resources, among which, Indicates u i Learning C j position embedding.

[0098] p i,t-j =e -μ(t-j) (2)

[0099] In formula (2), μ represents the decay rate factor.

[0100] It should be noted that the exponential decay method (time decay weighting) is often used in sequence recommendation to capture users' recent preferences. Drawing on this idea, referring to formula (2), the position index is used here instead of the time difference to calculate the weight.

[0101] In step S32, equation (3) is used to obtain u i Learning resources embedded in H i =[h i，1 ,h i，2 ,…,h i，j ,…,h i，t ];

[0102] H i=E(S i +P i ) (3)

[0103] In formula (3), E(·) represents the pre-trained Transformer base encoder, h i，j Indicates u i Learning C j The encoding embedding.

[0104] In step S33, considering the importance of different courses in the sequence, the attention mechanism is used to assign different weights to each course. First, the average H i Get u i Session embedding

[0105]

[0106] In step S34, h is obtained using equation (5) i，j The weight α i，j ;

[0107]

[0108] In formula (5), the superscript L represents long-term intention learning. represents the learnable weight matrix, q L , represents a learnable vector.

[0109] In step S35, equation (6) is used to obtain u i Weighted long-term learning of intent embedding

[0110]

[0111] In step S4, based on the paired learning resource conversion of the historical learning resource sequence set, a learning resource graph is constructed, and the last historical learning resource of each historical learning resource sequence is used as a guide, combined with the attention mechanism, to extract the user's short-term learning intention embedding.

[0112] In an optional embodiment, the extraction process of short-term learning intention embedding includes:

[0113] S41. Construct a learning resource graph based on the pairwise learning resource conversion of the historical learning resource sequence set, and extract a first-order Markov transition probability distribution from the learning resource transfer probability matrix of the learning resource graph using the last historical learning resource of each historical learning resource sequence as a guide.

[0114] S42. Combined with the attention mechanism, weighted calculation is performed on the importance of each historical learning resource in the first-order Markov transition probability distribution.

[0115] S43. Based on the weighted calculation result of the importance of each historical learning resource and combined with the initial embedding of each historical learning resource, the weighted short-term learning intention embedding of the user is obtained.

[0116] Specifically:

[0117] In step S41, a learning resource graph is constructed from S Among them, ε c ={(ε ab , w ab )|c a , c b ∈C, c a →c b , w ab ∈W} represents the learning resource edge matrix, c a represents the ath learning resource, c b represents the bth learning resource, c a →c b Indicates c a The next learning resource is c b , ε ab Indicated by c a Convert to c b The connection edge; W represents the learning resource transfer probability matrix, w ab ∈W means from c a to c b The transition probability.

[0118] In step S42, c is extracted from W i，t The first-order Markov transition probability distribution C i，t ={(c i，k , w tk )|c i，t →c i，k},c i，k Indicates u i The next history learning resource, w tk Indicates that from c t to c k The transition probability, k = {1, 2, ..., |C i，t |}, using formula (7) to calculate C i，t The importance of each historical learning resource is weighted.

[0119]

[0120] In step S43, u is calculated using equation (8) iShort-term learning intention embedding

[0121]

[0122] In formula (8), e i，k Indicates u i The learned initial embedding of the kth learning resource.

[0123] In step S5, each historical learning resource sequence is randomly enhanced to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss.

[0124] This step uses random enhancement operations and self-supervised contrastive learning to improve quality and alleviate the noise interaction problem of S.

[0125] In an optional embodiment, the process of constructing the self-supervised contrastive learning loss includes:

[0126] S51. Randomly enhance each historical learning resource sequence to obtain a corresponding enhanced historical learning resource sequence.

[0127] S52, using the pre-trained Transformer as the basic encoder and combining it with the attention mechanism, captures the user's enhanced long-term learning intention embedding from each enhanced historical learning resource sequence;

[0128] S53, setting a learning batch and obtaining a batch embedding with enhancement; wherein the long-term learning intention embedding and the enhanced long-term learning intention embedding of the same user are used as the first positive sample pair, and the long-term learning intention embedding and the enhanced long-term learning intention embedding of other users are used as the first negative sample set;

[0129] S54: Based on the first positive sample pair and the first negative sample set, construct a self-supervised contrastive learning loss.

[0130] Specifically:

[0131] In step S51, S i Perform random enhancement operations to obtain u i Enhanced sequence of history learning resources

[0132]

[0133] In formula (9), represents random augmentation operation. There are three augmentation schemes: random masking, random deletion, and reordering. Random masking and random deletion generate augmented learning sequences to simulate the situation where users skip or mistakenly click on some learning resources. Reordering represents the change of learning path. Random augmentation operation allows Si projected into different augmented views which are treated as positive pairs during contrastive learning.

[0134] In step S52, S i and Substituting into step 3, we get u i Weighted long-term learning intention embedding h i and weighted enhanced long-term learning intention embedding (The superscript L is omitted here for brevity).

[0135] In step S53, the learning batch is set to B, with the enhanced batch embedding as Set h i and for The first positive sample pair of for The first negative sample set, where express The first negative sample.

[0136] In step S54, the self-supervised contrastive learning loss Loss is calculated using formula (10): self ;

[0137]

[0138] In formula (10), s(·) represents the cosine similarity, and τ1 represents the first temperature parameter, which is used to adjust the sensitivity of similarity measurement in the process of self-supervised contrastive learning.

[0139] In step S6, similar historical learning resource sequences with short-term similar learning intentions to each historical learning resource sequence are retrieved, and the user's retrieval long-term learning intention embedding is obtained to construct a supervised contrastive learning loss.

[0140] This step uses supervised contrastive learning to align sequences of short-term similar learning intentions to improve recommendation accuracy.

[0141] In an optional implementation, the construction process of supervised contrastive learning loss includes:

[0142] S 61. Retrieve similar historical learning resource sequences that have the same last historical learning resource and predicted next learning resource as each historical learning resource sequence, and regard them as the positive sample set of the user's historical learning resource sequence, and calculate the similarity between each historical learning resource sequence and any similar historical learning resource sequence in its positive sample set.

[0143] S62. Use the pre-trained Transformer as the basic encoder and combine it with the attention mechanism to capture the user's retrieval long-term learning intention embedding from each similar historical learning resource sequence.

[0144] S63. Set a learning batch and obtain a batch embedding with retrieval; wherein the long-term learning intention embedding and the retrieval long-term learning intention embedding of the same user are used as the second positive sample pair, and the long-term learning intention embedding and the retrieval long-term learning intention embedding of other users are used as the second negative sample set.

[0145] S64: construct a supervised contrastive learning loss based on the similarity between each historical learning resource sequence and any similar historical learning resource sequence in the positive sample set, and the corresponding second positive sample pairs and second negative sample sets.

[0146] Specifically:

[0147] In step S61, the search is performed on the i The historical learning sequences with the same last learning resource and predicted next learning resource are considered as positive sample sets. Calculate S using formula (11) i and S i′ The similarity sim(S i , S i′ );

[0148]

[0149] In formula (11), S i (T), S i′ (T) means S i and S i′ The last T learning resources in S i (T) and S i′ Any learning resource that overlaps in (T), Respectively represent c in S i (T) and S i′ (T) is the reverse order of the position sorting, for example, set T = 10, c in S i (10) Ranked 4th, then is a regularization term that prevents the denominator from approaching 0.

[0150] In step S62, set the learning batch to B and randomly select an S from B. i Positive samples S i and Substituting into step 3, we get u i Weighted long-term learning intention embedding h iand weighted retrieval of long-term learning intent embeddings (Again, the superscript L is omitted here for brevity).

[0151] In step S63, the batch embedding with retrieval is obtained as Set h i and for The second positive sample pair, for The second negative sample set, where express The second negative sample.

[0152] In step S64, the supervised contrastive learning loss Loss is calculated using formula (12): sup ;

[0153]

[0154] In formula (12), τ2 represents the second temperature parameter.

[0155] In step S7, the long-term learning intention embedding and the short-term learning intention embedding are fused to obtain the prediction result of the user's next learning resource to construct the cross entropy loss function of the main recommendation task.

[0156] In an optional implementation, the fusion of the long-term learning intention embedding and the short-term learning intention embedding to obtain the prediction result of the user's next learning resource to construct a cross entropy loss function of the main recommendation task includes:

[0157] S71. Use a multi-layer perceptron MLP to assign weights to the long-term learning intention embedding and the short-term learning intention embedding.

[0158] S72. Based on the long-term learning intention embedding and its weight, and the short-term learning intention embedding and its weight, obtain the user's final learning intention embedding.

[0159] S73. Obtain the prediction result of the user's next learning resource by performing a dot product operation on the user's final learning intention embedding and the initial embedding of the learning resource.

[0160] S74. Based on the prediction result and label embedding of the user's next learning resource, construct a cross entropy loss function for the main recommendation task.

[0161] Specifically:

[0162] In step S71, the multi-layer perceptron (MLP) is used to allocate and The weight wi and 1-w i ;

[0163]

[0164] In formula (13), MLP(·) represents the multi-layer perceptron operation, Express and The two embeddings are concatenated.

[0165] In step S72, the final learning intention embedding h of ui is obtained using equation (14) i ;

[0166]

[0167] In step S73, by h i and e j Dot product operation, using formula (15) to get the prediction result

[0168]

[0169] In formula (15), softmax(·) represents the normalization function.

[0170] In step S74, the cross entropy loss function of the main recommendation task is constructed using formula (16);

[0171]

[0172] In formula (16), y ij Indicates u i C j The embedding of real interaction is user u i Label embedding of the next learning resource.

[0173] In step S8, based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function, an overall loss function is constructed and optimized until the network converges, and the learning resource recommendation network with optimal parameters is used to predict the next learning resource of the target user.

[0174] In an optional implementation, the process of constructing the overall loss function includes:

[0175] Using formula (17) to combine the loss functions of the main recommendation task, the self-supervised contrastive learning task and the global supervised contrastive learning task, we get the overall loss function Loss total ;

[0176] Loss total =Lo ssmain+λ1Loss self +λ2Loss sup +λ3||θ|| 2 (17)

[0177] In formula (17), λ1, λ2, λ3 represent hyperparameters, which are used to adjust the relative importance of different loss functions during training; θ represents model parameters; ||·|| represents the L2 regularization constraint.

[0178] Furthermore, in an optional implementation, the Adam optimization algorithm is used to optimize the learning resource recommendation network and calculate the overall loss function Loss total , to update the network parameters until convergence, so as to obtain the learning resource recommendation model with the optimal parameters for predicting u i The next interactive learning resource for .

[0179] So far, the embodiment of the present invention has completed the entire process of the learning resource recommendation method of intention-aware dual contrastive learning.

[0180] Embodiment 2:

[0181] The embodiment of the present invention provides a learning resource recommendation system for intention-aware dual contrast learning. Based on a preset learning resource recommendation network, the system includes:

[0182] The acquisition module is used to obtain the historical learning resource sequence set of all users;

[0183] A pre-training module, used for pre-training a Transformer using a masked language model (MLM) strategy based on the historical learning resource sequence set;

[0184] The capture module is used to use the pre-trained Transformer as the basic encoder and combine it with the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence;

[0185] An extraction module, configured to construct a learning resource graph based on the pairwise learning resource conversion of the historical learning resource sequence set, and extract the user's short-term learning intention embedding by taking the last historical learning resource of each historical learning resource sequence as a guide and combining an attention mechanism;

[0186] The contrastive learning module is used to randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss;

[0187] and for retrieving similar historical learning resource sequences with short-term similar learning intentions as each historical learning resource sequence, obtaining the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss;

[0188] A fusion module, used to fuse the long-term learning intention embedding and the short-term learning intention embedding, obtain the prediction result of the user's next learning resource, and construct a cross entropy loss function of the main recommendation task;

[0189] The prediction module is used to construct an overall loss function based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function and optimize until the network converges, and use the learning resource recommendation network with optimal parameters to predict the next learning resource of the target user.

[0190] Embodiment 3:

[0191] An embodiment of the present invention provides a storage medium storing a computer program for recommending learning resources for intention-aware double contrastive learning, wherein the computer program enables a computer to execute the learning resource recommendation method for double contrastive learning as described in Example 1.

[0192] Embodiment 4:

[0193] An embodiment of the present invention provides an electronic device, including:

[0194] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a learning resource recommendation method for performing double contrastive learning as described in Example 1

[0195] It can be understood that the learning resource recommendation system, storage medium and electronic device for intention-aware dual comparative learning provided in the embodiments of the present invention correspond to the learning resource recommendation method for intention-aware dual comparative learning provided in the embodiments of the present invention, and the explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the learning resource recommendation method, which will not be repeated here.

[0196] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0197] 1. The embodiment of the present invention introduces the masked language model MLM strategy, uses the pre-trained Transformer encoder to better capture the contextual knowledge between learning resources, and introduces more supervisory signals to guide the training of model parameters. At the same time, using the pre-trained parameters to initialize the Transformer and further fine-tune it on the next learning resource prediction task can better model long-term learning intentions. Position embedding and attention mechanisms help the model pay more attention to recently accessed learning resources, which is conducive to the representation of long-term learning intentions.

[0198] 2. The embodiment of the present invention breaks the historical learning resource sequence, extracts paired learning resource conversions, constructs a learning resource conversion graph, analyzes the resources that the user last visited and their related relationships, pays more attention to short-term interests than long-term learning intentions, and realizes complementary learning intention representation. Combined with the attention mechanism, it can more effectively capture the user's short-term learning intention.

[0199] 3. The embodiment of the present invention uses a supervised and self-supervised dual contrastive learning strategy to alleviate data sparsity and noise problems. Self-supervised contrastive learning enhances the model's robustness to noise by generating different versions of the sequence; while supervised contrastive learning uses sequences with the same recommendation target to align sequences with similar learning intentions, thereby improving recommendation accuracy.

[0200] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0201] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A learning resource recommendation method for intention-aware double contrastive learning, characterized in that: Based on the preset learning resource recommendation network, the method includes: Get the historical learning resource sequence set of all users; Based on the historical learning resource sequence set, a masked language model MLM strategy is used to pre-train the Transformer; The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's long-term learning intention embedding from each historical learning resource sequence; Based on the pairwise learning resource conversion of the historical learning resource sequence set, a learning resource graph is constructed, and the last historical learning resource of each historical learning resource sequence is used as a guide, combined with an attention mechanism, to extract the user's short-term learning intention embedding; Randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss; Retrieve similar historical learning resource sequences that have short-term similar learning intentions as each historical learning resource sequence, obtain the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss; The long-term learning intention embedding and the short-term learning intention embedding are integrated to obtain the prediction result of the user's next learning resource to construct a cross entropy loss function of the main recommendation task; Based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function, an overall loss function is constructed and optimized until the network converges, and the learning resource recommendation network with optimal parameters is used to predict the next learning resource of the target user.

2. The learning resource recommendation method for double contrastive learning as claimed in claim 1, characterized in that: The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's long-term learning intention embedding from each historical learning resource sequence; including: Construct the inverse position matrix of each historical learning resource sequence; Based on each historical learning resource sequence and its inverse position matrix, the pre-trained Transformer is used as the basic encoder to obtain the user's learning resource embedding; Obtaining the user's session embedding by averaging the learning resource embeddings; Based on the session embedding, an attention mechanism is used to assign a weight to each learning resource embedding; Based on each learning resource embedding and its weight, the user's weighted long-term learning intention embedding is obtained.

3. The learning resource recommendation method for double contrastive learning as claimed in claim 1, characterized in that: The pairwise learning resource conversion based on the historical learning resource sequence set constructs a learning resource graph, and uses the last historical learning resource of each historical learning resource sequence as a guide, combined with the attention mechanism, to extract the user's short-term learning intention embedding; including: Based on the pairwise learning resource conversion of the historical learning resource sequence set, a learning resource graph is constructed, and the first-order Markov transition probability distribution is extracted from the learning resource transition probability matrix of the learning resource graph using the last historical learning resource of each historical learning resource sequence as a guide; In combination with the attention mechanism, weighted calculation is performed on the importance of each historical learning resource in the first-order Markov transition probability distribution; Based on the weighted calculation result of the importance of each historical learning resource and combined with the initial embedding of each historical learning resource, the user's weighted short-term learning intention embedding is obtained.

4. The learning resource recommendation method for double contrastive learning as claimed in claim 2, characterized in that: The randomly enhancing each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss includes: Randomly enhance each historical learning resource sequence to obtain a corresponding enhanced historical learning resource sequence; The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's enhanced long-term learning intention embedding from each enhanced historical learning resource sequence; Setting a learning batch to obtain an enhanced batch embedding; wherein the long-term learning intention embedding and the enhanced long-term learning intention embedding of the same user are used as the first positive sample pair, and the long-term learning intention embedding and the enhanced long-term learning intention embedding of other users are used as the first negative sample set; Based on the first positive sample pair and the first negative sample set, a self-supervised contrastive learning loss is constructed.

5. The learning resource recommendation method for double contrastive learning as claimed in claim 2, characterized in that: The method retrieves similar historical learning resource sequences that have short-term similar learning intentions with each historical learning resource sequence, obtains the user's retrieval long-term learning intention embedding, and constructs a supervised contrastive learning loss; including: Retrieve similar historical learning resource sequences that have the same last historical learning resource and predicted next learning resource as each historical learning resource sequence, and regard them as positive sample sets of the user's historical learning resource sequences, and calculate the similarity between each historical learning resource sequence and any similar historical learning resource sequence in its positive sample set; The pre-trained Transformer is used as the basic encoder, combined with the attention mechanism, to capture the user's long-term retrieval learning intention embedding from each similar historical learning resource sequence; Set a learning batch and obtain a batch embedding with retrieval; wherein the long-term learning intention embedding and the retrieval long-term learning intention embedding of the same user are used as the second positive sample pair, and the long-term learning intention embedding and the retrieval long-term learning intention embedding of other users are used as the second negative sample set; Based on the similarity between each historical learning resource sequence and any similar historical learning resource sequence in the positive sample set, and the corresponding second positive sample pairs and second negative sample sets, a supervised contrastive learning loss is constructed.

6. The learning resource recommendation method for double contrastive learning according to claim 1, characterized in that: The method of fusing the long-term learning intention embedding and the short-term learning intention embedding to obtain the prediction result of the user's next learning resource to construct the cross entropy loss function of the main recommendation task includes: Using a multi-layer perceptron MLP to assign weights to the long-term learning intention embedding and the short-term learning intention embedding; Based on the long-term learning intention embedding and its weight, and the short-term learning intention embedding and its weight, obtaining the user's final learning intention embedding; Obtain the prediction result of the user's next learning resource through the dot product operation of the user's final learning intention embedding and the initial embedding of the learning resource; Based on the prediction results and label embedding of the user's next learning resource, the cross entropy loss function of the main recommendation task is constructed.

7. The learning resource recommendation method for double contrastive learning according to claim 1, characterized in that: The Adam optimization algorithm is used to optimize and solve the learning resource recommendation network, and the overall loss function is calculated to update the network parameters until convergence.

8. An intention-aware double contrastive learning learning resource recommendation system, characterized in that: Based on the preset learning resource recommendation network, the system includes: The acquisition module is used to obtain the historical learning resource sequence set of all users; A pre-training module, used for pre-training a Transformer using a masked language model (MLM) strategy based on the historical learning resource sequence set; The capture module is used to use the pre-trained Transformer as the basic encoder and combine it with the attention mechanism to capture the user's long-term learning intention embedding from each historical learning resource sequence; An extraction module, configured to construct a learning resource graph based on the pairwise learning resource conversion of the historical learning resource sequence set, and extract the user's short-term learning intention embedding by taking the last historical learning resource of each historical learning resource sequence as a guide and combining an attention mechanism; The contrastive learning module is used to randomly enhance each historical learning resource sequence to obtain the user's enhanced long-term learning intention embedding to construct a self-supervised contrastive learning loss; and for retrieving similar historical learning resource sequences with short-term similar learning intentions as each historical learning resource sequence, obtaining the user's retrieval long-term learning intention embedding to construct a supervised contrastive learning loss; A fusion module, used to fuse the long-term learning intention embedding and the short-term learning intention embedding, obtain the prediction result of the user's next learning resource, and construct a cross entropy loss function of the main recommendation task; The prediction module is used to construct an overall loss function based on the self-supervised contrastive learning loss, supervised contrastive learning loss and cross entropy loss function and optimize until the network converges, and use the learning resource recommendation network with optimal parameters to predict the next learning resource of the target user.

9. A storage medium, characterized in that: The computer program for recommending learning resources for intention-aware double contrastive learning is stored therein, wherein the computer program enables a computer to execute the method for recommending learning resources for double contrastive learning as claimed in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs comprising a learning resource recommendation method for executing double contrastive learning as described in any one of claims 1 to 7.

Images