A multimedia recommendation method based on information bottleneck denoising

Abstract

The present invention discloses a multimedia recommendation method based on information bottleneck denoising, including: 1. constructing heterogeneous data: user-to-product interaction matrix, product multimedia content data; 2. extracting product multimedia content features using a pre-trained model; 3. constructing a product association matrix based on multimedia content features; 4. learning user and product representation matrices based on a deep graph neural model; 5. minimizing the mutual information between multimedia content and its representation based on the information bottleneck theory, and calculating the loss function; 6. reconstructing the interaction matrix based on the user and product representation matrix, and calculating the loss function; 7. performing multi-task learning in conjunction with the loss function of step 5-6, and updating the model parameters until the model converges. Based on the idea of ​​information bottleneck, the present invention learns the least multimedia content information while meeting the requirements of the recommendation task, and can effectively remove redundant information in the multimedia content, thereby enabling more accurate multimedia recommendations.

Description

Technical Field

[0001] The present invention relates to the field of multimedia recommendation, and in particular to a multimedia recommendation method based on information bottleneck denoising. Background Art

[0002] Recommendation system is an information technology that provides personalized services and is often used to alleviate the information overload problem caused by massive data. Traditional ID-based collaborative filtering methods characterize user preferences through matrix decomposition, graph neural network and other technologies to achieve personalized product recommendations. With the development of deep learning and multimedia information processing technology, recommendation methods based on multimedia data assistance have been widely studied.

[0003] Existing multimedia recommendation methods use pre-trained models to extract features from multimedia content, and then combine ID information to model user preferences. Although multimedia recommendation methods have made some progress, there is still the problem of feature noise: pre-trained models are usually task-oriented. For example, visual pre-trained models are trained based on visual tasks, and the extracted visual feature information contains noise information for recommendation tasks and needs to be denoised. Since feature noise is difficult to formally define, how to implicitly design a denoising module to improve recommendation tasks is a key issue. Summary of the invention

[0004] In order to solve the shortcomings of the prior art, the present invention proposes a multimedia recommendation method based on information bottleneck denoising, in order to obtain the minimum multimedia content that meets the recommendation task and implicitly reduce the impact of multimedia feature noise on the recommendation task, thereby improving the recommendation performance.

[0005] The present invention adopts the following technical solutions to solve the technical problems:

[0006] The multimedia recommendation method based on information bottleneck denoising of the present invention is characterized in that it is performed according to the following steps:

[0007] Step 1: construct heterogeneous data, including: user interaction matrix for products, and multimedia content data of products;

[0008] Step 1.1: Let U represent the user set, and U = {u1, ...u a , ..., u b ,...u M},u a represents the ath user, u b represents the b-th user, M represents the total number of users, 1≤a, b≤M; let V represent the product set, and V={v1,...,v i , ..., v j , ..., v N},vi represents the i-th product, v j represents the jth product, N represents the total number of products, 1≤i, j≤N; let r ai represents the a-th user u a For the i-th product v i The interaction data of users on products, then the user interaction matrix R = {r ai} M×N ;

[0009] Step 1.2, let C = {C 1 , ..., C k , ..., C K} represents the multimedia content of product set V, where Table C k represents the multimedia content of the kth modality in the product set V, and represents the multimedia content of the kth modality in the product set V, represents the i-th product v i The multimedia content of the kth modality, K represents the total number of modalities, 1≤k≤K;

[0010] Step 2: Use the pre-trained model to extract product multimedia content features:

[0011] Use the pre-trained models of each modality to extract features from the multimedia content of the corresponding modality, and obtain a set of multimedia feature matrices of K modalities {F 1 , ..., F k , ..., F K}; where F k Represents the multimedia feature matrix of the kth modality;

[0012] Step 3: According to the multimedia feature matrix set {F 1 , ..., F k , ..., F K}Construct product association matrix:

[0013] Step 3.1: Calculate the similarity matrix set {S 1 , ..., S k , ..., S K}:

[0014] S k =L2(F k )×(L2(F k )) T (1)

[0015] In formula (1), S k represents the similarity matrix of the kth mode in the product set V, L2(F k) represents the normalized multimedia feature matrix of the kth modality in the product set V, and T represents the matrix transpose;

[0016] Step 3.2: According to formula (2), the similarity matrix set {S 1 , ..., S k , ..., S K} is sparsely processed to obtain the association matrix set {S′ 1 , ..., S′ k , ..., S′ K}:

[0017]

[0018] In formula (92), Represents the similarity matrix S of the kth mode in the product set V k The element value of the i-th row and j-th column of Represents the correlation matrix S′ of the kth mode in the product set V k The element value of the i-th row and j-th column of Represents the similarity matrix S of the kth mode in the product set V k All element values ​​of the i-th row, TopK′ represents the first K′ in descending order;

[0019] Step 3.3: Calculate the association matrix S′ of the product set V according to formula (3):

[0020]

[0021] Step 3.4: Calculate the representation propagation matrix A according to formula (4):

[0022]

[0023] The normalized representation propagation matrix is ​​calculated by formula (5):

[0024]

[0025] In formula (5), D represents the degree matrix of matrix A;

[0026] Step 4: Learn the user and product representation matrix based on the deep graph neural model:

[0027] Step 4.1: Use Gaussian distribution to randomly initialize the user representation matrix P = (p1, ..., p a , ..., p M}, where p a represents the a-th user u ad1+Kd2-dimensional representation vector; where d1 represents the user collaborative representation dimension, and d2 represents the content representation dimension;

[0028] Use Gaussian distribution to randomly initialize the product co-representation matrix Q = {q1, ..., q i， …q N}, where q i represents the i-th product v j The d1-dimensional representation vector of

[0029] Step 4.2: Calculate the multimedia content representation matrix T of each modality in the product set V by formula (6) = [T 1 , ..., T k , ..., T K ]:

[0030] T k =MLP k (F k ) (6)

[0031] In formula (5), MLP k is the kth multilayer perceptron, represents the multimedia content representation matrix of the kth modality;

[0032] Step 4.3: Construct the product representation matrix E = [Q, T]; and use the graph neural network to process {P, E} to obtain the user final representation matrix H u And the final product characterization matrix H v ;

[0033] Step 5: Minimize the mutual information between multimedia content and its representation based on the information bottleneck theory and calculate the loss function:

[0034] Step 5.1: Use PCA algorithm to analyze the multimedia feature matrix set {F 1 , ..., F k , ..., F K} to reduce the dimension and obtain the multimodal feature matrix {F′ 1 , ..., F′ k , ..., F′ K},in, represents the kth mode F k The multimedia feature matrix after dimensionality reduction;

[0035] Step 5.2: Construct the information bottleneck loss function L(θ1) according to formula (10):

[0036]

[0037] In formula (10), θ1 represents the parameters corresponding to K multi-layer perceptrons, HSIC(,) represents the Hilbert-Schmidt independence index;

[0038] Step 6: Reconstruct the interaction matrix based on the user and product representation matrices and calculate the loss function:

[0039] Step 6.1: Predict the ath user u according to formula (11): a For the i-th product v i The interaction probability r′ ai :

[0040]

[0041] In formula (11), δ is the sigmoid activation function; Represents the user's final representation matrix H u The ath row in represents the transpose of the vector; Represents the final product characterization matrix H u The i-th row in represents the vector;

[0042] Step 6.2: Calculate the reconstruction loss function L(θ2) according to formula (12):

[0043]

[0044] In formula (12), θ2 represents all parameters involved in reconstructing the interaction matrix, and θ2 = [P, Q, θ1]; D a is the ath user u a All training data of D a ={(i, j)|i∈R a , j∈VR a}; R a represents the a-th user u a A collection of interactive products; Represents the final product characterization matrix H v The j-th row vector in ;

[0045] Step 7: Combine various loss functions to perform multi-task learning to update model parameters until the model converges:

[0046] Step 7.1: Establish the multi-task optimization objective L(θ) according to formula (13):

[0047] L(θ)=L(θ1)+αL(θ2)+β||[P, Q,]|| 2 (13)

[0048] In formula (13), θ = [P, Q, θ2] is all the parameters to be optimized, α and β are used to adjust the weights of the information bottleneck loss function and the regularization term respectively;

[0049] Step 7.2: Solve the optimization target L(θ) by gradient descent method to update the parameter θ until L(θ) converges to the minimum value, thereby obtaining the optimal parameter θ. * ;

[0050] Step 7.3: Predict the ath user u according to formula (14) a For the i-th product v i The optimal interaction probability Thus, we get the interaction matrix of user set U predicting product set V: To implement product recommendations:

[0051]

[0052] In formula (14), Denotes the optimal user representation matrix H u* The ath user u a The optimal representation vector of Represents the optimal product representation matrix H v* The i-th product v i The optimal representation vector of .

[0053] The multimedia recommendation method based on information bottleneck denoising described in the present invention is also characterized in that the graph neural network in step 4.3 has a total of L convolutional layers, the current convolutional layer is l, and l=0 is initialized;

[0054] Initialize the l-th layer node representation matrix H l ={P, E};

[0055] The l-th layer node representation matrix H l It is sent into the graph neural network, and the l+1th layer node representation matrix H is calculated according to formula 97) l+1 :

[0056]

[0057] According to formula (8), the output of each graph convolution layer is aggregated, and the final node representation matrix H is obtained:

[0058]

[0059] According to formula (9), the user's final representation matrix H is obtained u And the final product characterization matrix H v :

[0060] H u =H[:M],H v =H[M:] (9)

[0061] In formula (9), H[:M] represents the 1st to Mth rows of the matrix H, and H[M:] represents the (M+1)th to (M+N)th rows of the matrix H.

[0062] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the multimedia recommendation method, and the processor is configured to execute the program stored in the memory.

[0063] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the multimedia recommendation method when the computer program is executed by a processor.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0065] 1. This paper proposes a multimedia denoising method based on information bottleneck, which obtains the minimum multimedia content that meets the recommendation task by means of information bottleneck constraints. This method can effectively denoise pre-trained multimedia features, thereby improving the quality of user preference modeling and improving recommendation performance.

[0066] 2. The multimedia denoising method based on information bottleneck proposed in the present invention is universal and can be embedded in any recommendation method based on representation learning and is applicable to many recommendation scenarios.

[0067] 3. The multimedia denoising method based on information bottleneck proposed in the present invention is very lightweight, does not introduce any model parameters, and has strong practical value.

[0068] 4. The present invention utilizes Hilbert-Schmidt independence constraints to construct an information bottleneck module, which is used to minimize the mutual information between multimedia representations and original features, and can implicitly reduce the impact of multimedia noise on user modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 The figure is a flow chart of the multimedia denoising method based on information bottleneck of the present invention. DETAILED DESCRIPTION

[0070] In this embodiment, a multimedia recommendation method based on information bottleneck denoising is to consider the noise problem of pre-trained multimedia features, design an information bottleneck module based on Hilbert-Schmidt independence constraints, obtain the minimum multimedia content that meets the recommendation task, thereby achieving multimedia feature denoising and improving recommendation performance. Specifically, Figure 1 As shown, the steps are as follows:

[0071] Step 1: construct heterogeneous data, including: user interaction matrix for products, and multimedia content data of products;

[0072] Step 1.1: Let U represent the user set, and U = {u1, ...u a , ..., u b ,...u M},u a represents the ath user, u b represents the b-th user, M represents the total number of users, 1≤a, b≤M; let V represent the product set, and V={v1,...,v i , ..., v j , ..., v N},v i represents the i-th product, v j represents the jth product, N represents the total number of products, 1≤i, j≤N; let r ai represents the a-th user u a For the i-th product v i The interaction data of users on products, then the user interaction matrix R = {r ai} M×N ; If the ath user u a For the i-th product v i If there is behavioral data (viewing, purchasing, clicking, etc.), then r ai =1, otherwise r ai =0;

[0073] Step 1.2, let C = {C 1 , C 2 , C 3} represents the multimedia content of product set V, where C 1 , C 2 , C 3 denote the visual content, text content, and audio content of the product set V, respectively, and Respectively represent the i-th product v i Visual content, text content and audio content;

[0074] Step 2: Use the pre-trained model to extract product multimedia content features:

[0075] Step 2.1: Use the pre-trained models of each modality to extract features from the corresponding modality multimedia content, and obtain three multimedia feature matrix sets {F 1 , F 2 , F 3}; where F 1 represents the visual feature matrix, F 2 represents the text feature matrix, F 3 represents the audio feature matrix;

[0076] Step 2.1: Use the visual pre-training model to train the i-th product v i Extract features from the visual content and obtain its visual feature vector where d v Represents the dimension of the visual feature vector; thus obtaining the visual feature matrix of the product set V

[0077] Step 2.2: Use the text pre-training model to train the i-th product v i Extract features from the text content and obtain its visual feature vector where d t Represents the dimension of the text feature vector; thus obtaining the text feature matrix of the product set V

[0078] Step 2.3: Use the visual pre-training model to train the i-th product v i Extract features from the visual content and obtain its visual feature vector where d a Represents the dimension of the audio feature vector; thus obtaining the audio feature matrix of the product set V

[0079] Step 3: According to the multimedia feature matrix set {F 1 , F 2 , F 3}Construct product association matrix:

[0080] Step 3.1: Calculate the similarity matrix set {S 1 , S k , S 3}:

[0081] S k =L2(F k )×(L2(F k )) T (1)

[0082] In formula (1), S k represents the similarity matrix of the kth mode of the product set V, L2(F k ) represents the normalized multimedia feature matrix of the kth modality of the product set V, T represents the matrix transpose, k = {1, 2, 3};

[0083] Step 3.2: According to formula (2), the similarity matrix set {S 1 , S k , S 3} is sparsely processed to obtain the association matrix set {S′ 1 , S′2 , S′ 3}:

[0084]

[0085] In formula (2), Represents the element value of the i-th row and j-th column of the similarity matrix of the k-th mode of the product set V, represents the element value of the i-th row and j-th column of the k-th mode of the product set V, represents all element values ​​of the ith row of the similarity matrix of the kth mode of the product set V, k = {1, 2, 3}; TopK′ represents the first K′ in descending order, K′ = 10;

[0086] Step 3.3: Calculate the association matrix S′ of the product set V according to formula (3):

[0087]

[0088] Step 3.4: Calculate the representation propagation matrix A according to formula (4):

[0089]

[0090] The normalized representation propagation matrix is ​​calculated by formula (5):

[0091]

[0092] In formula (5), D represents the degree matrix of matrix A;

[0093] Step 4: Learn the user and product representation matrix based on the deep graph neural model:

[0094] Step 4.1: Use Gaussian distribution to randomly initialize the user representation matrix P = {p1, ..., p a , ..., p M}, where p a represents the a-th user u a The d1+3d2-dimensional representation vector of , and d1=d2=64;

[0095] Use Gaussian distribution to randomly initialize the product co-representation matrix Q = {q1, ..., q i , ..., q N}, where q i represents the i-th product v i The d1-dimensional representation vector of

[0096] Step 4.2: Calculate the multimedia content representation matrix T of each modality in the product set V by formula (6) = [T 1 , T2 , T 3 ]:

[0097] T k =MLP k (F k ) (6)

[0098] In formula (5), MLP k is the kth multilayer perceptron, Represent the multimedia content representation matrix of the kth modality; calculate the product representation matrix E = [Q, T];

[0099] Step 4.3, define the graph neural network to have a total of L = 3 convolutional layers, the current convolutional layer is l, and initialize l = 0;

[0100] Initialize the node representation matrix of the lth layer Hl = {P, E};

[0101] Step 4.4: Input the node representation matrix Hl of the lth layer into the graph neural network, and calculate the node representation matrix Hl+1 of the l+1th layer according to formula (7): l+1 :

[0102]

[0103] Step 4.5: Aggregate the outputs of each graph convolutional layer according to formula (8) and obtain the final node representation matrix H:

[0104]

[0105] Step 4.6: According to formula (9), the user's final representation matrix H is obtained u And the final product characterization matrix H v :

[0106] H u =H[:M],H v =H[M:] (9) In formula (9), H[:M] represents the 1st to Mth rows of the matrix H, and H[M:] represents the (M+1)th to (M+N)th rows of the matrix H;

[0107] Step 5: Minimize the mutual information between multimedia content and its representation based on the information bottleneck theory and calculate the loss function:

[0108] Step 5.1: Use PCA algorithm to analyze the multimedia feature matrix set {F 1 , F 2 , F 3} to reduce the dimension and obtain the multimodal feature matrix {F′ 1 , F′ 2 , F′ 3},in, represents the multimedia feature matrix after dimension reduction of the kth modality;

[0109] Step 5.2: Construct the information bottleneck loss function L(θ1) according to formula (10):

[0110]

[0111] In formula (10), θ1 represents the parameters corresponding to K multi-layer perceptrons, HSIC(,) represents the Hilbert-Schmidt independence index;

[0112] Step 6: Reconstruct the interaction matrix based on the user and product representation matrices and calculate the loss function:

[0113] Step 6.1: Predict the ath user u according to formula (11): a For the i-th product v i The interaction probability r′ ai :

[0114]

[0115] In formula (11), δ is the sigmoid activation function; Represents the user's final representation matrix H u The ath row in represents the transpose of the vector; Represents the final product characterization matrix H u The i-th row in represents the vector;

[0116] Step 6.2: Calculate the reconstruction loss function L(θ2) according to formula (12):

[0117]

[0118] In formula (12), θ2 represents all parameters involved in reconstructing the interaction matrix, and θ2 = [P, Q, θ1]; D a is the ath user u a All training data of D a ={(i, j)|i∈R a , j∈VR a}; R a represents the a-th user u a A collection of interactive products; Represents the final product characterization matrix H v The j-th row vector in ;

[0119] Step 7: Combine various loss functions to perform multi-task learning to update model parameters until the model converges:

[0120] Step 7.1: Establish the multi-task optimization objective L(θ) according to formula (13):

[0121] L(θ)=L(θ1)+αL(θ2)+β||[P, Q,]|| 2 (13)

[0122] In formula (13), θ = [P, Q, θ2] is all the parameters to be optimized, α and β are used to adjust the weights of the information bottleneck loss function and the regularization term respectively;

[0123] Step 7.2: Solve the optimization target L(θ) by gradient descent method to update the parameter θ until L(θ) converges to the minimum value, thus obtaining the optimal parameter θ. * ;

[0124] Step 7.3: Predict the ath user u according to formula (14) a For the i-th product v i The optimal interaction probability Thus, we get the interaction matrix of user set U predicting product set V: To implement product recommendations:

[0125]

[0126] In formula (14), Denotes the optimal user representation matrix H u* The ath user u a The optimal representation vector of Represents the optimal product representation matrix H v* The i-th product v i The optimal representation vector of .

[0127] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0128] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0129] Example:

[0130] In order to verify the effectiveness of this method, the present invention selects three commonly used Amazon data sets for experiments, namely Clothing, Sports and Baby. For each user, a product that the user interacts with is randomly selected as the test set, and the remaining interaction data is used as the training set. For each user-product interaction record, a product that the user has not interacted with is randomly sampled to form a triple for model training; in the test phase, the present invention selects all products that the user has not interacted with for sorting, and uses the recall rate Recall@N and the normalized discounted cumulative gain Normalized Discounted Cumulative Gain (NDCG@N) as evaluation criteria.

[0131] This paper selects 5 related methods for effect comparison, namely BPR-MF, LightGCN, VBPR, LightGCN-M, and LATTICE. Specifically, Tables 1 to 3 show the experimental results on the above three datasets respectively.

[0132] Table 1 Comparison of recommendation results between the proposed method and the comparative method on the Clothing dataset

[0133]

[0134]

[0135] Table 2 Comparison of recommendation results between the proposed method and the comparative method on the Sports dataset

[0136] Models Recall@10 NDCG@10 Recall@20 NDCG@20 BPR-MF 0.0444 0.0247 0.0661 0.0304 LightGCN 0.0572 0.0324 0.0853 0.0398 VBPR 0.0487 0.0271 0.0731 0.0335 LightGCN+M 0.0635 0.0360 0.0952 0.0443 LATTICE 0.0650 0.0366 0.0969 0.0450 IBRec (the present invention) 0.0667 0.0379 0.1000 0.0465

[0137] Table 3 Comparison of recommendation results between the proposed method and the comparative method on the Baby dataset

[0138] Models Recall@10 NDCG@10 Recall@20 NDCG@20 BPR-MF 0.0376 0.0202 0.0612 0.0264 LightGCN 0.0493 0.0268 0.0773 0.0342 VBPR 0.0446 0.0248 0.0688 0.0311 LightGCN+M 0.0554 0.0303 0.0868 0.0385 LATTICE 0.0583 0.0320 0.0891 0.0400 IBRec (the present invention) 0.0590 0.0322 0.0914 0.0408

[0139] It can be seen from the above three tables that on the three data sets, the IBRec method proposed in the present invention is significantly better than the five compared methods in terms of the four indicators of Recall@10, NDCG@10, Recall@20 and NDCG@20.

Claims

1. A multimedia recommendation method based on information bottleneck denoising, characterized in that: The steps are as follows: Step 1: construct heterogeneous data, including: user interaction matrix for products, and multimedia content data of products; Step 1.1: Let U represent the user set, and U = {u1,…u a ,…,u b ,…u M },u a represents the ath user, u b represents the b-th user, M represents the total number of users, 1≤a,b≤M; let V represent the product set, and V={v1,…,v i ,…,v j ,…,v N },v i represents the i-th product, v j represents the jth product, N represents the total number of products, 1≤i,j≤N; let r ai represents the a-th user u a For the i-th product v i The interaction data of users on products, then the user interaction matrix R = {r ai } M×N ; Step 1.2, let C = {C 1 ,...,C k ,...,C K } represents the multimedia content of product set V, where Table C k represents the multimedia content of the kth modality in the product set V, and represents the multimedia content of the kth modality in the product set V, represents the i-th product v i The multimedia content of the kth modality, K represents the total number of modalities, 1≤k≤K; Step 2: Use the pre-trained model to extract product multimedia content features: Use the pre-trained models of each modality to extract features from the multimedia content of the corresponding modality, and obtain a set of multimedia feature matrices of K modalities {F 1 ,...,F k ,...,F K }; where F k Represents the multimedia feature matrix of the kth modality; Step 3: According to the multimedia feature matrix set {F 1 ,...,F k ,...,F K }Construct product association matrix: Step 3.1: Calculate the similarity matrix set {S 1 ,...,S k ,...,S K }: S k =L2(F k )×(L2(F k )) T (1) In formula (1), S k represents the similarity matrix of the kth mode in the product set V, L2(F k ) represents the normalized multimedia feature matrix of the kth modality in the product set V, and T represents the matrix transpose; Step 3.2: According to formula (2), the similarity matrix set {S 1 ,...,S k ,...,S K } is sparsely processed to obtain the association matrix set {S' 1 ,...,S' k ,...,S' K }: In formula (2), Represents the similarity matrix S of the kth mode in the product set V k The element value of the i-th row and j-th column of Represents the association matrix S' of the kth mode in the product set V k The element value of the i-th row and j-th column of Represents the similarity matrix S of the kth mode in the product set V k All element values ​​of the i-th row, TopK' represents the first K' in descending order; Step 3.3, calculate the association matrix S' of the product set V according to formula (3): Step 3.4: Calculate the representation propagation matrix A according to formula (4): The normalized representation propagation matrix is ​​calculated by formula (5): In formula (5), D represents the degree matrix of matrix A; Step 4: Learn the user and product representation matrix based on the deep graph neural model: Step 4.1: Use Gaussian distribution to randomly initialize the user representation matrix P = {p1, ..., p a ,...,p M }, where p a represents the a-th user u a d1+Kd2-dimensional representation vector; where d1 represents the user collaborative representation dimension, and d2 represents the content representation dimension; Use Gaussian distribution to randomly initialize the product co-representation matrix Q = {q1,...,q i ,...,q N }, where q i represents the i-th product v j The d1-dimensional representation vector of Step 4.2: Calculate the multimedia content representation matrix T of each modality in the product set V by formula (6) = [T 1 ,...,T k ,...,T K ]: T k =MLP k (F k ) (6) In formula (5), MLP k is the kth multilayer perceptron, represents the multimedia content representation matrix of the kth modality; Step 4.3: Construct the product representation matrix E = [Q, T]; and use the graph neural network to process {P, E} to obtain the user final representation matrix H u And the final product characterization matrix H v ; Step 5: Minimize the mutual information between multimedia content and its representation based on the information bottleneck theory and calculate the loss function: Step 5.1: Use PCA algorithm to analyze the multimedia feature matrix set {F 1 ,...,F k ,...,F K } to reduce the dimension and obtain the multimodal feature matrix {F' 1 ,...,F' k ,...,F' K },in, represents the kth mode F k The multimedia feature matrix after dimensionality reduction; Step 5.2: Construct the information bottleneck loss function L(θ1) according to formula (10): In formula (10), θ1 represents the parameters corresponding to K multi-layer perceptrons, and HSIC(,) represents the Hilbert-Schmidt independence index; Step 6: Reconstruct the interaction matrix based on the user and product representation matrices and calculate the loss function: Step 6.1: Predict the ath user u according to formula (11): a For the i-th product v i The interaction probability r' ai : In formula (11), δ is the sigmoid activation function; Represents the user's final representation matrix H u The ath row in represents the transpose of the vector; Represents the final product characterization matrix H u The i-th row in represents the vector; Step 6.2: Calculate the reconstruction loss function L(θ2) according to formula (12): In formula (12), θ2 represents all parameters involved in reconstructing the interaction matrix, and θ2 = [P, Q, θ1]; D a is the ath user u a All training data of D a ={(i,j)|i∈R a ,j∈VR a }; R a represents the a-th user u a A collection of interactive products; Represents the final product characterization matrix H v The j-th row vector in ; Step 7: Combine various loss functions to perform multi-task learning to update model parameters until the model converges: Step 7.1: Establish the multi-task optimization objective L(θ) according to formula (13): L(θ)=L(θ1)+αL(θ2)+β||[P,Q,]|| 2 (13) In formula (13), θ = [P, Q, θ2] is all the parameters to be optimized, α and β are used to adjust the weights of the information bottleneck loss function and the regularization term respectively; Step 7.2: Solve the optimization target L(θ) by gradient descent method to update the parameter θ until L(θ) converges to the minimum value, thereby obtaining the optimal parameter θ. * ; Step 7.3: Predict the ath user u according to formula (14) a For the i-th product v i The optimal interaction probability Thus, we get the interaction matrix of user set U predicting product set V: To implement product recommendations: In formula (14), Denotes the optimal user representation matrix H u* The ath user u a The optimal representation vector of Represents the optimal product representation matrix H v* The i-th product v i The optimal representation vector of .

2. The multimedia recommendation method based on information bottleneck denoising according to claim 1, characterized in that: The graph neural network in step 4.3 has a total of L convolutional layers, the current convolutional layer is l, and l=0 is initialized; Initialize the l-th layer node representation matrix H l ={P,E}; The l-th layer node representation matrix H l It is sent into the graph neural network, and the l+1th layer node representation matrix H is calculated according to formula (7): l+1 : According to formula (8), the output of each graph convolution layer is aggregated, and the final node representation matrix H is obtained: According to formula (9), the user's final representation matrix H is obtained u And the final product characterization matrix H v : H u =H[:M],H v =H[M:] (9) In formula (9), H[:M] represents the 1st to Mth rows of the matrix H, and H[M:] represents the (M+1)th to (M+N)th rows of the matrix H.

3. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the multimedia recommendation method according to claim 1 or 2, and the processor is configured to execute the program stored in the memory.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multimedia recommendation method according to claim 1 or 2 are executed.

Images