Recommendation method based on graph convolutional neural network

Abstract

The invention provides a recommendation method based on a graph convolutional neural network. The recommendation method comprises the following steps: constructing an adjacent matrix A based on a user-project historical interaction matrix R; constructing a residual module with initial feature connection to capture a high-dimensional cooperative signal; projecting the original visual and text features into the same vector space; under the guidance of the ID features, redundant information is filtered out through a nonlinear activation function; an article-article relation graph of modal perception is constructed based on a KNN sparse technology, and modal features of articles are enhanced; a behavior-guided multi-modal feature mining module is introduced and is used for extracting a multi-modal cooperative signal; introducing a hyper-parameter gamma to fuse the image feature Ev and the text feature Et, predicting the most likely favorite article of the user u, and recommending the first N items with the highest score to the user; and for all-stage self-supervised learning, obtaining an all-stage self-supervised learning loss function # imgabs0 #, and optimizing the all-stage self-supervised learning loss function on the basis of Bayesian personalized ranking loss.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal recommendation algorithms, and in particular, to a recommendation method based on a graph convolutional neural network. Background Art

[0002] The multimodal recommendation task is a popular research direction that combines multimodal learning and recommendation systems. With the rise of graph convolutional neural networks, multimodal recommendation algorithms have made breakthrough developments. Researchers have constructed user-item interaction graphs and item-item relationship graphs to enhance item features and explore user preferences. Multimodal recommendation is based on the user's historical interaction information and the multimodal content features of items, and selects the N items that the user is most likely to like from a large number of items to be recommended. The input of this task has three categories, namely the user's historical interaction matrix and the image and text features of the item, and the output is the item ID recommended to the user.

[0003] The commonly used collaborative filtering method in traditional recommendation systems directly recommends items that the other party has not interacted with but has interacted with after identifying similar users. Figure 1 As shown in the figure. This method does not fully explore the multimodal preferences of users, but simply recommends based on the historical interaction information of users, ignoring the content characteristics of the items themselves. The multimodal recommendation algorithm first constructs a user-item bipartite graph based on the historical interaction information of users, then learns and integrates the preferences of similar users based on the text and image features of the items, and finally recommends items with fused features to users.

[0004] Early multimodal recommendation methods were mainly based on matrix decomposition. [1] , such as VBPR and CKE. Matrix decomposition is a classic algorithm in recommendation systems. It makes recommendations by decomposing the user-item rating matrix into a user feature matrix and an item feature matrix. First, the algorithm iteratively solves the existing user-item rating data to obtain the latent vectors of each user and item. Then the inner product of the user vector and the item vector is calculated to predict and sort the user's ratings of uninteracted items to generate a recommendation list. Matrix decomposition is one of the commonly used algorithms in recommendation systems. However, this method tends to recommend items that have been interacted with by users with similar behaviors, and is easily affected by sparse data. To address these limitations, researchers introduced a multimodal recommendation system based on graph convolutional neural networks.

[0005] Collaborative filtering [2]It is a recommendation algorithm based on user behavior, which is mainly divided into two categories: user-based collaborative filtering and item-based collaborative filtering. User-based collaborative filtering analyzes the user's historical behavior (such as ratings, clicks, purchases, etc.) to find other users with similar interests to the target user, and then recommends items that these users like to the target user. Item-based collaborative filtering analyzes the similarity between items and recommends other items that are similar to items that the user liked in the past. Commonly used similarity metrics include cosine similarity and Pearson correlation coefficient.

[0006] Due to the compatibility between graph convolutional neural networks and collaborative filtering algorithms, the latest research on multimodal recommendation systems often uses deep learning and graph learning techniques to incorporate modality-based information as auxiliary information into item representations. This approach improves the performance of traditional ID-based recommendation algorithms, especially in the case of sparse historical interactions and cold start problems. In order to obtain more accurate recommendation results, researchers have also constructed auxiliary graphs, such as item-item graphs based on item semantic similarity. [3] , user-user graph based on user similarity [4] However, existing multimodal recommendation methods only construct historical interaction graphs between users and items, without fully utilizing the multimodal features of items. In addition, the modal features contain a lot of noise, which will be amplified during the message propagation process, which will bring challenges to the accuracy of recommendations.

[0007] References

[0008] [1]Ruining He and Julian McAuley.2016.VBPR:visual bayesianpersonalized ranking from implicit feedback.In Proceedings of the AAAIconference on artificial intelligence,Vol.30.

[0009] [2]Harris Papadakis,Antonis Papagrigoriou,Costas Panagiotakis,etal.2022.Collaborative filtering recommender systems taxonomy.Knowledge andInformation Systems 64,1(2022),35-74.

[0010] [3]Penghang Yu, Zhiyi Tan, Guanming Lu, et al. 2023. Multi-view graphconvolutional network for multimedia recommendation. In Proceedings of the31st ACM International Conference on Multimedia.6576-6585.

[0011] [4] Qifan Wang, Yinwei Wei, Jianhua Yin, et al. 2021. Dual GNN: Dual graph neural network for multimedia recommendation. IEEE Transactions on Multimedia 25 (2021), 1074-1084. Summary of the invention

[0012] According to the technical problems mentioned in the above background technology, a recommendation method based on graph convolutional neural network is provided. The present invention utilizes three parallel graph convolution processes to mine and fuse multiple high-dimensional collaborative signals from user-item interaction graphs, item-item graphs in image modality, and item-item graphs in text modality. In addition, the present invention designs a full-stage self-supervised learning strategy to supervise multiple stages of representation learning, remove redundant information, and improve the accuracy of recommendations. Since the present method relies on hyperparameters for feature fusion and prunes the edges in the graph network in the feature preprocessing part, reducing the number of model parameters, the algorithm requires little storage space and has strong application capabilities.

[0013] The technical means adopted by the present invention are as follows:

[0014] A recommendation method based on graph convolutional neural network, comprising the following steps:

[0015] Step 1: Construct an adjacency matrix A based on the user-item historical interaction matrix R, remove the noise edges in the user-item interaction graph G at a ratio of ρ, and construct a pruned adjacency matrix A based on the retained edges ρ ;

[0016] Step 2: Construct a residual module with initial feature connections to capture high-dimensional collaborative signals;

[0017] Step 3: Project the original visual and text features into the same vector space; under the guidance of the ID features, filter out redundant information through a nonlinear activation function;

[0018] Step 4: Construct a modality-aware item-item relationship graph based on KNN sparsification technology to enhance the modal characteristics of the items;

[0019] Step 5: Introduce a behavior-guided multimodal feature mining module to extract multimodal collaborative signals;

[0020] Step 6: Introduce hyperparameter γ to fuse image features E v and text feature E t , predict the items that user u is most likely to like, and recommend the top N items with the highest scores to the user;

[0021] Step 7: Obtain the loss function of self-supervised learning for all stages

[0022] Step 8: Optimize the self-supervised learning loss function of the entire stage based on the Bayesian personalized ranking loss.

[0023] Furthermore, in step 1, the adjacency matrix A is:

[0024]

[0025] Among them, 0 represents an all-zero matrix; if user u interacts with item i, then A ui =1; otherwise A ui = 0; remove the noise edges in the user-item interaction graph G at a rate of ρ, then the number of edges retained is n = |ε|·(1-ρ), where |ε| is the total number of edges in the graph G. The probability of discarding each edge between nodes i and j is:

[0026]

[0027] Among them, d i ,d j ∈D represents the degree of nodes i and j; the probability of obtaining all edges p = {p0, p1, ... p |ε|-1}, the remaining edges are: ε m ={e0,e1,…e n-1}; where ε m Represents the n edges that are retained.

[0028] Furthermore, the step 2 includes the following steps:

[0029] Step 21: Concatenate the user and item IDs to get E id ; E id User-item interaction graph G after pruning ρ The formula for up propagation is:

[0030]

[0031] Among them, A ρ represents the pruned adjacency matrix; D ρ and denote the degree matrix and the normalized adjacency matrix respectively; and Represent the L-th layer and initial ID features respectively;

[0032] Step 22: Introduce the hyperparameter α to balance the influence of the initial features and The weight of is set to 1-α;

[0033] Step 23: Aggregate the features of each layer to get the final ID representation:

[0034]

[0035] Among them, L id Indicates the number of graph convolution layers; E id Represents the ID features of users and items.

[0036] Furthermore, the step 3 includes the following steps:

[0037] Step 31: Project the original visual and textual features into the same vector space:

[0038] E i,m =W1X i,m +b1;

[0039] Where m = {v, t}, represents the set of image and text modalities, Indicates the original characteristics of an item; and Represent the trainable transformation matrix and bias vector respectively;

[0040] Step 32: Under the guidance of ID features, use nonlinear activation function to filter out redundant information:

[0041]

[0042] in, represents the purified modal features; and They represent the trainable transformation matrix and bias vector respectively, ⊙ represents element-wise multiplication, and σ(·) represents the sigmoid function.

[0043] Furthermore, the step 4 includes the following steps:

[0044] Step 41: Calculate the similarity between items i and j using the cosine similarity function

[0045]

[0046] Among them, e m Represents the modal characteristics of an item;

[0047] Step 42: Keep the first k similar edges and construct the semantic similarity graph of items in image and text modalities:

[0048]

[0049] Step 43: Perform graph convolution on the purified features and enhance the features of item i by the k items most similar to item i:

[0050]

[0051] in, Denotes the enhanced item features, D i,m represents the degree matrix.

[0052] Furthermore, the step 5 includes the following steps:

[0053] Step 51: Generate the user’s initial preference by aggregating the modal features of the user’s neighbor nodes, i.e., the items they have interacted with.

[0054]

[0055] in, represents the number of neighbor nodes of the user, Indicates the enhanced features of the item;

[0056] Step 52: Construct a graph convolutional network based on modal features and aggregate:

[0057]

[0058] Among them, L m Indicates the number of layers of the modality feature convolutional network, E m Represent the multimodal features of users and items, m∈{v,t}.

[0059] Furthermore, the step 6 includes the following steps:

[0060] Step 61: Introduce hyperparameter γ to fuse image features E v and text feature E t :

[0061] E f =γE v +(1-γ)E t ;

[0062] Step 62: ID Feature E id and modal characteristics E f Fusion, get the final feature E:

[0063] E=E id +E f ;

[0064] Step 63: To predict the items that user u is most likely to like, calculate the score y for each item i u,i :

[0065]

[0066] Step 64: Recommend the top N items with the highest scores to the user.

[0067] Furthermore, the step 7 includes the following steps:

[0068] Step 71: Use the InfoNCE loss function for feature alignment, which is in the form of:

[0069]

[0070] Among them, C(A,B) represents InfoNCE loss; A and B represent different features; B + represents the positive sample related to A; B - represents negative samples; τ represents temperature parameter;

[0071] Step 72: Supervise the learning process at all stages by bringing positive sample pairs closer and negative sample pairs further away.

[0072] Furthermore, the step 72 includes the following steps:

[0073] Step 721: Before the modal graph convolution, use InfoNCE loss to align the enhanced image features and text features

[0074]

[0075] Among them, the subscript i represents an item;

[0076] Step 722: Use the text feature as an anchor point and initialize the ID feature before the ID graph convolution. With enhanced image features and text features Alignment:

[0077]

[0078] Step 723: The final ID features of the user and item id With the final modal characteristics e f To align:

[0079]

[0080] Step 724: Full-stage self-supervised learning loss for:

[0081]

[0082] Furthermore, the step 8 includes the following steps:

[0083] Step 81: Optimization function based on Bayesian personalized ranking BPR loss:

[0084]

[0085] In the triple (u,i,j), i and j represent the items that user u has interacted with and the items that he has not interacted with, respectively; u Represents user characteristics;

[0086] Step 82: Use BPR loss to constrain the modal features of users and items:

[0087]

[0088] Among them, x i,m and x j,m All represent the original modal characteristics of the item; then the total loss is:

[0089]

[0090] Among them, λ and μ both represent hyperparameters; represents the BPR loss function; represents the modal BPR loss function; Represents the InfoNCE loss function.

[0091] Compared with the prior art, the present invention has the following advantages:

[0092] The parallel graph structure proposed in this paper can effectively extract the user's preferences and the features of the items, and designs a full-stage self-supervised learning strategy for multimodal recommendation. This algorithm effectively improves the accuracy of the recommendation task and has strong application value. In addition, this algorithm designs a denoising module and hyperparameters to fuse different features, which solves the problem of large model parameters and slow training speed, and can ensure that the model can run effectively on devices such as mobile terminals with limited computing power and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0094] Figure 1 It is a traditional recommendation algorithm.

[0095] Figure 2 The figure is a flow chart of the algorithm of the present invention.

[0096] Figure 3 This is a schematic diagram of an example of the effects recommended by the present invention. DETAILED DESCRIPTION

[0097] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0098] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0099] like Figure 1-3 As shown, the present invention provides a recommendation method based on a graph convolutional neural network, comprising the following steps:

[0100] Step 1: Construct an adjacency matrix A based on the user-item historical interaction matrix R, remove the noise edges in the user-item interaction graph G at a ratio of ρ, and construct a pruned adjacency matrix A based on the retained edges ρ The adjacency matrix A is:

[0101]

[0102] Among them, 0 represents an all-zero matrix; if user u interacts with item i, then A ui =1; otherwise A ui = 0; remove the noise edges in the user-item interaction graph G at a rate of ρ, then the number of edges retained is n = |ε|·(1-ρ), where |ε| is the total number of edges in the graph G. The probability of discarding each edge between nodes i and j is:

[0103]

[0104] Among them, d i ,d j ∈D represents the degree of nodes i and j; the probability of obtaining all edges p = {p0, p1, ... p |ε|-1}, the remaining edges are: ε m ={e0,e1,…e n-1}; where ε m Represents the n edges that are retained.

[0105] Preferably, step 2: constructing a residual module with initial feature connection to capture high-dimensional collaborative signals; comprising the following steps:

[0106] Step 21: Concatenate the user and item IDs to get E id ; E id User-item interaction graph G after pruning ρ The formula for up propagation is:

[0107]

[0108] Among them, A ρ represents the pruned adjacency matrix; D ρ and denote the degree matrix and the normalized adjacency matrix respectively; and Represent the L-th layer and initial ID features respectively;

[0109] Step 22: Introduce the hyperparameter α to balance the influence of the initial features and The weight of is set to 1-α;

[0110] Step 23: Aggregate the features of each layer to get the final ID representation:

[0111]

[0112] Among them, L id Indicates the number of graph convolution layers; E id Represents the ID features of users and items.

[0113] As a preferred implementation, step 3: projecting the original visual and text features into the same vector space; under the guidance of the ID feature, filtering out redundant information through a nonlinear activation function. The following steps are included:

[0114] Step 31: Project the original visual and textual features into the same vector space:

[0115] E i,m =W1X i,m +b1;

[0116] Where m = {v, t}, represents the set of image and text modalities, Indicates the original characteristics of an item; and Represent the trainable transformation matrix and bias vector respectively;

[0117] Step 32: Under the guidance of ID features, use nonlinear activation function to filter out redundant information:

[0118]

[0119] in, represents the purified modal features; and They represent the trainable transformation matrix and bias vector respectively, ⊙ represents element-wise multiplication, and σ(·) represents the sigmoid function.

[0120] In this application, step 4: constructing a modality-aware item-item relationship graph based on KNN sparsification technology to enhance the modal features of the items; includes the following steps:

[0121] Step 41: Calculate the similarity between items i and j using the cosine similarity function

[0122]

[0123] Among them, e m Represents the modal characteristics of an item;

[0124] Step 42: Keep the first k similar edges and construct the semantic similarity graph of items in image and text modalities:

[0125]

[0126] Step 43: Perform graph convolution on the purified features and enhance the features of item i by the k items most similar to item i:

[0127]

[0128] in, Denotes the enhanced item features, D i,m represents the degree matrix.

[0129] Preferably, step 5: introducing a behavior-guided multimodal feature mining module to extract multimodal collaborative signals;

[0130] Furthermore, the step 5 includes the following steps:

[0131] Step 51: Generate the user’s initial preference by aggregating the modal features of the user’s neighbor nodes, i.e., the items they have interacted with.

[0132]

[0133] in, represents the number of user neighbor nodes, Indicates the enhanced features of the item;

[0134] Step 52: Construct a graph convolutional network based on modal features and aggregate:

[0135]

[0136] Among them, L m Indicates the number of layers of the modality feature convolutional network, E m Represent the multimodal features of users and items, m∈{v,t}.

[0137] Step 6: Introduce hyperparameter γ to fuse image features E v and text features E t , predict the items that user u is most likely to like, and recommend the top N items with the highest scores to the user; further, the step 6 includes the following steps:

[0138] Step 61: Introduce hyperparameter γ to fuse image features E v and text feature E t :

[0139] E f =γE v +(1-γ)E t ;

[0140] Step 62: ID Feature E id and modal characteristics E f Fusion, get the final feature E:

[0141] E=E id +E f ;

[0142] Step 63: To predict the items that user u is most likely to like, calculate the score y for each item i u,i :

[0143]

[0144] Step 64: Recommend the top N items with the highest scores to the user.

[0145] Step 7: Obtain the loss function of self-supervised learning for all stages Furthermore, the step 7 includes the following steps:

[0146] Step 71: Use the InfoNCE loss function for feature alignment, which is in the form of:

[0147]

[0148] Among them, C(A,B) represents InfoNCE loss; A and B represent different features; B + represents the positive sample related to A; B - represents negative samples; τ represents temperature parameter;

[0149] Step 72: Supervise the learning process at all stages by bringing the positive sample pairs closer and the negative sample pairs farther away. Specifically:

[0150] Step 721: Before the modal graph convolution, use InfoNCE loss to align the enhanced image features and text features

[0151]

[0152] Among them, the subscript i represents an item;

[0153] Step 722: Use the text feature as an anchor point and initialize the ID feature before the ID graph convolution. With enhanced image features and text features Alignment:

[0154]

[0155] Step 723: The final ID features of the user and item id With the final modal characteristics e f To align:

[0156]

[0157] Step 724: Full-stage self-supervised learning loss for:

[0158]

[0159] Step 8: Optimize the self-supervised learning loss function of the whole stage based on the Bayesian personalized ranking loss. Further, the step 8 includes the following steps:

[0160] Step 81: Optimization function based on Bayesian personalized ranking BPR loss:

[0161]

[0162] In the triple (u,i,j), i and j represent the items that user u has interacted with and the items that he has not interacted with, respectively; u Represents user characteristics;

[0163] Step 82: Use BPR loss to constrain the modal features of users and items:

[0164]

[0165] Among them, x i,m and x j,m All represent the original modal characteristics of the item; then the total loss is:

[0166]

[0167] Among them, λ and μ both represent hyperparameters; represents the BPR loss function; represents the modal BPR loss function; Represents the InfoNCE loss function.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A recommendation method based on graph convolutional neural network, characterized in that: The following steps are involved: Step 1: Construct an adjacency matrix A based on the user-item historical interaction matrix R, remove the noise edges in the user-item interaction graph G at a ratio of ρ, and construct a pruned adjacency matrix A based on the retained edges ρ ; Step 2: Construct a residual module with initial feature connections to capture high-dimensional collaborative signals; Step 3: Project the original visual and text features into the same vector space; under the guidance of the ID features, filter out redundant information through a nonlinear activation function; Step 4: Construct a modality-aware item-item relationship graph based on KNN sparsification technology to enhance the modal characteristics of the items; Step 5: Introduce a behavior-guided multimodal feature mining module to extract multimodal collaborative signals; Step 6: Introduce hyperparameter γ to fuse image features E v and text feature E t , predict the items that user u is most likely to like, and recommend the top N items with the highest scores to the user; Step 7: For the self-supervised learning of the whole stage, obtain the self-supervised learning loss function L cl ; Step 8: Optimize the self-supervised learning loss function of the entire stage based on the Bayesian personalized ranking loss.

2. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: In step 1, the adjacency matrix A is: Among them, 0 represents an all-zero matrix; if user u interacts with item i, then A ui =1; otherwise A ui = 0; remove the noise edges in the user-item interaction graph G at a rate of ρ, then the number of edges retained is n = |ε|·(1-ρ), where |ε| is the total number of edges in the graph G; the probability of discarding each edge between nodes i and j is: Among them, d i ,d j ∈D represents the degree of nodes i and j; the probability of obtaining all edges p = {p0, p1, ... p |ε|-1 }, the remaining edges are: ε n ={e0,e1,…e n-1 }; where ε m Represents the n edges that are retained.

3. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 2 includes the following steps: Step 21: Concatenate the user and item IDs to get E id ; E id User-item interaction graph G after pruning ρ The formula for up propagation is: Among them, A ρ represents the pruned adjacency matrix; D ρ and denote the degree matrix and the normalized adjacency matrix respectively; and Represent the L-th layer and initial ID features respectively; Step 22: Introduce the hyperparameter α to balance the influence of the initial features and The weight of is set to 1-α; Step 23: Aggregate the features of each layer to get the final ID representation: Among them, L id Indicates the number of graph convolution layers; E id Represents the ID features of users and items.

4. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 3 includes the following steps: Step 31: Project the original visual and textual features into the same vector space: AND i,m =W1X i,m +b1; Where m = {v, t}, represents the set of image and text modalities, Indicates the original characteristics of an item; and Represent the trainable transformation matrix and bias vector respectively; Step 32: Under the guidance of ID features, use nonlinear activation function to filter out redundant information: in, represents the purified modal features; and They represent the trainable transformation matrix and bias vector respectively, ⊙ represents element-wise multiplication, and σ(·) represents the sigmoid function.

5. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 4 includes the following steps: Step 41: Calculate the similarity between items i and j using the cosine similarity function Among them, e m Represents the modal characteristics of an item; Step 42: Keep the first k similar edges and construct the semantic similarity graph of items in image and text modalities: Step 43: Perform graph convolution on the purified features and enhance the features of item i by the k items most similar to item i: in, Denotes the enhanced item features, D i,m represents the degree matrix.

6. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 5 includes the following steps: Step 51: Generate the user’s initial preference by aggregating the modal features of the user’s neighbor nodes, i.e., the items they have interacted with. in, represents the number of user neighbor nodes, Indicates the enhanced features of the item; Step 52: Construct a graph convolutional network based on modal features and aggregate: Among them, L m Indicates the number of layers of the modality feature convolutional network, E m Represent the multimodal features of users and items, m∈{v,t}.

7. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 6 includes the following steps: Step 61: Introduce hyperparameter γ to fuse image features E v and text feature E t : THAT f =γE v +(1−γ)E t 4 Step 62: ID Feature E id and modal characteristics E f Combined, we get the final feature E: E=E id +E f ; Step 63: To predict the items that user u is most likely to like, calculate the score y for each item i u,i : Step 64: Recommend the top N items with the highest scores to the user.

8. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 7 includes the following steps: Step 71: Use the InfoNCE loss function for feature alignment, which is in the form of: Among them, C(A,B) represents InfoNCE loss; A and B represent different features; B + represents the positive sample related to A; B - represents negative samples; τ represents temperature parameter; Step 72: Supervise the learning process at all stages by bringing positive sample pairs closer and negative sample pairs further away.

9. The recommendation method based on graph convolutional neural network according to claim 8, characterized in that: The step 72 includes the following steps: Step 721: Before the modal graph convolution, use InfoNCE loss to align the enhanced image features and text features Among them, the subscript i represents an item; Step 722: Use the text feature as an anchor point and initialize the ID feature before the ID graph convolution. With enhanced image features and text features Alignment: Step 723: The final ID features of the user and item id With the final modal characteristics e f To align: Step 724: Full-stage self-supervised learning loss for:

10. The recommendation method based on graph convolutional neural network according to claim 1, characterized in that: The step 8 includes the following steps: Step 81: Optimization function based on Bayesian personalized ranking BPR loss: In the triple (u,i,j), i and j represent the items that user u has interacted with and the items that he has not interacted with, respectively; u Represents user characteristics; Step 82: Use BPR loss to constrain the modal features of users and items: Among them, x i,m and x j,m All represent the original modal characteristics of the item; then the total loss is: Among them, λ and μ both represent hyperparameters; represents the BPR loss function; represents the modal BPR loss function; Represents the InfoNCE loss function.

Images