Image recommendation method based on graph neural network

Abstract

The invention discloses an image recommendation method based on a graph neural network, and relates to the technical field of recommendation systems, and the method comprises the following steps: constructing model input, constructing a graph neural network model, comparing a learning module, carrying out joint training and optimization, and carrying out recommendation generation and evaluation. According to the method, the relationship embedding matrix in the knowledge graph is considered into the graph neural network model, the splicing vector of the knowledge graph is used as a construction expansion node embedding vector input model, and the complex relationship between the user and the image can be deeply mined through the graph neural network and comparative learning in combination with the interaction graph, so that the user preference is accurately captured, and the user experience is improved. Therefore, the recommendation accuracy is remarkably improved, the multi-dimensional features of the user and the image nodes are expanded, the context sensing technology is combined, the method can provide highly personalized recommendation services for the user, the unique requirements of the user are met, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of recommendation systems, and in particular to an image recommendation method based on a graph neural network. Background Art

[0002] In the current Internet and e-commerce fields, image recommendation systems have become an important bridge connecting users with rich multimedia content. However, traditional image recommendation algorithms often have the following significant defects when faced with massive data and complex user behaviors: Traditional image recommendation algorithms mostly rely on simple collaborative filtering or content-based recommendation strategies. These methods have limitations in capturing user preferences and image features. They are often unable to deeply explore the complex interactive relationship between users and images, and lack sufficient contextual information to accurately understand the real needs of users. As a result, the recommendation results often deviate from user expectations and are difficult to meet the personalized needs of users.

[0003] Existing recommendation systems tend to recommend images that are highly similar to the user's historical interaction behaviors. Although such recommendations guarantee accuracy to a certain extent, they ignore the diversity and novelty of recommendations. While enjoying personalized recommendation services, users are also eager to discover more unknown content related to their interests. The shortcomings of traditional algorithms in recommendation diversity limit the possibility of users exploring new content and affect user experience.

[0004] In the interaction process between users and images, there is a rich relationship network, including social relationships between users, similarity relationships between images, and interaction behaviors between users and images. Traditional recommendation algorithms are unable to cope with these complex relationships and find it difficult to fully explore and utilize the potential information behind these relationships. This not only limits the performance improvement of recommendation algorithms, but also hinders the development of recommendation systems towards intelligence and personalization.

[0005] In summary, a more efficient and intelligent recommendation algorithm is urgently needed to meet the current challenges. The image recommendation method based on graph neural network proposed in this invention solves the above problems. Summary of the invention

[0006] The purpose of the present invention is to make up for the shortcomings of the prior art and to provide an image recommendation method based on graph neural network. It can construct a user-item interaction graph and a knowledge graph as input, and combine graph neural network with comparative learning. It can deeply explore the complex relationship between users and images, accurately capture user preferences, and thus significantly improve the accuracy of recommendations, expand the multi-dimensional features of user and image nodes, and combine with context-aware technology. This method can provide users with highly personalized recommendation services, meet users' unique needs, and improve user experience.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides an image recommendation method based on a graph neural network, the method comprising the following steps: S100, build model input: Construct a user-item interaction graph that contains users and images as nodes. The nodes in the interaction graph are divided into two categories: one is user nodes and the other is image nodes. The edges represent the interaction between users and images, including but not limited to likes, browsing, and collections. Each edge is assigned a weight to form an initial adjacency matrix. The initial adjacency matrix represents the user-item interaction graph. Obtain a knowledge graph containing image and user related entities, wherein the entities in the knowledge graph are image attributes and user attributes and the relationships between the image attributes and user attributes; wherein the image attributes include but are not limited to author, style and theme, and the user attributes include but are not limited to age, gender and interests; Define a relation embedding matrix R, where the elements in the relation embedding matrix represent the similarity and connection strength between two nodes; have two relation embedding matrices, one representing the relationship within the attributes of the image, and the other representing the relationship within the attributes of the user; For each image node , extract relevant attribute information from the image’s knowledge graph and concatenate them into the image’s embedding vector As the extended node embedding vector, similarly, for the user node , extracting attribute information from the user’s knowledge graph and concatenating it into a user embedding vector as an extended node embedding vector; S200, build graph neural network model: by and , the user-item interaction graph is used as the input of the graph neural network model; the graph neural network model is composed of multiple layers of GNN stacking, and the update rule of GNN is:

[0008] in, Representation Node In the Embedding representation of the layer; Integer, starting from 0; represents nodes, including image nodes and user nodes; m is the neighbor node of v; For the The bias term of the layer; For the The weight matrix of the layer; is the activation function; Indicates Relationship embedding between layer node u and neighbor node m; represents the normalization constant; Indicates that node m is in The embedding representation of the layer; when for or ; is the attention coefficient The expression is: , in, is the weight vector in the attention mechanism, is the feature transformation matrix in the attention mechanism, Represents a splicing operation, is the activation function, Representation Node The embedding representation of Representation Node Neighbor nodes The embedding representation of Indicates that the node Neighbor nodes Neighbor nodes other than ; The final user embedding is represented as , the image embedding is represented as ,P is the number of layers of the graph neural network model; In the contrastive learning module, S300 extracts samples from the user behavior sequence, generates positive sample pairs by randomly flipping and cropping the user's historical interactive images, and generates negative sample pairs by randomly flipping and cropping the user's historical non-interactive images. The contrastive loss function is used for contrastive learning tasks, so that the model can effectively distinguish positive sample pairs from negative sample pairs during the optimization process. S400, joint training and optimization: Combine the contrastive learning module with the graph neural network model to jointly optimize the embedding representation of users and images through joint training. During the training process, the model parameters are continuously updated iteratively until the preset convergence conditions are met or the optimal performance is achieved; S500, recommendation generation and evaluation: based on the optimized vector representation of the user and the image, the similarity and matching degree between the user and the non-interacted image are calculated, and the similarity or matching degree is sorted to generate a personalized recommendation list.

[0009] Furthermore, before starting to build the user-item interaction graph, a comprehensive check is performed on the collected user data and image data to eliminate incomplete, duplicate and erroneous data records, where invalid user data includes but is not limited to long-term inactive user accounts and user records with serious information missing, and invalid image data includes but is not limited to images that cannot be loaded, copyright issues and low-quality images. The retained valid data is standardized, including but not limited to unified user ID format and image format and resolution, and the node types in the graph are clarified, including user nodes and image nodes, where user nodes represent independent user entities and image nodes represent specific image content with which users interact. The interaction behaviors between users and images are defined as edges in the graph, including but not limited to likes, browsing and collections. For each interaction behavior, a calculation rule for edge weights is formulated to reflect the intensity and frequency of interaction between users and images.

[0010] Furthermore, based on the cleaned user data and image data, a user node list and an image node list are generated respectively, and a unique identifier is assigned to each node. According to the interaction behavior between the user and the image, an adjacency matrix is ​​initialized to represent the user-item interaction graph. Each element in the matrix represents the interaction weight between the user and the image, and non-interaction is set to 0.

[0011] Furthermore, the specific steps of S300 are: S301, construction of positive and negative sample pairs: Positive sample pairs: They are generated by randomly flipping and cropping the images of the user's historical interactions. , whose historical interactive image set is , then the positive sample pair satisfy and Negative sample pairs: are non-interactive images generated by random flipping and cropping, that is, for users , randomly selected ,and Constitute a negative sample pair ; S302, contrast loss function: Use the normalized cross entropy loss in contrastive learning to distinguish positive sample pairs from negative sample pairs. Suppose image The representation vector is , the contrast loss function is defined as: in, Representation vector and The similarity between them is calculated using cosine similarity, that is, , is the temperature coefficient, which is used to control the concentration of the similarity distribution, Representing images A set of negative samples; S303, optimization process: During the training process, the contrastive learning module is jointly trained with the graph neural network model, and the model parameters are updated through the back-propagation algorithm to minimize the contrastive loss function. As the training progresses, the model can learn more refined node representations, thereby improving the accuracy and diversity of recommendations.

[0012] Furthermore, the S400 stage combines the contrastive learning module with the graph neural network model to jointly optimize the vector representation of users and images through joint training. The specific steps are as follows: Initialize model parameters: Before starting training, you need to initialize all weight matrices in the graph neural network model and related parameters in the contrastive learning module; Forward propagation: Use the extended node embedding vector constructed in step S100 as the input of the graph neural network model, and calculate the deep representation vector of each user and image through iterative updates of multi-layer GNN, which is recorded as user node The embedding representation of , image node The embedding representation of ,calculate Loss function; calculate Loss function: ,in , are the scores of the user on the positive and negative samples respectively; using the positive and negative sample pairs constructed in step S300, the contrast loss function is calculated; Backpropagation and optimization: Use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, update the parameters of the graph neural network model and the contrastive learning module to minimize the contrastive loss function, and repeat the above steps until the preset convergence conditions are met.

[0013] Furthermore, the specific content of S500 is: in the S500 stage, the personalized recommendation list is generated using the optimized user and image vector representations, and the recommendation results are evaluated. The specific steps are as follows: S501, calculate similarity and matching degree: for each user , calculate its difference with the uninteracted image The similarity or matching degree between them is directly calculated using the cosine similarity between the optimized vector representations; S502, generating a recommendation list: sorting the non-interacted images according to similarity or matching, selecting the image with the highest similarity or matching as the recommendation result, and generating a personalized recommendation list; S503, evaluate recommendation results: use multiple evaluation indicators to evaluate the recommendation results, including but not limited to accuracy, recall, F1 score and NDCG.

[0014] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program can implement the steps of the method when executed by a processor.

[0015] Compared with the existing technology, this image recommendation method based on graph neural network has the following beneficial effects: 1. The present invention constructs a user-item interaction graph and uses the knowledge graph to construct an extended node embedding vector and a relationship embedding matrix, so as to make full use of the rich information in the knowledge graph to expand the attribute information of user and image nodes. This not only enhances the context perception ability of the model, but also enables the model to have a deeper understanding of user preferences and image features. Combined with the graph neural network model and the contrastive learning module, the model can learn the deep representation of users and images, capture the complex relationships between nodes and the topological structure information of the graph. This combination of deep learning and contrastive learning significantly improves the accuracy of recommendations, making the recommendation results more in line with the personalized needs of users.

[0016] 2. By extracting positive sample pairs and negative sample pairs from user behavior sequences and adopting contrastive loss function for training, the model can distinguish between positive sample pairs and negative sample pairs, thereby enhancing the ability to recognize subtle differences. This ability enables the model to not only consider the user's known preferences when making recommendations, but also explore new content that the user may be interested in but has not yet discovered. It can generate more diverse and novel recommendation lists, help users discover new points of interest, and improve user experience.

[0017] 3. The present invention conducts a comprehensive check on the collected user data and image data, eliminates incomplete, duplicate and erroneous data records, and standardizes the retained valid data. This step ensures the consistency and compatibility of the data and provides a high-quality data foundation for subsequent model training. In addition, by constructing a user-item interaction graph and defining edge weight calculation rules, the method can flexibly handle different types of user and image interaction behaviors, and reflect the user's preference based on the intensity and frequency of the interaction. This flexibility and scalability enable the method to adapt to recommendation scenarios of different scales and complexities, improving the robustness and practicality of the recommendation system.

[0018] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flowchart of the overall architecture of the image recommendation method based on graph neural network of the present invention; Figure 2 This is a flowchart of the comparative learning module of the image recommendation method based on graph neural network of the present invention.

[0021] Figure 3 This is the overall model framework diagram of the image recommendation method based on graph neural network in the present invention, where the user node is , the image node is , the processed image node is , and are the user's knowledge graph and the image's knowledge graph, respectively. is the initial vector after the user knowledge graph is expanded, is the initial vector after image knowledge graph expansion, is the initial vector after random cropping and inversion of the image, and Embedded representations of final users and images, is the final embedded representation of the image after processing. Improve the relevance and accuracy of recommendations by maximizing the differences in users' preferences for images. By learning image similarities, the model is made more robust to different interferences. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The present invention provides an image recommendation method based on a graph neural network, and the method comprises the following steps: S100, build model input: Construct a user-item interaction graph that contains users and images as nodes. The nodes in the interaction graph are divided into two categories: one is user nodes and the other is image nodes. The edges represent the interaction between users and images, including but not limited to likes, browsing, and collections. Each edge is assigned a weight to form an initial adjacency matrix. The initial adjacency matrix represents the user-item interaction graph. Obtain a knowledge graph containing image and user related entities, wherein the entities in the knowledge graph are image attributes and user attributes and the relationships between the image attributes and user attributes; wherein the image attributes include but are not limited to author, style and theme, and the user attributes include but are not limited to age, gender and interests; Define a relation embedding matrix R, where the elements in the relation embedding matrix represent the similarity and connection strength between two nodes; have two relation embedding matrices, one representing the relationship within the attributes of the image, and the other representing the relationship within the attributes of the user; For each image node , extract relevant attribute information from the image’s knowledge graph and concatenate them into the image’s embedding vector As the extended node embedding vector, similarly, for the user node , extracting attribute information from the user’s knowledge graph and concatenating it into a user embedding vector as an extended node embedding vector; The knowledge graph is used to expand the attribute information of image and user nodes, and their splicing features are input into the graph neural network model as the extended node embedding vector. Through the entity and relationship links in the knowledge graph, a relationship embedding matrix is ​​constructed to add more dimensional features to the nodes in the graph network, enhance the representation distinction ability, and thus enhance the model's context perception ability.

[0024] In the present invention, image features such as color, texture, object, etc. can be extracted through a CNN network.

[0025] S200, build graph neural network model: by and , the user-item interaction graph is used as the input of the graph neural network model; the graph neural network model is composed of multiple layers of GNN stacking, and the update rule of GNN is:

[0026] in, Representation Node In the Embedding representation of the layer; Integer, starting from 0; represents nodes, including image nodes and user nodes; m is the neighbor node of v; For the The bias term of the layer; For the The weight matrix of the layer; is the activation function; Indicates Relationship embedding between layer node u and neighbor node m; represents the normalization constant; Indicates that node m is in The embedding representation of the layer; when for or ; is the attention coefficient for: , in, is the weight vector in the attention mechanism, is the feature transformation matrix in the attention mechanism, Represents a splicing operation, is the activation function, Representation Node The embedding representation of Representation Node Neighbor nodes The embedding representation of Indicates that the node Neighbor nodes Neighbor nodes other than ; The final user embedding is represented as , the image embedding is represented as ,P is the number of layers of the graph neural network model; The graph neural network model in the present invention is composed of multiple layers of GNN. Each layer of GNN learns the deep representation of users and images through the components of graph convolution layer and graph attention mechanism, and captures the complex relationship between nodes and the topological structure information of the graph.

[0027] S300, contrastive learning module: extract positive sample pairs (i.e., image pairs of user history interactions, generated by randomly flipping and cropping the images of user history interactions) and negative sample pairs (i.e., randomly combined non-interaction image pairs, generated by randomly flipping and cropping) from the user behavior sequence; use contrastive loss function to contrast learning tasks, so that the model can distinguish positive sample pairs from negative sample pairs during the optimization process, enhance the model's ability to recognize subtle differences, and further improve the accuracy and diversity of recommendations; S400, joint training and optimization: Combine the contrastive learning module with the graph neural network model to jointly optimize the embedding representation of users and images through joint training. During the training process, the model parameters are continuously updated iteratively until the preset convergence conditions are met or the optimal performance is achieved; S500, recommendation generation and evaluation: Based on the optimized user and image embedding representation, calculate the similarity and matching between the user and the non-interacted image, sort according to the similarity or matching, and generate a personalized recommendation list. Use multiple evaluation indicators including but not limited to accuracy, recall, F1 score and NDCG to evaluate the recommendation results.

[0028] Furthermore, before starting to build the user-item interaction graph, a comprehensive check is performed on the collected user data and image data to eliminate incomplete, duplicate and erroneous data records, where invalid user data includes but is not limited to long-term inactive user accounts and user records with serious information missing, and invalid image data includes but is not limited to images that cannot be loaded, copyright issues and low-quality images. The retained valid data is standardized, including but not limited to unified user ID format and image format and resolution, etc., to ensure data consistency and compatibility, and the node types in the graph are clarified, including user nodes and image nodes, where user nodes represent independent user entities and image nodes represent specific image content with which users interact. The interaction behaviors between users and images are defined as edges in the graph, including but not limited to clicks, browsing and purchases. For each interaction behavior, a calculation rule for edge weights is formulated to reflect the intensity and frequency of interaction between users and images, which can be based on a comprehensive consideration of multiple factors such as the number of interactions and timestamps. For example, the more interactions or the closer the time, the higher the weight, indicating that the user has a higher preference for the image.

[0029] Furthermore, based on the cleaned user data and image data, a user node list and an image node list are generated respectively, and a unique identifier is assigned to each node. Based on the interaction between the user and the image, an adjacency matrix is ​​initialized to represent the user-item interaction graph. Each element in the matrix represents the interaction weight between the user and the image, and non-interaction is set to 0. With the in-depth processing of subsequent data, the matrix will be continuously updated and improved; The specific steps of S100 are: S101, build a user-item interaction graph: (1) Define nodes: The nodes in the graph are divided into two categories: one is user nodes , the other is image node , where N and M represent the total number of users and images, respectively; (2) Define edges: Edges represent the interaction between users and images, including but not limited to clicks, browsing, and purchases. Each edge can be assigned a weight. , indicating the intensity or frequency of interaction, the interaction graph can be represented as an adjacency matrix ,in Indicates user With image The interaction weight between them is set to 0 if there is no interaction; S102, extended node attribute information: (1) Knowledge graph construction: First, there needs to be an image knowledge graph containing image-related entities and a user knowledge graph containing user-related entities. The entities in the knowledge graph are the attributes of the image and the attributes of the user, as well as the relationship between them and the user and the image. The attributes of the image include but are not limited to the author, style and theme, and the attributes of the user include but are not limited to age, gender and interests. The relationship between them and the user and the image includes but is not limited to image-created-by-author and user-like-style; Through the relational links in the knowledge graph, we further consider the contextual information between nodes, that is, if two images are created by the same author, they have similar representations in the embedding space, and define the relational embedding matrix R. The elements in the relational embedding matrix represent the similarity and connection strength between the two nodes; there are two relational embedding matrices, one representing the relationship between the attributes of the image, and the other representing the relationship between the attributes of the user.

[0030] (2) Node feature extraction: For each image node , extract relevant attribute information from the knowledge graph: author vector and style vector , and concatenate them into the image embedding vector , similarly, for user nodes , extract its attribute information: age and gender and concatenate them into user embedding vector ; The extracted node embedding vector is used as the initial embedding of the graph network node. , whose initial embedding is , for user nodes , whose initial embedding is ; Furthermore, the specific steps of S200 are: S201, define the graph neural network model: the extended embedding vectors of the user and image nodes obtained in step S100, record the user node The embedding vector of , image node The embedding vector of The graph neural network model consists of multiple GNN layers stacked together to learn the deep representation of nodes layer by layer; each layer of GNN includes a graph convolution layer and a graph attention layer. S202, graph convolution layer: The output of the original graph convolutional layer is defined as: , S203, graph attention layer: The graph attention layer can assign different attention weights to different neighbor nodes, so as to better capture the complex relationship between nodes. In the graph attention layer, nodes Every neighbor Calculate an attention coefficient , weighted aggregation of neighbor features, specifically: , in, is the weight vector of the attention mechanism, is the feature transformation matrix, Represents a splicing operation, is the activation function, Representation Node The embedding representation of Representation Node Neighbor nodes The embedding representation of Indicates that the node Neighbor nodes Neighbor nodes outside.

[0031] The attention coefficient obtained by the graph attention layer and the relationship embedding matrix obtained by the knowledge graph are added to the graph convolutional network to update the node representation. The update rule can be expressed as: , in, Representation Node In the Embedding representation of the layer; Integer, starting from 0; represents nodes, including image nodes and user nodes; m is the neighbor node of v; For the The bias term of the layer; For the The weight matrix of the layer; is the activation function; Indicates Relationship embedding between layer node u and neighbor node m; represents the normalization constant; Indicates that node m is in The embedding representation of the layer; when for or , Indicates that node m is in Embedding representation of the layer; S204, stacked GNN: stack multiple GNNs as needed. Through iterative updates of each layer, the node representation can gradually capture more graph structure information and context features. S205, output layer: The output of the multi-layer GNN is used as the final node representation for subsequent recommendation tasks. In this embodiment, a total of 3 layers of network structure are set, and the final user embedding representation is , the image embedding is represented as , is the number of layers of the graph neural network model, here we take ,when The input layer embedding representation of the model is represented as .

[0032] Furthermore, the specific steps of S300 are: S301, construction of positive and negative sample pairs: Positive sample pairs: They are generated by randomly flipping and cropping the images of the user's historical interactions. , whose historical interactive image set is , then the positive sample pair satisfy and Negative sample pairs: are non-interactive images generated by random flipping and cropping, that is, for users , randomly selected ,and Constitute a negative sample pair ; S302, contrast loss function: Use the normalized cross entropy loss in contrastive learning to distinguish positive sample pairs from negative sample pairs. Suppose image The representation vector is , the contrast loss function can be defined as: ,in, Representation vector and The similarity between is usually calculated by cosine similarity, that is, , is the temperature coefficient, which is used to control the concentration of the similarity distribution, Representing images The set of negative samples.

[0033] S303, optimization process: During the training process, the contrastive learning module is jointly trained with the graph neural network model, and the model parameters are updated through the back-propagation algorithm to minimize the contrastive loss function. As the training progresses, the model can learn more refined node representations, thereby improving the accuracy and diversity of recommendations.

[0034] Furthermore, the derivation process of the contrast loss function in S302 is as follows: Assume that negative samples, for positive samples , the denominator in the loss function can be expanded as: , therefore, the contrastive loss can be rewritten as: , this loss function encourages the model to maximize the similarity between the representation vectors of positive sample pairs and minimize the similarity between negative sample pairs.

[0035] Furthermore, the S400 stage combines the contrastive learning module with the graph neural network model to jointly optimize the embedding representation of users and images through joint training. The specific steps are as follows: Initialize model parameters: Before starting training, you need to initialize all weight matrices and bias items in the graph neural network model, as well as the feature transformation matrix and related parameters in the contrastive learning module (temperature coefficient ); Forward propagation: Use the extended node embedding representation constructed in steps S100 and S200 as the input of the graph neural network model, and calculate the deep representation vector of each user and image (denoted as user node The embedding representation of , image node The embedding representation of ), using the final user and image representation obtained in step S200, calculate Loss function; calculate Loss function: Mainly used to solve the recommendation problem of implicit feedback. Its goal is to optimize the model by maximizing the user's preference for positive samples relative to the preference for negative samples. The specific formula is: ,in , are the user's scores for positive and negative samples respectively.

[0036] Using the positive and negative sample pairs constructed in step S300, calculate the contrast loss function; Calculate contrast loss: ,in, Representation vector and The similarity between is usually calculated by cosine similarity, that is, , is the temperature coefficient, which is used to control the concentration of the similarity distribution, Representing images The set of negative samples.

[0037] Back propagation and optimization: Use the back propagation algorithm to calculate the gradient of the loss function with respect to the model parameters, update the parameters of the GNN and contrastive learning module to minimize the contrastive loss function, and repeat the above steps until the preset convergence conditions are met.

[0038] Furthermore, the specific content of S500 is: in the S500 stage, the optimized user and image embedding representations are used to generate a personalized recommendation list, and the recommendation results are evaluated. The specific steps are as follows: S501, calculating similarity and matching degree: for each user u, calculating the similarity or matching degree between it and the non-interacted image i. The similarity can be directly calculated using the cosine similarity between the optimized embedding representations; S502, generating a recommendation list: sorting the non-interacted images according to similarity or matching, selecting the image with the highest similarity or matching as the recommendation result, and generating a personalized recommendation list; S503, evaluate the recommendation results: use a variety of evaluation indicators to evaluate the recommendation results, including but not limited to precision, recall, F1 score and NDCG, where the precision measures the proportion of relevant images in the recommendation list, the recall measures the proportion of relevant images recommended, the F1 score is the harmonic mean of the precision and recall, and the NDCG takes into account the position and relevance of the image in the recommendation list.

[0039] Example: Data preparation: Collect user data and image data. For example, if there are 100 users and 1,000 images, clean the data and remove incomplete, duplicate, and erroneous data records. Assume that 10 invalid user data (such as long-term inactive user accounts and user records with serious information missing) and 50 invalid image data (such as images that cannot be loaded, copyright issues, and low-quality images) are removed, and the retained valid data is standardized to unify the user ID format and image format and resolution, etc. Constructing a user-item interaction graph: clarifying node types, including 90 user nodes ) and 950 image nodes ), define the edge, the interaction between users and images includes click, browse and purchase, assuming that users With image There is a click interaction, the number of interactions is 5, and the time is in the last week. Based on the number of interactions and timestamps, the edge weight is set to 0.8 and the adjacency matrix is ​​initialized. , represents the user-item interaction graph, for example, , set to 0 if non-interactive; Expand node attribute information: Build a knowledge graph that includes image and user related entities. For example, image attributes include author, style, and theme, user attributes include age, gender, and interests, and the relationship between them (such as image-created-by-author and user-like-style). , extract relevant attribute information from the knowledge graph of the image: such as the author vector and style vector , and concatenate them into an embedded representation of the image , for user nodes , extract the user's attribute information from the user's knowledge graph: age is 25 years old, gender is male, and concatenate them into a user embedding representation (Assume that 1 represents male and 0 represents female); Fusion of knowledge graph information: Feature enhancement, for image nodes , whose initial embedding is ; For user nodes , whose initial embedding is , context-aware, through relational links in the knowledge graph, e.g., discovering images and If they are created by the same author, they have similar representations in the embedding space, and their relationship embedding It reflects the similarity and connection strength between two images.

[0040] Constructing a graph neural network model: defining the graph neural network structure, assuming that the user node The embedding vector of , image node The embedding vector of , The graph convolution layer uses a graph convolutional network (GCN) to update the node representation. For example, for the node , its neighbor node set is , the normalization constant , , No. The weight matrix of the layer , bias term , activation function , Graph attention layer, for nodes Every neighbor Calculate an attention coefficient , for example, the weight vector of the attention mechanism , feature transformation matrix (Assumption), then: , introducing the attention coefficient and the relationship embedding matrix, the node In the The embedding of a layer is represented as: , The value starts from 0 and there are three levels in total.

[0041] The final user embedding is represented as , the image embedding is represented as , .

[0042] Contrastive learning module: construct positive and negative sample pairs. Positive sample pairs are generated by randomly flipping and cropping images of user historical interactions. For example, for user , whose historical interactive image set is , then the positive sample pair is satisfy and , negative samples are non-interactive images generated by random flipping and cropping, for example, randomly selected ,and Constitute a negative sample pair The contrast loss function is designed to distinguish positive sample pairs from negative sample pairs using the normalized cross entropy loss in contrastive learning. For images The representation vector is , for the image The representation vector is , temperature coefficient , then the contrast loss function is: , The calculation can be obtained: , assuming that the negative sample set , we can calculate: ; Joint training and optimization: Initialize model parameters, initialize all parameters in the graph neural network model, and compare related parameters in the learning module (temperature coefficient ), forward propagation, using the constructed extended node embedding vector as the input of the graph neural network model, through the iterative update of the multi-layer graph convolution layer and the graph attention layer, calculating the deep representation vector of each user and image, calculating the contrast loss, using the above contrast loss function formula to calculate the loss, back propagation and optimization, using the back propagation algorithm to calculate the gradient of the loss function with respect to the model parameters, updating the parameters of the graph neural network model and the contrastive learning module to minimize the contrast loss function, repeating the above steps until the preset convergence conditions are met; Recommendation generation and evaluation: Calculate similarity and matching for users , calculate its interaction with the last image The similarity between them is assumed to be: , , then the similarity is: , generate a recommendation list, sort the non-interacted images according to similarity, select the image with the highest similarity as the recommendation result, generate a personalized recommendation list, evaluate the recommendation results, and use multiple evaluation indicators to evaluate the recommendation results, such as precision, recall, F1 score and NDCG. Assuming that there are 10 images in the recommendation list, of which 6 are relevant images, the precision is 6 / 10=0.6. Assuming that there are 20 relevant images in total and 8 are recommended, the recall is 8 / 20=0.4. The F1 score is calculated based on the precision and recall rate, and the NDCG is calculated considering the position and relevance of the image in the recommendation list.

[0043] The knowledge graph in the present invention not only enhances the extracted node embedding vectors as the initial embedding of the graph network nodes, but also considers the contextual information between nodes through the relationship links in the knowledge graph. For example, by utilizing the relationship that two images are created by the same author, an attention mechanism is introduced into the graph neural network so that related images have similar representations in the embedding space.

[0044] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0045] Any matters not described in the present invention are applicable to the prior art.

Claims

1. An image recommendation method based on graph neural network, characterized in that: The method comprises the following steps: S100, build model input: Construct a user-item interaction graph that contains users and images as nodes. The nodes in the interaction graph are divided into two categories: one is user nodes and the other is image nodes. The edges represent the interaction between users and images, including but not limited to likes, browsing, and collections. Each edge is assigned a weight to form an initial adjacency matrix. The initial adjacency matrix represents the user-item interaction graph. Obtain a knowledge graph containing image and user related entities, wherein the entities in the knowledge graph are image attributes and user attributes and the relationships between the image attributes and user attributes; wherein the image attributes include but are not limited to author, style and theme, and the user attributes include but are not limited to age, gender and interests; Define a relation embedding matrix R, where the elements in the relation embedding matrix represent the similarity and connection strength between two nodes; have two relation embedding matrices, one representing the relationship within the attributes of the image, and the other representing the relationship within the attributes of the user; For each image node , extract relevant attribute information from the image’s knowledge graph and concatenate them into the image’s embedding vector As the extended node embedding vector, similarly, for the user node , extracting attribute information from the user’s knowledge graph and concatenating it into a user embedding vector as an extended node embedding vector; S200, build graph neural network model: by and , the user-item interaction graph is used as the input of the graph neural network model; the graph neural network model is composed of multiple layers of GNN stacking, and the update rule of GNN is: in, Representation Node In the Embedding representation of the layer; Integer, starting from 0; represents nodes, including image nodes and user nodes; m is the neighbor node of v; For the The bias term of the layer; For the The weight matrix of the layer; is the activation function; Indicates Relationship embedding between layer node u and neighbor node m; represents the normalization constant; Indicates that node m is in The embedding representation of the layer; when for or ; is the attention coefficient The expression is: ， in, is the weight vector in the attention mechanism, is the feature transformation matrix in the attention mechanism, Represents a splicing operation, is the activation function, Representation Node The embedding representation of Representation Node Neighbor nodes The embedding representation of Indicates that the node Neighbor nodes Neighbor nodes other than ; The final user embedding is represented as , the image embedding is represented as ,P is the number of layers of the graph neural network model; In the contrastive learning module, S300 extracts samples from the user behavior sequence, generates positive sample pairs by randomly flipping and cropping the user's historical interactive images, and generates negative sample pairs by randomly flipping and cropping the user's historical non-interactive images. The contrastive loss function is used for contrastive learning tasks, so that the model can effectively distinguish positive sample pairs from negative sample pairs during the optimization process. S400, joint training and optimization: Combine the contrastive learning module with the graph neural network model to jointly optimize the embedding representation of users and images through joint training. During the training process, the model parameters are continuously updated iteratively until the preset convergence conditions are met or the optimal performance is achieved; S500, recommendation generation and evaluation: based on the optimized vector representation of the user and the image, the similarity and matching degree between the user and the non-interacted image are calculated, and the similarity or matching degree is sorted to generate a personalized recommendation list.

2. The image recommendation method based on graph neural network according to claim 1, characterized in that: Before starting to build the user-item interaction graph, conduct a comprehensive check on the collected user data and image data to eliminate incomplete, duplicate and erroneous data records. Invalid user data includes but is not limited to long-term inactive user accounts and user records with serious information missing. Invalid image data includes but is not limited to images that cannot be loaded, copyright issues and low-quality images. The retained valid data is standardized, including but not limited to unified user ID format and image format and resolution. The node types in the graph are clarified, including user nodes and image nodes, where user nodes represent independent user entities and image nodes represent specific image content that users interact with. The interaction behaviors between users and images are defined as edges in the graph, including but not limited to likes, browsing and collections. For each interaction behavior, a calculation rule for edge weights is formulated to reflect the intensity and frequency of interaction between users and images.

3. The image recommendation method based on graph neural network according to claim 2, characterized in that: Based on the cleaned user data and image data, a user node list and an image node list are generated respectively, and a unique identifier is assigned to each node. According to the interaction behavior between the user and the image, an adjacency matrix is ​​initialized to represent the user-item interaction graph. Each element in the matrix represents the interaction weight between the user and the image, and non-interaction is set to 0.

4. The image recommendation method based on graph neural network according to claim 1, characterized in that: The specific steps of S300 are: S301, construction of positive and negative sample pairs: Positive sample pairs: They are generated by randomly flipping and cropping the images of the user's historical interactions. , whose historical interactive image set is , then the positive sample pair satisfy and Negative sample pairs: are non-interactive images generated by random flipping and cropping, that is, for users , randomly selected ,and Constitute a negative sample pair ; S302, contrast loss function: Use the normalized cross entropy loss in contrastive learning to distinguish positive sample pairs from negative sample pairs. Suppose image The representation vector is , the contrast loss function is defined as: in, Representation vector and The similarity between them is calculated using cosine similarity, that is, , is the temperature coefficient, which is used to control the concentration of the similarity distribution, Representing images A set of negative samples; S303, optimization process: During the training process, the contrastive learning module is jointly trained with the graph neural network model, and the model parameters are updated through the back-propagation algorithm to minimize the contrastive loss function. As the training progresses, the model can learn more refined node representations, thereby improving the accuracy and diversity of recommendations.

5. The image recommendation method based on graph neural network according to claim 1, characterized in that: The S400 stage combines the contrastive learning module with the graph neural network model to jointly optimize the vector representation of users and images through joint training. The specific steps are as follows: Initialize model parameters: Before starting training, you need to initialize all weight matrices in the graph neural network model and related parameters in the contrastive learning module; Forward propagation: Use the extended node embedding vector constructed in step S100 as the input of the graph neural network model, and calculate the deep representation vector of each user and image through iterative updates of multi-layer GNN, which is recorded as user node The embedding representation of , image node The embedding representation of ,calculate Loss function; calculate Loss function: ,in , are the scores of the user on the positive and negative samples respectively; using the positive and negative sample pairs constructed in step S300, the contrast loss function is calculated; Backpropagation and optimization: Use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, update the parameters of the graph neural network model and the contrastive learning module to minimize the contrastive loss function, and repeat the above steps until the preset convergence conditions are met.

6. The image recommendation method based on graph neural network according to claim 1, characterized in that: The specific content of S500 is: in the S500 stage, the optimized user and image vector representations are used to generate a personalized recommendation list, and the recommendation results are evaluated. The specific steps are as follows: S501, calculate similarity and matching degree: for each user , calculate its difference with the uninteracted image The similarity or matching degree between them is directly calculated using the cosine similarity between the optimized vector representations; S502, generating a recommendation list: sorting the non-interacted images according to similarity or matching, selecting the image with the highest similarity or matching as the recommendation result, and generating a personalized recommendation list; S503, evaluate recommendation results: use multiple evaluation indicators to evaluate the recommendation results, including but not limited to accuracy, recall, F1 score and NDCG.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of any method described in claims 1-5 can be implemented.

Images