Data recommendation method and device and data recommendation model processing method and device

Abstract

The invention relates to a data recommendation method which is suitable for a video recommendation scene. The method comprises the following steps: acquiring an article knowledge graph, an interaction graph between an object and an article, and modal content of the article; fusing the article knowledge graph and the interaction graph to obtain a collaborative knowledge graph, determining an article relation graph based on modal content of the article, and constructing an object co-occurrence graph based on the interaction graph; heterogeneous graph learning is carried out based on the interaction graph and the collaborative knowledge graph, and object features and article features are obtained; performing isomorphic composition learning based on the object co-occurrence graph and the object features to obtain a target object representation, and performing isomorphic composition learning based on the object relation graph and the object features to obtain a target object representation; and according to the target object representation of the target object and the target object representation of each object, performing object recommendation on the target object. By adopting the method, the recommendation performance of the strict cold start article can be improved on the premise of keeping the recommendation performance of the hot start article.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data recommendation method, apparatus, computer equipment, storage medium and computer program product, as well as a data recommendation model processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of graph neural networks, more and more people are paying attention to graph data. The graph in graph data is not a picture but a topological graph. This kind of graph is a data structure that uses points (Vertex / Node) and edges (Edge) to represent entities and relationships. Common graph data includes knowledge graphs. In data recommendation scenarios, as the amount of data and interactive features become richer, knowledge graphs are sometimes introduced to help data recommendation in order to improve the accuracy of data recommendation.

[0003] Traditional recommendation methods that introduce knowledge graphs can enhance the representation learning of hot start items by introducing side information in the knowledge graph, and can achieve relatively good performance to a certain extent. However, for strictly cold start items, since there is no interactive data in both the training and application stages, such methods still cannot obtain effective feature representations for strictly cold start items, resulting in poor recommendation performance for strictly cold start items and facing a serious problem of imbalanced recommendation performance. Summary of the invention

[0004] Based on this, it is necessary to provide a data recommendation method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0005] In one aspect, the present application provides a data recommendation method. The method comprises:

[0006] Obtain the knowledge graph of items, the interaction graph between objects and items, and the modal content of items;

[0007] The item knowledge graph and the interaction graph are integrated to obtain a collaborative knowledge graph, an item relationship graph is determined based on the modal content of the items, and an object co-occurrence graph is constructed based on the interaction graph;

[0008] Perform heterogeneous graph learning based on interaction graphs and collaborative knowledge graphs to obtain object features and item features;

[0009] Performing isomorphic graph learning based on the object co-occurrence graph and the object features to obtain a target object representation, and performing isomorphic graph learning based on the item relationship graph and the item features to obtain a target item representation;

[0010] According to the target object representation of the target object and the target item representation of each item, an item recommendation is performed for the target object.

[0011] In some embodiments, the fusing the item knowledge graph and the interaction graph to obtain a collaborative knowledge graph includes:

[0012] Determine the edges in the interaction graph, and convert each edge in the interaction graph into a multi-tuple; the entities in the multi-tuple are objects and items connected by the edges in the interaction graph;

[0013] Based on the entities in the multi-group and the entities in the item knowledge graph, the interaction graph and the item knowledge graph are fused by entity alignment to obtain a collaborative knowledge graph.

[0014] In some embodiments, performing sparse processing on the first similarity matrix to obtain an unweighted matrix includes:

[0015] For any row in the first similarity matrix, sort the values ​​of multiple elements in the row from large to small;

[0016] The element with a ranking less than the preset ranking is taken as the first element, and the other elements are taken as the second element;

[0017] Update the value of the first element in the targeted row to a first value, and update the value of the second element in the targeted row to a second value, where the second value is smaller than the first value;

[0018] After the values ​​of the elements in each row of the first similarity matrix are updated, an unweighted matrix is ​​obtained.

[0019] In some embodiments, performing sparse processing on the second similarity matrix to obtain an object co-occurrence graph includes:

[0020] For any row in the second similarity matrix, sort the values ​​of multiple elements in the row from large to small;

[0021] The element with a ranking lower than the preset ranking is taken as the third element, and the other elements are taken as the fourth element;

[0022] The value of the third element in the targeted row is retained, and the value of the fourth element in the targeted row is updated to the second value;

[0023] After each row of the second similarity matrix is ​​updated, an object co-occurrence graph is obtained.

[0024] In some embodiments, determining the importance weight of each multi-tuple in the egocentric network includes:

[0025] For any multi-group in the egocentric network, determining the importance of the multi-group according to the identifier embedding of the entities and edges in the multi-group;

[0026] According to the importance of all multi-groups in the egocentric network, the importance of each multi-group is normalized to obtain the importance weight of each multi-group.

[0027] In some embodiments, for any object entity, aggregating the identification embedding of the object entity and the corresponding egocentric network representation to obtain a third object embedding of the object entity includes:

[0028] Performing an addition operation on the identity embedding of the targeted object entity and the corresponding egocentric network representation to obtain an addition operation result;

[0029] Performing element-by-element multiplication operation on the identity embedding of the targeted object entity and the corresponding egocentric network representation to obtain a multiplication result;

[0030] The addition operation result and the multiplication operation result are fused to obtain a third object embedding of the targeted object entity.

[0031] In some embodiments, fusing the object text representation and the object image representation to obtain a target object representation includes:

[0032] Performing multi-head attention processing based on the object text representation and the object image representation;

[0033] The results after multi-head attention processing are average pooled to obtain the target object representation.

[0034] In some embodiments, recommending items to the target object based on the target object representation and the target item representation of each item includes:

[0035] Get a target object representation of a target object;

[0036] The matching degree between the target object representation of the target object and the target object representation of each item is calculated, and the items whose matching degree meets the preset matching condition are recommended to the target object.

[0037] In some embodiments, the item includes hot start media data and strict cold start media data, the interaction graph between the object and the item includes an interaction graph between the user and the media data, and the modal content of the item includes text content and image content in the media data.

[0038] On the other hand, the present application also provides a data recommendation device. The device includes:

[0039] The acquisition module is used to obtain the item knowledge graph, the interaction graph between objects and items, and the modal content of items;

[0040] A construction module, used to fuse the item knowledge graph and the interaction graph to obtain a collaborative knowledge graph, determine an item relationship graph based on the modal content of the items, and construct an object co-occurrence graph based on the interaction graph;

[0041] Heterogeneous graph learning module, which is used to perform heterogeneous graph learning based on interaction graphs and collaborative knowledge graphs to obtain object features and item features;

[0042] A homogeneous graph learning module, configured to perform homogeneous graph learning based on the object co-occurrence graph and the object features to obtain a target object representation, and perform homogeneous graph learning based on the item relationship graph and the item features to obtain a target item representation;

[0043] The recommendation module is used to recommend items to the target object according to the target object representation of the target object and the target item representation of each item.

[0044] On the other hand, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the above data recommendation method when executing the computer program.

[0045] On the other hand, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above data recommendation method when executed by a processor.

[0046] On the other hand, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the data recommendation method is implemented.

[0047] The above-mentioned data recommendation method, device, computer equipment, storage medium and computer program product construct a collaborative knowledge graph based on the observed interaction graph between objects and items and the item knowledge graph, and obtain the object features and item features associated with the interaction through the way of side information perception, so as to reduce the influence of irrelevant side information on the recommendation performance of hot start items. In addition, an item relationship graph is constructed according to the modal content of the items, and an object co-occurrence graph is constructed according to the similarity of the interaction behavior between objects. Based on the item relationship graph and the object co-occurrence graph, the object features and item features encoding the interaction relationship between objects and items are propagated and aggregated, which can mine the association structure between hot start items and objects on the one hand, and transfer the features of hot start items to strict cold start items on the other hand, make up for the representation of strict cold start items due to the lack of interaction data, and can greatly improve the recommendation performance of strict cold start items. This application utilizes the modal content of items, the side information of item knowledge graph, and the interaction information in the interaction graph, and effectively improves the recommendation performance of strict cold start items under the premise of maintaining the recommendation performance of hot start items through the above-mentioned heterogeneous graph learning and homogeneous graph learning methods.

[0048] In one aspect, the present application provides a data recommendation model processing method. The method comprises:

[0049] Obtaining a sample item knowledge graph, a sample interaction graph between a sample object and a sample item, and a sample modal content of the sample item;

[0050] The sample item knowledge graph and the sample interaction graph are integrated to obtain a sample collaborative knowledge graph, a sample item relationship graph is determined based on the sample modal content, and a sample object co-occurrence graph is constructed based on the sample interaction graph;

[0051] Through the data recommendation model to be trained, heterogeneous graph learning is performed on the sample interaction graph and the sample collaborative knowledge graph to obtain sample object features and sample item features;

[0052] Through the data recommendation model to be trained, performing isomorphic graph learning on the sample object co-occurrence graph and the sample object features to obtain a predicted object representation, and performing isomorphic graph learning based on the sample item relationship graph and the sample item features to obtain a predicted item representation;

[0053] Determining a prediction loss according to the prediction object representation and the prediction item representation, and constructing a first loss function according to the prediction loss;

[0054] The data recommendation model to be trained is trained using the first loss function, and a data recommendation model is obtained after the training is completed.

[0055] In some embodiments, determining a sample target interaction graph based on the sample interaction graph includes:

[0056] Normalizing the sample interaction graph to obtain a normalized sample interaction graph;

[0057] determining auxiliary information according to the predicted object representation and the predicted item representation;

[0058] A sample-target interaction graph is determined according to the normalized sample interaction graph and the auxiliary information.

[0059] On the other hand, the present application also provides a data recommendation model processing device. The device includes:

[0060] An acquisition module, used to acquire a sample item knowledge graph, a sample interaction graph between a sample object and a sample item, and a sample modal content of the sample item;

[0061] A construction module, used to fuse the sample item knowledge graph and the sample interaction graph to obtain a sample collaborative knowledge graph, determine a sample item relationship graph based on the sample modal content, and construct a sample object co-occurrence graph based on the sample interaction graph;

[0062] A heterogeneous graph learning module, used to perform heterogeneous graph learning on the sample interaction graph and the sample collaborative knowledge graph through a data recommendation model to be trained, so as to obtain sample object features and sample item features;

[0063] A homogeneous graph learning module is used to perform homogeneous graph learning on the sample object co-occurrence graph and the sample object features through the data recommendation model to be trained to obtain a predicted object representation, and to perform homogeneous graph learning based on the sample item relationship graph and the sample item features to obtain a predicted item representation;

[0064] a determination module, configured to determine a prediction loss according to the prediction object representation and the prediction item representation, and construct a first loss function according to the prediction loss;

[0065] The training module is used to train the data recommendation model to be trained by using the first loss function, and obtain the data recommendation model after the training is completed.

[0066] On the other hand, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned data recommendation model processing method when executing the computer program.

[0067] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above-mentioned data recommendation model processing method when executed by a processor.

[0068] On the other hand, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the data recommendation model processing method is implemented.

[0069] The above-mentioned data recommendation model processing method, device, computer equipment, storage medium and computer program product construct a sample collaborative knowledge graph based on the sample interaction graph and sample item knowledge graph observed between the sample object and the sample item, and obtain the sample object features and sample item features associated with the interaction through the side information perception method, so as to reduce the influence of irrelevant side information on the recommendation performance of the hot start item. In addition, a sample item relationship graph is constructed according to the sample modal content of the sample item, and a sample object co-occurrence graph is constructed according to the similarity of the interaction behavior between the sample objects. Based on the sample item relationship graph and the sample object co-occurrence graph, the sample object features and sample item features encoding the interaction relationship between the sample object and the sample item are propagated and aggregated to obtain the predicted object representation and the predicted item representation. The prediction loss is determined according to the predicted object representation and the predicted item representation, and the data recommendation model to be trained is trained according to the prediction loss, and the data recommendation model is obtained after the training is completed. The data recommendation model trained in this way can, on the one hand, mine the association structure between the hot start item and the object, and on the other hand, transfer the features of the hot start item to the strict cold start item, make up for the representation of the strict cold start item due to the lack of interaction data, and can greatly improve the recommendation performance of the strict cold start item. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A diagram of an application environment for data recommendation and / or a data recommendation model processing method in one embodiment;

[0071] Figure 2 A schematic diagram of a flow chart of a data recommendation method in one embodiment;

[0072] Figure 3 A comparison of hot start items, general item cold start, and strict item cold start items in one embodiment;

[0073] Figure 4 is a schematic diagram of an item knowledge graph in one embodiment;

[0074] Figure 5 A schematic flow chart of steps for determining an item relationship graph based on modal contents of items in one embodiment;

[0075] Figure 6 A flowchart of steps for performing heterogeneous graph learning based on an interaction graph and a collaborative knowledge graph to obtain object features and item features in one embodiment;

[0076] Figure 7A flowchart of steps for performing graph attention processing based on a collaborative knowledge graph to obtain a knowledge-aware third object embedding and a third item embedding in one embodiment;

[0077] Figure 8 A flowchart of a step of determining a first weight corresponding to a second object text embedding and a second weight corresponding to a second object image embedding in one embodiment;

[0078] Fig. 9 It is a schematic diagram of the application of a data recommendation method in one embodiment;

[0079] Fig.10 A schematic diagram of a process flow of a data recommendation model processing method in one embodiment;

[0080] Fig.11 A schematic diagram of the principle of a data recommendation method in an embodiment;

[0081] Fig.12 is a structural block diagram of a data recommendation device in one embodiment;

[0082] Fig.13 It is a structural block diagram of a data recommendation model processing device in one embodiment;

[0083] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] Before describing the data recommendation method of this application in detail, some terms involved in this application are introduced first:

[0086] Warm-start recommendation: predicts items for which there is interaction data for users who have appeared in the training phase. Items recommended by warm start are also called hot start items.

[0087] Normal item cold-start recommendation: predicts new items in the test phase for users who have appeared in the training phase, and these items have interaction data in the test phase. The items recommended by normal item cold-start are also called normal cold-start items.

[0088] Strict item cold-start recommendation: predicts new items in the test phase for users who have appeared in the training phase, and there is no interaction data for these items in the test phase. Items recommended by strict item cold-start are also strict cold-start items.

[0089] The data recommendation method and / or data recommendation model processing method provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or integrated on the cloud or other servers. The terminal and the server can be used separately to execute the data recommendation method and / or data recommendation model processing method provided in the embodiment of the present application, or they can be used together to execute the data recommendation method and / or data recommendation model processing method provided in the embodiment of the present application.

[0090] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, portable wearable devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0091] The data recommendation method and data recommendation model processing method of this application are designed into machine learning in artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0092] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0093] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0094] The solution provided in the embodiments of the present application involves artificial intelligence, machine learning and other technologies, which are specifically described by the following embodiments:

[0095] In some embodiments, Figure 2 As shown, a data recommendation method is provided, which is applied to a computer device (the computer device may be Figure 1 The terminal or server in the example is used to illustrate, including the following steps:

[0096] Step 202, obtaining the item knowledge graph, the interaction graph between the object and the item, and the modal content of the item.

[0097] Among them, an item is a product with a specific meaning, which can be a physical product or a data object carried by an electronic device. Physical products include clothes, food, electrical appliances, or furniture; data objects carried by electronic devices include electronic coupons, promotional information, audio, video, images, or text.

[0098] In the field of data recommendation, items usually include hot start items, general cold start items, and strict cold start items. To understand the meaning of hot start items, general cold start items, and strict cold start items, please refer to Figure 3 , Figure 3The figure shows a comparison of hot start items, general cold start items, and strict cold start items. The solid color squares indicate that there are interaction records between the hot start items and the users, and the diagonal squares indicate that there are interaction records between the general cold start items and the users.

[0099] Knowledge Graph can be understood as a knowledge domain visualization or knowledge domain mapping map. It is a series of various graphics that show the knowledge development process and structural relationship. It uses visualization technology to describe knowledge resources and their carriers, and can show different knowledge and their mutual connections. know It is a knowledge graph built for various items, which contains various information about the items, such as the category of the items, the brand of the items, etc.

[0100] In an exemplary embodiment, the item knowledge graph may include four types of entities and six types of relationships. The specific description of each type of entity and relationship is as follows: Figure 4 Entities include: items (referring to products recommended to users), features (referring to product feature words from user reviews), brands (referring to product brands or manufacturers), and categories (product classifications). Relationships include: described as, produced, belongs to, purchased, browsed, and purchased at the same time. For an example of the item knowledge graph, please refer to Figure 4 It should be noted that since the "feature" entity from the review data may contain redundant and meaningless words that reflect the characteristics of the item, the TF-IDF algorithm (term frequency algorithm) can be used in the preprocessing stage to eliminate less significant features. For example, feature words with a frequency between 10 and 1000 and a TF-IDF score of 0.1 can be retained.

[0101] The object in this application is an object that has or may interact with an object in the future, usually a user. inter It is a graph used to record the interaction between users and items. The interaction between users and items, such as purchase, click, browse, watch or download, can reflect the user's preference for items to a certain extent. The interaction graph between objects and items can be derived from the interaction records. The computer device analyzes the interaction records, takes users and items as nodes, and takes the interaction relationship between users and items as edges to construct the interaction graph between objects and items.

[0102] The modal content of an item refers to the content of the item carried in a certain data form, such as the text content, image content, video content or audio content of the item. In this application, only one modal content of an item can be used for data recommendation, or multi-modal content of an item can be used for data recommendation, which is not limited in the embodiments of this application.

[0103] Step 204, integrate the item knowledge graph and the interaction graph to obtain a collaborative knowledge graph, determine the item relationship graph based on the modal content of the items, and construct an object co-occurrence graph based on the interaction graph.

[0104] Specifically, the computer device can first construct a heterogeneous graph between objects, items and entities based on the interaction graph between objects and items, the knowledge graph of items, and the modal characteristics of items, so as to encode the collaborative relationship between objects and items, and then construct an isomorphic graph between items and between objects, so as to respectively encode the internal structural associations between items and objects.

[0105] Among them, there is only one type of node and edge in the data in the homogeneous graph. The data in the heterogeneous graph can have more than one type of node and edge, allowing different types of nodes to have different dimensions of features or attributes.

[0106] In some embodiments, the computer device may treat each node in the item knowledge graph as an entity and each node in the interaction graph as an entity, thereby fusing the item knowledge graph and the interaction graph through entity alignment to obtain a collaborative knowledge graph.

[0107] In some embodiments, the computer device may encode the modal content of the item to obtain the corresponding modal features, and then calculate the similarity between the modal features of the two items. Thus, the modal adjacency matrix is ​​constructed according to the similarity between the two items, and the item relationship graph is represented by the modal adjacency matrix.

[0108] In some embodiments, assuming that there are N items, where N is a natural number greater than 1, the computer device may construct an N*N matrix, where the value of the jth element in the i-th row of the matrix is ​​the similarity between the i-th item and the j-th item. The N*N matrix may be used to represent an item relationship graph.

[0109] In some embodiments, the modal content of the item includes multimodal content, and the computer device can construct a corresponding item relationship graph for any modal content. For example, for text, the computer device can construct a text-based item relationship graph according to the text content of the item, and for images, the computer device can construct an image-based item relationship graph according to the image content of the item.

[0110] In other embodiments, the computer device may further encode the multimodal content of the item separately to obtain multimodal encoded data, thereby fusing the multimodal encoded data to obtain fused modal features, and then determining the item relationship graph based on the fused modal features, that is, the similarity may be calculated based on the fused modal features between two items. Thus, a neighbor matrix is ​​constructed based on the similarity between two items, and the item relationship graph is represented by the adjacency matrix.

[0111] For example, the multimodal content of an item includes text content (including title and description text), image content and audio content. The computer device can calculate the text features, image features and audio features respectively, fuse the text features, image features and audio features to obtain the fused modal features, and then determine the item relationship graph based on the fused modal features. Among them, for text encoding, the title and description text of the item can be spliced ​​into a complete sentence, and then Sentence-Transformers are used to extract text features. For image content, feature extraction can be performed through a convolutional neural network to obtain image features. For audio content, audio features can be extracted through an MFCC (Mel Frequency Cepstrum Coefficient) algorithm. Of course, other algorithms can also be used to extract modal features, and this is not limited to the embodiments of the present application.

[0112] Since objects that have interacted with similar items often have similar preferences, in some embodiments, the computer device can construct an object adjacency matrix to represent the object co-occurrence graph, and then learn the internal relationships between objects. Specifically, the computer device can determine the items that every two objects have interacted with based on the interaction graph. Assuming that there are M objects, where M is a natural number greater than 1, an M*M matrix can be constructed, and the value of the jth element in the i-th row of the matrix is ​​the number of items that the i-th object and the j-th object have interacted with. This M*M matrix can be used to represent the object co-occurrence graph.

[0113] Step 206, performing heterogeneous graph learning based on the interaction graph and the collaborative knowledge graph to obtain object features and item features.

[0114] Among them, both the interaction graph and the collaborative knowledge graph belong to heterogeneous graphs. Heterogeneous graph learning can specifically be graph convolution processing or graph attention processing based on heterogeneous graphs. Computer devices can perform side information-aware heterogeneous graph learning based on interaction graphs to obtain object embedding and item embedding; and perform side information-aware heterogeneous graph learning based on collaborative knowledge graphs to obtain object embedding and item embedding. Thus, the object embeddings obtained by different heterogeneous graph learning are integrated to obtain object features, and the item embeddings obtained by different heterogeneous graph learning are integrated to obtain item features.

[0115] In some embodiments, the computer device may perform graph convolution processing based on the interaction graph to obtain behavior-aware first object embedding and first item embedding; aggregate the modal features of the items on the interaction graph to obtain modality-aware second object embedding and second item embedding; and perform graph attention processing based on the collaborative knowledge graph to obtain knowledge-aware third object embedding and third item embedding.

[0116] Furthermore, the computer device may fuse at least two embeddings of the first object embedding, the second object embedding, or the third object embedding to obtain the object feature. At least two embeddings of the first item embedding, the second item embedding, or the third item embedding may be fuse to obtain the item feature.

[0117] The specific method of fusion may be weighted addition or splicing, etc., which is not limited in the embodiments of the present application.

[0118] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first item embedded can be referred to as a second item embedded, and similarly, a first object embedded can be referred to as a second object embedded.

[0119] Step 208, performing isomorphic graph learning based on the object co-occurrence graph and the object features to obtain the target object representation, and performing isomorphic graph learning based on the item relationship graph and the item features to obtain the target item representation.

[0120] Among them, both the object co-occurrence graph and the item relationship graph belong to isomorphic graphs. Isomorphic graph learning can specifically be based on isomorphic graphs for graph convolution processing or graph attention processing. Specifically, the computer device can transfer and aggregate neighbor information based on the constructed item relationship graph and object co-occurrence graph, thereby realizing the mining of potential associations between items and objects.

[0121] In some embodiments, considering that objects that share more interactive items with the central object should have a greater impact on it, a graph attention process is designed on the object co-occurrence graph to achieve message transmission between objects. The computer device can perform graph attention process based on the pre-constructed object co-occurrence graph and the object features learned based on the heterogeneous graph in step 206 to obtain the target object representation.

[0122] In some embodiments, the computer device may use a lightweight graph convolutional network to propagate and aggregate information on the item relationship graph based on a pre-constructed item relationship graph and the item features obtained based on the heterogeneous graph learning in step 206, that is, perform graph convolution processing to obtain a target item representation.

[0123] In some embodiments, the item relationship graph includes a text-based item relationship graph and an image-based item relationship graph. The computer device may perform graph convolution processing based on the text-based item relationship graph and the features of each item to obtain an item text representation; and perform graph convolution processing based on the image-based item relationship graph and the features of each item to obtain an item image representation. The item text representation and the item image representation are then fused to obtain a target item representation. The specific fusion method may be averaging, weighted summation or other methods, which are not limited in the embodiments of the present application.

[0124] Step 210 , recommending items to the target object based on the target object representation of the target object and the target item representations of each item.

[0125] Specifically, the computer device can perform pairwise similarity calculations on the target object representation and the target item representation, and for any target object, recommend items with a similarity greater than or equal to a preset similarity threshold, or the top K (K is a natural number greater than or equal to 1) items with the largest similarity, to the target object. The similarity can be specifically represented by cosine similarity, Euclidean distance, or Manhattan distance, etc., which is not limited in the embodiments of the present application.

[0126] In some embodiments, item recommendations are made to the target object based on the target object representation of the target object and the target item representation of each item, including: obtaining the target object representation of the target object; calculating the matching degree between the target object representation of the target object and the target item representation of each item respectively; and recommending items whose matching degree meets preset matching conditions to the target object.

[0127] Specifically, the computer device may calculate the similarity between the target object representation of the target object and the target object representation of each object, and determine the matching degree based on the similarity. For example, the similarity may be directly used as the matching degree, or the similarity may be mathematically calculated to obtain the matching degree.

[0128] Furthermore, the computer device can recommend items whose matching degree meets the preset matching condition to the target object. The preset matching condition can specifically be that the matching degree is greater than or equal to the preset matching degree threshold, or the top K (K is a natural number greater than or equal to 1) names with the largest matching degree, or, for different startup categories, the top P (P is a natural number greater than or equal to 1) names under the startup category. The startup categories include hot start, general cold start, and strict cold start.

[0129] In some embodiments, for different startup categories, the computer device may determine the recommended items under the startup category respectively. That is, for any startup category, the computer device may recommend the items under the startup category to the target object when the matching degree corresponding to the items under the startup category meets the preset matching condition. For example, for any startup category, the matching degrees of all items under the startup category may be sorted from large to small, and the top P items are recommended to the target object as recommended items.

[0130] In this way, by making item recommendations for the target object based on the matching degree between the target object representation and the target item representation, more matching hot start items and cold start items can be recommended to the user.

[0131] The above data recommendation method constructs a collaborative knowledge graph based on the observed interaction graph between objects and items and the item knowledge graph, and obtains the object features and item features associated with the interaction through the way of side information perception, thereby reducing the impact of irrelevant side information on the recommendation performance of hot start items. In addition, an item relationship graph is constructed according to the modal content of the items, and an object co-occurrence graph is constructed according to the similarity of the interaction behaviors between objects. Based on the item relationship graph and the object co-occurrence graph, the object features and item features that encode the interaction relationship between objects and items are propagated and aggregated. On the one hand, the association structure between hot start items and objects is respectively mined, and on the other hand, the features of hot start items are transferred to strict cold start items to make up for the representation of strict cold start items due to the lack of interaction data, which can greatly improve the recommendation performance of strict cold start items. This application utilizes the modal content of items, the side information of the item knowledge graph, and the interaction information in the interaction graph. Through the above-mentioned heterogeneous graph learning and isomorphic graph learning methods, the recommendation performance of strict cold start items is effectively improved while maintaining the recommendation performance of hot start items.

[0132] In some embodiments, the object knowledge graph and the interaction graph are fused to obtain a collaborative knowledge graph, including: determining the edges in the interaction graph, and converting each edge in the interaction graph into a tuple; the entities in the tuple are the objects and objects connected by the edges in the interaction graph; based on the entities in the tuple and the entities in the object knowledge graph, the interaction graph and the object knowledge graph are fused by entity alignment to obtain a collaborative knowledge graph.

[0133] Specifically, the computer device can find the edges in the interaction graph between the object and the item. The edges in the interaction graph connect the object and the items that have an interactive relationship with it. Then, each edge in the interaction graph is represented by a tuple. The tuple here can be a triple, that is, a triple (u, Interact, i). Among them, u in the triple refers to one of the objects, i refers to one of the items, and Interact refers to the interactive relationship between the object and the item. Then, the computer device can, based on the alignment relationship of the entity of the item, G inter With G know Integrate into a unified collaborative knowledge graph in And ε′=ε know ∪Interact, Represents the node combination of the item knowledge graph, Represents the set of nodes in the interaction graph, ε know Represents the edge set of the item knowledge graph.

[0134] In the above embodiment, by performing entity alignment on the entities of the objects, the object behavior and the object knowledge can be encoded into a unified relationship graph, namely, a collaborative knowledge graph.

[0135] In some embodiments, reference Figure 5 , determine the item relationship graph based on the modal content of the item, including:

[0136] Step 502, constructing an original relationship graph based on the modal content of the item.

[0137] In some embodiments, the computer device may encode the modal content of the items to obtain the corresponding modal features, and then calculate the similarity between the modal features of the two items, thereby constructing a modal adjacency matrix based on the similarity between the two items, and representing the original relationship graph through the adjacency matrix.

[0138] In some embodiments, the modal content of the item includes text content and image content, and the computer device can construct a text-based original relationship graph according to the text content of the item, and construct an image-based original relationship graph according to the image content of the item.

[0139] Step 504, determining a mask matrix according to the startup category corresponding to the item; the mask matrix is ​​used to isolate the information transfer from the strict cold start item to the hot start item.

[0140] It should be noted that the data recommendation method of the present application is applicable to the recommendation of hot start items, general cold start items and strict cold start items. Since there is no interaction record with the user when the strict cold start items are recommended, in the subsequent isomorphic graph learning process, the main learned item representation is learned from the hot start items, so a mask matrix needs to be added to shield the information from the strict cold start items to the hot start items, so as to ensure the recommendation performance for the hot start items.

[0141] In some embodiments, for the element in the ath row and bth column of the mask matrix, if the ath item is a hot start item and the bth item is a strict cold start item, the value of the element is 0; otherwise, the value of the element is 1. Specifically, the mask matrix can be expressed by Formula 1:

[0142]

[0143] Among them, I h is the hot start item set, I c is a strict cold start item set, a∈I h Indicates that the ath item is a hot start item, b∈I c Indicates that the bth item is a strict cold start item.

[0144] Step 506, determining the item relationship graph according to the original relationship graph and the mask matrix.

[0145] Specifically, the computer device may perform matrix multiplication on the original relationship graph and the mask matrix to obtain the item relationship graph. Specifically, the item relationship graph may be represented by Formula 2:

[0146]

[0147] in, Represents the original relationship graph, represents the mask matrix, and ⊙ represents the element-by-element multiplication of two matrices.

[0148] In some embodiments, if the modal content of the item is multimodal content, the original interaction graph constructed for each modal content can be multiplied with the mask matrix to obtain an item relationship graph corresponding to each modality.

[0149] In some embodiments, the modal content of the item includes text content and image content, and the computer device constructs a text-based original relationship graph according to the text content of the item, and constructs an image-based original relationship graph according to the image content of the item. According to the text-based original relationship graph and the mask matrix, determine the text-based item relationship graph Determine the image-based item relationship graph based on the image-based original relationship graph and the mask matrix

[0150] It should be noted that the data recommendation method can be executed through a data recommendation model. During the training phase of the data recommendation model, since no strict cold start items are introduced, the original relationship graph does not need to be corrected by the mask matrix, and the model can directly use the original relationship graph as the sample item relationship graph in the training phase.

[0151] In the above embodiment, an item relationship graph is introduced to narrow the difference between strict cold start items and hot start items while maintaining the performance of hot start recommendations. Based on the modal features of items, the potential structure between items can be fully explored. In the subsequent graph learning process, the collaborative signal of hot start items can be transmitted to strict cold start items, thereby improving the recommendation performance of strict cold start scenarios. In addition, a mask matrix is ​​added to shield the information transmitted from strict cold start items to hot start items. On the premise of improving the recommendation performance of strict cold start items, the recommendation performance of hot start items can be guaranteed.

[0152] In some embodiments, an original relationship graph is constructed based on the modal content of the items, including: calculating the similarity between any two items based on the modal content of the any two items; constructing a first similarity matrix based on the similarity between any two items; sparsifying the first similarity matrix to obtain an unweighted matrix, and determining the original relationship graph based on the unweighted matrix.

[0153] Since different edge information can reflect different features of items, the item relationship graph can be represented by an adjacency matrix. Specifically, for any two items, such as the ath item and the bth item, the computer device can use the modal features of the ath item and the bth item to represent the item relationship graph. and Calculate the similarity between the two (such as cosine similarity). Specifically, the similarity between the two can be calculated by formula 3:

[0154]

[0155] Then, the first similarity matrix is ​​constructed based on the similarity between any two items as the value of the matrix element. Next, the computer device may perform a sparse process on the first similar phase matrix to obtain an unweighted matrix, and perform a normalization process on the unweighted matrix to obtain an original relationship graph.

[0156] In some embodiments, the computer device may perform sparse processing on the first similarity matrix, such as binarization, that is, setting the values ​​of elements in the matrix that are less than a preset value to 0 and setting the values ​​of elements that are greater than or equal to the preset value to 1.

[0157] In some embodiments, the first similarity matrix is ​​sparsely processed to obtain an unweighted matrix, including: for any row in the first similarity matrix, sorting the values ​​of multiple elements in the row from large to small; taking the elements with a ranking less than a preset ranking as the first element, and the other elements as the second element; updating the value of the first element in the row to a first value, and updating the value of the second element in the row to a second value, the second value being less than the first value; after the values ​​of the elements in each row of the first similarity matrix are updated, an unweighted matrix is ​​obtained.

[0158] Specifically, in the first similarity matrix, each row corresponds to an item, and the elements in the row are the similarities between other items and the item. For the row corresponding to the ath item, the computer device can sort the values ​​of multiple elements in the row from large to small, and use the elements with a ranking less than the preset ranking as the first element, and the other elements as the second element. Then, the value of the first element in the row is updated to the first value (such as 1), and the value of the second element in the row is updated to the second value (such as 0). In this way, the above processing is performed on each row, and the sparseness of the first similarity matrix is ​​completed to obtain an unweighted matrix.

[0159] In some embodiments, for the a-th item, the computer device may retain only the top-K similar items similar to it (that is, retain the top K similar items with the greatest similarity), which can be sparsely processed by Formula 4:

[0160]

[0161] in, represents the first similarity matrix, Represents an unweighted matrix, where each element in the unweighted matrix is ​​1 or 0, 1 indicates that there is an association between the two items, and 0 indicates that there is no association between the two items.

[0162] In this way, by retaining the top-K similar items that are similar to the item to perform sparse processing on the first similarity, important side information can be retained and irrelevant side information can be eliminated.

[0163] Furthermore, the computer device can normalize the unweighted matrix by formula 5: Get the original relationship diagram between items

[0164]

[0165] Among them, D m express Considering the two modalities of text and image, we can obtain the original relationship graph based on text Original image-based graph

[0166] In the above embodiment, the first similarity matrix is ​​constructed by the similarity between the items, so that the potential connection between the items can be fully explored. The first similarity matrix is ​​sparsely processed and then normalized, so that important side information can be retained and the stability of calculation can be improved.

[0167] In some embodiments, constructing an object co-occurrence map based on an interaction map includes: determining the number of items that any two objects interact with in common according to the interaction map; constructing a second similarity matrix according to the number of items that any two objects interact with in common; and performing sparse processing on the second similarity matrix to obtain the object co-occurrence map.

[0168] Since users who have interacted with similar items often have similar preferences, the object co-occurrence graph can be represented by an adjacency matrix to learn the internal relationships between objects. Specifically, the computer device can determine the number of items that any two objects interact with based on the interaction graph, and then construct a second similarity matrix based on the number of items that any two objects interact with as the value of the matrix element. Next, the computer device can perform sparse processing on the second similarity matrix to obtain the object co-occurrence graph.

[0169] In the above embodiment, the second similarity matrix is ​​constructed by the number of items that different objects have interacted with, so that the preference relationship between different objects can be fully explored. After the second similarity matrix is ​​sparse, important side information can be retained, thereby improving the accuracy of subsequent calculations.

[0170] In some embodiments, the second similarity matrix is ​​sparsely processed to obtain an object co-occurrence map, including: for any row in the second similarity matrix, sorting the values ​​of multiple elements in the row from large to small; using the elements with a ranking less than a preset ranking as the third element, and the other elements as the fourth element; retaining the value of the third element in the row, and updating the value of the fourth element in the row to the second numerical value; after each row of the second similarity matrix is ​​updated, the object co-occurrence map is obtained.

[0171] Specifically, in the second similarity matrix, each row corresponds to an object, and the elements in the row are the number of items that other objects have interacted with the object. For the row corresponding to the cth object, the computer device can sort the values ​​of multiple elements in the row from large to small, and use the elements with a ranking less than the preset ranking as the third element, and the other elements as the fourth element. Then, the value of the third element is retained, and the value of the fourth element in the row is updated to the second value (such as 0). In this way, the above processing is performed on each row, that is, the sparseness of the second similarity matrix is ​​completed, and an unweighted matrix is ​​obtained.

[0172] Specifically, for the c-th user ∈U, for any other user, the computer device can retain the number of common interaction items as the top-K number, which can be sparsely processed by Formula 6:

[0173]

[0174] Among them, G u-u (c, d) represents the number of shared interaction items between the cth user and the dth user in the second similarity matrix. Representing object co-occurrence graph The element in row c and column d.

[0175] In the above embodiment, for any object, objects having the top-K identical interactive items are retained to perform sparse processing on the second similarity matrix, thereby retaining important side information and eliminating irrelevant side information.

[0176] In some embodiments, reference Figure 6 , based on the interaction graph and collaborative knowledge graph, heterogeneous graph learning is performed to obtain object features and item features, including:

[0177] Step 602: Perform graph convolution processing based on the interaction graph to obtain behavior-aware first object embedding and first item embedding.

[0178] Specifically, the computer device can construct an encoder based on the graph convolutional network (the encoder can be considered as a part of the data recommendation model), perform iterative message propagation on the interaction graph between objects and items, and explore high-order synergistic effects without introducing item side information.

[0179] In some embodiments, the computer device may construct an encoder based on a multi-layer graph convolutional network to perform graph convolution processing on the interaction graph.

[0180] For the l+1th layer, the embedding of any object node can be determined based on the embedding of the neighboring nodes of the object node in the lth layer, for example, the average embedding of the neighboring nodes of the object node in the lth layer, specifically through the public

[0181] Formula 7 indicates:

[0182]

[0183] in, represents the embedding of object u at layer l+1, represents the embedding of item i at layer l, Represented in the interaction graph G inter The neighbor set of object u in , Represents the number of neighbor nodes in the neighbor set of object u.

[0184] For the l+1th layer, the embedding of any item node can be determined based on the embedding of the neighboring nodes of the item node in the lth layer, for example, the average embedding of the neighboring nodes of the item node in the lth layer, for example, expressed by Formula 8:

[0185]

[0186] in, represents the embedding of item i at the l+1th layer, represents the embedding of object u at layer l, Represented in the interaction graph G inter The neighbor set of item i in , Represents the number of neighbor nodes in the neighbor set of item i.

[0187] It should be noted that the neighbor nodes of the object node can be determined through the interaction graph, and the neighbor nodes of the item node can be determined through the interaction graph.

[0188] In addition, it should be noted that the object embedding for the initial layer can be represented by object identity embedding, which can be obtained by encoding the attributes of the object, or it can be randomly initialized, or it can be a feature that the data recommendation model continuously learns during the training process. The item embedding for the initial layer can be represented by item identity embedding, which can be obtained by encoding the attributes of the item, or it can be randomly initialized, or it can be a feature that the data recommendation model continuously learns during the training process.

[0189] In some embodiments, the computer device may use the object embedding output by the last layer as the first object embedding, and use the item embedding output by the last layer as the first item embedding. In other embodiments, after the multi-layer graph neural network is processed, the computer device may aggregate the object embedding output by each layer to obtain the first object embedding, and aggregate the item embedding output by each layer to obtain the first item embedding. The aggregation method may be averaging.

[0190] In some embodiments, the interaction graph includes object nodes and item nodes; graph convolution processing is performed based on the interaction graph to obtain a behavior-aware first object embedding and a first item embedding, including: performing multiple graph convolution processing based on the interaction graph, and obtaining the output of each graph convolution processing; for any object node, aggregating the embedding corresponding to the targeted object node in the output of each graph convolution processing to obtain the first object embedding of the targeted object node; for any item node, aggregating the embedding corresponding to the targeted item node in the output of each graph convolution processing to obtain the first item embedding of the targeted item node.

[0191] Specifically, the computer device may perform multiple graph convolution processes based on the interaction graph and obtain the output of each graph convolution process. For any object node, the embedding corresponding to the targeted object node in the output of each graph convolution process is aggregated by mean pooling to obtain the first object embedding of the targeted object node; for any item node, the embedding corresponding to the targeted item node in the output of each graph convolution process is aggregated by mean pooling to obtain the first item embedding of the targeted item node.

[0192] For example, after processing multiple layers of graph convolution, the computer device can use mean pooling to aggregate the embeddings of each layer to obtain the final behavior-aware first object embedding. Embed with first item Where L represents the number of layers of the graph convolutional network, that is, the number of graph convolution processing.

[0193] It should be noted that there are no edges connected to the strictly cold-start items in the interaction graph. Therefore, after the behavior-aware graph convolution, the embedding of the strictly cold-start items is a zero vector, which is equivalent to skipping the collaborative filtering based on interaction data.

[0194] In the above embodiment, since the information scope aggregated by graph convolutions of different layers is different, the characteristics of objects and items can be better represented by aggregating the results of multi-layer graph convolution processing.

[0195] Step 604, determining a modality-aware second object embedding and a second item embedding according to the interaction graph and the modal content of the item.

[0196] Specifically, in order to mine modal-aware object preferences and item features, the computer device can map the multimodal features of the item extracted by the feature extractor into interaction-related multimodal features, and then aggregate them on the interaction graph between the object and the item to obtain the modal-aware second object embedding and second item embedding.

[0197] In some embodiments, the computer device may extract modal features of modal content, such as extracting text features of text content through a text encoder and extracting image features of image content through a visual encoder.

[0198] In some embodiments, the computer device may map the multimodal features into interactively correlated multimodal features through a linear layer.

[0199] In some embodiments, the interaction graph includes object nodes and item nodes, and a modality-aware second object embedding and a second item embedding are determined based on the modal content of the interaction graph and the items, including: for any object node, adjacent item nodes are determined based on the interaction graph, and the second object embedding of the targeted object node is determined based on the modal content of the adjacent item nodes; for any item node, adjacent object nodes are determined based on the interaction graph, and the second item embedding of the targeted item node is determined based on the second object embedding features of the adjacent object nodes.

[0200] Specifically, for any object node, the computer device can determine the item nodes adjacent to the object node according to the interaction graph, average the modal features of the adjacent multiple item nodes, and obtain the second object embedding of the object node. The second object embedding is calculated for each object node in the above manner.

[0201] For any item node, the computer device can determine the object nodes adjacent to the item node according to the interaction graph, average the modal features of multiple adjacent object nodes, and obtain the second item embedding of the item node. The second item embedding is calculated for each item node in the above manner.

[0202] Exemplarily, the computer device may determine the second object embedding and the second object item embedding by using Formulas 9 and 10:

[0203]

[0204]

[0205] in, represents the second object embedding of object u, Represented in the interaction graph G inter The neighbor set of object u in , Represents the number of neighbor nodes in the neighbor set of object u. Linear(·) is a fully connected layer that uses Dropout to map the modal features of items to the interactive perception vector space. i m Represents the modal characteristics of item i. represents the second item embedding of item i, Represented in the interaction graph G inter The neighbor set of item i in , Represents the number of neighbor nodes in the neighbor set of item i.

[0206] In some embodiments, the modal features of the object are multimodal features, including text features and image features, so the modality-aware second object embedding includes a second object text embedding based on text feature perception and a second object image embedding based on image feature perception. The modality-aware second object embedding includes a second object text embedding based on text feature perception and a second object image embedding based on image feature perception.

[0207] In the above embodiment, based on the modal content of the object, aggregation is performed on the interaction graph between the object and the object, so that modality-aware object embedding and item embedding can be obtained, which can better explore user preferences and item characteristics.

[0208] Step 606: Perform graph attention processing based on the collaborative knowledge graph to obtain knowledge-aware third object embedding and third item embedding.

[0209] It should be noted that, since there are different edges in the collaborative knowledge graph, different edges have different specific meanings and different weights. In order to better propagate messages between entities, the knowledge-aware third object embedding and third item embedding can be generated based on the heterogeneous graph attention aggregation method.

[0210] Step 608: Fuse the first object embedding, the second object embedding, and the third object embedding to obtain object features.

[0211] Specifically, the computer device may perform weighted processing on the first object embedding, the second object embedding, and the second object embedding to obtain the object feature, wherein the weighted coefficient of the weighted processing may be a preset coefficient or a coefficient learned by the data recommendation model during the training phase.

[0212] Step 610, fusing the first item embedding, the second item embedding and the third item embedding to obtain item features.

[0213] Specifically, the computer device may perform weighted processing on the first item embedding, the second item embedding, and the third item embedding to obtain item features. Similarly, the weighted coefficients of the weighted processing may be preset coefficients or coefficients learned by the data recommendation model during the training phase.

[0214] In the above embodiment, after obtaining the behavior-aware, knowledge-aware, and modality-aware object embeddings and item embeddings, they are fused to obtain object features and item features. The fused object features and item features can better reflect the user preferences and item characteristics related to the interaction based on external knowledge and different modalities.

[0215] In some embodiments, reference Figure 7 , based on the collaborative knowledge graph, graph attention processing is performed to obtain knowledge-aware third object embedding and third item embedding, including:

[0216] Step 702, for any entity in the collaborative knowledge graph, determine the egocentric network of the targeted entity; entities include object entities and item entities.

[0217] Among them, the nodes in the egocentric network are composed of a unique central node and the neighbor nodes of the central node. The edges of the egocentric network include the edges between the central node and its neighbor nodes, and the edges between each neighbor node.

[0218] For any entity in the collaborative knowledge graph, a computer device can use the entity as the central node, the neighboring nodes connected to the entity, the edges between the entity and each neighboring node, and the edges between each neighboring node to form an egocentric network of the entity.

[0219] Step 704, determining the importance weight of each multi-group in the egocentric network, and determining the representation of the egocentric network according to the importance weight of each multi-group.

[0220] The entity identity embedding can be obtained by encoding the entity's attributes, or it can be randomly initialized, or it can be a feature that the data recommendation model continuously learns during the training process. Similarly, the edge identity embedding in the collaborative knowledge graph can be obtained by encoding the edge's attributes, or it can be randomly initialized, or it can be a feature that the data recommendation model continuously learns during the training process.

[0221] For each entity in the collaborative knowledge graph, the computer device determines the representation of the egocentric network corresponding to the entity in the following manner:

[0222] First, the computer device may determine at least one tuple in the egocentric network, which may be a triplet including a head entity, a tail entity, and an edge connecting the head entity and the tail entity. Further, the computer device may determine the importance weight of the tuple according to the identifier embedding of the head entity, the identifier embedding of the tail entity, and the identifier embedding of the edge.

[0223] Further, the computer device may determine the representation of the egocentric network based on the importance weight of each multi-tuple and the identity embedding of the tail entity in each multi-tuple. For example, the computer device may perform a weighted sum based on the importance weight of each multi-tuple and the identity embedding of the tail entity in each multi-tuple to obtain the representation of the egocentric network.

[0224] In some embodiments, the importance weight of each multi-group in the egocentric network is determined, including: for any multi-group in the egocentric network, determining the importance of the multi-group based on the identity embedding of the entities and edges in the multi-group; based on the importance of all multi-groups in the egocentric network, normalizing the importance of each multi-group to obtain the importance weight of each multi-group.

[0225] Specifically, for any multi-tuple in the egocentric network, the computer device can determine the importance of the multi-tuple according to the identification embedding of the entities and edges in the multi-tuple. Then, according to the importance of all multi-tuples in the egocentric network, the importance of each multi-tuple is normalized to obtain the importance weight of each multi-tuple.

[0226] Exemplarily, the computer device may use knowledge-aware attention to aggregate neighbor information in the collaborative knowledge graph and determine the representation of the egocentric network by the following formula:

[0227]

[0228]

[0229] π(h,r,t)=(W r x t ) T tanh(W r x h +x r )) (Formula 13);

[0230] in, represents the egocentric network corresponding to entity h, Egocentric Network (h,r,t) represents the triple consisting of the head entity h, the tail entity t, and the edge r connecting the head entity h and the tail entity t. The head entity h here can be any entity in the collaborative knowledge graph, including objects, items, and entities in the item knowledge graph. π(h,r,t) represents the importance of the triple (h,r,t), x h represents the identity embedding of the head entity h, x r represents the identity embedding of edge r, x t represents the identity embedding of the tail entity t, W r Represents the transformation matrix of relation r, which is the matrix learned by the data recommendation model during training. α(h,r,t) represents the importance weight of the triple (h,r,t), and (h,r′,t′) is the egocentric network Any triple in .

[0231] In the above embodiment, for any multi-group in the ego-centric network, the importance of the multi-group can be determined based on the identifier embedding of the entities and edges in the multi-group, and then the importance of each multi-group is normalized to obtain the importance weight of each multi-group.

[0232] Step 706: for any object entity, aggregate the identification embedding of the object entity and the corresponding egocentric network representation to obtain a third object embedding of the object entity.

[0233] Specifically, for any object entity, the computer device may aggregate the identification embedding of the object entity and the corresponding representation of the egocentric network to obtain a third object embedding of the object entity. The aggregation method may be addition, multiplication or concatenation, etc., which is not limited in the embodiment of the present application.

[0234] In some embodiments, for any object entity, the identity embedding of the targeted object entity and the corresponding egocentric network representation are aggregated to obtain a third object embedding of the targeted object entity, including: performing an addition operation on the identity embedding of the targeted object entity and the corresponding egocentric network representation to obtain an addition operation result; performing an element-by-element multiplication operation on the identity embedding of the targeted object entity and the corresponding egocentric network representation to obtain a multiplication operation result; and fusing the addition operation result and the multiplication operation result to obtain a third object embedding feature of the targeted object entity.

[0235] The specific fusion method for fusing the addition operation result and the multiplication operation result may be weighted summation. For example, when obtaining the representation of the egocentric network After that, the computer equipment can be aggregated by dual interactive aggregation With the embedding of the entity itself, for example, the computer device can obtain the third object embedding of the object entity through formula 14:

[0236]

[0237] Among them, W1, W2 represent trainable weight matrices, ⊙ represents element-by-element product, and LeakyReLU represents an activation function.

[0238] In the above embodiment, the representation of the egocentric network of the entity and the embedding of the entity itself are aggregated by dual interactive aggregation, so that the information embedded in the third object after knowledge perception is richer and more accurate.

[0239] Step 708: for any item entity, aggregate the identification embedding of the item entity and the corresponding representation of the egocentric network to obtain a third item embedding of the item entity.

[0240] Specifically, for any item entity, the computer device may aggregate the identification embedding of the item entity and the corresponding representation of the egocentric network to obtain a third item embedding of the item entity. The aggregation method may be addition, multiplication, or concatenation, etc., which is not limited in the present embodiment.

[0241] In some embodiments, for any object entity, the identification embedding of the targeted object entity and the corresponding representation of the egocentric network are aggregated to obtain a third object embedding of the targeted object entity, including: performing an addition operation on the identification embedding of the targeted object entity and the corresponding representation of the egocentric network to obtain an addition operation result; performing an element-by-element multiplication operation on the identification embedding of the targeted object entity and the corresponding representation of the egocentric network to obtain a multiplication operation result; and fusing the addition operation result and the multiplication operation result to obtain a third object embedding feature of the targeted object entity.

[0242] The specific fusion method for fusing the addition operation result and the multiplication operation result may be weighted summation. For example, when obtaining the representation of the egocentric network After that, the computer equipment can be aggregated by dual interactive aggregation In this way, the representation of the entity's egocentric network and the entity's own embedding are aggregated through dual interactive aggregation, making the information of the knowledge-aware third-party embedding richer and more accurate.

[0243] In the above embodiments, a message propagation mechanism is constructed between objects, items, and connected entities, thereby generating knowledge-aware object embedding and item embedding based on a heterogeneous attention aggregation method.

[0244] In some embodiments, a modality-aware second object embedding includes a text-aware second object text embedding and an image-aware second object image embedding; fusing the first object embedding, the second object embedding, and the second object embedding to obtain object features, including: determining a first weight corresponding to the second object text embedding, and a second weight corresponding to the second object image embedding; determining an object multimodal embedding based on the first weight, the second object text embedding, the second weight, and the second object image embedding; and performing weighted processing on the first object embedding, the object multimodal embedding, and the third object embedding to obtain object features.

[0245] It is understandable that after obtaining the behavior-aware, knowledge-aware, and modality-aware object embeddings, the computer device fuses them to obtain object features, which can reflect user preferences related to interaction based on external knowledge and different modalities. Different modal embeddings may correspond to different weights, so the computer device can determine a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding. Then, the second object text embedding, the second weight, and the second object image embedding are weighted according to their respective corresponding weights to obtain a multimodal embedding of the object.

[0246] Furthermore, the computer device may perform weighted processing on the first object embedding, the object multimodal embedding, and the third object embedding to obtain the object feature, wherein the weighted coefficient may be a preset weight or a weight learned by the data recommendation model during training.

[0247] Exemplarily, the computer device may determine the object feature e by the following formula: u :

[0248]

[0249] in, is the first object embedding, Embedded as a third object, is the second object text embedding, is the second object image embedding. k ,λ m represents the hyperparameter used to control the weight of knowledge-aware and modality-aware representations, β t ,β i They respectively represent the relative importance of text and image, which are the first weight and the second weight mentioned above. The first weight and the second weight will be optimized during the training process.

[0250] In the above embodiment, after obtaining behavior-aware, knowledge-aware, and modality-aware object embeddings, they are fused to obtain object features, which can reflect user preferences related to interaction based on external knowledge and different modalities.

[0251] refer to Figure 8 In some embodiments, the modality-aware second item embedding includes a text-aware second item text embedding and an image-aware second item image embedding, and determining a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding includes:

[0252] Step 802: construct a text-based first virtual interaction graph according to the similarity between the second object text embedding of any object and the second item text embedding of any item.

[0253] Specifically, the computer device can calculate the similarity between the second object text embedding of any object and the second item text embedding of any item, construct a text-based similarity matrix according to each similarity, and use the text-based similarity matrix as the first text-based virtual interaction graph.

[0254] Exemplarily, the computer device may construct the first virtual interaction graph by using Formula 16:

[0255]

[0256] in, represents the first text-based virtual interaction diagram, is the second object text embedding, Text embed for the second item.

[0257] Step 804 : constructing an image-based second virtual interaction graph according to the similarity between the second object image embedding of any object and the second item image embedding of any item.

[0258] Specifically, the computer device can calculate the similarity between the second object image embedding of any object and the second item image embedding of any item, construct an image-based similarity matrix according to each similarity, and use the image-based similarity matrix as the image-based second virtual interaction graph.

[0259] Exemplarily, the computer device may construct the second virtual interaction graph by using Formula 17:

[0260]

[0261] in, represents a second image-based virtual interaction graph, is the second object image embedding, Embed the image for the second item.

[0262] Step 806 : determining a first discriminant score for characterizing the difference between the first virtual interaction graph and the interaction graph.

[0263] Specifically, an auxiliary discriminator can be introduced to distinguish the first virtual interaction graph from the interaction graph. The discriminator can be constructed through a nonlinear network, and D(·) represents the output of the discriminator. The computer device can input the first virtual interaction graph into the discriminator and output a first discriminant score.

[0264] Step 808: Determine a second discriminant score for characterizing the difference between the second virtual interaction graph and the interaction graph.

[0265] The computer device may input the second virtual interaction graph into the discriminator and output a second discriminant score.

[0266] Step 810: Determine a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding according to the first discrimination score and the second discrimination score.

[0267] The computer device may normalize the first discrimination score and determine the first weight according to the normalization result; normalize the second discrimination score and determine the second weight according to the normalization result.

[0268] Exemplarily, the computer device may determine the first weight β by using formulas 18 and 19. t and the second weight β i :

[0269]

[0270]

[0271] Among them, η represents the number of t With β i The parameter is updated during the training process of the data recommendation model.

[0272] In the above embodiments, since the importance of different modalities depends on their contribution to capturing and predicting user preferences and behaviors, the present application updates the importance weights by the difficulty of distinguishing between the virtual interaction graph generated by the multimodal content and the observed real interaction graph, which can greatly improve the accuracy of object multimodal embedding and item multimodal embedding.

[0273] In some embodiments, a modality-aware second item embedding, including a text-aware second item text embedding and an image-aware second item image embedding, fuses the first item embedding, the second item embedding, and the third item embedding to obtain item features, including: determining a first weight corresponding to the second item text embedding, and a second weight corresponding to the second item image embedding; determining an item multimodal embedding based on the first weight, the second item text embedding, the second weight, and the second item image embedding; and weighted processing the first item embedding, the item multimodal embedding, and the third item embedding to obtain item features.

[0274] It is understandable that after obtaining the behavior-aware, knowledge-aware, and modality-aware item embeddings, the computer device fuses them to obtain item features, which can reflect the item characteristics related to interaction based on external knowledge and different modalities. Different modal embeddings may correspond to different weights, so the computer device can determine the first weight corresponding to the second item text embedding and the second weight corresponding to the second item image embedding. Then, the second item text embedding, the second weight, and the second item image embedding are weighted according to their respective corresponding weights to obtain the multimodal embedding of the item.

[0275] Furthermore, the computer device may perform weighted processing on the first item embedding, the item multimodal embedding, and the third item embedding to obtain item features, wherein the weighted coefficient may be a preset weight or a weight learned by the data recommendation model during training.

[0276] For example, the computer device may determine the item feature e by the following formula 20: i :

[0277]

[0278] in, For the first item embed, Embedded for third objects, For the second item text embedding, is the second item image embedding. k ,λ m represents the hyperparameter used to control the weight of knowledge-aware and modality-aware representations, β t ,β i They respectively represent the relative importance of text and image, which are the first weight and the second weight mentioned above. The first weight and the second weight will be optimized during the training process.

[0279] In the above embodiment, after obtaining the behavior-aware, knowledge-aware, and modality-aware item embeddings, they are fused to obtain item features, which can reflect the interaction-related item characteristics based on external knowledge and different modalities.

[0280] In some embodiments, isomorphic graph learning is performed based on object co-occurrence graphs and object features to obtain a target object representation, including: taking the object co-occurrence graph as a first adjacency matrix, and constructing a first input matrix according to the object features of each object; performing graph attention processing based on the first adjacency matrix and the first input matrix to obtain a first output matrix; the elements in the first output matrix are the target object representations.

[0281] In some embodiments, the computer device may perform multi-layer graph attention processing on the object co-occurrence graph, and the representation of any node in each layer of graph attention processing is obtained by weighted summation based on the representation of its neighbor nodes in the previous layer. Among them, the neighbor nodes of the node can be determined by the first adjacency matrix, and the weight coefficient corresponding to any neighbor node can be determined based on the value of the element corresponding to the neighbor node in the first adjacency matrix (i.e., an element in the matrix) and the values ​​of the elements corresponding to all neighbor nodes. It should be noted that for the initial layer, the representation of each node is the object feature e obtained through heterogeneous graph learning. u .

[0282] After multi-layer graph attention processing, the computer device may represent the value of each element in the first output matrix output by the last layer as the target object.

[0283] Exemplarily, the computer device may determine the representation of any object in the lth layer by the following formula:

[0284]

[0285] in, represents the representation of the cth object at layer l, Represents the representation of the d-th object at the l-1th layer. Represents the set of neighbor objects of the cth object, is the object co-occurrence graph, i.e., the first adjacency matrix. d′ is any object in the neighboring node of the cth object.

[0286] Stacking L on the object co-occurrence graph u-u After the convolutional layers, the final target object representation can be obtained after the message passing between objects. (That is, the output of the last layer).

[0287] In the above embodiment, considering that objects that share more interactive items with the central object should have a greater impact on it, by designing a method to realize message transmission between objects through graph attention on the object co-occurrence graph, the final target object representation can be more accurate.

[0288] In some embodiments, the item relationship graph includes a text-based item relationship graph and an image-based item relationship graph; isomorphic graph learning is performed based on the item relationship graph and item features to obtain a target item representation, including: using the text-based item relationship graph as a second adjacency matrix, and using the image-based item relationship graph as a third adjacency matrix; constructing a second input matrix according to the item features of each item; performing graph convolution processing based on the second adjacency matrix and the second input matrix to obtain a second output matrix; the elements in the second output matrix are item text representations; performing graph convolution processing based on the third adjacency matrix and the second input matrix to obtain a third output matrix; the elements in the third output matrix are item image representations; fusing the item text representation and the item image representation to obtain the target item representation.

[0289] In some embodiments, the computer device may perform multi-layer graph convolution processing on the item relationship graph, and the representation of any node in each layer of graph convolution processing is obtained by aggregating the representations of its neighboring nodes in the previous layer. For a text-based item relationship graph, graph convolution processing may be performed based on the features of each item to obtain an item text representation; for an image-based item relationship graph, graph convolution processing may be performed based on the features of each item to obtain an item image representation.

[0290] For a text-based item relationship graph, the neighbor nodes of any node can be determined by the second adjacency matrix. For an image-based item relationship graph, the neighbor nodes of any node can be determined by the third adjacency matrix. For the initial layer, the representation of each node is the item feature obtained through heterogeneous graph learning.

[0291] Exemplarily, the computer device may calculate the text representation of the item by the following formula 22:

[0292]

[0293] in, represents the representation of the a-th item in the l-th layer, Represents the b-th item in the l-1th layer. represents the set of neighbor items of the a-th item, It is a text-based item relationship graph, which is also the second adjacency matrix.

[0294] Stacking L on a text-based item relationship graph i-i After the convolutional layers, the text representation of the item is obtained

[0295] Exemplarily, the computer device may calculate the object image representation by the following formula 23:

[0296]

[0297] in, represents the representation of the a-th item in the l-th layer, Represents the b-th item in the l-1th layer. represents the set of neighbor items of the a-th item, It is an image-based item relationship graph, also known as the third adjacency matrix.

[0298] Stacking L on the image-based item relationship graph i-i After the convolutional layers, the text representation of the item is obtained

[0299] Furthermore, after obtaining the object text representation and the object image representation, the computer device may fuse them to obtain the target object representation. The fusion method may be averaging or weighted summing, etc., which is not limited in the embodiment of the present application.

[0300] In the above embodiment, neighbor information is transferred and aggregated based on the text-based object relationship graph to obtain the object text representation, and neighbor information is transferred and aggregated based on the image-based object relationship graph to obtain the object image representation. The object text representation and the object image representation are fused to obtain the target object representation, so as to mine the multimodal features between the objects.

[0301] In some embodiments, the object text representation and the object image representation are fused to obtain the target object representation, including: performing multi-head attention processing based on the object text representation and the object image representation; and performing mean pooling on the result after the multi-head attention processing to obtain the target object representation.

[0302] Specifically, in order to capture the association of item representations extracted from the modality-specific item relationship graph, a multi-head attention mechanism and mean pooling can be used to fuse the item text representation and the item image representation.

[0303] Exemplarily, the computer device may determine the target item representation by the following formulas 24-26:

[0304]

[0305]

[0306]

[0307] Where H represents the number of attention heads. σ(·) represents the softmax function. They represent the transformation matrices of the head-th attention head on the query and key, respectively, and d represents the number of dimensions of embedding. M is a set of modalities, including text modality and image modality. Indicates the number of modes.

[0308] In the above embodiment, the multi-head attention mechanism and mean pooling method can be used to well integrate the object text representation and the object image representation to obtain the target object representation.

[0309] In some embodiments, the item includes hot start media data and strict cold start media data, the interaction graph between the object and the item includes an interaction graph between the user and the media data, and the modal content of the item includes text content and image content in the media data.

[0310] The media data may specifically be audio or video data, such as a long video or a short video. A long video is a video whose duration is greater than a first preset duration, and a short video is a video whose duration is less than a second preset duration, wherein the second preset duration is less than or equal to the first preset duration.

[0311] refer to Fig. 9 , Fig. 9 The figure shows the knowledge graph related to the video, the image content and text content of the video, and the historical interaction graph between the user and the video. Based on the above data, the hot start videos and cold start videos that the user is interested in can be predicted.

[0312] In an exemplary scenario, a computer device may obtain a knowledge graph of media data, an interaction graph between a user and media data, and text content and image content of the media data, fuse the knowledge graph and interaction graph of the media data to obtain a collaborative knowledge graph, determine an item relationship graph based on the text content and image content of the media data, construct an object co-occurrence graph based on the interaction graph, perform heterogeneous graph learning based on the interaction graph and the collaborative knowledge graph to obtain object features and media features; perform homogeneous graph learning based on the object co-occurrence graph and object features to obtain a target object representation, perform homogeneous graph learning based on the item relationship graph and media features to obtain a target media representation, and recommend media data for the target object based on the target object representation of the target object and the target media representation of each piece of media data.

[0313] In the above embodiment, through the data recommendation method based on heterogeneous-homogeneous graph learning, the visual content, text content and structural knowledge in the knowledge graph of media data (such as video) can be utilized to enrich the representation of strict cold start videos and hot start videos. While maintaining the hot start effect, the representation deviation of strict cold start videos caused by insufficient historical interaction data is compensated, thereby improving the recommendation performance of the recommendation system for strict cold start videos.

[0314] It should be noted that the above application scenarios are only illustrative, and it is understood that the application of the data recommendation method provided in each embodiment of the present application is not limited to the above scenarios. The data recommendation method of the present application can be applied to e-commerce scenarios, video recommendation scenarios, advertising push scenarios, etc. For example, it can realize the recommendation of promotional information, electronic coupons, and physical goods.

[0315] In some embodiments, the isomorphic graph learning and heterogeneous graph learning in the above embodiments are realized through a trained data recommendation model. The trained data recommendation model is applied, and the items involved include hot start items, general cold start items, and strict cold start items. Correspondingly, the item knowledge picture is also an item knowledge graph about hot start items, general cold start items, and strict cold start items. The interaction graph is an interaction graph about objects and hot start items and general cold start items. The modal content of the items includes the modal content of hot start items, general cold start items, and strict cold start items. The computer device can first retrain and learn the data recommendation model based on these data, so that the model can learn relevant information about cold start items, and then perform heterogeneous graph and isomorphic graph learning based on these data to achieve data recommendation.

[0316] Please refer to Fig.10 The present application provides a data recommendation model processing method, which is described by taking the method applied to a computer device as an example, and includes the following steps:

[0317] Step 1002, obtaining a sample item knowledge graph, a sample interaction graph between sample objects and sample items, and sample modal content of the sample items.

[0318] Among them, the sample item knowledge graph is the item knowledge graph used as a training sample, the sample interaction graph is the interaction graph used as a training sample, and the sample modal content is the modal content used as a training sample.

[0319] It should be noted that during the training phase, sample items do not include strict cold start items, but usually include hot start items, and in some scenarios, may also include general cold start items.

[0320] Step 1004, integrate the sample item knowledge graph and the sample interaction graph to obtain the sample collaborative knowledge graph, determine the sample item relationship graph based on the sample modal content, and construct the sample object co-occurrence graph based on the sample interaction graph.

[0321] Regarding the specific content of fusing the sample item knowledge graph and the sample interaction graph to obtain the sample collaborative knowledge graph, please refer to the description of step 204 in the aforementioned embodiment regarding fusing the item knowledge graph and the interaction graph to obtain the relevant content of the collaborative knowledge graph.

[0322] For relevant content about determining the sample item relationship graph based on the sample modal content, please refer to the relevant description of determining the item relationship graph based on the modal content of the item in the aforementioned step 204. For content about constructing the sample object co-occurrence graph based on the sample interaction graph, please refer to the description of constructing the object co-occurrence graph based on the interaction graph in the aforementioned step 204.

[0323] Step 1006, through the data recommendation model to be trained, heterogeneous graph learning is performed on the sample interaction graph and the sample collaborative knowledge graph to obtain sample object features and sample item features.

[0324] Specifically, the computer device can obtain sample object embedding and sample item embedding by performing side information-aware heterogeneous graph learning based on the sample interaction graph through the data recommendation model; and obtain sample object embedding and sample item embedding by performing side information-aware heterogeneous graph learning based on the sample collaborative knowledge graph through the data recommendation model. Thus, the sample object embedding obtained by different heterogeneous graph learning is integrated through the data recommendation model to obtain sample object features; and the sample item embedding obtained by different heterogeneous graph learning is integrated through the data recommendation model to obtain sample item features.

[0325] In some embodiments, heterogeneous graph learning is performed on the sample interaction graph and the sample collaborative knowledge graph to obtain sample object features and sample item features, including: determining modality-aware sample object embedding and sample item embedding based on the sample interaction graph and the sample modal content of the sample item; performing graph attention processing based on the sample collaborative knowledge graph to obtain knowledge-aware sample object embedding and sample item embedding; determining sample object features based on the modality-aware sample object embedding and knowledge-aware sample item embedding; and determining sample item features based on the modality-aware sample item embedding and knowledge-aware sample item embedding.

[0326] The computer device may determine the modality-aware sample object embedding and the sample item embedding according to the sample interaction diagram and the sample modal content of the sample item. The method of obtaining the modality-aware sample object embedding and the modality-aware sample item embedding may refer to the description of determining the modality-aware second object embedding and the second item embedding according to the interaction diagram and the modal content of the item in the aforementioned step 604.

[0327] The computer device may perform graph attention processing based on the sample collaborative knowledge graph to obtain knowledge-aware sample object embedding and sample item embedding. The method of obtaining the knowledge-aware sample object embedding and the knowledge-aware sample item embedding may refer to the description of performing graph attention processing based on the collaborative knowledge graph to obtain the knowledge-aware third object embedding and the third item embedding in the aforementioned step 606.

[0328] Furthermore, the computer device may aggregate the modality-aware sample object embedding and the knowledge-aware sample item embedding to obtain sample object features; aggregate the modality-aware sample item embedding and the knowledge-aware sample item embedding to obtain sample item features. The aggregation method may be weighted summation, averaging, or vector concatenation, which is not limited in the present embodiment.

[0329] In the above embodiment, after obtaining the knowledge-aware and modality-aware sample object / item embeddings, they are fused to obtain sample object features and sample item features. The fused sample object features and sample item features can better reflect user preferences and item characteristics based on external knowledge and different modalities.

[0330] Step 1008, through the data recommendation model to be trained, isomorphic graph learning is performed on the sample object co-occurrence graph and the sample object features to obtain the predicted object representation, and isomorphic graph learning is performed based on the sample item relationship graph and the sample item features to obtain the predicted item representation.

[0331] In some embodiments, considering that sample objects that have more sample interaction items in common with the central object should have a greater impact on it, a method is designed to realize message transmission between sample objects through graph attention on the sample object co-occurrence graph. The computer device can perform graph attention processing through a data recommendation model based on a pre-constructed sample object co-occurrence graph and sample object features obtained through heterogeneous graph learning to obtain a predicted object representation.

[0332] In some embodiments, the computer device may perform graph convolution processing through a data recommendation model based on a pre-constructed sample item relationship graph and sample item features obtained based on heterogeneous graph learning to obtain a predicted item representation.

[0333] In some embodiments, the sample item relationship graph includes a text-based sample item relationship graph and an image-based sample item relationship graph. The computer device can perform graph convolution processing based on the text-based sample item relationship graph and the features of each sample item through a data recommendation model to obtain a text representation of the sample item; and perform graph convolution processing based on the image-based sample item relationship graph and the features of each sample item through a data recommendation model to obtain an image representation of the sample item. The sample item text representation and the sample item image representation are then fused to obtain a predicted item representation. The specific fusion method can be averaging, weighted summation or other methods, which are not limited in the embodiments of the present application.

[0334] Step 1010, determining a prediction loss according to the prediction object representation and the prediction item representation, and constructing a first loss function according to the prediction loss.

[0335] Specifically, the computer device may determine the positive items that interact with the sample object according to the sample interaction graph, and determine the negative items that do not interact with the sample object according to the sample interaction graph. Then, the first sample matching degree is determined according to the predicted object representation of the sample object and the predicted item representation of the positive item, and the second sample matching degree is determined according to the predicted object representation of the sample object and the predicted item representation of the negative item.

[0336] It can be understood that for positive samples, the larger the corresponding first sample matching degree is theoretically, the more accurate it is, and for negative samples, the smaller the corresponding second sample matching degree is theoretically, the more accurate it is. Therefore, the computer device obtains the predicted loss based on the negative of the first sample matching degree. Alternatively, the computer device can obtain the predicted loss based on the second sample matching degree. Alternatively, the computer device can obtain the predicted loss based on the difference between the first sample matching degree and the second sample matching degree.

[0337] Furthermore, after determining the predicted loss, the computer device may construct a first loss function directly based on the predicted loss, or may construct a first loss function based on the predicted loss and in combination with other losses.

[0338] In some embodiments, a prediction loss is determined based on a predicted object representation and a predicted item representation, including: determining a first sample matching degree based on a predicted object representation of a sample object and a predicted item representation of a positive item; a positive item is an item that interacts with the sample object; determining a second sample matching degree based on a predicted object representation of the sample object and a predicted item representation of a negative item; a negative item is an item that does not interact with the sample object; and determining a prediction loss based on a difference between the first sample matching degree and the second sample matching degree.

[0339] Specifically, the computer device may construct a triple consisting of a sample object, a positive item, and a negative item. The computer device calculates similarity based on the predicted object representation of the sample object and the predicted item representation of the positive item, and uses the calculated similarity as the first sample matching degree. The computer device calculates similarity based on the predicted object representation of the sample object and the predicted item representation of the negative item, and uses the calculated similarity as the second sample matching degree. The similarity may specifically be cosine similarity.

[0340] Furthermore, the computer device may calculate a comparison value between the first sample matching degree and the second sample matching degree. The comparison value may specifically be a difference, a quotient, or other numerical values, and then obtain a predicted loss based on the comparison value.

[0341] Exemplarily, the computer device may determine the predicted loss by the following formula 27:

[0342]

[0343] Among them, L BPR is the prediction loss, (u,i p ,i n ) represents a triple consisting of a sample object, a positive item, and a negative item. represents the first sample matching degree, Indicates the second sample matching degree.

[0344] In the above embodiment, a first sample matching degree is determined based on the predicted object representation of the sample object and the predicted item representation of the positive item, and a second sample matching degree is determined based on the predicted object representation of the sample object and the predicted item representation of the negative item, so as to construct a prediction loss in a direction that maximizes the difference between the first matching degree and the second matching degree. This can make the model's prediction closer to the direction of the positive sample prediction, thereby improving the model's prediction accuracy.

[0345] Step 1012: Train the data recommendation model to be trained using the first loss function, and obtain the data recommendation model after the training is completed.

[0346] Specifically, when the computer device performs batch training on the data recommendation model, each time the model parameters can be adjusted in the direction of minimizing the first loss function, and the training is stopped when the training stop condition is reached, so as to obtain a trained data recommendation model. Among them, the training stop condition can specifically be reaching a preset number of iterations, reaching a preset iteration time, the prediction effect of the model reaching a preset effect, or the improvement degree of the prediction effect of the model is less than a preset degree, etc., which is not limited in the embodiments of the present application.

[0347] The above-mentioned data recommendation model processing method constructs a sample collaborative knowledge graph based on the sample interaction graph and sample item knowledge graph observed between the sample objects and the sample items, and obtains the sample object features and sample item features associated with the interaction through the way of side information perception, thereby reducing the influence of irrelevant side information on the recommendation performance of hot start items. In addition, a sample item relationship graph is constructed according to the sample modal content of the sample items, and a sample object co-occurrence graph is constructed according to the similarity of the interaction behaviors between the sample objects. Based on the sample item relationship graph and the sample object co-occurrence graph, the sample object features and sample item features encoding the interaction relationship between the sample objects and the sample items are propagated and aggregated to obtain the predicted object representation and the predicted item representation. The prediction loss is determined according to the predicted object representation and the predicted item representation, and the data recommendation model to be trained is trained according to the prediction loss, and the data recommendation model is obtained after the training is completed. The data recommendation model trained in this way can, on the one hand, mine the association structure between hot start items and objects, and on the other hand, transfer the features of hot start items to strict cold start items, make up for the representation of strict cold start items due to the lack of interaction data, and can greatly improve the recommendation performance of strict cold start items.

[0348] In some embodiments, the data recommendation model processing method further includes the step of constructing a second loss function, which specifically includes: constructing valid multi-groups and invalid multi-groups based on the knowledge graph of sample items; determining the first multi-group score according to the identifier embedding of entities and edges in the valid multi-group; determining the second multi-group score according to the identifier embedding of entities and edges in the invalid multi-group; constructing a second loss function based on the difference between the first multi-group score and the second multi-group score. The data recommendation model to be trained is trained by the first loss function, and the data recommendation model is obtained after the training is completed, including: alternately training the data recommendation model to be trained by the first loss function and the second loss function, and obtaining the data recommendation model after the training is completed.

[0349] The valid multi-tuple is a multi-tuple in the sample item knowledge graph, and the invalid multi-tuple is a multi-tuple that does not exist in the sample item knowledge graph. The multi-tuple can be a triple.

[0350] In some embodiments, the computer device may determine the first tuple score based on the identifier embedding of the head entity, the identifier embedding of the edge, and the identifier embedding of the tail entity in the valid tuple. The second tuple score is determined based on the identifier embedding of the head entity, the identifier embedding of the edge, and the identifier embedding of the tail entity in the invalid tuple. The identifier embedding of the entity and the edge may specifically be features learned by the data recommendation model in the previous training process; for the first iteration, it may be obtained by random initialization assignment.

[0351] In some embodiments, for both valid and invalid tuples, their tuple scores can be determined in the following manner: the computer device can obtain the relational r transformation matrix, multiply the relational r transformation matrix with the identification embedding of the head entity, convert it to the relational dimension, and add the identification embedding of the edge to obtain a result. The tail entity is multiplied by the relational r transformation matrix, and after converting the tail entity to the relational dimension, the above result is subtracted from the tail entity converted to the relational dimension to obtain the tuple score of the tuple. It can be understood that the higher the score of the tuple, the higher its validity.

[0352] Exemplarily, the computer device may calculate the multi-tuple score by the following formula 28:

[0353]

[0354] Among them, h represents the head entity, r represents the edge, and t represents the tail entity; W r Represents the relation r transformation matrix, which is the matrix learned by the model during training; e h Indicates the identifier embedding of the head entity, e r represents the identity embedding of the edge, e t Represents the identity embedding of the tail entity.

[0355] It should be noted that the computer device may also calculate the multi-tuple score in other ways, such as using the scoring function in Table 1 below to calculate the multi-tuple score:

[0356] Table 1. Schematic diagram of triple scoring function

[0357]

[0358] Among them, h, r, t represent the representation of the head entity, relation and tail entity respectively, M r ,w r represents the relational transformation matrix, w h represents the head entity transformation matrix, w t Represents the tail entity transformation matrix, R represents the real space, and the exponent of R represents the dimension of the real space.

[0359] Further, the computer device may construct a second loss function based on the difference between the first multi-group score and the second multi-group score. Exemplarily, the computer device may construct the second loss function by the following formula 29:

[0360]

[0361] in,

[0362] Furthermore, the computer device may train the data recommendation model to be trained alternately through the first loss function and the second loss function, and obtain the data recommendation model after the training is completed.

[0363] In the above embodiment, by introducing a second loss function for distinguishing valid multi-groups from invalid multi-groups, the representation learning of the item knowledge graph of multi-group semantics can be promoted.

[0364] In some embodiments, the data recommendation model processing method further includes: determining the similarity between the modality-aware sample object embedding and the predicted object representation, and determining the contrast loss of the object preference according to the similarity. Constructing a first loss function according to the predicted loss includes: determining the first loss function according to the predicted loss and the contrast loss.

[0365] Specifically, the computer device may introduce a contrast loss for learning object preferences of diversified modalities, thereby improving the perceptual accuracy of modal perception. That is, the computer device may calculate the similarity for any sample object based on the sample object embedding of the modal perception of the sample object and the predicted object representation of the sample object, normalize the similarity, and obtain the contrast loss based on the normalization result. Further, the prediction loss and the contrast loss may be weighted and summed to construct the first loss function.

[0366] Exemplarily, the computer device may calculate the contrast loss by the following formula:

[0367]

[0368]

[0369] Among them, L contr represents contrast loss, m refers to any modality, such as text modality or image modality, M refers to the set of modalities; U represents the set of sample objects; s represents cosine similarity; represents the predicted object representation of sample object u, represents the modality-aware sample object embedding of sample object u; u′ refers to any sample object in the sample object set; Refers to the predicted object representation of any sample object u′; Refers to the modality-aware sample object embedding of any sample object u′.

[0370] In the above embodiment, by maximizing the modality-aware sample object embedding and the final predicted object representation to construct a contrast loss and add it to the model training, the data recommendation model can learn more accurate modality-aware features during the training process.

[0371] In some embodiments, the data recommendation model processing method further includes a step of determining a modal adversarial loss, specifically including: constructing a sample virtual interaction graph based on the similarity between a modal-aware sample object embedding of any sample object and a modal-aware sample item embedding of any sample item; determining a sample target interaction graph based on the sample interaction graph; and determining a modal adversarial loss based on the difference between the sample virtual interaction graph and the sample target interaction graph. Determining a first loss function based on prediction loss and contrast loss includes: determining the first loss function based on prediction loss, contrast loss, and modal adversarial loss.

[0372] Since the object modal features extracted by this encoder and the visual encoder may contain information irrelevant to object preference, this application designs an adversarial self-supervised learning task to obtain interactively related modal embeddings.

[0373] Specifically, the computer device can calculate the similarity between the modal-aware sample object embedding of any sample object and the modal-aware sample item embedding of any sample item, construct a virtual similarity matrix based on the similarities, and use the virtual similarity matrix as a sample virtual interaction graph.

[0374] Exemplarily, the computer device may construct a sample virtual interaction graph by the following formula:

[0375]

[0376] in, represents a sample virtual interaction graph based on modality m, where modality m can specifically be a text modality or an image modality. is the modality-aware sample object embedding, Sample item embeddings for modality awareness.

[0377] Furthermore, the computer device can determine the sample target interaction graph based on the sample interaction graph, for example, the sample interaction graph can be directly used as the sample target interaction graph, or the sample interaction graph can be normalized to obtain the sample target interaction graph, or the sample interaction graph can be normalized and then enhanced information can be superimposed to obtain the sample target interaction graph.

[0378] In some embodiments, determining a sample target interaction graph based on the sample interaction graph includes: normalizing the sample interaction graph to obtain a normalized sample interaction graph; determining auxiliary information based on a predicted object representation and a predicted item representation; and determining a sample target interaction graph based on the normalized sample interaction graph and the auxiliary information.

[0379] Specifically, the computer device may perform normalization processing on each element in the sample interaction graph to obtain a normalized sample interaction graph. The auxiliary information is determined according to the similarity between the predicted object representation and the predicted item. The sample interaction graph is enhanced by the auxiliary information, that is, the normalized sample interaction graph and the auxiliary information are weighted to obtain a sample target interaction graph.

[0380] Exemplarily, the computer device may determine the sample target interaction graph by the following formulas 33-35:

[0381]

[0382]

[0383] g = -log(-log(uni)), uni~Uniform(0,1) (Formula 35);

[0384] in, is the representation of a matrix element (a, b) in the sample target interaction graph, G inter (a,b) is the representation of a matrix element (a, b) in the sample interaction graph. g is random noise, τ is the temperature coefficient, and γ is the weight adopted to control the strength of auxiliary information. φ represents auxiliary information, is the prediction object representation, is the predicted item representation.

[0385] In the above embodiment, the real sample interaction graph is normalized to be converted into a continuous interaction graph, and then auxiliary information is provided by the predicted object representation and predicted item representation obtained by the final prediction to enhance the real interaction graph. On the one hand, the real interaction graph can be smoothed, and on the other hand, the model training can be made more robust.

[0386] Furthermore, after constructing the sample target interaction graph, the computer device can determine the modal adversarial loss based on the difference between the sample virtual interaction graph and the sample target interaction graph.

[0387] In some embodiments, an auxiliary discriminator may be introduced to distinguish between sample virtual interaction graphs and sample target interaction graphs. The discriminator may be constructed through a nonlinear network, for example, it may be expressed in the following form: sigmoid(Drop(BN(LeakyReLU(Linear(x))))). Wherein, sigmoid refers to an activation function, Drop refers to a method of regularizing a neural network, BN refers to Batch-Normalization, LeakyReLU refers to an activation function, and Linear refers to a linear layer.

[0388] Exemplarily, the computer device may determine the modal adversarial loss by the following formula:

[0389]

[0390]

[0391] Among them, D represents the output of the discriminator, E represents the expectation, express and interpolation, p represents the gradient penalty value, which is used to constrain the gradient, and its weight is ξ.

[0392] Furthermore, after the computer device determines the modal adversarial loss, the first loss function may be determined based on the prediction loss and the modal adversarial loss. In some embodiments, the computer device may determine the first loss function based on the prediction loss, the contrast loss, and the modal adversarial loss. For example, the first loss function may be obtained by weighting the prediction loss, the contrast loss, and the modal adversarial loss.

[0393] Exemplarily, the computer device may determine the first loss function by the following formula:

[0394] L Rec =L BPR +λ adv L adv +λ contr L contr +λ reg ||θ|| 2 (Formula 38);

[0395] Among them, λ adv ,λ contr and reg They represent the weights used to control different loss terms, and θ is a learnable model parameter.

[0396] In the above embodiment, a sample virtual interaction graph is constructed based on the similarity between the modality-aware sample object embedding of any sample object and the modality-aware sample item embedding of any sample item. The modality adversarial loss is determined based on the difference between the sample virtual interaction graph and the sample target interaction graph. Thus, the prediction loss, contrast loss and modality adversarial loss are combined to perform multi-task learning, thereby achieving joint optimization of the recommendation task and multiple auxiliary tasks.

[0397] It should be noted that the data recommendation model processing method provided in this application can be applied to a variety of scenarios, such as e-commerce scenarios, video recommendation scenarios, advertising push scenarios, etc. When applied in various scenarios, the actual training data is derived from the data of the scenario.

[0398] In a specific embodiment, the data recommendation model processing method includes the following steps:

[0399] Obtain the sample item knowledge graph, the sample interaction graph between the sample object and the sample item, and the sample modal content of the sample item. Fuse the sample item knowledge graph and the sample interaction graph to obtain the sample collaborative knowledge graph, determine the sample item relationship graph based on the sample modal content, and construct the sample object co-occurrence graph based on the sample interaction graph.

[0400] Through the data recommendation model to be trained, the modality-aware sample object embedding and sample item embedding are determined according to the sample interaction graph and the sample modal content of the sample item; graph attention processing is performed based on the sample collaborative knowledge graph to obtain the knowledge-aware sample object embedding and sample item embedding; based on the modality-aware sample object embedding and knowledge-aware sample item embedding, the sample object characteristics are determined; based on the modality-aware sample item embedding and knowledge-aware sample item embedding, the sample item characteristics are determined.

[0401] Through the data recommendation model to be trained, isomorphic graph learning is performed on the sample object co-occurrence graph and the sample object features to obtain the predicted object representation. Isomorphic graph learning is performed based on the sample item relationship graph and the sample item features to obtain the predicted item representation.

[0402] A first sample matching degree is determined based on the predicted object representation of the sample object and the predicted item representation of the positive item; the positive item is an item that interacts with the sample object. A second sample matching degree is determined based on the predicted object representation of the sample object and the predicted item representation of the negative item; the negative item is an item that does not interact with the sample object. A prediction loss is determined based on the difference between the first sample matching degree and the second sample matching degree.

[0403] The similarity between modality-aware sample object embeddings and predicted object representations is determined, and a contrastive loss for object preference is determined based on the similarity.

[0404] According to the similarity between the modality-aware sample object embedding of any sample object and the modality-aware sample item embedding of any sample item, a sample virtual interaction graph is constructed; the sample interaction graph is normalized to obtain a normalized sample interaction graph; auxiliary information is determined according to the predicted object representation and the predicted item representation; a sample target interaction graph is determined according to the normalized sample interaction graph and the auxiliary information. According to the difference between the sample virtual interaction graph and the sample target interaction graph, a modality adversarial loss is determined.

[0405] The first loss function is determined according to the prediction loss, contrast loss and modality adversarial loss.

[0406] Based on the knowledge graph of sample items, valid multi-groups and invalid multi-groups are constructed; the score of the first multi-group is determined according to the identifier embedding of the entities and edges in the valid multi-group; the score of the second multi-group is determined according to the identifier embedding of the entities and edges in the invalid multi-group; based on the difference between the score of the first multi-group and the score of the second multi-group, a second loss function is constructed.

[0407] The data recommendation model to be trained is trained alternately by using the first loss function and the second loss function, and a data recommendation model is obtained after the training is completed.

[0408] The above-mentioned data recommendation model processing method constructs a sample collaborative knowledge graph based on the sample interaction graph and sample item knowledge graph observed between the sample objects and the sample items, and obtains the sample object features and sample item features associated with the interaction through the way of side information perception, thereby reducing the influence of irrelevant side information on the recommendation performance of hot start items. In addition, a sample item relationship graph is constructed according to the sample modal content of the sample items, and a sample object co-occurrence graph is constructed according to the similarity of the interaction behaviors between the sample objects. Based on the sample item relationship graph and the sample object co-occurrence graph, the sample object features and sample item features encoding the interaction relationship between the sample objects and the sample items are propagated and aggregated to obtain the predicted object representation and the predicted item representation. The prediction loss is determined according to the predicted object representation and the predicted item representation, and the data recommendation model to be trained is trained according to the prediction loss, and the data recommendation model is obtained after the training is completed. The data recommendation model trained in this way can, on the one hand, mine the association structure between hot start items and objects, and on the other hand, transfer the features of hot start items to strict cold start items, make up for the representation of strict cold start items due to the lack of interaction data, and can greatly improve the recommendation performance of strict cold start items.

[0409] In a specific embodiment, please refer to Fig.11 ,The data recommendation method includes the following steps:

[0410] Obtain the knowledge graph (i.e. the item knowledge graph mentioned above), the user-item interaction graph, and the modal content of the item (including text content and image content).

[0411] The knowledge graph and the user-item interaction graph are integrated to obtain a collaborative knowledge graph. The item relationship graph (including text-based item relationship graph and image-based item relationship graph) is determined based on the modal content of the items. The user co-occurrence graph is constructed based on the interaction graph.

[0412] Based on the interaction graph, the user's ID embedding (ID embedding), the hot start item's ID embedding (ID embedding), and the strict cold start item's ID embedding (ID embedding), graph convolution processing is performed to obtain the behavior-aware first object embedding and first item embedding.

[0413] Determining a modality-aware second object embedding and a second item embedding based on the interaction graph and the modal content of the items (including content features of the hot start items and content features of the strictly cold start items);

[0414] Based on the collaborative knowledge graph, the identity embedding (ID embedding) of users, the identity embedding (ID embedding) of hot start items, the identity embedding (ID embedding) of strict cold start items, and the identity embedding (ID embedding) of general entities, graph attention processing is performed to obtain knowledge-aware third object embedding and third item embedding.

[0415] The first object embedding, the second object embedding and the third object embedding are fused to obtain fused object features. The first item embedding, the second item embedding and the third item embedding are fused to obtain item features (including item features of hot start items and item features of strictly cold start items).

[0416] Based on the user co-occurrence graph and object features, graph attention processing is performed to obtain the final target user representation. Based on the item relationship graph and item features, graph convolution processing is performed to obtain item text representation and item image representation. Through multi-head self-attention and mean pooling, the item text representation and item image representation are fused to obtain the target item representation.

[0417] According to the matching degree between the target user representation of the user and the target item representation of the hot start item, the hot start item is recommended to the user; according to the matching degree between the target user representation of the user and the target item representation of the strict cold start item, the strict cold start item is recommended to the user.

[0418] The above data recommendation method constructs a collaborative knowledge graph based on the observed interaction graph between objects and items and the item knowledge graph, and obtains the object features and item features associated with the interaction through the way of side information perception, thereby reducing the impact of irrelevant side information on the recommendation performance of hot start items. In addition, an item relationship graph is constructed according to the modal content of the items, and an object co-occurrence graph is constructed according to the similarity of the interaction behaviors between objects. Based on the item relationship graph and the object co-occurrence graph, the object features and item features that encode the interaction relationship between objects and items are propagated and aggregated. On the one hand, the association structure between hot start items and objects can be mined, and on the other hand, the features of hot start items are transferred to strict cold start items to make up for the representation of strict cold start items due to the lack of interaction data, which can greatly improve the recommendation performance of strict cold start items. This application utilizes the modal content of items, the side information of the item knowledge graph, and the interaction information in the interaction graph. Through the above-mentioned heterogeneous graph learning and isomorphic graph learning methods, the recommendation performance of strict cold start items is effectively improved while maintaining the recommendation performance of hot start items.

[0419] The following is an explanation of the experimental process of the data recommendation method provided in this application:

[0420] (1) Experimental data

[0421] The effect test of this application uses three public recommendation datasets from a public platform for experiments. After reorganization, it is used to uniformly evaluate the recommendation performance of models using different side information in strict cold start scenarios.

[0422] The dataset used for the experiment contains user comments on various types of products on a shopping platform, as well as text descriptions and image information of the products. This application conducts experiments on three types of product subsets: Beauty, Cell Phones, and Clothing. In the dataset, each rating is regarded as a positive user-item interaction record. For each subset of raw data, users and items are filtered, and the statistical data of the processed dataset are shown in Table 1, where the data sparsity is calculated using the following formula:

[0423]

[0424] Among them, |Inter|, and Represent the number of interactions between users and items, the number of users, and the number of items respectively.

[0425] Table 2 Statistical data of experimental data sets and construction of collaborative knowledge graph

[0426]

[0427] For the visual information of the product, a 4096-dimensional visual vector that has been pre-extracted and made public is used. For the text information of the product, the title and text description of the product are first concatenated into a complete sentence, and then Sentence-Transformers is used to extract 384-dimensional sentence features. For each subset, 20% of the items are randomly sampled from all items as strict cold start items, and the interaction data associated with these items are split into a strict cold start validation set and a strict cold start test set in a 1:1 ratio. The remaining items are used as hot start items, and the associated interaction data are divided into a training set, a hot start validation set, and a hot start test set in a ratio of 8:1:1. The validation set and the test set are used to determine the hyperparameters and evaluate the model performance, respectively. In order to test the knowledge graph-based method on the dataset, a knowledge graph is constructed for each subset. Specifically, the knowledge graph contains four types of entities and six types of relationships. The specific description of each type of entity and relationship is as follows. Figure X As shown in the figure. Since the "feature" entity from the review data may contain words that are redundant and not very meaningful for reflecting item features, the TF-IDF algorithm is used in the preprocessing stage to eliminate features that are not significant enough. For each data set, feature words with a frequency between 10 and 1000 and a TF-IDF score of 0.1 are retained.

[0428] (2) Experimental settings and evaluation indicators

[0429] The experiment uses a full ranking method to calculate the performance of the recommendation system. Under the hot start recommendation setting, all items that have not been interacted with by the user are considered candidate items. Under the strict cold start recommendation setting, all strict cold start items are considered candidate items. The five indicators widely used in Top-K recommendation, Recall, Mean Reciprocal Ranking (MRR), Normalized Discounted Cumulative Gain (NDCG), Hit Ratio (HR) and Precision, are used to evaluate the experimental results. In this experiment, K=20 is uniformly set, and the five indicators are simplified as: R@20, M@20, N@20, H@20, P@20. The larger the value of the five indicators, the better the recommendation result in the cold start or hot start scenario. In addition, in order to simultaneously evaluate the performance balance of the model in the strict cold start and hot start scenarios, the harmonic mean of each indicator in the strict cold start and hot start scenarios is calculated:

[0430]

[0431] The harmonic mean gives equal importance to strict cold start and warm start recommendations, and penalizes models that fall short on one of the attributes.

[0432] (3) Comparison method

[0433] This application conducted experimental comparisons with various types of recommendation models: 1. General collaborative filtering method (CF); 2. Knowledge-aware recommendation method (KG); 3. Multimodal recommendation method (MM); 4. Method for cold start of items (CS); 5. Recommendation method based on multimodal content and knowledge graph (MM+KG).

[0434] (4) Experimental results

[0435] Table 3. Comparative experimental results of strict item cold start and hot start on the Beauty dataset. Underline They represent the best method and the second best method among all the compared methods. All the numbers in the table represent percentage values, and "%" is omitted.

[0436]

[0437]

[0438]

[0439] Table 4 Comparative experimental results of strict item cold start and hot start on Cell Phones dataset. Underline Wire They represent the best method and the second best method among all the compared methods. All the numbers in the table represent percentage values, and "%" is omitted.

[0440]

[0441]

[0442]

[0443] Table 5 Comparative experimental results of strict item cold start and hot start on Clothing dataset. Underline They represent the best method and the second best method among all the compared methods. All the numbers in the table represent percentage values, and "%" is omitted.

[0444]

[0445]

[0446]

[0447] As can be seen from Tables 3 to 5, the present application has generally achieved better strict item cold start and hot start recommendation effects on multiple data sets. The side information-aware heterogeneous graph learning in the present application can enable hot start items to capture the collaborative signals of behavior perception, modality perception, and knowledge perception, while strict cold start items can capture the semantics of items related to interaction, including multimodal content related to interactive behavior and knowledge graph representation. The modality-specific item relationship graph can transfer collaborative signals based on the semantic structure between items, especially from hot start items to strict cold start items.

[0448] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0449] Based on the same inventive concept, the embodiment of the present application also provides a data recommendation device for implementing the data recommendation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data recommendation device embodiments provided below can refer to the limitations on the data recommendation method above, and will not be repeated here.

[0450] In one embodiment, Fig.12 As shown, a data recommendation device 1200 is provided, comprising: an acquisition module 1201, a construction module 1202, a heterogeneous graph learning module 1203, a homogeneous graph learning module 1204 and a recommendation module 1205, wherein:

[0451] The acquisition module 1201 is used to obtain the item knowledge graph, the interaction graph between the object and the item, and the modal content of the item.

[0452] The construction module 1202 is used to fuse the item knowledge graph and the interaction graph to obtain a collaborative knowledge graph, determine the item relationship graph based on the modal content of the items, and construct an object co-occurrence graph based on the interaction graph.

[0453] The heterogeneous graph learning module 1203 is used to perform heterogeneous graph learning based on the interaction graph and the collaborative knowledge graph to obtain object features and item features.

[0454] The isomorphic graph learning module 1204 is used to perform isomorphic graph learning based on the object co-occurrence graph and the object features to obtain the target object representation, and to perform isomorphic graph learning based on the item relationship graph and the item features to obtain the target item representation.

[0455] The recommendation module 1205 is used to recommend items to the target object according to the target object representation of the target object and the target item representations of each item.

[0456] In one embodiment, a construction module is specifically used to: determine the edges in the interaction graph, and convert each edge in the interaction graph into a multi-tuple; the entities in the multi-tuple are the objects and items connected by the edges in the interaction graph; based on the entities in the multi-tuple and the entities in the item knowledge graph, the interaction graph and the item knowledge graph are fused by entity alignment to obtain a collaborative knowledge graph.

[0457] In one embodiment, the construction module is specifically used to: construct an original relationship graph based on the modal content of the items; determine a mask matrix according to the startup category corresponding to the items; the mask matrix is ​​used to isolate the information transfer from the strict cold start items to the hot start items; determine the item relationship graph according to the original relationship graph and the mask matrix.

[0458] In one embodiment, the construction module is specifically used to: calculate the similarity between any two items based on the modal content of any two items; construct a first similarity matrix based on the similarity between any two items; perform sparse processing on the first similarity matrix to obtain an unweighted matrix, and determine the original relationship graph based on the unweighted matrix.

[0459] In one embodiment, the construction module is specifically used to: for any row in the first similarity matrix, sort the values ​​of multiple elements in the row from large to small; use the element with a ranking less than a preset ranking as the first element, and the other elements as the second element; update the value of the first element in the row to a first numerical value, and update the value of the second element in the row to a second numerical value, the second numerical value being less than the first numerical value; after the values ​​of the elements in each row of the first similarity matrix are updated, an unweighted matrix is ​​obtained.

[0460] In one embodiment, the construction module is specifically used to: determine the number of items that any two objects interact with in common according to the interaction graph; construct a second similarity matrix according to the number of items that any two objects interact with in common; and perform sparse processing on the second similarity matrix to obtain an object co-occurrence graph.

[0461] In one embodiment, the construction module is specifically used to: for any row in the second similarity matrix, sort the values ​​of multiple elements in the row from large to small; use the elements with a ranking less than a preset ranking as the third element, and the other elements as the fourth element; retain the value of the third element in the row, and update the value of the fourth element in the row to the second numerical value; after each row of the second similarity matrix is ​​updated, obtain an object co-occurrence map.

[0462] In one embodiment, the heterogeneous graph learning module is specifically used to: perform graph convolution processing based on the interaction graph to obtain a behavior-aware first object embedding and a first item embedding; determine a modality-aware second object embedding and a second item embedding according to the interaction graph and the modal content of the item; perform graph attention processing based on the collaborative knowledge graph to obtain a knowledge-aware third object embedding and a third item embedding; fuse the first object embedding, the second object embedding, and the second object embedding to obtain object features; fuse the first item embedding, the second item embedding, and the third item embedding to obtain item features.

[0463] In one embodiment, the heterogeneous graph learning module is specifically used to: perform multiple graph convolution processes based on the interaction graph and obtain the output of each graph convolution process; for any object node, aggregate the embedding corresponding to the targeted object node in the output of each graph convolution process to obtain the first object embedding of the targeted object node; for any item node, aggregate the embedding corresponding to the targeted item node in the output of each graph convolution process to obtain the first item embedding of the targeted item node.

[0464] In one embodiment, the interaction graph includes object nodes and item nodes, and the heterogeneous graph learning module is specifically used to: for any object node, determine the adjacent item nodes according to the interaction graph, and determine the second object embedding of the targeted object node according to the modal content of the adjacent item nodes; for any item node, determine the adjacent object nodes according to the interaction graph, and determine the second item embedding of the targeted item node according to the second object embedding features of the adjacent object nodes.

[0465] In one embodiment, the heterogeneous graph learning module is specifically used to: determine the egocentric network of any entity in the collaborative knowledge graph; entities include object entities and item entities; determine the importance weight of each multi-group in the egocentric network, and determine the representation of the egocentric network based on the importance weight of each multi-group; for any object entity, aggregate the identity embedding of the object entity and the corresponding representation of the egocentric network to obtain a third object embedding of the object entity; for any item entity, aggregate the identity embedding of the item entity and the corresponding representation of the egocentric network to obtain a third item embedding of the item entity.

[0466] In one embodiment, the heterogeneous graph learning module is specifically used to: for any multi-group in the ego-centric network, determine the importance of the multi-group based on the identity embedding of the entities and edges in the multi-group; based on the importance of all multi-groups in the ego-centric network, normalize the importance of each multi-group to obtain the importance weight of each multi-group.

[0467] In one embodiment, the heterogeneous graph learning module is specifically used to: perform an addition operation on the identity embedding of the targeted object entity and the representation of the corresponding egocentric network to obtain an addition operation result; perform an element-by-element multiplication operation on the identity embedding of the targeted object entity and the representation of the corresponding egocentric network to obtain a multiplication operation result; and fuse the addition operation result and the multiplication operation result to obtain a third object embedding of the targeted object entity.

[0468] In one embodiment, the modality-aware second object embedding includes text-aware second object text embedding and image-aware second object image embedding; the heterogeneous graph learning module is specifically used to: determine a first weight corresponding to the second object text embedding, and a second weight corresponding to the second object image embedding; determine the object multimodal embedding according to the first weight, the second object text embedding, the second weight, and the second object image embedding; and perform weighted processing on the first object embedding, the object multimodal embedding, and the third object embedding to obtain object features.

[0469] In one embodiment, the modality-aware second item embedding, including text-aware second item text embedding and image-aware second item image embedding, and the heterogeneous graph learning module are specifically used to: construct a text-based first virtual interaction graph according to the similarity between the second object text embedding of any object and the second item text embedding of any item; construct an image-based second virtual interaction graph according to the similarity between the second object image embedding of any object and the second object image embedding of any item; determine a first discriminant score for characterizing the difference between the first virtual interaction graph and the interaction graph; determine a second discriminant score for characterizing the difference between the second virtual interaction graph and the interaction graph; determine a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding according to the first discriminant score and the second discriminant score.

[0470] In one embodiment, the isomorphic graph learning module is specifically used to: use the object co-occurrence graph as the first adjacency matrix, and construct a first input matrix according to the object characteristics of each object; perform graph attention processing based on the first adjacency matrix and the first input matrix to obtain a first output matrix; the elements in the first output matrix are representations of the target objects.

[0471] In one embodiment, the item relationship graph includes a text-based item relationship graph and an image-based item relationship graph; the isomorphic graph learning module is specifically used to: use the text-based item relationship graph as the second adjacency matrix, and use the image-based item relationship graph as the third adjacency matrix; construct a second input matrix according to the item features of each item; perform graph convolution processing based on the second adjacency matrix and the second input matrix to obtain a second output matrix; the elements in the second output matrix are item text representations; perform graph convolution processing based on the third adjacency matrix and the second input matrix to obtain a third output matrix; the elements in the third output matrix are item image representations; and fuse the item text representation and the item image representation to obtain the target item representation.

[0472] In one embodiment, the isomorphic graph learning module is specifically used to: perform multi-head attention processing based on the object text representation and the object image representation; and perform mean pooling on the results after the multi-head attention processing to obtain the target object representation.

[0473] In one embodiment, the recommendation module is specifically used to obtain a target object representation of the target object; calculate the matching degree between the target object representation of the target object and the target object representation of each item, and recommend items whose matching degree meets the preset matching conditions to the target object.

[0474] In one embodiment, the item includes hot start media data and strict cold start media data, the interaction graph between the object and the item includes an interaction graph between the user and the media data, and the modal content of the item includes text content and image content in the media data.

[0475] Based on the same inventive concept, the embodiment of the present application also provides a data recommendation model processing device for implementing the data recommendation model processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data recommendation model processing device embodiments provided below can refer to the limitations on the data recommendation model processing method above, and will not be repeated here.

[0476] In one embodiment, Fig.13 As shown, a data recommendation model processing device 1300 is provided, including: an acquisition module 1301, a construction module 1302, a heterogeneous graph learning module 1303, a homogeneous graph learning module 1304, a determination module 1305 and a training module 1306, wherein:

[0477] The acquisition module 1301 is used to obtain the sample item knowledge graph, the sample interaction graph between the sample object and the sample item, and the sample modal content of the sample item.

[0478] Construction module 1302 is used to fuse the sample item knowledge graph and the sample interaction graph to obtain the sample collaborative knowledge graph, determine the sample item relationship graph based on the sample modal content, and construct the sample object co-occurrence graph based on the sample interaction graph.

[0479] The heterogeneous graph learning module 1303 is used to perform heterogeneous graph learning on the sample interaction graph and the sample collaborative knowledge graph through the data recommendation model to be trained, so as to obtain the sample object features and the sample item features.

[0480] The isomorphic graph learning module 1304 is used to perform isomorphic graph learning on the sample object co-occurrence graph and the sample object features through the data recommendation model to be trained to obtain the predicted object representation, and to perform isomorphic graph learning based on the sample item relationship graph and the sample item features to obtain the predicted item representation.

[0481] The determination module 1305 is used to determine the prediction loss according to the prediction object representation and the prediction item representation, and construct a first loss function according to the prediction loss.

[0482] The training module 1306 is used to train the data recommendation model to be trained by using the first loss function, and obtain the data recommendation model after the training is completed.

[0483] In one embodiment, the determination module is also used to construct valid multi-groups and invalid multi-groups based on the knowledge graph of sample items; determine the first multi-group score based on the identity embedding of entities and edges in the valid multi-group; determine the second multi-group score based on the identity embedding of entities and edges in the invalid multi-group; construct a second loss function based on the difference between the first multi-group score and the second multi-group score; the training module is also used to train the data recommendation model to be trained alternately through the first loss function and the second loss function, and obtain the data recommendation model after the training is completed.

[0484] In one embodiment, the determination module is specifically used to: determine a first sample matching degree based on a predicted object representation of the sample object and a predicted item representation of a positive item; a positive item is an item that interacts with the sample object; determine a second sample matching degree based on the predicted object representation of the sample object and a predicted item representation of a negative item; a negative item is an item that does not interact with the sample object; and determine a prediction loss based on a difference between the first sample matching degree and the second sample matching degree.

[0485] In one embodiment, a heterogeneous graph learning model is specifically used to determine modality-aware sample object embedding and sample item embedding based on sample interaction graphs and sample modal content of sample items; perform graph attention processing based on sample collaborative knowledge graphs to obtain knowledge-aware sample object embedding and sample item embedding; determine sample object features based on modality-aware sample object embedding and knowledge-aware sample item embedding; determine sample item features based on modality-aware sample item embedding and knowledge-aware sample item embedding.

[0486] In one embodiment, the determination module is further used to determine the similarity between the modality-aware sample object embedding and the predicted object representation, and determine the contrast loss of the object preference based on the similarity; and determine the first loss function based on the prediction loss and the contrast loss.

[0487] In one embodiment, the determination module is also used to: construct a sample virtual interaction graph based on the similarity between the modal-aware sample object embedding of any sample object and the modal-aware sample item embedding of any sample item; determine the sample target interaction graph based on the sample interaction graph; determine the modal adversarial loss based on the difference between the sample virtual interaction graph and the sample target interaction graph; and determine the first loss function based on the prediction loss, contrast loss and modal adversarial loss.

[0488] In one embodiment, the determination module is further used to: normalize the sample interaction graph to obtain a normalized sample interaction graph; determine auxiliary information based on the predicted object representation and the predicted item representation; and determine a sample target interaction graph based on the normalized sample interaction graph and the auxiliary information.

[0489] Each module in the above-mentioned data recommendation device and / or data recommendation model processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0490] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.14As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store graph data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data recommendation method and / or a data recommendation model processing method is implemented.

[0491] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0492] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0493] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0494] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0495] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0496] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0497] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0498] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data recommendation method, characterized in that: The method comprises: Obtain the knowledge graph of items, the interaction graph between objects and items, and the modal content of items; The item knowledge graph and the interaction graph are integrated to obtain a collaborative knowledge graph, an item relationship graph is determined based on the modal content of the items, and an object co-occurrence graph is constructed based on the interaction graph; Perform heterogeneous graph learning based on interaction graphs and collaborative knowledge graphs to obtain object features and item features; Performing isomorphic graph learning based on the object co-occurrence graph and the object features to obtain a target object representation, and performing isomorphic graph learning based on the item relationship graph and the item features to obtain a target item representation; According to the target object representation of the target object and the target item representation of each item, an item recommendation is performed for the target object.

2. The method according to claim 1, characterized in that The determining of the item relationship graph based on the modal content of the item includes: constructing an original relationship graph based on the modal content of the items; Determining a mask matrix according to the startup category corresponding to the item; the mask matrix is ​​used to isolate information transfer from a strictly cold-start item to a hot-start item; An item relationship graph is determined according to the original relationship graph and the mask matrix.

3. The method according to claim 2, characterized in that The constructing an original relationship graph based on the modal content of the item includes: Calculate the similarity between any two items according to the modal contents of the two items; Constructing a first similarity matrix based on the similarity between the arbitrary two items; The first similarity matrix is ​​subjected to sparse processing to obtain an unweighted matrix, and an original relationship graph is determined according to the unweighted matrix.

4. The method according to claim 1, characterized in that: The constructing an object co-occurrence graph based on the interaction graph includes: According to the interaction graph, determining the number of items that any two objects interact with in common; According to the number of items that any two objects interact with in common, a second similarity matrix is ​​constructed; The second similarity matrix is ​​subjected to sparse processing to obtain an object co-occurrence graph.

5. The method according to claim 1, characterized in that The heterogeneous graph learning based on the interaction graph and the collaborative knowledge graph to obtain object features and item features includes: Performing graph convolution processing based on the interaction graph to obtain a behavior-aware first object embedding and a first item embedding; determining a modality-aware second object embedding and a second item embedding based on the interaction graph and the modal content of the item; Performing graph attention processing based on the collaborative knowledge graph to obtain knowledge-aware third object embedding and third item embedding; fusing the first object embedding, the second object embedding, and the second object embedding to obtain an object feature; The first item embedding, the second item embedding and the third item embedding are integrated to obtain an item feature.

6. The method according to claim 5, characterized in that The interaction graph includes object nodes and item nodes; and performing graph convolution processing based on the interaction graph to obtain a behavior-aware first object embedding and a first item embedding includes: Perform multiple graph convolution processes based on the interaction graph, and obtain the output of each graph convolution process; For any object node, aggregate the embedding corresponding to the object node in the output of each graph convolution process to obtain the first object embedding of the object node; For any item node, aggregate the embeddings corresponding to the item node in the output of each graph convolution process to obtain the first item embedding of the item node.

7. The method according to claim 5, characterized in that The interaction graph includes object nodes and item nodes, and determining a modality-aware second object embedding and a second item embedding according to the interaction graph and the modal content of the item includes: For any object node, determine the adjacent item node according to the interaction graph, and determine the second object embedding of the targeted object node according to the modal content of the adjacent item node; For any item node, the adjacent object nodes are determined according to the interaction graph, and the second item embedding of the item node is determined according to the second object embedding features of the adjacent object nodes.

8. The method according to claim 5, characterized in that The performing graph attention processing based on the collaborative knowledge graph to obtain a knowledge-aware third object embedding and a third item embedding includes: For any entity in the collaborative knowledge graph, determine the egocentric network of the targeted entity; the entity includes an object entity and an item entity; Determining an importance weight of each multi-group in the egocentric network, and determining a representation of the egocentric network based on the importance weight of each multi-group; For any object entity, aggregate the identity embedding of the object entity and the representation of the corresponding egocentric network to obtain a third object embedding of the object entity; For any item entity, the identification embedding of the item entity and the corresponding egocentric network representation are aggregated to obtain the third item embedding of the item entity.

9. The method according to claim 5, characterized in that The modality-aware second object embedding includes a text-aware second object text embedding and an image-aware second object image embedding; The fusing the first object embedding, the second object embedding, and the second object embedding to obtain an object feature includes: Determining a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding; Determine an object multimodal embedding according to the first weight, the second object text embedding, the second weight, and the second object image embedding; The first object embedding, the object multimodal embedding, and the third object embedding are weighted to obtain object features.

10. The method according to claim 9, characterized in that The modality-aware second object embedding includes a text-aware second object text embedding and an image-aware second object image embedding, and determining a first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding includes: constructing a first text-based virtual interaction graph according to the similarity between the second object text embedding of any object and the second item text embedding of any item; constructing an image-based second virtual interaction graph according to a similarity between a second object image embedding of any object and a second object image embedding of any item; determining a first discriminant score for characterizing a difference between the first virtual interaction graph and the interaction graph; determining a second discriminant score for characterizing a difference between the second virtual interaction graph and the interaction graph; A first weight corresponding to the second object text embedding and a second weight corresponding to the second object image embedding are determined according to the first discriminant score and the second discriminant score.

11. The method according to claim 1, characterized in that: The performing isomorphic graph learning based on the object co-occurrence graph and the object features to obtain a target object representation includes: Using the object co-occurrence graph as a first adjacency matrix, constructing a first input matrix according to the object features of each object; Graph attention processing is performed based on the first adjacency matrix and the first input matrix to obtain a first output matrix; the elements in the first output matrix are representations of target objects.

12. The method according to claim 1, characterized in that The item relationship graph includes a text-based item relationship graph and an image-based item relationship graph; performing isomorphic graph learning based on the item relationship graph and the item features to obtain a target item representation includes: Using the text-based item relationship graph as a second adjacency matrix and using the image-based item relationship graph as a third adjacency matrix; constructing a second input matrix based on the item characteristics of each item; Performing graph convolution processing based on the second adjacency matrix and the second input matrix to obtain a second output matrix; the elements in the second output matrix are text representations of items; Performing graph convolution processing based on the third adjacency matrix and the second input matrix to obtain a third output matrix; the elements in the third output matrix are object image representations; The object text representation and the object image representation are fused to obtain a target object representation.

13. A data recommendation model processing method, characterized in that: The method comprises: Obtaining a sample item knowledge graph, a sample interaction graph between a sample object and a sample item, and a sample modal content of the sample item; The sample item knowledge graph and the sample interaction graph are integrated to obtain a sample collaborative knowledge graph, a sample item relationship graph is determined based on the sample modal content, and a sample object co-occurrence graph is constructed based on the sample interaction graph; Through the data recommendation model to be trained, heterogeneous graph learning is performed on the sample interaction graph and the sample collaborative knowledge graph to obtain sample object features and sample item features; Through the data recommendation model to be trained, performing isomorphic graph learning on the sample object co-occurrence graph and the sample object features to obtain a predicted object representation, and performing isomorphic graph learning based on the sample item relationship graph and the sample item features to obtain a predicted item representation; Determining a prediction loss according to the prediction object representation and the prediction item representation, and constructing a first loss function according to the prediction loss; The data recommendation model to be trained is trained using the first loss function, and a data recommendation model is obtained after the training is completed.

14. The method according to claim 13, characterized in that The method further comprises: Based on the sample item knowledge graph, construct valid multi-tuples and invalid multi-tuples; Determining a first multi-tuple score based on the identifier embeddings of the entities and edges in the valid multi-tuple; Determining a second multi-tuple score based on the identifier embeddings of the entities and edges in the invalid multi-tuple; constructing a second loss function based on a difference between the first multi-group score and the second multi-group score; The step of training the data recommendation model to be trained by using the first loss function to obtain the data recommendation model after the training is completed includes: The data recommendation model to be trained is trained alternately by using the first loss function and the second loss function, and a data recommendation model is obtained after the training is completed.

15. The method according to claim 13, characterized in that The determining the predicted loss according to the predicted object representation and the predicted item representation includes: Determining a first sample matching degree according to a predicted object representation of the sample object and a predicted item representation of a positive item; the positive item is an item that interacts with the sample object; Determining a second sample matching degree according to the predicted object representation of the sample object and the predicted item representation of a negative item; the negative item is an item that has not interacted with the sample object; A prediction loss is determined according to a difference between the first sample matching degree and the second sample matching degree.

16. The method according to claim 13, characterized in that The performing heterogeneous graph learning on the sample interaction graph and the sample collaborative knowledge graph to obtain sample object features and sample item features includes: determining a modality-aware sample object embedding and a sample item embedding based on the sample interaction graph and the sample modal content of the sample item; Performing graph attention processing based on the sample collaborative knowledge graph to obtain knowledge-aware sample object embedding and sample item embedding; Determining sample object features based on the modality-aware sample object embedding and the knowledge-aware sample item embedding; Based on the modality-aware sample item embedding and the knowledge-aware sample item embedding, sample item features are determined.

17. The method according to claim 16, characterized in that The method further comprises: determining a similarity between a modality-aware sample object embedding and the predicted object representation, and determining a contrastive loss for object preference based on the similarity; The constructing a first loss function according to the predicted loss includes: A first loss function is determined according to the prediction loss and the contrast loss.

18. The method according to claim 17, characterized in that The method further comprises: constructing a sample virtual interaction graph based on the similarity between the modality-aware sample object embedding of any sample object and the modality-aware sample item embedding of any sample item; Determine a sample target interaction graph based on the sample interaction graph; Determining a modal adversarial loss according to a difference between the sample virtual interaction graph and the sample target interaction graph; The determining a first loss function according to the prediction loss and the contrast loss includes: A first loss function is determined according to the prediction loss, the contrast loss, and the modality adversarial loss.

19. A data recommendation device, characterized in that: The device comprises: The acquisition module is used to obtain the item knowledge graph, the interaction graph between objects and items, and the modal content of items; A construction module, used to fuse the item knowledge graph and the interaction graph to obtain a collaborative knowledge graph, determine an item relationship graph based on the modal content of the items, and construct an object co-occurrence graph based on the interaction graph; Heterogeneous graph learning module, used to perform heterogeneous graph learning based on interaction graphs and collaborative knowledge graphs to obtain object features and item features; A homogeneous graph learning module, configured to perform homogeneous graph learning based on the object co-occurrence graph and the object features to obtain a target object representation, and perform homogeneous graph learning based on the item relationship graph and the item features to obtain a target item representation; The recommendation module is used to recommend items to the target object according to the target object representation of the target object and the target item representation of each item.

20. A data recommendation model processing device, characterized in that: The device comprises: An acquisition module, used to acquire a sample item knowledge graph, a sample interaction graph between a sample object and a sample item, and a sample modal content of the sample item; A construction module, used to fuse the sample item knowledge graph and the sample interaction graph to obtain a sample collaborative knowledge graph, determine a sample item relationship graph based on the sample modal content, and construct a sample object co-occurrence graph based on the sample interaction graph; A heterogeneous graph learning module, used to perform heterogeneous graph learning on the sample interaction graph and the sample collaborative knowledge graph through a data recommendation model to be trained, so as to obtain sample object features and sample item features; A homogeneous graph learning module is used to perform homogeneous graph learning on the sample object co-occurrence graph and the sample object features through the data recommendation model to be trained to obtain a predicted object representation, and to perform homogeneous graph learning based on the sample item relationship graph and the sample item features to obtain a predicted item representation; a determination module, configured to determine a prediction loss according to the prediction object representation and the prediction item representation, and construct a first loss function according to the prediction loss; The training module is used to train the data recommendation model to be trained by using the first loss function, and obtain the data recommendation model after the training is completed.

Images