Product Recommendation Method Based on Dual-Path Preference Enhancement

Abstract

The present invention discloses a product recommendation method based on dual - path preference enhancement, including: obtaining the final feature vectors of users and products; determining a first similarity between a user and a product according to the final feature vectors of the user and the product; the final feature vectors of the user and the product are obtained based on the feature vectors of the user and the product determined by a dual - path preference enhancement model based on self - supervised learning according to the user interaction preference and the modality preference; recommending products for the user according to the first similarity. According to the method provided by the present invention, by utilizing more factors, such as the user's modality preference and interaction preference, to determine the final feature vectors of the user and the product, and training a dual - path preference enhancement network model according to the self - supervised loss; it is possible to remove semantic noise in the product - product graph, improve the accuracy of identifying user preferences, and enhance the effectiveness of product recommendation for users.

Description

Technical Field

[0001] The present invention belongs to the technical field of information processing, and particularly relates to a product recommendation method based on dual-channel preference enhancement. Background Art

[0002] With the development of the Internet and social media, communicating, presenting, and interacting through multimodal information, such as image information and text information, has become a trend, resulting in a sharp increase in multimodal information. Therefore, how to recommend valuable information for users from a vast amount of information has become a popular research direction. Currently, most multimodal recommendation methods are used according to the characteristics of the information. The multimodal recommendation method encodes each user and product through user data and product data to obtain their feature vectors, and then calculates the similarity between the feature vectors of the user and the product, and recommends products for the user according to the similarity. However, the current multimodal recommendation method only shows the similarity relationship between products by using the multimodal information of the products to obtain the interaction preference of the user for the products or the modal preference of the user for the product information, and determines the feature vector of the user through this single factor. However, the feature vector of the user obtained by this method cannot well represent the preference of the user, and there is also noise in the product-product graph for determining the modal preference of the user. This results in a low effectiveness of recommending information for the user according to this method, that is, many of the products recommended for the user are not of interest to the user and the user will not view or click on them. Summary of the Invention

[0003] An embodiment of the present invention provides a product recommendation method based on dual-channel preference enhancement, which can solve the problem of low effectiveness of recommending products for users according to the current product recommendation method.

[0004] In a first aspect, an embodiment of the present invention provides a product recommendation method based on dual-channel preference enhancement. The method includes: obtaining a first user final feature vector of a user and first product final feature vectors of multiple products; determining a first similarity between the user and the multiple products according to the first user final feature vector and the multiple first product final feature vectors, wherein the first user final feature vector and the first product final vectors are determined by a trained dual-channel preference enhancement network model based on self-supervised learning according to an initial user ID feature vector of the user, a first initial text feature vector, and a first initial image feature vector, initial product ID feature vectors of the multiple products, a second initial text feature vector, and a second initial image feature. The initial user ID feature vector is a vector related to the user obtained according to the interaction preference of the user for product information, and the initial product ID feature vector is a vector related to the product obtained according to the interaction preference of the user for product information. The first initial image feature vector and the first initial text feature vector are vectors related to the user obtained according to the modality preference of the user for product information, and the second initial image feature vector and the second initial text feature vector are vectors related to the product obtained according to the modality preference of the user for product information. The dual-channel preference enhancement network model is used to enhance the second initial text feature vector and the second initial image feature vector to obtain an enhanced text feature vector and an enhanced image feature vector, and determine the self-supervised loss of the modality preference enhancement model according to the enhanced text feature vector and the enhanced image feature vector, and perform training under the constraint of the self-supervised loss; recommend products for the user according to the multiple first similarities.

[0005] In a possible implementation of the first aspect, before obtaining the first final feature vectors of the user and the products, the user data and the product data may be preprocessed to obtain the initial user ID feature vector of the user, the initial product ID feature vectors of multiple products, the first initial text feature vector, the first initial image feature vector, the second initial text feature vector, the second initial image feature vector, the user-product interaction graph, and the product-product graph. The initial user ID feature vector, the initial product ID feature vectors, the first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vector are input into the dual-path preference enhancement network model. Through the user-product interaction graph and the product-product graph, the second user final feature vector of the user, the second product final feature vectors of multiple products, the enhanced text feature vectors of multiple products, and the enhanced image feature vectors are obtained. Then, according to the second user final feature vector and the multiple second product final feature vectors, the second similarity between the user and the multiple products is determined. The Bayesian personalized ranking loss of the dual-path preference enhancement network model is determined according to the second similarity. The self-supervised loss of the dual-path preference enhancement network model is determined according to the enhanced text feature vectors and the enhanced image feature vectors. The dual-path preference enhancement network model is trained according to the Bayesian personalized ranking loss and the self-supervised loss to obtain the trained dual-path preference enhancement network model. The initial user ID feature vector, the initial product ID feature vectors of multiple products, the first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vector are input into the trained dual-path preference enhancement network model to obtain the first user final feature vector of the user and the first product final feature vectors of multiple products.

[0006] Exemplarily, the user data and the product data include the interaction data between the user and multiple products, the user information of the user, and the product information of multiple products. The user-product interaction graph is determined according to the interaction data between the user and multiple products, and the product-product graph is determined according to the product information.

[0007] In a possible implementation of the first aspect, the dual-path preference enhancement network model may specifically be used for:

[0008] Set two masks for the second initial image feature vector to respectively mask some neighbors in the product-product graph, and correspondingly obtain two masked product-product graphs. Then, perform graph convolution on the second initial image feature vector on the two masked product-product graphs respectively to obtain two enhanced image feature vectors.

[0009] Exemplarily, the masks follow the Bernoulli binary distribution with a probability of p.

[0010] Exemplarily, the enhanced image feature vectors satisfy the following formula: ·

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017] Among them, is the image feature of the l-th layer of the masked product-product graph, is the enhanced image feature vector obtained by the first graph convolution, is the enhanced image feature vector obtained by the second graph convolution, is the image feature of the last layer of the product-product graph, S ij is the product-product graph, is the mask, is the image feature of the l-th layer output by the product-product graph, is the second initial image feature vector.

[0018] In a possible implementation of the first aspect, the enhanced text feature vector and the enhanced image feature vector obtained by the first graph convolution and the second graph convolution can be respectively fused to obtain the multi-modal feature vector of the product. Then, according to the multi-modal feature vector, the self-supervised loss is determined through the following formula:

[0019]

[0020] Among them, L InfoNCE is the self-supervised loss, is the multi-modal feature vector corresponding to the first graph convolution of the i-th product, or is the multi-modal feature vector corresponding to the second graph convolution of the i-th product or the j-th product, i≠j, sim(·) is the cosine similarity calculation function, τ ssl is the adjustable temperature coefficient hyperparameter, is the set of positive sample products in each batch of training samples, and there is an interaction between the user and the positive sample products.

[0021] In a possible implementation of the first aspect, the dual-path preference enhancement network model can specifically be used for:

[0022] Based on the initial user ID feature vector and multiple initial product ID feature vectors, through the user-product interaction graph and the product-product graph, determine the final user ID feature vector and the final product ID feature vector. Based on the first initial image feature vector and the second initial image feature vector, through the user-product interaction graph and the product-product graph, determine the first final image feature vector and the second final image feature vector. Based on the first initial text feature vector and the second initial text feature vector, through the user-product interaction graph and the product-product graph, determine the first final text feature vector and the second final text feature vector. Fuse the final user ID feature vector, the first final image feature vector, and the first final text feature vector to obtain the second user final feature vector. Fuse the final product ID feature vector, the second final image feature vector, and the second final text feature vector to obtain the second product final feature vector.

[0023] In a possible implementation of the first aspect, the dual-channel preference enhancement network model can specifically be used for:

[0024] Through the following formula, fuse the final user ID feature vector, the first final image feature vector, and the first final text feature vector to obtain the second user final feature vector:

[0025]

[0026] Where, is the second user final feature vector, is the first final image feature vector, is the first final text feature vector, is the trainable weight of the first final image feature vector, is the trainable weight of the first final text feature vector, and || is the concatenation operator.

[0027] In a possible implementation of the first aspect, the dual-channel preference enhancement network model can specifically be used for:

[0028] Through the following formula, fuse the final product ID feature vector, the second final image feature vector, and the second final text feature vector to obtain the second product final feature vector:

[0029]

[0030] Where, is the second product final feature vector, is the second final image feature vector, is the second final text feature vector.

[0031] In a possible implementation of the first aspect, a user-product interaction graph can be determined based on the interaction data between the user and multiple products. The original image feature vectors of multiple products in the product information are input into a pre-trained model to obtain the original text feature vectors of multiple products. Based on the original text feature vectors and the original image feature vectors, a product-product graph is determined. The ID features representing the user and multiple products are initialized to obtain an initial user ID feature vector and initial product ID feature vectors of multiple products. The image preference feature and text preference feature representing the user are initialized to obtain a first initial image feature vector and a first initial text feature vector. The original image feature vectors and the original text feature vectors are dimensionally reduced to obtain a second initial image feature vector and a second initial text feature vector.

[0032] In a possible implementation of the first aspect, the Bayesian ranking loss satisfies the following formula:

[0033]

[0034] O = {(u, i, j)|y ui = 1, y uj = 0}

[0035] where L BPR is the Bayesian ranking loss, is the second similarity between the user and the i-th product, is the second similarity between the user and the j-th product, i ≠ j, and μ is the sigmoid non-linear activation function.

[0036] In a possible implementation of the first aspect, the total loss function of the dual-path preference enhancement network model can be determined based on the Bayesian personalized ranking loss and the self-supervised loss. Then, under the constraint of the total loss function, the dual-path preference enhancement network model is trained through the backpropagation algorithm to obtain a trained modal preference enhancement model.

[0037] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows: According to the method provided by the present invention, by using more factors, such as the user's modal preference for product information and the user's interaction preference for products, the final feature vectors of the user and the products are determined, and products are recommended to the user based on the similarity between the two feature vectors, which can improve the accuracy of identifying the user's preference, thereby enhancing the effectiveness of product recommendation for the user; by training the dual-path preference enhancement network model according to the self-supervised loss, the semantic noise in the product-product graph can be removed to obtain more accurate user preferences, thereby further improving the effectiveness of product recommendation for the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1Schematic flowchart of a product recommendation method based on dual-channel preference enhancement provided by an embodiment of the present invention;

[0039] Figure 2 Schematic flowchart of a product recommendation method based on dual-channel preference enhancement provided by an embodiment of the present invention;

[0040] Figure 3 Schematic structural diagram of a dual-channel preference enhancement network model provided by an embodiment of the present invention

[0041] Figure 4 Schematic flowchart of a method for determining a final feature vector provided by an embodiment of the present invention. Detailed implementation manners

[0042] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures and technologies are proposed to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0043] It should be understood that when used in the specification and appended claims of the present invention, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0044] It should also be understood that the term "and / or" as used in the specification and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0045] As used in the specification and appended claims of the present invention, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0046] In addition, in the description of the specification and appended claims of the present invention, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0047] References to "one embodiment" or "some embodiments" etc. described in the specification of the present invention mean that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present invention. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants mean "including but not limited to", unless otherwise specifically emphasized.

[0048] A multi-modal recommendation method based on dual-path preference enhancement provided by an embodiment of the present invention can be applied to an electronic device, and the specific type of the electronic device is not limited in any way in the embodiments of the present invention.

[0049] Figure 1 FIG. shows a schematic flowchart of a multi-modal recommendation method based on dual-path preference enhancement provided by an embodiment of the present invention. By way of example and not limitation, this method can be applied to the above-mentioned electronic device. Method 100 may include steps S101 - S103, and each step will be described below.

[0050] S101, obtain a first user final feature vector of the user and first product final feature vectors of multiple products.

[0051] Exemplarily, the first user final feature vector and the first product feature vector are obtained by a trained dual-path preference enhancement network model through the initial user ID feature vector of the user, the first initial text feature vector, and the first initial image feature vector, and the initial product ID feature vectors of the multiple products, the second initial text feature vector, and the second initial image feature. The obtained first user final feature vector of the user and the first product final feature vectors of the multiple products will be stored in a database.

[0052] Exemplarily, the initial user ID feature vector is a vector related to the user obtained according to the user's interaction preference for product information, the initial product ID feature vector is a vector related to the product obtained according to the user's interaction preference for product information, the first initial image feature vector and the first initial text feature vector are vectors related to the user obtained according to the user's modal preference for product information, and the second initial image feature vector and the second initial text feature vector are vectors related to the product obtained according to the user's modal preference for product information.

[0053] Exemplarily, each user corresponds to a first user final feature vector, and each product corresponds to a first product final feature vector. The dual-path preference enhancement network model updates the first user final feature vector of the user and the first product final feature vector of the product at regular intervals based on the newly added user data and product data.

[0054] Exemplarily, the dual-path preference enhancement network model is used to enhance the second initial text feature vector and the second initial image feature vector to obtain an enhanced text feature vector and an enhanced image feature vector. And determine the self-supervised loss of the modality preference enhancement model according to the enhanced text feature vector and the enhanced image feature vector, and perform training under the constraint of the self-supervised loss.

[0055] S102. Determine a first similarity between the user and the product according to the first user final feature vector and the first product final feature vector.

[0056] Exemplarily, the first similarity between the user and the product can be determined by the following formula:

[0057]

[0058] where is the similarity between the user and the product, is the first user final feature vector, is the first product final feature vector.

[0059] S103. Recommend products for the user according to the first similarity.

[0060] The first similarity between the user and multiple products can be determined, and products are recommended for the user according to the first similarity.

[0061] For example, the first similarity between the user and multiple products of the same category can be determined, and then the product with the highest first similarity is recommended to the user. It is also possible to determine the first similarity between the user and multiple different types of products, and then recommend all products with a similarity greater than the threshold to the user.

[0062] It should be understood that the present invention does not limit the specific method of recommending products for the user according to the first similarity.

[0063] Since the first similarity is obtained based on the modality preference of the user for product information and the interaction preference of the user for the product, and since the self-supervised loss of the model is determined according to the enhanced feature vector; according to the method provided by the present invention, recommending products for the user through the first similarity between the user and the product obtained by the dual-path preference enhancement network model can explore the interest preferences of the user in a finer granularity, reduce the semantic noise when determining the user preferences, and thus improve the success rate when recommending products.

[0064] Figure 2 Shows a schematic flowchart of a multi-modal recommendation method based on dual-channel preference enhancement provided by an embodiment of the present invention. As an example rather than a limitation, method 200 is a specific possible implementation of method 100. Method 200 may include steps S201-S210, and each step will be described below.

[0065] S201, preprocess the user data and product data to obtain the initial user ID feature vector of the user, the initial product ID feature vector of the product, the first initial text feature vector, the first initial image feature vector, the second initial text feature vector, the second initial image feature vector, the user-product interaction graph, and the product-product graph.

[0066] Exemplarily, the user data and product data may include: interaction data of the user and the product, product information, and user information.

[0067] Exemplarily, the product information may include information such as the original image feature vector of the product, the product specifications, and the product model.

[0068] Exemplarily, the user information may include the identity information of the user, etc.

[0069] Optionally, the user data and product data may include data of at least one user and multiple products.

[0070] In some embodiments, the user-product interaction graph may be determined according to the interaction data of the user and the product.

[0071] In a possible implementation, the user-product interaction graph may satisfy the following formula:

[0072] G = {(u, y ui , i)|u ∈ U, y ui ∈ {0, 1}, i ∈ I}

[0073] Wherein, U is the user set, I is the product set, and y ui is the connection relationship between user u and product i.

[0074] In an example, if there is a positive feedback relationship between the product and the user, such as click, purchase, etc., y ui = 1, otherwise, y ui = 0.

[0075] In some embodiments, the text information of the product in the product information may be input into a pre-trained model to obtain the original text feature vector of the product, and the publicly available original image feature vector of the product may be directly obtained from the product information. Then, the product-product graph may be determined according to the original image feature vector and the original text feature vector of the product.

[0076] Exemplarily, the original image feature vector can be a 4096-dimensional feature vector extracted from BAIR Reference CaffeNet.

[0077] Exemplarily, the text information of the product can include information such as title, table of contents, etc.

[0078] In one example, the original text feature vector satisfies the following formula:

[0079]

[0080] where, is the original text feature vector, S t is the pre-trained model, t i is the text information of the product, and α is the network parameter of the pre-trained model.

[0081] Exemplarily, the pre-trained model can be all-MiniLM-L6-v2, which is a model fine-tuned on the MiniLM-L12-H384-uncased model using more than 100 million sentences and can extract 384-dimensional original text features.

[0082] In one possible implementation, the cosine similarity matrix of the product image and text can be determined based on the original image feature vector and the original text feature vector. Then, k-nearest neighbor sparsification is performed on the two similarity matrices respectively to obtain an image sparse matrix and a text sparse matrix. Next, normalization is performed on the image sparse matrix and the text sparse matrix respectively to obtain the feature maps of the image and the text. The product-product graph can be obtained by fusing the image feature map and the text feature map with fixed weights.

[0083] In one example, the cosine similarity matrix of the image and the text can be determined by the following formula:

[0084]

[0085]

[0086] where, is the cosine similarity matrix of the image, is the original image feature vector of the i-th product, is the original image feature vector of the j-th product, is the original text feature vector of the i-th product, is the original text feature vector of the j-th product, i≠j, and ||·|| is the modulus value of the feature vector.

[0087] In one example, the image sparse matrix and the text sparse matrix satisfy the following formula:

[0088]

[0089]

[0090] where is the image sparse matrix between the i-th product and the j-th product, is the text sparse matrix between the i-th product and the j-th product. The input of top-k(·) is the i-th row elements of the similarity matrix, and it returns the highest k values.

[0091] In one example, the image feature map and the text feature map can be obtained through the following formula:

[0092]

[0093]

[0094] where is the image feature map, is the text feature map, D v is the diagonal matrix of the element and D t is the diagonal matrix with elements .

[0095] In one example, the product-product graph satisfies the following formula:

[0096]

[0097] where S is the product-product graph, λ v is the weight of the image feature map, and λ t is the weight of the text feature map.

[0098] Exemplarily, the sum of the weight of the image feature map and the weight of the text feature map is 1, that is, λ v + λ t = 1.

[0099] In one possible implementation, the ID features representing users and products can be randomly initialized to obtain the initial user ID feature vector of the user and the initial product ID feature vector of the product.

[0100] In one possible implementation, similarly, the image modality preference features and text modality preference features representing users can be randomly initialized to obtain the first initial image feature vector and the first initial text feature vector.

[0101] In a possible implementation, the original image feature vector and the original text feature vector of the product can be dimensionally reduced to obtain a second initial image feature vector and a second initial text feature vector.

[0102] In one example, the original text feature vector and the original image feature vector of the product can be dimensionally reduced through a multi-layer perceptron to obtain a second initial image feature vector and a second initial text feature vector.

[0103] Exemplarily, the second initial image feature vector and the second initial text feature vector can be obtained through the following formula:

[0104]

[0105]

[0106] where is the second initial image feature vector, and are learnable weight matrices corresponding to the second initial image feature vector, and are learnable biases corresponding to the second initial image feature vector, and LeakyReLu(·) is a non-linear activation function corresponding to the second initial image feature vector or the second initial text feature vector; is the second initial text feature vector, and are learnable weight matrices corresponding to the second initial text feature vector, and are learnable biases corresponding to the second initial text feature vector.

[0107] S202. Input the initial user ID feature vector, the initial product ID feature vector, the first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vector into the dual-channel preference enhancement network model, and through the user-product interaction graph and the product-product graph, obtain the second user final feature vector of the user, the second product final feature vector of the product, the enhanced text feature vector of the product and the enhanced image feature vector.

[0108] In some embodiments, referring to Figure 3 , the dual-channel preference enhancement network model 300 may include an interaction preference enhancement module 301, a modality preference enhancement module 302 and a dual-channel joint prediction module 303.

[0109] The initial user ID feature vector and the initial product ID feature vector can be input into the interaction preference enhancement module. The interaction preference enhancement module can obtain the final user ID feature vector and the final product ID feature vector based on the initial user ID feature vector and the initial product ID feature vector through the user-product interaction graph and the product-product graph.

[0110] The first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vector can be input into the modality preference enhancement module. The modality preference enhancement module can obtain the first final image feature vector and the second final image feature vector, the first final text feature vector and the second final text feature vector based on the first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vector through the user-product interaction graph and the product-product graph.

[0111] Finally, the interaction preference enhancement module can send the final user ID feature vector and the final product ID feature vector to the dual-path joint prediction module. The modality preference enhancement module can send the first final image feature vector and the second final image feature vector, the first final text feature vector and the second final text feature vector to the dual-path joint prediction module. The dual-path joint prediction module can fuse the final user ID feature vector, the first final text feature vector and the first final image feature vector to obtain the second final user feature vector; fuse the final product ID feature vector, the second final image feature vector and the second final text feature vector to obtain the second final product feature vector.

[0112] By way of example and not limitation, the dual-path preference enhancement network model can determine the second final user feature vector and the second final product feature vector through steps S401 - S410 in method 400. The following explains steps S401 - S410.

[0113] S401, the interaction preference enhancement module takes the initial user ID feature vector and the initial product ID feature vector as the nodes of the user and the product on the user-product interaction graph, performs graph convolution on the user-product interaction graph, and obtains the final user ID feature vector and the first product ID feature vector.

[0114] In a possible implementation, the interaction preference enhancement module can take the initial user ID feature vector and the initial product ID feature vector as the nodes of the user and the product on the user-product interaction graph, perform graph convolution on the user-product interaction graph, update the product ID feature vector and the user ID feature vector, and a new graph can be obtained for each execution of the graph convolution operation. Combining the product ID feature vector and the user ID feature vector obtained for each layer respectively, the final user ID feature vector and the first product ID feature vector can be obtained.

[0115] Exemplarily, the ID feature vector of the l-th layer obtained by performing l graph convolution operations satisfies the following formula:

[0116]

[0117]

[0118] where, is the set of users who have interacted with product i, is the set of products that have interacted with user u, is the user ID feature vector of the l-th layer, the product ID feature vector of the l-th layer.

[0119] In one example, the product ID feature vector and the user ID feature vector obtained for each layer can be combined respectively through the following formula to obtain the final user ID feature vector and the first product ID feature vector:

[0120]

[0121]

[0122]

[0123] where L ui is the number of graph convolution operations, is the final user ID feature vector, is the first product ID feature vector.

[0124] S402. The interaction preference enhancement module uses the initial product ID feature vector as the feature of the product nodes on the product-product graph, performs graph convolution on the product-product graph, and obtains the second product ID feature vector.

[0125] In one possible implementation, the initial product ID feature vector can be used as the feature of the product nodes on the product-product graph, graph convolution can be performed on the product-product graph to update the product ID feature vector, and the product ID feature vector obtained from the last graph convolution is determined as the second product ID feature vector.

[0126] In one example, the second product ID feature vector can be determined through the following formula:

[0127]

[0128]

[0129] where, is the second product ID feature vector, is the set of products adjacent to product i in the product-product graph, is the product ID feature vector obtained from the last graph convolution operation, is the product ID feature vector S of the l-th layer ij is the product-product graph.

[0130] S403. The interaction preference enhancement module fuses the first product ID feature vector and the second product ID feature vector to obtain the final product ID feature vector.

[0131] Exemplarily, the final product ID feature vector can satisfy the following formula:

[0132]

[0133] where, is the final product ID feature vector, is the first product ID feature vector, is the second product ID feature vector.

[0134] S404. The interaction preference enhancement module sends the final product ID feature vector and the final user ID feature vector to the dual-path joint prediction module.

[0135] Correspondingly, the dual-path joint prediction module receives the final product ID feature vector and the final user ID feature vector.

[0136] S405. The modality preference enhancement module uses the first initial modality feature vector and the second initial modality feature vector as the nodes of users and products on the user-product interaction graph, performs graph convolution on the user-product interaction graph, and obtains the final user modality feature vector and the first product modality feature vector.

[0137] Exemplarily, the first initial modality feature vector includes the first initial image feature vector and the first initial text feature vector.

[0138] Exemplarily, the second initial modality feature vector can include the second initial image feature vector and the second initial text feature vector.

[0139] Exemplarily, the final user modality feature vector can include the first final image feature vector and the first final text feature vector.

[0140] Exemplarily, the first product modality feature vector can include the first product image feature vector and the first product text feature vector.

[0141] In a possible implementation, similarly, the first initial image feature vector and the second initial image feature vector can be used as the features of the nodes of the user and the product on the user-product interaction graph respectively, and graph convolution can be performed to update the image features. Each time an update is made, a new graph is obtained. The image features of each layer of the graph are accumulated to obtain the first final image feature vector and the first product image feature vector.

[0142] Exemplarily, the image features of the l-th layer satisfy the following formula:

[0143]

[0144]

[0145] Among them, is the user image feature of the l-th layer, is the product image feature of the l-th layer.

[0146] Exemplarily, the first final image feature vector and the first product image feature vector satisfy the following formula:

[0147]

[0148]

[0149]

[0150] Among them, is the first final image feature vector, is the first product image feature vector.

[0151] In a possible implementation, similarly, the first initial text feature vector and the second initial text feature vector can be used as the features of the nodes of the user and the product on the user-product interaction graph respectively, and graph convolution can be performed to update the text features. Each time an update is made, a new graph is obtained. The text features of each layer of the graph are accumulated to obtain the first final text feature vector and the first product text feature vector.

[0152] Exemplarily, the text features of the l-th layer satisfy the following formula:

[0153]

[0154]

[0155] Among them, is the user text feature of the l-th layer, is the product text feature of the l-th layer.

[0156] Exemplarily, the first final text feature vector and the first product text feature vector satisfy the following formula:

[0157]

[0158]

[0159]

[0160] wherein, is the first final text feature vector, is the first product text feature vector.

[0161] S406. The modality preference enhancement module uses the second initial modality feature vector as the feature of the product node on the product-product graph, performs graph convolution on the product-product graph, and obtains the second product modality feature vector.

[0162] Exemplarily, the second product modality feature vector may include a second product image feature vector and a second product text feature vector.

[0163] In a possible implementation, similarly, the second initial image feature can be used as the feature of the product node on the product-product graph, and graph convolution operations can be performed to update the image feature vector. The result of the last image convolution is used as the second product image feature vector.

[0164] Exemplarily, the second product image feature vector satisfies the following formula:

[0165]

[0166]

[0167]

[0168] wherein, is the product image feature vector of the l-th layer, is the second product image feature vector, is the product image feature vector obtained by the last graph convolution operation.

[0169] In a possible implementation, similarly, the second initial text feature can be used as the feature of the product node on the product-product graph, and graph convolution operations can be performed to update the text feature vector. The result of the last text convolution is used as the second product text feature vector.

[0170] Exemplarily, the second product text feature vector satisfies the following formula:

[0171]

[0172]

[0173] Among them, is the product text feature vector of the l-th layer, is the second product text feature vector, the product text feature vector obtained by the last graph convolution operation.

[0174] S407. The modality preference enhancement module fuses the first product modality feature vector and the second product modality feature vector to obtain the final product modality feature vector.

[0175] In a possible implementation, the first product image feature vector and the second product image feature vector can be fused to obtain the second final image feature vector. The first product text feature vector and the second product text feature vector are fused to obtain the second final text feature vector.

[0176] In an example, the second final image feature vector satisfies the following formula:

[0177]

[0178] Among them, is the second final image feature vector.

[0179] In an example, the second final text feature vector satisfies the following formula:

[0180]

[0181] Among them, is the second final text feature vector.

[0182] S408. The modality preference enhancement module sends the final product modality feature vector and the final user modality feature vector to the dual-path joint prediction module.

[0183] Correspondingly, the dual-path joint prediction module receives the final product modality feature vector and the final user modality feature vector.

[0184] S409. The dual-path joint prediction module fuses the final user ID feature vector and the final user modality feature vector to obtain the second user final feature vector.

[0185] In a possible implementation, the dual-path joint prediction module can fuse the final user ID feature vector, the first final image feature vector, and the first final text feature vector through the following formula to obtain the second user final feature vector:

[0186]

[0187] Among them, is the second user's final feature vector, is the first final image feature vector, is the first final text feature vector, is the trainable weight of the first final image feature vector, is the trainable weight of the first final text feature vector, || is the concatenation operator.

[0188] S410, the dual-path joint prediction module fuses the final product ID feature vector and the final product modality feature vector to obtain the second product's final feature vector.

[0189] In one possible implementation, the dual-path joint prediction module can fuse the final product ID feature vector, the second final image feature vector, and the second final text feature vector through the following formula to obtain the second product's final feature vector:

[0190]

[0191] where, is the second product's final feature vector, is the second final image feature vector, is the second final text feature vector.

[0192] In some embodiments, the modality preference enhancement module can also determine the enhanced text feature vector and the enhanced image feature vector of the product according to the second initial text feature vector and the second initial image feature vector.

[0193] By way of example and not limitation, the modality preference enhancement module can determine the enhanced text feature vector and the enhanced image feature vector of the product through steps S410 and S412 of method 400. The following explains each step.

[0194] S411, the modality preference enhancement module sets a mask for the second initial image feature to mask some neighbors of the product-product graph, and performs graph convolution on the masked product-product graph to obtain the enhanced image feature vector.

[0195] In one possible implementation, digital enhancement can be performed on the second initial image feature vector, and then 2 masks can be set for the digitally enhanced second initial image feature to mask different partial neighbors on the product-product graph respectively, corresponding to obtaining 2 masked product-product graphs. The second initial image feature performs graph convolution on these 2 product-product graphs respectively, and two enhanced image feature vectors can be obtained.

[0196] Exemplarily, the mask follows a Bernoulli binary distribution with a probability of p.

[0197] Exemplarily, the enhanced image feature vector satisfies the following formula:

[0198]

[0199]

[0200]

[0201]

[0202]

[0203]

[0204] where, is the image feature of the l-th layer of the masked product-product graph, is the enhanced image feature vector obtained by the first graph convolution, is the enhanced image feature vector obtained by the second graph convolution, is the image feature of the last layer of the product-product graph, S ij is the product-product graph, is the mask, is the image feature of the l-th layer output by the product-product graph, is the second initial image feature vector after digital enhancement.

[0205] S412. The modality preference enhancement module sets a mask for the second text image feature, masks some neighbors of the product-product graph, and performs graph convolution on the masked product-product graph to obtain an enhanced text feature vector.

[0206] In a possible implementation, the second initial text feature vector can be digitally enhanced, and then 2 masks are set for the digitally enhanced second initial text feature, respectively masking different partial neighbors on the product-product graph, corresponding to obtaining 2 masked product-product graphs. The second initial text feature performs graph convolution on these 2 product-product graphs respectively, and two enhanced text feature vectors can be obtained.

[0207] Exemplarily, the mask follows a Bernoulli binary distribution with a probability of p.

[0208] Exemplarily, the enhanced text feature vector satisfies the following formula:

[0209]

[0210]

[0211]

[0212]

[0213]

[0214]

[0215] Among them, is the text feature of the l-th layer of the masked product - product graph, is the enhanced text feature vector obtained by the first graph convolution, is the enhanced text feature vector obtained by the second graph convolution, is the text feature of the last layer of the product - product graph, S ij is the product - product graph, is the mask, is the text feature of the l-th layer output by the product - product graph, is the second initial text feature vector after digital enhancement.

[0216] S203. Determine the second similarity between the user and the product according to the second user final feature vector and the second product final feature vector.

[0217] Exemplarily, the calculation method of the second similarity is similar to that of the first similarity, and it is the inner product of the second user final feature vector and the second product final feature vector.

[0218] Exemplarily, the second similarity satisfies the following formula:

[0219]

[0220] Among them, is the second user final feature vector, is the second product final feature vector, is the second similarity.

[0221] S204. Determine the Bayesian personalized ranking loss of the dual - path preference enhancement network model according to the second similarity.

[0222] Exemplarily, the Bayesian personalized ranking loss satisfies the following formula:

[0223]

[0224] O = {(u, i, j)|y ui = 1, y uj = 0}

[0225] Among them, L BPR is the Bayesian ranking loss, is the second similarity between the user and the i-th product, is the second similarity between the user and the j-th product, i ≠ j, and μ is the sigmoid non-linear activation function.

[0226] S205. Determine the self-supervised loss of the dual-path preference enhancement network model according to the enhanced text feature vector and the enhanced image feature vector.

[0227] In one example, the enhanced text feature vector and the enhanced image feature vector obtained by respectively fusing the first graph convolution and the second graph convolution for the first time can be fused to obtain the multi-modal feature vector of the product. Then, according to the multi-modal feature vector, the self-supervised loss is determined. Since the two enhanced multi-modal features are learned on the product-product graph with different neighbors masked, using the self-supervised learning loss function can maximize the consistency of the two enhanced multi-modal features of the same product, while minimizing the two enhanced multi-modal features of different products, which can reduce the influence of unimportant neighbors in the product-product graph during the graph convolution process and achieve the effect of removing noise in the product-product graph.

[0228] Exemplarily, the enhanced text feature vector and the enhanced image feature vector can be fused through the following formula:

[0229]

[0230]

[0231] where is the multi-modal feature vector of the first graph convolution, is the enhanced image feature vector of the first graph convolution, is the enhanced text feature vector of the first graph convolution; is the multi-modal feature vector of the second graph convolution, is the enhanced image feature vector of the second graph convolution, is the enhanced text feature vector of the second graph convolution.

[0232] Exemplarily, the self-supervised loss satisfies the following formula:

[0233]

[0234] where L InfoNCE is the self-supervised loss, is the multi-modal feature vector corresponding to the first graph convolution of the i-th product, or is the multi-modal feature vector corresponding to the second graph convolution of the i-th product or the j-th product, i ≠ j, sim(·) is the cosine similarity calculation function, and τ ssl is the adjustable temperature coefficient hyperparameter, is the set of positive sample products in each batch of training samples, and there is an interaction between the user and the positive sample products.

[0235] S206. Train the dual-path preference enhancement network model according to the Bayesian personalized ranking loss and the self-supervised loss to obtain the trained dual-path preference enhancement network model.

[0236] In one example, the loss function of the dual-path preference enhancement network model can be determined according to the Bayesian personalized ranking loss and the self-supervised loss. Under the constraint of the loss function, the dual-path preference enhancement network model is trained through the backpropagation algorithm to obtain the trained dual-path preference enhancement network model.

[0237] Exemplarily, the loss function of the dual-path preference enhancement network model satisfies the following formula:

[0238]

[0239] where L BPR is the Bayesian personalized ranking loss, L InfoNCE is the self-supervised loss, is the regularization term loss of the model, Θ is the parameter of the model, and α and β are hyperparameters.

[0240] S207. Input the initial user ID feature vector, multiple initial product ID feature vectors, the first initial image feature vector, the second initial image feature vector, the first initial text feature vector, and the second initial text feature vector into the trained dual-path preference enhancement network model to obtain the first user final feature vector of the user and the first product final feature vectors of multiple products.

[0241] Exemplarily, the methods for determining the first user final feature vector and the second user final feature vector are the same, except that the first user final feature vector is determined by the trained dual-path preference enhancement model, and the second user final feature vector is determined by the dual-path preference enhancement model that has not been completed training. Similarly, the methods for determining the first product final feature vector and the second product final feature vector are the same, except that the first product final feature vector is determined by the trained dual-path preference enhancement model, and the second product final feature vector is determined by the dual-path preference enhancement model that has not been completed training.

[0242] For the specific content of the method for determining the first user final feature vector and the second user final feature vector, please refer to the content in steps S401 - S410 in method 400, which will not be elaborated here.

[0243] S208. Obtain the first user final feature vector of the user and the first product final feature vectors of multiple products.

[0244] S209. Determine the first similarity between the user and multiple products according to the first user final feature vector and the first product final feature vector.

[0245] S210. Recommend products for the user according to the multiple first similarities.

[0246] The content of steps S208 - S210 in method 200 is the same as the content of steps S101 - S103 in method 100. For specific reference, please refer to the relevant descriptions of steps S101 - S103 in method 100, which will not be elaborated here.

[0247] In order to better illustrate the beneficial effects of the method provided by the present invention, the following simulation experiment is designed:

[0248] Exemplarily, the operating system used in the simulation experiment is ubuntu 20.04 and python3.7; the experimental platform is pytorch - 1.7.1; the graphics card of the experimental device is NVIDIA Tesla V100 - SXM2, and the video memory is 32GB.

[0249] Exemplarily, the dataset used in the experiment is a classic multi - modal task dataset. By comparing the benchmark methods based on the user - product interaction graph and the product - product graph, and the task accuracy of the product recommendation method based on dual - path preference enhancement provided by the present invention.

[0250] Exemplarily, the experimental dataset may include data of multiple users and multiple products. During training, all user information, product information, and partial interaction data of users and products are used as the training set to train the two models. During testing, according to the user final feature vectors and product final vectors output by the two models respectively, calculate the similarity between the user and the product, and predict whether the user and the product will generate interaction actions through the similarity. Compare the prediction results with the interaction data of users and products in the test set to determine the accuracy of the two models.

[0251] The following table shows the recall rate and accuracy rate tested by the existing recommendation method, that is, the benchmark method, and the method provided by the present invention.

[0252] Table 1 Comparison of recall rate and accuracy rate between the benchmark method and the method proposed by the present invention It can be seen from Table 1 that when using different datasets, the method provided by the present invention exceeds the benchmark method in all indicators. When using Dataset 1, the R@20 and P@20 indicators of the method provided by the present invention are improved by 3.10% and 3.51% compared with the benchmark method; when using Dataset 2, they are improved by 5.58% and 4.84%; when using Dataset 3, they are improved by 6.34% and 6.00%.

[0253] The following table shows the recall rate and accuracy comparison of the recommendation methods of the method provided according to the present invention, training the model without using self-supervised loss based on the method provided in the present invention, and the models based on the model without interaction preference enhancement and the preference enhancement model, which only includes the user-product interaction graph.

[0254] Table 2 Visualization experiment on the importance comparison of self-supervised method and dual-path preference enhancement method in the model As can be seen from Table 2, the performance of the method without self-supervised learning has decreased. The method provided by the present invention has a significant improvement in the R@20 and P@20 metrics compared with the other two methods.

[0255] According to the method provided by the present invention, by using more factors, such as the user's modal preference for product information and the user's interaction preference for products, to determine the final feature vectors of the user and the product, and recommending products to the user based on the similarity between the two feature vectors, the accuracy of identifying the user's preference can be improved, thereby enhancing the effectiveness of product recommendation for the user; by training the dual-path preference enhancement network model according to the self-supervised loss, the semantic noise in the product-product graph can be removed, and more accurate user preferences can be obtained, thereby further improving the effectiveness of product recommendation for the user.

[0256] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

Claims

1. A product recommendation method based on dual - path preference enhancement, characterized in that, Including: Obtain the first user final feature vector of the user and the first product final feature vectors of multiple products; Determine a first similarity between the user and the multiple products according to the first user final feature vector and the multiple first product final feature vectors, wherein, the first user final feature vector and the first product final vector are based on a trained dual-path preference enhancement network model of self-supervised learning according to the initial user ID feature vector, the first initial text feature vector and the first initial image feature vector of the user, the initial product ID feature vectors, the second initial text feature vectors and the second initial image features of the multiple products. The initial user ID feature vector is a vector related to the user obtained according to the interaction preference of the user for product information, and the initial product ID feature vector is a vector related to the product obtained according to the interaction preference of the user for product information. The first initial image feature vector and the first initial text feature vectors are vectors related to the user obtained according to the modality preference of the user for product information, and the second initial image feature vector and the second initial text feature vectors are vectors related to the product obtained according to the modality preference of the user for product information. The dual-path preference enhancement network model is used to enhance the second initial text feature vector and the second initial image feature vector to obtain an enhanced text feature vector and an enhanced image feature vector, and determine a self-supervised loss according to the enhanced text feature vector and the enhanced image feature vector, and perform training under the constraint of the self-supervised loss; Recommend products for the user according to the multiple first similarities.

2. The method according to claim 1, characterized in that, Before obtaining the first user final feature vector of the user and the first product final feature vectors of multiple products, the method further includes: Preprocess user data and product data to obtain the initial user ID feature vector of the user, the initial product ID feature vectors of the multiple products, the first initial text feature vector, the first initial image feature vector, the second initial text feature vectors, the second initial image feature vectors, the user-product interaction graph and the product-product graph. The user data and product data include the interaction data of the user and the multiple products, the user information of the user and the product information of the multiple products. The user-product interaction graph is determined according to the interaction data of the user and the multiple products, and the product-product graph is determined according to the product information; Input the initial user ID feature vector, the initial product ID feature vector, the first initial image feature vector and the second initial image feature vector, the first initial text feature vector and the second initial text feature vectors into the dual-path preference enhancement network model, and obtain the second user final feature vector of the user, the second product final feature vectors of the multiple products, the enhanced text feature vectors and the enhanced image feature vectors of the multiple products through the user-product interaction graph and the product-product graph; Determine a second similarity between the user and the multiple products according to the second user final feature vector and the multiple second product final feature vectors; Determine the Bayesian personalized ranking loss of the dual-channel preference enhancement network model according to the second similarity; Determine the self-supervised loss of the dual-channel preference enhancement network model according to the enhanced text feature vector and the enhanced image feature vector; Train the dual-channel preference enhancement network model according to the Bayesian personalized ranking loss and the self-supervised loss to obtain the trained dual-channel preference enhancement network model; Input the initial user ID feature vector, the multiple initial product ID feature vectors, the first initial image feature vector, the second initial image feature vector, the first initial text feature vector, and the second initial text feature vector into the trained dual-channel preference enhancement network model to obtain the first user final feature vector of the user and the first product final feature vectors of the multiple products.

3. The method according to claim 2, characterized in that, The dual-channel preference enhancement network model is specifically configured to: Set two masks for the second initial image feature vector to respectively mask partial neighbors in the product-product graph, and correspondingly obtain two masked product-product graphs, where the masks follow a Bernoulli binary distribution with a probability of p; Perform graph convolution on the second initial image feature vector on the two masked product-product graphs respectively to obtain two enhanced image feature vectors, and the enhanced image feature vectors satisfy the following formula: Among them, is the image feature of the l-th layer of the masked product-product diagram, is the enhanced image feature vector obtained by the first graph convolution, is the enhanced image feature vector obtained by the second graph convolution, is the image feature of the last layer of the product-product diagram, S ij is the product-product diagram, is the mask, is the image feature of the (l-1)-th layer output by the product-product diagram, is the second initial image feature vector, is the set of products adjacent to product i in the product-product diagram.

4. The method according to any one of claims 1 - 3, characterized in that, Determine the self-supervised loss of the dual-channel preference enhancement network model according to the enhanced text feature vector and the enhanced image feature vector, including: Fuse the enhanced text feature vector and the enhanced image feature vector obtained by the first graph convolution and the second graph convolution respectively to obtain the multi-modal feature vector of the product; Determine the self-supervised loss according to the multi-modal feature vector through the following formula: where, L InfoNCE is the self-supervised loss, is the multi-modal feature vector corresponding to the first graph convolution of the i-th product, or is the multi-modal feature vector corresponding to the second graph convolution of the i-th product or the j-th product, i≠j, sim(·) is the cosine similarity calculation function, τ ssl is an adjustable temperature coefficient hyperparameter, is the set of positive sample products in each batch of training samples, and there is an interaction between the user and the positive sample products.

5. The method according to claim 2, characterized in that, The dual-channel preference enhancement network model is specifically configured to: Determine the final user ID feature vector and the final product ID feature vector according to the initial user ID feature vector and the multiple initial product ID feature vectors through the user-product interaction graph and the product-product graph; Determine the first final image feature vector and the second final image feature vector according to the first initial image feature vector and the second initial image feature vector through the user-product interaction graph and the product-product graph; Determine the first final text feature vector and the second final text feature vector according to the first initial text feature vector and the second initial text feature through the user-product interaction graph and the product-product graph; Fuse the final user ID feature vector, the first final image feature vector, and the first final text feature vector to obtain the second user final feature vector; Fuse the final product ID feature vector, the second final image feature vector, and the second final text feature vector to obtain the second product final feature vector.

6. The method according to claim 5, wherein The dual-channel preference enhancement network model is specifically configured to: Fuse the final user ID feature vector, the first final image feature vector, and the first final text feature vector through the following formula to obtain the second user final feature vector: wherein, is the final feature vector of the second user, is the first final image feature vector, is the first final text feature vector, is the trainable weight of the first final image feature vector, is the trainable weight of the first final text feature vector, || is the concatenation operator.

7. The method according to claim 5, wherein The dual-path preference enhancement network model is specifically used for: Fuse the final product ID feature vector, the second final image feature vector, and the second final text feature vector through the following formula to obtain the second product final feature vector: Among them, is the final feature vector of the second product, is the final image feature vector of the second, is the final text feature vector of the second.

8. The method according to claim 2, wherein Preprocess the user data and product data to obtain the initial user ID feature vector of the user, the initial product ID feature vectors of the multiple products, the first initial text feature vector, the first initial image feature vector, the second initial text feature vector, the second initial image feature vector, the user-product interaction graph, and the product-product graph, including: Determine the user-product interaction graph according to the interaction data of the user and the multiple products; Input the original image feature vectors of the multiple products in the product information into a pre-trained model to obtain the original text feature vectors of the multiple products; Determine the product-product graph according to the original text feature vector and the original image feature vector; Initialize the ID features representing the user and the multiple products to obtain the initial user ID feature vector and the initial product ID feature vectors of the multiple products; Initialize the image preference feature and text preference feature representing the user to obtain the first initial image feature vector and the first initial text feature vector; Reduce the dimensions of the original image feature vector and the original text feature vector to obtain the second initial image feature vector and the second initial text feature vector.

9. The method according to claim 2, wherein The Bayesian personalized ranking loss satisfies the following formula: O = {(u, i, j)|y ui = 1, y uj = 0} Among them, L BPR is the Bayesian personalized ranking loss, is the second similarity between the user and the i-th product, is the second similarity between the user and the j-th product, i≠j, and μ is the sigmoid non-linear activation function.

10. The method according to claim 2, wherein Train the dual-path preference enhancement network model according to the Bayesian personalized ranking loss and the self-supervised loss to obtain the trained dual-path preference enhancement network model, including: Determine the total loss function of the dual-path preference enhancement network model according to the Bayesian personalized ranking loss and the self-supervised loss; Under the constraint of the total loss function, train the dual-path preference enhancement network model through the backpropagation algorithm to obtain the trained dual-path preference enhancement network model.

Images