Method, device, electronic device and medium for predicting user preferences

Abstract

The present disclosure provides a method, apparatus, electronic device, and medium for predicting user preferences, which can be applied to the field of artificial intelligence technology. The method includes: obtaining a user-item interaction matrix, wherein the user-item interaction matrix includes at least one user's evaluation of at least one item; obtaining feature data of multiple categories of images related to a target item among the at least one item; using an attention model to generate a visual feature vector for the target item based on the feature data of the multiple categories of images; and using a deep matrix factorization model to predict the degree of preference of the target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically, to a method, device, electronic device, and medium for predicting user preferences. Background Art

[0002] With the gradual development of information technology, analyzing user interests and preferences has been widely applied in various fields. For example, it can recommend items or information to users based on their preferences, thereby achieving more accurate recommendations. In the process of implementing the concepts of the present disclosure, the inventors found that traditional methods often analyze user preferences based solely on historical user evaluation data, resulting in inaccurate analysis results. Summary of the Invention

[0003] In view of this, the present disclosure provides a method, apparatus, electronic device, and medium for predicting user preferences. By using an attention model to generate a visual feature vector for a target item based on feature data of multiple categories of images, and predicting the target user's preference based on the visual feature vector and the user-item interaction matrix,

[0004] One aspect of the present disclosure provides a method for predicting user preferences, comprising:

[0005] Obtaining a user-item interaction matrix, wherein the user-item interaction matrix includes an evaluation value of at least one user for at least one item;

[0006] Acquiring feature data of multiple types of images related to a target item among the at least one item;

[0007] Generate a visual feature vector for the target object based on the feature data of the multiple categories of images using an attention model;

[0008] A deep matrix decomposition model is used to predict a preference degree of a target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix.

[0009] According to an embodiment of the present disclosure, the step of acquiring feature data of multiple types of images related to the target object includes: for each type of image,

[0010] extracting a first feature vector from each image in the class of images using the trained convolutional neural network;

[0011] Clustering the extracted multiple first eigenvectors using a clustering algorithm to obtain N clusters, where N is an integer greater than 1;

[0012] By selecting a first eigenvector from each cluster, N first eigenvectors are obtained as feature data of the image of this type.

[0013] According to an embodiment of the present disclosure, the attention model includes a first attention network layer and a second attention network layer, and using the attention model to generate a visual feature vector for the target object based on the feature data of the multiple categories of images includes:

[0014] Use the first attention network to generate a second feature vector for each type of image based on the N first feature vectors of the image;

[0015] A second attention network is used to generate a visual feature vector for the target item based on the second feature vectors of the multi-class images.

[0016] According to an embodiment of the present disclosure, the method of using the first attention network to generate a second feature vector for each type of image according to the N first feature vectors of the image includes: generating a second feature vector for each type of image according to the N first feature vectors of the image by the following equation:

[0017] u ict =ReLU(W α f ict +b α )

[0018]

[0019] f ic =∑ t a ict f ict

[0020] Where i represents the item, c represents the image category, t represents the tth first eigenvector among the N first eigenvectors of each type of image, and f ict represents the tth first eigenvector of the cth category image of item i, ReLU represents the activation function, u ict represents f ict The implicit expression of a ict represents f ict The importance of f ic The second eigenvector of the c-th image of item i, W α represents the first permutation matrix between the first attention network layer and the second attention network layer, b α Represents the first bias term between the first attention network layer and the second attention network layer, u α Represents the first context vector of the first attention network layer and the second attention network layer.

[0021] According to an embodiment of the present disclosure, the generating of a visual feature vector for the target item based on the second feature vector of the multi-class image using the second attention network includes: generating a visual feature vector for the target item based on the second feature vector of the multi-class image using the following equation:

[0022] u ic =ReLU(W β f ic +b β )

[0023]

[0024] f i =∑ c a ic f ic

[0025] where u ic represents f ic The implicit expression of a ic represents f ic The importance of f i Represents the visual feature vector of item i, where item i is the target item, W β represents the second permutation matrix between the first attention network layer and the second attention network layer, b β b α Represents the second bias term between the first attention network layer and the second attention network layer, u β Represents the second context vector of the first and second attention network layers.

[0026] According to an embodiment of the present disclosure, the method of using a deep matrix decomposition model to predict the target user's preference for the target item based on the visual feature vector and the user-item interaction matrix includes:

[0027] Generating a feature vector of a target user based on the evaluation values ​​of the target user on all items in the user-item interaction matrix;

[0028] generating an evaluation feature vector of the target item based on the evaluation values ​​of the target item by all users in the user-item interaction matrix;

[0029] generating a feature vector of the target object according to the evaluation feature vector of the target object and the visual feature vector of the target object;

[0030] The preference degree of the target user for the target item is predicted according to the cosine similarity between the feature vector of the target user and the feature vector of the target item.

[0031] According to an embodiment of the present disclosure, the method further includes:

[0032] Obtaining the target user's actual preference for the target item;

[0033] Adjust parameters of at least one of the attention model and the deep matrix decomposition model according to the difference between the predicted preference level and the actual preference level.

[0034] According to an embodiment of the present disclosure, the method further includes performing at least one of the following operations before clustering the extracted multiple first feature vectors using a clustering algorithm:

[0035] removing invalid first eigenvectors and erroneous first eigenvectors from the extracted plurality of first eigenvectors;

[0036] adjusting parameters of the trained convolutional neural network according to the mean and variance of the plurality of first feature vectors; and

[0037] The eigenvalues ​​in the plurality of first eigenvectors are standardized and normalized.

[0038] According to an embodiment of the present disclosure, the clustering algorithm includes a K-Means clustering algorithm.

[0039] According to an embodiment of the present disclosure, the evaluation value of each user for each item in the user-item interaction matrix is ​​calculated based on at least one of the following:

[0040] whether the user has browsed display information of the item, wherein the display information includes at least one of a text description of the item and an image of the item;

[0041] The number of times the user has viewed the item's display information; and

[0042] The user's evaluation information for the item.

[0043] According to an embodiment of the present disclosure, the method further includes: recommending items based on the predicted target user's preference for the target item.

[0044] Another aspect of the present disclosure provides an apparatus for predicting user preferences, comprising:

[0045] A first acquisition module is configured to acquire a user-item interaction matrix, wherein the user-item interaction matrix includes an evaluation value of at least one user on at least one item;

[0046] a second acquisition module, configured to acquire feature data of multiple types of images related to a target item among the at least one item;

[0047] a feature extraction module, configured to generate a visual feature vector for the target object based on the feature data of the multiple categories of images using an attention model;

[0048] A prediction module is configured to use a deep matrix decomposition model to predict a preference of a target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix.

[0049] Another aspect of the present disclosure provides an electronic device, comprising:

[0050] one or more processors;

[0051] a storage device for storing one or more programs,

[0052] When the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 11.

[0053] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0054] Another aspect of the present disclosure provides a computer program, which includes computer-executable instructions. When the instructions are executed, the computer program is used to implement the method described above.

[0055] According to the embodiments of the present disclosure, the problem of inaccurate prediction results in traditional user preference prediction methods can be at least partially solved / mitigated / suppressed / even avoided, and thus the technical effect of improving the prediction accuracy of user preferences can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0057] Figure 1 The following schematically illustrates an application scenario of the method for predicting user preferences according to an embodiment of the present disclosure;

[0058] Figure 2 A flowchart of a method for predicting user preferences according to an embodiment of the present disclosure is schematically shown;

[0059] Figure 3 The flowchart of the method for obtaining feature data of each type of image according to an embodiment of the present disclosure is schematically shown.

[0060] Figure 4 A schematic diagram schematically illustrates a method for obtaining feature data of each type of image according to an embodiment of the present disclosure;

[0061] Figure 5 A flowchart of a method for predicting user preferences according to another embodiment of the present disclosure is schematically shown;

[0062] Figure 6 A schematic diagram schematically illustrates a method for predicting user preferences according to another embodiment of the present disclosure;

[0063] Figure 7 A block diagram schematically illustrates an apparatus for predicting user preferences according to an embodiment of the present disclosure;

[0064] Figure 8 The block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0065] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0066] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0067] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0068] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0069] Embodiments of the present disclosure provide a method and apparatus for predicting user preferences. This method uses an attention model to generate a visual feature vector for a target item based on feature data from multiple categories of images associated with the target item, and then uses a deep matrix factorization model to predict the target user's preference for the target item based on the visual feature vector and a user-item interaction matrix. This method enables more accurate prediction of user preferences for items.

[0070] Figure 1 The following schematically illustrates an exemplary system architecture 100 that can be applied to a method for predicting user preferences according to an embodiment of the present disclosure. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0071] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0072] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0073] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0074] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0075] It should be noted that the method for predicting user preferences provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the apparatus for predicting user preferences provided in the embodiments of the present disclosure can generally be set in the server 105. The method for predicting user preferences provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the apparatus for predicting user preferences provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0076] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0077] Figure 2 The flowchart of the method for predicting user preferences according to an embodiment of the present disclosure is schematically shown.

[0078] In operation S201 , a user-item interaction matrix is ​​obtained, where the user-item interaction matrix includes an evaluation value of at least one user on at least one item.

[0079] The so-called items here can be goods for sale, including but not limited to physical goods, virtual goods, and services. For example, physical goods include but are not limited to books, clothing, accessories, etc., virtual goods include but are not limited to music, videos, games, etc., and services include but are not limited to hotel services, housekeeping services, etc.

[0080] The user-item interaction matrix can be a matrix containing multiple evaluation values, where each element in the matrix is ​​a user's evaluation value for an item. For example, the matrix R can be in the following form:

[0081]

[0082] The matrix is ​​an H×K matrix, and the element r in the matrix ui represents the evaluation value of user u on item i, where H, K, u and i are all positive integers, 1≤u≤H, 1≤i≤K.

[0083] The evaluation value r can be calculated based on various factors ui These factors include, but are not limited to, implicit and explicit evaluation information. Implicit evaluation information includes, but is not limited to, whether a user has viewed the item's display information and / or the number of views. Explicit evaluation information includes, but is not limited to, the user's evaluation of the item.

[0084] In some embodiments, the evaluation value r can be calculated based on whether the user u has browsed the display information of the item i. ui , where the displayed information includes but is not limited to the text description of item i, the image of item i, etc. For example, on the display interface of the hotel service item i, the user u can be shown the introduction, image and link address of hotel i as the display information. If the user u clicks on any element on the display interface of hotel i, the evaluation value r ui Counted as 1, otherwise the evaluation value r ui Counted as 0.

[0085] In some embodiments, the evaluation value r can be calculated based on the number of times user u views the display information of item i. ui For example, if user u clicks on any element on the display interface of hotel i 3 times, the evaluation value r can be set to ui If the number is 4, the evaluation value r ui Counted as 4.

[0086] In some embodiments, the evaluation value r can be calculated based on the evaluation information of user u on item i. ui The evaluation information includes but is not limited to the user's rating of the item, the content of the user's text comment on the item, etc. For example, if user u gives item i a rating of 4, then the evaluation value r can be ui If the rating is 5, the evaluation value r ui 5. Optionally, semantic analysis can be performed on the content of the text review of user u on item i. If the analysis result is a positive evaluation, the evaluation value r ui If it is a negative review, the evaluation value r ui =-1, if it is a neutral evaluation, the evaluation value r ui Counted as 0.

[0087] Although the above describes the calculation method of the evaluation value with different examples, the embodiments of the present disclosure are not limited thereto, and the evaluation value can be calculated in any other way as needed. For example, the evaluation value can be calculated based on the number of clicks on the item-related images by the user and the rating of the item by the user.

[0088] In operation S202 , feature data of multiple types of images related to a target item among the at least one item are acquired.

[0089] According to embodiments of the present disclosure, each item can correspond to multiple categories of images. For example, for hotel services, multiple categories of hotel images can be displayed to users on a sales interface. Each category of images can correspond to an aspect of the hotel service, such as, but not limited to, images of guest rooms (first category images), images of public areas (second category images), images of catering services (third category images), images of surrounding transportation (fourth category images), and so on. A trained neural network model can be used to extract features from each category of images to obtain feature data for that category of images, which will be further explained below.

[0090] In operation S203 , a visual feature vector for the target item is generated based on the feature data of the multiple categories of images using an attention model.

[0091] In operation S204 , a Deep Matrix Factorization Model (DMF) is used to predict a preference of a target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix.

[0092] While the various operations of the method are described above in a specific order, the embodiments of the present disclosure are not limited thereto, and the operations described above may be performed in another order as needed. For example, step S201 may be performed after step S202, or simultaneously. In some embodiments, the generation of the user-item interaction matrix and the extraction of image features may be performed independently of the prediction process.

[0093] The following will refer to Figure 3 and Figure 4 To illustrate an example implementation of the above step S202. Figure 3 The flowchart of the method for obtaining feature data of each type of image according to an embodiment of the present disclosure is schematically shown. Figure 4 A schematic diagram schematically illustrates a method for obtaining feature data of each type of image according to an embodiment of the present disclosure.

[0094] In operation S3011, a first feature vector is extracted from each image in the class of images using a trained convolutional neural network.

[0095] like Figure 4 As shown, it is assumed that the multiple images belonging to category c include images M1 to M n In this step, images M1 to M n Input to the trained convolutional neural network. The trained convolutional neural network extracts the first feature vector F1 from the image M1, extracts the second feature vector F2 from the image M2, and so on, thereby obtaining feature vectors F1 to F nas the first eigenvector for class c.

[0096] Trained convolutional neural networks include, but are not limited to, VGG neural networks, Inception neural networks, and ResNet neural networks. You can select an appropriate neural network for image feature extraction as needed. Extracted image features include, but are not limited to, color, texture, shape, and spatial relationship features.

[0097] Deep learning fits data through the back propagation algorithm (BP), employing nonlinear activation functions to process complex nonlinear relationships within the data. Leveraging the power of massively parallel computing, end-to-end deep learning models use multi-layer neural networks to learn layer-by-layer representations from raw data. These representations are considered the data features of that layer. Compared to traditional manually designed features, data features extracted through deep learning are more concise and efficient, and both the trained models and features possess a certain degree of generalization. Deep convolutional neural networks (CNNs) replace fully connected layers with convolutional layers, reducing computational overhead and accelerating model fitting. They are commonly used in tasks such as image processing.

[0098] The following describes image feature extraction in detail using the VGG neural network model as an example. The VGG neural network model has various implementations, characterized by varying numbers of convolutional layers. The number of layers in different models ranges from 11 to 19. For example, the E-type VGG19 model takes a (224, 224) RGB three-channel color image as input (thus, images with larger pixels must be resized first). After two convolutional layers of conv3-64 (where 3 is the kernel size and 64 is the channel size), a maxpooling operation is performed. This is repeated four times with the same kernel size. Three fully connected layers of sizes 4096, 4096, and 1000 are then connected. The output of the final layer is soft-max normalized. It is important to note that the end-to-end pre-trained VGG19 model can extract image visual feature vectors at any layer within it. However, the feature vectors extracted by lower layers are more basic and simple, while the feature vectors extracted by higher layers have higher-dimensional feature representations. Considering that the last fully connected layer of the model has stronger model integration capabilities, and in order to obtain a lower-level and more original visual expression of the image, the embodiment of the present disclosure can use the output vector of the first fully connected layer as the first feature vector extracted from the pre-trained convolutional neural network.

[0099] In operation S3012, the extracted plurality of first feature vectors are preprocessed.

[0100] In some embodiments, the preprocessing includes but is not limited to at least one of the following operations: removing invalid first eigenvectors (e.g., first eigenvectors extracted from a solid color image) and erroneous first eigenvectors (e.g., first eigenvectors whose eigenvalues ​​are all 0) from the extracted multiple first eigenvectors; adjusting the parameters of the trained convolutional neural network according to the mean and variance of the multiple first eigenvectors (e.g., adjusting the relevant parameters of the pre-trained convolutional neural network for the case where the mean and variance of the overall eigenvector values ​​do not meet the preset requirements); and standardizing and normalizing the eigenvalues ​​in the multiple first eigenvectors (e.g., standardizing and normalizing the eigenvalues ​​for the case where the eigenvalues ​​of the eigenvectors are too large or too small). Operation S3012 may be an optional operation, and in some embodiments, this operation may be performed or not performed as needed.

[0101] In operation S3013 , the plurality of first eigenvectors are clustered using a clustering algorithm to obtain N clusters, where N is an integer greater than 1.

[0102] In some embodiments, the K-Means clustering algorithm can be used to cluster the extracted multiple first feature vectors to reduce visual information redundancy while retaining as much visual information of each image category as possible. For example, the first feature vectors can be clustered into N clusters based on the vector distance (such as but not limited to the Euclidean distance) between the first feature vectors. For example, Figure 4 As shown, N = 2. Through this step, the first feature vector is clustered into two clusters CS1 and CS2. Assuming that the first feature vector includes F1 to F8, in this step, F1, F3, and F8 are clustered into cluster CS1 and F2, F4, F5, F6, and F7 are clustered into cluster CS2 using the K-Means clustering algorithm.

[0103] In operation S3014 , N first eigenvectors are obtained as feature data of the image class by selecting a first eigenvector from each cluster.

[0104] For example, one or more first eigenvectors closest to the cluster center can be retained in each cluster. Figure 4 As shown, the first eigenvector F3 closest to the cluster center can be selected from cluster CS1, and the first eigenvector F7 closest to the cluster center can be selected from cluster CS2. Thus, two first eigenvectors F3 and F7 are obtained as feature data for category c.

[0105] For an item with C categories of images, by performing steps S3011 to S3014 on the C categories of images of the item, the same number of N first eigenvectors can be obtained for each category of images as the feature data of that category of images, thereby obtaining visual feature data of size (C×N×S) for the item, where C represents the total number of image categories of the item, and N represents the number of samples. That is, N first eigenvectors are selected from each image category as the feature data of that category of images, and S represents the length of the first eigenvector, which can depend on the trained neural network model. Next, the user's preference for the item can be predicted based on the feature data of each category of images of the item and the user-item interaction matrix.

[0106] Figure 5 The following schematically illustrates a method for predicting user preferences according to another embodiment of the present disclosure. Figure 6 Schematically shows Figure 5 Schematic diagram of the method. Figure 6 As shown, this embodiment adopts an attention model and a deep matrix decomposition model, where the attention model can include a first attention network layer and a second attention network layer.

[0107] In operation S501, feature data of various images of items and a user-item interaction matrix are obtained. Figure 6 In the example, it is assumed that item i has images of three categories (c=1, c=2, c=3, where c represents the image category), which can be obtained by the above reference. Figure 3 and Figure 4 The method described above can be used to obtain the feature data of each type of image. Figure 2 Described method to obtain the user-item interaction matrix.

[0108] In operation S502 , a first attention network is used to generate a second feature vector for each class of images based on the N first feature vectors of the class of images.

[0109] For example, reference Figure 6 , the first feature vector of each category c (respectively c=1, c=2, c=3) is input into the first attention network, and the first attention network can generate the second feature vector for each category c according to the following equations (1) to (3):

[0110] u ict =ReLU(W α f ict +b α ) (1)

[0111]

[0112] fic =∑ t a ict f ict (3)

[0113] Where i represents the item, c represents the image category, t represents the tth first eigenvector among the N first eigenvectors of each type of image, and f ict represents the tth first eigenvector of the cth category image of item i, ReLU represents the activation function, u ict represents f ict The implicit expression of a ict represents f ict The importance of f ic The second eigenvector of the c-th image of item i, W α represents the first permutation matrix between the first attention network layer and the second attention network layer, b α Represents the first bias term between the first attention network layer and the second attention network layer, u α Represents the first context vector of the first attention network layer and the second attention network layer.

[0114] According to equation (1), the ReLU function is used as the activation function and f is transformed into ict Mapped to an implicit representation u ict According to Equation (2), a jointly learned first context vector u is used α To measure the importance of the visual features of each image, where u ict and u α The dot product of u gives ict The score indicates that u ict The importance of the current category; and the normalized value a of the visual importance is calculated using the softmax function ict According to equation (3), by calculating f ict and a ict Weighted sum, for the category c, the second eigenvector f is obtained ic The first context vector u α It can be preset as a default value and can be adjusted during the prediction process based on the difference between the predicted result and the true value.

[0115] In this way, for class c=1, the second eigenvector f is obtained i1 ; For category c = 2, get the second eigenvector f i2 ; For category c = 3, get the second eigenvector f i3 .

[0116] In operation S503 , a second attention network is used to generate a visual feature vector for the target item according to the second feature vectors of the multi-class images.

[0117] For example, the second feature vector f for each category can be i1 、f i2 and f i3 Input to the second attention network, the second attention network generates the visual feature vector f of item i according to the following equations (4) to (6) i .

[0118] u ic =ReLU(W β f ic +b β ) (4)

[0119]

[0120] f i =∑ c a ic f ic (6)

[0121] where u ic represents f ic The implicit expression of a ic represents f ic The importance of f i Represents the visual feature vector of item i, where item i is the target item, W β represents the second permutation matrix between the first attention network layer and the second attention network layer, b β b α Represents the second bias term between the first attention network layer and the second attention network layer, u β Represents the second context vector of the first attention network layer and the second attention network layer. The second context vector u β It can be preset as a default value and can be adjusted during the prediction process based on the difference between the predicted result and the true value.

[0122] refer to Figure 6 , the visual feature vector f of item i output by the second attention network can be i Provided to the deep matrix decomposition model, the deep matrix decomposition model can be used to perform the following steps S504 to S507 based on the visual feature vector f of the object i i and the user-item interaction matrix to predict user u’s preference for item i.

[0123] In operation S504 , a feature vector of the target user is generated based on the evaluation values ​​of the target user on all items in the user-item interaction matrix.

[0124] refer to Figure 6 , for the target user u, the deep matrix factorization model can use the u-th row element in the user-item interaction matrix to generate the feature vector r of the target user u u , as mentioned above, the u-th row element in the user-item interaction matrix R represents the evaluation value of user u on all items.

[0125] In operation S505 , an evaluation feature vector of the target item is generated based on the evaluation values ​​of the target item by all users in the user-item interaction matrix.

[0126] refer to Figure 6 , for the target item i, the deep matrix factorization model can use the i-th column element in the user-item interaction matrix to generate the evaluation feature vector r of the target item i i , as mentioned above, the elements in the i-th column of the user-item interaction matrix R represent the evaluation values ​​of all users on all items i.

[0127] In operation S506 , a feature vector of the target item is generated according to the evaluation feature vector of the target item and the visual feature vector of the target item.

[0128] refer to Figure 6 , the visual feature vector f of the target item i output by the second attention network in the above operation S503 can be i The evaluation feature vector r of the target item i obtained in the above operation 505 u Add them together to get the feature vector of target item i.

[0129] In operation S507 , the preference of the target user for the target item is predicted based on the cosine similarity between the feature vector of the target user and the feature vector of the target item.

[0130] refer to Figure 6 The deep matrix decomposition model can be based on the following equations (7) to (9), by calculating the feature vector of the target item i obtained in the above operation S506 and the feature vector r of the target user obtained in the above operation S504. u The cosine similarity of is used to predict the preference of user u for item i.

[0131]

[0132] in represents the preference of user u for item i (e.g., predicted score). θ represents the model parameters, and F(·) represents the function that maps the parameters to the predicted score. The feature vector q u and p iRepresenting user u and item i respectively, the two feature vectors are defined as follows:

[0133] q u =φ u (r u ) (8)

[0134]

[0135] where φ u and φ i denote the fully connected layers of user u and item i respectively. φ denotes a fully connected network that learns the spatial representation of the input vector and permutes its dimensions to keep it consistent.

[0136] In some embodiments, after step S507, the model may be tested and iteratively updated. For example, the target user's actual preference for the target item may be obtained, and the parameters of at least one of the attention model and the deep matrix decomposition model may be adjusted based on the difference between the predicted preference and the actual preference. For example, a normalized cross entropy loss may be used as a loss function, which is normalized by max(R) (R represents the user rating, and max(R) represents the maximum value in the rating data). Therefore, the objective function of the model is defined as follows:

[0137]

[0138] Where Ω is the regularizer and λ is a hyperparameter. u，i represents the true preference of user u for item i from the dataset, represents the degree of preference of user u for item i predicted by equation (7).

[0139] In some embodiments, after completing the prediction of the user's preference for an item, for example, completing the prediction of the preference of multiple users for multiple items, items may be recommended to the user based on the prediction results.

[0140] Item recommendation methods include but are not limited to content-based recommendation, collaborative filtering recommendation, hybrid recommendation, etc.

[0141] Content-based recommendation is a continuation and development of information filtering technology. It makes recommendations based on the content of items. In a content-based recommendation system, items or objects are defined by their associated feature attributes. Based on the characteristics of the user-evaluated objects, the system learns the user's interests and examines the degree of compatibility between the user profile and the item being recommended. The user model is determined by the learning method used; common methods include decision trees, neural networks, and vector-based representations.

[0142] Collaborative filtering is one of the earliest and most successful technologies used in recommendation systems. It typically employs the nearest neighbor technique, leveraging historical user preferences to calculate distances between users. It then uses the weighted product ratings of the target user's nearest neighbors to predict the target user's preference for a particular product. The system then makes recommendations based on this preference. The advantage of collaborative filtering is that it has no specific requirements for the recommended items and can handle complex, unstructured content such as music and movies.

[0143] Hybrid recommendation combines multiple recommendation methods to maximize their strengths and minimize their weaknesses. In practice, a variety of methods can be used to combine single recommendation algorithms.

[0144] The disclosed embodiments are applicable to recommendation systems and data mining scenarios involving multi-classified images, such as online shopping platforms. By analyzing the intrinsic relationships between images of different classifications based on an attention algorithm and a deep matrix factorization algorithm model, the disclosed embodiments can mine potential user interests, predict a user's interest in an item, and identify effective user needs and provide personalized item recommendations and rankings.

[0145] Figure 7 The block diagram schematically shows an apparatus for predicting user preferences according to an embodiment of the present disclosure.

[0146] like Figure 7 As shown, the apparatus 700 for predicting user preferences includes a first acquisition module 710 , a second acquisition module 720 , a feature extraction module 730 and a prediction module 740 .

[0147] The first acquisition module 710 may acquire a user-item interaction matrix, where the user-item interaction matrix includes an evaluation value of at least one user on at least one item.

[0148] The second acquisition module 720 may acquire feature data of multiple types of images related to a target item among the at least one item.

[0149] The feature extraction module 730 may use an attention model to generate a visual feature vector for the target item based on the feature data of the multiple categories of images.

[0150] The prediction module 740 may use a deep matrix decomposition model to predict the preference of a target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix.

[0151] According to the modules, submodules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any of them. Alternatively, according to the modules, submodules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0152] For example, any multiple of the first acquisition module 710, the second acquisition module 720, the feature extraction module 730, and the prediction module 740 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 710, the second acquisition module 720, the feature extraction module 730, and the prediction module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 710 , the second acquisition module 720 , the feature extraction module 730 and the prediction module 740 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0153] Figure 8 The block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. Figure 8The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0154] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0155] Various programs and data required for the operation of the system 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 executes the programs in the ROM 802 and / or RAM 803 to perform various operations of the method flow according to the embodiment of the present disclosure. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and RAM 803. The processor 801 may also execute the programs stored in the one or more memories to perform various operations of the method flow according to the embodiment of the present disclosure.

[0156] According to an embodiment of the present disclosure, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. System 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. Drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0157] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0158] The present disclosure also provides a computer-readable storage medium, which may be included in the apparatus / electronic device described in the above embodiments, or may exist independently and not be incorporated into the apparatus / electronic device. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0159] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or couplings are intended to fall within the scope of this disclosure.

[0162] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for predicting user preferences, comprising: Obtaining a user-item interaction matrix, wherein the user-item interaction matrix includes an evaluation value of at least one user for at least one item; Acquiring feature data of multiple types of images related to a target item among the at least one item includes: For each class of images, a first eigenvector is extracted from each image in the class using a trained convolutional neural network, where the first eigenvector represents an output vector of a first fully connected layer; the plurality of extracted first eigenvectors are clustered using a clustering algorithm to obtain N clusters, where N is an integer greater than 1; and N first eigenvectors are obtained as feature data for the image class by selecting a first eigenvector from each cluster; Generating a visual feature vector for the target object based on the feature data of the multiple categories of images using the attention model includes: Generating a second feature vector for each class of images based on the N first feature vectors of each class of images using the first attention network includes: generating a second feature vector for each class of images based on the N first feature vectors of each class of images using the following equation: in, Indicates items, represents the image category, Represents the first feature vector of each class of images The first eigenvector, Indicates items No. Class image The first eigenvector, ReLU represents the activation function, express The implicit expression of express The importance of Indicates items No. The second eigenvector of the class image, represents the first permutation matrix between the first attention network layer and the second attention network layer, represents the first bias term of the first attention network layer and the second attention network layer, Representing the first context vector of the first attention network layer and the second attention network layer; Generating a visual feature vector for the target item based on the second feature vectors of the multi-class images using a second attention network includes: generating a visual feature vector for the target item based on the second feature vectors of the multi-class images using the following equation: in, express The implicit expression of express The importance of Indicates items The visual feature vector of the item is the target item, represents the second permutation matrix between the first and second attention network layers, represents the second bias term between the first and second attention network layers, a second context vector representing a first attention network layer and a second attention network layer, wherein the attention model includes the first attention network layer and the second attention network layer; Predicting the target user's preference for the target item based on the visual feature vector and the user-item interaction matrix using a deep matrix decomposition model includes: Generating a feature vector of a target user based on row elements in the user-item interaction matrix that represent evaluation values ​​of the same target user on all items; generating an evaluation feature vector of the target item based on column elements in the user-item interaction matrix that represent evaluation values ​​of the target item by all users; generating a feature vector of the target object according to the evaluation feature vector of the target object and the visual feature vector of the target object; The preference degree of the target user for the target item is predicted according to the cosine similarity between the feature vector of the target user and the feature vector of the target item.

2. The method according to claim 1, further comprising: Obtaining the target user's actual preference for the target item; Adjust parameters of at least one of the attention model and the deep matrix decomposition model according to the difference between the predicted preference level and the actual preference level.

3. The method according to claim 1 , further comprising performing at least one of the following operations before clustering the extracted plurality of first feature vectors using a clustering algorithm: removing invalid first eigenvectors and erroneous first eigenvectors from the extracted plurality of first eigenvectors; adjusting parameters of the trained convolutional neural network according to the mean and variance of the plurality of first feature vectors; and The eigenvalues ​​in the plurality of first eigenvectors are standardized and normalized.

4. The method according to claim 1, wherein The clustering algorithm includes a K-Means clustering algorithm.

5. The method according to any one of claims 1 to 4, wherein The evaluation value of each user for each item in the user-item interaction matrix is ​​calculated according to at least one of the following: whether the user has browsed display information of the item, wherein the display information includes at least one of a text description of the item and an image of the item; The number of times the user has viewed the item's display information; and The user's evaluation information for the item.

6. The method according to any one of claims 1 to 4, further comprising: Item recommendations are made based on the predicted target user's preference for the target item.

7. A device for predicting user preferences, comprising: A first acquisition module is configured to acquire a user-item interaction matrix, wherein the user-item interaction matrix includes an evaluation value of at least one user on at least one item; a second acquisition module, configured to acquire feature data of multiple types of images related to a target item among the at least one item; The second acquisition module is further configured to: for each class of images, extract a first feature vector from each image in the class using a trained convolutional neural network, wherein the first feature vector represents an output vector of the first fully connected layer; cluster the extracted multiple first feature vectors using a clustering algorithm to obtain N clusters, where N is an integer greater than 1; and select a first feature vector from each cluster to obtain N first feature vectors as feature data of the class of images; a feature extraction module, configured to generate a visual feature vector for the target object based on the feature data of the multiple categories of images using an attention model; The feature extraction module is further configured to: Generating a second feature vector for each class of images based on the N first feature vectors of each class of images using the first attention network includes: generating a second feature vector for each class of images based on the N first feature vectors of each class of images using the following equation: in, Indicates items, represents the image category, Represents the first feature vector of each class of images The first eigenvector, Indicates items No. Class image The first eigenvector, ReLU represents the activation function, express The implicit expression of express The importance of Indicates items No. The second eigenvector of the class image, represents the first permutation matrix between the first attention network layer and the second attention network layer, represents the first bias term of the first attention network layer and the second attention network layer, Representing the first context vector of the first attention network layer and the second attention network layer; Generating a visual feature vector for the target item based on the second feature vectors of the multi-class images using a second attention network includes: generating a visual feature vector for the target item based on the second feature vectors of the multi-class images using the following equation: in, express The implicit expression of express The importance of Indicates items The visual feature vector of the item is the target item, represents the second permutation matrix between the first and second attention network layers, represents the second bias term between the first and second attention network layers, a second context vector representing a first attention network layer and a second attention network layer, wherein the attention model includes the first attention network layer and the second attention network layer; a prediction module, configured to predict a preference of a target user among the at least one user for the target item based on the visual feature vector and the user-item interaction matrix using a deep matrix decomposition model; The prediction module is further configured to: Generating a feature vector of a target user based on row elements in the user-item interaction matrix that represent evaluation values ​​of the same target user on all items; generating an evaluation feature vector of the target item based on column elements in the user-item interaction matrix that represent evaluation values ​​of the target item by all users; generating a feature vector of the target object according to the evaluation feature vector of the target object and the visual feature vector of the target object; The preference degree of the target user for the target item is predicted according to the cosine similarity between the feature vector of the target user and the feature vector of the target item.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 6.

Images