Information recommendation method and device based on multi-classification images

Abstract

The present invention discloses a method and device for recommending information based on multi-classified images, which can be applied to, but not limited to, the field of artificial intelligence. The method comprises: collecting multiple images of a target object in multiple visual categories; inputting each collected object image into a pre-trained image feature vector extraction model to output an image feature vector corresponding to each image; determining the visual feature vector of the target object in each visual category based on the image feature vectors of the multiple object images in each visual category; determining the target user's preference for each visual category of the target object based on the visual feature vectors of the target object in each visual category and the target user's preference for different visual categories; and determining information to be recommended to the target user based on the target user's preference for each visual category of the target object. The present invention can effectively analyze the user's interest in different visual aspects of an object.

Description

Technical Field

[0001] The present invention discloses an information recommendation method and device based on multi-classification images, which can be applied to the field of artificial intelligence and any field other than the field of artificial intelligence. The application field of the information recommendation method and device based on multi-classification images disclosed by the present invention is not limited. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] Recommendation systems, as an effective data mining method, are widely used in e-commerce, online video viewing, and other fields. By analyzing user interests, preferences, and historical interaction data, recommendation systems intelligently generate personalized information recommendation lists for each user based on recommendation algorithms from massive amounts of data. The recommendation system identifies user interests, guides users to discover desired information or product needs, and provides precise recommendations.

[0004] In real-world applications like e-commerce and online video viewing, it's often necessary to incorporate unstructured data like images and videos as visual aids to enhance the performance of recommendation systems. Therefore, feature engineering of these visual aids, as well as methods for integrating visual features into recommendation systems, have become key research topics in the product recommendation field.

[0005] Existing feature engineering for visual features has shifted from traditional manual feature extraction to more efficient automatic feature extraction using deep neural networks. However, most recommendation systems use only a single visual feature vector, or a global visual feature vector, to represent an item's visual information. This approach ignores the correlation between local and global representations of visual information.

[0006] For a specific object, there are often different sides to represent the visual information of the object. Different sides can show different visual features. If you simply use average pooling or maximum pooling to integrate multiple image features of the object, the visual information of the object from different perspectives will be lost.

[0007] To address the above issues, no effective solutions have been proposed so far. Summary of the Invention

[0008] An embodiment of the present invention provides an information recommendation method based on multi-classification images, which is used to solve the technical problem that the visual auxiliary information used in the existing information recommendation system is the visual information of the object represented by a single visual feature vector, and the expression of the visual information of the object is incomplete. The method includes: collecting multiple object images of the target object in multiple visual categories, wherein each visual category corresponds to a side of the target object; inputting each collected object image into a pre-trained image feature vector extraction model, and outputting the image feature vector corresponding to each image; determining the visual feature vector of the target object in each visual category based on the image feature vectors of the multiple object images of the target object in each visual category; determining the target user's preference for each visual category of the target object based on the visual feature vector of the target object in each visual category and the target user's preference for different visual categories; and determining the information to be recommended to the target user based on the target user's preference for each visual category of the target object.

[0009] An embodiment of the present invention further provides an information recommendation device based on multi-classification images, which is used to solve the technical problem that the visual auxiliary information used in existing information recommendation systems is the visual information of the object represented by a single visual feature vector, resulting in incomplete expression of the object visual information. The device includes: an image acquisition module, which is used to acquire multiple object images of the target object in multiple visual categories, wherein each visual category corresponds to a side of the target object; an image feature vector extraction module, which is used to input each acquired object image into a pre-trained image feature vector extraction model and output an image feature vector corresponding to each image; an object visual feature vector determination module, which is used to determine the visual feature vector of the target object in each visual category based on the image feature vectors of the multiple object images of the target object in each visual category; a user preference determination module, which is used to determine the target user's preference for each visual category of the target object based on the visual feature vector of the target object in each visual category and the target user's preference for different visual categories; and an information recommendation module, which is used to determine the information to be recommended to the target user based on the target user's preference for each visual category of the target object.

[0010] An embodiment of the present invention also provides a computer device to solve the technical problem that the visual auxiliary information used in the existing information recommendation system is the visual information of the object represented by a single visual feature vector, and the expression of the visual information of the object is incomplete. The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned information recommendation method based on multi-classification images is implemented.

[0011] An embodiment of the present invention also provides a computer-readable storage medium to solve the technical problem that the visual auxiliary information used in the existing information recommendation system is the visual information of the object represented by a single visual feature vector, and the expression of the visual information of the object is incomplete. The computer-readable storage medium stores a computer program for executing the above-mentioned information recommendation method based on multi-classification images.

[0012] The information recommendation method, apparatus, computer device, and computer-readable storage medium based on multi-classification images provided in the embodiments of the present invention, after collecting multiple object images of a target object in multiple visual categories, input each collected object image into a pre-trained image feature vector extraction model, output the image feature vector corresponding to each image, and then determine the visual feature vector of the target object in each visual category based on the image feature vectors of the multiple object images of the target object in each visual category. Combined with the target user's preference for different visual categories, the target user's preference for each visual category of the target object is determined. Finally, based on the target user's preference for each visual category of the target object, the information to be recommended to the target user is determined.

[0013] Compared with the information recommendation scheme in the prior art that only uses a single visual feature vector, the embodiment of the present invention takes into account the different visual preferences of users for objects from different perspectives and can effectively analyze the user's interest in the different perspectives of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0015] Figure 1 This is a flow chart of an information recommendation method based on multi-classification images in an embodiment of the present invention;

[0016] Figure 2 This is a flow chart of a method for analyzing user visual feature information in an embodiment of the present invention;

[0017] Figure 3 This is a flow chart of a method for performing dimensionality reduction analysis on user visual feature information in an embodiment of the present invention;

[0018] Figure 4 A schematic diagram of a VGG deep neural network model architecture in an embodiment of the present invention;

[0019] Figure 5Schematic diagram of an information recommendation model based on multi-classification images in an embodiment of the present invention;

[0020] Figure 6 Schematic diagram of an information recommendation device based on multi-classification images in an embodiment of the present invention;

[0021] Figure 7 Schematic diagram of an optional information recommendation device based on multi-classification images in an embodiment of the present invention;

[0022] Figure 8 Schematic diagram of an optional information recommendation device based on multi-classification images in an embodiment of the present invention;

[0023] Figure 9 Schematic diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0025] Before introducing the embodiments of the present invention, the professional terms involved in the embodiments of the present invention are explained as follows:

[0026] 1. Recommendation algorithm: Common recommendation algorithms include content-based recommendation, collaborative filtering recommendation and hybrid recommendation.

[0027] 1) Content-based recommendation algorithms are a continuation and development of information filtering technology. They make recommendations based on the content of items, rather than relying on user evaluations of the items. Instead, they rely on machine learning to derive user interest profiles from examples describing the content's characteristics. In content-based recommendation systems, items or objects are defined by their associated feature attributes. The system evaluates the characteristics of the objects based on user evaluations, learns user interests, and examines the degree of match between the user profile and the item being predicted. The user profile model depends on the learning method used; commonly used methods include decision trees, neural networks, and vector-based representations. Content-based user profiles require historical user data, and the user profile model may change as user preferences evolve.

[0028] 2) The Collaborative Filtering Recommendation algorithm is one of the earliest and most successful technologies used in recommendation systems. It generally employs the nearest neighbor technique, leveraging historical user preference information to calculate the distance between users. It then uses the weighted product ratings of the target user's nearest neighbors to predict the target user's preference for a specific product, and then makes recommendations based on this preference. The greatest advantage of collaborative filtering is that it has no specific requirements for the recommended items and can handle complex, unstructured content such as music and movies.

[0029] 3) Hybrid recommendation algorithms combine multiple recommendation methods to maximize their strengths and minimize their weaknesses. In theory, hybrid recommendation algorithms are no worse than any single recommendation algorithm. In practice, various methods are used to combine single algorithms.

[0030] 2. Visual Feature Vectors: Vision, as one of the most important channels for human sensory information, plays an irreplaceable role in human life. Visual information, such as images and videos, as abstract information that is easier for humans to understand, has long been quantified and applied to data mining and recommendation tasks. Computers extract features from image visual information. Common visual features include color, texture, shape, and spatial relationship features. Thanks to the rapid development of deep learning, complex deep neural networks, with their deep nonlinear networks, can capture features from low-level concrete to highly abstract representations, and adaptively learn distributed features from heterogeneous data. This lays the foundation for extracting high-dimensional visual feature vectors from images that are consistent with human description and cognition and suitable for large-scale machine learning. Research based on transfer learning has shown that deep neural networks trained on benchmark datasets (such as the ImageNet dataset) can generalize to extract visual features from other datasets, demonstrating excellent generalization performance and high accuracy across various visual tasks on these new datasets.

[0031] The embodiment of the present invention provides an information recommendation method based on multi-classification images, which can be applied to but not limited to the recommendation of information such as videos and products.

[0032] Figure 1 This is a flow chart of an information recommendation method based on multi-classification images provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0033] S101 , collecting multiple object images of a target object under multiple visual categories, wherein each visual category corresponds to a side of the target object.

[0034] It should be noted that the target item in the embodiments of the present invention can be a product that a user is browsing on an e-commerce platform or a video that a user is watching on a video playback platform. Because different users have different visual preferences for the same item from different viewing angles, when recommending related products or video objects to a user based on the products they browse or the video they watch, making recommendations based on the user's visual preferences for different aspects of the item can further improve recommendation efficiency.

[0035] S102: Input each collected object image into a pre-trained image feature vector extraction model, and output the image feature vector corresponding to each image.

[0036] It should be noted that the above-mentioned image feature vector extraction model refers to a model obtained in advance through machine learning training, which can extract image feature vectors based on object images. Optionally, in an embodiment of the present invention, machine learning can be performed on a deep learning neural network model to train an image feature vector extraction model.

[0037] S103 , determining a visual feature vector of the target object in each visual category based on the image feature vectors of the plurality of object images of the target object in each visual category.

[0038] It should be noted that, for each visual category (ie, each side of the target object), determining the visual feature vector of the object in the visual category based on the image feature vectors of multiple images has a higher accuracy rate.

[0039] Optionally, the above S103 may adopt a mean pooling algorithm to determine the visual feature vector of the target object in each visual category based on the image feature vectors of multiple object images of the target object in each visual category.

[0040] S104 , determining the target user's preference for each visual category of the target item based on the visual feature vector of the target item in each visual category and the target user's preference for different visual categories.

[0041] It should be noted that different users have different visual preferences for different visual categories of an item. This can be achieved by analyzing a large amount of historical user interaction data when browsing products or watching videos. After determining the visual feature vectors of the target item in each visual category, the target user's preference for each visual category can be determined based on the target user's preference for each visual category and the visual feature vectors of the target item in each visual category.

[0042] S105 , determining the information to be recommended to the target user based on the target user's preference for each visual category of the target item.

[0043] After determining the target user's preference for each visual category of the target item, item information under the visual category of interest may be recommended to the target user.

[0044] In one embodiment, Figure 2 As shown, the information recommendation method based on multi-classification images provided in the embodiment of the present invention can analyze user visual feature information through the following steps:

[0045] S201, collecting user-item interaction data of multiple users on multiple items;

[0046] S202, generating a user-item interaction matrix based on the collected user-item interaction data;

[0047] S203, decomposing the user-item interaction matrix based on a Bayesian optimization matrix decomposition model to obtain a user visual feature vector matrix and an item visual feature vector matrix;

[0048] S204: Determine the target user's preference for different visual categories based on the user's visual feature vector matrix.

[0049] In one embodiment, Figure 3 As shown, the information recommendation method based on multi-classified images provided in the embodiment of the present invention can perform dimensionality reduction processing on the user's visual feature information through S301, and then the above S204 can determine the target user's preference for different visual categories through S302:

[0050] S301, using the embedding matrix, reducing the dimension of the user visual feature vector matrix to obtain a reduced-dimensional user visual feature vector matrix;

[0051] S302: Determine the target user's preference for different visual categories based on the user visual feature vector matrix after dimensionality reduction.

[0052] In an embodiment of the present invention, a pre-trained deep neural network model is used to extract the visual feature vector of an image; a Bayesian optimized matrix decomposition model is used to effectively process a large amount of data in a short period of time, thereby providing an information recommendation system for multi-classified images.

[0053] The information recommendation method based on multi-classified images provided in the embodiments of the present invention can analyze the impact of different image classifications on the expression of the visual features of items, thereby realizing the discovery of potential points of interest for users. In specific implementation, the information recommendation method based on multi-classified images provided in the embodiments of the present invention can be implemented through the following process:

[0054] 1) Use pre-trained deep neural network to extract image feature vectors:

[0055] Deep learning fits data through the back propagation algorithm (BP), employing nonlinear activation functions to process complex nonlinear relationships within the data. Assisted by the computing power of massively parallel computing, end-to-end deep learning models use multi-layer neural networks to learn representations from raw data layer by layer. These representations are considered the data features of that layer. Compared to traditional manually designed features, data features extracted through deep learning are more concise and efficient, and both the trained models and features possess a certain degree of generalization. Among them, deep convolutional neural networks (CNNs) replace fully connected layers with convolutional layers, reducing computational overhead and accelerating model fitting. They are commonly used in tasks such as image processing.

[0056] like Figure 4 The figure below shows the architecture of the VGG deep neural network model. The VGG model has various implementations, characterized by varying numbers of convolutional layers. The number of layers in different models increases from 11 in the A-type model on the left to 19 in the E-type model on the right. Taking the E-type model VGG19 as an example, the model takes a (224, 224) RGB three-channel color image as input (thus, images with larger pixels must be resized first). After two convolutional layers (conv3-64 (where 3 is the kernel size and 64 is the channel size), the model performs a maximum pooling operation. This is repeated four times with the same kernel size, followed by three fully connected layers of sizes 4096, 4096, and 1000, respectively. The output of the final layer is normalized using soft-max. It should be noted that the end-to-end pre-trained VGG19 model can extract image visual feature vectors at any layer within it, but the feature vectors extracted by the earlier network layers are lower-level and simpler, while the feature vectors extracted by the later network layers have higher-dimensional feature expressions. Considering that the final fully connected layer of the model has stronger model integration capabilities, and in order to obtain the lower-level, more primitive visual expression of the image, in this embodiment of the present invention, the output vector of the first fully connected layer is used as the image feature extracted from the pre-trained deep neural network.

[0057] It should be noted that common convolutional neural network models (e.g., VGG, Inception, ResNet, etc.) can be used as visual feature vector extraction models. Those skilled in the art can modify the extraction model accordingly based on the actual application scenario.

[0058] In an embodiment of the present invention, a deep neural network model pre-trained with large-scale image data is used to extract image feature vectors. The image feature vectors are used as visual auxiliary information to analyze the user's visual feature preference, thereby enhancing the performance of the information recommendation system.

[0059] 2) Dataset and data preprocessing (including collecting data information to obtain the user-item interaction (rating) matrix and visual feature vector analysis):

[0060] The recommendation system is built on a wide range of user-item behavior data, and data preprocessing and analysis are often required before making recommendations.

[0061] ① User-item interaction matrix:

[0062] The user-item interaction data in the present invention utilizes implicit user feedback, such as the number of times a user clicks on an image as a measure of user interest in the image. In such cases, it can be directly assumed that the number of times a user clicks on an image, or whether or not a user clicks on an image, serves as an indicator of user preference for the image.

[0063] ②Visual feature vector analysis:

[0064] After extracting visual features, the visual feature vectors need to be quantitatively analyzed. Visual features extracted from invalid images (such as solid color images) should be eliminated, as should erroneous feature vectors, such as those containing all zeros. If the mean and variance of the overall feature vector values ​​are unreasonable, the relevant parameters of the pre-trained model need to be adjusted. If the feature vector values ​​are too large or too small, the feature values ​​need to be normalized.

[0065] ③Use average pooling for multiple image features under each image category:

[0066] Assume that an object has c image categories, and each category has a different number of images. In order to retain as much visual information as possible from each image category while reducing visual information redundancy, the present invention uses mean pooling to obtain the visual feature expression of each image category. Therefore, after visual feature extraction and mean pooling, a visual feature vector of dimension (c×s) is obtained to identify the visual information of c different aspects of the object, where s represents the length of the feature vector extracted by the pre-trained network, which is controlled by the pre-trained deep neural network model.

[0067] 3) Modeling the user's implicit visual features, simulating the user's visual feature preferences through adaptive parameters learned together with the model, and using the matrix decomposition model based on Bayesian optimization to train the data:

[0068] Based on the Bayesian optimization matrix decomposition model, the present invention predicts the user's preference for an item by modeling the user's implicit features, the item's implicit features, the user's visual implicit features, and the item's visual implicit features.

[0069]

[0070] in, represents the degree of preference of user u for item i predicted by the model, α is the global offset, β u and β i This part indicates the deviation of users and items. and γ i Represent the implicit feature vectors of users and items respectively, θ u represents the visual implicit features of user u, and E is the embedding matrix. i To represent the visual features of item i. Considering that users have different visual preferences for different perspectives, we introduce a parameter W u To express the user’s personal preference for the visual information of different viewpoints of an object. Here, W u is a weight vector (of dimension c×1) corresponding to the c visual views of the item. Finally, A is the overall bias weight of the multi-view visual features. W is the category visual preference matrix of all users.

[0071] Figure 5 The model structure proposed in the present invention is shown. First, the visual features of each image are extracted using a pre-trained neural network model. For each image, visual features of a specific dimension (such as 4096 dimensions) are extracted from the pre-trained neural network model. The visual feature vectors of multiple images under each category (different perspectives) are average pooled to obtain a new visual feature vector to represent the visual information of the item under this image category. The visual feature vectors of the different perspectives of the item are then spliced ​​together to form a visual feature matrix to represent the multi-perspective visual features of the item. The visual feature matrix is ​​represented by the weights w associated with the user. u (w1,w2,...,w c ) is projected into a feature vector, whose weight reflects the user u’s personalized visual preference for different viewing angles of the object. Finally, the visual feature vector is reduced in dimensionality using the embedding matrix E to obtain a low-dimensional feature vector.

[0072] The model proposed in this invention is a pairwise sorting method based on the maximum a posteriori estimation of Bayesian analysis. s It consists of a triple (u,i,j):

[0073]

[0074] Where u represents the user, i represents the user's positive feedback, and j represents the unknown user feedback. Matrix decomposition is used to predict user preferences, and the predicted value is expressed as express:

[0075]

[0076] and Defined by formula (1). Therefore, the prediction model It can be expressed as:

[0077]

[0078] Here ij Yes i and γ j The difference between the two, F ij It's F i and F j The difference between .

[0079] The optimization criterion C for personalized ranking is defined as:

[0080]

[0081] Where σ is the sigmoid function, λ θ is the regularization hyperparameter adjusted through experimental results.

[0082] The ranking model can be effectively learned using stochastic gradient descent. The gradient of the optimization criterion C with respect to the model parameter θ is:

[0083]

[0084]

[0085] All parameters are updated at a speed adjusted by the learning rate η:

[0086]

[0087] The model inputs a feedback dataset consisting of user-item interaction pairs and item visual features. The output is all parameters learned by the model. In each iteration, the model is updated using all user-item interaction pairs. For each user-item interaction pair in the training data, an item j unknown to the user is randomly selected to form a triplet (u, i, j), and the parameters of each triplet are updated according to the formula. For each sampled triplet (u, i, j), the model's time complexity is O(K2×F). F is the dimensionality of the visual features. Assuming that a complete iteration has n randomly sampled triplets, the time complexity of each iteration is O(n(K2×F)). The number of triplets n depends on the number of user-item interaction pairs.

[0088] In an embodiment of the present invention, a multi-class image recommendation system is implemented based on deep neural network feature extraction and a Bayesian optimization matrix decomposition model. It supports inputting multiple visual category image features of a target item and uses adaptive parameters to learn the visual feature representations of different aspects of the target item. A user-item interaction (rating) matrix is ​​obtained and modeled using a Bayesian optimization matrix decomposition model. A permutation matrix learned jointly within the model is used to train the implicit representation of visual features, and model prediction is performed using the Bayesian optimization matrix decomposition model.

[0089] Based on the same inventive concept, embodiments of the present invention also provide an information recommendation device based on multi-classified images, as described in the following embodiments. Because the principles underlying the problem solved by this device are similar to those of the information recommendation method based on multi-classified images, the implementation of this device can be referenced to the implementation of the information recommendation method based on multi-classified images, and any repetitions will not be repeated.

[0090] Figure 6 FIG. 1 is a schematic diagram of an information recommendation device based on multi-classification images according to an embodiment of the present invention. Figure 6 As shown, the device includes: an image acquisition module 601, an image feature vector extraction module 602, an item visual feature vector determination module 603, a user preference determination module 604 and an information recommendation module 605.

[0091] Among them, the image acquisition module 601 is used to collect multiple object images of the target object in multiple visual categories, wherein each visual category corresponds to a side of the target object; the image feature vector extraction module 602 is used to input each collected object image into a pre-trained image feature vector extraction model, and output the image feature vector corresponding to each image; the object visual feature vector determination module 603 is used to determine the visual feature vector of the target object in each visual category based on the image feature vectors of multiple object images of the target object in each visual category; the user preference determination module 604 is used to determine the target user's preference for each visual category of the target object based on the visual feature vector of the target object in each visual category and the target user's preference for different visual categories; the information recommendation module 605 is used to determine the information to be recommended to the target user based on the target user's preference for each visual category of the target object.

[0092] It should be noted that the image acquisition module 601, image feature vector extraction module 602, item visual feature vector determination module 603, user preference determination module 604, and information recommendation module 605 described above correspond to S101 to S105 in the method embodiment. The examples and application scenarios implemented by these modules and corresponding steps are the same, but are not limited to the contents disclosed in the method embodiment described above. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0093] In one embodiment, Figure 7 As shown, the information recommendation device based on multi-classification images provided in the embodiment of the present invention may also include: a user-item interaction data acquisition module 606, a user interaction matrix generation module 607, a matrix decomposition module 608 and a user visual feature analysis module 609.

[0094] Among them, the user-item interaction data collection module 606 is used to collect user-item interaction data of multiple users on multiple items; the user interaction matrix generation module 607 is used to generate a user-item interaction matrix based on the collected user-item interaction data; the matrix decomposition module 608 is used to decompose the user-item interaction matrix based on the Bayesian optimization matrix decomposition model to obtain a user visual feature vector matrix and an item visual feature vector matrix; the user visual feature analysis module 609 is used to determine the target user's preference for different visual categories based on the user visual feature vector matrix.

[0095] It should be noted that the user-item interaction data collection module 606, user interaction matrix generation module 607, matrix decomposition module 608, and user visual feature analysis module 609 described above correspond to S201 to S204 in the method embodiment. The examples and application scenarios implemented by these modules and corresponding steps are the same, but are not limited to the contents disclosed in the method embodiment described above. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0096] In one embodiment, Figure 8 As shown, the information recommendation device based on multi-classification images provided in the embodiment of the present invention may also include: a user visual feature dimensionality reduction module 610, which is used to use the embedding matrix to reduce the dimensionality of the user visual feature vector matrix to obtain the reduced dimensionality user visual feature vector matrix; in this embodiment, the above-mentioned user preference determination module 604 is also used to determine the target user's preference for different visual categories based on the reduced dimensionality user visual feature vector matrix.

[0097] It should be noted that the user visual feature dimensionality reduction module 610 and the user preference determination module 604 described above correspond to S301-S302 in the method embodiment. The examples and application scenarios implemented by these modules and corresponding steps are the same, but are not limited to the contents disclosed in the method embodiment described above. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system, such as a set of computer-executable instructions.

[0098] In one embodiment, in the information recommendation device based on multi-classification images provided in an embodiment of the present invention, the object visual feature vector determination module 603 is further used to: adopt a mean pooling algorithm to determine the visual feature vector of the target object in each visual category based on the image feature vectors of multiple object images of the target object in each visual category.

[0099] Based on the same inventive concept, an embodiment of the present invention further provides a computer device to solve the technical problem that the visual auxiliary information used in the existing information recommendation system is the object visual information represented by a single visual feature vector, resulting in incomplete expression of the object visual information. Figure 9 FIG. 1 is a schematic diagram of a computer device according to an embodiment of the present invention, Figure 9 As shown, the computer device 90 includes a memory 901, a processor 902, and a computer program stored in the memory 901 and executable on the processor 902. When the processor 902 executes the computer program, the above-mentioned information recommendation method based on multi-classification images is implemented.

[0100] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium to solve the technical problem that the visual auxiliary information used in the existing information recommendation system is the object visual information represented by a single visual feature vector, and the expression of the object visual information is incomplete. The computer-readable storage medium stores a computer program for executing the above-mentioned information recommendation method based on multi-classification images.

[0101] In summary, the information recommendation method, apparatus, computer device, and computer-readable storage medium based on multi-classification images provided in the embodiments of the present invention, after collecting multiple object images of the target object in multiple visual categories, input each collected object image into a pre-trained image feature vector extraction model, output the image feature vector corresponding to each image, and then determine the visual feature vector of the target object in each visual category based on the image feature vectors of the multiple object images of the target object in each visual category. Combined with the target user's preference for different visual categories, the target user's preference for each visual category of the target object is determined. Finally, based on the target user's preference for each visual category of the target object, the information to be recommended to the target user is determined.

[0102] Compared with the information recommendation scheme in the prior art that only uses a single visual feature vector, the embodiment of the present invention takes into account the different visual preferences of users for objects from different perspectives and can effectively analyze the user's interest in the different perspectives of the object.

[0103] The information recommendation method based on multi-classification images provided in the embodiments of the present invention can achieve but is not limited to the following technical effects: ① The data set uses implicit feedback information. Compared with explicit feedback data (such as user ratings), implicit feedback data has a large data volume in actual production, a denser and more stable interaction matrix, and implicit feedback is naturally generated by users, which naturally has a more comprehensive and effective expression of user information; ② By training users' visual preferences for different perspectives and sides through adaptive parameters learned together in the model, compared with the existing method of single-perspective visual information mining, this method can effectively analyze the user's interest in different perspectives and sides of an object; ③ Using a pre-trained deep neural network model to extract the visual feature vector of the image instead of using an end-to-end deep neural model to directly perform data mining, it effectively utilizes the deep neural network model to extract rich feature information and avoids the problem of deep neural network models generally taking a long time. Using a Bayesian optimized matrix decomposition model, this model has the advantages of fast modeling speed and high data mining efficiency. Compared with various end-to-end deep neural network model algorithms, it can effectively process large amounts of data in a short time.

[0104] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0106] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0108] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An information recommendation method based on multi-classification images, characterized in that: include: Collect multiple images of the target object under multiple visual categories, where each visual category corresponds to a side of the target object; Input each collected object image into a pre-trained image feature vector extraction model, and output the image feature vector corresponding to each image; Determining, based on the image feature vectors of the plurality of object images of the target object in each visual category, the visual feature vector of the target object in each visual category as visual auxiliary information to analyze the user's visual feature preference; Collecting user-item interaction data from multiple users on multiple items, using implicit user feedback data; generating a user-item interaction matrix based on the collected user-item interaction data; decomposing the user-item interaction matrix based on a Bayesian optimized matrix decomposition model to obtain a user visual feature vector matrix and an item visual feature vector matrix; and determining the target user's preference for different visual categories based on the user visual feature vector matrix and the item visual feature vector matrix; Determining the target user's preference for each visual category of the target item based on the visual feature vector of the target item in each visual category and the target user's preference for different visual categories; According to the target user's preference for each visual category of the target item, the information to be recommended to the target user is determined, so as to achieve recommendation based on the user's visual preference for different aspects of the item.

2. The method according to claim 1, wherein Before determining the target user's preference for different visual categories based on the user visual feature vector matrix, the method further includes: Using the embedding matrix, the user visual feature vector matrix is ​​reduced in dimension to obtain the reduced-dimensional user visual feature vector matrix; The step of determining the target user's preference for different visual categories based on the user's visual feature vector matrix includes determining the target user's preference for different visual categories based on the user's visual feature vector matrix after dimensionality reduction.

3. The method according to claim 1, wherein Determining a visual feature vector of the target object in each visual category based on the image feature vectors of the plurality of object images of the target object in each visual category includes: The mean pooling algorithm is used to determine the visual feature vector of the target object in each visual category based on the image feature vectors of multiple object images of the target object in each visual category.

4. An information recommendation device based on multi-classification images, characterized in that: include: An image acquisition module, configured to acquire multiple images of a target object in multiple visual categories, wherein each visual category corresponds to a side of the target object; The image feature vector extraction module is used to input each collected object image into a pre-trained image feature vector extraction model and output the image feature vector corresponding to each image; an item visual feature vector determination module, configured to determine, based on the image feature vectors of multiple item images of the target item in each visual category, a visual feature vector of the target item in each visual category as visual auxiliary information, so as to analyze the user's visual feature preference; A user-item interaction data collection module is used to collect user-item interaction data from multiple users on multiple items, using user implicit feedback data; a user interaction matrix generation module is used to generate a user-item interaction matrix based on the collected user-item interaction data; a matrix decomposition module is used to decompose the user-item interaction matrix based on a Bayesian optimized matrix decomposition model to obtain a user visual feature vector matrix and an item visual feature vector matrix; a user visual feature analysis module is used to determine the target user's preference for different visual categories based on the user visual feature vector matrix and the item visual feature vector matrix; A user preference determination module is configured to determine the target user's preference for each visual category of the target item based on the visual feature vector of the target item in each visual category and the target user's preference for different visual categories; The information recommendation module is used to determine the information to be recommended to the target user based on the target user's preference for each visual category of the target item, so as to make recommendations based on the user's visual preferences for different aspects of the item.

5. The device according to claim 4, characterized in that The device further comprises: A user visual feature dimensionality reduction module is used to reduce the dimensionality of the user visual feature vector matrix using the embedding matrix to obtain a user visual feature vector matrix after dimensionality reduction; The user preference determination module is further configured to determine the target user's preference for different visual categories based on the user visual feature vector matrix after dimensionality reduction.

6. The device according to claim 4, characterized in that The object visual feature vector determination module is further configured to: The mean pooling algorithm is used to determine the visual feature vector of the target object in each visual category based on the image feature vectors of multiple object images of the target object in each visual category.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the information recommendation method based on multi-classification images according to any one of claims 1 to 3 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for executing the information recommendation method based on multi-classification images according to any one of claims 1 to 3.

Images