Media object recommendation method, device, computer equipment and storage medium

Abstract

The present application relates to a media object recommendation method, apparatus, computer equipment, and storage medium. The method includes: determining a media object graph to which a reference media object belongs as a first media object graph; the media object graph is pre-clustered based on feature information of each media object to obtain multiple clusters, and is constructed based on each media object corresponding to the same cluster; determining graph similarity between the first media object graph and each second media object graph; the second media object graph is a media object graph other than the first media object graph; determining a target media object graph from each second media object graph based on the graph similarity; determining a first target media object to be recommended from the target media object graph, and recommending the first target media object. Using the present application, it is possible to improve the accuracy of media object recommendations.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for recommending media objects. Background Art

[0002] With the rapid development of digital media technology and Internet applications, as well as the popularization of digital media devices such as digital cameras and smart phones, people can easily generate and share various types of media objects, including photos, videos, and audio, resulting in an explosive growth in the data volume of these media objects. Finding the desired media objects has become a thorny problem.

[0003] Traditional methods typically require the uploader or platform administrator to manually add descriptive information and tags to each media object. These descriptive information and tags are then used to recommend media objects to users. However, the limited information manually added to media objects cannot accurately represent their characteristics, leading to inaccurate media object recommendations. Summary of the Invention

[0004] Based on this, it is necessary to provide a media object recommendation method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of media object recommendation.

[0005] In a first aspect, the present application provides a media object recommendation method, comprising:

[0006] Determining a media object graph to which the reference media object belongs as a first media object graph; the media object graph is constructed by clustering the media objects into a plurality of clusters based on feature information of the media objects, and then constructing the media objects corresponding to the same cluster;

[0007] Determining graph similarities between the first media object graph and each second media object graph; the second media object graph is a media object graph other than the first media object graph;

[0008] Determining a target media object graph from each of the second media object graphs based on the graph similarity;

[0009] A first target media object to be recommended is determined from the target media object graph, and the first target media object is recommended.

[0010] In a second aspect, the present application provides a media object recommendation device, comprising:

[0011] A search module is configured to determine a media object graph to which a reference media object belongs as a first media object graph; the media object graph is pre-clustered based on characteristic information of each media object to obtain a plurality of clusters, and is constructed based on each media object corresponding to the same cluster;

[0012] a calculation module, configured to determine graph similarities between the first media object graph and each second media object graph; wherein the second media object graph is a media object graph other than the first media object graph;

[0013] a determination module, configured to determine a target media object graph from each second media object graph based on graph similarity;

[0014] The recommendation module is configured to determine a first target media object to be recommended from the target media object graph and recommend the first target media object.

[0015] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above-mentioned method when executed by a processor.

[0017] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps in the above method.

[0018] The above-mentioned media object recommendation method, apparatus, computer device, computer-readable storage medium and computer program product, by clustering the characteristic information of each media object in advance to construct a media object graph, and then recommending media objects based on the graph similarity between media object graphs, can more accurately understand and classify media objects, and further more accurately determine the graph similarity between media object graphs, thereby improving the accuracy of media object recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A diagram illustrating an application environment of a media object recommendation method provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of a media object recommendation method provided in an embodiment of the present application;

[0021] Figure 3 A visualization diagram of a clustering result provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of feature extraction in the spatial and temporal domains provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a network hierarchical structure for feature extraction provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of a feature extraction result provided in an embodiment of the present application;

[0025] Figure 7 A structural block diagram of a media object recommendation device provided in an embodiment of the present application;

[0026] Figure 8 A structural block diagram of another media object recommendation device provided in an embodiment of the present application;

[0027] Figure 9 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0028] Figure 10 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0029] Figure 11 A diagram of the internal structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0031] The media object recommendation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the computer device 104 through a communication network. The data storage system can store data that the computer device 104 needs to process. The data storage system can be integrated on the computer device 104, or it can be placed on the cloud or other network computer devices. The terminal 102 can send the information of the reference media object to the computer device 104, and the computer device 104 can execute the media object recommendation method in each embodiment of the present application, determine the target media object to be recommended, and send the target media object to be recommended to the terminal 102 for recommendation. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The computer device 104 can be implemented using an independent server or a server cluster consisting of multiple servers, or can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc., and the portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0032] like Figure 2 As shown, the embodiment of the present application provides a media object recommendation method, which is applied to Figure 1 The terminal 102 or computer device 104 in the example is used for explanation. It is understood that the computer device may include at least one of a terminal and a server. The method includes the following steps:

[0033] S202. Determine a media object graph to which the reference media object belongs as a first media object graph; the media object graph is pre-clustered according to feature information of each media object to obtain multiple clusters, and is constructed according to each media object corresponding to the same cluster.

[0034] A media object is a multimedia object. Multimedia is a combination of various media types, including text, sound, and images. Reference media objects are used as a reference to determine target media objects for recommendation. A media object graph is a data structure consisting of nodes and edges, constructed based on multiple media objects corresponding to the same cluster.

[0035] In some embodiments, the media object may include at least one of an image, a video, an audio, and the like.

[0036] In some embodiments, the reference media object may be a media object with which the user has interacted. The interaction may include at least one of searching, viewing, liking, commenting, adding to favorites, and stopping. The interaction may be currently ongoing or has occurred in the past.

[0037] In some embodiments, the terminal used by the user can send information of the reference media object to the computer device, and the computer device can search for the media object graph to which the reference media object belongs from multiple pre-built media object graphs as the first media object graph.

[0038] S204: Determine graph similarities between the first media object graph and each second media object graph; the second media object graph is a media object graph other than the first media object graph.

[0039] Among them, graph similarity is used to measure the similarity between two media object graphs.

[0040] In some embodiments, the computer device may represent the first media object graph and each second media object graph as an undirected graph, calculate the similarity between the undirected graph of the first media object graph and the undirected graph of each second media object graph, and obtain graph similarity.

[0041] S206: Determine a target media object graph from each second media object graph based on graph similarity.

[0042] In some embodiments, the number of target media object graphs can be one or more.

[0043] In some embodiments, the computer device may determine the target media object graph from each second media object graph based on the degree of graph similarity.

[0044] S208: Determine a first target media object to be recommended from the target media object graph, and recommend the first target media object.

[0045] In some embodiments, the computer device may further determine a second target media object to be recommended from the first media object graph and recommend the second target media object, that is, recommend a media object from the media object graph to which the reference media object belongs.

[0046] The first target media object refers to a target media object to be recommended determined from the target media object graph, and the second target media object refers to a target media object to be recommended determined from the first media object graph.

[0047] In some embodiments, the first target media object to be recommended may be all or part of the target media object graph.The second target media object to be recommended may be all or part of the first media object graph.

[0048] In some embodiments, the number of the first target media objects or the second target media objects to be recommended may be one or more.

[0049] In some embodiments, the computer device may select a first preset number of media objects from the target media object graph as first target media objects to be recommended. The first preset number may be one or more. The computer device may sort the media objects in the target media object graph in descending order of distance from the cluster center within the cluster, and select the first preset number of media objects ranked first as the first target media objects to be recommended.

[0050] In some embodiments, the computer device may select a second preset number of media objects from the first media object graph as second target media objects to be recommended. The second preset number may be one or more. The computer device may select the second preset number of media objects as second target media objects to be recommended based on similarities between each media object in the first media object graph and the reference media object.

[0051] In some embodiments, after determining the first target media object or the second target media object to be recommended, the computer device may send information about the first target media object or the second target media object to be recommended to the terminal, and the terminal may display the information about the first target media object or the second target media object to be recommended to recommend the first target media object or the second target media object.

[0052] In some embodiments, the terminal may directly play the first target media object or the second target media object. In other embodiments, the terminal may display preview information or introduction information of the first target media object or the second target media object, and play the first target media object or the second target media object after receiving a triggering operation for the displayed preview information or introduction information.

[0053] As can be seen, in the embodiments of the present application, by pre-clustering the media objects based on their feature information to construct a media object graph, and then recommending media objects based on the graph similarity between the media object graphs, media objects can be more accurately understood and classified, and the graph similarity between the media object graphs can be more accurately determined, thereby improving the accuracy of media object recommendations. In addition, by calculating graph similarity to recommend media objects, the computational complexity is reduced, thereby improving recommendation efficiency.

[0054] In some embodiments, determining graph similarity between the first media object graph and each second media object graph includes:

[0055] Determine the Laplacian matrices corresponding to the first media object graph and each second media object graph;

[0056] Determine a plurality of eigenvectors corresponding to each Laplace matrix;

[0057] For each second media object graph, the graph similarity between the first media object graph and the second media object graph is determined based on the similarity between each feature vector corresponding to the second media object graph and each feature vector corresponding to the first media object graph.

[0058] Among them, the Laplace matrix is ​​a matrix representation of the graph.

[0059] In some embodiments, the computer device may determine the adjacency matrix and degree matrix of the undirected graphs corresponding to the first media object graph and each second media object graph, and then determine the Laplacian matrices corresponding to the first media object graph and each second media object graph based on the difference between the degree matrix and the adjacency matrix. For example, if the undirected graphs corresponding to the first media object graph and the second media object graph are G1 and G2, respectively, and the corresponding adjacency matrices are A1 and A2, respectively, and the corresponding degree matrices are D1 and D2, respectively, then the Laplacian matrices corresponding to the first media object graph and the second media object graph are L1 = D1 - A1 and L2 = D2 - A2, respectively.

[0060] In some embodiments, the computer device may calculate multiple eigenvalues ​​corresponding to the Laplacian matrix and the eigenvectors corresponding to each eigenvalue. The relationship between the Laplacian matrix, the eigenvalues, and the eigenvectors may be expressed as follows:

[0061] L1v i =λ i v i ;(1)

[0062] L2u j =μ j u j ;(2)

[0063] Wherein, L1 in formula (1) represents the Laplacian matrix corresponding to the first media object graph, λ i represents the i-th eigenvalue corresponding to L1, v i In formula (2), L2 represents the Laplacian matrix corresponding to the second media object graph, μ j represents the jth eigenvalue corresponding to L2, u jrepresents the j-th eigenvector corresponding to L2.

[0064] In some embodiments, the similarity between each feature vector corresponding to the second media object graph and each feature vector corresponding to the first media object graph may be a cosine similarity.

[0065] In some embodiments, the computer device can generate the feature vector matrices corresponding to the first media object graph and the second media object graph respectively based on the feature vectors corresponding to the first media object graph and the second media object graph. That is, the feature vector matrix corresponding to the first media object graph is V = [v1, v2, ..., v n ], the eigenvector matrix corresponding to the second media object graph is U=[u1,u2,……,u n ], where n represents the number of eigenvectors. The computer device may calculate the cosine similarity between each eigenvector in the eigenvector matrices corresponding to the first media object graph and the second media object graph, based on the eigenvector matrices corresponding to the first media object graph and the second media object graph, respectively.

[0066] In some embodiments, the computer device may calculate the cosine similarity according to the following formula:

[0067]

[0068] Among them, V i Represents one of the eigenvectors in the eigenvector matrix corresponding to the first media object graph, that is, one of the eigenvectors corresponding to the first media object graph. j Represents one of the eigenvectors in the eigenvector matrix corresponding to the second media object graph, that is, one of the eigenvectors corresponding to the second media object graph. ||V i || and ||U j || represent V i The module and U j The model. ij Indicates V i and U j The cosine similarity between .

[0069] In some embodiments, the computer device may determine the graph similarity between the first media object graph and the second media object graph based on the average of the similarities between each feature vector corresponding to the second media object graph and each feature vector corresponding to the first media object graph. The formula is as follows:

[0070]

[0071] Wherein, sim(G1, G2) represents the graph similarity between the first media object graph G1 and the second media object graph G2. ij Represents the cosine similarity between the feature vector corresponding to the first media object graph and the feature vector corresponding to the second media object graph. n represents the total number of feature vectors.

[0072] It can be seen that in this embodiment, by calculating the Laplace matrices corresponding to the first media object graph and each second media object graph, and then determining the multiple eigenvectors corresponding to each Laplace matrix, and calculating the cosine similarity between each eigenvector, the graph similarity between the first media object graph and the second media object graph can be accurately calculated.

[0073] In some embodiments, determining a target media object graph from each second media object graph based on graph similarity includes:

[0074] Arrange the second media object graphs in descending order according to their corresponding graph similarities;

[0075] From each second media object graph, a preset number of second media object graphs are selected as target media object graphs.

[0076] In some embodiments, the preset number may be one or more.

[0077] In some embodiments, the computer device may further arrange each second media object graph in ascending order according to the corresponding graph similarity, and select a preset number of second media object graphs from each second media object graph as the target media object graph.

[0078] It can be seen that in this embodiment, by sorting the second media object graphs according to the size of the graph similarity, and then selecting a preset number of second media object graphs based on the sorting results, it is possible to accurately determine the second media object graph that is more similar to the first media object graph, thereby recommending media objects in the second media object graph, thereby improving the accuracy of media object recommendation.

[0079] In some embodiments, the first media object graph and any second media object graph each include nodes and edges; the nodes are connected by edges; the nodes are used to represent media objects included in the media object graph; the edges are used to represent similarities between the media objects included in the media object graph; after determining the media object graph to which the reference media object belongs as the first media object graph, the method further includes:

[0080] From the first media object graph, based on the similarity represented by the edge connected to the node of the reference media object, a second target media object to be recommended is determined from the media objects corresponding to each node connected to the node of the reference media object, and the second target media object is recommended.

[0081] In some embodiments, the computer device may determine the similarity between each media object based on feature information of each media object contained in the same media object graph in advance, and then assign values ​​to the edges between the nodes of each media object contained in the media object graph based on the similarity.

[0082] In some embodiments, the edges in the media object graph can also be used to represent the categories of the media objects contained in the media object graph. For example, if the category of a media object is sports, the edge connected to the node of the media object can be assigned the value of sports.

[0083] In some embodiments, the computer device may sort the media objects corresponding to the nodes connected to the node of the reference media object in descending order of similarity based on the similarity represented by the edges connected to the nodes of the reference media object, select the media objects that are ranked second to a preset number as the second target media objects to be recommended, and recommend the second target media objects.

[0084] It can be seen that in this embodiment, from the first media object graph, based on the similarity represented by the edge connected to the node of the reference media object, the second target media object to be recommended is determined from the media objects corresponding to each node connected to the node of the reference media object, and the second target media object is recommended. This can accurately recommend more similar media objects from the first media object graph to which the reference media object belongs, thereby improving the accuracy of media object recommendation.

[0085] In some embodiments, the reference media object is a media object with which the user has interacted; and recommending the first target media object includes:

[0086] The first target media object is recommended to a terminal corresponding to a user who interacts with the reference media object.

[0087] In some embodiments, the computer device can send information about the first target media object or the second target media object to a terminal corresponding to a user who has interacted with the reference media object, and the terminal corresponding to the user who has interacted with the reference media object can display the information about the first target media object or the second target media object to recommend the media object.

[0088] It can be seen that in this embodiment, based on the reference media object with which the user has interacted, the target media object to be recommended is determined, and then the target media object is recommended to the terminal corresponding to the user with whom the reference media object has interacted. This makes it possible to accurately recommend media objects to the user based on the user's interactive behavior.

[0089] In some embodiments, before determining the media object graph to which the reference media object belongs as the first media object graph, the method further includes:

[0090] Extract features of each media object in advance to obtain feature information of each media object;

[0091] Clustering is performed based on feature information to divide each media object into multiple clusters;

[0092] A media object graph is constructed based on the media objects corresponding to the same cluster.

[0093] In some embodiments, the feature information may be a feature vector or a feature matrix.

[0094] In some embodiments, when the media object is a video, the computer device can perform feature extraction on the image, audio, and text information in the video to obtain multimodal feature information of the video. By integrating multimodal feature information, the computer device can fully understand the media object and improve the accuracy of recommendations.

[0095] In some embodiments, the computer device may perform clustering using a k-means algorithm.

[0096] In some embodiments, the computer device can determine an initial cluster center that meets the target number as the current cluster center, calculate the distance between each feature information and the current cluster center, divide each feature information into the cluster cluster corresponding to the current cluster center that is closest to it, and then determine the new current cluster center based on the average value of the feature information divided into the same cluster cluster, and return to the step of calculating the distance between each feature information and the current cluster center to iteratively perform clustering until the iteration stop condition is met to obtain the final multiple cluster clusters.

[0097] The target number is the preset number of clusters to be divided.

[0098] In some embodiments, the computer device may randomly determine a target number of initial cluster centers within the data range to which each feature information belongs.

[0099] In some embodiments, the iteration stopping condition may be that the cluster center no longer changes or the number of iterations is greater than or equal to a preset threshold.

[0100] For example, if there is a dataset containing feature vectors of 10 media objects, each feature vector contains 5 dimensions, the dataset can be represented as a 10×5 matrix:

[0101] [2,3,4,1,5]

[0102] [1,4,3,7,2]

[0103] [3,2,5,6,1]

[0104] [4,1,2,5,8]

[0105] [2,2,3,4,9]

[0106] [1,3,4,6,8]

[0107] [7,4,3,1,5]

[0108] [8,9,2,3,1]

[0109] [5,6,1,2,4]

[0110] [2,5,6,7,1]

[0111] Assuming the target number is 3, three initial cluster centers can be randomly generated. For example, the three initial cluster centers can be:

[0112] [2,4,5,1,8]

[0113] [7,3,2,6,4]

[0114] [1,5,3,7,9]

[0115] If the iteration is stopped after 5 iterations, the final 3 cluster centers are:

[0116] [1.5,3.5,4,1.5,7.5]

[0117] [5.5,4,2.5,5.25,4]

[0118] [2.6667,4.6667,3.3333,5,4.6667]

[0119] Therefore, the clusters corresponding to the final three cluster centers are the three clusters finally divided.

[0120] like Figure 3 Figure 1 shows a schematic diagram of a clustering result visualization. The five-pointed star in the figure represents the final cluster center, and the circles with the same texture represent the features that are grouped into the same cluster. It can be seen that features close to the same cluster center are grouped into the cluster corresponding to the same cluster center.

[0121] In some embodiments, a computer device may construct a media object image based on media objects corresponding to feature information that are classified into the same cluster, use each media object in the cluster as a node in a media object graph, and use the similarity between each media object as the value of the edge connecting the nodes in the media object graph.

[0122] In some embodiments, clustering can be performed in parallel using multiple computing devices, thereby improving clustering efficiency. The amount of data processed by each computing device can be balanced to achieve load balancing. Processing results from a computing device can be cached in other computing devices for subsequent processing, achieving efficient data utilization and enhancing the scalability of the solution.

[0123] It can be seen that in this embodiment, feature extraction is performed on each media object in advance to obtain feature information of each media object, and clustering is performed based on the feature information to divide each media object into multiple clusters, so that a media object graph can be accurately constructed based on each media object corresponding to the same cluster, making the media objects in the same media object graph more similar.

[0124] In some embodiments, after constructing a media object graph based on the media objects corresponding to the same cluster, the method further includes:

[0125] After obtaining user feedback information for any media object, updating feature information of the media object according to the user feedback information for the media object;

[0126] Re-determine the target cluster to which the media object belongs based on the updated feature information of the media object;

[0127] Add the media object to the media object graph corresponding to the target cluster.

[0128] The user feedback information is information related to the feedback made by the user on any media object.

[0129] In some embodiments, user feedback information may include at least one of user interaction information and user evaluation information. User interaction information refers to the user's interaction with the media object. User evaluation information refers to the user's evaluation of the media object.

[0130] In some embodiments, the user's interaction situation may include at least one of operations such as searching, viewing, liking, commenting, collecting, and staying.

[0131] In some embodiments, the terminal may obtain user feedback information regarding the media object and send the user feedback information to the computer device. The computer device may update feature information of the media object based on the user feedback information regarding the media object.

[0132] In some embodiments, after updating the feature information of a media object, the computer device may determine the distance between the updated feature information and the cluster centers of each cluster based on the updated feature information of the media object, and assign the media object to the target cluster corresponding to the cluster center with the smallest distance. The media object is then added to the media object graph corresponding to the target cluster to update the media object graph corresponding to the target cluster.

[0133] It can be seen that in this embodiment, the media object graph can be updated and optimized based on user feedback information, further improving the accuracy of media object recommendations, and improving the timeliness and effectiveness of the media object graph, and enhancing the scalability and robustness of the solution.

[0134] In some embodiments, feature extraction is performed on each media object in advance to obtain feature information of each media object, including:

[0135] Convert each media object in the spatial domain to the temporal domain;

[0136] Convolution and pooling are performed on each media object in the spatial domain and temporal domain respectively to obtain the spatial domain features and temporal domain features of each media object;

[0137] The spatial domain features and temporal domain features of each media object are fused to obtain the feature information of each media object.

[0138] In some embodiments, the computer device may use a three-dimensional convolution kernel (3D CNN) to perform convolution processing and pooling processing on each media object.

[0139] In some embodiments, when the media object is a video, the computer device may perform convolution processing and pooling processing on the video frame sequence in the spatial domain and the temporal domain respectively to obtain the spatial domain features and temporal domain features of each media object. Figure 4 As shown in the figure, the video frame sequence is convolved and pooled in the spatial domain and temporal domain respectively, and the spatial domain features and temporal domain features are output respectively.

[0140] In some embodiments, processing in the spatial and temporal domains may be performed in parallel.

[0141] In some embodiments, the computer device may divide the video into multiple video frame sequences, each video frame sequence comprising multiple video frames. The computer device may perform convolution and pooling processing on each video frame sequence to obtain feature information corresponding to each video frame sequence, that is, to obtain multiple sets of feature information for each video. In some embodiments, adjacent video frame sequences may overlap. For example, a video may be divided into multiple video frame sequences comprising 16 frames, with adjacent video frame sequences overlapping by 8 frames.

[0142] The network with convolution and pooling can be Figure 5 As shown in the figure, for example: the dimension of the input video frame sequence is 3×16×128×171, the size of the convolution kernel of the pooling layer is d×k×k, the d of the first pooling layer is 1 to ensure that the time domain information is not fused prematurely, the d of the subsequent pooling layer is 2, and the convolution kernel size of the convolution layer is 3×3×3. Figure 5 The convolutional layer and pooling layer in the

[15] perform feature extraction layer by layer, and finally obtain a 4096-dimensional feature vector as the feature information of the input video frame sequence. Figure 6 The features output by the last convolutional layer are visualized in Figure 3. It can be seen that the features at the beginning emphasize the image information in the video frame, and the features at the end emphasize the motion information in the video frame.

[0143] It can be seen that in this embodiment, convolution processing and pooling processing are performed on each media object in the spatial domain and time domain respectively, which can avoid the loss of time domain information after convolution processing and pooling processing, increase the amount of information in the feature information, and improve the accuracy of the feature information.

[0144] In some embodiments, when the media object is a video, the computer device can pre-extract features from each video to obtain feature information of each video, and then cluster the videos based on the feature information to divide each video into multiple clusters, and then construct a video graph based on the videos belonging to the same cluster. When recommending a video, the computer device can determine the video graph to which the reference video belongs as the first video graph, and then determine the graph similarity between the first video graph and each second video graph. The second video graph is a video graph other than the first video graph. The computer device can determine the target video graph from each second video graph based on the graph similarity, and then determine the first target video to be recommended from the target video graph, and recommend the target video to the terminal corresponding to the user who has interacted with the reference video.

[0145] It should be understood that, although the steps in the flowcharts of the above-mentioned embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0146] Based on the same inventive concept, embodiments of the present application also provide a media object recommendation device. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more media object recommendation device embodiments provided below can be found in the above-mentioned limitations of the media object recommendation method and will not be further elaborated here.

[0147] like Figure 7 As shown, an embodiment of the present application provides a media object recommendation device 700, comprising:

[0148] Search module 702 is used to determine the media object graph to which the reference media object belongs as a first media object graph; the media object graph is pre-clustered based on the characteristic information of each media object to obtain multiple clusters, and is constructed based on each media object corresponding to the same cluster;

[0149] A calculation module 704 is configured to determine graph similarities between the first media object graph and each second media object graph; the second media object graph is a media object graph other than the first media object graph;

[0150] A determination module 705 is configured to determine a target media object graph from each second media object graph based on graph similarity;

[0151] The recommendation module 706 is configured to determine a first target media object to be recommended from the target media object graph and recommend the first target media object.

[0152] In some embodiments, in determining the graph similarity between the first media object graph and each second media object graph, the calculation module 704 is specifically configured to:

[0153] Determine the Laplacian matrices corresponding to the first media object graph and each second media object graph;

[0154] Determine a plurality of eigenvectors corresponding to each Laplace matrix;

[0155] For each second media object graph, the graph similarity between the first media object graph and the second media object graph is determined based on the similarity between each feature vector corresponding to the second media object graph and each feature vector corresponding to the first media object graph.

[0156] In some embodiments, in determining the target media object graph from each second media object graph based on graph similarity, the determination module 705 is specifically configured to:

[0157] Arrange the second media object graphs in descending order according to their corresponding graph similarities;

[0158] From each second media object graph, a preset number of second media object graphs are selected as target media object graphs.

[0159] In some embodiments, the first media object graph and any second media object graph each include nodes and edges; the nodes are connected by edges; the nodes are used to represent the media objects included in the media object graph; the edges are used to represent the similarity between the media objects included in the media object graph;

[0160] The recommendation module 706 is also used to: from the first media object graph, based on the similarity represented by the edge connected to the node of the reference media object, determine the second target media object to be recommended from the media objects corresponding to each node connected to the node of the reference media object, and recommend the second target media object.

[0161] In some embodiments, the reference media object is a media object with which the user has interacted; in terms of recommending the first target media object, the recommendation module 706 is specifically configured to:

[0162] The first target media object is recommended to a terminal corresponding to a user who interacts with the reference media object.

[0163] In some embodiments, as Figure 8 As shown, the apparatus 700 further includes:

[0164] The construction module 708 is used to extract features of each media object in advance to obtain feature information of each media object; cluster the media objects according to the feature information to divide the media objects into multiple clusters; and construct a media object map based on the media objects corresponding to the same cluster.

[0165] In some embodiments, the construction module 708 is further configured to: after obtaining user feedback information for any media object, update the feature information of the media object according to the user feedback information for the media object;

[0166] Re-determine the target cluster to which the media object belongs based on the updated feature information of the media object;

[0167] Add the media object to the media object graph corresponding to the target cluster.

[0168] In some embodiments, in pre-extracting features from each media object to obtain feature information of each media object, the construction module 708 is specifically configured to:

[0169] Convert each media object in the spatial domain to the temporal domain;

[0170] Convolution and pooling are performed on each media object in the spatial domain and temporal domain respectively to obtain the spatial domain features and temporal domain features of each media object;

[0171] The spatial domain features and temporal domain features of each media object are fused to obtain the feature information of each media object.

[0172] Each module in the aforementioned media object recommendation device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0173] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store media object maps. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned media object recommendation method are implemented.

[0174] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned media object recommendation method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0175] Those skilled in the art will understand that Figure 9 or Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0176] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0177] In some embodiments, as Figure 11 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0178] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0180] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0181] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A media object recommendation method, characterized in that: include: Determining a media object graph to which the reference media object belongs as a first media object graph; the media object graph is pre-clustered according to feature information of each media object to obtain a plurality of clusters, and is constructed based on each media object corresponding to the same cluster; Determining graph similarities between the first media object graph and each second media object graph; The second media object graph is a media object graph other than the first media object graph; determining a target media object graph from each of the second media object graphs according to the graph similarity; A first target media object to be recommended is determined from the target media object graph, and the first target media object is recommended.

2. The method according to claim 1, characterized in that The determining of the graph similarity between the first media object graph and each second media object graph includes: Determining Laplacian matrices corresponding to the first media object graph and each second media object graph; Determine a plurality of eigenvectors corresponding to each of the Laplace matrices; For each of the second media object graphs, the graph similarity between the first media object graph and the second media object graph is determined based on the similarity between each feature vector corresponding to the second media object graph and each feature vector corresponding to the first media object graph.

3. The method according to claim 1, characterized in that Determining a target media object graph from each of the second media object graphs according to the graph similarity includes: Arrange the second media object graphs in descending order of their corresponding graph similarities; From each of the second media object graphs, a preset number of second media object graphs are selected as target media object graphs.

4. The method according to claim 1, wherein The first media object graph and any one of the second media object graphs each include nodes and edges; the nodes are connected by edges; the nodes are used to represent media objects included in the media object graph; the edges are used to represent similarities between media objects included in the media object graph; After determining the media object graph to which the reference media object belongs as the first media object graph, the method further includes: From the first media object graph, based on the similarity represented by the edge connected to the node of the reference media object, the second target media object to be recommended is determined from the media objects corresponding to each node connected to the node of the reference media object, and the second target media object is recommended.

5. The method according to claim 1, wherein The reference media object is a media object with which the user interacts; and the recommending the first target media object includes: The first target media object is recommended to a terminal corresponding to a user who has interacted with the reference media object.

6. The method according to any one of claims 1 to 5, characterized in that Before determining the media object graph to which the reference media object belongs as the first media object graph, the method further includes: Extracting features of each media object in advance to obtain feature information of each media object; performing clustering according to the feature information to divide each of the media objects into a plurality of clusters; A media object graph is constructed based on the media objects corresponding to the same cluster.

7. The method according to claim 6, characterized in that After constructing the media object graph based on the media objects corresponding to the same cluster, the method further includes: After obtaining user feedback information for any of the media objects, updating feature information of the media object according to the user feedback information for the media object; Re-determining the target cluster to which the media object belongs based on the updated feature information of the media object; Add the media object to the media object graph corresponding to the target cluster.

8. The method according to claim 6, characterized in that The step of extracting features from each media object in advance to obtain feature information of each media object includes: Convert each media object in the spatial domain to the temporal domain; Performing convolution processing and pooling processing on each media object in the spatial domain and the temporal domain respectively to obtain spatial domain features and temporal domain features of each media object; The spatial domain features and the temporal domain features of each of the media objects are fused to obtain feature information of each of the media objects.

9. A media object recommendation device, characterized in that: include: A search module is configured to determine a media object graph to which a reference media object belongs as a first media object graph; the media object graph is pre-clustered according to feature information of each media object to obtain a plurality of clusters, and is constructed based on each media object corresponding to the same cluster; a calculation module, configured to determine graph similarities between the first media object graph and each second media object graph; The second media object graph is a media object graph other than the first media object graph; a determination module, configured to determine a target media object graph from each of the second media object graphs according to the graph similarity; The recommendation module is configured to determine a first target media object to be recommended from the target media object graph and recommend the first target media object.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images