Short video personalized recommendation method and system based on multi-modal graph convolutional network

A technology of convolutional network and recommendation method, which is applied in the field of short video personalized recommendation based on multimodal graph convolutional network, can solve the problem of ineffective and poor accuracy of short video personalized recommendation, inability to express multimodal content information, and user Preference expression deviation and other issues to achieve the effect of improving accuracy and comprehensibility, accurate and effective personalized recommendation

Active Publication Date: 2019-10-15
SHANDONG UNIV
View PDF10 Cites 17 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Indiscriminate direct modeling of multi-modal information will lead to deviations in the representation of user preferences. Therefore, how to represent the user's modality-specific preferences is the first reuse challenge in short video personalized recommendation.
2. Multimodal content complicates the relationship between users and the information of short videos
[0005] The inventors found that the curre...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Short video personalized recommendation method and system based on multi-modal graph convolutional network
  • Short video personalized recommendation method and system based on multi-modal graph convolutional network
  • Short video personalized recommendation method and system based on multi-modal graph convolutional network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0043] figure 1 The flow chart of the short video personalized recommendation method based on the multi-modal graph convolutional network of this embodiment is given.

[0044] Such as figure 1 As shown, the short video personalized recommendation method based on the multimodal graph convolutional network in this embodiment includes:

[0045]S101: Build a user-short video graph structure based on the image mode, audio mode, and text mode of the short video; the points in the user-short video graph structure represent the user and the short video, and the lines between the points represent both interaction between.

[0046] In the specific implementation, data sets are randomly extracted from the three platforms of Douyin, Kuaishou and Movielens, including users, short videos and users' playback history.

[0047] According to the user's playback history, on each modality m (including image v, audio a and text t), namely Build the graph structure separately Points in the g...

Embodiment 2

[0094] figure 2 A schematic structural diagram of the short video personalized recommendation system based on the multi-modal graph convolutional network of this embodiment is given.

[0095] Such as figure 2 As shown, the short video personalized recommendation system based on the multimodal graph convolutional network of this embodiment includes:

[0096] (1) User-short video graph structure building module, which is used to construct the user-short video graph structure based on the image mode, audio mode and text mode of the short video respectively; the point in the user-short video graph structure represents the user and short videos, the connection between the dots indicates the interaction between the two;

[0097] (2) User and short video expression module, which is used to input the user-short video graph structure of each mode into the corresponding graph convolutional neural network, and calculate and express each modality through the aggregation layer of the c...

Embodiment 3

[0105] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following figure 1 The steps in the short video personalized recommendation method based on multimodal graph convolutional network are shown.

[0106] In this embodiment, through the construction of a multimodal graph convolutional neural network, the user's preferences in different modalities are modeled, and at the same time, the multimodal content information is used to express the user's preferences. On this basis, different modalities According to the fusion of user preferences, the interest expression of users is obtained, and the relationship between user preferences and short video content information is calculated to provide users with more accurate short video personalized recommendations.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a short video personalized recommendation method and system based on a multi-modal graph convolutional network. The short video personalized recommendation method based on the multi-modal graph convolutional network comprises the following steps: respectively constructing user-short video graph structure based on an image mode, an audio mode and a text mode of a short video;wherein points in the short video graph structure represent a user and a short video, and a connection line between the points represents interaction between the user and the short video; inputting the user-short video graph structure into a corresponding graph convolutional neural network, respectively calculating a user and a short video for expressing each mode through a polymerization layer of the corresponding graph convolutional neural network, and combining expressions of each mode of the user and the short video by utilizing a fusion layer of the corresponding graph convolutional neural network to obtain final expressions of the user and the short video; and using a Bayesian personalized sorting algorithm to sequentially recommend paired sequences of final expressions of the userand the short video.

Description

technical field [0001] The disclosure belongs to the field of short video personalized recommendation, and in particular relates to a short video personalized recommendation method and system based on a multi-modal graph convolutional network. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] Personalized recommendations have become a core component of many online content sharing services, especially short video platforms. Various short video applications, such as Vine, Instagram, Kuaishou, Douyin, Meipai, WeChat, Weibo, Tencent Weishi, etc., have developed rapidly in recent years. The short video is seamlessly connected to various social platforms on the Internet, so that it can be directly shared on social networks after shooting. Short video combines multiple modes of text, audio, and image, which can meet users' expression and communi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N21/258H04N21/25H04N21/466H04N21/45G06F16/735G06F16/738G06F16/78G06N3/04
CPCH04N21/25891H04N21/251H04N21/4666H04N21/4663H04N21/4668H04N21/4667H04N21/4532G06F16/735G06F16/738G06F16/78G06N3/045
Inventor 尉寅玮王翔聂礼强刘萌刘伟锋高赞刘威
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap