Video recommendation method based on multimodal video content and multi-task learning

Abstract

The invention discloses a video recommendation method based on multi-modal video content and multi-task learning, comprising the steps of: extracting visual, audio, and text features of short videos through a pre-trained model; State features are fused; the deep walk method is used to learn the feature representation of the user's social relationship; a deep neural network model based on the attention mechanism is proposed to learn multi-domain feature representation; the feature embedding generated based on the above steps is used as the shared layer of the multi-task model, and then The prediction results are generated by multi-layer perceptron respectively. The present invention uses the attention mechanism combined with user features to fuse multi-modal features of video, making the whole recommendation richer and more personalized; at the same time, considering the importance of interactive features in recommendation learning for multi-domain features, a The deep neural network model based on the attention mechanism enriches the learning of high-level features and provides users with more accurate personalized video recommendations.

Description

technical field [0001] The invention relates to the technical field of network video and recommendation systems, in particular to a video recommendation method based on multi-modal video content and multi-task learning. Background technique [0002] With the rapid popularization of smart mobile terminals and the development of multimedia technology, video has gradually become the carrier of information dissemination. In recent years, short videos have risen rapidly. Video has become a main way of entertainment for people, and users' interests have also shown wider. The rapid increase in the number of short videos has brought about a serious problem of information overload. How to find videos of interest to users from massive amounts of data has become a hot topic and research object. A good recommendation system can not only help consumers find interesting or even potentially interesting videos faster and more conveniently, but also help content providers increase profits an...

AI Technical Summary

Problems solved by technology

[0003] At present, the short video recommendation technology is facing two important challenges: (1) Most of the current recommendation algorithms are based on user preferences and user behavior to make recommendations, ignoring the content of items, and there is also a serious cold start problem, which leads to large Most videos are ignored, and even traditional content-based recommendation methods do not perform well because they rely on metadata rather than original video content

However, the metadata of micro-videos is uploaded by users, which may be inaccurate for videos, how to effectively utilize the multi-modal information of videos becomes an important challenge for video recommendation

(2) The single-task recommendation model cannot meet the current needs for multi-tasks. In video recommendation, it is not only necessary to predict whether the user will watch the video, but also predict the user's rating of the video, whether to like it, whether to forward it, etc.

Images