Video Recommendation Feature Fusion via Bilinear Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing attentive collaborative filtering (ACF) model for video recommendations faces challenges due to the significant property difference between user features and video features, leading to information loss and gradient vanishing issues, which affect the accuracy of video recommendations.
Innovation Solution
A method involving feature extraction using separate networks for video and user data, followed by fusion through bilinear pooling and dot multiplication to generate recommendation probabilities, while also incorporating text features for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the ACF model is used to extract joint features between user and video, then video recommendation can be performed, but information loss occurs due to the property difference between user features and video features
Solution Approach 1:
The patent divides the feature extraction process into two separate networks: a first feature extraction network for video features and a second feature extraction network for user features. This segmentation allows each network to be optimized for its specific input type, preventing information loss that would occur in a unified network while managing complexity through modular design.
Solution Approach 2:
The patent applies different network architectures and extraction strategies to different feature types based on their local characteristics. Video features use a network optimized for visual data, while user features use a network optimized for behavioral and demographic data, ensuring each feature type is processed with appropriate quality and methodology.
2Reliability
If the ACF model extracts joint features from video library, then recommendation ranking can be obtained, but gradient vanishing occurs due to property difference between features
Solution Approach 1:
By segmenting the feature extraction into separate networks for videos and users, the patent eliminates the gradient vanishing problem that occurs when trying to process heterogeneous feature types in a single network. Each network maintains stable gradients for its specific feature type while the fusion layer combines them without gradient conflicts.
Solution Approach 2:
The patent introduces a feature fusion network as an intermediary between the video feature extraction network and user feature extraction network. This mediator handles the combination of features from different sources, isolating the gradient flow in each extraction network and preventing vanishing gradients while enabling effective feature integration.
3Loss of information
If separate feature extraction networks are used for video and user data, then information loss is reduced, but device complexity increases
Solution Approach 1:
The patent segments the feature extraction system into specialized networks for different data types, which improves information preservation by optimizing each network for its specific input. The complexity increase is managed through modular architecture where each segment can be independently optimized and maintained.
Solution Approach 2:
The patent creates a composite feature extraction system that combines multiple specialized networks (video feature network, user feature network, and fusion network) into a unified recommendation system. This composite structure leverages the strengths of each component while achieving overall system optimization that balances information preservation with manageable complexity.
Data Source
AI summary
A video recommendation method is provided, including: inputting a video to a first feature extraction network, performing feature extraction on at least one consecutive video frame in the video, and outputting a video feature of the video; inputting user data of a user to a second feature extraction network, performing feature extraction on the discrete user data, and outputting a user feature of the user; performing feature fusion based on the video feature and the user feature, and obtaining a recommendation probability of recommending the video to the user; and determining, according to the recommendation probability, whether to recommend the video to the user.