Distributed recommendation method based on Spark platform

Abstract

The invention relates to a distributed recommendation method based on a Spark platform. When related parameters input by users are legal and historical behavior data clicked by the users is not empty, a recommendation sequence A based on ItemBased collaborative filtering is generated; community finding is carried out with the users as the vertexes and the number of common clicks of the users as the edge, and a recommendation sequence B based on the UserBased collaborative filtering is generated; the A and the B are merged according to different weights to generate a recommended sequence C based on collaborative filtering; on the basis of the C, personal click historical behaviors of the users are paid attention to, training is carried out by utilizing a factor decomposition machine model, a training model is generated for prediction, and a prediction recommendation sequence result P is generated; the C and the P are merged according to a merging rule, a final recommendation sequence F is generated and ordered, and the final recommendation sequence F is written into a real-time database. By means of the method, the requirement for recommending massive big data can be met, and collective wisdom recommendation and personal wisdom recommendation are combined to form the final recommendation sequence.

Description

technical field [0001] The invention relates to a distributed recommendation system, in particular to a distributed recommendation method based on the Spark platform. Background technique [0002] Collaborative Filtering recommendation is a popular technique in information filtering and recommendation systems. Different from traditional content-based filtering that directly analyzes content for recommendation, collaborative filtering analyzes user interests, finds similar (interested) users of the specified user in the user group, and synthesizes the evaluation of a certain information by these similar users to form a system for the specified user. Prediction of liking for this information. Traditional item-based collaborative filtering evaluates the similarity between items based on user ratings for different items, and makes recommendations based on the similarity between items; traditional user-based collaborative filtering evaluates items based on the ratings of differe...

AI Technical Summary

Problems solved by technology

[0008] At present, there are few technical solutions for implementing commercial recommendation systems using the Spark distributed memory iterative computing framework

Images