Distributed recommendation method based on Spark platform

A recommendation method and distributed technology, applied in database distribution/replication, structured data retrieval, marketing, etc., can solve problems such as few technical solutions

Active Publication Date: 2017-12-08
NORTHEASTERN UNIV
View PDF2 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] At present, there are few technical solutions for implementing commercial ...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed recommendation method based on Spark platform
  • Distributed recommendation method based on Spark platform
  • Distributed recommendation method based on Spark platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0085] The present invention will be further elaborated below in conjunction with the accompanying drawings of the description.

[0086]The invention is a distributed recommendation method based on the Spark platform, which is applicable to a business scenario in which users have no ratings for clicked commodities, and belongs to the field of recommendation systems. Adopt master-slave architecture, use HDFS (Hadoop Distributed File System, Hadoop Distributed File System) of Hadoop cluster as storage, Spark cluster as computing engine, combine historical data of user exposure clicks and exposure non-clicks, commodity tags and user interest tags, and apply ItemBasedCF (product-based collaborative filtering recommendation algorithm) and UserBased CF (user-based collaborative filtering algorithm) work together to recommend products to users. Further use the Factorization Machine (factorization machine) model to predict the user's click on the recommended product, so as to obtain t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a distributed recommendation method based on a Spark platform. When related parameters input by users are legal and historical behavior data clicked by the users is not empty, a recommendation sequence A based on ItemBased collaborative filtering is generated; community finding is carried out with the users as the vertexes and the number of common clicks of the users as the edge, and a recommendation sequence B based on the UserBased collaborative filtering is generated; the A and the B are merged according to different weights to generate a recommended sequence C based on collaborative filtering; on the basis of the C, personal click historical behaviors of the users are paid attention to, training is carried out by utilizing a factor decomposition machine model, a training model is generated for prediction, and a prediction recommendation sequence result P is generated; the C and the P are merged according to a merging rule, a final recommendation sequence F is generated and ordered, and the final recommendation sequence F is written into a real-time database. By means of the method, the requirement for recommending massive big data can be met, and collective wisdom recommendation and personal wisdom recommendation are combined to form the final recommendation sequence.

Description

technical field [0001] The invention relates to a distributed recommendation system, in particular to a distributed recommendation method based on the Spark platform. Background technique [0002] Collaborative Filtering recommendation is a popular technique in information filtering and recommendation systems. Different from traditional content-based filtering that directly analyzes content for recommendation, collaborative filtering analyzes user interests, finds similar (interested) users of the specified user in the user group, and synthesizes the evaluation of a certain information by these similar users to form a system for the specified user. Prediction of liking for this information. Traditional item-based collaborative filtering evaluates the similarity between items based on user ratings for different items, and makes recommendations based on the similarity between items; traditional user-based collaborative filtering evaluates items based on the ratings of differe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06Q30/02G06Q30/06
CPCG06F16/182G06F16/219G06F16/2462G06F16/27G06F16/285G06F16/9535G06Q30/0255G06Q30/0631
Inventor 陈东明胡阳黄新宇
Owner NORTHEASTERN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap