Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A distributed recommendation method based on spark platform

A recommended method and distributed technology, applied in the direction of database distribution/replication, instrumentation, relational database, etc., can solve the problem of few technical solutions

Active Publication Date: 2020-02-21
NORTHEASTERN UNIV LIAONING
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] At present, there are few technical solutions for implementing commercial recommendation systems using the Spark distributed memory iterative computing framework

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A distributed recommendation method based on spark platform
  • A distributed recommendation method based on spark platform
  • A distributed recommendation method based on spark platform

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0085] The present invention will be further elaborated below in conjunction with the accompanying drawings of the description.

[0086]The invention is a distributed recommendation method based on the Spark platform, which is applicable to a business scenario in which users have no ratings for clicked commodities, and belongs to the field of recommendation systems. Adopt master-slave architecture, use HDFS (Hadoop Distributed File System, Hadoop Distributed File System) of Hadoop cluster as storage, Spark cluster as computing engine, combine historical data of user exposure clicks and exposure non-clicks, commodity tags and user interest tags, and apply ItemBasedCF (product-based collaborative filtering recommendation algorithm) and UserBased CF (user-based collaborative filtering algorithm) work together to recommend products to users. Further use the Factorization Machine (factorization machine) model to predict the user's click on the recommended product, so as to obtain t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a distributed recommendation method based on the Spark platform. When the relevant parameters input by the user are legal and the historical behavior data clicked by the user is not empty, a recommendation sequence A based on ItemBased collaborative filtering is generated; based on the user as the vertex, the user and The number of users’ common clicks is used as a side to discover communities, and generate a recommendation sequence B based on UserBased collaborative filtering; merge A and B according to different weights to generate a recommendation sequence C based on collaborative filtering; on the basis of C, focus on individual users Click on the historical behavior, use the factorization machine model to train, generate the training model for prediction, and generate the predicted recommendation sequence result P; merge C and P according to the merging rules, generate the final recommended sequence F and sort it, and write it into the real-time database. The present invention can meet the recommendation requirements of massive big data, and combines collective wisdom recommendation and individual wisdom recommendation to form a final recommendation sequence.

Description

technical field [0001] The invention relates to a distributed recommendation system, in particular to a distributed recommendation method based on the Spark platform. Background technique [0002] Collaborative Filtering recommendation is a popular technique in information filtering and recommendation systems. Different from traditional content-based filtering that directly analyzes content for recommendation, collaborative filtering analyzes user interests, finds similar (interested) users of the specified user in the user group, and synthesizes the evaluation of a certain information by these similar users to form a system for the specified user. Prediction of liking for this information. Traditional item-based collaborative filtering evaluates the similarity between items based on user ratings for different items, and makes recommendations based on the similarity between items; traditional user-based collaborative filtering evaluates items based on the ratings of differe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/9535G06Q30/02G06Q30/06
CPCG06F16/182G06F16/219G06F16/2462G06F16/27G06F16/285G06F16/9535G06Q30/0255G06Q30/0631
Inventor 陈东明胡阳黄新宇
Owner NORTHEASTERN UNIV LIAONING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com