Hadoop-based network recommendation method

A recommendation method and network technology, applied in the recommendation field, can solve the problems of high computing time consumption and poor scalability of the recommendation system, and achieve the effect of improving scalability

CN104503967AInactive Publication Date: 2015-04-08LANGCHAO ELECTRONIC INFORMATION IND CO LTD
6 Cites 4 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2015-04-08
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a Hadoop-based network recommendation method. Specific to a network based on a Hadoop platform, the scale of a cluster is increased according to a practical computation requirement; a dynamic storage space is provided for mass data sets by using an HDFS (Hadoop Distributed File System) to store the ever-increasing mass data sets; big tasks are decomposed into small tasks through a MapReduce framework; the computation speed of an algorithm is increased through a parallel computation way; and network recommendation is realized by using a substance diffusion algorithm and a heat conduction algorithm. Through adoption of the Hadoop-based network recommendation method, the problem that a large electronic commerce website cannot be well reached since a conventional algorithm running in a standalone mode tends to be impeded easily by multiple computer hardware performance such as processor speed and storage capacity is solved; a system is reliably and efficiently recommended to perform calculation, analysis and mining through massive user dialogs; and the problems of high time consumption, poor expandability, polynomial increase of the complexity of the algorithm and worse and worse algorithm performance along with the increase of the numbers of users and goods, and the like in other recommendation algorithms are solved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of recommendation methods, in particular to a hadoop-based network recommendation method. Background technique

[0002] With the rapid innovation of Internet technology, the total number of web servers in the world continues to increase, and the number of web pages therefore shows an explosive growth trend. People can read articles, listen to music, communicate socially, share photos, and buy goods online, but it also brings a series of troubles: on the one hand, it is difficult for users to search for valuable parts from the explosive accumulation of massive information; On the one hand, the quality of information itself varies, and some potentially valuable information is buried due to the lack of auxiliary tools. This phenomenon is called "information overload". In order to effectively deal with the above problems, people have developed search engines, such as Google, Baidu, Bing, etc., which use web crawlers to c...

Examples

Embodiment Construction

[0021] Below according to accompanying drawing of description, in conjunction with specific embodiment, the present invention is further described:

[0022] A hadoop-based network recommendation method, the design structure of the recommendation method includes: (1) a recommendation application interface module, (2) a recommendation system module, and (3) a data storage module.

[0023] Among them, the operating mode of Hadoop adopts a fully distributed operating mode, which is composed of 4 PCs, one of which is used as the master control node, and the remaining 3 are used as slaver computing nodes. The hardware of each PC is Intel Dual-core 2.6GHz CPU, 2G memory and 100G hard disk, the operating system is Ubuntu-10.10-server-i386, the JDK version is 1.6.0_27, the MySQL version is 5.6.10, and the Hadoop version is 0.20. 2. The specific configuration of hadoop is as follows figure 1 shown.

[0024] Such as figure 2 As shown, it is recommended that the system install Apache,...