Distributed real-time search engine based on p2p

Abstract

The invention discloses a distributed real-time search engine based on p2p. A cluster of the distributed real-time search engine comprises a plurality of nodes, each node comprises more than one index, each index is divided into more than one index fragmentation, and each index fragmentation only comprises a main fragmentation or simultaneously comprises a main fragmentation and more than one duplicate; the nodes are independent from one another, are connected with one another through peer-to-peer networks, and are communicated with one another in a broadcast or multicast mode, and each independent node stores a cluster index metadata sheet reflecting index information of all the nodes in the whole cluster. When the indexes are updated and searched, the nodes read the cluster index metadata sheets and send requests to the corresponding nodes to conduct indexing or search according to attributes of the requests. By means of the method, main nodes are omitted, a searching route is shortened, and the searching speed is improved. The distributed real-time search engine has high fault tolerance, so that even if one node breaks down, other nodes still can work normally, and no burden is caused on a system.

Description

technical field [0001] The invention relates to a search engine, in particular to a distributed real-time search engine based on p2p. Background technique [0002] The information on the Internet is growing explosively, and the information on the network is also updated in real time at a high speed. Therefore, how to obtain information accurately, quickly and in a timely manner has become the primary problem that search engines need to solve. [0003] The existing distributed search methods are mainly centralized, with a master-slave structure. A centralized server acts as the master node to manage all child nodes, and the master node distributes requests to other child nodes to realize distributed search. The master node uniformly provides retrieval services. This centralized search engine has the following disadvantages: (1) The real-time performance of the data cannot be guaranteed. Since the data is updated to the master node first, and then the master node is updated t...

AI Technical Summary

Problems solved by technology

[0003] The existing distributed search methods are mainly centralized, with a master-slave structure. A centralized server acts as the master node to manage all child nodes, and the master node distributes requests to other child nodes to realize distributed search. The master node uniformly provides retrieval services. This centralized search engine has the following disadvantages: (1) The real-time performance of the data cannot be guaranteed. Since the data is updated to the master node first, and then the master node is updated to the sub-nodes, so the data in the slave There will be a time interval when the master node is updated to the child node. If the client queries immediately when the master node updates the data, and the child node is queried at the same time, and the child node has not synchronized to the data at this time, then the newly updated That piece of data cannot be queried, so that the client cannot obtain the latest data immediately; (2) Since the entire system passes through the master node when indexing and searching, once the master node fails, the entire cluster is in an unsearchable state , causing the entire system to collapse. At the same time, when the network is busy, a large number of clients frequently visit, resulting in excessive pressure on the master node, and it is very easy to fail due to transition saturation. Even if no failure occurs, the master node will also be due to excessive pressure , running at reduced speed

Images