Distributed search system, index distribution method and storage medium

A distributed search and distributed technology, applied in the computer field, can solve the problems of index distribution limited master node machine performance and network bandwidth, unable to expand the number of search machines, affecting the stability of search services, etc.

Inactive Publication Date: 2020-02-07
BEIJING SANKUAI ONLINE TECH CO LTD
View PDF8 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In the case of a large index file and a large search cluster, the speed of index distribution is severely limited by the machine performance and network bandwidth of the master node, which cannot scale with the size of the index and the number of search machines. At the same time, the pressure on the master node is too high, which affects the search service stability

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed search system, index distribution method and storage medium
  • Distributed search system, index distribution method and storage medium
  • Distributed search system, index distribution method and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0026] A distributed search system disclosed in this embodiment, such as figure 1 As shown, the distributed search system includes:

[0027] The distributed index construction platform 110 is used to construct an index, store the index in the index storage cluster according to the index fragmentation, and write the resource description information of the index into the state storage cluster, and the index fragmentation includes multiple file block;

[0028] An index storage cluster 120, configured to store the index according to the index slices;

[0029] The state storage cluster 130 is configured to store the resource description information, register the service of the search node, store the service state of the search node, and receive the index download command sent by the master control cluster;

[0030] The master control cluster 140 is configured to monitor the state storage cluster, and if the resource description information is detected, send an index download inst...

Embodiment 2

[0049] An index distribution method disclosed in this embodiment, such as Figure 4 As shown, the index distribution method is executed by the distributed search system disclosed in the above-mentioned embodiments, including: step 410 to step 440 .

[0050] Step 410, the distributed index construction platform constructs the index, stores the index in the index storage cluster according to the index fragments, and writes the resource description information of the index into the state storage cluster, and the index fragments include multiple files piece.

[0051] The distributed index construction platform 110 builds the latest full index, divides the index into a preset number of index fragments, compresses them according to the index fragments, stores the compressed indexes in the index storage cluster 120 according to the index fragments, and Write the indexed resource description information to the state storage cluster 130 . The nodes in the index storage cluster 120 ar...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a distributed search system, an index distribution method and a storage medium, and the system comprises a distributed index construction platform which is used for constructing an index, storing an index fragment in an index storage cluster, and writing resource description information into a state storage cluster, and the index fragment comprises a plurality of file blocks; an index storage cluster; a state storage cluster; a main control cluster which is used for sending an index downloading instruction to the state storage cluster; and a search cluster, including multiple search nodes, a plurality of search nodes form a peer-to-peer network, if the search nodes monitor an index downloading instruction, index fragments and file blocks to be downloaded are determined, and if at least two other search nodes have file blocks, the search nodes determine one of the at least two other search nodes as a downloading source of the file blocks and download the file blocks from the downloading source. According to the system of the invention, the index distribution speed is increased, and the search service stability is improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a distributed search system, an index distribution method and a storage medium. Background technique [0002] In the distributed search service, after the index construction platform builds the index, it needs to quickly distribute the index to the search cluster. In the case of large index files and many search cluster machines, how to quickly distribute the index to the search service is an inevitable problem for the distributed search service. [0003] At present, the mainstream search services include Solr and Elasticsearch. The index distribution schemes they adopt are to synchronize the index from the master node to the slave node, and multiple slave nodes download index files from a master node. [0004] In the case of a large index file and a large search cluster, the speed of index distribution is severely limited by the machine performance and network bandw...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/182G06F16/13G06F16/16G06F16/17
CPCG06F16/182G06F16/134G06F16/16G06F16/1734
Inventor 张桂贤赵朝兴项肖华张俊浩张弓王仲远
Owner BEIJING SANKUAI ONLINE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap