Unlock instant, AI-driven research and patent intelligence for your innovation.

Distributed system, data aggregation method and computer readable storage medium

A distributed system and computer program technology, applied in the computer field, can solve the problems of hang up, not considering the memory pressure of the coordinator node, and the instability of the coordinator node, so as to reduce the risk of hang up and the effect of reducing the memory pressure.

Active Publication Date: 2021-07-16
BEIJING QIANXIN TECH +1
View PDF3 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to provide a distributed system, a data aggregation method, a computer device and a computer-readable storage medium, which can solve the problem that the coordinating node may be unstable or hang up without considering the memory pressure of the coordinating node in the ElasticSearch retrieval of the prior art technical issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed system, data aggregation method and computer readable storage medium
  • Distributed system, data aggregation method and computer readable storage medium
  • Distributed system, data aggregation method and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] figure 1 Schematically shows a block diagram of a data aggregation system according to Embodiment 1 of the present invention, as shown in figure 1 As shown, the distributed system may include several query nodes 101, the query nodes 101 are used to receive query requests sent by users, and the query nodes 101 that receive the query requests are called coordinating nodes 102, wherein:

[0031] The coordinating node 102 is configured to: parse the query request to determine the target index segment pointed to by the query request, obtain segment information of the target index segment, and determine a query node that locally stores the target index segment 101. As a target node 103, construct a query subrequest according to the fragment information, and send the query subrequest to the target node 103.

[0032] Wherein, the target node 103 is also a query node 101 .

[0033] In this embodiment, if the index label is carried in the query request, the coordinating node 10...

Embodiment 2

[0053] Embodiment 2 of the present invention provides a data aggregation method. Part of the steps of the data aggregation method are the same as those in the above-mentioned embodiment 1. These steps will not be repeated in this embodiment. For details, please refer to the above-mentioned embodiment. one. specifically, figure 2 Schematically shows a flow chart of a data aggregation method according to Embodiment 2 of the present invention, as figure 2 As shown, the data aggregation method described in the second embodiment may include steps S21 to S25, wherein:

[0054] Step S21, receiving the query request sent by the coordinating node as the first query request.

[0055] Step S22, parsing the first query request to obtain fragmentation information.

[0056] Step S23, locating a locally stored target index segment according to the segment information.

[0057] Step S24, running the target index segment to obtain the query result requested by the first query request.

...

Embodiment 3

[0063] Embodiment 3 of the present invention provides a data aggregation method. Part of the steps of the data aggregation method are the same as the steps in the above-mentioned embodiment 1. These steps will not be repeated in this embodiment. For details, please refer to the above-mentioned embodiment one. specifically, image 3 Schematically shows a flow chart of the data aggregation method according to Embodiment 3 of the present invention, as image 3 As shown, the data aggregation method described in the third embodiment may include steps S31 to S36, wherein:

[0064] Step S31, receiving the query request sent by the user as a second query request.

[0065] Step S32, parsing the second query request to determine the target index segment pointed to by the second query request.

[0066] Step S33, acquiring the segment information of the target index segment.

[0067] Step S34, determining the node locally storing the target index fragment as the target node.

[0068]...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed system, which is characterized in that a coordination node is used for analyzing a query request to determine a target index fragment pointed by the query request, acquiring fragment information of the target index fragment, determining a query node for locally storing the target index fragment as a target node, constructing a query sub-request according to the fragment information, and sending the query sub-request to the target node; the target node is used for receiving and analyzing the query sub-request to obtain fragment information, positioning a locally stored target index fragment according to the fragment information, running the target index fragment to obtain a query result requested by the query sub-request, and when all the locally stored target index fragments are not unique, aggregating the query results of all the target index fragments to obtain a first aggregation result, and returning the first aggregation result to the coordination node; and the coordination node is also used for receiving the first aggregation results returned by the target nodes and aggregating all the first aggregation results returned by the plurality of target nodes.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a distributed system, a data aggregation method, computer equipment and a computer-readable storage medium. Background technique [0002] ElasticSearch is a distributed retrieval system based on Apache Lucene, which can implement retrieval functions through indexes. The retrieval process of ElasticSearch is divided into query stage and fetch stage. The query stage is used to query the doc id list according to the query request, and the fetch stage is used to obtain doc documents according to the doc id list. Among them, in the ElasticSearch system, an index can be composed of several index fragments, and the ElasticSearch system includes multiple query nodes, and each query node stores multiple index fragments. The query node that receives the query request is also called Coordinator node. [0003] In the query stage, after the coordinating node receives the query req...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L12/911H04L29/08
CPCH04L47/827H04L67/1044
Inventor 于平韩洪伟
Owner BEIJING QIANXIN TECH