Data aggregation method and server

A data aggregation and server technology, applied in the field of data aggregation methods and servers, can solve problems such as waste of computing power, reduction of original business I/O performance, and instantaneous resource usage peaks, so as to reduce I/O pressure and reduce data retention time , reduce the effect of mutual interference

Active Publication Date: 2020-08-07
北京北龙超级云计算有限责任公司
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method is not suitable for applications in scenarios with large amounts of data, and the single point of failure problem may produce inaccurate analysis results
Moreover, when a data aggregation request occurs, the server reads, calculates, and aggregates data synchronously, which is not friendly to some scenarios that require relatively high response speed, and for non-relational databases, the pressure will also be great. Momentary resource usage spikes may occur
[0004] In another solution, different data processing processes are designed according to different business needs, resulting in low efficiency and flexibility of data processing business; and the results of each business have strong business correlations, which cannot be provided to other businesses as input data, resulting in A lot of waste of computing power
Or each business will directly read the original data when requesting, which will cause a large number of repeated I / O operations, and even reduce the I / O performance of the original business, thereby affecting the original business system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data aggregation method and server
  • Data aggregation method and server
  • Data aggregation method and server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] figure 1 A schematic diagram of a data aggregation system 100 according to an embodiment of the present invention is shown. As shown in the figure, the system 100 includes at least one computing node 110 , message middleware 120 , server 130 and data index library 140 . s, figure 1 The data aggregation system 100 in is only exemplary. In a specific practical situation, there may be different numbers of computing nodes 110,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data aggregation method, which is suitable for being executed in a server. The server is respectively in communication connection with message middleware and a data index library. The data aggregation method comprises the following steps of: reading theme data of each computing node for different theme types from the message middleware, wherein the theme data comprises atimestamp and at least one index value; temporarily storing the theme data in a preset data flow queue, and extracting the theme data from the data flow queue piece by piece by adopting a preset filtering flow queue; aggregating the same theme data of the same computing node according to at least one aggregation period, and calculating an index statistical value of the computing node in each aggregation period; and summarizing the index statistical values of the plurality of aggregation periods in the preset index time period, taking the summarized index statistical values as data indexes of the computing nodes in the preset index time period, and storing the data indexes into a data index library. The invention further discloses a server for executing the data aggregation method.

Description

technical field [0001] The invention relates to the field of log processing, in particular to a data aggregation method and server. Background technique [0002] In recent years, high-performance computing technology has gradually developed and matured. With the increasing demand for computing power, the expansion of computing scale, and the changes in computing models, more and more users not only pay attention to the monitoring and analysis of macro performance data, but also pay great attention to micro performance. Association analysis of data. The E-level system has a large scale and supports complex subject applications. It needs to realize high-frequency and low-load collection, storage, analysis and mining of multi-dimensional management data and performance big data. For the massive performance data collected, it is very important to design data aggregation to realize effective multi-dimensional and multi-indicator performance data visual analysis, which puts forwa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/242
CPCG06F16/22G06F16/244
Inventor 梁敏孙园萍
Owner 北京北龙超级云计算有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products