A data aggregation method and server

A data aggregation and server technology, applied in the field of data aggregation methods and servers, can solve the problems of reducing the original business I/O performance, wasting computing power, and instantaneous resource usage peaks, achieving low data retention time and reducing I/O pressure. , the effect of avoiding waste of resources

Active Publication Date: 2020-11-06
北京北龙超级云计算有限责任公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method is not suitable for applications in scenarios with large amounts of data, and the single point of failure problem may produce inaccurate analysis results
Moreover, when a data aggregation request occurs, the server reads, calculates, and aggregates data synchronously, which is not friendly to some scenarios that require relatively high response speed, and for non-relational databases, the pressure will also be great. Momentary resource usage spikes may occur
[0004] In another solution, different data processing processes are designed according to different business needs, resulting in low efficiency and flexibility of data processing business; and the results of each business have strong business correlations, which cannot be provided to other businesses as input data, resulting in A lot of waste of computing power
Or each business will directly read the original data when requesting, which will cause a large number of repeated I / O operations, and even reduce the I / O performance of the original business, thereby affecting the original business system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data aggregation method and server
  • A data aggregation method and server
  • A data aggregation method and server

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] figure 1 A schematic diagram of a data aggregation system 100 according to an embodiment of the present invention is shown. As shown in the figure, the system 100 includes at least one computing node 110 , message middleware 120 , server 130 and data index library 140 . s, figure 1 The data aggregation system 100 in is only exemplary. In a specific practical situation, there may be different numbers of computing nodes 110,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data aggregation method, which is suitable for execution in a server, and the server is respectively connected to a message middleware and a data index database. type of topic data, the topic data includes a timestamp and at least one indicator value; temporarily store the topic data in a preset data flow queue, and use a preset filter flow queue to extract topics one by one from the data flow queue Data; aggregate the same topic data of the same computing node according to at least one aggregation period, and calculate the indicator statistics of the computing node in each aggregation period; aggregate the indicator statistics of multiple aggregation periods within the preset index period Summarize and store in the data index library as the data index of the computing node in the preset index period. The invention also discloses a server for executing the method.

Description

technical field [0001] The invention relates to the field of log processing, in particular to a data aggregation method and server. Background technique [0002] In recent years, high-performance computing technology has gradually developed and matured. With the increasing demand for computing power, the expansion of computing scale, and the changes in computing models, more and more users not only pay attention to the monitoring and analysis of macro performance data, but also pay great attention to micro performance. Association analysis of data. The E-level system has a large scale and supports complex subject applications. It needs to realize high-frequency and low-load collection, storage, analysis and mining of multi-dimensional management data and performance big data. For the massive performance data collected, it is very important to design data aggregation to realize effective multi-dimensional and multi-indicator performance data visual analysis, which puts forwa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/242
CPCG06F16/22G06F16/244
Inventor 梁敏孙园萍
Owner 北京北龙超级云计算有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products