Data summerization method, query method and device, equipment, and storage medium

A technology of data aggregation and data query, applied in the field of data processing, can solve the problems of data aggregation, high delay, poor fault tolerance, etc., and achieve the effect of avoiding low efficiency, improving efficiency, and improving timeliness

Inactive Publication Date: 2019-01-25
SHENZHEN LEXIN SOFTWARE TECH CO LTD
View PDF4 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In the existing technology, scripts are used to summarize the data of sub-databases and sub-tables into a single database and single table. This method cannot summarize the data generated online in a timely manner, and the delay is high, and the script is used to call the data , poor fault tolerance, thus affecting the efficiency of summarizing data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data summerization method, query method and device, equipment, and storage medium
  • Data summerization method, query method and device, equipment, and storage medium
  • Data summerization method, query method and device, equipment, and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] figure 1 It is a flow chart of a data summarization method provided by Embodiment 1 of the present invention. This embodiment is applicable to the situation of summarizing sub-databases and sub-table books. This method can be executed by a data summarization device, which can be controlled by hardware and / or software, and generally can be integrated into a device with data aggregation function, which can be an electronic device such as a server, a mobile terminal, or a server cluster. Such as figure 1 As shown, the method specifically includes the following steps:

[0045] Step 110, when it is detected that a data log is generated online, data corresponding to the data log is collected.

[0046] When data is generated in the network and sub-databases and tables are performed, a database log (binlog) will be generated, and the database log will be detected in real time. When the data log is generated, the data corresponding to the data log will be collected. In this e...

Embodiment 2

[0060] image 3 It is a flow chart of a data query method provided by Embodiment 2 of the present invention. This embodiment is applicable to the case of querying summarized data. The method can be executed by a data query device, which can be controlled by hardware and / or software, and generally can be integrated into a device with data query function, which can be an electronic device such as a server, a mobile terminal, or a server cluster. Such as image 3 As shown, the method specifically includes the following steps:

[0061] Step 310, obtaining query statements corresponding to the data to be queried.

[0062] When the business personnel query the summarized data in the ciphertext database, they input the query statement of the data to be queried into the query system. The query statement includes keyword information identifying the data to be queried.

[0063] Step 320, acquire a plurality of sub-data from the distributed ciphertext library according to the query s...

Embodiment 3

[0069] Figure 4 It is a schematic structural diagram of a data summarization device provided by Embodiment 3 of the present invention. Such as Figure 4 As shown, the device includes: a data collection module 410 , a message queue writing module 420 , a data reading module 430 and a data summarization module 440 .

[0070] The data collection module 410 is used to collect data corresponding to the data log when it is detected that the data log is generated online;

[0071] A message queue writing module 420, configured to write data into a message queue corresponding to a data subject;

[0072] Data reading module 430, used for reading the data of setting quantity from message queue;

[0073] The data summarization module 440 is used for distributing and summarizing the set amount of data into the plaintext database, and encrypting the set amount of data according to the set rules and then distributingly summarizing them into the ciphertext database.

[0074] Optionally, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a data summerization method, a query method and device, equipment and a storage medium. The method comprises the following steps: when the data log generatedon line is detected, the data corresponding to the data log is collected; Writing the data into a message queue corresponding to a data subject; reading a set amount of data from the message queue; Distributed aggregation of the set amount of data into a plaintext library, and distributed aggregation of the set amount of data encrypted according to the set rules into a ciphertext library. According to the data summarization method provided by an embodiment of the present invention, when the data log is detected, the data corresponding to the data log is written into the message queue, which can improve the timeliness of data summerization, and the distributed summerization of the set amount of data into the plaintext library and ciphertext library can avoid the inefficiency caused by the stand-alone aggregation, thereby improving the efficiency of data summerization.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data processing, and in particular to a data summarization method, a query method, a device, a device, and a storage medium. Background technique [0002] With the rapid development of the Internet, a large amount of data is generated all the time in the Internet. In order to effectively manage the generated large amount of data, these data are divided into databases and tables. In order to conduct big data analysis, if operators need to count various indicators and trends, they need to summarize the data in the sub-database and sub-table into a single database and single table to facilitate data analysis and solve operational analysis problems. . [0003] In the existing technology, scripts are used to summarize the data of sub-databases and sub-tables into a single database and single table. This method cannot summarize the data generated online in a timely manner, and the del...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/18G06F16/25G06F9/54
CPCG06F9/546G06F2209/548
Inventor 林伟平王雨春熊志坤张路
Owner SHENZHEN LEXIN SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap