Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data analysis method, computing device and storage medium

A data analysis and data grouping technology, applied in the field of data analysis, can solve problems such as high memory usage and slow data query efficiency, achieve fast high-dimensionality, reduce resource consumption, and improve data query efficiency

Active Publication Date: 2022-03-11
ZHANGYUE TECH CO LTD
View PDF15 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, when using existing real-time data analysis tools to analyze and process high-dimensional, large-base data, there will be problems such as high memory usage and slow data query efficiency.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data analysis method, computing device and storage medium
  • Data analysis method, computing device and storage medium
  • Data analysis method, computing device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] figure 1 shows a schematic flow chart of a data analysis method according to Embodiment 1 of the present invention, as figure 1 As shown, the method includes the following steps:

[0024] Step S101, selecting a target grouping dimension from multiple data grouping dimensions.

[0025] The method is used to analyze the log data generated by the business server, and the log data can be analyzed by combining real-time analysis and offline analysis during the analysis process. Among them, the offline analysis is performed on the specified target grouping dimension. Specifically, according to the daily data analysis and query requirements of the data query terminal, one or more commonly used data grouping dimensions can be selected from multiple data grouping dimensions involved in data analysis as the target grouping dimension. +The data grouping dimension of "Large cardinality field grouping" is used as the target grouping dimension. Among them, high-dimensional specif...

Embodiment 2

[0034] Figure 2a shows a schematic flow chart of a data analysis method according to Embodiment 2 of the present invention, as Figure 2a As shown, the method includes the following steps:

[0035] Step S201, selecting a target grouping dimension from multiple data grouping dimensions.

[0036] In order to make the selected target grouping dimension more in line with the data analysis and query requirements that are frequently used in daily life, it is possible to collect information on various data analysis and query requirements from the data query terminal and the operation and maintenance personnel of the business server, etc., and determine each data analysis and query requirement The data grouping dimension corresponding to the information. For example, if a certain data analysis query demand information is to analyze the number of clicks of UV (UniqueVisitor, independent user) corresponding to each display resource displayed on the client in each city in the country ...

Embodiment 3

[0063] Embodiment 3 of the present invention provides a non-volatile storage medium, where at least one executable instruction is stored in the storage medium, and the executable instruction can execute the data analysis method in any of the above method embodiments.

[0064] Specifically, the executable instruction can be used to make the processor perform the following operations: select the target grouping dimension from multiple data grouping dimensions; extract the source data from the source data table, and perform offline calculation on the source data according to the target grouping dimension to obtain the target grouping dimension The offline calculation result corresponding to the dimension; among them, the source data in the source data table is the real-time data obtained by rolling up the log data generated by the business server according to multiple data grouping dimensions; Data analysis query request, which fuses the offline calculation results corresponding t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data analysis method, computing equipment and a storage medium, and the data analysis method comprises the steps: selecting a target grouping dimension from a plurality of data grouping dimensions; extracting source data from the source data table, and performing offline calculation on the source data according to the target grouping dimension to obtain an offline calculation result corresponding to the target grouping dimension; wherein the source data in the source data table is real-time data obtained by performing data volume loading on log data generated by the business server according to a plurality of data grouping dimensions; and in response to a data analysis query request containing the target grouping dimension, fusing the offline calculation results corresponding to the target grouping dimension to obtain a data query result. According to the scheme, the offline calculation result corresponding to the target grouping dimension is obtained by utilizing the offline calculation mechanism, and the offline calculation result corresponding to the target grouping dimension is fused, so that the data query efficiency is improved, and the resource consumption of data analysis is reduced.

Description

technical field [0001] The present invention relates to the technical field of data analysis, in particular to a data analysis method, computing equipment and a storage medium. Background technique [0002] With the development of Internet technology, the business volume has gradually increased, which has further led to a rapid increase in the amount of data generated by the business. Therefore, higher requirements are put forward for the analysis and processing of data. While obtaining effective information, it is also necessary to meet the timeliness requirements of data processing, so that real-time data analysis technology is also paid more and more attention accordingly. For example, real-time data analysis tools such as Clickhouse and StarRocks. Through real-time data analysis tools, some analysis and processing of massive data can be done in real time, and corresponding real-time indicators can be fed back. However, when using existing real-time data analysis tools ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/2453G06F16/2455G06F16/2458
CPCG06F16/24539G06F16/24556G06F16/2462
Inventor 陈杰文蔡东海
Owner ZHANGYUE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com