Data analysis system and method, storage medium and electronic equipment

A data analysis system and data technology, applied in the computer field, can solve the problems of lack of data security audit function, real-time calculation does not support, large amount of data, etc.

Active Publication Date: 2019-04-26
江苏满运物流信息有限公司
View PDF5 Cites 32 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

On the one hand, the amount of daily user behavior data is very large, some of which require real-time calculations, and some of which require offline calculations, and the complexity is very high
On the other hand, the needs of data analysts are flexible and changeable, and each data analysis is very time-consuming
[0003] Now there are some

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data analysis system and method, storage medium and electronic equipment
  • Data analysis system and method, storage medium and electronic equipment
  • Data analysis system and method, storage medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0037] Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data analysis system and method, a storage medium and electronic equipment, and the system comprises a data collection module which is used for collecting real-time data and offline data, and storing the collected offline data in an HDFS in a Hive form; The real-time calculation module responds to a query instruction of a user, real-time data are consumed through a distributed processing engine Flink to form a real-time data wide table, and data of the real-time data wide table are transferred into a Druid through message middleware kafka; The offline calculation module is used for cleaning and calculating offline data by using Hive to form an offline data width table and synchronizing the offline data width table into a distributed analysis engine Kylin to form amulti-dimensional offline data pre-summary table; And a query engine module. According to the invention, data can be accessed in a real-time or offline manner, and the core model can be abstractly calculated. A data analyst can screen the content to be analyzed by himself/herself in a supporting and pulling manner, and then can display the content in a manner of a rich visual chart.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data analysis system, method, storage medium and electronic equipment. Background technique [0002] With the development of the Internet and data technology, the amount of data stored and calculated by various Internet platforms is very large, especially for the vehicle-cargo matching platform, which usually faces drivers and cargo owners across the country. On the one hand, the amount of user behavior data is very large every day, some of which require real-time calculations, and some require offline calculations, which is very complex. On the other hand, the needs of data analysts are flexible and changeable, and each data analysis is very time-consuming. [0003] Now there are some mature commercial analysis software. The common problems of these software are: when accessing a large amount of data, the calculation is very slow, or even stuck; some do not support ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/242G06F16/182
Inventor 王东李大学严旭东张超
Owner 江苏满运物流信息有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap