Near-real-time data acquisition method and device, electronic device and storage medium

A data acquisition and near-real-time technology, applied in the computer field, can solve the problems of high professionalism of developers, long development cycle, and high development difficulty, and achieve the effect of reducing development cost and development cycle, lowering development threshold, and reducing processes

Active Publication Date: 2019-11-26
JIANGSU MANYUN SOFTWARE TECH CO LTD
View PDF4 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] However, because batch processing acceleration is based on Hive computing, it cannot update the data, which prolongs the processing process and cannot guarantee that the entire job chain will be completed within the specified time, making it difficult to meet business requirements; real-time stream computing requires developers High professionalism, difficult development, long development cycle, and inability to efficiently respond to urgent business needs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Near-real-time data acquisition method and device, electronic device and storage medium
  • Near-real-time data acquisition method and device, electronic device and storage medium
  • Near-real-time data acquisition method and device, electronic device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0051]Furthermore, the drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a near-real-time data collection method and device, an electronic device and a storage medium, and the method comprises the steps: receiving a first data collection instructionwhich is used for calling first data information; calling the first data information from a near-real-time database based on the first data acquisition instruction, the near-real-time database at least comprising: a first data layer which acquires and stores service data from a service database; a second data layer, comprising a plurality of data models, each data model being associated with a business theme, and the data of the first data layer being classified into a plurality of business themes through the plurality of data models of the second data layer; and a third data layer, comprisinga plurality of wide tables, and each wide table at least comprising statistical data of service data classified to a plurality of service themes through the second data layer. According to the methodand the device provided by the invention, near-real-time data processing is realized.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a near real-time data acquisition method, device, electronic equipment, and storage medium. Background technique [0002] With the development of big data, near real-time data processing is particularly important in various fields. In particular, for business data in various fields, how to obtain constantly changing business data within a higher timeliness to support internal process management and external business expansion is an urgent problem to be solved in the era of big data. [0003] In order to solve this problem, there are currently two implementations of near real-time data processing: [0004] 1) Batch processing acceleration [0005] Batch acceleration is based on components of the existing Hadoop (distributed system infrastructure developed by the Apache Foundation) ecosystem. Batch processing acceleration includes data extraction layer, logic transformation la...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/28G06F16/22
CPCG06F16/25G06F16/28G06F16/22
Inventor 张超
Owner JIANGSU MANYUN SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap