Real-time data processing method and system

A real-time data processing and data processing technology, applied in software maintenance/management, version control, creation/generation of source code, etc., can solve problems such as difficulty in reusing code, inability to maintain project stability, and reduction in project development efficiency. , to reduce the development threshold, simplify development costs, and prevent resubmission

Pending Publication Date: 2022-03-29
MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present invention provides a real-time data processing method and system, which solves the need to learn the coding technology of big data technology components, and the code between projects is difficult to reuse, resulting in reduced project development efficiency, and project stability cannot maintain consistent stability sexual problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time data processing method and system
  • Real-time data processing method and system
  • Real-time data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0056] On the one hand, if figure 1 As shown, a real-time data processing method performs real-time tasks according to the following steps, including:

[0057] Step S100, dynamically create an instance object of the data processing engine according to the information of the data processing engine recorded in the pre-specified task configuration; and,

[0058] Step S101, execute the data processing process of the real-time task according to the specified runni...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a real-time data processing method and system, and the method comprises the steps: abstracting a data processing process of each big data technology component into a plurality of stages; by specifying the data processing engine corresponding to the big data technology component in the task configuration and the configuration of each stage, the corresponding big data technology component can be added into the project for real-time data processing without compiling codes, the threshold of using the big data technology component by developers is reduced, and the development efficiency is improved.

Description

technical field [0001] The present invention relates to the field of real-time computing, in particular to a real-time data processing method and system. Background technique [0002] Real-time data is a set of sequential, massive, fast, and continuous arrival data sequences, and the data stream can be regarded as a dynamic data collection that grows infinitely over time. In order to detect real-time data and efficiently calculate a large amount of data, some big data technology components such as Storm, spark, and flink are usually used to process real-time data. Storm is a stream computing framework open sourced by Twitter and hosted on GitHub. Spark, namely Apache Spark, is a fast and general-purpose computing engine designed for large-scale data processing. Flink is a distributed processing engine for stream data and batch data. However, in the existing technology, there are many types of real-time data processing technologies. Each has its own strengths, but for develo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/36G06F8/71
CPCG06F8/36G06F8/71
Inventor 谢县东李洪勋
Owner MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap