Unlock instant, AI-driven research and patent intelligence for your innovation.

Streaming data structured processing system and method

A processing system and processing method technology, applied in the field of stream data structured processing systems, to achieve the effect of improving scalability and facilitating product iteration

Pending Publication Date: 2020-01-17
G CLOUD TECH
View PDF16 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical problem solved by the present invention is to provide a stream data structured processing system and method; to realize the plug-in of the stream data structured processing function, and to avoid problems such as high coupling, insufficient flexibility, and weak scalability

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Streaming data structured processing system and method
  • Streaming data structured processing system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] See figure 1 As shown, the stream data structured processing system of the present invention mainly focuses on the structured processing of various devices, application logs, real-time geographic location information, and sensor data. The system of the present invention is divided into three modules, 1. a plug-in module with specific functional flow data analysis; 2. a data bus module for realizing mutual communication between plug-ins; 3. a plug-in arrangement module.

[0028] 1. Plug-in module for streaming data analysis

[0029] like figure 1 , the plug-in modules for streaming data analysis are independent plug-ins. All the original data parsing functions in this system will be completely decoupled from the streaming data processing system, and will be implemented in the form of plug-ins.

[0030] A plug-in is a completely independent software entity capable of processing specific data, such as a jar package.

[0031] The plug-ins are loaded uniformly when the sy...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a streaming data structured processing system and method. The streaming data structured processing system comprises a plug-in module, a data bus module and a plug-in arrangement module, wherein the plug-in module is a plug-in with a specific stream data analysis function; the data bus module is used for accessing a plug-in to realize consumption of data before analysis andproduction of data after analysis; the plug-in arrangement module arranges a plug-in module of the system, constructs a complete data chain, and completely analyzes the streaming data. According to the streaming data structured processing method, a complete streaming data analysis link is defined by using a script file, and the script file is analyzed to sequentially process streaming data through different plug-ins and finally output the streaming data. According to the streaming data structured processing method, the plug-ins are decoupled, and the plug-ins and the system are decoupled, sothat the expandability of an analysis function is improved; and the streaming data structured processing method can be used for structured processing of streaming data.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a stream data structured processing system and method. Background technique [0002] Streaming data, such as log files generated by applications, device logs, online shopping data, in-game player activities, social networking site information, financial trading halls, or geospatial service data, has the characteristics of rich sources, huge differences in data formats, and real-time nature. The structured processing of data can be understood as the data cleaning stage in data mining, which provides an operational basis for subsequent data association, data fusion, and data analysis. Therefore, the structured processing of streaming data with different data formats is of great significance to the field of data mining. [0003] In the traditional stream data analysis platform, the data structure processing code is seriously coupled with other system function codes, which l...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/445
CPCG06F9/44526
Inventor 罗平季统凯
Owner G CLOUD TECH