多协议实时数据处理方法、装置、计算机设备和存储介质
By employing Kafka data source and Flink real-time computing engine in multi-protocol real-time data processing, and combining protocol parsing, data filtering, deduplication, and metric aggregation processes, the compatibility and efficiency issues of multi-protocol real-time data processing are resolved, enabling efficient data processing across industries and domains.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA POWER IND INTERNET CO LTD
- Filing Date
- 2023-10-17
- Publication Date
- 2026-07-17
AI Technical Summary
Existing technologies for multi-protocol real-time data processing suffer from several problems, including incompatibility with multiple data protocols, complex processing procedures, high degree of manual coding, poor cross-domain scalability, low degree of automation, and complex system architecture, resulting in low real-time processing efficiency and high development costs.
A multi-protocol real-time data processing approach is adopted. By selecting the Kafka data source, configuring the consumption topic and data transformation rules on the transformation rule management page, the data processing process is divided into protocol parsing, data filtering, data deduplication and indicator aggregation processes using a real-time processing process model. Data modeling and entity transformation are performed using a rule configuration combination model and multiple transformation rule modeling models. Finally, data processing is implemented on the Flink real-time computing engine.
It enables real-time data processing across industries and domains using multiple protocols, simplifies the development process, reduces the need for manual coding, improves code reusability and project development efficiency, and ensures sub-second message latency for the system.
Smart Images
Figure CN117370438B_ABST