Unlock instant, AI-driven research and patent intelligence for your innovation.

Real-time streaming data processing method, system and equipment and storage medium

A data processing and data storage technology, applied in the computer field, can solve the problems of lack of stream data consistency, high availability, and high design complexity, and achieve the effects of convenient consumption speed, maintaining consistency, and preventing memory overflow

Pending Publication Date: 2022-04-29
北京滴普科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, this method has the following disadvantages: it needs to implement its own set of serialization and deserialization design, the design complexity of the whole process is high, and it lacks the consistency and high availability of stream data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time streaming data processing method, system and equipment and storage medium
  • Real-time streaming data processing method, system and equipment and storage medium
  • Real-time streaming data processing method, system and equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0051] In the description of the present invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc. indicated orientations or positional relationships are based on the orientations or positional relationships shown in the drawings, and are only In order to facilitate the description of the present invention and simplify the description, it does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a real-time streaming data processing method, system and device and a storage medium. The real-time streaming data processing method relates to the technical field of computers, and comprises the following steps: obtaining preset initial SQL statement data; preprocessing the initial SQL statement data to obtain target SQL statement data; performing analysis processing on the target SQL statement data to obtain table mode data; and inputting the table mode data into a preset Flink framework for conversion processing to generate real-time streaming data, and storing the real-time streaming data in preset middleware. Through the embodiment disclosed by the invention, the initial SQL statement data can be pre-processed to maintain the consistency and high availability of the data, so that the real-time streaming data stored in the preset middleware is generated by utilizing the Flink framework.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a real-time streaming data processing method, system, device and storage medium. Background technique [0002] In related technologies, most of the streaming data obtained through the Flink platform is based on the direct communication between the service and the cluster. The real-time streaming data is serialized inside the cluster, and the data is captured or called back to the service layer through a custom protocol. By deserializing the byte stream data into the original data, the acquisition of real-time stream data is completed. [0003] However, this method has the following disadvantages: it needs to implement its own serialization and deserialization design, the design complexity of the whole process is high, and it lacks the consistency and high availability of stream data. Contents of the invention [0004] The present invention aims to solve at least one ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F16/242
CPCG06F16/24568G06F16/2433
Inventor 贺子江杨磊
Owner 北京滴普科技有限公司