Multi-data-stream processing method and device, terminal and storage medium

A technology with multiple data streams and processing methods, applied in the field of data processing, can solve the problems of low accuracy and completeness, and achieve the effect of ensuring completeness and accuracy

Pending Publication Date: 2021-02-09
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] This application provides a multi-data stream processing method, device, terminal and storage medium, which has solved the problem of low accuracy and completeness in the existing Flink when processing multiple data streams

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-data-stream processing method and device, terminal and storage medium
  • Multi-data-stream processing method and device, terminal and storage medium
  • Multi-data-stream processing method and device, terminal and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0025] The terms "first", "second", and "third" in this application are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, features defined as "first", "second", and "third" may explicitly or implicitly include at least one of these features. In the description of the present application, "plurality" means at least t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-data-stream processing method and device, a terminal and a storage medium, and the method comprises the steps: receiving a plurality of data streams transmitted by message middleware, and obtaining the type of each data stream; extracting a first data stream of which the type is insertion in the plurality of data streams, and inserting the first data stream into a pre-constructed final result table by utilizing an Flink window function; extracting a second data stream of which the type is not inserted in the plurality of data streams, constructing a synchronization table, and inserting the second data stream into the synchronization table by utilizing an Flink window function; and combining the synchronization table with a pre-constructed final result tableto update corresponding fields in the final result table to obtain a new final result table. By means of the mode, under the condition that data integrity and accuracy are guaranteed, the Flink is used for processing multiple inserted or updated data volumes sent by the message-oriented middleware.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a multi-data stream processing method, device, terminal and storage medium. Background technique [0002] In the field of data processing, data processing methods can be divided into batch processing and stream processing in a certain way. Batch processing is about packaging a certain amount of data for unified processing. The calculation time is measurable and bounded, while stream processing is It is non-stop calculation, once the program is started, it will not stop, and the amount of data and time-consuming processing tasks will increase infinitely. [0003] The Flink framework is widely used in the field of streaming data and batch data processing, because Flink can be executed on a large scale, has very high throughput, and can provide correct results even in the case of out-of-order or data delays. Benefiting from Flink's excellent performance and exactl...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F16/23G06F16/27
CPCG06F16/2365G06F16/24568G06F16/27
Inventor 许浩奇
Owner CHINA PING AN PROPERTY INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap