Stream computing processing method, equipment and system

A processing method and stream computing technology, applied in the field of communication, can solve the problems of reduced concurrent access performance, achieve the effect of improving concurrent access performance, saving disk I/O, and avoiding the limitation of the number of access connections

Inactive Publication Date: 2014-12-31
ALIBABA GRP HLDG LTD
View PDF3 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0013] The present application provides a stream computing processing method, device and system, which can solve the problem of upstream applications directly accessing data from disks while implementing persistent operations in the process of consumption of data streams by existing stream computing technologies. Issues that degrade concurrent access performance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stream computing processing method, equipment and system
  • Stream computing processing method, equipment and system
  • Stream computing processing method, equipment and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] As mentioned in the background technology, in the existing flow computing process, on the one hand, due to the requirements of fault tolerance and fault recovery, persistent processing is required. In the prior art, it can be realized through caching strategies. However, another On the one hand, providing real-time data query services through the database may lead to excessive database access and affect I / O performance, but the existing caching strategy cannot solve such problems well.

[0050] In order to solve such problems, this application proposes a flow computing processing method, so that upstream applications do not directly deal with the database, avoiding the limitation of the number of database access connections, and saving a lot of disk I / O, thereby achieving a significant increase in concurrent access performance effect.

[0051] like image 3 As shown, it is a schematic flowchart of a stream computing processing method provided in the embodiment of the p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a stream computing processing method, equipment and system. By applying the technical scheme provided by the embodiment of the application, in the process that a distributed stream computing cluster processes the current batch of information, intermediate computing detail data of the current batch of information is stored to a database for performing the persistence operation; meanwhile, under the premise of meeting the persistence operation requirement and realizing fault tolerance and fault recoverability guarantee, a query operation performed by an upstream application on a real-time computing result is provided by a memory, and the trend of data flow is redefined, so that the upstream application does not directly come into contact with the database, and the limitation of database access linking number is avoided; meanwhile, a significant amount of disk I/O caused by frequent access to the database is saved, and the effect of greatly enhancing a concurrent access performance is realized.

Description

technical field [0001] The present application relates to the communication field, and in particular, to a stream computing processing method, device and system. Background technique [0002] Stream computing (Stream Computing) products are mainly used for real-time computing. like figure 1 As shown in , it is a schematic diagram of the most basic stream computing data flow in the prior art. [0003] exist figure 1 In the scenario shown, the message flow sent by the message middleware is one by one, and usually the flow computing product consumes the message flow in batches, that is, several messages are bundled into one batch, and one batch The number of message streams can be adjusted. [0004] In order to meet the requirements of order preservation and fault tolerance, the flow computing framework designs a general consumption strategy: only after the consumption of a batch is completely successful and the message sending component responds, the next batch of messages...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F12/02H04L29/08
CPCG06F16/2219G06F16/24539G06F16/2471H04L67/10
Inventor 刘健男
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap