Distributed message system

A message system and distributed technology, applied in transmission systems, digital transmission systems, electrical components, etc., can solve problems such as server-side performance bottlenecks

Active Publication Date: 2017-12-12
杭州多麦电子商务股份有限公司
View PDF4 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Not only the message itself but also the message consumption information must be strictly syn...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed message system
  • Distributed message system
  • Distributed message system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0022] like image 3 Shown is the distributed message system framework of the present invention, including a production cluster for generating topic message data, a service cluster for storing topic message data, a consumption cluster for consuming topic message data, and a management cluster for managing consumption clusters and service clusters. . The service cluster includes topic partitions for storing topic message data. The topic partition includes the master copy (Leader) and the slave copy (Follower) distributed in different servers of the service cluster, and the slave copy is a redundant backup of the master copy. A service cluster consists of one or more Agent servers (servers), each of which is responsible for external services or data redundancy for several topic partitions. The server provides external message submission / consumption / partition addressing functions. One of the servers (Agents) in the same topic partition is the aforementioned master copy, and the...

Embodiment 2

[0068] For the sake of brevity, the parts of this embodiment that are the same as those of the first embodiment will not be repeated here, and only the different parts of the second embodiment and the first embodiment will be described.

[0069] In this embodiment, there are two storage modes for topic message data on the server. One is the same file mode as the first embodiment, and the other is the memory mode. The file mode is suitable for storing message data with a long expiration time and a large volume, including logs, mysql binlog, and simple activity information of applications, etc.

[0070] The memory mode is suitable for simple fast message delivery. These messages usually have a short life cycle (maybe only a few minutes), require small transmission delays, and need to reach the consumer relatively quickly. It is inconvenient to store this type of topic message data in a file. First of all, file storage is designed for messages with a relatively long life cycle,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of message processing, and specifically relates to a distributed message system. The distributed message system comprises a production cluster used for generating theme message data, a service cluster used for storing the theme message data, a consumption cluster used for consuming the theme message data, and a management cluster used for managing the consumption cluster and the service cluster. The service cluster comprises a theme subarea used for storing the theme message data, the theme subarea comprises a master transcript and a salve transcript, which are distributed in different servers of the service cluster, and the salve transcript is redundant backup of the master transcript; a production end in the production cluster accesses the master transcript of the theme subarea to store the generated theme message data, and a consumption end in the consumption cluster accesses the salve transcript of the theme subarea to consume the message data of the theme subarea. During consumption, the consumption end only extracts data from the master transcript of the theme subarea for consumption, and if the consumption message is normal, the demand of synchronizing to the salve transcript does not exists, thereby relieving the burden of the server.

Description

technical field [0001] The invention relates to the field of message processing, in particular to a distributed message system. Background technique [0002] There are many message systems currently used for message (such as log) processing, and the most popular is the distributed message system. [0003] Distributed messaging system frameworks such as figure 1 shown. Including Producer (message producer, referred to as PD), Agent (message buffer), Consumer (message processor, referred to as CS) and third-party management cluster, each role can have multiple. The Producer sends the message to the Agent, the message is persistently stored on the Agent, and the Consumer obtains the message from the Agent for processing. The third-party management cluster is used to store some state information of the Producer, Consumer and Agent. [0004] Distributed message systems manage messages based on message topics. In the message storage device, it is also stored based on the topi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08H04L12/26
CPCH04L43/10H04L67/10H04L67/1095H04L67/1001
Inventor 胡悦吴文龙
Owner 杭州多麦电子商务股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap