Distributed message processing method and system

A message processing and distributed technology, applied in the direction of electrical digital data processing, communication between programs, instruments, etc., can solve problems such as high cost, high cost, and inability to guarantee the order of message delivery

Active Publication Date: 2022-03-04
ZHEJIANG CHUANGLIN TECH CO LTD
View PDF18 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the existing message processing system is too specialized / complex or too simple, such as Microsoft's MSMQ is an industrial-grade message processing system with powerful functions and hard delivery guarantees, but its deployment requires Microsoft's proprietary system, which is costly; such as Amazon Provides a low-cost Simple Queue Service (Simple Queue Service, SQS), which only has basic messaging operations (send / receive), and cannot guarantee the order of message delivery
[0004] Therefore, when providing Hadoop users with a message queue function that guarantees the order of message delivery, it is often necessary to additionally deploy an overly specialized / complex message processing system, which is too costly and will increase the complexity of the corresponding application.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed message processing method and system
  • Distributed message processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0049] Embodiment 1. A distributed message processing method, such as figure 1 shown, including the following steps:

[0050] S100. Receive a message sending request sent by the client 200, generate a message including a sending time stamp according to the message sending request, and add the message to a corresponding message queue, where the sending time stamp is unique in the message queue , arranging the messages in the message queue based on the sending timestamp;

[0051] In this embodiment, the message line is written in the corresponding message queue based on the message sending request in advance, and then the corresponding timestamp management node 300 is requested to send the timestamp, and the obtained sending timestamp is inserted into the message line to complete the generation of the message. mark it as sent, so that the sending of the message is completed based on the message sending request, and this process is implemented based on the atomic method.

[005...

Embodiment 2

[0095] Embodiment 2. A distributed message processing system, such as figure 2 As shown, it is connected to the distributed file system 400, including several message management nodes 100, several time stamp management nodes 300 and several clients 200;

[0096] The message management node 100 includes:

[0097] The message publishing module is configured to receive a message sending request sent by the client 200, generate a message including a sending time stamp according to the message sending request, and add the message to a corresponding message queue, and the sending time stamp is determined by the corresponding time stamp Issued by the management node 300, the sending time stamp is unique in the message queue; the message publishing module 110 is also configured to arrange the messages in the message queue based on the sending time stamp;

[0098] Overtime judging module, for judging whether the message is overtime, marking the overtime message according to the judgm...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed message processing method and system, and the method comprises the following steps: receiving a message sending request sent by a client, generating a message containing a sending timestamp according to the message sending request, adding the message to a corresponding message queue, the sending timestamp having uniqueness in the message queue, and sending the message to the client according to the message queue. Arranging messages in a message queue based on the sending timestamps; judging whether the message is overtime or not, marking an overtime message according to a judgment result, and backing up the overtime message to a distributed file system; and periodically deleting the overtime messages in the message queue according to preset clearing time. According to the invention, the sequence of message transmission is ensured through the design of the sending timestamps; through the design of judging and clearing the overtime message, the distributed file system and the message queue can be matched with each other, so that the data storage pressure brought by the message queue is relieved while the client queries the message.

Description

technical field [0001] The invention relates to the technical field of computer software, in particular to a distributed message processing method and system. Background technique [0002] The Hadoop platform is the most famous big data platform. Today's big data processing technology stacks are all based on Hadoop, and most applications based on the big data platform need to process distributed messages; [0003] However, the existing message processing system is too specialized / complex or too simple, such as Microsoft's MSMQ is an industrial-grade message processing system with powerful functions and hard delivery guarantees, but its deployment requires Microsoft's proprietary system, which is costly; such as Amazon Provides a low-cost Simple Queue Service (Simple Queue Service, SQS), which only has basic message delivery operations (send / receive), and cannot guarantee the order of message delivery. [0004] Therefore, when providing Hadoop users with a message queue func...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/54
CPCG06F9/546G06F2209/548G06F2209/547G06F2209/541
Inventor 张晨
Owner ZHEJIANG CHUANGLIN TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap