Message sending method and device based on Kafka system

A technology for message and message distribution, applied in transmission systems, multi-program devices, inter-program communication, etc., can solve the problems of repeated message transmission, loss, etc.

Active Publication Date: 2020-10-23
CHINA MOBILE GROUP SICHUAN +1
View PDF17 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] How to avoid repeated sending or message loss in the process of s

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Message sending method and device based on Kafka system
  • Message sending method and device based on Kafka system
  • Message sending method and device based on Kafka system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention. The drawing numbers in this application are only used to distinguish each step in the solution, and are not used to limit the execution order of each step. The specific execution order is subject to the description in the specification.

[0026] First, introduce the Kafka system structure involved in this solution. Such as figure 1 As shown, the Kafka system usually includes Producer, Server Broker, Block Partition, and Consumer. Wherein, the server Br...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a message sending method and device based on Kafka. The method and device are used for avoiding the situation of repeated sending or message loss in amessage sending process. According to the scheme provided by the invention, the method comprises the following steps: allocating a message sent by a Producer to at least one server Broker in a cluster through a partition device; monitoring the allocation completion degree of the message allocated to the at least one server through a monitor; and when the distribution completion degree meets a preset completion degree standard, indicating the Consumer to read the message through at least one server. According to the scheme provided by the invention, based on the Kafka system, the partition device can avoid the situationthat the same message sent by a producer is issued or missed for many times. Meanwhile, the monitor can avoid the situation that the issued message is incomplete. Only when the distribution completion degree meets the preset completion degree standard, the consumer is indicated to read the message, and it is guaranteed that the consumer can read complete information.

Description

technical field [0001] The present invention relates to the field of distributed message systems, in particular to a method and device for sending messages based on the Kafka system. Background technique [0002] Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action streaming data in consumer-scale websites. Streaming data is mostly used in web sites, and the data includes page views (PageView), information on viewed content, search conditions, and the like. [0003] In the existing Kafka message distribution mechanism, the classifier Topic obtains the producer's message and sends it to the consumer through multiple server brokers. However, in the existing message distribution mechanism, each message is only transmitted once, and once a server goes down, the message may be lost. If the message is resent when the message is lost, the consumer may receive the message repeatedly. [0004] How to avoid repeated sending or message...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F9/54
CPCH04L67/02H04L67/10G06F9/546
Inventor 杨名
Owner CHINA MOBILE GROUP SICHUAN
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap