Production and consumption method based on kafka cluster

A cluster and message technology, which is applied in the direction of response error generation, hardware redundancy for data error detection, and inter-program communication. It can solve problems such as single point failures, avoid message loss, and eliminate data fluctuations. The effect of solving data consistency and single point of failure problems

Pending Publication Date: 2020-11-13
银盛支付服务股份有限公司
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0015] In order to overcome the deficiencies of the prior art, the present invention provides a production and consumption method based on kafka clusters. By setting up multiple servers, the method eliminates the situation of data fluctuation caused by the instability of a single server, and solves data consistency and single-point failure problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Production and consumption method based on kafka cluster
  • Production and consumption method based on kafka cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0034] The idea, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, features and effects of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative efforts belong to The protection scope of the present invention. In addition, all the connection / connection relationships involved in the patent do not simply refer to the direct connection of components, but mean that a better connection structure can be formed by adding or reducing connection accessories according to specific implementat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a production and consumption method based on a kafka cluster, and relates to the technical field of information processing. The method comprises the following steps: S1, setting a message submission rule; S2, setting an automatic message retry rule; S3, setting the number of copies of the theme topic, and setting the number of copies of the theme topic to be greater than orequal to 3; S4, setting the number of message write-in copies, and ensuring that the number of copies of the topic is greater than the number of message write-in copies; S5, submitting the offset Offset after ensuring that the message consumption is finished; S6, setting threads of a plurality of servers. The method is characterized in that the partition number of multiple data topics is counted,the following formula is adopted for calculation: P/n = concurrence, P is the data of the total partition number, n is the number of servers, and concurrence is the number of threads; the method hasthe advantages that data fluctuation caused by instability of a single server is completely eradicated, and the problems of data consistency and single-point faults are solved.

Description

technical field [0001] The present invention relates to the technical field of message processing, and more specifically, the present invention relates to a production and consumption method based on a kafka cluster. Background technique [0002] Kafka is a distributed publish-subscribe messaging system with features such as high throughput, persistence, replica set mechanism, and distributed support for horizontal expansion. The design of producers and consumers is completely decoupled, traffic can be peak-shaving, and message processing can be asynchronous. It is now used by many companies as various types of data pipelines and messaging systems. [0003] In order to better illustrate and understand the technical solution of the present invention, the basic concept of Kafka is introduced as follows: [0004] 1. Producers and consumers [0005] For Kafka, there are two basic types of clients, including: producers (Producer), consumers (Consumer), producers (also known as...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/54G06F11/20
CPCG06F9/546G06F11/2023G06F2209/548
Inventor 曹浩刘跃红夏菊平李雨峰
Owner 银盛支付服务股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap