Unlock instant, AI-driven research and patent intelligence for your innovation.

Monitoring method and related equipment for topic data of kafka cluster

A cluster and data technology, applied in the field of data processing, to achieve the effect of enhancing the amount of data changes

Active Publication Date: 2022-03-25
PING AN TECH (SHENZHEN) CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Based on this, it is necessary to provide a monitoring method and related equipment for the topic data of the Kafka cluster in view of the technical problems of the current Kafka monitoring tools in terms of topic variation, multi-cluster management, and historical data storage.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Monitoring method and related equipment for topic data of kafka cluster
  • Monitoring method and related equipment for topic data of kafka cluster
  • Monitoring method and related equipment for topic data of kafka cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048]In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0049] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first threshold could be referred to as a second threshold, and, similarly, a second threshold could be referred to as a first threshold, without departing from the scope of this application.

[0050] figure 1 An overall flow chart of a method for monitoring topic data of a Kafka cluste...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application relates to the technical field of data processing, in particular to a method for monitoring topic data of a kafka cluster and related equipment. The method for monitoring the topic data of the kafka cluster includes: accessing all kafka clusters, obtaining topic data in all said kafka clusters; classifying and storing the acquired topic data according to the unique identifier of said kafka cluster, and in the form of charts Displaying the topic data; monitoring the latest stored topic data, and sending an alarm prompt when the latest stored topic data reaches an alarm threshold. This technical solution can monitor multiple Kafka clusters at the same time, and the topic data is presented in a dynamic form, providing traceable historical topic data.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular, to a method and related equipment for monitoring topic data of a Kafka cluster. Background technique [0002] With the wide application of Kakfa in stream processing, there may be hundreds or thousands of topics in Kafka, and there may be one or more Kafka clusters. Therefore, the data changes of each topic in Kafka and the consumption progress of consumers monitoring is increasingly important. [0003] Currently, the more commonly used Kafka monitoring tools on the market are KafkaOffsetMonitor and KafkaManager. Among them, KafkaOffsetMonitor can obtain relevant data such as the change amount and consumption progress of each consumer's corresponding topic, but it can only monitor a single kafka cluster, and cannot monitor the change amount of topics that have not been consumed; although KafkaManager can manage multiple clusters , but the related data of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L43/045H04L43/16H04L41/0681H04L67/10
Inventor 吴善鹏周敏
Owner PING AN TECH (SHENZHEN) CO LTD