Message queue backlog alarm method and device, and medium

Abstract

The invention discloses a message queue backlog alarm method and device and a medium, and the method comprises the steps: a monitoring alarm service sends an information query instruction to each distributed message queue cluster through a user-defined packaged application program interface; receiving meta-information responded by each distributed message queue cluster, and determining message backlog information corresponding to the distributed message queue cluster based on the meta-information; querying user configuration information of related users based on the service management platform; and based on the meta-information, the message backlog information and the user configuration information, sending alarm information to a corresponding related user. If the theme information and the consumption information in the message queue are abnormal, the configuration of the related user can be obtained based on the service management platform, so that the alarm information is generated and sent to the related user, continuous monitoring and rapid alarm of the message queue are realized, and stable operation of the message queue is ensured.

Description

[0001] This application claims priority based on the invention patent application filed with the China National Patent Office on December 29, 2023, with the application number 202311866027.4 and the invention title "A Method, Device, and Medium for Message Queue Backlog Warning". The entire content of the above-mentioned Chinese patent application is incorporated herein by reference. Technical Field

[0002] This application relates to the field of computers, and specifically to a method, device, and medium for message queue backlog warning. Background Art

[0003] In a microservices architecture system, a message queue (MQ) is an essential and crucial part. It refers to middleware that stores and distributes business event messages to consumers, aiming to decouple different systems or different modules.

[0004] When consumers consume messages, if the consumption speed cannot keep up with the production speed, message backlogs will occur. When there are large-scale message backlogs, it means that the business has been severely affected. For example, it may lead to business interruptions, user complaints, etc.

[0005] Based on this, monitoring message queue backlogs is particularly important. Summary of the Invention

[0006] To solve the above problems, this application proposes a method for message queue backlog warning, including:

[0007] Trigger a monitoring and warning task through a scheduled task component;

[0008] The monitoring and warning service sends an information query instruction to each distributed message queue cluster through a custom encapsulated application programming interface;

[0009] Receive the meta-information responded by each distributed message queue cluster, where the meta-information includes: topic information and consumption information;

[0010] Query the user configuration of relevant users based on the service management platform;

[0011] Based on the topic information and consumption messages, determine the warning information to be sent, and send the warning information to the relevant users based on the user configuration through the API opened by a preset office program.

[0012] In one example, before receiving the meta-information responded by each distributed message queue cluster, the method further includes:

[0013] Define the relevant functions of the interfaces corresponding to each distributed message queue cluster;

[0014] Among them, the related functions include a configuration acquisition function, a consumer thread control function, a consumer acquisition function, a partition acquisition function, a topic acquisition function, a topic information acquisition function, and a consumption information acquisition function.

[0015] In one example, the configuration acquisition function acquires the configuration of the message queue in the service management platform, where the return type is a subclass of the basic message queue class, and the specific configuration type is specified through generic parameters;

[0016] The consumer thread control function starts the consumer thread of the corresponding message queue and shuts down the consumer thread of the corresponding message queue;

[0017] The consumer acquisition function acquires all consumer IDs through the returned list containing consumer IDs;

[0018] The partition acquisition function acquires all partition IDs consumed by the consumer through the list containing partition IDs returned by the parameters corresponding to the consumer ID;

[0019] The topic acquisition function acquires the topic ID corresponding to the consumer through the list containing the topic ID returned by the parameters corresponding to the consumer ID;

[0020] The topic information acquisition function acquires the topic information through the list containing the topic information corresponding to the partition ID returned by the parameters corresponding to the topic ID;

[0021] The consumption information acquisition function acquires the consumption information through the list containing the consumption information returned by the parameters corresponding to the consumer ID and the topic ID.

[0022] In one example, the topic information corresponds to a single topic, each topic corresponds to multiple node partitions, the partition ID corresponds to a single node partition, and all node partitions form a node cluster;

[0023] The distributed message queue cluster is connected to and corresponds to the node cluster and the service management platform;

[0024] The topic information includes: cluster ID, topic ID, partition ID, topic location, and the consumption information includes: consumption ID, consumption location.

[0025] In one example, the distributed message queue cluster includes: multiple Kafka clusters, multiple RockerMQ clusters;

[0026] The monitoring and alerting service sends information query instructions to each distributed message queue cluster through a custom encapsulated application programming interface, specifically including:

[0027] The monitoring and alarming service sends information query instructions to each Kafka cluster and each RocketMQ cluster respectively through the custom encapsulated kafka - client - api and rocker - client - api.

[0028] In one example, based on the topic information and the consumed messages, determining the alarming information to be sent specifically includes:

[0029] Based on the topic information and the consumed messages, the determined alarming information to be sent includes: alarm time, alarm cluster, alarm topic, consumption ID, queue alarm threshold, queue production position, queue consumption position, and queue backlog situation.

[0030] In one example, the method further includes:

[0031] Displaying the alarming information to the relevant users through the office program, where when displaying the alarming information, a preset part of the alarming information is rendered and displayed;

[0032] Obtaining the acquisition status of the relevant users for the alarming information through the API opened by the office program, where the acquisition status includes: unread status, read status, and replied status;

[0033] If it is determined that the acquisition status is in the preset status for more than a preset duration, then through the office program, alarm the relevant users in other ways, and the other ways at least include: voice call, video call.

[0034] In one example, before determining that the acquisition status is in the preset status for more than a preset duration, the method further includes:

[0035] When the number of messages in the queue backlog does not exceed a preset number, determining that the preset status includes the unread status, and based on the queue backlog situation, determining the first preset duration corresponding to the unread status, where the more the number of messages in the queue backlog, the shorter the first preset duration;

[0036] When the number of messages in the queue backlog exceeds a preset number, determining that the preset status includes the unread status and the read status, and based on the queue backlog situation, determining the first preset duration corresponding to the unread status and the second preset duration corresponding to the read status, and the second preset duration is longer than the first preset duration.

[0037] For unread or unreplied situations, other methods can be set for prompting to ensure that relevant users can indeed see this alarming information.

[0038] For different degrees of message queue backlog, corresponding preset durations can be set to represent the urgency of this time, thereby enhancing the attention of relevant users to this troubleshooting and repair. Moreover, by setting different preset durations, if the relevant users have completed the troubleshooting and repair, there is no need to use other methods for prompting, thus ensuring the user experience of relevant users.

[0039] On the other hand, this application also proposes a method for message queue backlog warning, including:

[0040] The monitoring and warning service sends information query instructions to each distributed message queue cluster through a custom encapsulated application programming interface;

[0041] Receive the meta-information responded by each distributed message queue cluster, and determine the message backlog information corresponding to the distributed message queue cluster based on the meta-information;

[0042] Query the user configuration information of relevant users based on the service management platform;

[0043] Send warning information to the corresponding relevant users based on the meta-information, the message backlog information, and the user configuration information.

[0044] In one example, the method further includes:

[0045] Trigger the monitoring and warning task through the timed task component, so that the monitoring and warning service sends information query instructions to each distributed message queue cluster.

[0046] In one example, the distributed message queue cluster includes multiple clusters, each cluster corresponds to multiple topics, each topic corresponds to one or more partitions, and the message queue is formed by the topics; the cluster types include at least one of Kafka cluster and RockerMQ cluster.

[0047] In one example, the monitoring and warning service sends information query instructions to each distributed message queue cluster through a custom encapsulated application programming interface, specifically including:

[0048] The monitoring and warning service determines the custom encapsulated application programming interface;

[0049] Through the application programming interface, determine the cluster identifier corresponding to the distributed message queue cluster to be queried, and determine its corresponding cluster type according to the cluster identifier;

[0050] In the Kafka cluster or RockerMQ cluster corresponding to the cluster type, select the corresponding cluster according to the cluster identifier and send the information query instruction.

[0051] In one example, determining the message backlog information corresponding to the distributed message queue cluster based on the meta information specifically includes:

[0052] Based on the meta information, determining the production position and consumption position of the corresponding message queue in the distributed message queue cluster;

[0053] Based on the production position and the consumption position, obtaining the number of backlogged messages, and determining the number of backlogged messages as the message backlog information.

[0054] And / or,

[0055] Based on the meta information, the message backlog information, and the user configuration information, sending an alarm message to the corresponding relevant users, specifically including:

[0056] Based on the user configuration information, obtaining the queue alarm threshold preset for the distributed message queue cluster;

[0057] If the number of backlogged messages is higher than the queue alarm threshold, generating an alarm message based on the meta information, the number of backlogged messages, and the user configuration information;

[0058] Based on the user configuration information, sending the alarm message to the corresponding relevant users.

[0059] In one example, generating an alarm message based on the meta information, the number of backlogged messages, and the user configuration information specifically includes:

[0060] Based on the number of backlogged messages, the user configuration information, and the production position, consumption position, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and current timestamp included in the meta information, generating an alarm message;

[0061] Or, determining the message backlog degree based on the number of backlogged messages and a preset message backlog threshold, and determining the message backlog situation based on the message backlog degree and the number of backlogged messages; based on the message backlog situation, the user configuration information, and the production position, consumption position, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and current timestamp included in the meta information, generating an alarm message;

[0062] And / or, based on the user configuration information, sending the alarm message to the corresponding relevant users, specifically including:

[0063] Obtaining the user information of the relevant users according to the user configuration, and sending the alarm message to the account corresponding to the user information in the third-party program through the API interface of the pre-obtained third-party program.

[0064] In one example, before receiving the meta-information of the responses of each distributed message queue cluster, the method further includes:

[0065] Define the relevant functions of the interfaces and classes corresponding to each distributed message queue cluster; wherein, the interfaces and classes include at least one of the following: MqSoureFactory class, MQSource interface, BaseMQAlarmService class, MessageAssemble class, Sendable interface, MQAlarmService class;

[0066] The MqSoureFactory class is used to create a distributed message queue cluster, instantiate the distributed message queue cluster, and implement the underlying connection function of the distributed message queue cluster;

[0067] The MQSource interface is used to abstract the distributed message queue cluster and define the interface functions of the API interface;

[0068] The BaseMQAlarmService class is used to implement the actions of the distributed message queue cluster and interface calls;

[0069] The MessageAssemble class is used to encapsulate relevant metadata into objects;

[0070] The Sendable interface is used to define an alarm interface and transmit alarm information;

[0071] The MQAlarmService class is used to start an alarm task;

[0072] Preferably, the interface functions include at least one of the following: configuration acquisition function, consumer thread control function, consumer acquisition function, partition acquisition function, topic acquisition function, topic information acquisition function, consumption information acquisition function; wherein, the meta-information further includes: topic information and consumption information;

[0073] The configuration acquisition function is used to acquire the configuration of the message queue in the service management platform, wherein the return type of the configuration acquisition function is a subclass of the basic message queue class, and the specific configuration type is specified through a generic parameter;

[0074] The consumer thread control function is used to start the consumer thread of the corresponding message queue and close the consumer thread of the corresponding message queue;

[0075] The consumer acquisition function is used to acquire all consumer identifiers through the returned list containing consumer identifiers;

[0076] The partition acquisition function is used to obtain all partition identifiers consumed by the consumer by means of a list containing partition identifiers returned according to parameters corresponding to the consumer identifier;

[0077] The topic acquisition function is used to obtain the topic identifier corresponding to the consumer by means of a list containing topic identifiers returned according to parameters corresponding to the consumer identifier;

[0078] The topic information acquisition function is used to obtain the topic information by means of a list containing topic information corresponding to partition identifiers returned according to parameters corresponding to the topic identifier;

[0079] The consumption information acquisition function is used to obtain the consumption information by means of a list containing consumption information returned according to parameters corresponding to the consumer identifier and the topic identifier.

[0080] In one example, the method further includes:

[0081] Display the alarm information to the relevant user in text form through the third-party program, and when displaying the alarm information, render and display a preset part of the alarm information;

[0082] Obtain the reading status of the relevant user for the alarm information through the API interface of the third-party program, where the reading status includes: unread status, read status, and replied status;

[0083] If the reading status remains in a preset status for more than a preset duration, alarm the relevant user in a form other than the text form according to the alarm information;

[0084] Wherein, if the number of messages to be processed in the message backlog information does not exceed a preset number, the preset status includes the unread status, and if the number of messages to be processed in the message backlog information exceeds the preset number, the preset status includes the unread status and the read status.

[0085] On the other hand, the present application also proposes a message queue backlog alarm device, including:

[0086] At least one processor; and,

[0087] A memory communicatively connected to the at least one processor; wherein,

[0088] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the message queue backlog alarm method as described in any of the above examples.

[0089] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured as: the message queue backlog warning method described in any of the above examples.

[0090] The message queue backlog warning method proposed by the present application can bring the following beneficial effects:

[0091] Through the timing task component, the scheduling and triggering of tasks can be performed to ensure the effective operation of the monitoring and warning tasks. Through the monitoring and warning service, the meta-information of each distributed message queue cluster can be obtained at all times. If the topic information and consumption information are abnormal, the relevant user configurations can be obtained based on the service management platform, so as to generate and send the warning information to the relevant users, realizing the continuous monitoring and rapid alarm of the message queue, ensuring the stable operation of the message queue, and reducing the possibility of negative impacts on the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0093] Figure 1 is a schematic flowchart of the message queue backlog warning method in an embodiment of the present application;

[0094] Figure 2 is a schematic diagram of the roles of the message queue components in the traditional solution in an embodiment of the present application;

[0095] Figure 3 is a schematic diagram of the message queue backlog in the traditional solution in an embodiment of the present application;

[0096] Figure 4 is a schematic flowchart of the message queue backlog warning method in a certain situation in an embodiment of the present application;

[0097] Figure 5 is a schematic diagram of the interface for displaying warning information in a certain situation in an embodiment of the present application;

[0098] Figure 6 is a schematic diagram of the message queue backlog warning device in an embodiment of the present application;

[0099] Figure 7 is a schematic flowchart of the message queue backlog warning method in another situation in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0100] To make the objectives, technical solutions, and advantages of this application clearer, the following will clearly and completely describe the technical solutions of this application in combination with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0101] The following will, with reference to the drawings, elaborate on the technical solutions provided by each embodiment of this application.

[0102] As Figure 1 and Figure 7 shown, an embodiment of this application provides a method for warning about message queue backlog, including:

[0103] S101: Trigger a monitoring and warning task through a timed task component.

[0104] As Figure 2 shown, in the traditional solution, the message queue component usually includes three roles: a producer that produces messages, a message queue that stores and distributes messages, and a consumer that consumes messages. Once the consumption speed cannot keep up with the production speed, a message backlog situation as Figure 3 shown will occur.

[0105] Based on this, a timed task component is set to trigger a monitoring and warning task. Among them, as Figure 4 shown, the timed task component can be XXL-JOB. XXL-JOB is an open-source distributed task scheduling platform used to solve large-scale distributed task scheduling problems. It provides a complete set of task scheduling and task management solutions, supporting the scheduling execution of timed tasks and resident tasks. XXL-JOB has functions such as a visual task management interface, distributed task scheduling execution, and task log viewing. At the same time, it also provides advanced features such as task execution alarm and task failure retry, which can help developers easily implement task scheduling and management.

[0106] S102: The monitoring and warning service sends an information query instruction to each distributed message queue cluster through a custom encapsulated application programming interface.

[0107] The application programming interface can be called client-api. As Figure 4 shown, the distributed message queue cluster includes: multiple Kafka clusters (for example, Figure 4 the Kafka cluster 1 to Kafka cluster 3 shown in Figure 4 ), multiple RockerMQ clusters (for example,

[0108] Among them, Kafka and RocketMQ are both distributed message queue systems. The design goal of Kafka is to build a high-performance, persistent, and distributed publish-subscribe message system. It adopts a distributed log storage model, appends messages to the log in a high-throughput manner, and supports batch read and write operations. The design goal of RocketMQ is to build a highly available, high-performance, and scalable message middleware. It adopts a master-slave architecture, supports ordered message transmission and transactional messages, and has good fault tolerance.

[0109] At this time, the monitoring and alerting service sends information query instructions to each Kafka cluster and each RocketMQ cluster respectively through the custom encapsulated kafka-client-api and rocker-client-api.

[0110] Similarly, as Figure 7 shown, in step S701, it can also be achieved that the monitoring and alerting service sends information query instructions to each distributed message queue cluster through the custom encapsulated application programming interface.

[0111] As Figure 4 shown, the distributed message queue cluster includes multiple clusters, each cluster corresponds to multiple topics, each topic corresponds to one or more partitions, and the message queue is formed through the topics; the cluster types include at least one of Kafka cluster and RocketMQ cluster.

[0112] Furthermore, when sending the information query instruction, the monitoring and alerting service determines the custom encapsulated application programming interface, and then through the application programming interface, determines the cluster identifier corresponding to the distributed message queue cluster to be queried, and determines its corresponding cluster type according to the cluster identifier. For example, the staff defines the distributed message queue cluster to be queried, and in it, uses the cluster identifier to indicate which clusters need to be queried, and different cluster identifiers represent different clusters, and different clusters have their own corresponding cluster types.

[0113] At this time, in the Kafka cluster or RocketMQ cluster corresponding to the cluster type, select the corresponding cluster according to the cluster identifier and send the information query instruction.

[0114] S103: Receive the meta-information responded by each distributed message queue cluster, and the meta-information includes: topic information and consumption information.

[0115] Before receiving the meta information, the relevant functions of the interfaces and classes corresponding to each distributed message queue cluster can be defined, which facilitates the subsequent sending of information query instructions, as well as the acquisition and return of meta information. Among them, the interfaces and classes include at least one of the following: MqSoureFactory class, MQSource interface, BaseMQAlarmService class, MessageAssemble class, Sendable interface, MQAlarmService class.

[0116] The MqSoureFactory class is used to create a distributed message queue cluster, instantiate the distributed message queue cluster, and implement the underlying connection function of the distributed message queue cluster.

[0117] The MQSource interface is used to abstract the distributed message queue cluster and define the interface functions of the API interface.

[0118] The BaseMQAlarmService class is used to implement the actions of the distributed message queue cluster and interface calls.

[0119] The MessageAssemble class is used to encapsulate relevant metadata into objects.

[0120] The Sendable interface is used to define an alarm interface and transmit alarm information.

[0121] The MQAlarmService class is used to start an alarm task.

[0122] Among them, the interface functions include at least one of the following: configuration acquisition function, consumer thread control function, consumer acquisition function, partition acquisition function, topic acquisition function, topic information acquisition function, consumption information acquisition function. Among them, the meta information also includes: topic information and consumption information.

[0123] Specifically, the configuration acquisition function is used to obtain the configuration of the message queue in the service management platform through the getMQConfig() method. Among them, the return type of the configuration acquisition function is a subclass of the base message queue class (BaseMQConfig class), and the specific configuration type is specified through the generic parameter <T extends BaseMQConfig>.

[0124] The consumer thread control function is used to start the consumer thread of the corresponding message queue through the start method. At this time, the consumption of messages in the MQ starts. And the consumer thread of the corresponding message queue is closed through the shutdownO method. At this time, the consumption of messages in the MQ stops.

[0125] Consumer acquisition function, which is used to obtain all consumer IDs (also known as consumer identifiers) through the listConsumer method, which can return a list containing consumer IDs. For example, the parameter listConsumerOption is set to specify the option for obtaining consumers, so as to obtain a list containing consumer IDs.

[0126] Partition acquisition function, which is used to obtain all partition IDs consumed by consumers through the listConsumerPartitionInfo method, which can return a list containing partition IDs (also known as partition identifiers or ConsumerPartitionInfo objects) corresponding to the parameter of the consumer ID (for example, consumerld). Of course, parameters for specifying options for obtaining partition information can also be added to the parameters, such as option.

[0127] Topic acquisition function, which is used to obtain the topic ID corresponding to the consumer through the listConsumerTopic method, which can return a list containing topic IDs (also known as topic identifiers) corresponding to the parameter of the consumer ID. Of course, parameters for specifying options for obtaining topic information can also be added to the parameters, such as listConsumerTopicOption.

[0128] Topic information acquisition function, which is used to obtain topic information through the listTopidnfo method, which can return a list containing topic information corresponding to the partition ID corresponding to the parameter of the topic ID (for example, topic).

[0129] Consumption information acquisition function, which is used to obtain consumption information through the listConsumerConsumelnf method, which can return a list containing consumption information corresponding to the parameters of the consumer ID and the topic ID.

[0130] Among them, as Figure 4 shown, the topic information corresponds to a single topic, each topic corresponds to multiple node partitions, the partition ID corresponds to a single node partition, and all node partitions form a node cluster. The distributed message queue cluster is connected and corresponding to the node cluster and the service management platform.

[0131] In this way, after setting up the architecture of the monitoring and alerting service, the corresponding consumption information and topic information can be obtained.

[0132] Such as Figure 4As shown, after each distributed message queue cluster receives an information query instruction, it continues to query the connected node cluster to obtain relevant meta-information and returns it to the monitoring and alerting service. The meta-information includes at least topic information and consumption information. Topic information refers to the relevant information of the topics used to organize and classify messages in the message queue system, while consumption information refers to the information provided by consumers when consuming data in the system. Topic information may include: cluster ID (also known as cluster name), topic ID (also known as topic name), partition ID (also known as topic partition), topic location. Consumption information may include: consumption ID (also known as consumption group name), consumption location.

[0133] Similarly, as Figure 7 shown, in step S702, it can be achieved that: receiving the meta-information responded by each distributed message queue cluster, and determining the message backlog information corresponding to the distributed message queue cluster based on the meta-information.

[0134] Among them, when calculating the message backlog information, it is possible to first determine the production location and consumption location of the corresponding message queue in the distributed message queue cluster based on the meta-information. At this time, the difference between the two is calculated according to the production location and consumption location, so as to obtain the number of backlogged messages, and the number of backlogged messages is determined as the message backlog information.

[0135] Of course, it is also possible to further preset the corresponding queue alert threshold in advance. This threshold can be set based on expert experience. For example, as Figure 5 set to 1000000 in. Of course, for different scenarios, this threshold can be set to different values. At this time, based on the user configuration information (i.e., user configuration), the queue alert threshold corresponding to the preset distributed message queue cluster is obtained.

[0136] Compare the message backlog quantity with the queue alert threshold. If the message backlog quantity is higher than the queue alert threshold, it is considered that the current message backlog is relatively serious. Generate an alert message based on the meta-information, message backlog quantity and user configuration information, and then send the alert message to the corresponding relevant users based on the user configuration information.

[0137] S104: Query the user configuration of relevant users based on the service management platform.

[0138] As Figure 4 shown, the service management platform can be nacos. Nacos is an open-source distributed configuration center and service discovery platform. It provides unified configuration management and service registration and discovery functions. It can help developers better manage and configure various parameters and attributes of applications, and can dynamically register, discover and manage the services of applications.

[0139] User configuration may include: alarm cluster backlog threshold, user information of relevant users for alarm sending, etc.

[0140] Similarly, as Figure 7 shown, in step S703, it can be achieved that: query the user configuration information of relevant users based on the service management platform. Among them, the user configuration information has the same meaning as the user configuration.

[0141] S105: Determine the alarm information to be sent based on the theme information and consumption messages, and send the alarm information to the relevant users based on the user configuration through the API opened by the preset office program.

[0142] Determining the alarm information to be sent based on the theme information and consumption messages includes: alarm time, alarm cluster, alarm theme, consumption ID, queue alarm threshold, queue production location, queue consumption location, queue backlog situation.

[0143] Through the timed task component, the scheduling and triggering of tasks can be carried out to ensure the effective operation of the monitoring and alarm tasks. Through the monitoring and alarm service, the meta-information of each distributed message queue cluster can be obtained at all times. If the theme information and consumption information are abnormal, the configuration of relevant users can be obtained based on the service management platform, so as to generate and send the alarm information to the relevant users, realizing the continuous monitoring and rapid alarm of the message queue, ensuring the stable operation of the message queue, and reducing the possibility of negative impacts on the business.

[0144] As Figure 5 shown, the alarm information is displayed to relevant users through the office program (or other third-party programs). Among them, when displaying the alarm information, it is usually displayed in text form, and the preset part of the alarm information is rendered and displayed. For example, the alarm theme and queue backlog situation are used as the preset part and rendered and displayed in different colors.

[0145] It is also possible to obtain the acquisition status (also called the read status) of relevant users for the alarm information through the API opened by the office program (or the API interface of the third-party program). The acquisition status (or read status) includes: unread status, read status, and replied status.

[0146] If the acquisition status (or read status) is in the preset status for more than the preset duration (for example, 3 minutes) according to the alarm information, alarm relevant users in other forms except text form, such as voice call, video call, so as to be able to remind relevant users to conduct troubleshooting and repair as soon as possible.

[0147] Among them, if the number of messages to be processed in the message backlog information does not exceed a preset number (for example, a million-level number of messages), the preset status includes the unread status. If the number of messages to be processed in the message backlog information exceeds the preset number, the preset status includes the unread status and the read status, so as to classify the urgency of different situations.

[0148] Specifically, when the unread status exceeds a preset duration (for example, 3 minutes), through the office program, an alarm is sent to the relevant user in other ways. The other ways at least include: voice call and video call, so as to be able to remind the relevant user to conduct a troubleshooting and repair as soon as possible.

[0149] Further, when the number of messages in the queue backlog does not exceed the preset number (for example, a million-level number of messages), at this time, the backlog situation is not particularly serious, and the preset status includes the unread status, that is, only when the relevant user has not read, an alarm will be sent in other ways. Based on the queue backlog situation, the first preset duration corresponding to the unread status is determined. Among them, the more the number of messages in the queue backlog, the shorter the first preset duration, that is, the more serious the situation, the shorter the first preset duration.

[0150] When the number of messages in the queue backlog exceeds the preset number, it means that the current backlog situation is relatively serious. At this time, the preset status includes the unread status and the read status, and based on the queue backlog situation, the first preset duration corresponding to the unread status and the second preset duration corresponding to the read status are determined. Even if the relevant user has read but not replied, in order to ensure that they can respond, an alarm still needs to be sent in other ways to prompt the relevant user to reply as soon as possible and improve their attention. Of course, the second preset duration can be set to be longer than the first preset duration. If the user has completed the troubleshooting and repair within the second preset duration, there is no need to prompt anymore.

[0151] For unread or unreplied situations, other ways can be set for prompting to ensure that the relevant user can indeed see this alarm information.

[0152] For different degrees of message queue backlog, corresponding preset durations can be set to represent the urgency of this time, so as to improve the attention of relevant users to this troubleshooting and repair, and different preset durations are set. If the relevant user has completed the troubleshooting and repair, there is no need to use other ways for prompting, thus ensuring the user experience of the relevant user.

[0153] Similarly, as Figure 7 shown, in step S704, it can be realized that: based on the meta information, the message backlog information and the user configuration information, an alarm message is sent to the corresponding relevant user.

[0154] Specifically, asFigure 5 As shown, alarm information can be generated based on the message backlog quantity, user configuration information, and production location, consumption location, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and current timestamp included in the meta-information. Of course, other information can also be added to the alarm information based on requirements.

[0155] Alternatively, determine the degree of message backlog based on the message backlog quantity and a preset message backlog threshold, and determine the message backlog situation based on the degree of message backlog and the message backlog quantity. For example, determine the degree of message backlog according to the level of the difference between the two. The higher the difference, the higher the degree of message backlog.

[0156] At this time, based on the message backlog situation, user configuration information, and production location, consumption location, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and current timestamp included in the meta-information, generate alarm information.

[0157] Then, obtain the user information of relevant users according to the user configuration, and send the alarm information to the account corresponding to the user information in the third-party program through the API interface of the pre-obtained third-party program. The third-party program can be an office software, social software, communication software, etc. The alarm information can be sent in the form of text, voice, etc.

[0158] As Figure 6 shown, an embodiment of the present application also proposes a message queue backlog alarm device, including:

[0159] At least one processor; and,

[0160] A memory communicatively connected to the at least one processor; wherein,

[0161] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the message queue backlog alarm method as described in any of the above embodiments.

[0162] An embodiment of the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set to: the message queue backlog alarm method as described in any of the above embodiments.

[0163] Each embodiment in the present application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0164] The device, medium, and method provided by the embodiments of the present application correspond one by one. Therefore, the device and the medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and the medium will not be elaborated here.

[0165] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified function in Figure 1 one or more of the processes Figure 1 or blocks or a combination of multiple blocks.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified function in Figure 1 one or more of the processes Figure 1 or blocks or a combination of multiple blocks.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Therefore, the instructions executed on the computer or other programmable device provide steps for implementing the specified function in Figure 1 one or more of the processes Figure 1 or blocks or a combination of multiple blocks.

[0169] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0170] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0171] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transitory media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0172] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0173] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for warning of message queue backlog, characterized in that Including: The monitoring and alarming service sends information query instructions to each distributed message queue cluster through a custom encapsulated application programming interface; Receiving the meta-information responded by each distributed message queue cluster, and determining the message backlog information corresponding to the distributed message queue cluster based on the meta-information; Querying the user configuration information of relevant users based on the service management platform; Sending an alarm message to the corresponding relevant users based on the meta-information, the message backlog information and the user configuration information.

2. The method according to claim 1, characterized in that, The method further includes: Triggering a monitoring and alarming task through a timing task component, so that the monitoring and alarming service sends information query instructions to each distributed message queue cluster.

3. The method according to claim 1, wherein The distributed message queue cluster includes multiple clusters, each cluster corresponds to multiple topics, each topic corresponds to one or more partitions, and message queues are formed through the topics; the cluster types include at least one of Kafka cluster and RockerMQ cluster.

4. The method according to claim 3, characterized in that, The monitoring and alarming service sends information query instructions to each distributed message queue cluster through a custom encapsulated application programming interface, specifically including: The monitoring and alarming service determines a custom encapsulated application programming interface; Through the application programming interface, determining the cluster identifier corresponding to the distributed message queue cluster to be queried, and determining its corresponding cluster type according to the cluster identifier; In the Kafka cluster or RockerMQ cluster corresponding to the cluster type, selecting the corresponding cluster according to the cluster identifier and sending an information query instruction.

5. The method according to claim 1, wherein Determining the message backlog information corresponding to the distributed message queue cluster based on the meta-information, specifically including: Based on the meta-information, determining the production position and consumption position of the corresponding message queue in the distributed message queue cluster; According to the production position and the consumption position, obtaining the number of backlogged messages, and determining the number of backlogged messages as the message backlog information; and / or Sending an alarm message to the corresponding relevant users based on the meta-information, the message backlog information and the user configuration information, specifically including: Based on the user configuration information, obtaining the queue alarm threshold preset for the distributed message queue cluster; If the number of backlogged messages is higher than the queue alarm threshold, generating an alarm message based on the meta-information, the number of backlogged messages and the user configuration information; Based on the user configuration information, sending the alarm message to the corresponding relevant users.

6. The method according to claim 5, characterized in that, Generating an alarm message based on the meta-information, the number of backlogged messages and the user configuration information, specifically including: Generating an alarm message based on the number of backlogged messages, the user configuration information, and the production position, consumption position, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and current timestamp included in the meta-information; Alternatively, determine the message backlog degree based on the message backlog quantity and a preset message backlog threshold, and determine the message backlog situation based on the message backlog degree and the message backlog quantity; generate an alarm message based on the message backlog situation, the user configuration information, and the production location, consumption location, corresponding cluster identifier, corresponding topic identifier, corresponding partition identifier, and the current timestamp included in the meta information; And / or, based on the user configuration information, send the alarm message to the corresponding relevant users, specifically including: Obtain the user information of the relevant users according to the user configuration, and send the alarm message to the account corresponding to the user information in the third-party program through the API interface of the pre-obtained third-party program.

7. The method according to claim 1, wherein Before receiving the meta information responded by each distributed message queue cluster, the method further includes: Define the relevant functions of the interfaces and classes corresponding to each distributed message queue cluster; wherein, the interfaces and classes include at least one of the following: MqSoureFactory class, MQSource interface, BaseMQAlarmService class, MessageAssemble class, Sendable interface, MQAlarmService class; The MqSoureFactory class is used to create a distributed message queue cluster, instantiate the distributed message queue cluster, and implement the underlying connection function of the distributed message queue cluster; The MQSource interface is used to abstract the distributed message queue cluster and define the interface functions of the API interface; The BaseMQAlarmService class is used to implement the actions of the distributed message queue cluster and interface calls; The MessageAssemble class is used to encapsulate relevant metadata into objects; The Sendable interface is used to define an alarm interface and transmit and send alarm messages; The MQAlarmService class is used to start an alarm task; Preferably, the interface functions include at least one of the following: configuration acquisition function, consumer thread control function, consumer acquisition function, partition acquisition function, topic acquisition function, topic information acquisition function, consumption information acquisition function; wherein, the meta information further includes: topic information and consumption information; The configuration acquisition function is used to acquire the configuration of the message queue in the service management platform, wherein the return type of the configuration acquisition function is a subclass of the basic message queue class, and the specific configuration type is specified through a generic parameter; The consumer thread control function is used to start the consumer thread of the corresponding message queue and close the consumer thread of the corresponding message queue; The consumer acquisition function is used to acquire all consumer identifiers through a list containing consumer identifiers returned; The partition acquisition function is used to acquire all partition identifiers consumed by a consumer through a list containing partition identifiers returned by parameters corresponding to the consumer identifier; The subject obtaining function is used to return a list containing subject identifiers by the parameters corresponding to the consumer identifier, and obtain the subject identifiers corresponding to the consumer; The subject information obtaining function is used to return a list containing the subject information corresponding to the partition identifier by the parameters corresponding to the subject identifier, and obtain the subject information; The consumption information obtaining function is used to return a list containing consumption information by the parameters corresponding to the consumer identifier and the subject identifier, and obtain the consumption information.

8. The method according to claim 6, wherein The method further includes: Using the third-party program to display the warning information to the relevant user in text form, and when displaying the warning information, rendering and displaying a preset part of the warning information; Obtaining the reading status of the relevant user for the warning information through the API interface of the third-party program, where the reading status includes: unread status, read status, and replied status; If the reading status remains in a preset status for more than a preset duration, then issue a warning to the relevant user in a form other than the text form according to the warning information; Wherein, if the number of messages to be processed in the message backlog information does not exceed a preset number, the preset status includes the unread status, and if the number of messages to be processed in the message backlog information exceeds the preset number, the preset status includes the unread status and the read status.

9. A message queue backlog warning device, characterized in that, Including: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the message queue backlog warning method described in any one of claims 1 to 8.

10. A non-volatile computer storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions are set to: the message queue backlog warning method described in any one of claims 1 to 8.

Images