Dial testing method and device, electronic equipment and storage medium

Abstract

The invention discloses a dial testing method and device, electronic equipment and a storage medium. The method comprises the following steps: on the basis of cluster attribute information of a distributed message cluster, a dial test message of a current batch is generated, and the dial test message carries partition address information and batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster; based on the partition address information of each dial test message of the current batch, distributing each dial test message to a partition of the distributed message cluster; consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information; and determining whether the distributed message cluster is abnormal or not based on the consumption statistical parameters of the dial test messages of the current batch. According to the embodiment of the invention, the fault detection efficiency of the distributed message system in related technologies can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of distributed systems, and in particular to a dialing method, apparatus, device and storage medium. Background Art

[0002] Distributed messaging systems are usually composed of multiple nodes, each of which includes one or more physical machines. Each physical machine usually has multiple physical disks mounted on it. Many nodes and disks provide services to the outside world as a whole. Distributed systems usually have very high requirements for cluster availability and response time. However, when a physical disk in the system fails, the availability of the cluster will be greatly affected and the system response time will also be longer. In related technologies, the system indicators obtained based on the input and output performance of the node's processor, memory, and disk cannot accurately determine the failed disk, and the fault detection efficiency of the distributed messaging system is low. Summary of the invention

[0003] In view of the above problems, the present application provides a dialing method, apparatus, device and storage medium to at least solve the technical problem of low fault detection efficiency of distributed message systems in related technologies.

[0004] According to a first aspect of an embodiment of the present application, a dial test method is provided, comprising: generating a current batch of dial test messages based on cluster attribute information of a distributed message cluster, the dial test messages carrying partition address information and batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster, and each disk includes one or more dial test message topics; based on the partition address information of each dial test message in the current batch, distributing each dial test message to a partition of the distributed message cluster; wherein each dial test message topic corresponds to at least one partition; consuming the dial test messages belonging to the current batch from the distributed message cluster based on the batch identification information; and determining whether an abnormality occurs in the distributed message cluster based on consumption statistical parameters corresponding to the dial test messages of the current batch.

[0005] According to a second aspect of an embodiment of the present application, a dialing test device is provided, the above-mentioned dialing test device comprising: a generating unit, based on cluster attribute information of a distributed message cluster, generating a fingerprint message dialing test message of a current batch, the fingerprint message dialing test message carrying partition address information and batch identification information; the partition address information of each fingerprint message dialing test message in the current batch covers the addresses of all disks in the distributed message cluster; each disk comprises one or more dialing test message topics; a distributing unit, for distributing each fingerprint message dialing test message to a partition in the distributed message cluster based on the partition address information of each fingerprint message dialing test message of the current batch; wherein each dialing test message topic corresponds to at least one partition; a consuming unit, for consuming the fingerprint message dialing test message belonging to the current batch from the distributed message cluster based on the batch identification information; a determining unit, for determining whether an abnormality occurs in the distributed message cluster based on consumption statistical parameters corresponding to the consumed fingerprint message dialing test messages of the current batch.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the dialing method of the first aspect through the computer program, or execute the dialing method of the first aspect.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the dialing method of the first aspect mentioned above when running, or execute the dialing method of the first aspect.

[0008] In an embodiment of the present application, cluster attribute information based on a distributed message cluster is used to generate a dial test message of the current batch, and the dial test message carries partition address information and batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster; each disk includes one or more dial test message topics, and based on the partition address information of each dial test message, each dial test message is distributed to a partition in the distributed message cluster, and each dial test message topic corresponds to at least one partition; based on the batch identification information, the dial test message belonging to the current batch is consumed from the distributed message cluster; based on the consumption statistical parameters corresponding to the dial test message of the current batch, a method for determining whether an abnormality occurs in the distributed message cluster; because the dial test batch and message timestamp are encapsulated into the dial test message, and accurately sent to each partition in the distributed message cluster, and the dial test message is synchronously consumed from the partition, it is possible not only to accurately feedback the health status of each partition through the dial test message, but also to efficiently and accurately determine the faulty disk, thereby improving the fault detection efficiency of the distributed message system, thereby solving the technical problem of low fault detection efficiency of the distributed message system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Moreover, the same reference numerals are used throughout the drawings to represent the same components. In the drawings:

[0010] Figure 1 is a schematic diagram of an application environment of an optional dialing method according to an embodiment of the present application;

[0011] Figure 2 is a schematic diagram of an application environment of another optional dialing method according to an embodiment of the present application;

[0012] Figure 3 is a flowchart of an optional dialing method according to an embodiment of the present application;

[0013] Figure 4 is a flowchart of another optional dialing method according to an embodiment of the present application;

[0014] Figure 5 is a flowchart of another optional dialing method according to an embodiment of the present application;

[0015] Figure 6 is a flowchart of another optional dialing method according to an embodiment of the present application;

[0016] Figure 7is a flowchart of another optional dialing method according to an embodiment of the present application;

[0017] Figure 8 is a flowchart of another optional dialing method according to an embodiment of the present application;

[0018] Fig. 9 It is a structural schematic diagram of a dial testing device provided in an embodiment of the present application;

[0019] Fig.10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] The following are the technical terms involved in this application:

[0023] Topic: message topic, first-level message type, messages are classified by Topic.

[0024] Producer: The producer and sender of messages in a distributed messaging system.

[0025] Consumer: The consumer of messages in a distributed messaging system.

[0026] Broker: A proxy server in a distributed messaging system that exposes the message sending and consuming interfaces. Message producers and consumers interact with it to complete the core message sending and receiving logic.

[0027] Partition: The partition referred to in this application is Partition. A Topic is stored by one or more queues. The minimum concurrent granularity for consumers to pull messages from the server is one queue. The multi-partition referred to in this invention refers to queues located on different Brokers (message servers).

[0028] Configuration Management Database (CMDB): A configuration management center built on a database where users store basic operation and maintenance data.

[0029] As an optional implementation, the above dialing method can be but is not limited to being applied to Figure 1 In the application environment shown. The application environment may include, but is not limited to: a dialing device 102 for human-computer interaction with a user, a network 110, and a distributed message cluster 112. A message dialing application client is running in the terminal device 102. The above-mentioned dialing device 102 includes a display 108, a memory 104 and a processor 106. The display 108 is used to present cluster attribute information of the distributed message cluster and information such as the address of the partition. The processor 106 is used to generate and consume different batches of dialing messages, and to determine whether the distributed message cluster 112 has an abnormality. The memory 104 is used to store cluster attribute information of the distributed message cluster and information such as the address of the partition. In addition, the distributed message cluster 112 includes at least one server (such as Figure 1 Server 1 to server N in the , N is a positive integer), each server is used to receive and store different batches of dial test messages.

[0030] The specific implementation process of the dial test method is as follows: Assume that Figure 1 The dial test device 102 shown runs a dial test client, and the dial test device 102 executes steps S102-S108, generates a current batch of dial test messages based on the cluster attribute information of the distributed message cluster, and the dial test messages carry partition address information and batch identification information; the partition address information of each dial test message in the current batch covers the addresses of all disks in the distributed message cluster; based on the partition address information of each dial test message, distributes each dial test message to the partition in the distributed message cluster 112. Based on the batch identification information, consume the dial test messages belonging to the current batch from the distributed message cluster 112; based on the consumed dial test messages of the current batch, determine whether the distributed message cluster is abnormal.

[0031] In one or more embodiments, the above-mentioned dialing method of the present application can be applied to Figure 2 In the application environment shown. Figure 2As shown, human-computer interaction can be performed between user 202 and dialing device 102. Dialing device 102 includes memory 104 and processor 106. In this embodiment, dialing device 102 can determine whether the above distributed message cluster is abnormal by referring to but not limited to the operations performed by the above dialing method.

[0032] Optionally, the dialing device 102 and the dialing device 204 include but are not limited to mobile phones, laptops, tablet computers, PDAs, MIDs (Mobile Internet Devices), desktop computers, smart TVs, etc. The application client with a dialing function in the embodiment of the present application includes but is not limited to a video client, an instant messaging client, a browser client, etc. The above network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that implement wireless communication. The server in the above distributed message cluster includes a cloud server, and the above cloud server includes but is not limited to a private cloud server or a public cloud server. The above is only an example, and no limitation is made to this in this embodiment.

[0033] Distributed messaging systems are usually composed of multiple nodes, each of which includes one or more physical machines. Each physical machine usually has multiple physical disks mounted on it. Many nodes and disks provide services to the outside world as a whole. Distributed systems usually have very high requirements for cluster availability and response time. However, when a physical disk in the system fails, the availability of the cluster will be greatly affected and the system response time will also be longer. In related technologies, the system indicators obtained based on the input and output performance of the node's processor, memory, and disk cannot accurately determine the failed disk, and the fault detection efficiency of the distributed messaging system is low.

[0034] In order to solve the above technical problems, as an optional implementation method, Figure 3 As shown, the embodiment of the present application provides a dialing method, comprising the following steps:

[0035] S302, based on the cluster attribute information of the distributed message cluster, generate the current batch of dial test messages, the above-mentioned dial test messages carry partition address information and batch identification information; the partition address information of each dial test message in the above-mentioned current batch covers the address of each disk in the above-mentioned distributed message cluster; each disk includes one or more dial test message topics.

[0036] S304, based on the partition address information of each dial test message in the current batch, distribute each dial test message to the partition of the distributed message cluster; wherein each of the dial test message topics corresponds to at least one partition;

[0037] S306, consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information;

[0038] S308: Determine whether the distributed message cluster is abnormal based on the consumption statistical parameters corresponding to the current batch of dial-up test messages.

[0039] Specifically, in the embodiment of the present application, the cluster attribute information here includes the IP address of the server in the cluster, the name of each hard disk in the server, and the partition name of each partition in the hard disk. In one example, based on the IP addresses of all servers in the distributed cluster, the identification of all hard disks, and the identification of each partition (Partition) contained in each hard disk, the address of each partition is obtained, and the current batch of dial test messages carrying the partition address information and batch identification information are generated to each partition of the distributed message cluster, that is, each partition in the cluster will detect whether an abnormality occurs through the current batch of dial test messages. Then based on the consumption test module, the dial test message belonging to the above-mentioned current batch is consumed from the above-mentioned distributed message cluster according to the current batch identification information, and the consumption statistical parameters corresponding to the fingerprint information of the current batch can be used to determine whether the distributed message cluster is abnormal. Here, the consumption statistical parameters include but are not limited to the message loss rate and the processing time parameters, and the processing time parameters include but are not limited to the consumption delay, etc.; for example, when the consumption delay of the fingerprint information of a certain partition exceeds the preset threshold, or the partition does not receive the fingerprint information of the current batch, it is determined that the current partition is abnormal, so that it can be further determined that the distributed message cluster is abnormal.

[0040] In another example, each disk in the distributed cluster includes one or more message topics. Since each message topic is evenly distributed to each disk according to a preset average distribution algorithm, in order not to affect the normal operation of existing services or applications in the distributed cluster, the total number of all disks in the distributed cluster is obtained from the above cluster attribute information, and a number of dial-test message topics greater than or equal to the above total number is created. Here, each of the above dial-test message topics corresponds to at least one partition; it should be noted that the partition corresponding to the dial-test message topic here is different from the partition corresponding to the existing service or application. Based on the number of dial-test message topics greater than or equal to the above total number, the dial-test messages can be evenly distributed to each disk in the distributed message cluster. Then, based on the consumption test module, the dial-test messages belonging to the above current batch are consumed from each disk in the above distributed message cluster according to the current batch identification information, and the consumption statistical parameters corresponding to the fingerprint information of the current batch can be used to determine whether each hard disk in the distributed message cluster is abnormal.

[0041] In the embodiment of the present application, since the dial test batch and message timestamp are encapsulated into the dial test message and accurately sent to each partition in the distributed message cluster, and the dial test message is consumed from the partition synchronously, it is possible not only to accurately feedback the health status of each of the above partitions through the dial test message, but also to efficiently and accurately determine the faulty disk, thereby improving the fault detection efficiency of the distributed message system, thereby solving the technical problem of low fault detection efficiency of the distributed message system.

[0042] In one or more embodiments, Figure 4 As shown, the embodiment of the present application also provides a dialing method, comprising the following steps:

[0043] S402, obtaining the total number of disks of the distributed cluster from the cluster attribute information, and creating a number of dial test message topics greater than or equal to the total number; each of the dial test message topics corresponds to at least one partition.

[0044] Specifically, in an embodiment of the present application, assuming that there are 5 servers in the current distributed message cluster, each server includes 4 disks, then the cluster has a total of 20 disks, and an integer of 20 dial-up message topics can be created here. Each topic can correspond to creating 3 to 5 new partitions, so that the dial-up message can cover all disks in the cluster.

[0045] S404, obtaining the partition address information of the partition corresponding to each of the dial test message topics, and obtaining the batch identification information used to identify the current batch.

[0046] S406: Generate the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information.

[0047] Specifically, in an embodiment of the present application, the partition address information of the partition corresponding to each of the above-mentioned dial test message topics is obtained from the cluster attribute information, and the batch identification information used to identify the above-mentioned current batch is obtained. As shown in Table (1), the message body of the dial test message includes Server IP (network address of the server), Cluster ID (unique number of the cluster in the database), Partition_ID (partition serial number), TimeStamp (message sending timestamp) and Batch ID (message batch) and other fields.

[0048] Table (1)

[0049] Dial test message body fields Type illustrate Server IP String Server IP Cluster ID String Unique ID of the cluster in the database Partition_ID Int Partition number

[0050] Table (1) continued

[0051] Dial test message body fields Type illustrate TimeStamp timestamp Message sending timestamp Batch ID Int Message Batching

[0052] In an example, the message body of a dial test message in the current batch is body = {Server_IP: 10.222.111.1, Cluster_id: 44123, Partition_ID: 43, TimeStamp: 8812737111, BatchID: 10001}, that is, the batch identification information corresponding to the current dial test message is 10001, and the dial test message is to be sent to the partition 43 of the server with the IP address of 10.222.111.1 in cluster 44123, and the sending timestamp is 8812737111. According to the above message body, the dial test message can be accurately distributed to each partition corresponding to the above dial test message topic.

[0053] S408, based on the partition address information of each dial test message in the current batch, distribute each dial test message to the partition of the distributed message cluster;

[0054] S410, consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information;

[0055] S412: Determine whether the distributed message cluster is abnormal based on the consumption statistical parameters of the current batch of dial-test messages.

[0056] The above steps S408-S412 have been clearly explained in the previous text and will not be repeated here.

[0057] In one or more embodiments, the obtaining of batch identification information for identifying the current batch includes:

[0058] A hash operation is performed on the timestamp corresponding to the creation time of the current batch, and the obtained hash value is used as the batch identification information of the current batch.

[0059] Specifically, in the embodiment of the present application, for example, the creation time corresponding to the current batch is 202207131930, then the timestamp is hashed by a preset encryption algorithm to obtain a hash value of 9d00b20758bb8269f2e50a1104073996 as the batch identification information of the current batch. The consumption test module of the dial test system consumes the corresponding fingerprint information from each partition in the cluster based on the batch identification information.

[0060] In one or more embodiments, the generating of the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information includes:

[0061] The batch identification information is respectively combined with the partition address information of the partition corresponding to the topic of each dial test message to construct each dial test message of the current batch;

[0062] The partition address information includes the partition identifier, the IP address of the server where the partition is located, and the cluster identifier of the distributed cluster where the partition is located.

[0063] In one example, assuming that the batch identification information corresponding to the current dial test message is 10001, the dial test message is to be sent to cluster 44123, the server with IP address 10.222.111.1 and partition 43. The message body of the current batch of dial test messages includes {Server_IP:10.222.111.1, Cluster_id:44123, Partition_ID:43, BatchID:10001}, and the dial test messages of the current batch can be constructed based on the message body.

[0064] In one or more embodiments, Figure 5 As shown, the embodiment of the present application also provides a dialing method, comprising the following steps:

[0065] S502, obtaining the total number of disks of the distributed cluster from the cluster attribute information, and creating a number of dial test message topics greater than or equal to the total number; each of the dial test message topics corresponds to at least one partition.

[0066] S504, obtaining the partition address information of the partition corresponding to each of the dial test message topics, and obtaining the batch identification information used to identify the current batch.

[0067] S506: Generate the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information.

[0068] S508: Distribute each dial test message to a partition in the distributed message cluster based on the partition address information of each dial test message in the current batch.

[0069] S510: traverse the partitions corresponding to the dial test message topics based on the batch identification information, and consume the dial test messages belonging to the current batch from the partition corresponding to the dial test message topic.

[0070] Specifically, in an embodiment of the present application, based on the batch identification information of the fingerprint information of the current batch, the partitions corresponding to the above-mentioned dial test message topics are traversed, and then the dial test messages belonging to the above-mentioned current batch are consumed from the partitions corresponding to the above-mentioned dial test message topics. According to the consumption of the above-mentioned current batch of dial test messages, it can be determined whether each partition has an abnormality.

[0071] S512: Determine whether an abnormality occurs in the distributed message cluster based on consumption statistical parameters corresponding to the current batch of dial-up test messages.

[0072] The above steps S502-S508, S512 have been clearly explained in the previous text and will not be repeated here.

[0073] In one or more embodiments, Figure 6 As shown, the embodiment of the present application also provides a dialing method, comprising the following steps:

[0074] S602, obtaining the total number of all disks of the distributed cluster from the cluster attribute information, and creating a number of dial test message topics greater than or equal to the total number; each of the dial test message topics corresponds to at least one partition.

[0075] S604, obtaining the partition address information of the partition corresponding to each of the dial test message topics, and obtaining the batch identification information used to identify the current batch.

[0076] S606: Generate the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information.

[0077] S608: Distribute the dial test messages to the partitions in the distributed message cluster based on the partition address information of the dial test messages in the current batch.

[0078] S610: traverse the partitions corresponding to the dial test message topics based on the batch identification information, and consume the dial test messages belonging to the current batch from the partition corresponding to the dial test message topic.

[0079] S612, determining whether the distributed message cluster is abnormal based on the message loss rate and / or processing time parameter corresponding to the current batch of dial-up test messages; the statistical information includes at least one of the message loss rate and the processing time parameter.

[0080] Specifically, in an embodiment of the present application, based on the message loss rate and / or processing time parameters corresponding to the dial test messages of the current batch, it can be determined whether the above-mentioned distributed message cluster has an abnormality. Here, the processing time parameters include but are not limited to consumption delay, etc. For example, if the message loss rate of the dial test messages consumed from the current partition is too large, it means that the partition has an abnormality, or the disk where the partition is located may have an abnormality. For another example, if the consumption delay consumed from the current partition is too long, it may mean that the partition has an abnormality, or the disk where the partition is located may have an abnormality.

[0081] Specifically, steps S602-S610 in the embodiment of the present application have been clearly described above and will not be repeated here.

[0082] In one or more embodiments, determining whether the distributed message cluster is abnormal based on the statistical information corresponding to the dial test message of the current batch consumed includes:

[0083] Determine a first number of the current batch of dial test messages distributed to a first partition, and determine a second number of the current batch of dial test messages consumed in the first partition; the first partition is any partition corresponding to the dial test message topic;

[0084] Calculate the ratio of the second number to the first number, and determine the message loss rate corresponding to the first partition according to the ratio;

[0085] When the message loss rate is greater than or equal to the first threshold, it is determined that an abnormality occurs in the first partition.

[0086] Specifically, in an embodiment of the present application, assuming that the current batch of dial test messages distributed to the current partition is 10, and the current batch of dial test messages consumed from the partition is 6, then the message loss rate corresponding to the current partition is 60%. Assuming that the first threshold is 80%, it can be confirmed that an abnormality has occurred in the current partition.

[0087] In one or more embodiments, determining whether the distributed message cluster is abnormal according to the processing time parameter corresponding to the dial test message of the current batch consumed includes:

[0088] Determine the consumption delay corresponding to each dial test message consumed in the first partition according to the distribution timestamp of the current batch of dial test messages distributed to the first partition and the consumption timestamp of the current batch of dial test messages consumed from the first partition; the first partition is any partition corresponding to the dial test message topic;

[0089] Based on the fact that the consumption delay of the dial test message is greater than or equal to the second threshold, it is determined that an abnormality occurs in the first partition; wherein the processing time parameter includes the consumption delay.

[0090] Specifically, in an embodiment of the present application, assuming that the distribution time of the current batch of dial test messages distributed to the current partition is 15:03:01:002 microseconds, and the time of consuming the above current batch of dial test messages from the partition is 15:03:03:002 microseconds, there are 6 messages, then the consumption delay corresponding to the current partition is 2 seconds. Assuming that the second threshold is 800 microseconds, it can be confirmed that an abnormality has occurred in the current partition.

[0091] In one or more embodiments, Figure 7 As shown, the embodiment of the present application also provides a dialing method, comprising the following steps:

[0092] S702, obtaining the total number of disks of the distributed cluster from the cluster attribute information, and creating a number of dial test message topics greater than or equal to the total number; each of the dial test message topics corresponds to at least one partition;

[0093] S704, obtaining the partition address information of the partition corresponding to each of the dial test message topics, and obtaining the batch identification information used to identify the current batch;

[0094] S706, generating the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information;

[0095] S708: Distribute each dial test message to a partition in the distributed message cluster based on the partition address information of each dial test message in the current batch.

[0096] S710, traversing the partitions corresponding to the dial test message topics based on the batch identification information, and consuming the dial test messages belonging to the current batch from the partitions corresponding to the dial test message topics;

[0097] S712, based on the consumption statistical parameter indicating that multiple partitions corresponding to the dial test message topic in the same disk are abnormal, determining that the disk is abnormal; and / or,

[0098] S714, based on the consumption statistical parameter indicating that multiple disks in the same server are abnormal, determining that the server is abnormal.

[0099] Specifically, in the embodiment of the present application, assuming that it is determined that multiple partitions corresponding to the above-mentioned dial test message topic in the same disk A are abnormal, it means that the abnormality of the disk A is not an accidental situation, and it can be determined that the disk A has an abnormality.

[0100] Assume that the server Server01 includes 4 disks. When it is determined that 3 disks in Server01 are abnormal, it can be determined that the server Server01 is abnormal.

[0101] The above steps S702-S710 have been clearly explained in the previous text and will not be repeated here.

[0102] Based on the above embodiment, as an optional implementation mode, the embodiment of the present application further provides a dialing device of a real-time dialing method, such as Figure 8 As shown, the dial testing device includes the following modules:

[0103] 1. Dial test message production module: responsible for the production and sending of dial test messages. The message body structure of the dial test message is shown in Table (1). After the message body of the dial test message is generated, the dial test message is distributed to each partition in the specific distributed cluster. Each dial test message can be sent as a normal message in batches, and only a certain amount of messages are sent in a batch. For example, a batch sends a total of 1,000 dial test messages. The dial test message production module is scheduled to send a batch of dial test messages every 5 minutes.

[0104] 2. Dial test message consumption module: The dial test message consumption module is responsible for consuming dial test messages from the cluster through the specified partition. During the consumption process, only the messages of the current batch are processed, and messages outside the current batch are directly discarded. And stop further consumption. Statistics are collected on the message loss rate and delay (such as consumption delay) during the consumption of dial test messages, and delivered to the monitoring system in units of partitions. It should be noted that the dial test message production module and the dial test message consumption module are in the same process, corresponding to the same batch of dial test messages.

[0105] 3. Partition configuration collection module (Agent): The partition configuration collection module is a service module deployed on each Broker (server). It is responsible for collecting the specific information of each partition on the disk of each server, formatting and encoding the specific information of the partition and uploading it to the CMDB.

[0106] 4. Alarm association aggregation device (Aggregator): After the dial test message consumption module delivers the calculated consumption statistics of the dial test message to the monitoring system, it will automatically trigger an abnormal alarm. The alarm information will trigger the alarm association aggregation device to actively obtain the partition and disk information in the CMDB, associate the alarm information with the specific information of the disk, and generate the latest abnormal alarm information and send it to the user.

[0107] Based on the above embodiment, as an optional implementation mode, the embodiment of the present application further provides a real-time dialing method, including the following steps:

[0108] S1. The dial test message production module and the dial test message consumption module are started synchronously, and the dial test messages of the current dial test batch are obtained synchronously. The dial test batch can be obtained, for example, through Hash (current timestamp).

[0109] S2. The dial test message production module dynamically obtains cluster attribute information from the online cluster (distributed message cluster), including cluster size, total disk quantity, etc., and dynamically creates the total number of dial test topics for the dial test message, which is usually the total number of disks*N, where N is a positive integer greater than or equal to 1. In this way, the dial test message can cover all disks of the distributed message cluster.

[0110] S3. The dial test message production module generates a dial test message corresponding to each partition corresponding to the dial test topic in the distributed message cluster, and sends it to each partition through the LowLevelAPI (low-level interface).

[0111] S4. After the dial test message is produced and sent, the dial test message consumption module consumes the current batch of dial test messages immediately, and consumes the current batch of dial test messages from the specified partition using the simpleconsumerAPI (simple consumer interface). During the consumption process, the acquired dial test message is parsed, and it is determined whether the batch of the message is consistent with the batch of messages held by the dial test message production module. If not, it is directly discarded and consumption is stopped. The total number of received messages is counted, and the message loss rate (lossRatio) is calculated. At the same time, the delivery timestamp and the system current timestamp in the message body are parsed, and the consumption delay (latency) is calculated based on the delivery timestamp and the system current timestamp.

[0112] S5. The dial-up message consumption module measures the message loss rate and consumption delay of the received message delivered to the monitoring system in units of partitions.

[0113] S6. The partition configuration collection module deployed on each server regularly collects the specific information of the partitions on each disk, formats and encodes the specific information of the disk, and uploads it to the CMDB. The real-time information of the disks and partitions in the CDMB can provide quasi-real-time feedback on the distribution of partitions on servers and disks. This part of the information is static information and will not be affected by abnormal situations such as server node downtime.

[0114] S7. When a disk (for example, disk / data5 of the server with IP address 10.44.3.4) fails, the consumption latency corresponding to the dial-up message increases or the message loss rate increases, which will cause the alarm index value to rise. At this time, the monitoring system alarm is triggered. The alarm content includes, for example: [High] partition45 latency over 10ms! [High] partition12 latency over 10ms! That is, the latency of partition45 exceeds 10ms, and the latency of partition12 exceeds 10ms.

[0115] S8. After the above original alarm is triggered, the alarm aggregation device will automatically aggregate the disk information {10.4.3.4:{ / data5:[p45, p12, p90]}} of the CMDB, quickly locate the problem of the disk / data5 with the IP address 10.44.3.4, and generate the alarm information: [High] / data5 in 10.44.3.4Error! , and send the alarm information to the operation and maintenance personnel for processing (manual or automatic processing).

[0116] S9. If a server has a problem, many partitions may alarm at the same time. By aggregating the alarm information, the server failure can be located. For example, the alarm after the above step is aggregated, [High] / data1 in10.44.3.4Error! [High] / data2 in 10.44.3.4Error! [High] / data3 in 10.44.3.4Error! [High] / data4 in 10.44.3.4Error! . After the above alarm information is aggregated by the alarm association aggregation device, the aggregated alarm information can be quickly obtained as: [High]Broker 10.44.3.4Error! That is, the server with IP address 10.44.3.4 has a fault.

[0117] The above technical solution of the present application has the following beneficial technical effects:

[0118] 1. By setting the dial test message for the dial test, encapsulating the dial test batch and message timestamp into the dial test message, and accurately distributing it to each partition for synchronous and accurate consumption, the health status of each partition can be accurately fed back through the dial test message.

[0119] 2. Association aggregation of disk information: Collect and analyze disk information and partition information. When an alarm occurs in a partition, the node, disk, or cluster with the problem can be accurately located.

[0120] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0121] According to another aspect of the embodiment of the present application, a dialing test device for implementing the above-mentioned dialing test method is also provided. Fig. 9 As shown, the device comprises:

[0122] The generating unit 902 generates a dial test message of the current batch based on the cluster attribute information of the distributed message cluster, wherein the dial test message carries the partition address information and the batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster; each disk includes one or more dial test message topics;

[0123] The distribution unit 904 is used to distribute each dial test message to the partition of the above-mentioned distributed message cluster based on the partition address information of each dial test message in the above-mentioned current batch; wherein each of the dial test message topics corresponds to at least one partition;

[0124] A consumption unit 906, configured to consume the dial test messages belonging to the current batch from the distributed message cluster based on the batch identification information;

[0125] The determination unit 908 is used to determine whether the distributed message cluster is abnormal based on the consumption statistical parameters corresponding to the dial test messages of the current batch.

[0126] In the embodiment of the present application, since the dial test batch and message timestamp are encapsulated into the dial test message and accurately sent to each partition in the distributed message cluster, the dial test message is consumed from the partition synchronously. Therefore, not only can the health status of each of the above partitions be accurately fed back through the dial test message, but the faulty disk can also be efficiently and accurately determined, thereby improving the fault detection efficiency of the distributed message system, thereby solving the technical problem of low fault detection efficiency of the distributed message system.

[0127] In one or more embodiments, the generating unit 902 includes:

[0128] A creation module, used to obtain the total number of disks of the distributed cluster from the cluster attribute information, and create a number of dial-test message topics greater than or equal to the total number; each of the dial-test message topics corresponds to at least one partition;

[0129] An acquisition module, used to acquire the partition address information of the partition corresponding to each of the above-mentioned dial test message topics, and to acquire the batch identification information used to identify the above-mentioned current batch;

[0130] A generating module is used to generate the dial test message of the current batch corresponding to the dial test message subject based on the partition address information and the batch identification information.

[0131] In one or more embodiments, the acquisition module includes:

[0132] The operation subunit is used to perform a hash operation on the timestamp corresponding to the creation time of the above-mentioned current batch, and use the obtained hash value as the batch identification information of the above-mentioned current batch.

[0133] In one or more embodiments, the generation module includes:

[0134] A construction subunit is used to construct each dialing message of the current batch by respectively combining the above-mentioned batch identification information with the partition address information of the partition corresponding to each of the above-mentioned dialing message topics; wherein the above-mentioned partition address information includes the partition identification, the IP address of the server where the partition is located, and the cluster identification of the distributed cluster where the above-mentioned partition is located.

[0135] In one or more embodiments, the consumption unit 906 includes:

[0136] The consumption module is used to traverse the partitions corresponding to the dial test message topics based on the batch identification information, and consume the dial test messages belonging to the current batch from the partitions corresponding to the dial test message topics.

[0137] In one or more embodiments, the determining unit 908 includes:

[0138] The first determination module is used to determine whether the above-mentioned distributed message cluster has an abnormality according to the message loss rate and / or processing time parameter corresponding to the above-mentioned current batch of dial-test messages; the above-mentioned statistical information includes the message loss rate and processing time parameter.

[0139] In one or more embodiments, the above-mentioned determination module includes:

[0140] A second determination module is used to determine a first number of the current batch of dial test messages distributed to a first partition, and to determine a second number of the current batch of dial test messages consumed in the first partition; the first partition is any partition corresponding to the dial test message topic;

[0141] A third determination module is used to calculate the ratio of the second number to the first number, and determine the message loss rate corresponding to the first partition according to the ratio;

[0142] The fourth determination module is used to determine that an abnormality occurs in the first partition when the message loss rate is greater than or equal to the first threshold.

[0143] In one or more embodiments, the above-mentioned determination module includes:

[0144] A fifth determination module, configured to determine a consumption delay corresponding to each dial test message consumed in the first partition according to a distribution timestamp of the current batch of dial test messages distributed to the first partition and a consumption timestamp of the current batch of dial test messages consumed from the first partition; the first partition is any partition corresponding to the dial test message topic;

[0145] A sixth determination module is used to determine that an abnormality occurs in the first partition based on the existence of a consumption delay of the dial test message being greater than or equal to a second threshold; wherein the processing time parameter includes the consumption delay. In one or more embodiments, the determination unit 908 includes:

[0146] A sixth determination module is configured to determine that an abnormality occurs in the disk based on the indication by the consumption statistical parameter that multiple partitions corresponding to the dial test message topic in the same disk are abnormal; and / or,

[0147] The seventh determination module is used to determine that an abnormality occurs in the server based on the consumption statistical parameter indicating that multiple disks in the same server are abnormal.

[0148] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned dialing method is also provided. The electronic device may be Figure 1 The dial test device or distributed message cluster shown in the figure. This embodiment is described by taking the electronic device as a dial test device as an example. Fig.10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments through the computer program.

[0149] Optionally, in this embodiment, the electronic device may be at least one network device among a plurality of network devices of a computer network.

[0150] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0151] S1, based on the cluster attribute information of the distributed message cluster, generate the dial test message of the current batch, the dial test message carries the partition address information and the batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster, and each disk includes one or more dial test message topics;

[0152] S2, based on the partition address information of each of the dial test messages in the current batch, distribute each of the dial test messages to the partition of the distributed message cluster; wherein each of the dial test message topics corresponds to at least one partition;

[0153] S3, consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information;

[0154] S4: Determine whether an abnormality occurs in the distributed message cluster based on consumption statistical parameters corresponding to the dial test messages of the current batch.

[0155] Among them, the memory 1002 can be used to store software programs and modules, such as program instructions / modules corresponding to the dialing method and device in the embodiment of the present application. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, realizing the above-mentioned dialing method. The memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include a memory remotely arranged relative to the processor 1004, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. Among them, the memory 1002 can be specifically used, but not limited to, for storing dialing messages for dialing.

[0156] As an example, Fig.10 As shown, the memory 1002 may include but is not limited to the generation unit 902, distribution unit 904, consumption unit 906 and determination unit 908 in the dialing device. In addition, it may also include but is not limited to other module units in the dialing device, which will not be repeated in this example.

[0157] Optionally, the transmission device 1006 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1006 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0158] In addition, the electronic device mentioned above further includes: a connection bus 1008, which is used to connect various module components in the electronic device mentioned above.

[0159] In other embodiments, the electronic device may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes in the form of network communication. Among them, the nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0160] In one or more embodiments, the present application further provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned dialing method. The computer program is configured to execute the steps of any of the above-mentioned method embodiments when it is run.

[0161] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0162] S1, based on the cluster attribute information of the distributed message cluster, generate the dial test message of the current batch, the dial test message carries the partition address information and the batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster, and each disk includes one or more dial test message topics;

[0163] S2, based on the partition address information of each of the dial test messages in the current batch, distribute each of the dial test messages to the partition of the distributed message cluster; wherein each of the dial test message topics corresponds to at least one partition;

[0164] S3, consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information;

[0165] S4: Determine whether an abnormality occurs in the distributed message cluster based on consumption statistical parameters corresponding to the dial test messages of the current batch.

[0166] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0167] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0168] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0169] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0173] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0174] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

Claims

1. A dialing method, characterized in that: The method comprises: Based on the cluster attribute information of the distributed message cluster, generate the dial test message of the current batch, the dial test message carries the partition address information and the batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster, and each disk includes one or more dial test message topics; Based on the partition address information of each of the dial test messages in the current batch, each of the dial test messages is distributed to the partition of the distributed message cluster; wherein each of the dial test message topics corresponds to at least one partition; Consuming the dial test messages belonging to the current batch from the distributed message cluster based on the batch identification information; Based on the consumption statistical parameters corresponding to the dial test messages of the current batch, it is determined whether an abnormality occurs in the distributed message cluster.

2. The method according to claim 1, characterized in that The generating of the current batch of dial test messages based on the cluster attribute information of the distributed message cluster includes: Acquire the total number of disks of the distributed cluster from the cluster attribute information, and create a dial test message topic with a number greater than or equal to the total number; Obtaining the partition address information of the partition corresponding to each of the dial test message topics, and obtaining the batch identification information for identifying the current batch; Based on the partition address information and the batch identification information, the dial test message of the current batch corresponding to the dial test message topic is generated.

3. The method according to claim 2, characterized in that The acquiring of batch identification information for identifying the current batch includes: A hash operation is performed on the timestamp corresponding to the creation time of the current batch, and the obtained hash value is used as the batch identification information of the current batch.

4. The method according to claim 2 or 3, characterized in that: The step of generating the current batch of dial test messages corresponding to the dial test message subject based on the partition address information and the batch identification information includes: The batch identification information is respectively combined with the partition address information of the partition corresponding to the topic of each dial test message to construct each dial test message of the current batch; The partition address information includes the partition identifier, the IP address of the server where the partition is located, and the cluster identifier of the distributed cluster where the partition is located.

5. The method according to claim 1 or 2, characterized in that: The consuming the dial test message belonging to the current batch from the distributed message cluster based on the batch identification information includes: The partitions corresponding to the dial test message topics are traversed based on the batch identification information, and the dial test messages belonging to the current batch are consumed from the partitions corresponding to the dial test message topics.

6. The method according to claim 1 or 2, characterized in that: The consumption statistical parameters include a message loss rate and a processing time parameter, and the consumption statistical parameters corresponding to the dial test messages of the current batch are used to determine whether the distributed message cluster is abnormal, including: Whether an abnormality occurs in the distributed message cluster is determined according to the message loss rate and / or processing time parameter corresponding to the current batch of dial test messages.

7. The method according to claim 6, characterized in that The determining whether the distributed message cluster is abnormal according to the message loss rate corresponding to the dial test message consumed in the current batch includes: Determine a first number of the current batch of dial test messages distributed to a first partition, and determine a second number of the current batch of dial test messages consumed in the first partition; the first partition is any partition corresponding to the dial test message topic; Calculating a ratio of the second number to the first number, and determining a message loss rate corresponding to the first partition according to the ratio; When the message loss rate is greater than or equal to a first threshold, it is determined that an abnormality occurs in the first partition.

8. The method according to claim 6, characterized in that The determining whether an abnormality occurs in the distributed message cluster according to the processing time parameter corresponding to the dial test message of the current batch consumed includes: Determine the consumption delay corresponding to each dial test message consumed in the first partition according to the distribution timestamp of the current batch of dial test messages distributed to the first partition and the consumption timestamp of the current batch of dial test messages consumed from the first partition; the first partition is any partition corresponding to the dial test message topic; Based on the fact that the consumption delay of the dial test message is greater than or equal to a second threshold, it is determined that an abnormality occurs in the first partition; wherein the processing time parameter includes the consumption delay.

9. The method according to claim 1 or 2, characterized in that: The determining whether an abnormality occurs in the distributed message cluster based on the consumption statistical parameters corresponding to the dial test messages of the current batch includes: Based on the consumption statistical parameter indicating that multiple partitions corresponding to the dial test message topic in the same disk are abnormal, determining that the disk is abnormal; and / or, Based on the consumption statistical parameter indicating that multiple disks in the same server are abnormal, it is determined that the server is abnormal.

10. A dialing device, characterized in that: The device comprises: A generating unit generates a dial test message of the current batch based on cluster attribute information of the distributed message cluster, wherein the dial test message carries partition address information and batch identification information; the partition address information of each dial test message in the current batch covers the address of each disk in the distributed message cluster; each disk includes one or more dial test message topics; A distribution unit, configured to distribute each of the dial test messages to a partition of the distributed message cluster based on the partition address information of each of the dial test messages in the current batch; wherein each of the dial test message topics corresponds to at least one partition; A consumption unit, configured to consume the dial test messages belonging to the current batch from the distributed message cluster based on the batch identification information; The determination unit is used to determine whether an abnormality occurs in the distributed message cluster based on the consumption statistical parameters corresponding to the dial test messages of the current batch.

11. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images