Consistency detection method and device, and electronic device

Abstract

This application relates to the field of computer technology, specifically to a consistency detection method, apparatus, and electronic device. The method includes acquiring a message forwarding link in a messaging system; acquiring message transmission record files of each node in the message forwarding link and statistical information on the number of transmitted messages for each node, according to the message forwarding link; the message transmission record files include message transmission records generated for the transmitted messages; performing message content consistency verification based on the message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result; performing message quantity consistency verification based on the statistical information on the number of nodes in the message forwarding link to obtain a second verification result; and determining the message consistency verification result of the messaging system based on the first and second verification results. The method provided in this application realizes automated detection of message consistency in a messaging system, improving the detection efficiency of consistency detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a consistency detection method, apparatus, and electronic device. Background Technology

[0002] With the explosive growth of data, distributed architectures are increasingly needed to achieve massive data processing capabilities and scalable computing power. In distributed scenarios, data messages are replicated, forwarded, and consumed across different nodes in the messaging system. Because these nodes are decoupled from each other, data loss or misdelivery can easily occur, leading to data inconsistencies across multiple nodes.

[0003] In related technologies, testers typically manually verify a small amount of data between multiple nodes to determine whether the data between multiple nodes is consistent, which results in low detection efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application propose a consistency detection method, apparatus, and electronic device, which can automatically detect message consistency among multiple nodes to solve the problem of low efficiency of manual detection.

[0005] The embodiments of this application are implemented using the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a consistency detection method, the method comprising: acquiring a message forwarding link in a messaging system; acquiring message transmission record files of each node in the message forwarding link and quantity statistics of each node for transmitted messages, wherein the message transmission record files include message transmission records generated for transmitted messages, and each message transmission record includes a message identifier and message content of the transmitted message; performing message content consistency verification based on the message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result; performing message quantity consistency verification based on the quantity statistics of multiple nodes in the message forwarding link to obtain a second verification result; and determining the message consistency verification result of the messaging system based on the first verification result and the second verification result.

[0007] Secondly, embodiments of this application provide a consistency detection device, comprising: a first acquisition module, configured to acquire a message forwarding link in a messaging system; a second acquisition module, configured to acquire message transmission record files of each node in the message forwarding link and quantity statistics of each node for transmitted messages, according to the message forwarding link; the message transmission record files include message transmission records generated for transmitted messages, and each message transmission record includes a message identifier and message content of the transmitted message; a first verification module, configured to perform message content consistency verification based on the message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result; a second verification module, configured to perform message quantity consistency verification based on the quantity statistics of multiple nodes in the message forwarding link to obtain a second verification result; and an output module, configured to determine the message consistency verification result of the messaging system based on the first verification result and the second verification result.

[0008] In some implementations, the message transmission records generated by different nodes have the same format; the first verification module includes a deduplication module for deduplicating the message transmission record files of each node; a hash calculation module for performing hash calculation on the deduplicated message transmission record files of each node to obtain the hash value corresponding to each node; and a matching module for comparing the hash values ​​of multiple nodes in the message forwarding link to generate the first verification result.

[0009] In some implementations, the message transmission record also includes the topic to which each transmitted message belongs; a deduplication module is specifically used to filter out duplicate message transmission records with the same topic and the same message identifier in the message transmission record file of each node.

[0010] In some implementations, the consistency detection device further includes a third acquisition module, configured to acquire, based on consistency test information, the target topic to which the message used for consistency testing belongs; a subscription prompt module, configured to send topic subscription prompt information to nodes after the first node in the message forwarding link based on the target topic, so that each node after the first node subscribes to the target topic; and a topic prompt module, configured to send topic prompt information to the first node based on the target topic, so that the first node sends a message belonging to the target topic during the test.

[0011] In some implementations, the consistency detection device further includes a content anomaly prompting module, which is used to generate anomaly node prompting information based on the anomaly node in the message forwarding link if the first verification result indicates that the message content in the message system is inconsistent; wherein, the anomaly node refers to a node located after the first node in the message forwarding link, and whose corresponding hash value is different from the hash value of the first node.

[0012] In some implementations, message transmission between adjacent nodes in the message forwarding link is performed via a message queue; the message forwarding link includes a producer node as a producer, a consumer node as a consumer, and an intermediate node as a transmission intermediary between the producer node and the consumer node; the quantity statistics of the producer node are the number of transmissions; the quantity statistics of the intermediate node include the number of receptions and the number of transmissions; the quantity statistics of the consumer node are the number of receptions; the second verification module includes a first comparison unit, used to compare the number of transmissions of the preceding node with the number of transmissions of the following node for any two adjacent nodes in the message forwarding link. The receiving quantity statistics of the nodes are compared to determine the quantity verification result corresponding to the two adjacent nodes; the quantity verification result is used to indicate whether the number of messages successfully sent by the preceding node and the number of messages successfully received by the following node are the same; the second comparison unit is used to perform local quantity verification based on the receiving quantity statistics and sending quantity statistics of the same intermediate node to determine the local quantity verification result of each intermediate node, the local quantity verification result is used to indicate whether the number of messages successfully received and the number of messages successfully sent by the intermediate node are the same; the determination unit is used to determine the second verification result based on the quantity verification result corresponding to the two adjacent nodes and the local quantity verification result of each intermediate node.

[0013] In some implementations, the consistency detection device further includes a quantity anomaly prompting module, used to determine a quantity anomaly node based on the second verification result if the second verification result indicates that the number of messages in the message system is inconsistent; the quantity anomaly node refers to at least one of two adjacent nodes whose quantity verification results indicate different quantities, and an intermediate node whose quantity verification results indicate different quantities; and to generate message quantity anomaly prompting information based on the quantity anomaly node.

[0014] In some implementations, the consistency detection device further includes a preparation module for sending a file clearing instruction to each node in the message forwarding link, so that each node clears its own message transmission record file.

[0015] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory storing computer instructions, which, when executed by the processor, implement the above-described method.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the above-described method.

[0017] Fifthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the above-described method.

[0018] This application provides a consistency detection method, apparatus, and electronic device. The method involves acquiring a message forwarding link in a messaging system; acquiring message transmission record files for each node in the message forwarding link and statistical information on the number of transmitted messages for each node; the message transmission record file includes message transmission records generated for the transmitted messages, and each message transmission record includes the message identifier and message content of the transmitted message; performing message content consistency verification based on the message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result; obviously, the first verification result can show whether there is message content inconsistency caused by message mistransmission at each node; subsequently, performing message quantity consistency verification based on the statistical information of multiple nodes in the message forwarding link to obtain a second verification result. The result is clear; the second verification result can show whether there is any inconsistency in the number of messages due to missing messages at each node; then, based on the first and second verification results, the message consistency verification result of the message system is determined; the method provided in this application performs consistency verification of message content through the message transmission record file of each node, and performs consistency verification of message quantity through the statistical information of the number of transmitted messages by each node. This not only realizes automatic verification of message consistency of multiple nodes involved in the message forwarding link, improving the efficiency of message consistency verification, but also performs consistency verification from both the message content and message quantity aspects, improving the accuracy of message consistency verification. It can effectively identify nodes with abnormal message content and abnormal message quantity, and can identify situations such as missing or incorrect messages in the message forwarding link.

[0019] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A A schematic diagram of an application scenario involving an embodiment of this application is shown.

[0022] Figure 1B A schematic diagram of a message forwarding link provided in an embodiment of this application is shown.

[0023] Figure 1C A schematic diagram of a messaging system provided in an embodiment of this application is shown.

[0024] Figure 2 A schematic diagram of the first process of a consistency detection method provided in an embodiment of this application is shown.

[0025] Figure 3 A schematic diagram of a message forwarding link provided in another embodiment of this application is shown.

[0026] Figure 4 An embodiment of this application is shown. Figure 2 A flowchart of step S130.

[0027] Figure 5 An embodiment of this application is shown. Figure 2 A flowchart of step S140.

[0028] Figure 6 The diagram shows a second flowchart of a consistency detection method provided in an embodiment of this application.

[0029] Figure 7 The diagram illustrates the third process flow of a consistency detection method provided in an embodiment of this application.

[0030] Figure 8 A schematic diagram of the overall process of consistency detection provided in an embodiment of this application is shown.

[0031] Figure 9 The diagram illustrates the processing flow of different nodes in an embodiment of this application.

[0032] Figure 10 A schematic diagram of a consistency detection device according to an embodiment of this application is shown.

[0033] Figure 11 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0034] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0036] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In this document, "multiple" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. In the following description, references to "some embodiments or some embodiment methods" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0038] To facilitate understanding of this application, some terms will be explained below.

[0039] Centralized strip deployment: A distributed deployment scenario that typically refers to dividing the modules involved in the data system (including upstream and downstream calling modules) into multiple centers in the network link. Each center is deployed independently and does not interfere with each other.

[0040] Fast Message Queue (FMQ) is a message queue system that enables asynchronous communication. It creates message queues based on required attributes, allowing producers to send messages and consumers to pull messages from them for processing.

[0041] Producer: A service that produces messages.

[0042] Consumer: Receives messages from the message queue and consumes them.

[0043] With the explosive growth of data, distributed architectures are increasingly needed to achieve massive data processing capabilities and scalable computing power. In distributed scenarios, data is replicated, forwarded, and consumed between different nodes. Because the nodes are decoupled from each other, data can be sent multiple times, missed times, or incorrect times, leading to data inconsistency across multiple nodes.

[0044] In related technologies, testers typically manually verify a small amount of data between multiple nodes to determine whether the data between multiple nodes is consistent, which results in low detection efficiency.

[0045] To address the aforementioned issues, this application proposes a consistency detection method, apparatus, and electronic device.

[0046] Please see Figure 1A , Figure 1A A schematic diagram of an application scenario involving an embodiment of this application is provided, including a terminal 10 and a messaging system 20, wherein the terminal 10 and the messaging system 20 are connected via a wired or wireless network.

[0047] The messaging system 20 can have multiple nodes deployed. These nodes in the same message forwarding link forward messages according to their order of appearance within the link. Figure 1A The example provides three nodes: 201, 202, and 203. It should be noted that... Figure 1A The three nodes shown do not constitute a limit on the number of nodes in the messaging system 20.

[0048] Please see Figure 1B , Figure 1B A schematic diagram of a message forwarding link according to an embodiment of this application is provided. According to the role of the nodes, the nodes in the message forwarding link can be divided into nodes as producers, nodes as message brokers, nodes as workers, and nodes as consumers.

[0049] In this process, the producer produces and sends messages; after receiving a message from the producer, the message broker finds the subscription configuration for the corresponding topic in the configuration file and forwards the message to the workers who have subscribed to that topic; after receiving the message, the worker finds the consumption subscription configuration for the corresponding topic in the configuration file and forwards the message to the consumers who have subscribed to that topic; finally, the consumer receives and consumes the message.

[0050] Please see Figure 1C , Figure 1CA schematic diagram of a messaging system 20 according to an embodiment of this application is provided, wherein multiple nodes in the messaging system 20 are deployed in city A and city B respectively.

[0051] Each city has multiple central clusters deployed, and each central cluster contains message brokers and workers. Different central clusters located in the same city can be divided into intra-domain clusters and extra-domain clusters according to the network environment in which the cluster is located. Messages received by extra-domain clusters will not be forwarded to intra-domain clusters.

[0052] exist Figure 1C The message system shown can contain multiple message forwarding links. Taking the message forwarding link where producer 2 is located as an example, the message produced by producer 2 is sent to the message broker in the central cluster 4, and then forwarded by the message broker in the central cluster 4 to the worker in the central cluster 4. Finally, the worker in the central cluster 4 forwards the message to consumer 4.

[0053] Meanwhile, workers in central cluster 4 can also replicate the received messages to workers in central cluster 5, which is in the same domain as central cluster 4. Furthermore, workers in central cluster 4 can also replicate the received messages to workers in various central clusters (such as central clusters 1, 2, and 3) in external cluster A, which are in the same city as central cluster 4. Thus, intra-domain replication is achieved through workers.

[0054] In addition, workers in central cluster 4 can also transmit messages to message brokers in other domain clusters (such as central clusters 9 and 10) located in different cities from central cluster 4. The messages are then forwarded to consumers through message brokers and workers in the corresponding central clusters, thus enabling cross-city replication through message brokers.

[0055] When executing the method provided in this application, terminal 10 can send a message consistency detection command to message system 20. Then, message system 20 obtains the message forwarding link in the message system; according to the message forwarding link, it obtains the message transmission record file of each node in the message forwarding link and the quantity statistics information of each node for the transmitted message; the message transmission record file includes message transmission records generated for the transmitted message, and each message transmission record includes the message identifier and message content of the transmitted message; message content consistency verification is performed based on the message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result; message quantity consistency verification is performed based on the quantity statistics information of multiple nodes in the message forwarding link to obtain a second verification result; and the message consistency verification result of the message system is determined based on the first verification result and the second verification result.

[0056] Afterwards, the messaging system 20 can return the message consistency verification result to the terminal 10, so that testers can view the message consistency verification result through the terminal 10 and perform anomaly investigation based on the message consistency verification result.

[0057] Among them, terminal 10 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart TV, wearable device, smartwatch, virtual reality device, vehicle terminal, smart TV, etc.

[0058] The messaging system 20 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0059] In some embodiments, the method of this application can also be executed by terminal 10. Terminal 10 can perform consistency detection processing according to the method of this application to obtain the message consistency verification result of the message system.

[0060] The present application will now be described in detail with reference to the embodiments.

[0061] Please see Figure 2 , Figure 2 A first flowchart of the consistency detection method provided in this application is given. This method can be executed by an electronic device, which can be a server, terminal, etc., without specific limitations. Figure 2 As shown, the consistency detection method includes steps S110-S150:

[0062] S110. Obtain the message forwarding link in the messaging system.

[0063] The messaging system is a system used for message transmission; it consists of producers producing and sending messages, which are then forwarded through a series of nodes to reach consumers, who then receive and consume the messages.

[0064] A message forwarding chain refers to the path a message takes from production by a producer to consumption by a consumer; furthermore, a message produced by a producer can be forwarded to one or more consumers through a message forwarding chain.

[0065] In this system, a producer can produce messages on one or more topics, and a consumer can subscribe to messages on one or more topics. In a message forwarding chain, the topics to which the producer's messages belong, the topics subscribed to by the consumer, and the topics subscribed to by other nodes in the message forwarding chain are all the same.

[0066] The messaging system can be a distributed system, and the message forwarding links within the messaging system are pre-defined. In some embodiments, the message forwarding links can cover the entire path from the producer to all consumer nodes in the messaging system. There can be multiple consumers in the messaging system; therefore, in this case, the message forwarding links include the transmission path from the producer to multiple message consumers. For ease of description, the node acting as the producer is referred to as the producer node, the node acting as the consumer is referred to as the consumer node, and other nodes traversed between the producer node and the consumer node are referred to as intermediate nodes. It is understood that between a producer node and a consumer node, there can be one or more intermediate nodes. In this application, a message forwarding link includes one producer node, and the number of consumer nodes and intermediate nodes is unlimited.

[0067] Figure 3 An exemplary diagram of a message forwarding chain in a messaging system is shown. In this message forwarding chain, the message generated by producer node P1 is sent to intermediate nodes P2-1 and P2-2; intermediate node P2-1 can send the message from producer node P1 to consumer node P3-1; and intermediate node P2-2 can send the message from producer node P1 to consumer nodes P3-1 and P3-2.

[0068] In some embodiments, nodes in the messaging system that need to be tested for consistency can be specified according to testing needs, and then a message forwarding link can be constructed according to the specified nodes that need to be tested for consistency. In this case, the message forwarding link may include some or all of the consumer nodes in the messaging system in addition to the producer nodes.

[0069] A messaging system can be a distributed system. Different nodes in a messaging system can be physical device nodes or software nodes. When a node is a software node, two different software nodes can be deployed on different physical devices (such as servers, terminals, and edge computing devices) or on the same physical device.

[0070] The method described in this application can be executed by a node in the messaging system that is on the message forwarding link, or by a node in the messaging system that is not on the message forwarding link, or by other devices outside the messaging system; no specific limitations are made here.

[0071] S120. According to the message forwarding link, obtain the message transmission record file of each node in the message forwarding link and the quantity statistics of each node for the transmitted message; the message transmission record file includes the message transmission record generated for the transmitted message, and each message transmission record includes the message identifier of the transmitted message and the message content of the transmitted message.

[0072] The message forwarding link includes the node identifiers of each node. Therefore, the message transmission record file of the node and the statistical information of the number of transmitted messages can be obtained from the corresponding node according to the node identifier in the message forwarding link.

[0073] A node's message transmission record file is a file created locally by the node to store the message transmission records it generates during the message transmission process. The message transmission process can be a sending process, a receiving process, or both. For the sender, the message transmission record generated by the sender can include records generated for successfully sent messages as well as records generated for messages that failed to be sent. For the receiver, the message transmission record generated by the receiver can include records generated for successfully received messages as well as records generated for messages that failed to be received.

[0074] A message identifier can be a serial number used to uniquely identify a message; the message content of the transmitted message can include the specific content of the message itself. In addition, the message transmission record may also include the subject of the transmitted message, the timestamp information of the transmission, etc.

[0075] The quantity statistics for transmitted messages refer to the number of messages counted by the node during the sending or receiving process. If a node acts only as a message sender (e.g., a producer node), its quantity statistics are the number of messages sent. If a node is an intermediary node that both sends and receives messages, its quantity statistics include both the number of messages received and the number of messages sent. If a node acts only as a message receiver (e.g., a consumer node), its quantity statistics are the number of messages received. The sending quantity statistics can include the number of messages sent, the number of messages successfully sent, and the number of messages that failed to send. For the same node, the number of messages sent equals the sum of the number of messages successfully sent and the number of messages that failed to send. The receiving quantity statistics can include the number of messages received, the number of messages successfully received, and the number of messages that failed to receive. Similarly, for the same node, the number of messages received equals the sum of the number of messages successfully received and the number of messages that failed to receive.

[0076] In some implementations, the number of transmitted messages can be statistically analyzed by parsing the message transmission records in a message transmission log file. It is worth noting that, since message transmission records in the file may contain duplicate or retransmitted messages, deduplication can be performed on the message transmission records before obtaining the number of transmitted messages.

[0077] In some embodiments, before testing the nodes in the message forwarding link of the messaging system, detection programs can be deployed on each node in the message forwarding link. During the testing of the nodes in the message forwarding link of the messaging system, the detection programs running on the nodes can generate message transmission records based on the messages sent or received, and store the message transmission records in a message transmission record file on the node's local machine. In addition, the detection programs running on the nodes can count the number of messages sent and received by the nodes in real time to obtain the corresponding quantity statistics information.

[0078] In some implementations, prior to step S120, the consistency detection method further includes sending a file clearing instruction to each node in the message forwarding link so that each node clears its own message transmission record file.

[0079] Through the above implementation method, before performing consistency testing, each node clears its own message transmission record file by using a file clearing command. This ensures that the message transmission record file and statistical information obtained during the consistency testing process only contain information about messages transmitted during the consistency test, thus avoiding the impact on the accuracy of consistency testing if the message transmission record file includes message transmission records of messages transmitted outside of the consistency test period.

[0080] S130. Verify message content consistency based on message transmission record files of multiple nodes in the message forwarding link to obtain the first verification result.

[0081] The first verification result indicates whether the message content of the same message transmitted by different nodes in the message forwarding link of the messaging system is consistent. Furthermore, message content consistency verification also needs to verify whether the same message has passed through all nodes in the message forwarding link.

[0082] In some implementations, since each message transmission record in the message transmission record file includes the message identifier and message content of the transmitted message, the message content of each node in the message forwarding link for the same message (i.e., the message identifier is the same) can be obtained based on the message identifier. Then, the message content of multiple nodes in the message forwarding link for the same message can be compared. If the message transmission record files of all nodes in the message forwarding link include the message content for the same message and the message content is consistent, it can be determined that the message content for that message is consistent. For other messages, message content consistency verification can be performed in a similar manner.

[0083] In other implementations, the message transmission records generated by different nodes have the same format; therefore, file matching can be performed directly on the message transmission record files. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 Exemplary embodiments provided in this application are given. Figure 2 A flowchart of step S130 is shown. Step S130 includes steps S131-S133:

[0084] S131. Perform message transmission record deduplication processing on the message transmission record files of each node.

[0085] Because nodes may send multiple messages (send the same message to multiple nodes) or resend messages (send the same message to the same node multiple times) during the message transmission process, duplicate message transmission records may appear in both the message transmission records generated by the message sender and the message transmission records generated by the message receiver. Therefore, before calculating the hash value of the message transmission record file, it is necessary to remove duplicate message transmission records from the message transmission record file.

[0086] In some implementations, message transmission records of each node can be deduplicated using message identifiers; specifically, for multiple message transmission records with the same message identifier in the message transmission record file, only one message transmission record is retained.

[0087] In some other implementations, if the message transmission record also includes the topic to which each transmitted message belongs, step S131 may further include: filtering out duplicate message transmission records with the same topic and the same message identifier in the message transmission record file of each node.

[0088] For multiple message transmission records with the same topic and message identifier in a node's message transmission log file, only one record is retained. Filtering message transmission records by combining the message identifier with the topic effectively ensures message filtering and avoids misidentifying multiple message transmission records with different topics but the same message identifier as the same message.

[0089] S132. Perform hash calculation on the message transmission record files of each node after deduplication to obtain the hash value corresponding to each node.

[0090] Hash calculations can be implemented using hash functions, such as MD5 and SHA.

[0091] S133. Compare the hash values ​​of multiple nodes in the message forwarding link to generate the first verification result.

[0092] Specifically, if multiple nodes have the same hash value, the generated first verification result indicates that the message content transmitted by each node is the same; if at least one node among the multiple nodes has a different hash value, the generated first verification result indicates that there is a node in the message system whose transmitted message content is inconsistent with that of other nodes. It can be understood that if two nodes have the same hash value, it indicates that the data in the message transmission record files of the two nodes is the same.

[0093] In some implementations, if the first verification result indicates that the message content in the messaging system is inconsistent, an abnormal node prompt message is generated based on the abnormal node in the message forwarding link; wherein, an abnormal node refers to a node located after the first node in the message forwarding link, and whose corresponding hash value is different from the hash value of the first node.

[0094] The first node in the message forwarding chain is the producer node. During testing, all nodes in the chain except the first node theoretically receive messages produced and sent by the producer node. Therefore, the hash value of the first node is used as a reference, rather than the hash values ​​of other nodes, to avoid the possibility that message content has been modified or is abnormal at intermediate nodes, leading to misidentification of subsequent nodes' received messages as accurate message content.

[0095] It is understandable that the message transmission log file of the first node records the message content of all sent messages; therefore, if there is a node in the message forwarding chain after the first node with a hash value different from that of the first node, it means that the message content of the message transmitted by that node is abnormal.

[0096] In other implementations, an abnormal node may also refer to the first node in the message forwarding chain that is located after the first node and whose corresponding hash value is different from that of the first node.

[0097] In the above implementation, after deduplicating the message transmission record files of each node, the data in the entire deduplicated message transmission record file is hashed, and the message content of the messages transmitted between different nodes is verified by comparing the hash values. In this way, it is not necessary to perform consistency comparison on a single message basis, but to perform message content consistency verification on the entire message transmission record file, which can improve the efficiency of message content consistency verification.

[0098] S140. Verify message quantity consistency based on the quantity statistics of multiple nodes in the message forwarding link to obtain the second verification result.

[0099] The second verification result is used to indicate whether the number of messages transmitted by different nodes in the messaging system is consistent.

[0100] As mentioned above, for the first node in the message forwarding chain (i.e., the producer node), its quantity statistics are the number of messages sent; for the consumer node in the message forwarding chain, its quantity statistics are the number of messages received; and for the intermediate nodes in the message forwarding chain, its quantity statistics can include both the number of messages sent and the number of messages received.

[0101] In some embodiments, if adjacent nodes in a message forwarding link do not transmit messages through message queues, but instead send and receive messages directly, the node acting as the receiver can be determined for the node with the sender role (e.g., producer node and intermediate node). Figure 3 In the message forwarding chain shown, the nodes that act as the receivers of producer node P1 include intermediate node P2-1 and intermediate node P2-2; the nodes that act as the receivers of intermediate node P2-1 are consumer node P3-1; the nodes that act as the receivers of intermediate node P2-2 include consumer node P3-2 and consumer node P3-3. Then, message sending and receiving groups can be formed based on the nodes with the sender role and the corresponding receiver nodes. A message sending and receiving group includes one node with the sender role (referred to as the sender node for ease of description) and all the nodes that act as receivers of that sender node (referred to as receiver nodes). For example, based on... Figure 3The message forwarding chain shown can be divided into three message sending and receiving groups: (Producer node P1; Intermediate node P2-1, Intermediate node P2-2), (Intermediate node P2-1; Consumer node P3-1), and (Intermediate node P2-2; Consumer node P3-2, Consumer node P3-3). In each message sending and receiving group, the node before the semicolon (;) represents the sending node, and the node after the semicolon (;) represents the receiving node corresponding to the sending node.

[0102] Next, for each message sending and receiving group, based on the sending count statistics of the sending nodes and the receiving count statistics of all receiving nodes in the group, it is determined whether the number of messages sent by the sending nodes is consistent with the total number of messages received by all receiving nodes in the group, or whether the number of successfully sent messages by the sending nodes is consistent with the total number of successfully received messages by all receiving nodes in the group. If, in a message sending and receiving group, the number of successfully sent messages by the sending nodes is consistent with the total number of successfully received messages by all receiving nodes in the group, it can be determined that the number of messages transmitted between the sending nodes and receiving nodes in the group is essentially consistent.

[0103] If it is determined that the number of messages transmitted between the sender and receiver nodes in all message sending and receiving groups is consistent, it can be determined that the number of messages transmitted in the message system is consistent. Conversely, if there is a discrepancy in the number of messages transmitted between the sender and receiver nodes in at least one message sending and receiving group, it can be determined that there are nodes in the message system with abnormal message transmission numbers.

[0104] In other implementations, message transmission between adjacent nodes in the message forwarding link is performed via message queues; the message forwarding link includes a producer node as a producer, a consumer node as a consumer, and an intermediate node as a transmission intermediary between the producer and consumer nodes; such as Figure 5 As shown, Figure 5 The embodiments provided in this application are given Figure 2 The flowchart of step S140 is shown below. Step S140 includes steps S141-S143:

[0105] S141. For any two adjacent nodes in the message forwarding link, compare the sending statistics of the preceding node with the receiving statistics of the following node to determine the quantity verification result of the two adjacent nodes; the quantity verification result is used to indicate whether the number of messages successfully sent by the preceding node and the number of messages successfully received by the following node are the same.

[0106] In a message forwarding chain, two adjacent nodes refer to two nodes that have a message sending and receiving relationship and are adjacent to each other on the message forwarding chain. For example... Figure 3 In the example, producer node P1 and intermediate node P2-1 are both adjacent nodes.

[0107] In a pair of adjacent nodes, the "first" node refers to the node that sent the message, and the "second" node indicates the node that received the message. For example, in... Figure 3 In the producer node P1 and intermediate node P2-1, producer node P1 is the first node and intermediate node P2-1 is the second node.

[0108] In a messaging system, if any two adjacent nodes in the message forwarding link transmit messages via message queues, the sending node publishes the message to the message queue, and the receiving node retrieves the message from the message queue. In this way, if a sending node corresponds to multiple receiving nodes, the sending node does not need to send a message to each receiving node separately. For a single message, the sending node publishes the message to the message queue, and the multiple receiving nodes corresponding to that sending node then retrieve the message from the message queue respectively.

[0109] Therefore, assuming the messages are forwarded correctly, the preceding node should send all received messages to the message queue, and the following node should receive all messages from the message queue. In other words, the number of messages sent recorded in the sending statistics of the preceding node should be equal to the number of messages received recorded in the receiving statistics of the following node.

[0110] If the number of messages sent recorded in the sending statistics of the preceding node is not equal to the number of messages received recorded in the receiving statistics of the following node, it means that the message reception at the following node is abnormal.

[0111] Furthermore, the specific anomaly can be determined by the relationship between the number of messages sent by the preceding node and the number of messages received by the following node. Specifically, if the number of messages sent by the preceding node is greater than the number of messages received by the following node, it means that the following node has missed receiving messages; if the number of messages sent by the preceding node is less than the number of messages received by the following node, it means that the following node has received duplicate messages.

[0112] S142. Perform local quantity verification based on the received quantity statistics and sent quantity statistics messages of the same intermediate node, and determine the local quantity verification result of each intermediate node. The local quantity verification result is used to indicate whether the number of messages successfully received by the intermediate node is the same as the number of messages successfully sent.

[0113] In a messaging system, if any two adjacent nodes in the message forwarding chain transmit messages via message queues, when the intermediate node acts as the sender node, the sender node publishes the message to the message queue, and the receiver node, acting as the receiver, pulls the message from the message queue. In this way, even if one sender node corresponds to multiple receiver nodes, the sender node does not need to send a message to each receiver node separately. Similarly, when the intermediate node acts as the receiver node, the receiver node retrieves the message from the message queue.

[0114] Therefore, if messages are forwarded correctly, intermediate nodes should send all messages obtained from the previous message queue to the next message queue so that subsequent nodes can retrieve messages from the message queue. In other words, if messages are forwarded correctly, the number of messages successfully received by intermediate nodes should equal the number of messages successfully sent.

[0115] If the number of messages successfully received by an intermediate node differs from the number of messages successfully sent, it means that the intermediate node failed to send all the received messages to the next message queue, indicating that there is a message sending error at that intermediate node.

[0116] S143. Determine the second verification result based on the quantity verification results of the two adjacent nodes and the local quantity verification results of each intermediate node.

[0117] Specifically, if the quantity verification results of two adjacent nodes indicate that the number of messages successfully sent by the preceding node is different from the number of messages successfully received by the following node, or if the local quantity verification result of an intermediate node indicates that the number of messages successfully received by the intermediate node is different from the number of messages successfully sent, the second verification result is used to indicate that the number of messages transmitted by different nodes in the messaging system is inconsistent.

[0118] If the quantity verification results for any two adjacent nodes indicate that the number of messages successfully sent by the preceding node is the same as the number of messages successfully received by the following node, and the local quantity verification results for each intermediate node indicate that the number of messages successfully received by the intermediate node is the same as the number of messages successfully sent, then the second verification result is used to indicate that the number of messages transmitted by different nodes in the messaging system is consistent.

[0119] In addition, the sending statistics of intermediate nodes can be compared with those of producer nodes. If the number of messages sent by an intermediate node is equal to the number of messages sent by a producer node, it means that the intermediate node has successfully forwarded all messages. If the number of messages sent by an intermediate node is not equal to the number of messages sent by a producer node, it means that the intermediate node has failed to forward all messages. In this case, it may be that the intermediate node has a sending or receiving error, or it may be that the upstream node of the intermediate node has an error.

[0120] In other embodiments, if there is only one receiver node acting as the producer node in the message forwarding link, and message transmission is performed between any two adjacent nodes in the message forwarding link via a message queue, step S140 may further include steps one to three:

[0121] Step 1: For each node in the message forwarding chain other than the first node, based on the sending count statistics of the first node, the queue reading count statistics of the node, and the receiving count statistics, obtain the quantity verification result of the node. The quantity verification result is used to indicate whether the queue message reading count of the node is the same as the message successfully sent count of the first node; and whether the message successfully received count of the node is the same as the queue message reading count.

[0122] Understandably, assuming all nodes forward messages correctly, the number of messages read from a node's queue should equal the number of messages successfully received, meaning that the node can successfully retrieve all messages from the message queue. At the same time, its number of messages read from the queue should equal the number of messages successfully sent by the first node, meaning that the previous node successfully forwarded all messages.

[0123] Obviously, if the number of messages read from a node's queue is not equal to the number of messages successfully received, it means that the node cannot correctly receive all the messages in the message queue; if the number of messages read from a node's queue is not equal to the number of messages successfully sent by the first node, it means that the node's predecessor failed to forward all the messages.

[0124] In other implementations, the quantity statistics of other nodes in the message forwarding link besides the first node may also include the quantity statistics of failed receptions, wherein the number of queue messages read is equal to the sum of the number of successfully received messages and the number of failed message receptions; in this case, the quantity verification result can also be used to indicate whether the number of failed message receptions is 0. Obviously, if the number of failed message receptions of a node is not 0, it means that the node cannot correctly receive all messages in the message queue.

[0125] Step 2: Determine the second verification result based on the verification results of the quantities of each node other than the first node.

[0126] Specifically, if a node's verification result indicates that the number of queue messages read by that node is different from the number of messages successfully sent by the first node; or, if the number of queue messages read by that node is different from the number of messages successfully received, then the second verification result is determined to indicate that the number of messages among the nodes in the messaging system is inconsistent.

[0127] If the verification results for any node other than the first node indicate that the number of queue messages read by that node is the same as the number of messages successfully sent by the first node, and the number of queue messages read by that node is the same as the number of messages successfully received, it means that all data has been forwarded correctly, and the second verification result is used to indicate that the number of messages in each node of the messaging system is consistent.

[0128] Step 3: Determine the message consistency verification result of the messaging system based on the first verification result and the second verification result.

[0129] Specifically, if the first verification result indicates that the message content of all nodes in the message system is consistent, and the second verification result indicates that the message quantity of all nodes in the message system is consistent, then the message consistency verification result of the message system is determined to indicate that the messages in the message system are consistent.

[0130] If the first verification result indicates that there are nodes in the message system with abnormal message content, or if the second verification result indicates that there are nodes in the message system with abnormal message quantity, the message consistency verification result of the message system is determined to indicate that the messages in the message system are inconsistent.

[0131] The method provided in this application embodiment performs message content consistency verification based on message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result. The first verification result can show whether there is message content inconsistency caused by message mistransmission at each node. Then, message quantity consistency verification is performed based on the quantity statistics of multiple nodes in the message forwarding link to obtain a second verification result. The second verification result can show whether there is message quantity inconsistency caused by message omission at each node. Then, based on the first verification result and the second verification result, the message consistency verification result of the message system is determined. This not only realizes automatic verification of message consistency of multiple nodes involved in the message forwarding link, improving the efficiency of message consistency verification, but also performs consistency verification from both message content and message quantity aspects, which can effectively identify nodes with abnormal message content and abnormal message quantity, and can identify situations such as omission and mistransmission in the message forwarding link.

[0132] In some implementations, please refer to Figure 6 , Figure 6A second flowchart of the consistency detection method provided in this application embodiment is shown. After step S140, the consistency detection method may further include steps S210-S220:

[0133] S210. If the second verification result indicates that the number of messages in the message system is inconsistent, determine the abnormal number node based on the second verification result. The abnormal number node refers to at least one of the two adjacent nodes whose numbers are different according to the number verification result, or the intermediate node whose number is different from the number indicated by the local number verification result.

[0134] S220. Generate message quantity abnormality prompt information based on the number of abnormal nodes.

[0135] In some implementations, the message indicating an abnormal message quantity may include the node identifier of the node with the abnormal quantity; it may also include the quantity statistics of the node with the abnormal quantity.

[0136] In the above implementation, when the number of messages in the message system is inconsistent as indicated by the second verification result, message quantity anomaly prompts are generated through message anomaly nodes. This facilitates testers in troubleshooting based on the message quantity anomaly prompts and improves the efficiency of anomaly troubleshooting.

[0137] In some implementations, please refer to Figure 7 , Figure 7 The third flowchart of the consistency detection method according to the embodiments of this application is shown. Before step S120, the consistency detection method further includes steps S310-S330:

[0138] S310. Based on the consistency test information, obtain the target topic to which the message used for consistency testing belongs.

[0139] Among them, the consistency test information is used to indicate the message forwarding links that need to undergo consistency testing.

[0140] The target topic of the message used for consistency testing is the topic to which the first node in the message forwarding chain needs to send the message, which is also the topic that other nodes in the message forwarding chain need to subscribe to.

[0141] S320. Based on the target topic, send topic subscription prompts to all nodes after the first node in the message forwarding chain so that each node after the first node can subscribe to the target topic.

[0142] S330. Based on the target topic, send a topic hint message to the first node so that the first node can send messages belonging to the target topic during the test.

[0143] Through the above implementation method, the first node of the message forwarding link that needs to be tested for consistency will send a message belonging to the target topic during the test. After that, the message belonging to the target topic will be forwarded and subscribed to other nodes of the target topic, that is, other nodes after the first node in the message forwarding link, so as to forward the message according to the message forwarding link, so as to realize the consistency test of the message forwarding link.

[0144] In some implementations, please refer to Figure 8 , Figure 8 The diagram illustrates the overall process of consistency detection in an embodiment of this application. Taking a message transmission link as an example, the message transmission link includes a producer, a message broker, a worker, and a consumer.

[0145] After running the consistency check procedure, the messages used for consistency checks are... Figure 8 The message transmission link shown is used for transmission. After the transmission is completed, each node (including...) is retrieved. Figure 8 The message transmission log files (shown as producers, message brokers, workers, and consumers) are then used to perform consistency checks according to the method provided in this application. The quantity statistics can be obtained by parsing the message transmission log files or by statistically analyzing the message transmission process.

[0146] In some implementations, during the consistency check process, consumers can also report real-time quantity statistics to producers. These statistics include, for example, the number of messages received, successfully consumed, and failed to be consumed at any given moment. Producers can then match the real-time quantity statistics from consumers with their own to quickly determine if there are any missed or excessive messages. For instance, if a consumer reports receiving 100 messages, successfully consuming 98, and failing to consume 2, and the producer has already produced 120 messages, there is a high probability of missed messages. However, since message transmission also takes time, relying solely on the real-time quantity statistics from consumers and producers provides only a quick and rough assessment with low accuracy. Therefore, the methods described in the aforementioned embodiments are needed to accurately determine message consistency in the messaging system.

[0147] Please see Figure 9 , Figure 9 Figure a shows a schematic diagram of the producer's process in consistency detection.

[0148] For producers, after consistency checks begin and before producers produce and send messages, the corresponding message transmission record file for the producer is initialized, that is, the message transmission record file is cleared. After that, the producer produces and sends messages.

[0149] In some implementations, during message sending, the message sending status is recorded through a message transmission log file. At the same time, real-time statistics on the number of messages sent are written to shared memory. The shared memory also receives real-time statistics on the number of consumers. Thus, a rough real-time detection can be performed using the real-time statistics on the number of producers and consumers in the shared memory, further improving the efficiency of consistency detection.

[0150] In some implementations, the message sending efficiency of the producer can be improved by creating multiple threads.

[0151] In some implementations, the message transmission record for each message may include a message identifier (seq_no), a topic (topic_name), and a message content (msg). The storage format of the message transmission record may be as follows:

[0152] topic_name=xxxx&seq_no=xxxx&msg=xxxx.

[0153] After the message sending is completed, the number of messages sent is counted and reported to facilitate comparison with the number of messages reported by other nodes.

[0154] It is worth mentioning that due to the existence of multiple retransmissions, the message transmission record file may contain multiple records of the same message. Therefore, the obtained message transmission record file can be preprocessed by calling the preprocessor in the shared program library. Preprocessing operations include deduplication, sorting, and deleting redundant fields.

[0155] Meanwhile, in order to ensure that the message transmission log files of each node are hash-matched, the message transmission log files of each node need to have the same data format.

[0156] Figure 9 Figure b shows a flowchart of the consumer's consistency check process. Its processing flow is similar to that of the producer. The difference is that for the producer, its quantity statistics only include the number of messages sent, while for the consumer, its quantity statistics include the number of messages received, the number of messages successfully consumed, and the number of messages that failed to be consumed.

[0157] Figure 9Figure 'c' shows a flowchart of the message broker and worker in consistency detection. The processing flow is similar to that of the producer. The difference is that the message broker and worker do not report real-time quantity statistics to the shared memory. Their quantity statistics include the number of messages sent. For the message broker and worker, their quantity statistics also include the number of messages received, the number of messages successfully received, and the number of messages that failed to be received.

[0158] In the above application process, the detection program can be deployed on various nodes of the message system that are located across cities and domains. When the detection program is executed, it verifies the hash value of the message transmission record file of each node and the quantity statistics information, thereby achieving full coverage of each node in the message transmission link. And when the hash value of the message transmission record file of each node is consistent and the quantity statistics information is consistent, it is determined that the messages of each node in the message forwarding link of the message system are consistent.

[0159] Meanwhile, during the testing process, multiple threads can be created to improve the performance and efficiency of each node in the consistency testing process. Compared with manual testing in related technologies, this greatly improves the testing efficiency. Furthermore, due to the improved testing efficiency, stress testing and stability testing can be achieved by sending a large number of test messages, which greatly increases the confidence in the consistency testing.

[0160] It should be noted that in the above application process, before each consistency check, the shared memory and the message transmission log files of each node need to be initialized to clear the historical data and ensure the accuracy of the consistency check.

[0161] In some implementations, please refer to Figure 10 , Figure 10 A schematic diagram of a conformance detection device provided in an embodiment of this application is given. The conformance detection device 500 includes:

[0162] The first acquisition module 510 is used to acquire the message forwarding link in the messaging system.

[0163] The second acquisition module 520 is used to acquire message transmission record files of each node in the message forwarding link and the quantity statistics of each node for the transmitted messages according to the message forwarding link; the message transmission record file includes message transmission records generated for the transmitted messages, and each message transmission record includes the message identifier of the transmitted message and the message content of the transmitted message.

[0164] The first verification module 530 is used to verify the consistency of message content based on the message transmission record files of multiple nodes in the message forwarding link, and obtain the first verification result.

[0165] The second verification module 540 is used to verify the consistency of message quantity based on the quantity statistics of multiple nodes in the message forwarding link, and obtain the second verification result.

[0166] The output module 550 is used to determine the message consistency verification result of the messaging system based on the first verification result and the second verification result.

[0167] In some implementations, the message transmission records generated by different nodes have the same format; the first verification module 510 includes a deduplication module for deduplicating the message transmission record files of each node; a hash calculation module for performing hash calculation on the deduplicated message transmission record files of each node to obtain the hash value corresponding to each node; and a matching module for comparing the hash values ​​of multiple nodes in the message forwarding link to generate a first verification result.

[0168] In some implementations, the message transmission record also includes the topic to which each transmitted message belongs; a deduplication module is specifically used to filter out duplicate message transmission records with the same topic and the same message identifier in the message transmission record file of each node.

[0169] In some implementations, the consistency detection device 500 further includes a third acquisition module, used to acquire the target topic to which the message used for consistency testing belongs based on the consistency test information; a subscription prompt module, used to send topic subscription prompt information to nodes after the first node in the message forwarding link based on the target topic, so that each node after the first node subscribes to the target topic; and a topic prompt module, used to send topic prompt information to the first node based on the target topic, so that the first node sends messages belonging to the target topic during the test.

[0170] In some implementations, the consistency detection device 500 further includes a content anomaly prompting module, which generates anomaly node prompting information based on the anomaly node in the message forwarding link if the first verification result indicates that the message content in the message system is inconsistent; wherein, an anomaly node refers to a node located after the first node in the message forwarding link, and whose corresponding hash value is different from the hash value of the first node.

[0171] In some implementations, message transmission between adjacent nodes in the message forwarding link is performed via a message queue; the message forwarding link includes a producer node as a producer, a consumer node as a consumer, and an intermediate node as a transmission intermediary between the producer node and the consumer node; the quantity statistics of the producer node are the quantity statistics of the number of transmissions; the quantity statistics of the intermediate node include the quantity statistics of the number of receptions and the quantity statistics of the number of transmissions; the quantity statistics of the consumer node are the quantity statistics of the number of receptions; the second verification module 540 includes a first comparison unit, used to compare the quantity statistics of the first node with the quantity statistics of the second node in any two adjacent nodes in the message forwarding link. The receiving quantity statistics of the subsequent node are compared to determine the quantity verification result of the two adjacent nodes. The quantity verification result is used to indicate whether the number of messages successfully sent by the preceding node is the same as the number of messages successfully received by the following node. The second comparison unit is used to perform local quantity verification based on the receiving quantity statistics and sending quantity statistics of the same intermediate node to determine the local quantity verification result of each intermediate node. The local quantity verification result is used to indicate whether the number of messages successfully received by the intermediate node is the same as the number of messages successfully sent. The determination unit is used to determine the second verification result based on the quantity verification result of the two adjacent nodes and the local quantity verification result of each intermediate node.

[0172] In some implementations, the consistency detection device 500 further includes a quantity anomaly prompting module, used to determine a quantity anomaly node based on the second verification result if the second verification result indicates that the number of messages in the message system is inconsistent; the quantity anomaly node refers to at least one of two adjacent nodes whose quantity verification results indicate different quantities, or an intermediate node whose quantity verification results indicate different quantities; and to generate a message quantity anomaly prompting information based on the quantity anomaly node.

[0173] In some implementations, the consistency detection device 500 further includes a preparation module for sending a file clearing instruction to each node in the message forwarding link, so that each node clears its own message transmission record file.

[0174] Please see Figure 11 , Figure 11 A schematic diagram of a computer system suitable for implementing an electronic device according to embodiments of this application is shown. The electronic device can be the terminal described above, used to implement the consistency detection method provided in this application. It should be noted that... Figure 11 The computer system 1300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0175] like Figure 11As shown, the computer system 1300 includes a Central Processing Unit (CPU) 1301, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1302 or programs loaded from storage portion 1308 into Random Access Memory (RAM) 1303. The RAM 1303 also stores various programs and data required for system operation. The CPU 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An Input / Output (I / O) interface 1305 is also connected to the bus 1304.

[0176] The following components are connected to I / O interface 1305: an input section 1306 including a keyboard, mouse, microphone, etc.; an output section 1307 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to I / O interface 1305 as needed. Removable media 1311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1310 as needed so that computer instructions read from them can be loaded into storage section 1308 as needed.

[0177] In particular, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising computer instructions. When these computer instructions are executed by the central processing unit (CPU) 1301, various functions defined in the system of this application are performed.

[0178] This application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the methods described in any of the above method embodiments.

[0179] It should be noted that the computer-readable storage medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0180] In the embodiments of this application, the terms "module" or "unit" refer to computer instructions or a portion of computer instructions that have a predetermined function and work together with other related parts to achieve a predetermined goal. These instructions can be implemented, wholly or partially, using software, hardware (e.g., processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that functions as a whole.

[0181] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although this application has disclosed preferred embodiments as above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method of detecting consistency, characterized by, include: Obtain the message forwarding chain in the messaging system; According to the message forwarding link, obtain the message transmission record file of each node in the message forwarding link and the quantity statistics of each node for the transmitted message; the message transmission record file includes message transmission records generated for the transmitted message, and each message transmission record includes the message identifier of the transmitted message and the message content of the transmitted message. Based on the message transmission record files of multiple nodes in the message forwarding link, the message content consistency is verified to obtain the first verification result; Based on the statistical information of the number of multiple nodes in the message forwarding link, a message quantity consistency verification is performed to obtain a second verification result; Based on the first verification result and the second verification result, the message consistency verification result of the messaging system is determined.

2. The method of claim 1, wherein, The message transmission records generated by different nodes have the same format; The step of verifying message content consistency based on message transmission record files of multiple nodes in the message forwarding link to obtain a first verification result includes: Perform message transmission record deduplication processing on the message transmission record files of each node; The message transmission record files of each node after deduplication are hashed to obtain the hash value corresponding to each node. The hash values ​​of multiple nodes in the message forwarding link are compared to generate the first verification result.

3. The method of claim 2, wherein, The message transmission record also includes the topic to which each transmitted message belongs; The message transmission record deduplication process for the message transmission record files of each node includes: In the message transmission record files of each node, duplicate message transmission records with the same topic and the same message identifier are filtered out.

4. The method of claim 3, wherein, Before obtaining the message transmission record files of each node in the message forwarding link and the quantity statistics of each node for the transmitted messages according to the message forwarding link, the method further includes: Based on the consistency test information, obtain the target topic to which the message used for the consistency test belongs; Based on the target topic, topic subscription prompt information is sent to nodes after the first node in the message forwarding chain, so that each node after the first node subscribes to the target topic; Based on the target topic, a topic hint message is sent to the first node so that the first node sends messages belonging to the target topic during the test.

5. The method of claim 2, wherein, After comparing the hash values ​​of multiple nodes in the message forwarding link to generate the first verification result, the method further includes: If the first verification result indicates that the message content in the message system is inconsistent, an abnormal node prompt message is generated based on the abnormal node in the message forwarding link; wherein, the abnormal node refers to a node in the message forwarding link that is located after the first node and whose corresponding hash value is different from the hash value of the first node.

6. The method of claim 1, wherein, In the message forwarding link, adjacent nodes transmit messages through a message queue; the message forwarding link includes a producer node as a producer, a consumer node as a consumer, and an intermediate node as a transmission intermediary between the producer node and the consumer node. The quantity statistics of the producer nodes are the quantity statistics of the number of transmissions; the quantity statistics of the intermediate nodes include the quantity statistics of the number of receptions and the quantity statistics of the number of transmissions. The quantity statistics of the consumer nodes are the quantity statistics of the received data; The step of verifying message quantity consistency based on the quantity statistics of multiple nodes in the message forwarding link to obtain a second verification result includes: For any two adjacent nodes in the message forwarding link, the sending count statistics of the preceding node are compared with the receiving count statistics of the following node to determine the quantity verification result corresponding to the two adjacent nodes; the quantity verification result is used to indicate whether the number of messages successfully sent by the preceding node and the number of messages successfully received by the following node are the same. Local quantity verification is performed based on the received quantity statistics and sent quantity statistics messages of the same intermediate node to determine the local quantity verification result of each intermediate node. The local quantity verification result is used to indicate whether the number of messages successfully received and the number of messages successfully sent by the intermediate node are the same. The second verification result is determined based on the quantity verification results of the two adjacent nodes and the local quantity verification results of each intermediate node.

7. The method of claim 6, wherein, After determining the second verification result based on the quantity verification results corresponding to the two adjacent nodes and the local quantity verification results of each intermediate node, the method further includes: If the second verification result indicates that the number of messages in the messaging system is inconsistent, the abnormal number node is determined according to the second verification result; the abnormal number node refers to at least one of the following: two adjacent nodes whose numbers are different according to the number verification result, and an intermediate node whose number is different from the number indicated by the local number verification result. Based on the number of abnormal nodes, a message indicating an abnormal number of messages is generated.

8. The method according to any one of claims 1 to 7, characterized in that, Before obtaining the message transmission record files of each node in the message forwarding link and the quantity statistics of each node for the transmitted messages according to the message forwarding link, the method further includes: A file clearing command is sent to each node in the message forwarding link so that each node clears its own message transmission record file.

9. A consistency detection device characterized by comprising: include: The first acquisition module is used to acquire the message forwarding link in the messaging system; The second acquisition module is used to acquire, according to the message forwarding link, the message transmission record file of each node in the message forwarding link and the quantity statistics of each node for the transmitted message; the message transmission record file includes message transmission records generated for the transmitted message, and each message transmission record includes the message identifier of the transmitted message and the message content of the transmitted message; The first verification module is used to perform message content consistency verification based on message transmission record files of multiple nodes in the message forwarding link, and obtain a first verification result. The second verification module is used to perform message quantity consistency verification based on the quantity statistics of multiple nodes in the message forwarding link, and obtain a second verification result. The output module is used to determine the message consistency verification result of the message system based on the first verification result and the second verification result.

10. An electronic device, comprising: include: processor; A memory storing computer instructions that, when executed by the processor, implement the method as described in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-8.

12. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the method described in any one of claims 1-8.

Images