Message trajectory data analysis method, first server and storage medium

Abstract

The present application provides a method for analyzing message trajectory data, a first server, and a storage medium. The method includes: determining multiple common message topics and dividing a data file into multiple first areas; receiving message trajectory data corresponding to the common message topics through each first area; and obtaining multiple first target quantities, wherein one first target quantity corresponds to one common message topic, one first target quantity corresponds to one producer, one first target quantity corresponds to one consumer, and one first target quantity is the number of message trajectory data for messages belonging to the same producer, the same consumer, and the same common message topic. This method solves the problem in the prior art of being unable to determine the amount of message trajectory data under a specific topic between a pair of producers and consumers.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a message trajectory data analysis method, a first server, a computer-readable storage medium, a processor, and a message trajectory data analysis system. Background Art

[0002] In the prior art, RocketMQ's Broker server is used to complete message trace data retrieval and query.

[0003] The problems with the message track data retrieval and query methods in the prior art are as follows:

[0004] The broker server only supports configuring one message trace topic. All message trace data is sent to the message trace topic. All message trace data is mixed together, and the number of message trace data under a specific topic between a producer and consumer pair cannot be determined.

[0005] In the prior art, querying message trace data on a Broker server will occupy machine resources, and frequent use of the query function will affect the normal use of the Broker server. Summary of the Invention

[0006] The main purpose of this application is to provide a message trajectory data analysis method, a first server, a computer-readable storage medium, a processor, and a message trajectory data analysis system, so as to at least solve the problem in the prior art of being unable to determine the amount of message trajectory data under a specific topic between a pair of producers and consumers.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, a method for analyzing message trajectory data is provided, wherein a first server is communicatively connected to a second server, the second server is a Broker server, and the Broker server is configured with a data file. The method is applied to the first server, and the method includes: determining multiple common message topics and dividing the data file into multiple first areas, the common message topics corresponding to the first areas one-to-one, and the common message topics corresponding to any two of the first areas are different, and the first area is used to receive message trajectory data of messages belonging to the corresponding common message topics, and the message trajectory data includes at least one of the following: the name of the node where the message arrives during the transmission process, and the time of arrival at the node, the nodes including the producer, the second server and the consumer; receiving the message trajectory data of the corresponding common message topic through each first area; obtaining multiple first target quantities, wherein the first target quantity corresponds to one common message topic, the first target quantity corresponds to one producer, the first target quantity corresponds to one consumer, and the first target quantity is the number of message trajectory data of the messages belonging to the same producer, the same consumer and the same common message topic.

[0008] Optionally, the first server is communicatively connected to the third server to obtain multiple first target quantities, including: sending the message trajectory data of each of the common message topics to the third server; and obtaining multiple first target quantities through the third server.

[0009] Optionally, obtaining multiple first target quantities through the third server includes: configuring a first tag for each message track data through the third server, the first tag corresponding to the message track data one-to-one, and the first tag representing the common message topic to which the message track data belongs; configuring a second tag and a third tag for each message track data through the third server, the second tag corresponding to the message track data one-to-one, the second tag representing the name of the producer of the message to which the message track data belongs, and the third tag corresponding to the message track data one-to-one, and the third tag representing the name of the consumer of the message to which the message track data belongs; dividing the message track data into multiple groups according to the first tags, second tags and third tags of all the message track data in a group of the message track data, the first tags of all the message track data being the same, the second tags of all the message track data being the same, and the third tags of all the message track data being the same; determining multiple first target quantities, one first target quantity corresponding to a group of the message track data, and the first target quantity being the quantity of all the message track data in a group of the message track data.

[0010] Optionally, the third server is configured with a database, and the third server is communicatively connected to the front end. After obtaining multiple first target quantities through the third server, the method further includes: establishing a first mapping relationship through the third server, the first mapping relationship being a mapping relationship between the name of the producer, the name of the consumer, the ordinary message topic and the first target quantity; storing the first mapping relationship in the database through the third server; when the third server receives the name of the target producer, the name of the target consumer and the target ordinary message topic of the front end, determining the second target quantity according to the name of the target producer, the name of the target consumer, the target ordinary message topic and the first mapping relationship, and sending the second target quantity to the front end, the second target quantity being the quantity of the message trajectory data of the messages belonging to the target producer, the target consumer and the target ordinary message topic.

[0011] Optionally, the second server is configured with a log file, which includes multiple second areas. Before sending the message track data of each of the ordinary message topics to the third server, the method also includes: caching the message track data of each of the ordinary message topics to the corresponding second area of ​​the log file, the second area corresponding to the ordinary message topic one by one; and filtering out the message track data of the corresponding ordinary message topic from each second area.

[0012] Optionally, each message has an ID, the third server is configured with a database, and the third server is communicatively connected to the front end. After sending the message track data of each common message topic to the third server, the method further includes: obtaining the ID of the message to which each message track data belongs through the third server; dividing all the message track data into multiple groups according to the ID of the message to which each message track data belongs, one group of message track data corresponds to one message, and one group of message track data includes all the message track data of one message; determining the status of the corresponding message according to each group of message track data through the third server, the status of the message including at least one of the following: successful message production, successful message pulling, and successful message consumption; establishing a second mapping relationship, the second mapping relationship being a mapping relationship between the ID of the message and the status of the message; storing the second mapping relationship in the database through the third server; when the third server receives the ID of the target message of the front end, determining the status of the target message according to the ID of the target message and the second mapping relationship, and sending the status of the target message to the front end.

[0013] According to another aspect of the present application, a first server is provided, which is communicatively connected to a second server, the second server is a Broker server, and the Broker server is configured with a data file. The first server includes: a first processing unit, used to determine multiple common message topics and divide the data file into multiple first areas, the common message topics correspond to the first areas one-to-one, and the common message topics corresponding to any two of the first areas are different, one first area is used to receive message track data of messages belonging to the corresponding common message topics, and the message track data includes at least one of the following: the name of the node where the message arrives during the transmission process, the time of arrival at the node, and the nodes include producers, the second server and consumers; a receiving unit, used to receive the message track data of the corresponding common message topics through each of the first areas; a first acquisition unit, used to acquire multiple first target quantities, one first target quantity corresponding to one common message topic, one first target quantity corresponding to one producer, one first target quantity corresponding to one consumer, and one first target quantity being the number of message track data of the messages belonging to the same producer, the same consumer and the same common message topic.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the message trajectory data analysis methods.

[0015] According to another aspect of the present application, a processor is provided, which is used to run a program, wherein the program executes any one of the message trace data analysis methods when running.

[0016] According to one aspect of the present application, a message trajectory data analysis system is provided, comprising: a front end; a second server; a third server; and a first server, wherein the first server is communicatively connected to the front end, the second server, and the third server, respectively, and the first server is configured to execute any one of the message trajectory data analysis methods.

[0017] Applying the technical solution of the present application, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then, the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0019] Figure 1 A schematic diagram showing a flow chart of a method for analyzing message trajectory data provided according to an embodiment of the present application is shown;

[0020] Figure 2 The following is a structural block diagram of a message trajectory data analysis system provided according to an embodiment of the present application;

[0021] Figure 3 A structural block diagram of a first server provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] For ease of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0026] RocketMQ: It mainly consists of three parts: producer, broker server, and consumer. The producer is responsible for producing messages, the consumer is responsible for consuming messages, and the broker server is responsible for storing messages. The broker server can store messages from multiple common message topics.

[0027] Filebeat: A lightweight delivery tool for forwarding and centralizing log data, monitoring specified log files or locations, and collecting log events.

[0028] ELK: The Elastic Stack is a set of product components developed by Elastic that enables secure and reliable access to data from any source and in any format, and then searches, analyzes, and visualizes massive amounts of data in real time. The Elastic Stack includes Elasticsearch, Logstash, and Kibana. Logstash is a tool for collecting, analyzing, and filtering logs. Elasticsearch is an open source distributed search engine that provides data collection, analysis, and storage. Kibana is an open source analytics and visualization platform and a front-end component designed to work with Elasticsearch. Its browser-based interface enables the rapid creation and sharing of dynamic dashboards that display real-time changes to Elasticsearch query results.

[0029] As introduced in the background technology, the existing technology is unable to determine the amount of message trace data under a specific topic between a pair of producers and consumers. To solve the problem that the existing technology is unable to determine the amount of message trace data under a specific topic between a pair of producers and consumers, an embodiment of the present application provides a message trace data analysis method, a first server, a computer-readable storage medium, a processor, and a message trace data analysis system.

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] The first server is communicatively connected to the second server, where the second server is a Broker server in RocketMQ. The Broker server is configured with a data file, and the first server is a consumer in RocketMQ, receiving message trace data from the Broker server.

[0032] In this embodiment, a method for analyzing message trajectory data running on a first server is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 Flowchart of the method for analyzing message trajectory data according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0034] Step S201: determine a plurality of common message topics, and divide the data file into a plurality of first areas.

[0035] The common message topics correspond to the first regions one-to-one, and any two of the first regions correspond to different common message topics. The first region is used to receive message track data of messages belonging to the corresponding common message topics. The message track data includes at least one of the following: the name of the node to which the message arrives during the transmission process, and the time of arrival at the node. The nodes include the producer, the second server, and the consumer.

[0036] Step S202: receiving the message track data of the corresponding common message topic through each of the first areas;

[0037] Specifically, in the prior art, the Broker server only supports the configuration of one message track topic, and all message track data are sent to the message track topic, and all message track data are mixed together. The prior art cannot determine the amount of message track data under a specific topic between a pair of producers and consumers. This application uses the support for configuring multiple ordinary message topics in the Broker server. This application sets up multiple ordinary message topics, and each ordinary message topic receives the message track data of messages belonging to the ordinary message topic, that is, the message track data of messages of each topic are separated.

[0038] Step S203, obtain multiple first target quantities, one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0039] Specifically, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0040] The first server is in communication with a third server, and the third server is used to analyze the message trajectory data. Step S203 can be implemented as follows:

[0041] Step S2031, sending the message track data of each of the common message topics to the third server;

[0042] Step S2032: Acquire multiple first target quantities through the third server.

[0043] In this embodiment, in the prior art, message trace data is queried on the Broker server, which will occupy machine resources. Frequent use of the query function will affect the normal use of the Broker server. This application sends the message trace data to a third server, determines the amount of message trace data under a specific topic between a pair of producers and consumers through the third server, and queries the message trace data on the third server, thereby solving the problem that frequent use of the query function will affect the normal use of the Broker server.

[0044] The second server is configured with a log file, and the log file includes a plurality of second areas. Before step S2031, the method further includes:

[0045] caching the message trace data of each of the common message topics into the corresponding second area of ​​the log file, wherein the second area corresponds to the common message topic one by one;

[0046] The message track data of the corresponding common message topic is filtered out from each of the second areas.

[0047] In this embodiment, in some embodiments, Figure 2 As shown, Message-Consumer and Filebeat are configured on the first server. The Message-Consumer application obtains the message track data of each common message topic from the second server, and writes the message track data of each common message topic into the log file (.log file). Filebeat filters out the message track data of each common message topic from the log file. Message-Consumer is a service application for consuming messages.

[0048] The above step S2032 can be implemented as follows:

[0049] The third server configures a first tag for each message track data, wherein the first tag corresponds to the message track data one by one, and one first tag represents the common message topic to which the message track data belongs;

[0050] The third server configures a second tag and a third tag for each message track data, wherein the second tag corresponds to the message track data one by one, and each second tag represents the name of the producer of the message to which the message track data belongs; and the third tag corresponds to the message track data one by one, and each third tag represents the name of the consumer of the message to which the message track data belongs;

[0051] The third server divides the message track data into multiple groups according to the first tag, the second tag, and the third tag of all the message track data, wherein all the message track data in one group have the same first tag, the same second tag, and the same third tag;

[0052] A plurality of the first target quantities are determined, wherein one first target quantity corresponds to a group of the message track data, and one first target quantity is the quantity of all the message track data in a group of the message track data.

[0053] In this embodiment, in some embodiments, Figure 2As shown, Logstash and Elasticsearch are configured on the third server. Filebeat filters out the message trace data of each common message topic from the log file, and pushes the message trace data of the common message topic to Logstash of the third server. Logstash configures the first label, the second label and the third label for each message trace data. Then, Logstash sends the message trace data with the first label, the second label and the third label to Elasticsearch. Elasticsearch filters the message trace data of the same producer, the same consumer and the same common message topic into the same group based on the first label, the second label and the third label, and determines the number of message trace data in each group, so as to determine the number of message trace data under a specific topic between a pair of producers and consumers.

[0054] The third server is configured with a database, and the third server is in communication with the front end. After the step S2032, the method further includes:

[0055] Establishing a first mapping relationship through the third server, wherein the first mapping relationship is a mapping relationship between the name of the producer, the name of the consumer, the common message subject, and the first target quantity;

[0056] Storing the first mapping relationship in the database through the third server;

[0057] When the third server receives the name of the target producer, the name of the target consumer and the target general message topic of the front end, it determines the second target quantity based on the name of the target producer, the name of the target consumer, the target general message topic and the first mapping relationship, and sends the second target quantity to the front end. The second target quantity is the quantity of the message track data of the messages belonging to the target producer, the target consumer and the target general message topic.

[0058] In this embodiment, the target producer is one of multiple producers, the target consumer is one of multiple consumers, and the target general message topic is one of multiple general message topics. The mapping relationship between the producer's name, the consumer's name, the general message topic, and the first target quantity is stored in the database, that is, the number of message track data under a specific topic between the producer and the consumer is stored in the database, such as Figure 2As shown, in some embodiments, Elasticsearch is configured on the third server, and Kibana is configured on the front end. Elasticsearch receives the name of a producer, the name of a consumer, and a common message topic from Kibana, and Elasticsearch queries the number of message track data under the common message topic between the producer and the consumer from the database and sends it to Kibana. Kibana displays the number of message track data under the common message topic between the producer and the consumer.

[0059] Each message has an ID. The third server is configured with a database. The third server is in communication with the front end. After step S2032, the method further includes:

[0060] Obtaining, through the third server, the ID of the message to which each message trajectory data belongs;

[0061] dividing all the message trajectory data into a plurality of groups according to the ID of the message to which each message trajectory data belongs, wherein one group of the message trajectory data corresponds to one message, and one group of the message trajectory data includes all the message trajectory data of one message;

[0062] Determining, by the third server, the status of the corresponding message according to each set of the message trajectory data, the status of the message including at least one of the following: message production success, message pull success, and message consumption success;

[0063] Establishing a second mapping relationship, wherein the second mapping relationship is a mapping relationship between the ID of the message and the status of the message;

[0064] Storing the second mapping relationship in the second database through the third server;

[0065] When the third server receives the ID of the target message from the front end, it determines the state of the target message according to the ID of the target message and the second mapping relationship, and sends the state of the target message to the front end.

[0066] In this embodiment, the target ID is one of the IDs of multiple messages, and the state of the target message is one of the states of multiple messages, such as Figure 2As shown, in some embodiments, Elasticsearch is configured on the third server and Kibana is configured on the front end. Elasticsearch filters the message track data of the same ID into the same group. A message has an ID, that is, all the message track data of the same message are filtered into the same group. The status of the message is determined based on all the message track data of the message, and the mapping relationship between the ID of the message and the status of the message is stored in the database. When Elasticsearch receives a certain ID from Kibana, Elasticsearch queries the status of the message with the ID from the database, and sends the status of the message with the ID to Kibana. Kibana displays the status of the message with the ID.

[0067] Through the above embodiments, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then, the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0068] The embodiment of the present application also provides a first server. It should be noted that the first server of the embodiment of the present application can be used to execute the analysis method for message trajectory data provided by the embodiment of the present application. The first server is used to implement the above-mentioned embodiments and preferred implementation modes, and the details that have been explained will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware that implements a predetermined function. Although the first server described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0069] The following introduces the first server provided in the embodiment of the present application.

[0070] Figure 3 Schematic diagram of the first server according to an embodiment of the present application. Figure 3 As shown, the first server includes:

[0071] The first processing unit 10 is configured to determine a plurality of common message topics and divide the data file into a plurality of first regions, wherein the common message topics correspond to the first regions one-to-one, and any two of the first regions correspond to different common message topics. The first region is configured to receive message trajectory data of messages belonging to the corresponding common message topics, wherein the message trajectory data includes at least one of the following: the name of the node to which the message arrives during transmission and the time of arrival at the node, wherein the nodes include the producer, the second server, and the consumer;

[0072] A receiving unit 20, configured to receive the message trajectory data of the corresponding common message topic through each of the first areas;

[0073] Specifically, in the prior art, the Broker server only supports the configuration of one message track topic, and all message track data are sent to the message track topic, and all message track data are mixed together. The prior art cannot determine the amount of message track data under a specific topic between a pair of producers and consumers. This application uses the support for configuring multiple ordinary message topics in the Broker server. This application sets up multiple ordinary message topics, and each ordinary message topic receives the message track data of messages belonging to the ordinary message topic, that is, the message track data of messages of each topic are separated.

[0074] The first acquisition unit 30 is used to obtain multiple first target quantities, where one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0075] Specifically, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0076] The first server is communicatively connected to a third server, the third server is used to analyze message trajectory data, and the acquisition unit includes a sending module and an acquisition module.

[0077] The sending module is used to send the message trajectory data of each of the common message topics to the third server;

[0078] The acquisition module is configured to acquire a plurality of the first target quantities through the third server.

[0079] In this embodiment, in the prior art, message trace data is queried on the Broker server, which will occupy machine resources. Frequent use of the query function will affect the normal use of the Broker server. This application sends the message trace data to a third server, determines the amount of message trace data under a specific topic between a pair of producers and consumers through the third server, and queries the message trace data on the third server, thereby solving the problem that frequent use of the query function will affect the normal use of the Broker server.

[0080] The second server is configured with a log file, the log file includes a plurality of second areas, and the device includes a first storage unit and a screening unit.

[0081] The first storage unit is configured to cache the message trace data of each common message topic into the corresponding second area of ​​the log file, wherein the second area corresponds to each common message topic on a one-to-one basis;

[0082] The screening unit is configured to screen out the message track data corresponding to the common message topic from each of the second areas.

[0083] In this embodiment, in some embodiments, Figure 2 As shown, Message-Consumer and Filebeat are configured on the first server. The Message-Consumer application obtains the message track data of each common message topic from the second server, and writes the message track data of each common message topic into the log file (.log file). Filebeat filters out the message track data of each common message topic from the log file. Message-Consumer is a service application for consuming messages.

[0084] The acquisition module includes a first configuration submodule, a second configuration submodule, a division submodule and a determination submodule.

[0085] The first configuration submodule is configured to configure a first tag for each message track data through the third server, wherein the first tag corresponds to each message track data one by one, and each first tag represents the common message topic to which the message track data belongs;

[0086] The second configuration submodule is configured to configure a second tag and a third tag for each message track data through the third server, wherein the second tag corresponds to the message track data one by one, and each second tag represents the name of the producer of the message to which the message track data belongs; and the third tag corresponds to the message track data one by one, and each third tag represents the name of the consumer of the message to which the message track data belongs;

[0087] The division submodule is configured to divide the message track data into multiple groups according to the first tag, the second tag, and the third tag of all the message track data through the third server, wherein all the message track data in a group have the same first tag, the same second tag, and the same third tag;

[0088] The determination submodule is configured to determine a plurality of the first target quantities, wherein one first target quantity corresponds to a group of the message track data, and one first target quantity is the quantity of all the message track data in a group of the message track data.

[0089] In this embodiment, in some embodiments, Figure 2 As shown, Logstash and Elasticsearch are configured on the third server. Filebeat filters out the message trace data of each common message topic from the log file, and pushes the message trace data of the common message topic to Logstash of the third server. Logstash configures the first label, the second label and the third label for each message trace data. Then, Logstash sends the message trace data with the first label, the second label and the third label to Elasticsearch. Elasticsearch filters the message trace data of the same producer, the same consumer and the same common message topic into the same group based on the first label, the second label and the third label, and determines the number of message trace data in each group, so as to determine the number of message trace data under a specific topic between a pair of producers and consumers.

[0090] The third server is configured with a database, and the third server is in communication connection with the front end. The device includes a first establishing unit, a second storage unit and a second processing unit.

[0091] The first establishing unit is configured to establish a first mapping relationship through the third server, wherein the first mapping relationship is a mapping relationship between the name of the producer, the name of the consumer, the common message subject, and the first target quantity;

[0092] The second storage unit is configured to store the first mapping relationship in the database via the third server;

[0093] The above-mentioned second processing unit is used to determine the second target quantity based on the name of the above-mentioned target producer, the name of the above-mentioned target consumer, the above-mentioned target ordinary message topic and the above-mentioned first mapping relationship when the above-mentioned third server receives the name of the target producer, the name of the above-mentioned target consumer, the above-mentioned target ordinary message topic from the above-mentioned front-end, and send the second target quantity to the above-mentioned front-end. The above-mentioned second target quantity is the number of the above-mentioned message trajectory data of the messages belonging to the above-mentioned target producer, the above-mentioned target consumer and the above-mentioned target ordinary message topic.

[0094] In this embodiment, the target producer is one of multiple producers, the target consumer is one of multiple consumers, and the target general message topic is one of multiple general message topics. The mapping relationship between the producer's name, the consumer's name, the general message topic, and the first target quantity is stored in the database, that is, the number of message track data under a specific topic between the producer and the consumer is stored in the database, such as Figure 2 As shown, in some embodiments, Elasticsearch is configured on the third server, and Kibana is configured on the front end. Elasticsearch receives the name of a producer, the name of a consumer, and a common message topic from Kibana, and Elasticsearch queries the number of message track data under the common message topic between the producer and the consumer from the database and sends it to Kibana. Kibana displays the number of message track data under the common message topic between the producer and the consumer.

[0095] A message has an ID, the third server is configured with a database, the third server is in communication connection with the front end, and the device includes a second acquisition unit, a division unit, a determination unit, a second establishment unit, a third storage unit and a third processing unit.

[0096] The second obtaining unit is configured to obtain, through the third server, the ID of the message to which each message trajectory data belongs;

[0097] The division unit is configured to divide all the message trajectory data into a plurality of groups according to the ID of the message to which each message trajectory data belongs, through the third server, wherein a group of the message trajectory data corresponds to one message, and a group of the message trajectory data includes all the message trajectory data of one message;

[0098] The determining unit is configured to determine, through the third server, a status of the corresponding message based on each set of the message trajectory data, where the status of the message includes at least one of the following: message production success, message pull success, and message consumption success;

[0099] The second establishing unit is configured to establish a second mapping relationship, wherein the second mapping relationship is a mapping relationship between the ID of the message and the status of the message;

[0100] The third storage unit is configured to store the second mapping relationship in the second database via the third server;

[0101] The third processing unit is used to determine the status of the target message according to the ID of the target message and the second mapping relationship when the third server receives the ID of the target message from the front end, and send the status of the target message to the front end.

[0102] In this embodiment, the target ID is one of the IDs of multiple messages, and the state of the target message is one of the states of multiple messages, such as Figure 2 As shown, in some embodiments, Elasticsearch is configured on the third server and Kibana is configured on the front end. Elasticsearch filters the message track data of the same ID into the same group. A message has an ID, that is, all the message track data of the same message are filtered into the same group. The status of the message is determined based on all the message track data of the message, and the mapping relationship between the ID of the message and the status of the message is stored in the database. When Elasticsearch receives a certain ID from Kibana, Elasticsearch queries the status of the message with the ID from the database, and sends the status of the message with the ID to Kibana. Kibana displays the status of the message with the ID.

[0103] Through the above embodiments, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then, the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0104] The first server includes a processor and a memory. The first processing unit, receiving unit, and first acquisition unit are all stored in the memory as program units. The processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; alternatively, the modules can be located in different processors in any combination.

[0105] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be configured, and kernel parameters can be adjusted to address the existing problem of being unable to determine the amount of message trace data for a specific topic between a producer and consumer pair.

[0106] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0107] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is executed, the device where the computer-readable storage medium is located is controlled to execute the message trace data analysis method.

[0108] Specifically, the analysis method of message trajectory data includes:

[0109] The analysis method of the above message trajectory data includes:

[0110] Step S201: Determine multiple common message topics and divide the data file into multiple first regions. The common message topics correspond to the first regions one by one. Any two of the first regions correspond to different common message topics. The first region is used to receive message trajectory data of messages belonging to the corresponding common message topics. The message trajectory data includes at least one of the following: the name of the node to which the message arrives during the transmission process and the time of arrival at the node. The nodes include the producer, the second server, and the consumer.

[0111] Step S202: receiving the message track data of the corresponding common message topic through each of the first areas;

[0112] Step S203, obtain multiple first target quantities, one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0113] An embodiment of the present invention provides a processor, which is used to run a program, wherein the message trace data analysis method is executed when the program is running.

[0114] Specifically, the analysis method of message trajectory data includes:

[0115] The analysis method of the above message trajectory data includes:

[0116] Step S201: Determine multiple common message topics and divide the data file into multiple first regions. The common message topics correspond to the first regions one by one. Any two of the first regions correspond to different common message topics. The first region is used to receive message trajectory data of messages belonging to the corresponding common message topics. The message trajectory data includes at least one of the following: the name of the node to which the message arrives during the transmission process and the time of arrival at the node. The nodes include the producer, the second server, and the consumer.

[0117] Step S202: receiving the message track data of the corresponding common message topic through each of the first areas;

[0118] Step S203, obtain multiple first target quantities, one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0119] An embodiment of the present invention provides a message trajectory data analysis system, comprising: a front end; a second server; a third server; and a first server, wherein the first server is respectively communicatively connected to the front end, the second server, and the third server, and the first server is configured to execute the message trajectory data analysis method.

[0120] The analysis method of the above message trajectory data includes:

[0121] Step S201: Determine multiple common message topics and divide the data file into multiple first regions. The common message topics correspond to the first regions one by one. Any two of the first regions correspond to different common message topics. The first region is used to receive message trajectory data of messages belonging to the corresponding common message topics. The message trajectory data includes at least one of the following: the name of the node to which the message arrives during the transmission process and the time of arrival at the node. The nodes include the producer, the second server, and the consumer.

[0122] Step S202: receiving the message track data of the corresponding common message topic through each of the first areas;

[0123] Step S203, obtain multiple first target quantities, one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0124] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0125] The analysis method of the above message trajectory data includes:

[0126] Step S201: Determine multiple common message topics and divide the data file into multiple first regions. The common message topics correspond to the first regions one by one. Any two of the first regions correspond to different common message topics. The first region is used to receive message trajectory data of messages belonging to the corresponding common message topics. The message trajectory data includes at least one of the following: the name of the node to which the message arrives during the transmission process and the time of arrival at the node. The nodes include the producer, the second server, and the consumer.

[0127] Step S202: receiving the message track data of the corresponding common message topic through each of the first areas;

[0128] Step S203, obtain multiple first target quantities, one of the above-mentioned first target quantities corresponds to one of the above-mentioned common message topics, one of the above-mentioned first target quantities corresponds to one of the above-mentioned producers, one of the above-mentioned first target quantities corresponds to one of the above-mentioned consumers, and one of the above-mentioned first target quantities is the number of the above-mentioned message trajectory data of the above-mentioned messages belonging to the same above-mentioned producer, the same above-mentioned consumer and the same above-mentioned common message topic.

[0129] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0130] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0135] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0136] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0138] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0139] 1) In the message track data analysis method of the present application, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then, the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0140] 2) In the first server of the present application, the present application utilizes the support for configuring multiple common message topics in the Broker server. The present application sets up multiple common message topics, and each common message topic receives the message track data of the messages belonging to the common message topic, that is, the message track data of the messages of each topic are distinguished, and then, the number of message track data under a specific topic between a pair of producers and consumers is determined, thereby solving the problem in the prior art that the number of message track data under a specific topic between a pair of producers and consumers cannot be determined.

[0141] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for analyzing message trajectory data, characterized in that: A first server is communicatively connected to a second server, the second server being a Broker server, the Broker server being configured with a data file, and the method is applied to the first server, the method comprising: Determine a plurality of common message topics, and divide the data file into a plurality of first areas, wherein the common message topics correspond to the first areas on a one-to-one basis, and any two of the first areas correspond to different common message topics, and each of the first areas is used to receive message track data of messages belonging to the corresponding common message topics, wherein the message track data includes at least one of the following: a name of a node that the message arrives at during transmission and a time of arrival at the node, wherein the nodes include a producer, the second server, and a consumer; receiving the message track data of the corresponding common message topic through each of the first areas; Acquire multiple first target quantities, where one first target quantity corresponds to one common message topic, one first target quantity corresponds to one producer, one first target quantity corresponds to one consumer, and one first target quantity is the quantity of the message track data of the messages belonging to the same producer, the same consumer, and the same common message topic; The first server is in communication with the third server to obtain a plurality of first target quantities, including: Sending the message track data of each of the common message topics to the third server; The third server configures a first tag for each message track data, wherein the first tag corresponds to the message track data one by one, and each first tag represents the common message topic to which the message track data belongs; The third server configures a second tag and a third tag for each message track data, wherein the second tag corresponds to the message track data one by one, and the second tag represents the name of the producer of the message to which the message track data belongs; and the third tag corresponds to the message track data one by one, and the third tag represents the name of the consumer of the message to which the message track data belongs; Dividing the message track data into multiple groups by the third server according to the first tag, the second tag, and the third tag of all the message track data, wherein all the message track data in one group have the same first tag, the same second tag, and the same third tag; A plurality of first target quantities are determined, where one first target quantity corresponds to a group of the message track data, and one first target quantity is the quantity of all the message track data in a group of the message track data.

2. The method according to claim 1, characterized in that The third server is configured with a database and is in communication with the front end. After obtaining the plurality of first target quantities through the third server, the method further includes: Establishing a first mapping relationship by the third server, where the first mapping relationship is a mapping relationship between the name of the producer, the name of the consumer, the common message subject, and the first target quantity; storing the first mapping relationship in the database through the third server; When the third server receives the name of the target producer, the name of the target consumer and the target ordinary message topic of the front end, it determines the second target quantity based on the name of the target producer, the name of the target consumer, the target ordinary message topic and the first mapping relationship, and sends the second target quantity to the front end. The second target quantity is the quantity of the message track data of the messages belonging to the target producer, the target consumer and the target ordinary message topic.

3. The method according to claim 1, characterized in that The second server is configured with a log file, the log file including a plurality of second areas. Before sending the message trajectory data of each common message topic to the third server, the method further includes: caching the message trace data of each common message topic into the corresponding second area of ​​the log file, where the second area corresponds to the common message topic one by one; The message track data of the corresponding common message topic is filtered out from each second area.

4. The method according to claim 1, wherein Each message has an ID, the third server is configured with a database, and the third server is in communication with the front end. After sending the message trajectory data of each common message topic to the third server, the method further includes: Obtaining, through the third server, the ID of the message to which each message trajectory data belongs; dividing all the message trajectory data into multiple groups according to the ID of the message to which each message trajectory data belongs, wherein one group of message trajectory data corresponds to one message, and one group of message trajectory data includes all the message trajectory data of one message; Determining, by the third server, the status of the corresponding message according to each group of the message trajectory data, the status of the message including at least one of the following: message production success, message pulling success, and message consumption success; Establishing a second mapping relationship, where the second mapping relationship is a mapping relationship between the ID of the message and the status of the message; storing the second mapping relationship in the database through the third server; When the third server receives the ID of the target message from the front end, it determines the state of the target message according to the ID of the target message and the second mapping relationship, and sends the state of the target message to the front end.

5. A first server, characterized in that: The first server is communicatively connected to the second server, the second server being a Broker server, the Broker server being configured with a data file, the first server being configured to execute the message trace data analysis method according to any one of claims 1 to 4, the first server comprising: a first processing unit, configured to determine a plurality of common message topics and divide the data file into a plurality of first regions, wherein the common message topics correspond to the first regions on a one-to-one basis, and any two of the first regions correspond to different common message topics, wherein the first regions are configured to receive message track data of messages belonging to the corresponding common message topics, wherein the message track data includes at least one of the following: a name of a node at which the message arrives during a transmission process and a time of arrival at the node, wherein the nodes include a producer, the second server, and a consumer; a receiving unit, configured to receive the message track data of the corresponding common message topic through each of the first areas; The first acquisition unit is used to acquire multiple first target quantities, where one first target quantity corresponds to one common message topic, one first target quantity corresponds to one producer, one first target quantity corresponds to one consumer, and one first target quantity is the quantity of message trajectory data of the messages belonging to the same producer, the same consumer, and the same common message topic.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the message trajectory data analysis method according to any one of claims 1 to 4.

7. A processor, characterized in that: The processor is configured to run a program, wherein the program executes the message trace data analysis method according to any one of claims 1 to 4 when running.

8. A message trajectory data analysis system, characterized in that: The system comprises: front end; Second server; Third server; A first server, wherein the first server is respectively connected to the front end, the second server and the third server, and the first server is used to execute the message trajectory data analysis method according to any one of claims 1 to 4.

Images