Message middleware fault location method, device, equipment and medium
By creating a detector encapsulation class in the message middleware to automatically detect asynchronous link nodes, the cumbersome problem of message middleware failure analysis is solved, and the fault nodes and causes are quickly located, reducing labor costs and time.
Patent Information
- Application Number
- CN202111547612.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2041-12-16
AI Technical Summary
During the use of message middleware, fault analysis and location require manual logging into various platforms and viewing a large number of logs and configuration files, which is very cumbersome and time-consuming, resulting in high labor and time costs.
By creating and instantiating the detector packaging class, each node in the asynchronous link automatically detects, determines the fault nodes and the cause of the fault, and improves the efficiency of fault analysis.
It realizes rapid location and analysis of message middleware failures, reduces the time and cost of manual intervention, and improves the efficiency of fault analysis.
Smart Images
Figure CN114237956B_ABST
Abstract
Description
Background Art
[0002] Asynchrony is one of the commonly used methods for decoupling and performance optimization of background programs. Message middleware is a common method to achieve asynchrony. For example, open source message middleware such as Kafka, RabbitMQ, ActiveMQ, and RocketMQ are widely used.
[0003] Various faults may be encountered during the use of message middleware. Fault analysis and location of message middleware require manual logging into various platforms and checking a large number of logs and configuration files, which is very cumbersome and time-consuming, and consumes a lot of manpower and time costs. Summary of the invention
[0004] The purpose of this application is to provide a message middleware fault location method, device, equipment and computer-readable medium. By creating and instantiating a detector encapsulation class, each node in the asynchronous link is detected in turn, the faulty node and the fault cause corresponding to the faulty node are determined, and the fault analysis efficiency of the asynchronous link is improved.
[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0006] According to a first aspect of an embodiment of the present application, a message middleware fault locating method is provided, the method comprising:
[0007] Create a detector encapsulation class according to the preset detection method of each node in the asynchronous link;
[0008] Parse the preset configuration file to obtain the path of the physical configuration file of the message producer;
[0009] According to the path of the physical configuration file, read the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device;
[0010] According to the service name of the message producer and the IP information of the message production device, the log and message files of the message producer are obtained, and according to the log and message files of the message producer, the detector encapsulation class is instantiated;
[0011] The instantiated object of the detector encapsulation class is executed to detect each node in the asynchronous link in turn according to the preset detection method to determine the faulty node and the fault cause corresponding to the faulty node.
[0012] In some embodiments of the present application, based on the above solution, parsing the preset configuration file to obtain the path of the physical configuration file of the message producer includes:
[0013] Parse the preset configuration file to obtain the relative path and project root directory of the physical configuration file;
[0014] The relative path of the physical configuration file and the project root directory are concatenated to obtain the absolute path of the physical configuration file.
[0015] In some embodiments of the present application, based on the above solution, sequentially detecting each node in the asynchronous link to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0016] Triggering the message producer to produce asynchronous messages;
[0017] Obtaining a log sequence number corresponding to the asynchronous message in the message producer, wherein the log sequence numbers corresponding to the asynchronous message in each node of the asynchronous link are the same;
[0018] Based on the log sequence number corresponding to the asynchronous message, each node in the asynchronous link is detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0019] In some embodiments of the present application, based on the aforementioned solution, detecting each node in the asynchronous link in turn based on the log sequence number corresponding to the asynchronous message to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0020] Filtering the sending information of the asynchronous message in the log of the message producer according to the log sequence number corresponding to the asynchronous message, and determining whether the asynchronous message produced by the message producer meets the message sending condition according to the sending information of the asynchronous message;
[0021] If the asynchronous message does not meet the message sending condition, then the message producer is indeed a faulty node, and the fault cause is that the asynchronous message does not meet the message sending condition;
[0022] If the asynchronous message meets the message sending condition, the sending time and message subject of the asynchronous message are obtained according to the log of the message producer;
[0023] Determining whether the message file of the message producer contains the asynchronous message according to the sending time and the message subject of the asynchronous message;
[0024] If the message file of the message producer does not contain the asynchronous message, it is determined that the message producer is a faulty node, and the fault cause is the failure to write the asynchronous message;
[0025] If the message file of the message producer contains the asynchronous message, the subsequent nodes of the message producer are detected in turn to determine the faulty nodes and the fault causes corresponding to the faulty nodes.
[0026] In some embodiments of the present application, based on the aforementioned solution, sequentially detecting the subsequent nodes of the message producer to determine the faulty nodes and the fault causes corresponding to the faulty nodes includes:
[0027] Parse the configuration file of the message proxy node to obtain first configuration information of the message proxy node, wherein the first configuration information includes the instance name and address configuration information of the message server;
[0028] Querying the configuration center for first configuration information of the message server;
[0029] If the query is unsuccessful, it is determined that the message proxy node is a faulty node, and the cause of the fault is that the configuration information of the message server in the message proxy node is incorrect;
[0030] If the query is successful, determining whether the message proxy node correctly processes the asynchronous message based on the log of the message proxy node;
[0031] If the message proxy node does not correctly process the asynchronous message, it is determined that the message proxy node is a faulty node, and the fault cause is that the message proxy node does not correctly process the asynchronous message;
[0032] If the message proxy node processes the asynchronous message correctly, the subsequent nodes of the message proxy node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0033] In some embodiments of the present application, based on the aforementioned solution, sequentially detecting the subsequent nodes of the message proxy node to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0034] Determine, according to the log file of the message receiving node, whether the message receiving node writes the asynchronous message into the message file under the specified storage directory;
[0035] If the message receiving node fails to write the asynchronous message into the message file under the specified storage directory, it is determined that the message receiving node is a faulty node, and the fault reason is that the message receiving node fails to receive the asynchronous message or the message writing path of the message receiving node is wrong;
[0036] If the message receiving node writes the asynchronous message into a message file under a designated storage directory, the asynchronous message is searched for in the message file under the designated storage directory of the message server according to the service name of the message producer, the sending time of the asynchronous message and the subject information;
[0037] If the query is unsuccessful, it is determined that the message receiving node is a faulty node, and the fault cause is that the message receiving node fails to write a message.
[0038] If the query is successful, the subsequent nodes of the message receiving node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0039] In some embodiments of the present application, based on the foregoing solution, sequentially detecting subsequent nodes of the message receiving node to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0040] Parse the configuration file of the message forwarding node to obtain second configuration information of the message forwarding node, and determine whether the second configuration information is complete, wherein the second configuration information includes a subscription relationship with a message consumer and the message consumer address configuration information;
[0041] If the second configuration information is incomplete, determining that the message forwarding node is a faulty node, and the fault cause is that the configuration information of the message forwarding node is incorrect;
[0042] Querying the message consumer address configuration information in the configuration center;
[0043] If the query is unsuccessful, it is determined that the message forwarding node is a faulty node, and the cause of the fault is that the address source channel of the message forwarding node is wrong;
[0044] If the query is successful, determining whether the message forwarding node correctly processes the asynchronous message based on the log of the message forwarding node;
[0045] If the message forwarding node fails to correctly process the asynchronous message, it is determined that the message forwarding node is a faulty node, and the fault cause is that the message forwarding node fails to correctly process the asynchronous message;
[0046] If the message forwarding node correctly processes the asynchronous message, the subsequent nodes of the message forwarding node are detected in sequence to determine the faulty node and the fault cause corresponding to the faulty node.
[0047] According to a second aspect of an embodiment of the present application, a message middleware fault locating device is provided, the device comprising:
[0048] A detector encapsulation class, wherein the detector encapsulation class includes a detection method for each node in an asynchronous link;
[0049] Input and output devices are used to parse the preset configuration file and obtain the path of the physical configuration file of the message producer;
[0050] The input-output device is further used to read the physical configuration file according to the path of the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device;
[0051] The input and output device is also used to obtain the log and message files of the message producer according to the service name of the message producer and the IP information of the message production device, and instantiate the detector encapsulation class according to the log and message files of the message producer;
[0052] The use case executor is used to execute the instantiation object of the detector encapsulation class, detect each node in the asynchronous link in turn, and determine the faulty node and the fault cause corresponding to the faulty node.
[0053] According to one aspect of an embodiment of the present application, a computer-readable program medium is provided, which stores computer program instructions. When the computer program instructions are executed by a computer, the computer executes the above-mentioned financial data monitoring method.
[0054] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the computer device, the method described in the first aspect above is implemented.
[0055] According to a fourth aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by an electronic device, the method described in the first aspect above is implemented.
[0056] In an embodiment of the present application, by creating and instantiating a detector encapsulation class, each node in the asynchronous link can be automatically detected in turn through a preset detection method, the faulty node and the fault cause corresponding to the faulty node can be determined, and the fault analysis efficiency of the asynchronous link can be improved.
[0057] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The above and other features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
[0059] Figure 1 The figure shows the working principle of CQ type message middleware.
[0060] Figure 2 A schematic diagram showing the working principle of CMQ-type message middleware.
[0061] Figure 3 A flow chart of a message middleware fault locating method according to an embodiment of the present application is shown.
[0062] Figure 4 A flowchart of another message middleware fault locating method according to an embodiment of the present application is shown.
[0063] Figure 5 A schematic structural diagram of a message middleware fault locating device according to an embodiment of the present application is shown.
[0064] Figure 6 A schematic diagram of a program product for implementing the above method according to an embodiment of the present application is shown.
[0065] Figure 7 A schematic diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0066] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.
[0067] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.
[0068] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0069] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0070] It should also be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.
[0071] Figure 1 The following is a schematic diagram showing the working principle of CQ type message middleware. Figure 1 As shown, CQ uses a working process to complete the collection and delivery of messages. The workflow is as follows:
[0072] Step 110: The message producer calls the CQ API to write a message file.
[0073] Step 120: The file reading thread in the CQ work process reads messages from the directory specified in the configuration file, and writes messages that need to be delivered immediately directly to the shared memory, otherwise write Figure 1 The Mmap file shown in .
[0074] Step 130: The sending thread of the CQ work process is responsible for forwarding the message in the shared memory / calling the consumer for processing.
[0075] Step 140: The file reading thread of the CQ process periodically retrieves messages from the Mmap file to the shared memory, and the sending thread sends them.
[0076] CQ and the message producer are deployed on the same machine, which can reduce the time consumption of network calls. However, CQ is only applicable to message delivery within the same set and cannot support message delivery across sets, domains, computer rooms, or cities.
[0077] Figure 2 The following is a schematic diagram showing the working principle of CMQ type message middleware. Figure 2 As shown, CMQ has three working processes, namely proxy, processor and worker. The message transmission process within the same set is as follows:
[0078] Step 210: The producer calls the cmq API to write a message file.
[0079] Step 220: The message proxy node (cmq proxy) deployed on the producer device is responsible for reading messages from the directory specified in the configuration file and forwarding them to the message receiving node (cmq processor) of the message queue server (cmq server) of this set.
[0080] Step 230: The cmq processor is responsible for receiving the message and writing it to the message file in the specified storage directory.
[0081] Step 240: The message sending node (cmq worker) retrieves the message from the message file storage location and forwards / calls the consumer for processing.
[0082] It should be noted that when the consumer and producer are located in the domain and outside the domain respectively, the cmq worker needs to forward the message to the cmq processor of the cmq server outside the domain through the firewall protocol, and then the cmq worker outside the domain will deliver it. If the consumer and producer are not in the same set, the cmq worker needs to forward the message to the cmq processor of the cmq server in the set where the consumer is located, and then its cmq worker will deliver it. The transmission process of cross-computer room and cross-city messages is exactly the same as the cross-set message above, except that the message replication link is longer and more complicated.
[0083] Figure 3 A flow chart of a message middleware fault location method according to an embodiment of the present application is shown. Figure 3 As shown, the method includes at least the following steps.
[0084] Step 310: Create a detector encapsulation class according to the preset detection method of each node in the asynchronous link.
[0085] This application encapsulates the detection method of each node in the asynchronous link into a class so that a single or multiple asynchronous links can mobilize the encapsulation class to perform fault detection individually or in batches. The detection method of each node in the asynchronous link in the encapsulation class is applicable to simple message, cross-set message, cross-domain message, cross-computer room message and cross-city message transmission scenarios. The preset detection method of each node is specifically described below.
[0086] Step 320: Parse the preset configuration file to obtain the path of the physical configuration file of the message producer.
[0087] The preset configuration file includes the path of the physical configuration file of each node in the asynchronous link. In a specific implementation, the preset configuration file can be a file in Excel format. Input and output devices can be created, and the preset configuration file in Excel format can be read through the input and output devices.
[0088] Step 330: Read the physical configuration file according to the path of the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device.
[0089] Step 340: According to the service name of the message producer and the IP information of the message production device, the log and message files of the message producer are obtained, and according to the log and message files of the message producer, the detector encapsulation class is instantiated.
[0090] The message file of the message producer is stored in its specified directory. The log and message file of the message producer are obtained, and the relevant information of the message producer is used as a member parameter to instantiate the detector encapsulation class and obtain the detector instantiation object for subsequent execution of the detector instantiation object.
[0091] Step 350: Execute the instantiated object of the detector encapsulation class to detect each node in the asynchronous link in turn according to the preset detection method to determine the faulty node and the fault cause corresponding to the faulty node.
[0092] The present application can also write the faulty nodes and fault causes into a result display file in Excel format. One row in the result display file represents the detection result of the transmission process of an asynchronous message in each node in an asynchronous link. Different columns record the snapshots generated in each link of the detection process. The result display file simply and clearly displays the fault analysis results and the data involved in the intermediate process.
[0093] By creating and instantiating a detector encapsulation class, the embodiment of the present application can automatically detect each node in the asynchronous link in turn through a preset detection method, determine the faulty node and the fault cause corresponding to the faulty node, and improve the fault analysis efficiency of the asynchronous link.
[0094] In some solutions of the embodiments of the present application, based on the above solution, parsing the preset configuration file to obtain the path of the physical configuration file of the message producer includes:
[0095] Parse the preset configuration file to obtain the relative path and project root directory of the physical configuration file;
[0096] The relative path of the physical configuration file and the project root directory are concatenated to obtain the absolute path of the physical configuration file.
[0097] In a specific implementation, the OS mode in Python can be used, for example, the path concatenation function os.path.join() can be used to connect the relative path of the physical configuration file of the message producer and the project root directory to obtain the absolute path of the physical configuration file. The physical configuration file can be obtained through the absolute path of the physical configuration file.
[0098] In some solutions of the embodiments of the present application, based on the above solution, each node in the asynchronous link is detected in turn to determine the faulty node and the fault cause corresponding to the faulty node, including:
[0099] Triggering the message producer to produce asynchronous messages;
[0100] Obtaining a log sequence number corresponding to the asynchronous message in the message producer, wherein the log sequence numbers corresponding to the asynchronous message in each node of the asynchronous link are the same;
[0101] Based on the log sequence number corresponding to the asynchronous message, each node in the asynchronous link is detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0102] In the specific implementation, a test case can be created. When the test case is executed, the message producer is triggered to generate an asynchronous message for testing. The log sequence number can be MSGNO under the microservice framework. The log sequence number of the asynchronous message in each node in the asynchronous link is the same. By filtering the log information by MSGNO, the transmission process of the asynchronous message in the entire asynchronous link can be connected in series.
[0103] The following uses the CMQ type message middleware as an example to introduce the fault location method of each node in the asynchronous link.
[0104] In some solutions of the embodiments of the present application, based on the above solution, detecting each node in the asynchronous link in turn based on the log sequence number corresponding to the asynchronous message to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0105] Filtering the sending information of the asynchronous message in the log of the message producer according to the log sequence number corresponding to the asynchronous message, and determining whether the asynchronous message produced by the message producer meets the message sending condition according to the sending information of the asynchronous message;
[0106] If the asynchronous message does not meet the message sending condition, then the message producer is indeed a faulty node, and the fault cause is that the asynchronous message does not meet the message sending condition;
[0107] If the asynchronous message meets the message sending condition, the sending time and message subject of the asynchronous message are obtained according to the log of the message producer;
[0108] Determining whether the message file of the message producer contains the asynchronous message according to the sending time and the message subject of the asynchronous message;
[0109] If the message file of the message producer does not contain the asynchronous message, it is determined that the message producer is a faulty node, and the fault cause is the failure to write the asynchronous message;
[0110] If the message file of the message producer contains the asynchronous message, the subsequent nodes of the message producer are detected in turn to determine the faulty nodes and the fault causes corresponding to the faulty nodes.
[0111] Figure 4 A flowchart of another message middleware fault location method according to an embodiment of the present application is shown. Figure 4 In the detection points 1 and 2, the log at detection point 1 is used to detect whether the message producer has successfully sent the message, and at detection point 2, the message producer is detected to detect whether the asynchronous message has been successfully written into the message file, that is, whether the message has been successfully written.
[0112] The message producer publishes / sends a message on a specific message topic and calls the CQ or CMQ API of the message middleware to write a message file. The message middleware receives and forwards the message. Multiple active message consumers interested in this message topic can receive the message forwarded by the message middleware after subscribing. Then, starting from the source of the asynchronous link, detect whether the message producer has successfully sent and written the message.
[0113] It should be noted that the message sending conditions are set by the specific message sending business scenario. The message file directory and log directory of the message producer can be stored in a preset configuration file. The preset configuration file can be parsed to obtain the path of the log or message file, and then read and analyze the log or message file.
[0114] In some solutions of the embodiments of the present application, based on the above solution, sequentially detecting the subsequent nodes of the message producer to determine the faulty nodes and the fault causes corresponding to the faulty nodes includes:
[0115] Parse the configuration file of the message proxy node to obtain first configuration information of the message proxy node, wherein the first configuration information includes the instance name and address configuration information of the message server;
[0116] Querying the configuration center for first configuration information of the message server;
[0117] If the query is unsuccessful, it is determined that the message proxy node is a faulty node, and the cause of the fault is that the configuration information of the message server in the message proxy node is incorrect;
[0118] If the query is successful, determining whether the message proxy node correctly processes the asynchronous message based on the log of the message proxy node;
[0119] If the message proxy node does not correctly process the asynchronous message, it is determined that the message proxy node is a faulty node, and the fault cause is that the message proxy node does not correctly process the asynchronous message;
[0120] If the message proxy node processes the asynchronous message correctly, the subsequent nodes of the message proxy node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0121] The above detection method detects at detection point 2 whether the message producer successfully writes the asynchronous message into the message file, that is, whether the message is written successfully.
[0122] The message proxy node is deployed on the message production device, and the message receiving node and the message forwarding node are deployed on the message server. The configuration center can be a PaaS container cloud platform (such as Figure 1 cc configuration center in ). This method corresponds to the execution of Figure 4 Detection points 3, 4 and 5 determine whether the message broker node is a faulty node and the corresponding fault cause.
[0123] After the message producer writes the message, the message proxy node (cmq proxy) deployed on the producer device is responsible for reading the message from the directory specified in the configuration file (such as memory) and forwarding it to the cmq processor of the message server (cmqserver) of this set. The first configuration information is the configuration information necessary to establish a communication link between the message proxy node and the message server. Therefore, in the fault detection link, it is necessary to check whether the communication link between the cmq proxy and the cmq server (whether the configuration information matches) is correct. This method corresponds to execution at Figure 4 Detection points 3 and 4, that is, by matching the configuration file of the message proxy node at detection point 3 and the configuration information of the configuration center at detection point 4, determine that the configuration information of the message server in the message proxy node is wrong. If the configuration information is wrong, it means that the communication link between cmq proxy and cmq server is blocked, and the asynchronous message cannot be forwarded to cmq processor.
[0124] It should be noted that, in the process of determining whether the message proxy node correctly handles the asynchronous message according to the log of the message proxy node, the log sequence number of the asynchronous message can be used to match the log information related to the asynchronous message in the log of the message proxy node, and determine whether the message proxy node correctly handles the asynchronous message from the relevant log information. Figure 4 At detection point 5, that is, by matching the log of the message proxy node at detection point 5, it is determined whether the message proxy node correctly processes the asynchronous message.
[0125] Determining whether the message proxy node correctly processes the asynchronous message according to the log of the message proxy node specifically includes the following verification steps:
[0126] (1) Determine whether the file reading thread of cmq proxy opens the message file from the correct path. If so, proceed to the next verification step; otherwise, the detection ends and the cause of the failure can be determined to be incorrect configuration of the cmq proxy message file path or file permission;
[0127] (2) Determine whether the file reading thread of cmq proxy correctly parses the message file. If so, proceed to the next verification step; otherwise, the detection ends and it can be determined that the cause of the failure is that cmq proxy does not parse the message file correctly;
[0128] (3) Determine whether the sending thread of cmq proxy correctly reads the message from the shared memory. If so, proceed to the next verification step; otherwise, the detection ends and it can be determined that the cause of the failure is that the shared memory of cmq proxy is overwritten, resulting in the loss of message data;
[0129] (4) Determine whether the sending thread of cmq proxy successfully obtains the IP and port information of the target cmq server instance under the same set from the PaaS container cloud platform. If so, proceed to the next verification step; otherwise, the detection ends and the cause of the failure can be determined to be the timeout or failure of cmq proxy querying the PaaS container cloud platform;
[0130] (5) Determine whether the sending thread of the cmq proxy successfully calls the cmq processor of the target cmq server. If so, proceed to the next process; otherwise, the detection ends and it can be determined that the cause of the failure is that the cmq processor service of the cmq server is not started or is abnormal.
[0131] In some solutions of the embodiments of the present application, based on the above solution, sequentially detecting the subsequent nodes of the message proxy node to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0132] Determining, according to the log file of the message receiving node, whether the message receiving node writes the asynchronous message into a message file under a designated storage directory;
[0133] If the message receiving node fails to write the asynchronous message into the message file under the specified storage directory, it is determined that the message receiving node is a faulty node, and the fault reason is that the message receiving node fails to receive the asynchronous message or the message writing path of the message receiving node is wrong;
[0134] If the message receiving node writes the asynchronous message into a message file under a designated storage directory, the asynchronous message is searched for in the message file under the designated storage directory of the message server according to the service name of the message producer, the sending time of the asynchronous message and the subject information;
[0135] If the query is unsuccessful, it is determined that the message receiving node is a faulty node, and the fault cause is that the message receiving node fails to write a message.
[0136] If the query is successful, the subsequent nodes of the message receiving node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
[0137] This method corresponds to the execution of Figure 4 Detection points 6 and 7 are used to determine whether the message receiving node (cmq processor) is a faulty node and the corresponding fault cause. According to the log file of the message receiving node, it is determined whether the message receiving node writes the asynchronous message into the message file under the specified storage directory, which specifically includes the following verification steps:
[0138] (1) Check the log file of cmq processor at detection point 6 to determine whether cmq processor receives asynchronous messages correctly. If yes, proceed to the next verification step; otherwise, the detection ends and it can be determined that the cause of the failure is that cmqprocessor fails to receive the message successfully.
[0139] (2) Check the message file corresponding to the cmq processor at detection point 7 to determine whether the cmq processor correctly writes the asynchronous message to the message file in the specified directory. If so, proceed to the detection of subsequent nodes; otherwise, the detection ends and the cause of the fault can be determined to be an error in the cmq processor's message writing path.
[0140] In some solutions of the embodiments of the present application, based on the above solution, sequentially detecting subsequent nodes of the message receiving node to determine the faulty node and the fault cause corresponding to the faulty node includes:
[0141] Parse the configuration file of the message forwarding node to obtain second configuration information of the message forwarding node, and determine whether the second configuration information is complete, wherein the second configuration information includes a subscription relationship with a message consumer and the message consumer address configuration information;
[0142] If the second configuration information is incomplete, determining that the message forwarding node is a faulty node, and the fault cause is that the configuration information of the message forwarding node is incorrect;
[0143] Querying the message consumer address configuration information in the configuration center;
[0144] If the query is unsuccessful, it is determined that the message forwarding node is a faulty node, and the cause of the fault is that the address source channel of the message forwarding node is wrong;
[0145] If the query is successful, determining whether the message forwarding node correctly processes the asynchronous message based on the log of the message forwarding node;
[0146] If the message forwarding node fails to correctly process the asynchronous message, it is determined that the message forwarding node is a faulty node, and the fault cause is that the message forwarding node fails to correctly process the asynchronous message;
[0147] If the message forwarding node correctly processes the asynchronous message, the subsequent nodes of the message forwarding node are detected in sequence to determine the faulty node and the fault cause corresponding to the faulty node.
[0148] This method corresponds to the execution of Figure 4 Detection points 8, 9, and 10 in the middle determine whether the message forwarding node (cmq worker) is a faulty node and the corresponding fault cause. Among them, determining whether the second configuration information is complete corresponds to executing Figure 4 The detection point 8 includes the following verification steps:
[0149] (1) Determine whether cmq worker contains the configuration items corresponding to asynchronous messages. If so, proceed to the next verification step; otherwise, the detection ends and the cause of the failure can be determined to be that the conf configuration file of cmq worker lacks the configuration corresponding to asynchronous messages.
[0150] (2) Determine whether the subscription relationship of the asynchronous message in the cmq worker contains the message consumer information. If so, proceed to the subsequent steps; otherwise, the detection ends and it can be determined that the cause of the failure is that the subscription relationship of the asynchronous message in the conf configuration file of the cmq worker is incomplete.
[0151] Query the message consumer address configuration information in the configuration center, and execute Figure 4The detection point 9 specifically includes:
[0152] (1) Obtain information such as the instance name of the message consumer based on the subscription relationship of the asynchronous message in the configuration file of the cmq worker.
[0153] (2) According to the replication relationship of asynchronous messages in the configuration file of cmq worker, obtain the instance name and other information of the next replicator (cmqserver).
[0154] (3)According to the address configuration in the cmq worker configuration file, the address source channel of the message consumer and cmq server is obtained.
[0155] (4) According to the address source channel of cmq server, log in to PaaS container cloud platform (such as Figure 2 The cc configuration center in the cmq worker is used to obtain the IP and port information of the message consumer instance or the next cmq server (replicator) instance. If the query is successful, the subsequent process is entered; otherwise, the detection is completed and the cause of the failure can be determined to be an error in the address source channel in the cmq worker's conf configuration file.
[0156] It should be noted that the address source channel in the embodiment of the present application is not limited to the PaaS container cloud platform, and the method in the embodiment of the present application can be applied to various types of address sources.
[0157] According to the log of the message forwarding node, determine whether the message forwarding node correctly processes the asynchronous message, corresponding to the execution of Figure 4 Middle detection point 10.
[0158] It should be noted that after determining that the cmq worker is not a faulty node, it is necessary to detect whether the message consumer is a faulty node. This method is executed in Figure 4 At detection point 11, log in to the message consumer machine and check the consumer log to confirm the correctness and completeness of the message consumption process and results.
[0159] It is worth mentioning that message consumers include consumers of various protocol types, such as RPC protocol, firewall protocol, http / https protocol, tcp / udp protocol, etc. The corresponding fault location location of the present application is compatible with consumers of various protocol types. At the same time, the present application implements the TCPDUMP method to confirm whether the consumer has received the message, which can eliminate the influence of external factors such as the consumer machine disk is full.
[0160] It should be noted that the above-mentioned cmq server is in the same set as the message production device. When the consumer and producer are located in the domain and outside the domain respectively, the cmq worker needs to forward the message to the cmq processor of the cmq server outside the domain through the firewall protocol, and then the cmq worker outside the domain will deliver it; accordingly, when performing middleware fault detection, it is necessary to detect the cmq processor of the cmq server outside the domain and whether the cmq worker outside the domain is a faulty node. The method is the same as the detection process of the cmq processor and cmq worker mentioned above, which will not be repeated here.
[0161] In the case where the consumer and the producer are not in the same set, the cmq worker needs to first forward the message to the cmq processor of the cmq server in the set where the message consumer is located, and then its cmq worker delivers it; accordingly, when performing middleware fault detection, it is necessary to detect whether the cmq processor and cmqworker of the cmq server in the set where the message consumer is located are faulty nodes. The method is the same as the detection process of the cmq processor and cmq worker mentioned above, which will not be repeated here.
[0162] The process of transmitting messages across computer rooms and cities is exactly the same as the above cross-set message transmission process, except that the message replication link is longer, so it will not be repeated here.
[0163] The fault location method provided in the embodiments of the present application can detect the causes of failures of various message middleware occurring in each node of an asynchronous link, for example, failure to find the log sequence number in the message producer log, failure to write the message file of the producer device, the cmq proxy in the message producer device does not exist, there is no configuration information of asynchronous messages in the configuration file of cmq_proxy_, cmq_proxy does not forward asynchronous messages, the cmq processor or cmq worker in the consumer machine does not exist, the cmq processor does not forward asynchronous messages, the cmq_worker configuration file does not contain asynchronous messages or configuration information of message consumers, cmq worker does not forward asynchronous messages, the consumer service is abnormal or not started, and the consumer log does not effectively process asynchronous messages or fails to process.
[0164] The above is an example of CMQ-type message middleware to illustrate the fault location method of each node in the asynchronous link. For CQ-type message middleware, the detection method is similar to that of CMQ-type message middleware. For example, determining that the message producer in CQ is equivalent to the message producer in CMQ, determining whether the message producer in CQ is a faulty node and the corresponding fault cause are the same as the fault location method of the message producer in CMQ; the file reading thread in CQ is equivalent to the file reading thread in cmq proxy in CMQ, determining whether the file reading thread in CQ is a faulty node and the corresponding fault cause are the same as the fault location method of the file reading thread in cmq proxy in CMQ; the sending thread in CQ is equivalent to cmq worker in CMQ, determining whether the sending thread in CQ is a faulty node and the corresponding fault cause are the same as the fault location method of cmq worker in CMQ, which will not be repeated here.
[0165] The fault location method provided in the embodiment of the present application supports both single link detection and batch detection, and is suitable for scenarios of simple message transmission, cross-set message transmission, cross-domain message transmission, cross-computer room message transmission, and cross-city message transmission, to achieve full-process automated message middleware problem location.
[0166] The following describes an apparatus embodiment of the present application, which can be used in the message node fault location method in the above embodiment of the present application. For details not disclosed in the method embodiment of the present application, please refer to the above embodiment of the message node fault location method in the present application.
[0167] Figure 5 A schematic diagram of the structure of a message middleware fault location device according to an embodiment of the present application is shown. Figure 5 As shown, the device at least includes the following parts.
[0168] A detector package class 510, which includes a detection method for each node in the asynchronous link;
[0169] The input and output device 520 is used to parse the preset configuration file and obtain the path of the physical configuration file of the message producer;
[0170] The input-output device 520 is further used to read the physical configuration file according to the path of the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device;
[0171] The input-output device 520 is also used to obtain the log and message files of the message producer according to the service name of the message producer and the IP information of the message production device, and instantiate the detector encapsulation class according to the log and message files of the message producer;
[0172] The use case executor 530 is used to execute the instantiation object of the detector encapsulation class, detect each node in the asynchronous link in turn, and determine the faulty node and the fault cause corresponding to the faulty node.
[0173] In a specific implementation, the preset configuration file includes the project root directory, the relative path of the input / output Excel file, the message file directory and log directory of each node of the asynchronous link, the protocol type of the asynchronous message, etc.
[0174] The device may also include a public class, including some public function APIs, such as string processing, token calculation, etc., to facilitate calling the public class when detecting different nodes in the asynchronous link.
[0175] Each node of the asynchronous link can be connected to the interface automation test platform, and the use case executor can be connected to the interface automation test platform. An interface call is triggered through the interface automation test platform, which triggers the message producer to produce asynchronous messages.
[0176] It should be noted that although several units of the message middleware fault locating method and the message middleware fault locating device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the implementation mode of the present application, two or more units and functions described above can be concretized in one unit. Conversely, the features and functions of a unit described above can be further divided into multiple units for concretization. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.
[0177] As another aspect, the present application further provides a computer-readable storage medium on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present application may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present application described in the above "Exemplary Method" section of the present specification.
[0178] refer to Figure 6As shown, a program product 600 for implementing the above method according to an embodiment of the present application is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0179] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0180] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0181] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0182] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0183] As another aspect, the present application also provides an electronic device capable of implementing the above method.
[0184] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0185] Refer to the following Figure 7 hereinafter describes an electronic device 700 according to this embodiment of the present application. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0186] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).
[0187] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 executes the steps described in the above “Example Method” section of this specification according to various exemplary implementations of the present application.
[0188] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .
[0189] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0190] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0191] The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0192] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.
[0193] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0194] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be performed without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A message middleware fault location method, characterized in that: The method comprises: Create a detector encapsulation class according to the preset detection method of each node in the asynchronous link; Parse the preset configuration file to obtain the path of the physical configuration file of the message producer; According to the path of the physical configuration file, read the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device; According to the service name of the message producer and the IP information of the message production device, the log and message files of the message producer are obtained, and according to the log and message files of the message producer, the detector encapsulation class is instantiated; Execute the instantiated object of the detector encapsulation class to detect each node in the asynchronous link in turn according to the preset detection method, and determine the faulty node and the fault cause corresponding to the faulty node; The detecting each node in the asynchronous link in turn to determine the faulty node and the fault cause corresponding to the faulty node includes: Triggering the message producer to produce asynchronous messages; Obtaining a log sequence number corresponding to the asynchronous message in the message producer, wherein the log sequence numbers corresponding to the asynchronous message in each node of the asynchronous link are the same; Filtering the sending information of the asynchronous message in the log of the message producer according to the log sequence number corresponding to the asynchronous message, and determining whether the asynchronous message produced by the message producer meets the message sending condition according to the sending information of the asynchronous message; If the asynchronous message does not meet the message sending condition, then the message producer is indeed a faulty node, and the fault cause is that the asynchronous message does not meet the message sending condition; If the asynchronous message meets the message sending condition, the sending time and message subject of the asynchronous message are obtained according to the log of the message producer; Determining whether the message file of the message producer contains the asynchronous message according to the sending time and the message subject of the asynchronous message; If the message file of the message producer does not contain the asynchronous message, it is determined that the message producer is a faulty node, and the fault cause is the failure to write the asynchronous message; If the message file of the message producer contains the asynchronous message, the subsequent nodes of the message producer are detected in turn to determine the faulty nodes and the fault causes corresponding to the faulty nodes.
2. The message middleware fault location method according to claim 1, characterized in that: The step of parsing the preset configuration file to obtain the path of the physical configuration file of the message producer includes: Parse the preset configuration file to obtain the relative path and project root directory of the physical configuration file; The relative path of the physical configuration file and the project root directory are concatenated to obtain the absolute path of the physical configuration file.
3. The message middleware fault location method according to claim 1, characterized in that: The detecting the subsequent nodes of the message producer in sequence to determine the faulty nodes and the fault causes corresponding to the faulty nodes includes: Parse the configuration file of the message proxy node to obtain first configuration information of the message proxy node, wherein the first configuration information includes the instance name and address configuration information of the message server; Querying the configuration center for first configuration information of the message server; If the query is unsuccessful, it is determined that the message proxy node is a faulty node, and the cause of the fault is that the configuration information of the message server in the message proxy node is incorrect; If the query is successful, determining whether the message proxy node correctly processes the asynchronous message based on the log of the message proxy node; If the message proxy node does not correctly process the asynchronous message, it is determined that the message proxy node is a faulty node, and the fault cause is that the message proxy node does not correctly process the asynchronous message; If the message proxy node processes the asynchronous message correctly, the subsequent nodes of the message proxy node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
4. The message middleware fault location method according to claim 3, characterized in that: The detecting the subsequent nodes of the message proxy node in sequence to determine the faulty nodes and the fault causes corresponding to the faulty nodes includes: Determine, according to the log file of the message receiving node, whether the message receiving node writes the asynchronous message into the message file under the specified storage directory; If the message receiving node fails to write the asynchronous message into the message file under the specified storage directory, it is determined that the message receiving node is a faulty node, and the fault reason is that the message receiving node fails to receive the asynchronous message or the message writing path of the message receiving node is wrong; If the message receiving node writes the asynchronous message into a message file under a designated storage directory, the asynchronous message is searched for in the message file under the designated storage directory of the message server according to the service name of the message producer, the sending time of the asynchronous message and the subject information; If the query is unsuccessful, it is determined that the message receiving node is a faulty node, and the fault cause is that the message receiving node fails to write a message; If the query is successful, the subsequent nodes of the message receiving node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
5. The message middleware fault location method according to claim 4, characterized in that: The detecting the subsequent nodes of the message receiving node in sequence to determine the faulty node and the fault cause corresponding to the faulty node includes: Parse the configuration file of the message forwarding node to obtain second configuration information of the message forwarding node, and determine whether the second configuration information is complete, wherein the second configuration information includes a subscription relationship with a message consumer and the message consumer address configuration information; If the second configuration information is incomplete, determining that the message forwarding node is a faulty node, and the fault cause is that the configuration information of the message forwarding node is incorrect; Querying the message consumer address configuration information in the configuration center; If the query is unsuccessful, it is determined that the message forwarding node is a faulty node, and the cause of the fault is that the address source channel of the message forwarding node is wrong; If the query is successful, determining whether the message forwarding node correctly processes the asynchronous message based on the log of the message forwarding node; If the message forwarding node fails to correctly process the asynchronous message, it is determined that the message forwarding node is a faulty node, and the fault cause is that the message forwarding node fails to correctly process the asynchronous message; If the message forwarding node processes the asynchronous message correctly, the subsequent nodes of the message forwarding node are detected in turn to determine the faulty node and the fault cause corresponding to the faulty node.
6. A message middleware fault location device, characterized in that: The device comprises: A detector encapsulation class, wherein the detector encapsulation class includes a detection method for each node in an asynchronous link; Input and output devices are used to parse the preset configuration file and obtain the path of the physical configuration file of the message producer; The input-output device is further used to read the physical configuration file according to the path of the physical configuration file, and parse the physical configuration file to obtain the service name of the message producer and the IP information of the message production device, wherein the message producer is deployed on the message production device; The input and output device is also used to obtain the log and message files of the message producer according to the service name of the message producer and the IP information of the message production device, and instantiate the detector encapsulation class according to the log and message files of the message producer; A use case executor, used to execute the instantiation object of the detector encapsulation class, detect each node in the asynchronous link in turn, and determine the faulty node and the fault cause corresponding to the faulty node; The use case executor is also used to: Triggering the message producer to produce asynchronous messages; Obtaining a log sequence number corresponding to the asynchronous message in the message producer, wherein the log sequence numbers corresponding to the asynchronous message in each node of the asynchronous link are the same; Filtering the sending information of the asynchronous message in the log of the message producer according to the log sequence number corresponding to the asynchronous message, and determining whether the asynchronous message produced by the message producer meets the message sending condition according to the sending information of the asynchronous message; If the asynchronous message does not meet the message sending condition, then the message producer is indeed a faulty node, and the fault cause is that the asynchronous message does not meet the message sending condition; If the asynchronous message meets the message sending condition, the sending time and message subject of the asynchronous message are obtained according to the log of the message producer; Determining whether the message file of the message producer contains the asynchronous message according to the sending time and the message subject of the asynchronous message; If the message file of the message producer does not contain the asynchronous message, it is determined that the message producer is a faulty node, and the fault cause is the failure to write the asynchronous message; If the message file of the message producer contains the asynchronous message, the subsequent nodes of the message producer are detected in turn to determine the faulty nodes and the fault causes corresponding to the faulty nodes.
7. A computer device, characterized in that: The computer device comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the computer device, the method according to any one of claims 1 to 5 is implemented.
8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by an electronic device, the method according to any one of claims 1 to 5 is implemented.