A method for generating full-link logs without awareness

Abstract

The present invention discloses a method for generating full-link logs without awareness. It is characterized in that, through the MDC mechanism, the effects of injecting the link call serial number and the branch link serial number without awareness are realized, the link call serial number is dynamically injected, and various built-in link continuation point continuation logics are provided to realize the link tracing of full-link logs. The branch link serial number is dynamically injected to group and sort the logs of parallel logic links, determine the link continuation points of the log links, and introduce the corresponding link continuation logics according to their types; configure the injection points of the link call serial number and the branch link serial number in the global log format, and call the log printing method to print. This achieves the effect of avoiding repeated writing of the logic for injecting the link call serial number and the branch link serial number in the log printing logic, solving the problem of link disconnection in multi-threaded applications or distributed applications, avoiding the mixing of logs of parallel logic links, enhancing the organization of full-link logs, and improving the reading experience.

Description

Technical Field

[0001] The present invention relates to the field of link tracing in computer processing technology, and particularly to a method for generating full-link logs without awareness. Background Art

[0002] The logs of multi-threaded or distributed applications are disordered and discrete. When troubleshooting problems, it is difficult to find the logs related to the abnormal context in the disordered and discrete log content, which makes the difficulty of troubleshooting problems according to the log content in multi-threaded programs remain high.

[0003] In the existing public technology, the Slf4j framework provides the MDC mechanism, which supports dynamically injecting debugging information into the log content. However, the actual effect of the existing MDC mechanism cannot meet the above requirements, and its main disadvantages are as follows:

[0004] It cannot fully trace the call link: When the MDC mechanism was born, it did not consider the scenario of distributed application log tracing. In a distributed application, the link call sequence number and the branch link sequence number cannot be directly transmitted; in a multi-threaded application based on a thread pool, the historical data in the MDC containers of each working thread in the thread pool cannot be correctly processed.

[0005] It cannot be unaware: Since the MDC mechanism itself does not support distributed applications and multi-threaded applications based on thread pools, in these two types of applications, the application program needs to continuously write logic to process the link call sequence number and the branch link sequence number in the MDC.

[0006] In view of the above defects, the patent "Multi-threaded Link Tracing Method, Device and Computer Readable Storage Medium Based on MDC" (Application No.: 202010771440.2) provides a method for creating link identifiers through a preset method and inheriting multi-threaded link parameters by using preset rules. However, although this solution can solve the problem of link concatenation between multi-threads, it still cannot solve the problem of link concatenation in distributed applications, nor can it solve the problem of link grouping for multi-branch links.

[0007] Therefore, a method for generating full-link logs without awareness that supports multi-branch links is needed. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a method for generating full-link logs without awareness.

[0009] To solve the above technical problem, the present invention provides a method for generating full-link logs without awareness, which is characterized by including the following steps:

[0010] Step 1: Introduce a logging framework that supports the MDC mechanism in the application. The MDC mechanism is used to inject debugging information into the log, including a key-value pair mapping table bound to the thread that outputs the current log, so as to store the debugging information in the form of key-value pairs therein;

[0011] Step 2: Determine the starting point of the log link according to the type of the application. The position of the starting point of the log link is different under different application types;

[0012] Step 3: Introduce a link call sequence number generation logic at the starting point of the link. The link call sequence number generation logic includes parsing the link call sequence number and adding the link call sequence number to the MDC container. The MDC container is the specific implementation of the MDC mechanism in the logging framework. The link call sequence number generation logic includes checking whether there is a propagated link call sequence number in the request header. If not, a random string is generated as the link call sequence number for this call;

[0013] Step 4: Introduce a branch link sequence number generation logic at the starting point of the link. The branch link sequence number generation logic includes parsing the branch link sequence number and adding the branch link sequence number to the MDC container. The branch link sequence number generation logic first checks whether there is a propagated branch link sequence number in the request header. If not, a random string is generated as the branch link sequence number;

[0014] Step 5: Determine the continuation point of the log link according to the implementation logic of the application. The application and the continuation point of the log link are in a one-to-many mapping relationship, and there are multiple continuation points of the log link in one application;

[0015] Step 6: Introduce the corresponding link continuation logic according to the type of the continuation point of the log link;

[0016] Step 7: Configure the global log format, and set the injection points of the link call sequence number and the branch link sequence number in the global log format;

[0017] Step 8: Call the log printing method;

[0018] Step 9: Print the log, and append the link call sequence number and the branch link sequence number to the log content.

[0019] In Step 1, the logging framework includes but is not limited to log4j, log4j2, and logback. In Step 2, the application types are enumerated as distributed scheduled task applications and Servlet-based Web applications. In the scheduled task application, the starting point of the log link is the callback function of the scheduled task. In the Servlet-based Web application, the starting point of the log link is the request filter of the Web application.

[0020] In step 5, the link continuation points of the log link include Servlet request filter continuation points, distributed scheduled task callback function continuation points, thread pool continuation points, HTTP request processor continuation points, and Feign remote call continuation points; the continuation logic of the Servlet request filter continuation point includes: parsing the passed-through link call serial number and branch link serial number from the HTTP request header, and if the link call serial number and branch link serial number in the preset format can be parsed, adding the parsed link call serial number and branch link serial number to the MDC container; the continuation logic of the distributed scheduled task callback function continuation point includes: parsing the passed-through link call serial number and branch link serial number from the request parameters, and if the link call serial number and branch link serial number in the correct format can be parsed, adding the parsed link call serial number and branch link serial number to the MDC container; the continuation method of the thread pool continuation point includes: intercepting the execution method of the thread pool, clearing the historical data in the MDC container held by the current working thread before the original execution method is officially executed, copying the data in the MDC container held by the parent thread to the MDC container held by the current working thread, and generating a random string as the branch link serial number and setting it to the MDC container held by the current working thread; the continuation logic of the HTTP request processor continuation point includes: intercepting the HTTP request message and adding the link call serial number and branch link serial number stored in the MDC container of the current thread to the request message; the Feign remote call continuation point is a continuation point provided for Feign remote call operations; the continuation method of the Feign remote call continuation point includes: adding the logic of injecting the link call serial number and branch link serial number in the interceptor provided by the Feign framework; the logic of injecting the link call serial number and branch link serial number by the Feign remote call continuation point includes: first looking up the link call serial number and branch link serial number from the application request context, if the lookup fails, then continuing to look up the link call serial number and branch link serial number from the MDC container held by the current thread, and finally adding the found link call serial number and branch link serial number to the request header.

[0021] In step 5, the preset format of the link call serial number and branch link serial number is: an alphanumeric string with a length of 32 bits.

[0022] In step 5, the 32-bit alphanumeric string is generated by the UUID algorithm.

[0023] In step 5, one request is a link, and the sub-processes executed in parallel under one request are branch links, and the link and branch links are in a one-to-many mapping relationship.

[0024] In step 8, only the actual log content is passed in the method parameters; in step 9, logs with the same link call sequence number are logs of the same link, and logs with the same branch link sequence number are logs of the same branch link; the logs are grouped by the link call sequence number and the branch link sequence number, sorted by the log time, and multiple branch link logs are presented in parallel.

[0025] Advantages achieved by the present invention:

[0026] Through the MDC mechanism, the present invention realizes the effect of imperceptibly injecting the link call sequence number and the branch link sequence number, avoiding repeated writing of the logic for injecting the link call sequence number and the branch link sequence number in the log printing logic.

[0027] Through dynamically injecting the link call sequence number and building in multiple link continuation point continuation logics, the present invention realizes the link tracing of the full-link log, and solves the problem of link disconnection in multi-threaded applications or distributed applications.

[0028] By dynamically injecting the branch link sequence number, the present invention realizes the grouping and sorting of the logs of parallel logical links, avoids the mixing of the logs of parallel logical links, enhances the organization of the full-link log, and improves the reading experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow diagram of a method with multiple continuation points in an exemplary embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of the display form of the full-link log in an exemplary embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of the implementation process of the Servlet request filter continuation point logic in an exemplary embodiment of the present invention;

[0032] Figure 4 It is a schematic diagram of the implementation process of the timed task callback function continuation point logic in an exemplary embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of the implementation process of the thread pool continuation point logic in an exemplary embodiment of the present invention;

[0034] Figure 6 It is a schematic diagram of the implementation process of the HTTP request processor continuation point logic in an exemplary embodiment of the present invention;

[0035] Figure 7 It is a schematic diagram of the implementation process of the Feign remote call continuation point logic in an exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] In this case, multi-threaded or distributed applications are expected to have a means to group the logs generated by each call first by the link call sequence number, then by the branch link sequence number, then sort them horizontally by the log time, and finally sort them vertically by the log time of the first log on the branch link. By ordering and correlating the logs, the difficulty of tracing the full-link logs is reduced, and further the difficulty of troubleshooting problems based on the log content is reduced.

[0037] The multi-threaded application is an application that can execute multiple computer instructions in parallel at the same time; the distributed application refers to a large application that completes a task through cooperation over a network by micro-applications distributed on different computers; the full-link log refers to all the logs generated on the entire call link during one call; the link call sequence number is the sequence number generated when calling the service logic link, and each logical link generates a link call sequence number for each call; the branch link sequence number is the link number generated for each parallel logical link when there is parallel execution logic in the service logic link.

[0038] MDC (Mapped Diagnostic Contexts) is a thread-safe container for storing diagnostic logs.

[0039] The present invention will be further described below with reference to the accompanying drawings and exemplary embodiments:

[0040] As Figure 1 shown in the flow diagram of the implementation of a method with multiple continuation points in the embodiment of the present invention, the specific steps are as follows:

[0041] Step 101: Introduce a logging framework that supports the MDC mechanism in the application. The logging framework that supports the MDC mechanism selects Slf4j and log4j2.

[0042] Step 102: Determine the link starting point of the log link according to the type of the application. In the case of a Servlet-based Web application, the link starting point is the request filter.

[0043] Step 103: Introduce the link call sequence number generation logic in the Servlet request filter. The preset format of the link call sequence number is: an alphanumeric string with a length of 32 bits. The advantages of an alphanumeric string with a length of 32 bits include that it can be generated through the UUID algorithm with disorder and uniqueness.

[0044] Step 104: Introduce the generation logic of the branch link sequence number in the request filter of the Servlet. The preset format of the branch link sequence number is: an alphanumeric string with a length of 32 bits. The advantages of an alphanumeric string with a length of 32 bits include that it can be generated through the UUID algorithm with disorder and uniqueness.

[0045] Step 105: Determine the continuation points of the log link according to the implementation logic of the application. In the current application, the continuation points of the log link are the thread pool continuation point and the Feign remote call continuation point.

[0046] Step 106: Introduce the continuation logic of the thread pool continuation point and the continuation logic of the Feign remote call continuation point.

[0047] Step 107: Configure the global log format and set the injection points of the link call sequence number and the branch link sequence number in the global log format.

[0048] Step 108: Call the method of logging, and only pass the actual log content in the method parameters.

[0049] Step 109: Print the log, and append the link call sequence number and the branch link sequence number to the log content.

[0050] The corresponding technical effect is that the link call sequence number and the branch link sequence number are dynamically woven into the application log. The application log can be grouped according to the link call sequence number and the branch link sequence number, associate the logs of the same link, associate the logs of the same branch link, and separate the logs of the same link but different branch links, achieving the goal of unperceived, multi-dimensional, and full-link traceability of logs.

[0051] As Figure 2 shown in the schematic diagram of the presentation form of a full-link log in the embodiment of the present invention, this presentation form includes three main bodies, namely the main link, the branch link, and the log. Among them, Link 0 is the main link, and Link 1 and Link 2 are branch links.

[0052] The logs are first grouped according to the link call sequence number, then grouped according to the branch link sequence number, then sorted horizontally according to the log time, and finally sorted vertically according to the log time of the first log on the branch link.

[0053] The corresponding technical effect is that the application log is presented in a multi-branch link parallel display manner. When troubleshooting problems, it is very intuitive to distinguish between link logs and branch link logs, greatly reducing the difficulty of troubleshooting problems according to logs in multi-threaded application programs.

[0054] As Figure 3The flowchart of the logical implementation of the continuation point of a Servlet request filter in the embodiment of the present invention shown is as follows:

[0055] Step 301: Add a Servlet filter for processing the continuation logic of the processing link in the application and set the priority of the filter to the highest.

[0056] Step 302: Parse the link call sequence number from the Servlet request header. The specific method is to obtain the request header with the name "Trade-Call-ID" from the request header.

[0057] Step 303: Judge the value of the request header obtained in Step 302. If the value is empty, it is determined that the current Servlet is the starting point of the call link; if the value is not empty, it is determined that the current Servlet is the continuation point of the call link.

[0058] Step 304: If Step 303 determines that the current Servlet is the starting point of the call link, use the UUID algorithm to generate a random string, delete the "-" characters in the generated random string, and generate a 32-bit alphanumeric string. Assign the generated 32-bit alphanumeric string to the link call sequence number of the current request.

[0059] Step 305: If Step 303 determines that the current Servlet is the continuation point of the call link, assign the value of the request header with the name "Trade-Call-ID" to the link call sequence number of the current request.

[0060] Step 306: Transfer the request processing right to the request filter chain.

[0061] The corresponding technical effect is that in the Servlet request scenario, the log link is re-associated through the preset request filter, avoiding the loss of association of the link log when the distributed application provides services through the Servlet.

[0062] Such as Figure 4 The flowchart of the logical implementation of the continuation point of a distributed timed task callback function in the embodiment of the present invention shown is as follows:

[0063] Step 401: Replace the default timed task callback template method with a custom timed task callback template method.

[0064] Step 402: Add logic for cleaning the historical data of the MDC container held by the current thread in the custom timed task callback template method.

[0065] Step 403: Use the UUID algorithm to generate a random string in the custom timed task callback template method, remove the "-" characters in the generated random string to produce a 32-bit alphanumeric string. Assign the generated 32-bit alphanumeric string to the link call sequence number of the current request.

[0066] Step 404: Transfer the request processing right of the timed task processing request to the actual timed task callback function.

[0067] The corresponding technical effect is that in the distributed timed task scenario, the log link is associated through the link call sequence number generated by the custom timed task callback template method, avoiding the loss of association of the link log in the distributed timed task scenario.

[0068] As Figure 5 shown in the flowchart of the implementation of the thread pool continuation point logic in the embodiment of the present invention, the specific steps are as follows:

[0069] Step 501: Replace the default thread pool executor with a custom thread pool executor.

[0070] Step 502: Add the logic to clean the historical data of the MDC container held by the current working thread in the execution method of the custom thread pool executor.

[0071] Step 503: Add the logic to copy the context of the MDC container held by the parent thread to the MDC container held by the current working thread in the execution method of the custom thread pool executor.

[0072] Step 504: Add the logic to generate a branch link sequence number in the execution method of the custom thread pool executor.

[0073] Step 505: Add the logic to set the branch sequence number to the MDC container held by the current working thread in the execution method of the custom thread pool executor.

[0074] Step 506: Add the logic to call the default thread pool execution method in the execution method of the custom thread pool executor.

[0075] The corresponding technical effect is that in the thread pool scenario, the log link is re-associated through the custom thread pool executor, and the branch link is separated from the full link through the custom thread pool executor to generate log data that can be presented in multiple dimensions.

[0076] As Figure 6 shown in the flowchart of the implementation of the HTTP request processor continuation point logic in the embodiment of the present invention, the specific steps are as follows:

[0077] Step 601: Replace the default HTTP request processor with a custom HTTP request processor.

[0078] Step 602: Add logic to clean up the historical data of the MDC container held by the current thread in the execution method of the custom HTTP request processor.

[0079] Step 603: Obtain the link call sequence number in the execution method of the custom HTTP request processor, and set the link call sequence number into the request header. Specifically, set the value of the request header named "Trade-Call-ID" in the request header to the link call sequence number.

[0080] Step 604: Add logic to call the execution method of the default HTTP request processor in the execution method of the custom HTTP request processor.

[0081] The corresponding technical effect is that in the HTTP request scenario, the link call sequence number is processed by the custom HTTP request executor, and the link call sequence number is passed transparently to the application of the next link node in the request message header, providing continuous data support for the next link node.

[0082] As Figure 7 shown in the flowchart of the implementation of the Feign remote call continuation point logic in the embodiment of the present invention, the specific steps are as follows:

[0083] Step 701: Add a Feign interceptor for processing link continuation logic in the application program, set the priority of the interceptor to the highest, and place the execution order of the interceptor at the end.

[0084] Step 702: Set the link call sequence number in the HTTP request processor used in Feigin to null. Specifically, set the value of the request header named "Trade-Call-ID" in the request header to null.

[0085] Step 703: Obtain the link call sequence number from the link context. If the link call sequence number cannot be obtained from the link context, obtain the link call sequence number from the MDC container.

[0086] Step 704: Set the link call sequence number in the HTTP request processor used in Feigin to the link call sequence number obtained in Step 703. Specifically, set the value of the request header named "Trade-Call-ID" in the request header to the link call sequence number obtained in Step 703.

[0087] The corresponding technical effect is that in the scenario of Feign remote call, the link call sequence number is processed through a custom Feign request interceptor, and the link call sequence number is passed transparently to the application of the next link node in the request message header, providing continuous data support for the next link node.

[0088] The present invention is mainly used to provide a method for generating full-link logs without perception, and its beneficial effects are as follows:

[0089] Through the MDC mechanism, the present invention realizes the effect of injecting the link call sequence number and the branch link sequence number without perception, avoiding repeated writing of the logic for injecting the link call sequence number and the branch link sequence number in the log printing logic.

[0090] Through dynamic injection of the link call sequence number and built-in multiple link continuation point continuation logics, the present invention realizes the link tracing of full-link logs, solving the problem of link disconnection in multi-threaded applications or distributed applications.

[0091] By dynamically injecting the branch link sequence number, the present invention realizes the log grouping and sorting of parallel logical links, avoiding the mixing of logs of parallel logical links, enhancing the organization of full-link logs, and improving the reading experience.

[0092] The above embodiments do not limit the present invention in any way. Any other improvements and applications made to the above embodiments by equivalent transformation methods belong to the protection scope of the present invention.

Claims

1. A method for generating full-link logs without perception, characterized in that, Including the following steps: Step 1: Introduce a logging framework that supports the MDC mechanism in the application. The MDC mechanism is used to inject debugging information into the log, including a key-value pair mapping table bound to the thread that outputs the current log, so as to store the debugging information in the form of key-value pairs therein; Step 2: Determine the starting point of the log link according to the type of the application. The position of the starting point of the log link is different under different application types; Step 3: Introduce a link call sequence number generation logic at the starting point of the link. The link call sequence number generation logic includes parsing the link call sequence number and adding the link call sequence number to the MDC container. The MDC container is a specific implementation of the MDC mechanism in the logging framework. The link call sequence number generation logic includes checking whether there is a propagated link call sequence number in the request header. If not, a random string is generated as the link call sequence number for this call; Step 4: Introduce a branch link sequence number generation logic at the starting point of the link. The branch link sequence number generation logic includes parsing the branch link sequence number and adding the branch link sequence number to the MDC container. The branch link sequence number generation logic first checks whether there is a propagated branch link sequence number in the request header. If not, a random string is generated as the branch link sequence number; Step 5: Determine the continuation point of the log link according to the implementation logic of the application. The application and the continuation point of the log link have a one-to-many mapping relationship, and there are multiple continuation points of the log link in one application; Step 6: Introduce corresponding link continuation logics according to the types of the continuation points of the log link; Step 7: Configure the global log format and set the injection points for the link call sequence number and the branch link sequence number in the global log format; Step 8: Call the log printing method; Step 9: Print the log and append the link call sequence number and the branch link sequence number to the log content.

2. The method for generating a full-link log without perception according to claim 1, wherein: In Step 1, the logging framework includes but is not limited to log4j, log4j2, and logback. In Step 2, the application types are enumerated as distributed scheduled task applications and Servlet-based Web applications. In the scheduled task application, the starting point of the log link is the callback function of the scheduled task. In the Servlet-based Web application, the starting point of the log link is the request filter of the Web application.

3. The method for generating a full-link log without perception according to claim 2, characterized in that: In step 5, the link continuation points of the log link include Servlet request filter continuation points, distributed scheduled task callback function continuation points, thread pool continuation points, HTTP request processor continuation points, and Feign remote call continuation points; the continuation logic of the Servlet request filter continuation point includes: parsing the passed-through link call sequence number and branch link sequence number from the HTTP request header, and if the link call sequence number and branch link sequence number in the preset format can be parsed, adding the parsed link call sequence number and branch link sequence number to the MDC container; the continuation logic of the distributed scheduled task callback function continuation point includes: parsing the passed-through link call sequence number and branch link sequence number from the request parameters, and if the link call sequence number and branch link sequence number in the correct format can be parsed, adding the parsed link call sequence number and branch link sequence number to the MDC container; the continuation method of the thread pool continuation point includes: intercepting the execution method of the thread pool, clearing the historical data in the MDC container held by the current working thread before the original execution method is officially executed, copying the data in the MDC container held by the parent thread to the MDC container held by the current working thread, generating a random string as the branch link sequence number, and setting it to the MDC container held by the current working thread; the continuation logic of the HTTP request processor continuation point includes: intercepting the HTTP request message and adding the link call sequence number and branch link sequence number stored in the MDC container of the current thread to the request message; the Feign remote call continuation point is a continuation point provided for Feign remote call operations; the continuation method of the Feign remote call continuation point includes: adding the logic of injecting the link call sequence number and branch link sequence number in the interceptor provided by the Feign framework; the logic of injecting the link call sequence number and branch link sequence number of the Feign remote call continuation point includes: first looking up the link call sequence number and branch link sequence number from the application request context, if the lookup fails, then continuing to look up the link call sequence number and branch link sequence number from the MDC container held by the current thread, and finally adding the found link call sequence number and branch link sequence number to the request header.

4. The method for generating a full-link log without perception according to claim 3, wherein: In step 5, the preset format of the link call sequence number and branch link sequence number is: an alphanumeric string with a length of 32 bits.

5. The method for generating a full-link log without perception according to claim 4, wherein: In step 5, the 32-bit alphanumeric string is generated by the UUID algorithm.

6. The method for generating a full-link log without perception according to claim 5, wherein In step 5, one request is a link, and the sub-processes executed in parallel under one request are branch links. The link and the branch link have a one-to-many mapping relationship.

7. The method for generating a full-link log without perception according to claim 6, wherein, In step 8, only the actual log content is passed in the method parameters; in step 9, the logs with the same link call sequence number are logs of the same link, and the logs with the same branch link sequence number are logs of the same branch link; the logs are grouped by the link call sequence number and the branch link sequence number, sorted by the log time, and the multi-branch link logs are presented in parallel.

Images