Penetrating operation call chain log system

Abstract

The present invention provides a call chain log system and method for penetrating operation, which relates to the field of computer technology. The method includes: a link information module: at the entrance of a service request, a globally unique call link ID is assigned; a link position ID that records the position of the service in the entire link, and the link position ID records the upstream link position and the service position; a service log information module: a log ID is assigned to a business service, and relevant information about the business is recorded; a link information transmission module: mutual calls between services are completed, and call link information is transmitted; a log recording module: the recorded data is recorded separately in a file in the form of a log. The present invention can support operators to discover business failures and quickly locate problems; support performance analysis, audit analysis, and statistical analysis of services; and support audit analysis and statistical analysis of user behavior.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a call chain log system for penetrating operations. Background Art

[0002] With the continuous development of Internet technology, the promotion of distributed, microservices, cloud computing and other technical concepts, the complexity of application systems is also constantly evolving. A user's operation may involve multiple services and multiple systems for collaborative processing. These services may be on different machines, different clusters, or even different regions. The link relationship formed by the mutual calls between multiple services is called a call chain.

[0003] The invention patent with publication number CN112579406A discloses a log call chain generation method and device. A specific implementation of the method includes: generating an identification code according to the identifier of the device where the application is located and the current timestamp; monitoring the transmission order of the identification code between various application modules; extracting the logs corresponding to the identification code in each application module, and arranging and combining the extracted logs according to the transmission order to obtain a log call chain. This implementation method uses the identification code to automatically extract the logs in each log server, and sorts the logs according to the transmission order of the identification code between application modules, thereby realizing the automatic generation of the log call chain, which provides a basis for centralized log archiving and abnormal log positioning.

[0004] In increasingly complex systems, especially platform-level systems, operators need to promptly discover business failures and quickly locate problems. For systems involving distributed, microservices, and other architectures, it is difficult to quickly locate problems after discovering business failures, and a large number of business logs need to be analyzed for failures. It is also necessary to analyze, statistically analyze, and audit service performance. It is also necessary to regularly audit and statistically analyze system user behavior. At this time, data support within the system is needed, including call chain data, business data, user behavior data, etc.

[0005] In related technologies, distributed call chain data is used to help locate business failures and quickly locate problems. However, existing distributed call chain solutions have defects such as large data volume, no business attributes, inability to associate with business, and inability to count and audit user behavior. User behavior analysis technology also has defects such as inability to view business links, inability to associate with business failures, and inability to perform performance analysis on services. Summary of the invention

[0006] In view of the defects in the prior art, the present invention provides a call chain log system with penetrating operation.

[0007] According to a call chain log system for penetrating operation provided by the present invention, the scheme is as follows:

[0008] A call chain log system for penetrating operation, the system comprising:

[0009] Link information module: At the entrance of the service request, a globally unique call link ID is assigned; the link position ID that records the position of the service in the entire link, and the link position ID will record the upstream link position and the service position;

[0010] Service log information module: assigns log IDs to business services and records relevant information about the business;

[0011] Link information transmission module: completes the mutual calling between services and transmits the calling link information;

[0012] Logging module: records the recorded data in a separate file in the form of logs.

[0013] Preferably, the link information module serially calls the chain links so that the links can be serially connected in a complete tree structure.

[0014] Preferably, the service-related information recorded in the service log information module includes: status information recording the service execution status, user ID of user information, start time and end time of service execution, and other information used to describe the service.

[0015] Preferably, the link information transmission module includes: allocating a globally unique call link ID at the service entrance, generating a link position ID that records the position of the service in the link; when the first service calls the second service, passing to the second service the call link information: the call link ID, the link position ID of the second service; after the second service receives the link information transmitted by the first service, it uses the received call link ID as the call link ID of its own service, and uses the received link position ID as the link position ID of its own service.

[0016] Preferably, the content in the log recording module is recorded in the form of key-value pairs in JSON format; each service or more detailed event records a log, and each log ends with a line break.

[0017] The system's method of operation includes:

[0018] Step S1: Check at the system entry service whether there is a call chain link ID and link position ID passed;

[0019] Step S2: Start the service and record the service log information; before the service is finished, cache the service log information in the memory, wait for the service to end, and record other service information;

[0020] Step S3: If the current service calls other services during execution, a link location ID is assigned to the current service and then passed to the next service together with the calling link ID;

[0021] Step S4: When the service is finished, relevant service information from the service start time to the service end time is recorded, and other service information is supplemented;

[0022] Step S5: Log output, service execution ends, check whether the call chain log is complete.

[0023] Preferably, the step S1 includes: if the system entry service checks that there is a call chain link ID and a link position ID transmitted, inheriting the existing link information, and taking the received call chain link ID and link position ID as its own call chain link ID and link position ID;

[0024] If the system entry service checks that there is no call chain link ID and link position ID passed in, it generates a call chain link ID and link position ID.

[0025] Preferably, the service log information in step S2 includes: the log ID of the service log, the start time, the service name, the project to which the service belongs, and relevant service information including the application to which the service belongs.

[0026] Preferably, the relevant service information in step S4 includes: service end time, whether the service is successfully executed, and calculation time of the service.

[0027] Preferably, the step S5 includes: checking whether the call chain log is complete, and if complete, outputting it to a log file;

[0028] If it is incomplete, the missing information will be supplemented if possible and output to the log file. If it cannot be supplemented, an exception log will be recorded.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention provides a call chain recording method, which records the call chain information, so that operators can quickly find and locate problems; records service information, so that service performance can be analyzed; and records business information, so that business audits and statistical analysis can be performed;

[0031] 2. The present invention combines the call chain log and the system log to realize multiple uses of the call chain log data, which can be used for system performance analysis, service audit analysis, service statistical analysis, user audit analysis, user statistical analysis and other analyses, and can realize penetrating operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0033] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0034] Figure 2 A schematic diagram for link location ID delivery;

[0035] Figure 3 Schematic diagram of link content. DETAILED DESCRIPTION

[0036] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0037] The embodiment of the present invention provides a call chain log system for penetrating operation, referring to Figure 1 As shown, the system includes: a call chain link information module, a service log information module, a link information transmission module and a log recording module. Among them, the call chain link information module generates link information, which is transmitted between different services by the link information transmission module; after being transmitted to a service, the data is transmitted to the service log information module, which will add business information based on the call chain link information, and then send the data to the log recording module for log recording and output to a file.

[0038] Specifically, as shown in Table 1, the link information module includes: at the entrance of the service request, a globally unique call link ID is assigned to the request; a link position ID that records the position of the service in the entire link, and the link position ID records the upstream link position and the service position. In the call chain log information, the call link ID and link position ID will be passed along with the service call. The call chain log link information is used to connect the call chain links in series so that the links can be connected in series in a complete tree structure. Among them, at the service entrance, a globally unique call link ID is assigned to each request. Multiple requests for a service will be assigned multiple call link IDs.

[0039] Table 1

[0040]

[0041]

[0042] As shown in Table 2, the service log information module includes: assigning a service log ID to a service, recording it to form a service log, recording the status information of the business service execution status; recording the user ID of the user information; recording the start time and end time of the service execution; and other information used to describe the business. The service can be all types of services such as business services, program services, and data services.

[0043] Table 2

[0044]

[0045] The link information transmission module includes: allocating a globally unique call link ID at the service entrance, generating a link position ID that records the position of the service in the link; when the first service calls the second service, it will pass on to the second service the call link information: the call link ID, the link position ID of the second service; after the second service receives the link information passed by the first service, it uses the received call link ID as the call link ID of its own service, and uses the received link position ID as the link position ID of its own service.

[0046] Reference Figure 2 As shown, the link location ID is transmitted as follows: (1) When the entry service A has a link location ID of [1] and service B is called, the link location ID [1.1] of service B is generated and passed to service B.

[0047] (2) After service A calls service B, it calls service C. Service C is called by service A and follows service B. The link location ID [1.2] is generated and passed to service C.

[0048] (3) After receiving the message, service C uses [1.2] as the link location ID of this service and passes it on.

[0049] The logging module includes: recording the recorded data in a separate file in the form of logs; recording the content in the form of key-value pairs in JSON format; recording a log for each service or more detailed event, and each log ends with a line break. Log example:

[0050] {"endTime":"20201027105506736","project":"eplat","traceId":"6556cc9c4e61448b9586ceaddf40077c","spanId ":"1.1.1.1","status":"0","logId":"0201","cost":48,"startTime":"20201027105506688","app":"operation"}.

[0051] The present invention also provides a call chain log method for penetrating operation, and the method is implemented in the following steps:

[0052] 1) Check at the system entry service whether there is a call chain link ID and link position ID passed in. If so, inherit the existing link information and use the received call chain link ID and link position ID as its own call chain link ID and link position ID; if not, generate a call chain link ID and link position ID.

[0053] 2) When the service starts, the service log information is recorded. The service log includes the log ID, start time, service name, service project, service application and other service information. The service has not ended, and the above information is cached in the memory. Wait for the service to end and record other service information.

[0054] 3) If the service calls other services during execution, a link location ID is assigned to the service and then passed to the next service together with the call link ID.

[0055] 4) When the service is completed, record the service end time, whether the service is successfully executed, calculate the service time (service start time - service end time), and add other service information.

[0056] 5) Log output: After the service execution is completed, check whether the call chain log is complete. If it is complete, output it to the log file. If it is incomplete, the missing information can be supplemented and output to the log file. If it cannot be supplemented, record the exception log.

[0057] Next, the present invention will be described in more detail.

[0058] The requirements in the embodiment of the present invention include: 1. Implementing service tracking, being able to query which methods, services, and databases are called for a request; whether there are any exceptions, and capturing the exception information.

[0059] 2. Realize service analysis and user behavior auditing, and be able to query high-frequency services, abnormal services, slow services and other service analyses through data; be able to query user login lists, user access records and other user audit analyses.

[0060] Taking user login behavior as an example, the implementation method of call chain log is as follows:

[0061] Step 1: At the system HTTP layer, after receiving the request, a 16-bit unique call chain link ID is generated for this request, and a link location ID is generated [1].

[0062] Step 2: Receive the call chain link ID and link position ID [1] generated in the first step, start the HTTP request service with the service log ID

[0000] , and record the project, application information, and start time to which the HTTP request service belongs.

[0063] Step 3: After the HTTP request is received, the user login service with service log ID

[0001] will be called. Before the HTTP request calls the user login service, a link location ID [1.1] is generated for the user login service and passed to the user login service with service log ID

[0001] together with the call chain link ID generated in the first step.

[0064] Step 4: Receive the call chain link ID and link location ID [1.1] from step 3, and the user login service starts. The user login service also records the service project, application, start time and other information; and calls two services: first call the unified authentication service with service log ID

[0002] ; then call the security authentication service with service log ID

[0003] . Similarly, link location IDs [1.1.1] and [1.1.2] are assigned to the service before the service is called.

[0065] Step 5: The unified authentication service with service log ID

[0002] receives the call chain link ID and link position ID [1.1.1] from step 4; it also records the service's directory, application, and start time. After the service is executed, based on the execution result, it records whether the execution status is successful, records the end time, and calculates and records the service time. After recording the information, check the call chain log information. If there is any unrecorded information, supplement the record; then output it to the log file to form a log record. The security authentication service with service log ID

[0003] uses the same processing method.

[0066] Step 6: After the unified authentication service with service log ID

[0002] and the security authentication service with service log DI

[0003] are completed, the main thread continues to execute the user login service with service log ID

[0001] . When the user login service ends, the end time of the service is recorded, and the service time is calculated and recorded; after recording the information, check the call chain log information, and if there is any unrecorded information, supplement the record; then output it to the log file to form a log record.

[0067] Step 7: After the user login service with service log ID

[0001] is executed, the main thread continues to execute the HTTP request service. After the service is finished, the end time of the service is recorded, and the service time is calculated and recorded. After recording the information, check the call chain log information. If there is any unrecorded information, add the record; then output it to the log file to form a log record. At this point, a service request ends. The link content is as follows Figure 3 As shown, the log content is as follows:

[0068] {"endTime":"2021-10-2710:55:06.736","project":"eplat","traceId":"6556cc9c4e61448b9586ceaddf40077c","span Id":"1.1.1","status":"0","logId":"0002","cost":48,"startTime":"2021-10-2710:55:06.688","app":"operation"}

[0069] {"endTime":"2021-10-2710:55:06.757","project":"eplat","traceId":"6556cc9c4e61448b9586ceaddf40077c","span Id":"1.1.2","status":"0","logId":"0003","cost":20,"startTime":"2021-10-2710:55:06.737","app":"operation"}

[0070] {"endTime":"2021-10-2710:55:06.772","project":"eplat","traceId":"6556cc9c4e61448b9586ceaddf40077c","spanId":"1.1","status":"0","logId":"0001","cos t":105,"startTime":"2021-10-2710:55:06.667","app":"operation"}

[0071] {"endTime":"2021-10-2710:55:06.773","project":"eplat","traceId":"6556cc9c4e61448b9586ceaddf40077c","spa nId":"1","status":"0","logId":"0000","cost":110,"startTime":"2021-10-2710:55:06.663","app":"operation"}.

[0072] The embodiment of the present invention provides a call chain log system for penetrating operation, which records call chain information to facilitate operation personnel to quickly discover and locate problems; records service information to facilitate analysis of service performance; records business information to facilitate business auditing and statistical analysis; and the present invention combines call chain logs with system logs to achieve multiple uses of call chain log data, which can be used for system performance analysis, service audit analysis, service statistical analysis, user audit analysis, user statistical analysis, etc., and can achieve penetrating operation of the system.

[0073] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0074] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A call chain log system with penetrating operation, characterized in that: include: Link information module: assigns a globally unique call link ID at the entrance of the service request; The link location ID records the location of the service in the entire link. The link location ID records the upstream link location and the location of the service. Service log information module: assigns log IDs to business services and records relevant information about the business; Link information transmission module: completes the mutual calling between services and transmits the calling link information; Logging module: records the recorded data in a separate file in the form of logs; The method of operating the system includes: Step S1: Check at the system entry service whether there is a call chain link ID and link position ID passed; Step S2: Start the service and record the service log information; before the service is finished, cache the service log information in the memory, wait for the service to end, and record other service information; Step S3: If the current service calls other services during execution, a link location ID is assigned to the current service and then passed to the next service together with the calling link ID; Step S4: When the service is finished, relevant service information from the service start time to the service end time is recorded, and other service information is supplemented; Step S5: Log output, service execution ends, check whether the call chain log is complete.

2. The call chain log system for penetrating operation according to claim 1 is characterized in that: The link information module connects the call chain links in series, so that the links can be connected in series in a complete tree structure.

3. The call chain log system for penetrating operation according to claim 1 is characterized in that: The service log information module records the relevant information of the business, including: status information recording the business service execution status, user ID of user information, start time and end time of service execution, and other information used to describe the business.

4. The call chain log system for penetrating operation according to claim 1 is characterized in that: The link information transmission module includes: allocating a globally unique call link ID at the service entrance, generating a link position ID that records the position of the service in the link; when the first service calls the second service, passing the call link information to the second service: the call link ID, the link position ID of the second service; after the second service receives the link information transmitted by the first service, it uses the received call link ID as the call link ID of its own service, and uses the received link position ID as the link position ID of its own service.

5. The call chain log system for penetrating operation according to claim 1 is characterized in that: The content in the logging module is recorded in the form of key-value pairs in JSON format; each service or more detailed event records a log, and each log ends with a line break.

6. The call chain log system for penetrating operation according to claim 1 is characterized in that: The step S1 includes: if the system entry service checks that there is a call chain link ID and a link position ID transmitted, inheriting the existing link information, and taking the received call chain link ID and link position ID as its own call chain link ID and link position ID; If the system entry service checks that there is no call chain link ID and link position ID passed in, it generates a call chain link ID and link position ID.

7. The call chain log system for penetrating operation according to claim 1 is characterized in that: The service log information in step S2 includes: the log ID of the service log, the start time, the service name, the project to which the service belongs, and relevant service information including the application to which the service belongs.

8. The call chain log system for penetrating operation according to claim 1 is characterized in that: The relevant service information in step S4 includes: service end time, whether the service is successfully executed, and calculation of service time consumption.

9. The call chain log system for penetrating operation according to claim 1 is characterized in that: The step S5 includes: checking whether the call chain log is complete, and if complete, outputting it to the log file; If it is incomplete, the missing information will be supplemented if possible and output to the log file. If it cannot be supplemented, an exception log will be recorded.

Images