Real-time log tracking and positioning method

Abstract

The invention discloses a real-time log tracking and positioning method, and relates to the technical field of micro-service governance. Comprising the following steps: step 1, configuring configuration information of a log debugging user: step 11, configuring configuration items related to log opening and closing and storing the configuration items in a database, step 12, configuring a log monitoring path and storing the log monitoring path in the database, and step 13, configuring configuration items related to redis connection and storing the configuration items in the database; step 2, based on a spring framework, using AOP to create a slice monitoring log output class, checking whether a current login user is a log debugging user through slices according to the log output class, if yes, matching a log switch of the log debugging user to be in an on state and configuring a log of a log monitoring path, copying and outputting the log to redis, and if not, copying and outputting the log to the Redis; and 3, making a log viewing tool by utilizing the jar package, and retrieving related logs in the redis through the log viewing tool.

Description

Technical Field

[0001] The present invention discloses a method, which relates to the field of microservice governance technology, and specifically a real-time log tracking and positioning method. Background Art

[0002] The distributed architecture of microservices and containerization technology is gradually becoming popular. Under this solution, a service may run in thousands of containers, which can effectively distribute the user's access pressure to each container. However, this solution also exposes some problems. Once a problem occurs in the service, software debugging is extremely inconvenient. Log tracking and positioning can be performed to confirm the specific problem. However, the existing log query and positioning method uses the log collector to periodically aggregate the logs generated by the services in the container to the designated log server, and then use related tools to query and locate the aggregated logs. This method has some disadvantages, such as:

[0003] There is a delay in synchronizing logs to the log server, and logs cannot be viewed efficiently and in real time.

[0004] If there are many service requests at the same time, it may not be possible to accurately locate the required logs.

[0005] After the log server aggregates all logs, the log volume may be very large, and locating the logs may be very time-consuming.

[0006] To increase the retrieval speed, you may need to invest more software and hardware resources in the log server, increasing costs.

[0007] Logs are not highly relevant to users. To debug a service, the log level is usually lowered to debug to print more log information. However, turning on debug may cause all user requests to output a large amount of logs, which will put greater pressure on frequently accessed services and even cause downtime, causing disasters to normal business.

[0008] Changing the log level requires restarting the service application. Logging levels are typically adjusted through configuration files, and each adjustment typically requires a service restart, which can cause irreversible failures in running production applications. Summary of the Invention

[0009] The present invention aims to solve the problems of the prior art and provide a real-time log tracking and positioning method to make up for the shortcomings of the existing log processing mode. Without affecting the log processing, the method can track and accurately locate the logs in real time and efficiently.

[0010] The specific scheme proposed by the present invention is:

[0011] The present invention provides a real-time log tracking and positioning method, comprising:

[0012] Step 1: Configure the log debugging user configuration information:

[0013] Step 11: Configure the log opening and closing related configuration items and store them in the database.

[0014] Step 12: Configure the log monitoring path and store it in the database.

[0015] Step 13: Configure the configuration items related to Redis connection and store them in the database;

[0016] Step 2: Based on the Spring framework, use AOP to create a slice monitoring log output class. According to the log output class, verify whether the current logged-in user is a log debugging user through the slice. If so, match the log of the log debugging user with the log switch turned on and the log monitoring path configured, and copy the log to redis.

[0017] Step 3: Use the jar package to create a log viewing tool and use the log viewing tool to retrieve relevant logs in Redis.

[0018] Furthermore, in the real-time log tracking and positioning method, the configuration items related to the redis connection in step 13 include:

[0019] Configure the address log.redis.url for logging into redis.

[0020] Configure the password log.redis.password required to log in to redis: leave it blank if none exists.

[0021] Configure the Redis port log.redis.port.

[0022] Furthermore, in the real-time log tracking and positioning method, step 2 of copying and outputting the log to Redis includes:

[0023] The key corresponding to redis is composed of ${userId}_${class name}_${method name}_${log line number}, where ${userId} is the user's unique identifier, ${class name} is the class name corresponding to the service, ${method name} is the method name corresponding to the service, and ${log line number} is the line number corresponding to the log output in the source code file. The redis key is stored in the system thread variable in the form of an array list.

[0024] Furthermore, in step 3 of the real-time log tracking and positioning method, a log viewing tool is created using a jar package, and the log viewing tool is used to retrieve relevant logs in redis according to the key value in the thread variable.

[0025] The present invention also provides a real-time log tracking and positioning device, including a configuration module, a monitoring module, a matching module, a storage module and a viewing management module.

[0026] Configuration information of the configuration module configuration log debugging user:

[0027] Step 11: Configure the log opening and closing related configuration items and store them in the database.

[0028] Step 12: Configure the log monitoring path and store it in the database.

[0029] Step 13: Configure the configuration items related to Redis connection and store them in the database;

[0030] The monitoring module is based on the Spring framework and uses AOP to create a slice monitoring log output class. According to the log output class, the slice verifies whether the current logged-in user is a log debugging user. If so, the matching module matches the log of the log debugging user whose log switch is turned on and the log monitoring path is configured. The storage module copies the log and outputs it to redis.

[0031] The viewing management module uses the jar package to create a log viewing tool, and retrieves relevant logs in redis through the log viewing tool.

[0032] Furthermore, the configuration module of the real-time log tracking and positioning device configures configuration items related to redis connection, including:

[0033] Configure the address log.redis.url for logging into redis.

[0034] Configure the password log.redis.password required to log in to redis: leave it blank if none exists.

[0035] Configure the Redis port log.redis.port.

[0036] Furthermore, the storage module of the real-time log tracking and positioning device outputs the log copy to Redis, including:

[0037] The key corresponding to Redis is composed into the form of ${userId}_${class name}_${method name}_${log line number}, where ${userId} is the user's unique identifier, ${class name} is the class name corresponding to the service, ${method name} is the method name corresponding to the service, and ${log line number} is the line number corresponding to the log output in the source code file. The Redis key is stored in the system thread variable in the form of an array list.

[0038] Furthermore, the viewing management module of the real-time log tracking and positioning device uses a jar package to create a log viewing tool, and uses the log viewing tool to retrieve relevant logs in redis according to the key value in the thread variable.

[0039] The benefits of the present invention are:

[0040] To make up for the shortcomings of existing log processing methods, without affecting the log processing method, log configuration can be simply and conveniently operated according to actual conditions; turning logs on and off does not require restarting the service application, and does not affect normal business; log information is not stored on the log server disk, but on Redis, which is efficient and real-time; logs are highly bound to users, and logs are only turned on for debugging users, and logs are not printed for non-debugging users; it does not increase the burden on the server and does not require additional software and hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of obtaining a log switch in the method of the present invention.

[0042] Figure 2 It is a flow chart of the method of the present invention for obtaining the log monitoring path.

[0043] Figure 3 It is a flow chart of log recording implementation in the method of the present invention. DETAILED DESCRIPTION

[0044] AOP stands for Aspect-Oriented Programming. It is a supplement to Object-Oriented Programming (OOP) and has become a mature programming method.

[0045] Slicing is a technique for weaving aspect code into target code. Through slicing, code unrelated to business logic can be extracted to form independent aspects, thereby improving code modularity and maintainability.

[0046] Aspects are areas of concern that span multiple modules or classes during program execution. For example, logging, security checks, and transaction management are all typical aspects.

[0047] A thread local variable is a special variable that each thread has its own copy of. Each thread can independently access and modify its own copy of the variable without affecting the copies of other threads.

[0048] Log4j is an Apache open source project and a very popular logging framework in the Java language. It helps developers easily record application log information, including error messages, warning messages, debugging information, etc.

[0049] Redis is an open source key-value pair storage database written in ANSI C that supports network, memory, and optional persistence. It provides multiple data structures such as: String, list, set, zset, and hash.

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0051] Example 1

[0052] The present invention provides a real-time log tracking and positioning method, comprising:

[0053] Step 1: Configure the log debugging user configuration information:

[0054] Step 11: Combine Figure 2 , configure logging on and off and store the configuration items in the database. Logging can be enabled or disabled without restarting the service application; the debugging user simply needs to log out and log back in. The configuration items are stored in the database, and when the user logs in, the system reads the configuration and stores it in the user's session.

[0055] Step 12: Configure the log monitoring path and store it in the database. Figure 3 Configure the program path to monitor. This can be a package, such as com.example.order, a class, such as com.example.order.OrderService, or a method, such as com.example.order.OrderService.genOrder. Multiple paths are separated by semicolons. When a user logs in, the system checks whether the user is a debugging user. If so, the log monitoring path is retrieved from the database. Similar to the logging switch configuration, the read log monitoring path is also stored in the user session.

[0056] Step 13: Configure the configuration items related to the Redis connection and store them in the database. Logs are written to Redis for storage because Redis is an in-memory database, which is fast, has time-sensitive values, and is easy to manage. Redis configuration also needs to be maintained in the database, and the configuration items will be stored in the user session. Redis configuration items mainly include the following three items:

[0057] log.redis.url: the address for logging into redis,

[0058] log.redis.password: The password required to log in to redis, empty if none.

[0059] log.redis.port: redis port.

[0060] Step 2: Use Spring AOP to create a slice monitoring log output class, verify whether the current logged-in user is a log debugging user through the slice, and if so, match the logs whose user's log switch is turned on and the log monitoring path is configured. Figure 3 In the log recording process, the log is copied and output to redis.

[0061] The corresponding Redis key can be composed of ${userId}_${class name}_${method name}_${log line number}, where ${userId} is the user's unique identifier, ${class name} is the class name of the service, ${method name} is the method name of the service, and ${log line number} is the line number corresponding to the log output in the source code file. For example, if Zhang San's unique identifier is 001, the class name of the request service is OrderService, and the method name is genOrder. In the source code, line 45 uses log.debug("Generate Order"); the resulting Redis key is 001_OrderService_genOrder_45, and the corresponding value is "Generate Order". Storing the key in the system thread variable as an array list ensures accurate key matching in Redis and ensures log printing order.

[0062] Step 3: Use the jar package to create a log viewing tool and use the log viewing tool to retrieve relevant logs in Redis.

[0063] A log viewing tool is created using a JAR file to view monitored logs. Place the JAR file in the software project, and the JAR file will automatically start with the system. Once the project is started, it can be accessed. For example, if the web application service domain is www.example.com, the log viewing URL is http(s): / / www.example.com / log / redis / show. The log viewing tool only retrieves relevant logs from Redis based on the key value in the thread variable, preventing the disclosure of irrelevant data. This is secure and reliable, and the display page clearly displays the corresponding log key and value, enabling quick tracking and location of debug logs.

[0064] The method of the present invention locates logs faster, more accurately, quickly and conveniently, improves debugging efficiency and reduces enterprise costs.

[0065] Example 2

[0066] The present invention also provides a real-time log tracking and positioning device, including a configuration module, a monitoring module, a matching module, a storage module and a viewing management module.

[0067] Configuration information of the configuration module configuration log debugging user:

[0068] Step 11: Configure the log opening and closing related configuration items and store them in the database.

[0069] Step 12: Configure the log monitoring path and store it in the database.

[0070] Step 13: Configure the configuration items related to Redis connection and store them in the database;

[0071] The monitoring module is based on the Spring framework and uses AOP to create a slice monitoring log output class. According to the log output class, the slice verifies whether the current logged-in user is a log debugging user. If so, the matching module matches the log of the log debugging user whose log switch is turned on and the log monitoring path is configured. The storage module copies the log and outputs it to redis.

[0072] The viewing management module uses the jar package to create a log viewing tool, and retrieves relevant logs in redis through the log viewing tool.

[0073] Since the information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0074] Similarly, the device of the present invention makes up for the shortcomings of the existing log processing method. Without affecting the log processing method, the operation log configuration can be simply and conveniently performed according to the actual situation; turning on and off the log does not require restarting the service application, and does not affect normal business; the log information is not stored on the log server disk, but is stored in redis, which is efficient and real-time; the log is highly bound to the user, and the log is only turned on for debugging users, and the log is not printed for non-debugging users; it does not increase the server burden and does not require the addition of additional software and hardware.

[0075] It should be noted that not all steps and modules in the above-mentioned processes and device structures are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0076] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A real-time log tracking and positioning method, characterized in that include: Step 1: Configure the log debugging user's configuration information: Step 11: Configure the log opening and closing related configuration items and store them in the database. Step 12: Configure the log monitoring path and store it in the database. Step 13: Configure the configuration items related to redis connection and store them in the database; Step 2: Based on the spring framework, use AOP to create a slice listening log output class. According to the log output class, use the slice to verify whether the current logged-in user is a log debugging user. If yes, match the log of the log debugging user whose log switch is turned on and the log listening path is configured, and copy the log to redis. Step 3: Use the jar package to create a log viewing tool, and use the log viewing tool to retrieve relevant logs in redis.

2. A real-time log tracking and positioning method according to claim 1, characterized in that In step 13, configure the configuration items related to redis connection, including: Configure the address log.redis.url for logging into redis. Configure the password log.redis.password required to log in to redis: leave it blank if none. Configure the redis port log.redis.port.

3. A real-time log tracking and positioning method according to claim 1, characterized in that In step 2, the log is copied and output to redis, including: The key composition corresponding to redis is in the form of ${userId}_${class name}_${method name}_${log line number}, where ${userId} is the user's unique identifier, ${class name} is the class name corresponding to the service, ${method name} is the method name corresponding to the service, and ${log line number} is the line number corresponding to the log output in the source code file. The redis key is stored in the system's thread variable in the form of an array list.

4. A real-time log tracking and positioning method according to claim 3, characterized in that In step 3, use the jar package to create a log viewing tool, and use the log viewing tool to retrieve relevant logs in redis according to the key value in the thread variable.

5. A real-time log tracking and positioning device, characterized in that It includes configuration module, monitoring module, matching module, storage module and viewing management module. Configuration information of the configuration module configuration log debugging user: Step 11: Configure the log opening and closing related configuration items and store them in the database. Step 12: Configure the log monitoring path in the database. Step 13: Configure and store the configuration items related to redis connection in the database; The monitoring module is based on the spring framework and uses AOP to create a slice monitoring log output class. According to the log output class, it verifies whether the current logged-in user is a log debugging user through slicing. If so, the matching module matches the log of the log debugging user whose log switch is turned on and the log monitoring path is configured, and copies the log to redis. The viewing management module uses the jar package to create a log viewing tool, and retrieves relevant logs in redis through the log viewing tool.

6. A real-time log tracking and positioning device according to claim 5, characterized in that The configuration module configures the configuration items related to redis connection, including: Configure the address log.redis.url for logging into redis. Configure the password log.redis.password required to log in to redis: leave it blank if none. Configure the redis port log.redis.port.

7. A real-time log tracking and positioning device according to claim 5, characterized in that The storage module copies and outputs the log to redis, including: The key corresponding to redis is composed into the form of ${userId}_${class name}_${method name}_${log line number}, where ${userId} is the user's unique identifier, ${class name} is the class name corresponding to the service, ${method name} is the method name corresponding to the service, and ${log line number} is the line number corresponding to the log output in the source code file. The redis key is stored in the system's thread variable in the form of an array list.

8. A real-time log tracking and positioning device according to claim 7, characterized in that The management module uses the jar package to create a log viewing tool, and uses the log viewing tool to retrieve relevant logs in redis according to the key value in the thread variable.

Images