Non-invasive dynamic burying system for application, implementation method and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN VIPSHOP FUBON CONSUMER FINANCE CO LTD
- Filing Date
- 2022-07-28
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies cannot dynamically change the behavior of tracking points during code execution, thus failing to meet the requirement of non-intrusive tracking.
The system employs a transformer module, a tracking processor module, a bean factory module, and a startup class definition module, combined with HotSwap, to dynamically change tracking behavior. By defining bytecode transformers and tracking processors, and utilizing the factory pattern to decouple processor logic, the tracking strategy can be dynamically adjusted.
It enables flexible and seamless changes to event tracking behavior during code execution, supports event tracking optimization for a wide range of application frameworks and tool libraries, dynamically adjusts event tracking logic, and improves the flexibility and scalability of event tracking.
Smart Images
Figure CN115454563B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software development technology, specifically to a non-intrusive dynamic tracking system for applications, its implementation method, and its storage medium. Background Technology
[0002] Due to the huge demand for bytecode modification capabilities, the JDK introduced the java.lang.instrument package starting with JDK 5. It allows setting a ClassFileTransformer using the addTransformer method, which can then be used to perform class conversions.
[0003] JDK 1.5 supports static instrumentation. The basic idea is to add a proxy (Java agent) at JVM startup. Each proxy is a JAR file, and its MANIFEST.MF file specifies the proxy class, which contains a `premain` method. When the JVM loads a class, it first executes the `premain` method of the proxy class, then executes the Java program's `main` method. The `premain` method allows modification of the class file before loading. This mechanism can be considered virtual machine-level AOP, enabling dynamic modification and enhancement of classes without requiring any changes to the existing application. Starting with JDK 1.6, more powerful dynamic instruments are supported, loaded remotely via the Attach API after JVM startup.
[0004] However, both static and dynamic code enhancement require predefined pointcuts, similar to AOP pointcuts. "Static" and "dynamic" refer to the timing of the code enhancement—whether it occurs before or after the main function is executed. After enhancement, the code behavior remains unchanged; it cannot be changed during execution.
[0005] For non-intrusive event tracking requirements, the most common requirement is the ability to change the event tracking log level during execution. Therefore, how to enhance code functionality and change event tracking behavior during code execution is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] To address the above problems, this invention provides a non-intrusive dynamic data tracking system, implementation method, and storage medium for applications.
[0007] The adopted technical solution is a non-intrusive dynamic data tracking system for applications, including a converter module, a data tracking processor module, a bean factory module, a data tracking data processing module, and a startup class definition module.
[0008] The transformer module defines a bytecode transformer for executing the tracking strategy, weaving factory design pattern code into methods that conform to the strategy, and constructing a tracking processor module instance by the bean factory module.
[0009] The event tracking processor module contains predefined event tracking logic;
[0010] The bean factory module injects processor logic into the converter module at the matching position, thereby decoupling it from the specific implementation of the processor and then combining it with hotswap to dynamically change the behavior of the tracking point.
[0011] The data processing module is used to process data generated by different data processing processors;
[0012] The startup class definition module defines a startup class for loading the Java agent for the event tracking application, starts DCEVM, and creates a startup class implementation method for the startup class.
[0013] Furthermore, the converter module categorizes each method signature and reports unknown method signatures to the configuration server to achieve the purpose of dynamic instrumentation.
[0014] Optionally, the startup class implementation method in the startup class definition module includes the PID of the tracking application, the path of the Java agent, and the URL of the configuration server. The PID is the process ID of the tracking application used by the JVM to determine the target process for loading the Java agent. The path of the Java agent is the file path of the agent jar file used by the startup class to read the jar file. The URL is the configuration server used by the startup class to download bytecode from the given address.
[0015] This invention also provides a supporting method, a method for non-intrusive dynamic data embedding in applications, comprising:
[0016] S1. Build a server to provide configuration for event tracking strategies;
[0017] S2, predefined multiple event handlers;
[0018] S3. Define a Java agent;
[0019] S4. Construct a bytecode transformer and inject the logic that obtains the processor from the bean factory module and calls it;
[0020] S5. Define a startup class based on a Java agent and implement the agent main method in the startup class;
[0021] S6 and the agent interact with the configuration server via gRPC, reporting method qualified names and continuously building new Bloom filters and blacklists / whitelists.
[0022] S7. The agent dynamically detaches from the tracking application using the detach method and then reattaches to the application without affecting the application's operation, and then ends.
[0023] Optionally, in S1, the constructed server provides functions such as configuring method signature blacklists and whitelists, collecting method signatures, constructing method signature Bloom filters, and managing the corresponding bytecode.
[0024] Optionally, in S3, the bytecode of the Bloom filter and method blacklist / whitelist is loaded remotely, hotswap is implemented through DCEVM, the bytecode is hot-deployed, methods are classified, unknown method signatures are reported to the configuration server, new Bloom filters and blacklists / whitelists are constructed on the configuration server, and transformers are applied to methods in the whitelist.
[0025] Optionally, in S5, the target application for event tracking is started first, then the startup class is run and the process ID of the application is provided. The agent is injected into the target JVM, DCEVM is started, and then the bytecode transformation code is executed.
[0026] Optionally, in S6, the agent can also acquire new bytecode and then perform hot deployment.
[0027] Optionally, between S6 and S7, when the enhanced target method is called, the predefined instrumentation processors referenced in the execution strategy are executed, such as reporting OpenTracing specification flag data, and then the method call chain is displayed using an OpenTracing specification-compatible display server.
[0028] The present invention also provides a matching computer storage medium, wherein the computer storage medium stores a computer program that can drive a non-intrusive dynamic tracking system for the application, and when the computer program is executed by the processor, it can implement a non-intrusive dynamic tracking method for the application.
[0029] The beneficial effects of the present invention include at least one of the following;
[0030] 1. By defining a startup class for the event tracking application, this startup class connects to the process after the event tracking application starts and loads the transformer module encapsulated as a Java agent. Compared with existing annotation-based intrusive event tracking solutions and static instrumentation-based event tracking solutions, this invention is effective for any JVM application, while also taking into account targeted event tracking optimizations for common frameworks and utility libraries, as well as progressive event tracking processing for specific scenarios.
[0031] 2. For applications that require adding tracking logic, simply categorize methods into blacklists and whitelists on the server side. The categorization process generally targets the application's own methods; for widely used libraries, existing categorizations can usually be used. When an application needs to change or remove tracking logic, it only needs to change the blacklist / whitelist categorization and the processor associated with the whitelist, and recalculate the Bloom filter parameters on the server side based on the blacklist. This is not only simple to operate but also flexible in tracking, and the application is unaware of the tracking.
[0032] 3. This application utilizes the Factory pattern design pattern, similar to the core implementation of Spring IOC, to achieve the invocation of different event tracking logic. Adding new event tracking logic only requires adding a new handler implementation; the invocation logic remains unchanged. This allows the transformer to inject the same code while invoking different command implementations, achieving dynamic behavior transformation and easy extensibility. Attached Figure Description
[0033] Figure 1 A block diagram of a non-intrusive dynamic data tracking system for applications;
[0034] Figure 2 This is a flowchart illustrating the method for implementing non-intrusive dynamic data tracking in applications. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the scope of protection of this invention.
[0036] In the description of this invention, it should be noted that the terms "upper," "lower," etc., used to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention; the terms "first," "second," etc., used may be used for descriptive purposes only and should not be construed as indicating or implying relative importance. In addition, unless otherwise explicitly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.
[0037] Example 1
[0038] like Figure 1As shown, a non-intrusive dynamic data tracking system for applications includes a converter module, a data tracking processor module, a bean factory module, a data tracking data processing module, and a startup class definition module.
[0039] The transformer module defines a bytecode transformer for executing the tracking strategy, weaving factory design pattern code into methods that conform to the strategy, and constructing a tracking processor module instance by the bean factory module.
[0040] The event tracking processor module contains predefined event tracking logic;
[0041] The bean factory module injects processor logic into the converter module at the matching position, thereby decoupling it from the specific implementation of the processor and then combining it with hotswap to dynamically change the behavior of the tracking point.
[0042] The data processing module is used to process data generated by different data processing processors;
[0043] The startup class definition module defines a startup class for loading the Java agent for the event tracking application, starts DCEVM, and creates a startup class implementation method for the startup class.
[0044] The purpose of this implementation is to define a startup class for the event tracking application. This startup class connects to the process after the event tracking application starts and loads the transformer module encapsulated as a Java agent. Compared with existing annotation-based intrusive event tracking solutions and static instrumentation-based event tracking solutions, this invention is effective for any JVM application, while also taking into account targeted event tracking optimizations for commonly used frameworks and utility libraries, as well as progressive event tracking processing for specific scenarios. Furthermore, for applications that need to add event tracking logic, it is only necessary to classify methods into blacklists and whitelists on the configuration server side; the classification process generally focuses on the application's own methods, and for widely used libraries, existing classifications can generally be used. When the application needs to change or remove event tracking logic, it only needs to change the blacklist / whitelist classification and the processor associated with the whitelist, and recalculate the Bloom filter parameters on the server side based on the blacklist. This is not only simple to operate but also flexible in event tracking, and the application is unaware of the event tracking. This application utilizes the factory pattern of design patterns, similar to the core implementation of Spring IOC, to achieve the invocation of different event tracking logics. Adding new tracking logic only requires adding a new processor implementation, while the calling logic remains unchanged. This allows the transformer to inject the same code and call different commands, achieving dynamic behavior transformation and easy extensibility. It solves the problem of how to enhance code functionality and change tracking behavior during code execution.
[0045] Meanwhile, in this embodiment, the converter module can also classify each method signature and report unknown method signatures to the configuration server to achieve the purpose of dynamic data entry; the workflow design of collection-configuration-release takes into account both performance and flexibility.
[0046] As for the event tracking processor module, based on the predefined event tracking logic, it can log events such as method entry, exit, and exception throwing, report Open Tracing specification data, and report metrics data.
[0047] The Bean Factory module functions similarly to the factory pattern implementation in Spring IOC. The transformer injects logic that retrieves processors from the bean factory at matching locations, achieving decoupling from the specific processor implementation. This, combined with HotSwap, enables dynamic changes to the tracking behavior.
[0048] As for the event tracking data processing module, it processes the data generated by different event tracking processors. For example, for OpenTracing data, it can display performance, method call paths, etc.
[0049] The startup class definition module defines a startup class for the event tracking application to load the Java agent and simultaneously start DCEVM; and creates an implementation method for this startup class. This implementation method has three parameters: the process ID (PID) of the event tracking application, the path to the Java agent, and the configuration server URL. The PID is the process ID of the event tracking application, used by the JVM to determine the target process for loading the Java agent; the Java agent path is the file path of the agent JAR file, used by the startup class to read the JAR file; and the configuration server URL is used by the startup class to download bytecode from the given address.
[0050] Example 2
[0051] A method for implementing non-intrusive dynamic event tracking in applications includes:
[0052] S1. Build a server to provide configuration for event tracking strategies;
[0053] S2, predefined multiple event handlers;
[0054] S3. Define a Java agent;
[0055] S4. Construct a bytecode transformer and inject the logic that obtains the processor from the bean factory module and calls it;
[0056] S5. Define a startup class based on a Java agent and implement the agent main method in the startup class;
[0057] S6 and the agent interact with the configuration server via gRPC, reporting method qualified names and continuously building new Bloom filters and blacklists / whitelists.
[0058] S7. The agent dynamically detaches from the tracking application using the detach method and then reattaches to the application without affecting the application's operation, and then ends.
[0059] In practical implementation, the server built in S1 provides functions such as configuring method signature blacklists and whitelists, collecting method signatures, building method signature Bloom filters, and managing the corresponding bytecode.
[0060] In S2, multiple data tracking processors are predefined, such as OpenTracing specification data reporting processors and log tracking processors, and compiled into bytecode and uploaded to the management server.
[0061] In S3, a Java agent is defined to remotely load the bytecode of the Bloom filter and method blacklist / whitelist. Hotswap is implemented through DCEVM to hot-deploy the bytecode, classify methods, report the signatures of unknown methods to the configuration server, and construct new Bloom filters and blacklists / whitelists on the configuration server. Transformers are applied to methods in the whitelist. The purpose of this design is to avoid pre-defining method instrumentation in the agent, but to hot-deploy new processors when loading bytecode.
[0062] In S4, a bytecode transformer is built, and logic that obtains and calls the processor from the bean factory module is injected. The purpose of this design is to dynamically change the behavior of the tracking points based on the remotely loaded tracking bean factory implementation and the processor.
[0063] In S5, a startup class based on a Java agent is defined, and the agentmain method is implemented in the startup class. The target tracking application is started first, then the startup class is run and the process ID of the application is provided. The agent is injected into the target JVM, DCEVM is started, and then the bytecode transformation code is executed. The purpose of this design is that the agent is an independent process, does not start with the original application, and does not share the same JVM instance.
[0064] In S6, the Agent interacts with the configuration server via gRPC, reports the method qualified name, and continuously builds new Bloom filters and blacklists / whitelists. The Agent can also obtain new bytecode and then hot-deploy it. The purpose of this design is that, without a preset processor method, the behavior of the event handler can be dynamically adjusted through hot deployment of bytecode.
[0065] In S7, the Agent dynamically detaches from the tracking application and then reattaches to it using the detach method, without affecting application operation. End.
[0066] Meanwhile, before implementing S6, when the enhanced target method is called, the predefined instrumentation processors referenced in the execution strategy are used, such as reporting OpenTracing specification flag data. Then, an OpenTracing-compatible display server is used to display the method call chain, thereby realizing real-time dynamic display of the application method call chain, discovering hot execution paths, and revealing the code execution status of complex systems.
[0067] This allows for dynamic switching of predefined behaviors during application runtime through hot deployment of bytecode, meeting actual data tracking needs. In particular, it can optimize widely used Spring MVC framework applications, collecting data such as request and return codes of endpoints on the controller to generate metrics and alerts. For method tracking in general applications, it can also generate call graphs, thereby enabling hotspot analysis and change analysis.
[0068] Example 3
[0069] This application aims to design a solution for implementing non-intrusive dynamic event tracking. "Non-intrusive" means that when an application uses a third-party framework or component, it does not need to inherit the entity classes provided by the framework or component, implement the interfaces provided by the framework, or inherit the project scope of the third-party framework or component. In other words, the application can still run normally even when the dependent third-party framework or component is lost; in this case, the third-party framework or component is considered non-intrusive to the application. "Dynamic" refers to the property that the application can change its behavior after startup and during runtime without needing to be restarted; this is called the ability to dynamically change.
[0070] like Figure 2 As shown, this embodiment of the invention provides a method for non-intrusive dynamic data tracking in applications, which includes the following steps:
[0071] S1 is a web app that displays collected method signatures and a list of event handlers, defines blacklists and whitelists of methods, sets matching rules and associates event handlers, and calculates parameters for a blacklist Bloom filter.
[0072] S2 defines event tracker processors. Each event tracker processor is essentially a module that accepts a Map.<String,Object> A single-method interface. `public interface Processor{void do(Map`<String,Object> args)}
[0073] S3 defines a ClassFileTransformer implementation class, which implements the public byte[]transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ...).
[0074] The `ProtectionDomain` class, `byte[]bytes)throwsIllegalClassFormatException`, and the `public static void agentmain(String agentArgs,Instrumentation inst)throwsClassNotFoundException,UnmodifiableClassException` method are defined. In `agentmain`, a `Transformer` is loaded and initialized. Within the `Transformer`, the `transform` method is implemented. Methods are categorized using blacklists / whitelists, matching regulations, and Bloom filters. An `agent` class is defined, implementing the `public static void premain(String agentArgs,Instrumentation inst)throwsClassNotFoundException,UnmodifiableClassException` method. This method downloads bytecode from a specified URL, performs a hotswap using DCEVM, and then loads a `ClassFileTransformer` object through an `Instrumentation` object.
[0075] S4 reports unknown methods.
[0076] S5 ignores blacklisted methods and injects logic to obtain the processor through the bean factory into whitelisted methods. Processor p = beanFactory.getProcessor(FQN method name).
[0077] S6 execution processor, p.do(new HashMap)<String,Object> ("methodName",FQN methodname)).
[0078] Start the Java application on S7 and obtain the PID.
[0079] java -jar XXX.jar
[0080] jps / / Returns the PID of XXX, such as 10897
[0081] S8 defines a startup class that calls the attach(pid) and load(path) methods of the VirtualMachine class with the passed-in pid and starts DCEVM.
[0082] java-jar launcher.jar 10897path_to_agent_jar url_to_download_bytecode
[0083] Among them, 10897 is the PID of the tracking application obtained through jps, path_to_agent_jar is the path to agent.jar, and url_to_download_bytecode is the download endpoint provided by the configuration server.
[0084] In this embodiment, a computer storage medium is also provided, which stores a computer program. The computer storage medium can be one of magnetic random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, flash memory, magnetic surface memory, and optical disk. It can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, etc. The computer program can drive a non-intrusive dynamic tracking system for applications, and when the computer program is executed by the processor, it can implement a non-intrusive dynamic tracking method for applications.
[0085] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A non-intrusive dynamic data tracking system for applications, characterized in that: It includes a converter module, a data tracking processor module, a bean factory module, a data tracking data processing module, and a startup class definition module; The transformer module defines a bytecode transformer for executing the tracking strategy, weaving factory design pattern code into methods that conform to the strategy, and constructing a tracking processor module instance by the bean factory module; The event tracking processor module contains predefined event tracking logic; The bean factory module injects processor logic into the converter module at the matching position, thereby decoupling the processor's specific implementation and then using hotswap to dynamically change the event tracking behavior. The data processing module is used to process data generated by different data processing processors; The startup class definition module defines a startup class for the event tracking application to load the Java agent, simultaneously starts DCEVM, and creates a startup class implementation method for this startup class. The transformer module classifies each method signature and reports unknown method signatures to the configuration server to achieve dynamic event tracking. The startup class implementation method in the startup class definition module includes the event tracking application's PID, the Java agent's path, and the configuration server's URL. The PID is the event tracking application's process ID used by the JVM to determine the target process for loading the Java agent. The Java agent's path is the file path of the agent JAR file used by the startup class to read the JAR file. The URL is the configuration server address used by the startup class to download bytecode from the given address.
2. A method for non-intrusive dynamic data tracking in applications, characterized in that: The method includes: S1. Build a server to provide configuration for event tracking strategies; S2, predefined multiple event handlers; S3. Define a Java agent; S4. Construct a bytecode transformer and inject the logic that obtains and calls the processor from the bean factory module; S5. Define a startup class based on a Java agent and implement the agent main method in the startup class; S6 and the agent interact with the configuration server via gRPC, reporting method qualified names and continuously building new Bloom filters and blacklists / whitelists; S7: The agent dynamically detaches from the tracking application using the detach method and then reattaches to the application without affecting its operation. Then it ends. In S3, the bytecode for the Bloom filter and method blacklist / whitelist is remotely loaded. Hotswap is implemented through DCEVM to hot-deploy the bytecode. Methods are categorized, and unknown method signatures are reported to the configuration server. New Bloom filters and blacklists / whitelists are constructed on the configuration server. Transformers are applied to methods in the whitelist.
3. The method for implementing non-intrusive dynamic data embedding in an application according to claim 2, characterized in that: In S1, the built server provides functions such as configuring method signature blacklists and whitelists, collecting method signatures, building method signature Bloom filters, and managing the corresponding bytecode.
4. The method for non-intrusive dynamic data embedding in an application according to claim 2, characterized in that: In S5, the target application for event tracking is started first, then the startup class is run and the process ID of the application is provided. The agent is injected into the target JVM, DCEVM is started, and then the bytecode transformation code is executed.
5. A method for non-intrusive dynamic data embedding in an application according to claim 2, characterized in that: In S6, the agent retrieves the new bytecode and then performs hot deployment.
6. The method for non-intrusive dynamic data embedding in an application according to claim 2, characterized in that: Between S6 and S7, when the target method is called, the predefined instrumentation processors referenced in the execution strategy are executed, such as reporting OpenTracing specification flag data, and then using an OpenTracing specification-compatible display server to display the method call chain.
7. A computer storage medium storing a computer program, characterized in that, The computer program can drive the non-intrusive dynamic tracking system for applications as described in claim 1, and when the computer program is executed by the processor, it can implement the non-intrusive dynamic tracking method for applications as described in any one of claims 2 to 6.