Application monitoring data collection method and apparatus, electronic device, and storage medium

Abstract

The application provides an application monitoring data collection method and device, electronic equipment and a storage medium, wherein the application monitoring data collection method comprises the following steps: providing interface code for manual customized burying points in a business application; dynamically weaving the interface code into a third-party component; issuing at least one of the following instructions: burying point opening, burying point closing, burying point adding, burying point intercepting, and adjusting burying point sampling rate; periodically acquiring the instructions, and executing corresponding instructions on corresponding burying points.

Description

TECHNICAL FIELD

[0001] The present document relates to, but is not limited to, the field of application monitoring. BACKGROUND

[0002] In the field of application monitoring, it is often necessary to perform a point-injection on a business interface code, a framework execution process, to count the execution time, whether an exception occurs during the process, the state during the process, a specific event, etc. The application performance is counted, the 99 line is calculated, the success and failure rate is calculated, and the application is monitored and alarmed.

[0003] There are several common point-injection schemes, but each has its own shortcomings:

[0004] 1. CAT

[0005] Supports flexible point-injection by the business party using the cat client.

[0006] Cannot point-inject third-party components, and cannot dynamically switch some point-injections.

[0007] 2. Skywalking

[0008] Supports common third-party component point-injection collection through the Java agent.

[0009] Does not support flexible point-injection by the business party, and does not support dynamic switching of probes.

[0010] 3. ELK

[0011] Outputs logs at the point-injection location using logs, and collects, aggregates, and displays them through elk.

[0012] Also does not support third-party components and dynamic switching.

[0013] Generally speaking, intrusive point-injection can usually only be performed in the framework provided interceptor extension or in the self-owned code. It is almost impossible to extend point-injection for many third-party components that are finally delivered as dependencies. Although Java agent can theoretically perform aspect point-injection on any Java method, it is difficult for ordinary business parties to use extended point-injection due to high development costs. And once the cut-in is permanent, it will cause performance problems: for example, it cannot be dynamically switched, and excessive point-injection will affect performance. Usually, various point-injection components rely on very complex implementations and require various environment supports, such as using a unified client SDK for point-injection, which necessarily requires an SDK collection environment as support, which is not convenient for local development. SUMMARY

[0014] The following presents a summary of various exemplary technical solutions. In the following summary, some simplifications and omissions can be made in the interest of brevity and conciseness and these should not be interpreted as limiting the scope of the inventive concepts. A detailed description is set forth in the subsequent section for enabling one of ordinary skill in the art to make and use the inventive concepts presented herein.

[0015] To solve the above technical problems, the technical solutions of the present application provide an application monitoring data collection method, characterized in that it comprises: providing interface code for manually customizing a probe in a business application; dynamically weaving the interface code into a third-party component; issuing at least one of the following instructions: probe opening, probe closing, probe adding, probe intercepting, and adjusting a probe sampling rate; periodically acquiring the instructions and executing corresponding instructions on corresponding probes.

[0016] Preferably, the method further comprises: recording the issued instructions.

[0017] Preferably, the method further comprises: reporting a probe event model.

[0018] Preferably, the method further comprises: processing probe reporting data to form corresponding reports, links, or alarms.

[0019] Another technical solution of the present application provides an application monitoring data collection device, comprising: a probe weaving module comprising a client SDK and a Java agent, the client SDK being configured to provide interface code for manually customizing a probe in a business application, and the Java agent being configured to dynamically weave the interface code provided by the client SDK into a third-party component; a probe management module configured to issue at least one of the following instructions to a configuration service module: probe opening, probe closing, probe adding, probe intercepting, and adjusting an intercepting probe sampling rate; the Java agent is further configured to periodically acquire the instructions and execute corresponding instructions on corresponding probes.

[0020] Preferably, the device further comprises a configuration service module configured to record the instructions issued by the probe management module.

[0021] Preferably, the probe weaving module further comprises a probe reporting component configured to report a probe event model.

[0022] Preferably, the device further comprises an APM consumption module for processing the data reported by the probe weaving module to form corresponding reports, links, or alarms.

[0023] Another technical solution of the present application also provides an electronic device, comprising a processor, a memory and a computer program stored in the memory and run on the memory, wherein the processor implements the steps of the method according to any one of the above technical solutions when running the computer program.

[0024] Another technical solution of the present application also provides a computer readable storage medium, wherein the computer program is executed by the processor to implement the steps of the method according to any one of the above technical solutions.

[0025] The technical solution of the present application mainly has the following two aspects of beneficial effects:

[0026] First, scalability: some third-party components do not support extension, but the present application can support extension.

[0027] Second, dynamic control: the corresponding trace point can be dynamically switched, dynamically added and dynamically intercepted, so that the performance loss can be saved.

[0028] Third, flexibility and lightness: the business party can flexibly meet the trace point requirement, and does not need too many environment dependencies. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to better understand various exemplary embodiments, reference can be made to the accompanying drawings, in which:

[0030] Figure 1 A flowchart of an application monitoring data collection method provided by the embodiments is shown;

[0031] Figure 2 A schematic diagram of the execution process of step S104 in the application monitoring data collection method provided by the embodiments is shown;

[0032] Figure 3 A schematic diagram of the loading process of the Java agent provided by the embodiments is shown;

[0033] Figure 4 A schematic diagram of the module structure related to the Java agent provided by the embodiments is shown;

[0034] Figure 5 A schematic diagram of the structure of the application monitoring data collection device provided by the embodiments is shown.

[0035] For the convenience of understanding, the same reference signs have been used to refer to elements having substantially the same or similar structure and / or substantially the same or similar function. DETAILED DESCRIPTION

[0036] The description and figures illustrate the principles of the invention. Therefore, it will be understood that those skilled in the art will be able to design various arrangements, which, although not explicitly described or shown herein, embody the principles of the invention and are included within its scope. Furthermore, all examples cited herein are primarily intended for illustrative purposes to aid the reader in understanding the principles of the invention and the concepts provided by the inventors to deepen understanding in the field, and all examples should be considered as not being limited to such specific cited examples and conditions. Additionally, as used herein, unless otherwise indicated (e.g., “or additionally” or “or in an alternative”), the term “or” means non-exclusive or (i.e., and / or). Moreover, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

[0037] Definitions:

[0038] SDK: Software Development Kit. Literally, it means software development kit. In practical use, it refers to a software toolkit provided to clients. In Java, it is a JAR file that assembles software.

[0039] Java Agent: Java Agent technology was introduced in JDK 1.5. A Java Agent is an interceptor that runs before the main method. Typically, we use Java Agent and ASM bytecode technology to dynamically modify the loaded class file using ASM when the JVM loads the class binary. This facilitates adding timers before and after the monitored methods to calculate and monitor method execution time.

[0040] The first embodiment provides an application monitoring data acquisition method, see [link to relevant documentation]. Figure 1 The method includes:

[0041] Step S101: Provide interface code for manually customizing event tracking points in the business application;

[0042] Step S102: Dynamically weave the interface code into the third-party component;

[0043] A toolkit for manually customizing event tracking can be provided to business users via an event tracking SDK, including various event models and standard API interface code. Specific application programming interface (API) code is provided for users to manually track events, assembling specific event models through event tracking, or using SPAN models corresponding to the execution duration of API methods. Extended reporting component capabilities are provided through Service Provider Interface (SPI) technology. A client-side event tracking SDK allows business users to perform flexible event tracking.

[0044] The specific burying point event model reporting process can be responsible for a burying point reporting component, including memory queue caching, serialization, TCP connection maintenance, configuration dynamic acquisition and other functions.

[0045] The Java agent can be used to weave the API interface code provided by the client SDK into the specific method logic in the third-party component to obtain the burying point information. The related code weaving capability is used to weave the application program interface code (API) corresponding to the burying point SDK into the third-party component, and is also used to collect corresponding event, time consumption and other data. The Java agent guarantees the burying point of the third-party component. The third-party component is, for example, an open source binary component.

[0046] Step S103: issuing instructions of at least one of the following: burying point opening, burying point closing, burying point adding, burying point intercepting, adjusting burying point sampling rate;

[0047] The configuration management service can be used to send instructions to realize dynamic opening and closing of burying points, adding and intercepting, and also to adjust the sampling rate of the burying points.

[0048] Preferably, the method further comprises: recording the instructions.

[0049] Step S104: periodically obtaining the instructions and executing corresponding instructions on corresponding burying points.

[0050] The burying point SDK can be used to obtain the instructions. The periodicity here can be selected according to different needs, which can be one minute, one hour, one day, one week and the like.

[0051] Because excessive burying points will affect performance, part of the burying points in daily applications are only turned on in the need to troubleshoot problems or specific applications, so the dynamic switch burying point function of the embodiment is needed, and functions such as dynamic adding of burying points and dynamic intercepting of burying points can also be realized.

[0052] In the actual burying point switch process, only step S103 needs to be operated, and step S104 can be executed periodically. The step S104 execution process is shown in Figure 2 . Figure 2 The probe shown in the figure is a burying point, and Polaris is Java. The Polaris probe (Java burying point) configuration activities include the probe listener, Polaris configuration management and burying point SDK synchronization tasks.

[0053] At the beginning, the probe initializes, instantiates the Polaris configuration, reads the Polaris configuration, judges whether the start parameter is started, if yes, the configuration is overwritten, and then the configuration class is created, if not, the configuration class is directly created; the SDK initialization, the customer key start, the configuration synchronization task is started, polling for 3 minutes (other time can also be set), reading ServerConfig, checking changes, if the configuration changes, updating Client Cache, reading PolarisConfig, updating Polaris configuration; the probe operation, reading configuration, reading PolarisConfig. The above method realizes the timing polling execution of step S104.

[0054] Preferably, the method of the embodiment further comprises the step of: processing the embedded point reporting data to form corresponding reports, links, or alarms.

[0055] The specific loading process diagram of the Java agent can be as shown in Figure 3

[0056] 1. Load agent

[0057] 2. Exist java agent load agent

[0058] 2.1 Premain initializes agent context

[0059] 2.2 Normally load class

[0060] 2.3 Others

[0061] 3. Judge whether to contain enhanced class (including embedded point opening, embedded point closing, embedded point adding, embedded point intercepting, adjusting embedded point sampling rate, etc.)

[0062] 4. Transformer byte code enhances class

[0063] 5. Stub embedded point logic

[0064] 5.1 Embedded point logic trace and metric (byte code dynamic weaving enhancement)

[0065] 6. Agent loading is completed

[0066] 6.1 Check whether to contain self-embedded point dependence (including SDK provided to business for manual custom embedded point)

[0067] 6.2 Spring-Cat-Start

[0068] 6.3 Others

[0069] 7. Execute self-embedded point logic (i.e. flexible embedded point embedded by business team) ​

[0070] The various modules related to the Java agent can be as shown in Figure 4

[0071] Java agent API (application interface code), Java agent common, Java agent Core, Spring-boot-Cat-Start, Java-agent, Java agent Mybatis Plugin, and Java agent Cache Plugin.

[0072] The business team is provided with self-burial points and business indicators by Spring-boot-Cat-Start. The Java agent collects commonly used middleware indicators through bytecode enhancement in a non-invasive manner. In addition, the commonly used middleware architecture group can encapsulate the plug-in burial points as much as possible.

[0073] The second embodiment provides an application monitoring data collection device 200, which is shown in Figure 5 , and includes a burial point weaving module 201 and a burial point management module 202.

[0074] The burial point weaving module 201 includes a client SDK 2011 and a Java agent 2012. The client SDK is configured to provide interface code for manual custom burial points in a business application. The Java agent 2012 is configured to dynamically weave the interface code provided by the client SDK into a third-party component.

[0075] The burial point SDK 2011 can provide a tool set for manual custom burial points to the business side, including various event models and API standard interface code. Specific application program interface code (API) is provided to users for manual burial points. Specific burial point event models are assembled through the interface code, or SPAN models corresponding to the execution time of the interface code method. The specific reporting component capability is provided through the service providing interface code (SPI) technology.

[0076] The burial point weaving module 201 can also include a burial point reporting component configured to report burial point event models. The burial point reporting component is responsible for the specific burial point event model reporting process, including memory queue caching, serialization, TCP connection maintenance, configuration dynamic acquisition, and other functions.

[0077] ​The API interface code provided by the client-side tracking SDK 2011 can be woven into the specific method logic in the third-party component through the Java agent 2012 to obtain tracking information. The relevant code weaving capability is used to weave the application program interface code (API) corresponding to the tracking SDK 2011 into the third-party component, and is also used to collect corresponding event, time consumption and other data. The client-side tracking SDK 2011 can be provided to the business party for flexible tracking. The Java agent 2012 guarantees the tracking of the third-party component. The third-party component is, for example, an open-source binary component.

[0078] The tracking management module 202 is configured to issue at least one of the following instructions to the configuration service module: tracking on, tracking off, tracking addition, tracking interception, and adjusting the tracking sampling rate.

[0079] The sending of the instructions can achieve dynamic tracking on and off, addition of interception, and adjustment of the tracking sampling rate.

[0080] Preferably, the system further comprises a configuration service module 203 configured to record the instructions issued by the probe management module.

[0081] The Java agent 2012 is further configured to periodically obtain the instructions and execute the corresponding instructions on the corresponding tracking.

[0082] The periodicity here can also be selected according to different needs, which can be one minute, one hour, one day, one week, etc.

[0083] Because excessive tracking can affect performance, part of the tracking in daily applications is only turned on when problems need to be investigated or in specific applications, so the dynamic on and off tracking function of the embodiment is needed, and functions such as dynamic addition of tracking and dynamic interception of tracking can also be achieved.

[0084] Preferably, the system further comprises an APM consumption module 204 for processing the data reported by the tracking weaving module to form corresponding reports, links, or alarms.

[0085] The third embodiment further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor implements the steps of the method of any one of the above embodiments, such as steps S101-S104, or the processor implements the functions of the modules / units in the above embodiments, such as the functions of the units 201-204, when executing the computer program. The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0086] The electronic device can be a mobile terminal such as a smartphone, or a computing device such as a desktop computer, a notebook computer, a palm computer, and a cloud server. The electronic device can include, but is not limited to, a processor and a memory, and can include more or fewer components, or combine some components, for example, the electronic device can further include an input / output device, a network access device, a bus, and the like. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The memory can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory can include both the internal storage unit and the external storage device of the electronic device.

[0087] The fifth embodiment further provides a computer-readable storage medium, wherein the computer program is executable by the processor to implement the steps of the method of any one of the above embodiments.

[0088] The various function units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of a software function unit. When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, all or part of the flow of the method in the above-described embodiments can be implemented by a computer program, and the computer program can instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier wave signal and telecommunication signal.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above function units and modules is taken as an example, and in actual application, the above functions can be completed by different function units and modules according to needs, that is, the internal structure of the device is divided into different function units or modules to complete all or part of the functions described above. The function units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software function unit. In addition, the specific names of the function units and modules are only for convenient distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0090] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments. Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0091] In the embodiments provided in the present application, it should be understood that the disclosed system, electronic device and method can be implemented in other ways. For example, the above-described system, electronic device embodiments are merely illustrative. For example, the division of the modules or units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface codes, and can be electrical, mechanical or other forms. The units illustrated as separate components can or can not be physically separate, and the components illustrated as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0092] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones. Such modifications or replacements do not change the essence of the corresponding technical solutions, and should be included in the protection scope of the present application.

Claims

1. A method for collecting application monitoring data, characterized in that, include: Provide interface code for manually customizing event tracking in business applications; The interface code is dynamically woven into third-party components, and specific event models are assembled through instrumentation, or the SPAN model corresponding to the execution duration of the interface code method is used. The service provides interface code technology to extend the specific reporting component capabilities. Issue at least one of the following commands: enable tracking, disable tracking, add tracking, intercept tracking, adjust tracking sampling rate, to achieve dynamic enabling and disabling of tracking, adding and intercepting tracking, and adjusting the sampling rate of tracking. The instructions are periodically retrieved, and the corresponding instructions are executed at the corresponding embedded points.

2. The application monitoring data acquisition method according to claim 1, characterized in that, Also includes: Record the issued instructions.

3. The application monitoring data acquisition method according to claim 1, characterized in that, Also includes: Report the event tracking model.

4. The application monitoring data acquisition method according to claim 3, characterized in that, Also includes: Process the data reported by the tracking points to generate corresponding reports, tracking information, or alarms.

5. An application monitoring data acquisition device, characterized in that, include: The event tracking weaving module includes a client SDK and a Java agent. The client SDK is configured to provide interface code for manually customizing event tracking in business applications. The Java agent is configured to dynamically weave the interface code provided by the client SDK into third-party components. It assembles specific event tracking models through event tracking, or provides SPAN models corresponding to the execution duration of interface code methods. It also provides the ability to extend specific reporting components through service-provided interface code technology. The event tracking management module is configured to send at least one of the following commands to the configuration service module: event tracking enable, event tracking disable, event tracking add, event tracking intercept, and adjust the sampling rate of intercepted event tracking, thereby enabling dynamic enabling and disabling of event tracking, adding intercepted events, and adjusting the sampling rate of event tracking. The Java agent is also configured to periodically retrieve the instructions and execute the corresponding instructions at the corresponding data points.

6. The application monitoring data acquisition device according to claim 5, characterized in that, It also includes a configuration service module, which is configured to record instructions issued by the probe management module.

7. The application monitoring data acquisition device according to claim 5, characterized in that, The event tracking module also includes an event tracking reporting component, which is configured to report event tracking models.

8. The application monitoring data acquisition device according to claim 5, characterized in that, It also includes an APM consumption module: used to process the data reported by the data collection module and generate corresponding reports, links, or alarms.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory that runs on the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, When a computer program is executed by a processor, it implements the method of any one of claims 1-4.

Images