Monitoring log processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a monitoring log processing method, apparatus, device and storage medium, which obtains multiple call data generated at each monitoring point when the target application calls a specified method within each cache time period, and stores at least one call information corresponding to the multiple call data in the corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data. At each interval corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application and transmit them to the monitoring device. The monitoring device caches the multiple monitoring logs according to the data storage template preset at each monitoring point, and processes the cached multiple monitoring logs according to the preset analysis frequency, so as to determine the performance index of the target application, which is not affected by the QPS of the target application, thereby improving the performance analysis efficiency and analysis accuracy of the application.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a monitoring log processing method, apparatus, device and storage medium. Background Art

[0002] With the continuous development of distributed applications and cloud computing technology, the logical structure of business systems has become more and more complex, and applications have evolved into a series of services running on different platforms. Application performance is a key indicator that reflects the quality of service provided by applications to customers, and the level of application performance will directly affect the user experience.

[0003] In the prior art, in order to monitor the running performance of an application, it is usually determined based on the performance of a specified method executed by the application. Specifically, each time the application calls the specified method, a monitoring log is generated and synchronously written to the disk log file. The log collection module periodically collects incremental log data from the disk log file and sends it to the monitoring device. Correspondingly, the monitoring device caches and analyzes the received incremental log data to determine the performance data of the specified method, thereby determining the running performance of the application.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art: In actual applications, if the query rate per second (QPS) of a certain application to be monitored is relatively high, the specified method is called many times, and a large number of monitoring logs will be generated in a short time. This will not only increase the refresh frequency of the disk log file and increase the collection pressure of the log collection module, which will seriously affect the normal operation of the application or cause data loss, but also occupy a large amount of cache space of the monitoring device, increase the processing pressure of the monitoring device, and reduce the efficiency and accuracy of performance analysis. In general, the existing performance methods have the problem of low performance analysis efficiency and accuracy. Summary of the invention

[0005] The embodiments of the present application provide a monitoring log processing method, apparatus, device and storage medium to solve the problems of low performance analysis efficiency and accuracy in existing performance methods.

[0006] In a first aspect, an embodiment of the present application provides a monitoring log processing method, including:

[0007] Acquire multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0008] According to the monitoring point identifier and the cache time period identifier carried by each call data, at least one call information corresponding to the multiple call data is stored in the corresponding cache queue in sequence, each call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with the monitoring point identifier and the cache time period identifier;

[0009] At intervals corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application;

[0010] The multiple monitoring logs are sent to the monitoring device.

[0011] In a possible design of the first aspect, storing at least one piece of call information corresponding to the multiple call data in a corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data includes:

[0012] According to the monitoring point identifier and the cache time period identifier carried by each call data, the obtained multiple call data are divided into at least one data set, each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier;

[0013] For each data set, all the call data in the data set are processed according to the call execution time of each call data to determine at least one call information;

[0014] Each call information is sequentially stored in the cache queue corresponding to the data set.

[0015] Optionally, at each interval corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application, including:

[0016] At intervals corresponding to the cache time period, the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue, and the number of call executions of each call execution time are updated to the preset log template to generate multiple monitoring logs for the target application.

[0017] In another possible design of the first aspect, after processing all call information in each cache queue at each interval corresponding to the cache time period to generate multiple monitoring logs for the target application, the method further includes:

[0018] Storing multiple monitoring logs of the target application in a disk log file;

[0019] Obtaining incremental monitoring logs from the disk log file according to a preset collection cycle;

[0020] The sending the plurality of monitoring logs to the monitoring device includes:

[0021] The incremental monitoring log is sent to a monitoring device.

[0022] In a second aspect, an embodiment of the present application provides a monitoring log processing method, including:

[0023] Receive multiple monitoring logs of the target application from the application server;

[0024] According to the data storage template preset at each monitoring point, the plurality of monitoring logs are cached respectively;

[0025] According to the preset analysis frequency, the plurality of cached monitoring logs are processed to determine the performance index of the target application corresponding to the application server.

[0026] In a possible design of the second aspect, caching the plurality of monitoring logs respectively according to a data storage template preset at each monitoring point includes:

[0027] Analyze each monitoring log to determine the monitoring point identifier and cache time period identifier corresponding to each monitoring log;

[0028] According to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, at least one call information corresponding to each monitoring log is determined, each call information including: a call execution time and a call execution count of the call execution time;

[0029] At least one piece of call information corresponding to each monitoring log is cached based on the data storage template corresponding to each monitoring log.

[0030] In a third aspect, an embodiment of the present application provides a monitoring log processing method, including:

[0031] Acquire multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0032] According to the monitoring point identifier and the cache time period identifier carried by each call data, at least one call information corresponding to the multiple call data is stored in the corresponding cache queue in sequence, each call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with the monitoring point identifier and the cache time period identifier;

[0033] At intervals corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application;

[0034] The plurality of monitoring logs are processed according to a preset analysis frequency to determine the performance index of the target application.

[0035] In a fourth aspect, an embodiment of the present application provides a monitoring log processing device, including:

[0036] An acquisition module, used to acquire multiple call data generated at each monitoring point when the target application calls a specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0037] A cache module, for storing at least one call information corresponding to the plurality of call data in a corresponding cache queue in sequence according to a monitoring point identifier and a cache time period identifier carried by each call data, wherein each call information includes: a call execution time and a call execution count of the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier;

[0038] A processing module, configured to process all call information in each cache queue at intervals corresponding to the cache time period, and generate multiple monitoring logs for the target application;

[0039] The sending module is used to send the plurality of monitoring logs to the monitoring device.

[0040] In a possible design of the fourth aspect, the cache module is specifically configured to:

[0041] According to the monitoring point identifier and the cache time period identifier carried by each call data, the obtained multiple call data are divided into at least one data set, each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier;

[0042] For each data set, all the call data in the data set are processed according to the call execution time of each call data to determine at least one call information;

[0043] Each call information is sequentially stored in the cache queue corresponding to the data set.

[0044] Optionally, the processing module is specifically used to update the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue and the number of call execution times of each call execution time into a preset log template every time period corresponding to the cache time period, so as to generate multiple monitoring logs for the target application.

[0045] In another possible design of the fourth aspect, the cache module is further used to process all call information in each cache queue at intervals corresponding to the cache time period of the processing module, generate multiple monitoring logs for the target application, and then store the multiple monitoring logs of the target application in a disk log file;

[0046] The acquisition module is further used to acquire the incremental monitoring log in the disk log file according to a preset acquisition cycle;

[0047] The sending module is specifically used to send the incremental monitoring log to the monitoring device.

[0048] In a fifth aspect, an embodiment of the present application provides a monitoring log processing device, including:

[0049] A receiving module, used for receiving multiple monitoring logs of a target application from an application server;

[0050] A cache module, used to cache the multiple monitoring logs respectively according to the data storage template preset at each monitoring point;

[0051] The processing module is used to process the plurality of cached monitoring logs according to a preset analysis frequency, and determine the performance index of the target application corresponding to the application server.

[0052] In a possible design of the fifth aspect, the cache module is specifically configured to:

[0053] Analyze each monitoring log to determine the monitoring point identifier and cache time period identifier corresponding to each monitoring log;

[0054] According to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, at least one call information corresponding to each monitoring log is determined, each call information including: a call execution time and a call execution count of the call execution time;

[0055] At least one piece of call information corresponding to each monitoring log is cached based on the data storage template corresponding to each monitoring log.

[0056] In a sixth aspect, an embodiment of the present application provides a monitoring log processing device, including:

[0057] An acquisition module, used to acquire multiple call data generated at each monitoring point when the target application calls the designated device within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0058] A cache module, for storing at least one call information corresponding to the plurality of call data in a corresponding cache queue in sequence according to a monitoring point identifier and a cache time period identifier carried by each call data, wherein each call information includes: a call execution time and a call execution count of the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier;

[0059] The processing module is used to process all call information in each cache queue at intervals corresponding to the cache time period, generate multiple monitoring logs for the target application, and process the multiple monitoring logs according to a preset analysis frequency to determine the performance indicators of the target application.

[0060] In the seventh aspect, an embodiment of the present application provides an application server, comprising a processor, a memory, a transceiver, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect and various possible designs is implemented.

[0061] In an eighth aspect, an embodiment of the present application provides a monitoring device, comprising a processor, a memory, a transceiver, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the second aspect above is implemented.

[0062] In a ninth aspect, an embodiment of the present application provides an application server, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the third aspect above is implemented.

[0063] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect and various possible designs; or

[0064] The computer-executable instructions are used to implement the method according to the second aspect when executed by the processor; or

[0065] When the computer-executable instructions are executed by the processor, they are used to implement the method described in the third aspect above.

[0066] The monitoring log processing method, apparatus, device and storage medium provided in the embodiments of the present application obtain multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, and store at least one call information corresponding to the multiple call data in the corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data. At each interval corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application and transmit them to the monitoring device. In this way, the monitoring device can cache the multiple monitoring logs according to the data storage template preset at each monitoring point, and process the cached multiple monitoring logs according to the preset analysis frequency to determine the performance index of the application server corresponding to the target application. The performance index is not affected by the QPS of the target application, thereby improving the performance analysis efficiency and analysis accuracy of the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0068] Figure 1 This is a schematic diagram of an application architecture of the monitoring log processing method provided in an embodiment of the present application;

[0069] Figure 2 This is another application architecture diagram of the monitoring log processing method provided in the embodiment of the present application;

[0070] Figure 3 A flowchart of the monitoring log processing method embodiment 1 provided in this application;

[0071] Figure 4 A flowchart of the second embodiment of the monitoring log processing method provided by the present application;

[0072] Figure 5 An interactive diagram of the third embodiment of the monitoring log processing method provided by the present application;

[0073] Figure 6 A flowchart of a fourth embodiment of the monitoring log processing method provided in this application;

[0074] Figure 7 A flowchart of a fifth embodiment of the monitoring log processing method provided in this application;

[0075] Figure 8 A schematic diagram of the structure of the monitoring log processing device embodiment 1 provided in the present application;

[0076] Fig. 9 A schematic diagram of the structure of the monitoring log processing device embodiment 2 provided in the present application;

[0077] Fig.10 A schematic diagram of the structure of the monitoring log processing device embodiment 3 provided in the present application;

[0078] Fig.11 A schematic diagram of the structure of the first embodiment of the application server provided in the embodiment of the present application;

[0079] Fig.12 A schematic diagram of the structure of a monitoring device embodiment provided in an embodiment of the present application;

[0080] Fig.13 A schematic diagram of the structure of the second application server embodiment provided in the present application embodiment.

[0081] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0082] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0083] At present, with the rapid development of network technology, more and more services are provided by Internet applications. For example, e-commerce websites, financial transaction systems, and social networking website Internet application services are increasingly widely used in people's lives. The performance of application services directly affects the user experience, so how to monitor the operating performance of application services is very critical.

[0084] Normally, in order to monitor the operating performance of an application service, performance monitoring can be performed on important methods called by the application service, and then the performance of the application service can be determined based on the performance of the application method. Currently, a more common and safer method is to obtain the monitoring log generated on the application server side when the target application calls a specified method, and then analyze the monitoring log to determine the performance of the specified method, thereby determining the performance of the target application. Optionally, the application server can also report the obtained monitoring log to the monitoring device for analysis to obtain the performance analysis results of the specified method. The embodiments of the present application do not limit the device that performs log analysis.

[0085] The following first explains the application architecture of the embodiment of the present application. Figure 1 Schematic diagram of an application architecture of the monitoring log processing method provided in the embodiment of the present application. Figure 1 As shown, the application architecture may include an application server 11 and a monitoring device 12. Optionally, in actual applications, the monitoring device 12 may be a monitoring server.

[0086] Exemplarily, the application server 11 may include: an application module 111, a disk 112, and a collection module 113. The application module 111 is mainly used to generate a monitoring log when the target application calls a specified method, and refresh it to the log file of the disk 112. The collection module 113 can read the log data from the log file of the disk 112 at regular intervals and send it to the monitoring device 12 for processing.

[0087] Below we first Figure 1 The application architecture diagram shown illustrates the existing log processing flow and explains the problems existing in the existing log processing flow.

[0088] Optionally, in the existing log processing method, the generation and output method of the monitoring log is the same as the output method of the normal application business log. The application module 111 can synchronously calculate the response time of the specified method when the specified method is called, and generate the monitoring log based on the preset monitoring log template and refresh it to the log file of the disk 112. Correspondingly, the acquisition module 113 can collect the log file of the client monitoring disk, and collect incremental log data from the log file of the disk 112 at regular intervals (for example, 5 seconds) and send it to the monitoring device 12. Therefore, the monitoring device 12 can cache the received log data according to the preset analysis frequency (for example, 1 minute), and regularly analyze and process the cached data to determine the performance indicator information of the specified method, and then determine the performance of the target application to a certain extent, so as to subsequently issue alarms and generate reports.

[0089] In actual applications, the monitoring log may contain the call time (time), monitoring point identifier (key), application name (appName), host name (hostname), and call duration (elapsedTime, in milliseconds). Correspondingly, the data storage format of the monitoring device is: monitoring point key-(time-time / analysis frequency): {hostname:[elaspedTime1,elaspedTime2,…]}. For example, if the target application calls the specified method at dd hour mm minute nn second in xxxx year yy month zz day, the monitoring log generated is represented by the following storage format:

[0090]

[0091] Optionally, the monitoring device 12 analyzes the received log data as follows. Specifically, the monitoring device can periodically analyze the data from key to the last analysis frequency time point according to the configured monitoring point. For example, if the current time is 20200702211730 and the preset analysis frequency is 1 minute, the last analysis frequency time point is the current minute - 1, i.e., 202007022116.

[0092] For example, the monitoring device is identified as ap.v2.token.getToken at the monitoring point, and the data storage format of the monitoring log at the calling time of xxxx year yy month zz day dd hour mm minute is as follows:

[0093]

[0094] Optionally, after caching the received log data, the monitoring device calculates the performance indicators of the specified method at the whole machine level, for example, the average value, maximum value, minimum value, TP50, TP90, TP99, and TP999.

[0095] Among them, the meaning of TP indicator is: TP = Top Percentile, Top percentage, which means that within a time period, the time consumed by each call of the method is counted, and these times are sorted in ascending order, and the result is taken out: total number of times * number of indicators (note: percentage) = corresponding TP indicator value, and then the value corresponding to the sorted time is taken out.

[0096] For example, the TP99 value is calculated as follows: take out all the time-consuming data within the analysis frequency time point, sort them by the smallest arrival time, and take the time consumption of the 99th%+1st as the TP99 value. The business meaning expressed is that 99% of the method calls are below this value.

[0097] It is worth mentioning that in actual applications, the same application may be deployed in multiple instances, and each instance will generate its own monitoring log. In this way, the specific instance from which each monitoring log data comes can be determined based on the host (hostname) field identification data in the monitoring log. This can analyze the performance of the application method of each hostname instance, as well as the performance of the overall application (i.e., application).

[0098] Specifically, under normal circumstances, the shorter the execution time of a method when it is called, the higher the method performance is considered. However, the execution time of the same method when it is called multiple times may not be consistent, but it will always be stable in a certain range. In order to accurately reflect the performance of the method, the execution time of all methods within a period of time is usually counted, and the performance of the method is judged based on multiple performance indicators such as the maximum, minimum, average and various TP values ​​of the execution time. Similarly, the lower the value of these performance indicators, the higher the performance of the specified method.

[0099] It is understandable that in practical applications, according to the above method, not only the performance of the method can be monitored, but also whether an exception occurs when the method is called. If an exception occurs when the method is called, it will also be reflected in the monitoring log. Regarding the monitoring of abnormal conditions, the embodiments of the present application do not limit it, and it can be determined according to actual needs.

[0100] In actual applications, if the query rate per second (QPS, which is a measure of the amount of traffic processed by a specific query server within a specified time) of an application is relatively high, the application module monitoring method has to handle more core processes. In the case of high concurrency, the application module will generate a large number of monitoring logs in a short period of time, which will cause the following problems:

[0101] Problem 1: The application module generates a large amount of monitoring log data and continuously refreshes it to the log files on the disk. This will increase the disk and processor usage of the application server where the application module is located. The usage will continue to increase with the growth of QPS, which may seriously affect the normal operation of the application module.

[0102] Problem 2: When the log files on the disk refresh a large amount of log data in a short period of time, a large incremental log file is generated. This increases the collection pressure on the collection module and may cause a delay in the transmission of log data due to the inability to collect incremental log files in time and report them to the monitoring device. In severe cases, the log data may be lost due to the log file being split too quickly.

[0103] Question 3: Usually, the time-consuming value of a method will fluctuate within a certain range, for example, between 10 and 500 milliseconds. When the QPS increases, a large number of repeated time-consuming requests will be generated in a short period of time (for example, the execution time of the same method call varies, but it will always stabilize in a range. For example, the stable range of the call time of method A is between 10 and 50 milliseconds. If it is called 100 times, there will be at least 50 repeated execution times. If it is called 1000 times, there will be at least 950 repeated execution times. The more calls, the more repeated execution times, and thus the more repeated time-consuming requests). After receiving the log data, the monitoring device needs to store all the original data, which will take up a lot of cache space. At the same time, the sorting of large amounts of data will increase the processing pressure of the monitoring device, requiring a lot of resources and affecting the analysis efficiency.

[0104] In summary, existing performance methods have the problem of low performance analysis efficiency and accuracy.

[0105] For example, Figure 2 Schematic diagram of another application architecture of the monitoring log processing method provided in the embodiment of the present application. Figure 2 As shown, the monitoring log processing method is applied to an application server 20 , and the application server 20 may include: an application module 201 , a disk 202 , a collection module 203 and a monitoring module 204 .

[0106] according to Figure 2 and Figure 1 As shown in the architecture diagram, Figure 2 In the architecture diagram shown, the monitoring module 204 is integrated in the application server 20, that is, the application server 20 can generate a monitoring log through the application module 201 and refresh it to the log file on the disk 202. The collection module 203 can periodically read the log data from the log file on the disk 202 and transmit it to the monitoring module 204 for processing.

[0107] Regarding the specific implementation of the application module 201, the disk 202, and the acquisition module 203 Figure 1 The implementation of the application module 111, disk 112, and acquisition module 113 in the architecture diagram is similar, and will not be repeated here. The function of the monitoring module 204 is similar to that of the monitoring device 12, and will not be repeated here.

[0108] It is understandable that in Figure 1 and Figure 2 In the architecture diagram shown, since the log processing method is the same, there will be problems with low performance analysis efficiency and accuracy.

[0109] In response to the above problems, the technical conception process of the technical solution of the present application is as follows: The inventor has found through practice that when the QPS of the application is very high, if all the call data generated by the application in a period of time are processed in the application module, for example, the call data with the same time consumption generated at the same monitoring point are merged, so that all the log data of the same monitoring point within a period of time can be merged into a monitoring log. In this way, when the monitoring log is stored and transmitted to the monitoring device or monitoring module for processing, it will not be affected by the QPS, thereby avoiding the occurrence of the above problems 1 to 3, thereby improving the performance analysis efficiency and analysis accuracy of the application.

[0110] Specifically, the present application starts with the problems caused by high concurrency. By analyzing the monitoring log and the back-end data storage structure, the inventors found that the monitoring log content can be optimized and compressed without reducing the data accuracy and affecting the data analysis results. For example, by caching and counting the request time-consuming data (i.e., call data) generated during a cache period within the application module, and then merging and generating a monitoring log, and then starting the log thread of the application module to regularly refresh the merged monitoring log to the log file on the disk, the acquisition module can normally collect the generated monitoring log and report it to the monitoring device or monitoring module, and the monitoring device or monitoring module can complete the performance indicator data statistics by adjusting the data statistics calculation method for the newly merged monitoring log.

[0111] As can be seen from the above process, the technical solution of the present application needs to be improved when the application module generates the monitoring log, by caching the real-time monitoring data requested, and updating the format of the monitoring log, the transmission mode of the data is changed from synchronous output to asynchronous output, and space is used for time, thereby reducing the amount of monitoring log data generated, and reducing the frequency of the monitoring log being refreshed to the disk, realizing the control of the log output volume and the refresh frequency, and ensuring the stable data collection of the acquisition module. In this solution, it is necessary to upgrade the monitoring processing end at the same time, so as to be compatible with the new log data format, and the calculation of the performance index can be performed for the new log data format, for example, the calculation of the average, maximum, minimum and TP value is realized, so that the accurate real-time statistical analysis of the monitoring data can be ensured while reducing the amount of monitoring logs.

[0112] From the above analysis, it can be seen that the embodiment of the present application provides a monitoring log processing method, by obtaining multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, and according to the monitoring point identifier and the cache time period identifier carried by each call data, at least one call information corresponding to the multiple call data is stored in the corresponding cache queue in sequence, each call information includes a call execution time and the number of call executions of the call execution time, and the duration corresponding to each interval cache time period. By processing all call information in each cache queue, multiple monitoring logs for the target application can be generated and transmitted to the monitoring device, so that the monitoring device can cache multiple monitoring logs according to the data storage template preset at each monitoring point, and process the cached multiple monitoring logs according to the preset analysis frequency to determine the performance indicators of the application server corresponding to the target application, which are not affected by the QPS of the target application, thereby improving the performance analysis efficiency and analysis accuracy of the application.

[0113] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0114] Figure 3 The flowchart of the monitoring log processing method embodiment 1 provided in this application is as follows. Figure 1 The information interaction between the application server and the monitoring device in the architecture diagram shown in FIG. Figure 3 As shown, the monitoring log processing method may include the following steps:

[0115] S301: The application server obtains a plurality of call data generated at each monitoring point when the target application calls a specified method within each cache time period.

[0116] Each call data carries a monitoring point identifier and a cache time period identifier.

[0117] In this embodiment, when the performance of the target application needs to be monitored, the application server can calculate the performance of the target application when calling a specified method. Optionally, the specified method is usually one of a plurality of important methods executed by the target application in actual applications.

[0118] Exemplarily, before the specified method is called, monitoring posts can be set at multiple different locations to form multiple different monitoring points. Therefore, when the target application calls the specified method, multiple call data corresponding to each monitoring point will be generated on the application server side. In addition, in order to facilitate the processing of call data, the application server in this embodiment can obtain the call data generated at each monitoring point based on the set cache time period. Therefore, the application server can obtain multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period.

[0119] It is understandable that in this embodiment, since the target application can call the specified method in different cache time periods and the call data at different monitoring points may be different, each call data obtained by the application server carries a monitoring point identifier and a cache time period identifier.

[0120] S302: The application server stores at least one piece of call information corresponding to the plurality of call data in a corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each piece of call data.

[0121] Optionally, each piece of call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier.

[0122] In this embodiment, the application server may store the acquired multiple pieces of call data respectively based on the monitoring point and the cache time period to which the call data belongs.

[0123] Specifically, the application server first sets up a cache queue according to the monitoring point identifier and cache time period identifier carried in the call data, and then analyzes each of the multiple call data to determine the call execution time (i.e., call duration) carried in the multiple call data and the number of times each call execution time occurs (i.e., call execution count), and then generates a call information based on a call execution time and the call execution count of the call execution time. Accordingly, multiple call information can be generated according to the number of call execution times carried in the multiple call data; finally, each call information is stored in the cache queue corresponding to the corresponding monitoring point identifier and the corresponding cache time period identifier.

[0124] S303: The application server processes all call information in each cache queue at intervals corresponding to the cache time period, and generates multiple monitoring logs for the target application.

[0125] In this embodiment, since the application server obtains multiple call data generated at each monitoring point when the target application calls the specified method once with the duration corresponding to the cache time period, the frequency of caching call information in the cache queue is the duration corresponding to each interval cache time period. Therefore, the application server can process all call information in each cache queue for each duration corresponding to the interval cache time period, and generate a monitoring log based on all call information in each cache queue, thereby generating multiple monitoring logs for the target application in each cache time period.

[0126] It can be understood that since each cache queue is named with a monitoring point identifier and a cache time period identifier, each cache queue corresponds to a unique cache time period and a unique monitoring point, thereby ensuring that each monitoring point corresponds to a monitoring log within each cache time period, which is not affected by the target application calling the specified method (i.e., QPS), thereby ensuring the timeliness and efficiency of data processing.

[0127] S304: The application server sends multiple monitoring logs to the monitoring device.

[0128] In a possible design of an embodiment of the present application, the application server can transmit the obtained multiple monitoring logs to the monitoring device for analysis and processing, and then obtain the performance indicators of the specified method.

[0129] S305: The monitoring device caches the multiple monitoring logs received from the application server according to the preset data storage template of each monitoring point.

[0130] Optionally, a data storage template can be pre-installed inside the monitoring device. When the application server sends multiple monitoring logs to the monitoring device, the monitoring device can receive the multiple monitoring logs of the target application. After receiving the multiple monitoring logs, the multiple monitoring logs can be cached based on the data storage template, thereby facilitating subsequent data analysis.

[0131] Optionally, the data format of the data storage template may be a combination of a monitoring point identifier and a cache time period identifier as the first-level directory, a host name as the second-level directory, and under the host name is call information including the call execution time and the call execution count at the call execution time. Therefore, the monitoring device may process multiple monitoring logs in sequence according to the data format of the data storage template and cache them in corresponding directories respectively.

[0132] S306: The monitoring device processes the cached multiple monitoring logs according to a preset analysis frequency to determine the performance index of the target application corresponding to the application server.

[0133] Optionally, the monitoring device may store a preset analysis frequency for each monitoring point, and the preset analysis frequencies of different monitoring points may be the same or different. Exemplarily, the preset analysis frequency may be 30 seconds, 1 minute, 2 minutes, or other durations, and the actual preset analysis frequencies of different monitoring points may be set according to actual needs, which will not be described in detail here.

[0134] For example, in actual applications, the monitoring device can directly process the received multiple monitoring logs, or it can first cache the received monitoring logs according to the implementation in S305, and then perform performance analysis on the cached multiple monitoring logs according to the preset analysis frequency in the monitoring device to determine the performance indicators of the specified method, and then determine the performance indicators of the target application.

[0135] In practical applications, the indicators for judging application performance may include: the maximum value, minimum value, average value, TP value, etc. of the call execution time. In this embodiment, the monitoring device may sort all the call execution times under each combination of the monitoring point identifier and the cache time period identifier in order from small to large, so as to determine the maximum value and minimum value of the call execution time; then, using the formula sum(call execution time*call execution times) / sum(call execution times), the average value of the call execution time may be calculated; finally, to calculate the TP value, for example, TP99, the monitoring device sorts the call execution times from large to small, and then uses the sum(call execution times)*99% value as the call execution times (99); for all the call execution times sorted from large to small, firstly, for the first call execution time, the difference of sum(call execution times) minus the call execution times (99) is calculated, if the difference is less than or equal to the call execution times corresponding to the first call execution time, the first call execution time is used as TP99, otherwise, the calculation is continued for the second call execution time until the condition is met.

[0136] For example, suppose a set of call execution times and call execution times of a certain monitoring point in a certain cache time period are: 50ms:2; 54ms:4; 68ms:5; 70ms:6; 100ms:3. If the calculation is performed according to the algorithm of the embodiment of the present application, the call execution times are first sorted from large to small, that is, 100ms:3, 70ms:6, 68ms:5, 54ms:4, 50ms:2, and then TP99 and TP50 are calculated based on the order of the call execution times from large to small. Specifically, for TP99, count(99)=sum(count)*99%=(2+4+5+6+3)*99%=19.8. For the first call execution time of 100ms, at this time, sum1(count)=2+4+5+6+3=20, and 20-19.8=0.2; since 0.2 is less than the count value 3 corresponding to the call execution time of 100ms, the value of TP99=100ms; similarly, for TP50, count(50)=sum(count)*50%=(2+4+5+6+3)*50%=10; for the first call execution time of 100ms, ms, sum1(count)=2+4+5+6+3=20, and 20-10=10, since 10 is not less than the call execution number 3 corresponding to the call execution time 100ms, then for the second call execution time 70ms, sum2(count)=2+4+5+6=17, 17-10=7, since 7 is not less than the call execution number 6 corresponding to the call execution time 70ms, then for the third call execution time 68ms, sum3(count)=2+4+5=11, 11-10=1, 1 is less than the call execution number 5 corresponding to the call execution time 68ms, so TP50=68ms. The calculation method of other TP values ​​is similar and will not be repeated here.

[0137] The monitoring log processing method provided in the embodiment of the present application obtains multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, and stores at least one call information corresponding to the multiple call data in the corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data. Each call information includes a call execution time and the number of call executions at the call execution time, and the duration corresponding to each interval cache time period. By processing all call information in each cache queue, multiple monitoring logs for the target application can be generated and transmitted to the monitoring device. In this way, the monitoring device can cache the multiple monitoring logs according to the data storage template preset at each monitoring point, and process the cached multiple monitoring logs according to the preset analysis frequency to determine the performance index of the application server corresponding to the target application. It is not affected by the QPS of the target application, thereby improving the performance analysis efficiency and analysis accuracy of the application.

[0138] Based on the above embodiments, Figure 4 The following is a flow chart of the second embodiment of the monitoring log processing method provided by the present application. The method is explained with the application server as the execution subject. Figure 4 As shown, the above S302 can be implemented by the following steps:

[0139] S401: Divide the acquired multiple call data into at least one data set according to the monitoring point identifier and the cache time period identifier carried by each call data.

[0140] Each data set includes at least one piece of call data having the same monitoring point identifier and the same cache time period identifier.

[0141] Exemplarily, in order to reduce the amount of data transmitted, when the application server caches the acquired multiple call data, it first divides the multiple call data into sets according to the length of the cache time period. Usually, in order to ensure the efficiency of subsequent processing, the length of the cache time period is any value between 5 seconds and 10 seconds. Usually, the length of the cache time period is an integer. For example, the length of the cache time period is 10 seconds.

[0142] Optionally, the application server may divide the multiple call data generated between xxxx / yy / zz / dd:mm:00 and xxxx / yy / zz / dd:mm:10 into multiple data sets, and each data set is marked with a monitoring point identifier - xxxxyyzzddmm.

[0143] S402: For each data set, process all the call data in the data set according to the call execution time of each call data, and determine at least one call information.

[0144] Exemplarily, for each determined data set, first, according to the call execution time in each call data, the number of occurrences of each call execution time in the data set can be counted, and at least one call information corresponding to each data set can be generated. Each call information can include each call execution time and the number of call executions of the call execution time.

[0145] It is understandable that, for the call data in each data set, if multiple call data have the same call execution time, the call execution times of the call execution time are accumulated to obtain the correlation between the call execution time and the call execution times.

[0146] For example, for the data set of "ap.v2.token.getToken-xxxxyyzzddmm", the corresponding call information can be expressed in the form of "call execution time: call execution times".

[0147] S403: Store each piece of call information in sequence into a cache queue corresponding to the data set.

[0148] In this step, the cache queue corresponding to the data set can be expressed as follows:

[0149]

[0150] Optionally, when the name of the data set is ap.v2.token.getToken-xxxxyyzzddmm, and the call information of the data set includes: 2:7, 10:12, 16:5, the at least one call information corresponding to the data set is stored in the cache queue in the following format:

[0151]

[0152] Accordingly, the above S303 can be implemented by the following steps:

[0153] S404. For every duration corresponding to the cache time period, update the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue, and the number of call execution times of each call execution time into a preset log template to generate multiple monitoring logs for the target application.

[0154] Optionally, a preset log template is stored in the application server, and the format of the preset log template may be as follows:

[0155]

[0156] It is understandable that in the above-mentioned preset log template, the cache time period identifier, monitoring point identifier, application name, host name, call execution time, call execution count, etc. can all be updated, and the embodiments of the present application do not limit them.

[0157] Accordingly, updating the information in the cache queue determined in S403, that is, at least one piece of call information corresponding to the multiple call data in the cache time period xxxxyyzzddmm, into the above-mentioned preset log template can generate the following monitoring log:

[0158]

[0159] Optionally, in a specific implementation, after the application server stores at least one call information corresponding to each of the above-mentioned multiple call data in the corresponding cache queue, it can start a log thread, periodically (i.e., the duration corresponding to each interval cache time period, for example, 10 seconds) take out the identifiers of all monitoring points from the previous cache queue, and take out at least one call information in the current cache queue and merge them to generate a monitoring log.

[0160] If the current time is xxxxyyzzddmm13 and the duration of the cache time period is 10 seconds, the cache time period identifier of the current cache queue is xxxxyyzzddmm10, and the cache time period identifier of the previous cache queue is queuexxxxyyzzddmm00.

[0161] According to the records in S401 to S404 above, in this embodiment, each monitoring log corresponds to a unique monitoring point identifier and a unique cache time period identifier. This monitoring log format ensures that a monitoring point will only generate one monitoring log data in a cache time period, and will not increase due to the increase in the QPS of the target application, thereby ensuring the stable output of the monitoring log volume. At the same time, since in actual applications, the performance indicator analysis of a specified method only depends on the call execution time and the number of call executions, merging multiple call data into one monitoring log will not affect the calculation of the performance indicator, thereby ensuring the information and accuracy of the log data.

[0162] The monitoring log processing method provided in the embodiment of the present application divides the acquired multiple call data into at least one data set according to the monitoring point identifier and cache time period identifier carried by each call data. For each data set, all the call data in the data set are processed according to the call execution time of each call data, and at least one call information is determined, and each call information is sequentially stored in the cache queue corresponding to the data set, so that the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue, and the number of call executions of each call execution time can be updated to the preset log template every time the duration corresponding to the cache time period corresponds to, and multiple monitoring logs for the target application are generated. In this technical solution, by caching the call data of the specified method and generating the monitoring log by asynchronous merging, the number of monitoring logs is greatly reduced, the utilization rate of the application server processor is reduced, and the stable operation of the target application is ensured.

[0163] Optionally, based on the above embodiment, Figure 5 This is an interactive diagram of the third embodiment of the monitoring log processing method provided by this application. Figure 5 As shown, in this embodiment, before the above S303, the method may further include the following steps:

[0164] S501. Store multiple monitoring logs of the target application into a disk log file.

[0165] In this embodiment, a disk is usually installed on the application server, and the generated monitoring log is cached by the disk. Optionally, after the application server generates multiple monitoring logs of the target application, the multiple monitoring logs can be written into the disk log file of the disk.

[0166] S502: Obtain incremental monitoring logs in disk log files according to a preset collection cycle.

[0167] Exemplarily, the application server stores a preset collection cycle, which can periodically read the incremental log files stored within the duration corresponding to the previous preset collection cycle from the disk log file on the disk according to the preset collection cycle, thereby ensuring the real-time and stability of monitoring log collection.

[0168] Exemplarily, based on the number of monitoring points set in the specified method and the length of the cache time period, the number of monitoring logs in the target time period can be determined. For example, the number of monitoring logs generated by the application server per hour is equal to the number of monitoring points multiplied by (3600 seconds / cache time period), where the cache time period is in seconds. In actual applications, the larger the QPS of the target application, the more repeated call execution time in the call data generated in each cache time period, the more call data merged in the monitoring log in the present application, the higher the optimization ratio of the monitoring log, and the more obvious the reduction in the amount of monitoring logs generated relative to the prior art solution.

[0169] Accordingly, the above S304 can be replaced by the following steps:

[0170] Send incremental monitoring logs to monitoring devices.

[0171] In this embodiment, in order to reduce the processing burden of the monitoring device, the application server may only obtain the incremental monitoring log from the disk log file of the disk, and then send the collected incremental monitoring log to the monitoring device.

[0172] In the monitoring log processing method provided in the embodiment of the present application, the application server can first store multiple monitoring logs of the target application in a disk log file, and then obtain the incremental monitoring log in the disk log file according to the preset collection period, and send the incremental monitoring log to the monitoring device. This technical solution not only ensures the information content of the monitoring log obtained by the monitoring device, but also reduces the cache space occupied by the monitoring device, thereby improving the data processing efficiency while ensuring the data processing accuracy.

[0173] Optionally, based on the above embodiment, Figure 6 This is a flow chart of the fourth embodiment of the monitoring log processing method provided by this application. This method is explained with the monitoring device as the execution subject. Figure 6 As shown, the above S305 can be implemented by the following steps:

[0174] S601: parse each monitoring log to determine a monitoring point identifier and a cache time period identifier corresponding to each monitoring log.

[0175] In this embodiment, after receiving each monitoring log, the monitoring device first parses each monitoring log to determine the monitoring point identifier and cache time period identifier carried by each monitoring log. Furthermore, the monitoring device can also determine the host name, application name, etc. corresponding to each monitoring log, which provides a prerequisite for subsequent analysis of application performance.

[0176] S602: Determine at least one piece of call information corresponding to each monitoring log according to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log.

[0177] Each piece of call information includes: a call execution time and the number of call executions at the call execution time.

[0178] In this embodiment, since the performance indicators of the specified method called by the target application are mainly determined by the call execution time and the number of call executions, and the monitoring log has a monitoring point identifier and a cache time period identifier, the monitoring device can first determine at least one call information corresponding to each monitoring log based on the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, and then determine the call status of the target application at each monitoring point and in each cache time period, for example, the call execution time and the number of call executions, thereby laying a foundation for subsequent performance analysis of the target application.

[0179] S603: Cache at least one piece of call information corresponding to each monitoring log based on the data storage template corresponding to each monitoring log.

[0180] Optionally, a data storage template is pre-configured in the monitoring device, and the length of the time period in the data storage template corresponds to the preset analysis frequency. Therefore, after receiving the monitoring log, the monitoring device needs to process it according to the preset analysis frequency. Therefore, when the processing time has not been reached, the monitoring device can merge the call information within the same analysis time period according to the data storage template.

[0181] It can be understood that the processing unit of the monitoring log is the length of the preset time period, for example, 10 seconds. The preset analysis frequency of the monitoring device is 1 minute, and the analysis time period corresponding to the preset analysis frequency is 60 seconds. At this time, the monitoring device can merge all call information with the same monitoring point identifier and falling within the same preset analysis cycle.

[0182] For example, assuming that the preset analysis frequency of the monitoring point identifier key1 is 1 minute, and the start time of a preset analysis cycle is xxxxyyzz1600, then its end time should be xxxxyyzz1700. At this time, in the above-mentioned multiple monitoring logs, the monitoring data with the cache time period identifier falling between xxxxyyzz1600 and xxxxyyzz1700 can be cached in the data queue of key1-xxxxyyzz1600. At the same time, the monitoring data with the cache time period identifier falling between xxxxyyzz1700 and xxxxyyzz1800 can be cached in the data queue of key1-xxxxyyzz1800. The caching of monitoring logs is implemented in sequence according to this storage logic.

[0183] On the basis of the above embodiments, when the monitoring point is identified as ap.v2.token.getToken and the processing time period is xxxxyyzz1600, the called host names are 11.12.170.11 and hostname2. Among them, 11.12.170.11 has called the specified method 24 times in the duration of xxxxyyzz1600, of which 7 times were called with an execution time of 2ms, 12 times with an execution time of 10ms, and 5 times with an execution time of 16ms; hostname2 has called the specified method 12 times in the duration of xxxxyyzz1600, of which 6 times were called with an execution time of 4ms, 6 times with an execution time of 9ms, etc. Correspondingly, the data storage format of the monitoring device is as follows:

[0184]

[0185] According to the data cache format on the monitoring device side, since the data cache content is the call execution time and the corresponding number of call executions, the amount of cached data on the monitoring device side and the range of the call execution time will not increase due to the increase in the number of times the target application calls the specified method. Therefore, this solution ensures the stability of the call information without affecting the accuracy of the analysis, and reduces the resource consumption of the monitoring device processor and the size of the cache space occupied.

[0186] It is understandable that the embodiments of the present application may also include a solution in which the monitoring device displays the analysis results or pushes them to a display device or an output device for early warning, which is not described in detail in this embodiment.

[0187] The monitoring log processing method provided in the embodiment of the present application is that the monitoring device parses each monitoring log, determines the monitoring point identifier and cache time period identifier corresponding to each monitoring log, determines at least one call information corresponding to each monitoring log based on the monitoring point identifier and cache time period identifier corresponding to each monitoring log, and finally caches at least one call information corresponding to each monitoring log based on the data storage template corresponding to each monitoring log. In this technical solution, the monitoring device uses the monitoring log generated by the compatible application server to reduce the amount of data processed without affecting the analysis accuracy when the QPS of the target application is large, avoiding excessive resource consumption and reducing the use of cache space.

[0188] Figure 7 The flowchart of the fifth embodiment of the monitoring log processing method provided by this application is as follows. Figure 2 The application server in the architecture diagram is used as the execution subject for explanation. Figure 7 As shown, the monitoring log processing method may include the following steps:

[0189] S701. Acquire multiple pieces of call data generated at each monitoring point when a target application calls a specified method within each cache time period, each piece of call data carries a monitoring point identifier and a cache time period identifier.

[0190] S702: According to the monitoring point identifier and the cache time period identifier carried by each piece of call data, at least one piece of call information corresponding to the multiple pieces of call data is sequentially stored in a corresponding cache queue.

[0191] Each piece of call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier.

[0192] S703: Process all call information in each cache queue for a duration corresponding to each cache time period, and generate multiple monitoring logs for the target application.

[0193] In this embodiment, the specific implementation of the above S701 to S703 is consistent with the specific implementation of the above S301 to S303, and will not be repeated here.

[0194] S704: Process multiple monitoring logs according to a preset analysis frequency to determine the performance index of the target application.

[0195] In this embodiment, when a monitoring module is integrated in the application server, that is, it has a performance analysis function, the performance analysis of the target application can be implemented directly on the application server. Therefore, after the application server generates multiple monitoring logs for a cache time period, it can directly process the multiple monitoring logs according to the preset analysis frequency to determine the performance indicators of the target application.

[0196] It is understandable that when the preset analysis frequency is inconsistent with the frequency of data cache, the generated monitoring log can be first stored in the disk log file of the disk, and then the data can be read from the disk regularly according to the preset analysis frequency and analyzed and processed.

[0197] Optionally, the embodiment of the present application may also include a solution for displaying or pushing the analysis results, which is not described in detail in this embodiment.

[0198] The specific implementation of this step can refer to the scheme for the monitoring device to process the received monitoring logs recorded in the above embodiments, which will not be repeated here.

[0199] The monitoring log processing method provided in the embodiment of the present application can obtain multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, and store at least one call information corresponding to the multiple call data in the corresponding cache queue in turn according to the monitoring point identifier and cache time period identifier carried by each call data, and process all call information in each cache queue at intervals corresponding to the cache time period to generate multiple monitoring logs for the target application, and finally process the multiple monitoring logs according to the preset analysis frequency to determine the performance indicators of the target application. In this technical solution, when the application server is integrated with the monitoring log analysis function, the purpose of application performance analysis can also be achieved. It is also not affected by QPS and does not require interaction between devices, which simplifies the interaction process.

[0200] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0201] Figure 8 This is a schematic diagram of the structure of the monitoring log processing device embodiment 1 provided in this application. Figure 8 As shown, the device may include:

[0202] An acquisition module 801 is used to acquire multiple call data generated at each monitoring point when the target application calls a specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0203] The cache module 802 is used to store at least one call information corresponding to the multiple call data in a corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data, each call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with the monitoring point identifier and the cache time period identifier;

[0204] The processing module 803 is used to process all the call information in each cache queue at intervals corresponding to the cache time period, and generate multiple monitoring logs for the target application;

[0205] The sending module 804 is used to send the plurality of monitoring logs to the monitoring device.

[0206] In a possible design of the embodiment of the present application, the cache module 802 is specifically configured to:

[0207] According to the monitoring point identifier and the cache time period identifier carried by each call data, the obtained multiple call data are divided into at least one data set, each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier;

[0208] For each data set, all the call data in the data set are processed according to the call execution time of each call data to determine at least one call information;

[0209] Each call information is sequentially stored in the cache queue corresponding to the data set.

[0210] Optionally, the processing module 803 is specifically used to update the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue and the number of call execution times of each call execution time into a preset log template every time period corresponding to the cache time period, so as to generate multiple monitoring logs for the target application.

[0211] In another possible design of the embodiment of the present application, the cache module 802 is further configured to process all call information in each cache queue at intervals corresponding to the cache time period in the processing module 803, generate multiple monitoring logs for the target application, and store the multiple monitoring logs of the target application in a disk log file;

[0212] The acquisition module 801 is further used to acquire the incremental monitoring log in the disk log file according to a preset acquisition cycle;

[0213] The sending module 804 is specifically used to send the incremental monitoring log to the monitoring device.

[0214] The device provided in the embodiment of the present application can be used to perform the above Figures 3 to 6 The implementation principle and technical effect of the solution of the application server in the method embodiment are similar, and will not be repeated here.

[0215] Fig. 9 This is a schematic diagram of the structure of the second embodiment of the monitoring log processing device provided by this application. Fig. 9 As shown, the device may include:

[0216] The receiving module 901 is used to receive multiple monitoring logs of the target application from the application server;

[0217] A cache module 902, configured to cache the plurality of monitoring logs respectively according to a data storage template preset at each monitoring point;

[0218] The processing module 903 is used to process the plurality of cached monitoring logs according to a preset analysis frequency, and determine the performance index of the target application corresponding to the application server.

[0219] In a possible design of the embodiment of the present application, the cache module 902 is specifically configured to:

[0220] Analyze each monitoring log to determine the monitoring point identifier and cache time period identifier corresponding to each monitoring log;

[0221] According to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, at least one call information corresponding to each monitoring log is determined, each call information including: a call execution time and a call execution count of the call execution time;

[0222] At least one piece of call information corresponding to each monitoring log is cached based on the data storage template corresponding to each monitoring log.

[0223] The device provided in the embodiment of the present application can be used to perform the above Figures 3 to 6 The implementation principle and technical effect of the monitoring device solution in the method embodiment are similar and will not be repeated here.

[0224] Fig.10 This is a schematic diagram of the structure of the monitoring log processing device embodiment 3 provided in this application. Fig.10 As shown, the device may include:

[0225] An acquisition module 1001 is used to acquire multiple call data generated at each monitoring point when the target application calls the designated device in each cache time period, each call data carries a monitoring point identifier and a cache time period identifier;

[0226] The cache module 1002 is used to store at least one call information corresponding to the multiple call data in a corresponding cache queue in sequence according to the monitoring point identifier and the cache time period identifier carried by each call data, each call information includes: a call execution time and the number of call executions at the call execution time, and each cache queue is named with the monitoring point identifier and the cache time period identifier;

[0227] The processing module 1003 is used to process all call information in each cache queue at intervals corresponding to the cache time period, generate multiple monitoring logs for the target application, and process the multiple monitoring logs according to a preset analysis frequency to determine the performance indicators of the target application.

[0228] The device provided in the embodiment of the present application can be used to perform the above Figure 7The implementation principles and technical effects of the solutions in the method embodiment are similar and will not be repeated here.

[0229] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above-mentioned module is determined. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0230] Fig.11 This is a schematic diagram of the structure of the application server embodiment 1 provided in the present application embodiment. Fig.11 As shown, the application server may include: a processor 1101, a memory 1102, a transceiver 1103 and a system bus 1104. The memory 1102 and the transceiver 1103 are connected to the processor 1101 through the system bus 1104 and communicate with each other. The memory 1102 is used to store computer execution instructions, and the transceiver 1103 is used to communicate with other devices. When the processor 1101 executes the computer execution instruction, the above-mentioned Figures 3 to 6 The solution of the application server in the illustrated embodiment.

[0231] Fig.12 This is a schematic diagram of the structure of the monitoring device embodiment provided in the present application. Fig.12 As shown, the monitoring device may include: a processor 1201, a memory 1202, a transceiver 1203 and a system bus 1204. The memory 1202 and the transceiver 1203 are connected to the processor 1201 through the system bus 1204 and communicate with each other. The memory 1202 is used to store computer execution instructions, and the transceiver 1203 is used to communicate with other devices. When the processor 1201 executes the computer execution instruction, the above-mentioned Figures 3 to 6 The scheme of the monitoring device in the embodiment shown.

[0232] Fig.13 This is a schematic diagram of the structure of the second application server embodiment provided in the present application embodiment. Fig.13 As shown, the application server may include: a processor 1301, a memory 1302, a communication interface 1303 and a system bus 1304. The memory 1302 and the transceiver 1303 are connected to the processor 1301 through the system bus 1304 and communicate with each other. The memory 1302 is used to store computer execution instructions, and the communication interface 1303 is used to communicate with other devices. When the processor 1301 executes the computer execution instruction, the above-mentioned Figure 7 The solution of the application server in the illustrated embodiment.

[0233] In the above Figures 10 to 13 The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0234] The memory may include random access memory (RAM), may include read-only memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.

[0235] The transceiver or communication interface is used to implement communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries).

[0236] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0237] Optionally, the embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the above Figures 3 to 6 The solution of the application server in the method embodiment shown in the figure, or, making the computer execute the above Figures 3 to 6 The monitoring device scheme in the method embodiment shown, or, making the computer execute the above Figure 7 The solution of the application server in the method embodiment is shown.

[0238] Optionally, the present application embodiment further provides a chip for executing instructions, the chip being used to execute the above Figures 3 to 6 The solution of the application server in the method embodiment shown, or the above Figures 3 to 6 The monitoring device scheme in the method embodiment shown, or the above Figure 7 The solution of the application server in the method embodiment is shown.

[0239] The embodiment of the present application also provides a program product, wherein the program product includes a computer program, wherein the computer program is stored in a computer-readable storage medium, and at least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the above-mentioned Figures 3 to 6 The solution of the application server in the method embodiment shown, or the above Figures 3 to 6 The monitoring device scheme in the method embodiment shown, or the above Figure 7 The solution of the application server in the method embodiment is shown.

[0240] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in a formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items.

[0241] It is to be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0242] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0243] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A monitoring log processing method, characterized in that: include: Acquire multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier; According to the monitoring point identifier and the cache time period identifier carried by each call data, the obtained multiple call data are divided into at least one data set, each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier; for each data set, according to the call execution time of each call data, the number of occurrences of each call execution time is counted, and at least one call information corresponding to each data set is generated; Each call information is sequentially stored in a cache queue corresponding to the data set, each call information including: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier; At intervals corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application; wherein the monitoring log is obtained by taking out and merging at least one call information in the cache queue; The multiple monitoring logs are sent to the monitoring device.

2. The method according to claim 1, characterized in that At each interval corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application, including: At intervals corresponding to the cache time period, the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue, and the number of call executions of each call execution time are updated to the preset log template to generate multiple monitoring logs for the target application.

3. The method according to claim 1 or 2, characterized in that: After processing all call information in each cache queue at each interval corresponding to the cache time period and generating multiple monitoring logs for the target application, the method further includes: Storing multiple monitoring logs of the target application in a disk log file; Obtaining incremental monitoring logs from the disk log file according to a preset collection cycle; The sending the plurality of monitoring logs to the monitoring device includes: The incremental monitoring log is sent to a monitoring device.

4. A monitoring log processing method, characterized in that: Applied to monitoring equipment, the method comprises: Receive multiple monitoring logs of the target application from the application server; wherein the monitoring log is obtained by taking out and merging at least one call information in the cache queue; According to the data storage template preset at each monitoring point, the plurality of monitoring logs are cached respectively; Processing the plurality of cached monitoring logs according to a preset analysis frequency to determine a performance indicator of the target application corresponding to the application server; The caching of the plurality of monitoring logs respectively according to the data storage template preset at each monitoring point includes: Analyze each monitoring log to determine the monitoring point identifier and cache time period identifier corresponding to each monitoring log; According to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, at least one call information corresponding to each monitoring log is determined, and each call information includes: a call execution time and the number of call executions of the call execution time; wherein the at least one call information is generated for each data set by counting the number of occurrences of each call execution time according to the call execution time of each call data, and the data set is obtained by dividing the acquired multiple call data, and each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier; At least one piece of call information corresponding to each monitoring log is cached based on the data storage template corresponding to each monitoring log.

5. A monitoring log processing method, characterized in that: include: Acquire multiple call data generated at each monitoring point when the target application calls the specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier; According to the monitoring point identifier and the cache time period identifier carried by each call data, the obtained multiple call data are divided into at least one data set, each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier; for each data set, according to the call execution time of each call data, the number of occurrences of each call execution time is counted, and at least one call information corresponding to each data set is generated; Each call information is sequentially stored in a cache queue corresponding to the data set, each call information including: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier; At intervals corresponding to the cache time period, all call information in each cache queue is processed to generate multiple monitoring logs for the target application; wherein the monitoring log is obtained by taking out and merging at least one call information in the cache queue; The plurality of monitoring logs are processed according to a preset analysis frequency to determine the performance index of the target application.

6. A monitoring log processing device, characterized in that: include: An acquisition module, used to acquire multiple call data generated at each monitoring point when the target application calls a specified method within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier; A cache module, used to divide the acquired multiple call data into at least one data set according to the monitoring point identifier and the cache time period identifier carried by each call data, each data set including at least one call data with the same monitoring point identifier and the same cache time period identifier; for each data set, according to the call execution time of each call data, count the number of occurrences of each call execution time, and generate at least one call information corresponding to each data set; Each call information is sequentially stored in a cache queue corresponding to the data set, each call information including: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier; A processing module, configured to process all call information in each cache queue at intervals corresponding to the cache time period, and generate a plurality of monitoring logs for the target application; wherein the monitoring log is obtained by taking out and merging at least one call information in the cache queue; The sending module is used to send the plurality of monitoring logs to the monitoring device.

7. The device according to claim 6, characterized in that The processing module is specifically used to update the cache time period identifier and monitoring point identifier of each cache queue, at least one call execution time in the cache queue and the number of call execution times of each call execution time into a preset log template every time period corresponding to the cache time period, so as to generate multiple monitoring logs for the target application.

8. The device according to claim 6 or 7, characterized in that The cache module is further configured to process all call information in each cache queue at intervals corresponding to the cache time period in the processing module, generate multiple monitoring logs for the target application, and store the multiple monitoring logs of the target application in a disk log file; The acquisition module is further used to acquire the incremental monitoring log in the disk log file according to a preset acquisition cycle; The sending module is specifically used to send the incremental monitoring log to the monitoring device.

9. A monitoring log processing device, characterized in that: include: A receiving module, used to receive multiple monitoring logs of a target application from an application server; wherein the monitoring log is obtained by taking out and merging at least one call information in a cache queue; A cache module, used to cache the multiple monitoring logs respectively according to the data storage template preset at each monitoring point; A processing module, configured to process the plurality of cached monitoring logs according to a preset analysis frequency, and determine a performance indicator of a target application corresponding to the application server; The cache module is specifically used for: Analyze each monitoring log to determine the monitoring point identifier and cache time period identifier corresponding to each monitoring log; According to the monitoring point identifier and the cache time period identifier corresponding to each monitoring log, at least one call information corresponding to each monitoring log is determined, and each call information includes: a call execution time and the number of call executions of the call execution time; wherein the at least one call information is generated for each data set by counting the number of occurrences of each call execution time according to the call execution time of each call data, and the data set is obtained by dividing the multiple call data obtained, and each data set includes at least one call data with the same monitoring point identifier and the same cache time period identifier; At least one piece of call information corresponding to each monitoring log is cached based on the data storage template corresponding to each monitoring log.

10. A monitoring log processing device, characterized in that: include: An acquisition module, used to acquire multiple call data generated at each monitoring point when the target application calls the designated device within each cache time period, each call data carries a monitoring point identifier and a cache time period identifier; A cache module, used to divide the acquired multiple call data into at least one data set according to the monitoring point identifier and the cache time period identifier carried by each call data, each data set including at least one call data with the same monitoring point identifier and the same cache time period identifier; for each data set, according to the call execution time of each call data, count the number of occurrences of each call execution time, and generate at least one call information corresponding to each data set; Each call information is sequentially stored in a cache queue corresponding to the data set, each call information including: a call execution time and the number of call executions at the call execution time, and each cache queue is named with a monitoring point identifier and a cache time period identifier; A processing module is used to process all call information in each cache queue at intervals corresponding to the cache time period, generate multiple monitoring logs for the target application, and process the multiple monitoring logs according to a preset analysis frequency to determine the performance index of the target application, wherein the monitoring log is obtained by taking out and merging at least one call information in the cache queue.

11. An application server, comprising a processor, a memory, a transceiver, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

12. A monitoring device, comprising a processor, a memory, a transceiver, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to claim 4 is implemented.

13. An application server comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to claim 5 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 3 when executed by a processor; or The computer-executable instructions are used to implement the method according to claim 4 when executed by a processor; or When the computer-executable instructions are executed by a processor, they are used to implement the method according to claim 5.

Images