Index log processing method, device and system

Abstract

The invention relates to the technical field of log monitoring, in particular to an index log processing method, device and system. The method comprises the following steps: receiving a statistical log in a preset format output by a service system; matching an expanded index object according to the statistical log, and processing the statistical log by using the expanded index object to obtain a business index; converting the business index into a target index conforming to a three-party monitoring system format; and transmitting the target index to the three-party monitoring system, so that the three-party monitoring system displays the target index in a corresponding display form. Therefore, the problems that existing log monitoring is low in development efficiency, high in business code coupling, poor in robustness and the like can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of log monitoring technology, and in particular to an indicator log processing method, device and system. Background Art

[0002] In the existing log monitoring technology, it is necessary to embed points in the business system code, obtain indicator data and store indicator data through designed attributes, and perform secondary business processing on indicator data according to business needs, and then develop a log sender on the business system side, develop a log receiver on the log monitoring system side, send the indicator data to the log receiver through the log sender, and finally display it through the monitoring system. For example, when counting the payment status of a period of time, it is necessary to embed points in the business system code to store the real-time bank name, payment amount, payment method and other information into the database, and write code to implement the accumulation of payment amounts, etc., and then transmit the processed total amount and other information to the monitoring system through the written log sender.

[0003] In short, the entire implementation process is complicated, which not only leads to low development efficiency, but also the embedded code is highly invasive to the business system code, resulting in high coupling of business code, poor robustness, and difficulty in maintenance and expansion. Summary of the invention

[0004] In view of this, the embodiments of the present application provide an indicator log processing method, device and system, which can effectively solve the problems of low efficiency of existing log monitoring development, high coupling of business codes, and poor robustness.

[0005] In a first aspect, an embodiment of the present application provides an indicator log processing method, which is applicable to a log processing system, including:

[0006] Receive statistical logs in a preset format output by the business system;

[0007] Matching the expanded indicator object according to the statistical log, and processing the statistical log using the expanded indicator object to obtain the business indicator;

[0008] Convert the business indicators into target indicators that conform to the format of the third-party monitoring system;

[0009] The target indicator is transmitted to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form.

[0010] In some embodiments, the statistical log is output by the business system in the following manner:

[0011] A standard indicator log is inserted at a preset position in the business code of the business system; and the statistical log is output according to a preset path through the standard indicator log.

[0012] In some embodiments, matching the extended indicator object according to the statistical log, and processing the statistical log using the extended indicator object to obtain the business indicator includes:

[0013] Parse the statistical log according to the preset format to obtain an indicator processing type and an indicator parameter set;

[0014] Matching the extended indicator object according to the indicator processing type; the extended indicator object is obtained by extending the general indicator object based on business logic; wherein the general indicator object is provided by the introduced third-party monitoring system;

[0015] The extended indicator object performs business processing based on the indicator parameter set to obtain the business indicator.

[0016] In some embodiments, the method further comprises:

[0017] Before matching the extended indicator object, determining whether the indicator identifier parsed from the statistical log is in the indicator configuration file;

[0018] If it exists, the extended indicator object is matched according to the indicator processing type; wherein the indicator configuration file is pre-configured according to the indicator identifier of each target indicator to be received.

[0019] In some embodiments, the statistical log in the preset format is obtained by dividing the indicator identifier, the indicator processing type and the indicator parameter set by a first separator in a preset order, and dividing each indicator parameter in the indicator parameter set by a second separator;

[0020] The matching of the extended indicator object according to the statistical log and processing the statistical log using the extended indicator object to obtain the business indicator includes:

[0021] According to the preset order and the first separator, the indicator processing type, the indicator identifier and the indicator parameter set are parsed from the statistical log;

[0022] According to the second separator, each indicator parameter is parsed from the indicator parameter set of the preset data type.

[0023] In some embodiments, converting the business indicator into a target indicator that conforms to the format of a third-party monitoring system includes:

[0024] A conversion component is called to convert the business indicator into the target indicator; wherein the conversion component is a toolkit provided by the third-party monitoring system embedded in the log processing system.

[0025] In some embodiments, transmitting the target indicator to the third-party monitoring system includes:

[0026] The target indicator is transmitted to the third-party monitoring system through a crawling function of the third-party monitoring system; wherein the crawling function realizes the crawling indicator based on the monitoring indicator object, crawling frequency, target server and target address configured in the configuration file of the third-party monitoring system.

[0027] In some embodiments, the preset path is a log output path pointing to the log processing system configured in a configuration file of the business system.

[0028] In a second aspect, an embodiment of the present application provides an indicator log processing device, including: a receiving module, a processing module, an object conversion module and a transmission module;

[0029] A receiving module, used to receive the statistical log in a preset format output by the business system;

[0030] A processing module, used to match the expanded indicator object according to the statistical log, and use the expanded indicator object to process the statistical log to obtain the business indicator;

[0031] An object conversion module, used to convert the business indicators into target indicators that conform to the format of the third-party monitoring system;

[0032] The transmission module is used to transmit the target indicator to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form.

[0033] In a third aspect, an embodiment of the present application provides a log processing system, including: the log processing system is used to communicate with a business system and a three-party monitoring system;

[0034] The log processing system is used to implement an indicator log processing method provided according to the first aspect of the present application, obtain the target indicator and transmit it to the three-party monitoring system;

[0035] The three-party monitoring system is used to display the target indicators in a corresponding display form.

[0036] The embodiments of the present application have the following beneficial effects:

[0037] The present application receives a statistical log in a preset format output by a business system; matches an expanded indicator object according to the statistical log, and uses the expanded indicator object to process the statistical log to obtain a business indicator; converts the business indicator into a target indicator that conforms to the format of a third-party monitoring system; transmits the target indicator to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form. The present application isolates the business processing code from the business system code through a log processing system, thereby reducing the coupling of the business system, and the log processing system can be applied to other business systems, reducing the workload of log monitoring development and improving the efficiency of development work. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A structural block diagram of a log processing system according to an embodiment of the present application is shown;

[0040] Figure 2 A flow chart showing the indicator log processing method of an embodiment of the present application is shown;

[0041] Figure 3 The following is an architecture diagram of Prometheus in the log processing system of an embodiment of the present application;

[0042] Figure 4 A flow chart showing the indicator log processing method of an embodiment of the present application is shown;

[0043] Figure 5 A structural schematic diagram of an indicator log processing device according to an embodiment of the present application is shown.

[0044] Description of main component symbols:

[0045] 100 - log processing system; 200 - third-party monitoring system; 300 - business system; 510 - receiving module; 520 - processing module; 530 - object conversion module; 540 - transmission module. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0047] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0048] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0049] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0050] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0051] In order to solve the problems of low log monitoring efficiency, high business code coupling, poor robustness, difficulty in maintenance and expansion in the existing log monitoring system, the present application provides an indicator log processing method, device and system.

[0052] Figure 1 A structural block diagram of a log processing system according to an embodiment of the present application is shown. Exemplarily, the log processing system 100 is communicatively connected to a three-party monitoring system 200 and a business system 300 .

[0053] The business system is used to transmit the collected statistical logs to the log processing system. The business system collects statistical logs in the following ways:

[0054] Insert a standard indicator log at a preset position of the business system code, and obtain the statistical log through the standard indicator log.

[0055] Exemplarily, the transmission of statistical logs can be realized by configuring the preset path of log output. At this time, the log output path that needs to be configured in advance in the configuration file of the business system is the preset path pointing to the log processing system. It is understandable that the existing Java log distribution system can also be used to transmit the statistical log to the log processing system. The Java log distribution system is about to be embedded in the business system. Similarly, it is also necessary to configure the receiving object of the statistical log as the log processing system in the configuration file of the development environment of the business system in advance.

[0056] The log processing system is used to implement the indicator log processing method of this application.

[0057] The third-party monitoring system is used to display business indicators through various display forms.

[0058] In one implementation, the third-party monitoring system uses Prometheus+Grafana. Prometheus is a tool for storing, querying, and displaying time series data. It can collect various types of indicator data, such as server load, network traffic, etc., through various Exporter components. Grafana is an open source visualization tool that can be used to create beautiful and interactive dashboards to display the data collected by Prometheus.

[0059] The indicator log processing method is described below with reference to some specific embodiments.

[0060] Figure 2 A flow chart of the indicator log processing method of an embodiment of the present application is shown. Applicable to the log processing system of the present application. Exemplarily, the indicator log processing method includes the following steps:

[0061] S100, receiving a statistical log in a preset format output by a business system.

[0062] In one implementation, the statistical log is output by the business system in the following manner:

[0063] Insert the standard indicator log at a preset position in the business code of the business system; output the statistical log according to the preset path through the standard indicator log. It can be understood that the standard indicator log is a criterion for outputting the statistical log, and the standard indicator log specifies the preset format of the statistical log. For example, the log.info() function is used to output the statistical log in the form of a preset format character.

[0064] It can be understood that the preset path is pre-configured in the configuration file of the business system, and the business system outputs statistical logs to the log processing system based on the preset path. It can be seen that the present application does not need to develop an additional transmitter for sending statistical logs in the business system, and only a simple configuration in the configuration file of the business system can realize the sending of statistical logs to the log processing system, which saves development workload and reduces the complexity of the log processing system collecting statistical logs.

[0065] S200, matching the expanded indicator object according to the statistical log, and using the expanded indicator object to process the statistical log to obtain the business indicator.

[0066] The log processing system receives the statistical logs output by the business system and parses the statistical logs based on a preset format.

[0067] In one embodiment, the method further comprises:

[0068] Before matching the indicator object construction method, determine whether the indicator identifier parsed from the statistical log is in the indicator configuration file; if so, match the expanded indicator object according to the indicator processing type; wherein the indicator configuration file is pre-configured according to the indicator identifiers of each target indicator to be received.

[0069] It is understandable that different business systems need to monitor different business indicators. In the embodiment of the present application, the indicator identifier of the target indicator to be received is configured in the indicator configuration file. Each time a statistical log is received, it is necessary to first search the indicator configuration file according to the indicator identifier of the statistical log to see whether the indicator identifier exists. If it exists, the target indicator is read and transmitted to the log processing system. In one embodiment, in step S200, the expanded indicator object is matched according to the statistical log, and the statistical log is processed using the expanded indicator object to obtain the business indicator, including:

[0070] S211, parsing the statistical log according to a preset format to obtain an indicator processing type and an indicator parameter set.

[0071] S212, matching the extended indicator object according to the indicator processing type, wherein the extended indicator object is obtained by extending the general indicator object based on the business logic; wherein the general indicator object is provided by the introduced third-party monitoring system.

[0072] Exemplarily, the third-party monitoring system includes Prometheus. In the prior art, Prometheus regularly captures indicator data from the target system through a crawling mechanism. These indicator data include indicators of various indicator types, such as counters, summaries, histograms, etc. Prometheus stores the captured indicator data in its built-in time series database. The value of the counter will be continuously updated over time, and Prometheus will retain historical data for a period of time for query and analysis. In an embodiment of the present application, the corresponding Prometheus client library (for example, Prometheus Client Library, the core part) is introduced into the code of the log processing system, and then the counter is defined and updated, and the functions corresponding to the indicator objects such as general counters, summaries, histograms, etc. are expanded according to the business logic.

[0073] Since the data processing function of the common indicator object in the existing three-party monitoring system is limited, it cannot meet the indicator requirements of different business systems. Therefore, this embodiment also includes expanding the common indicator object according to business requirements to realize business processing of statistical logs and transferring the business indicators after business processing to the three-party monitoring system.

[0074] Furthermore, when defining a standard indicator log, the indicator processing type is defined as the name of the corresponding indicator object, so that when the indicator processing type is matched with the corresponding extended indicator object, the name of the indicator object is not modified when the indicator object is extended. The indicator processing type includes a counter type, a histogram type, and the like. For example, in order to realize the statistics of the payment amount in a certain time period, in the embodiment of the present application, the character counter corresponding to the indicator processing type corresponds to the counter indicator object counter, and the counter indicator object counter realizes the accumulation function according to each single amount.

[0075] S213, the expanded indicator object performs business processing based on the indicator parameter set to obtain the business indicator. The indicator parameter set is input into the expanded indicator object for business processing to obtain the business indicator. For example, the indicator parameter set includes the bank name, payment method, single payment amount, etc. The bank name, payment method, and single payment amount are input into the counter indicator object counter to obtain the total payment amount under the bank name.

[0076] It can be understood that the statistical log includes the indicator processing type, the indicator identifier and the indicator parameter set. The indicator parameter set includes multiple indicator parameters. In addition, the statistical log also includes the indicator name.

[0077] In one embodiment, the statistical log in a preset format is obtained by separating the indicator identifier, the indicator processing type and the indicator parameter set by a first separator in a preset order, and separating each indicator parameter in the indicator parameter set by a second separator.

[0078] In step S200, the indicator object construction method is matched according to the statistical log, and the statistical log is processed using the indicator object construction method to obtain the business indicator, including:

[0079] S221, according to a preset order and a first separator, parse the statistical log to obtain an indicator processing type, an indicator identifier, and an indicator parameter set.

[0080] S222: According to the second separator, parse the indicator parameter set of the preset data type to obtain each indicator parameter.

[0081] Exemplarily, in the order of indicator identifier, indicator name, indicator processing type and indicator parameter set, the first separator is a comma to construct a statistical log in the form of a string. Exemplarily, the statistical log is "indicator identifier, indicator name_indicator processing type, indicator parameter set". The data type of the indicator parameter set can be a dictionary type. For example, in the payment business, MET RIC_EXPORT_V1.demoPayCntV1_counter = {"payType":"cash","serverName":"serv22","bank":"Bank A"} is used. Among them, METRIC_EXPORT_V1 is the indicator identifier, demoPayCntV1 is the indicator name, counter is the indicator processing type, and "payType", "serverName", and "bank" are indicator parameters.

[0082] Understandably, in the business system, by inserting the log.info() function, the statistical log to be output is printed in the log.info() function. For example, log.info("{}.{}_{}={}", "METR IC_EXPORT_V1", "demoPayCntV1", "counter", "{\"payType\":\"Reverse Scan\",\"serverName\":\"serv11\",\"bank\":\"Bank C\"}"). Understandably, the format of the standard metric log is also "metric identifier, metric name_metric processing type, metric parameter set", such as "{}.{}_{}={}", "METR IC_EXPORT_V1", "demoPayCntV1", "counter", "{\"payType\":\"Reverse Scan\",\"serverName\":\"serv11\",\"bank\":\"Bank C\"}".

[0083] In step S221, according to the preset order and the first delimiter, the metric processing type, metric identifier, and metric parameter set are parsed from the statistical log, including:

[0084] The statistical log is split using the first delimiter. The first field is the metric identifier, which is used to determine whether the statistical log is the one to be received. The second field is the metric name and metric processing type. The third field is the metric parameter set. Furthermore, a regular expression can be used to determine whether the statistical log is the target metric.

[0085] In step S222, according to the second delimiter, each metric parameter is parsed from the metric parameter set of the preset data type, including:

[0086] The metric parameter set is split according to the second delimiter to obtain each metric parameter. For example, both the first delimiter and the second delimiter are commas. The metric parameter set is data of the dictionary data type, where the key is the metric parameter and the value is the parameter value.

[0087] S300. Convert the business metric into a target metric that conforms to the format of the third-party monitoring system.

[0088] Understandably, different monitoring systems receive data in different formats. Naturally, the third-party monitoring system in this application embodiment also has a suitable data format. Therefore, this application embodiment needs to convert the business metric into a target metric suitable for the third-party monitoring system.

[0089] In step S300, converting the business metric into a target metric that conforms to the format of the third-party monitoring system includes:

[0090] According to the indicator processing type, the corresponding type of general function in the conversion component is called, and the business indicator is transmitted to the general function to generate an indicator object; wherein the conversion component is a toolkit provided by a third-party monitoring system embedded in the log processing system. In the embodiment of the present application, the toolkit of the third-party monitoring system is embedded in the log processing system so that the log processing system can convert the business indicator into an indicator object.

[0091] Demonstratively, the third-party monitoring system uses Prometheus+Grafana. Prometheus collects various types of indicator objects through various Exporter components, and Grafana displays the indicator objects collected by the Exporter components. The preset conversion component is provided by the third-party monitoring system and is preset in the log processing system; the preset conversion component provides a set of conversion rules corresponding to the Json data format, based on which the protocol conversion of the json format log can be realized, and Grafana will then standardize the converted json log into a unified web page.

[0092] like Figure 3 As shown in the figure, the architecture of Prometheus includes: Prometheus server (Prometheus Server collection and storage), Service discovery, Time-Series Database (TSDB for short), targets, indicator collector exporters, PushGateway, AlertManager (alarm manager), Client Libraries and PromQL. Among them, the Prometheus server is the brain of the indicator-based monitoring system.

[0093] Prometheus Server: used to capture and store indicator data.

[0094] Prometheus Configuration: Configuration file (usually `prometheus.yml`) that defines how Prometheus grabs indicator data.

[0095] Exporter (optional), exposes indicator data for Prometheus to crawl.

[0096] Prometheus Web UI or PromQL: used to query and visualize the captured indicator data.

[0097] Prometheus Client Library: Define and update counters in business system code.

[0098] S400, transmitting the target indicator to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form. After extracting the relevant indicators, the extracted indicators can be displayed according to the patterns suitable for these indicators. For example, if the indicators are suitable for being represented by a line graph, a line graph is selected.

[0099] Furthermore, the target indicators are transmitted to the third-party monitoring system, including:

[0100] Call the conversion component to convert the business indicator into the target indicator; wherein, the crawling function implements the crawling indicator based on the monitoring indicator object, crawling frequency, target server and target address configured in the configuration file of the third-party monitoring system. In other words, it is necessary to pre-configure the monitored indicator object, data crawling frequency, target server, target address, etc. in the third-party monitoring system. According to the above configuration information, each indicator object is crawled through the crawling function of the third-party monitoring system. For example, if the third-party monitoring system uses Prometheus+Grafana, configure job_name, scrape_interval, metrics_path, targets, etc. in the configuration file. Use the indicator collector exporters to collect indicator objects.

[0101] For example, 1) Download and install the Exporter software. For example, if you choose to use `mysqld_exporter`, you can install it according to its official documentation. 2) Configure Exporter so that it can connect to the target application or device. For example, for `mysqld_exporter`, you need to configure the username, password and other information for connecting to the MySQL database. 3) Start Exporter. After starting the Exporter service, Exporter will provide HTTP service on the specified port for Prometheus to crawl indicator data. For example, configure Prometheus to crawl indicators:

[0102] 1) Add a job to the Prometheus configuration file (usually `prometheus.yml`) to tell Prometheus which port to grab data from. The specific configuration example is as follows:

[0103]

[0104] In the above configuration, `localhost:9104` is the address and port where `mysqld_exporter` runs. This application implements log monitoring based on the existing third-party monitoring system, and the business system and log processing system also use the existing log distribution system. Figure 4As shown in the figure, 1), in this application, it is only necessary to insert the output statement code of the standard metric log in a preset position in the business system code. For example, insert log.info("{}.{}_{}={}", "METRIC_EXPORT_V1", "demoPayCntV1", "counter", "{"payType":"Reverse Scan", "serverName":"serv11", "bank":"Bank C}") in the business system code. 2) The business system outputs the statistical log to the log processing system. 3) The log processing system receives and parses the statistical log based on the preset format. 4) Since the metric object自带 by the third-party monitoring system is not flexible enough when processing data and cannot meet the requirements of each business system, therefore, in this application, the metric object自带 by the third-party monitoring system is extended, and the extended metric object is used to perform specific business processing on the statistical log to obtain business metrics. 5) Use the conversion component引入 the third-party monitoring system to convert the business metrics into target metrics that conform to the data format of the third-party monitoring system. 6) Send the target metrics to the third-party monitoring system. 7) Display the target metrics through the third-party monitoring system.

[0105] In this application, the standard metric log is inserted into the business system code, with low invasiveness, and business processing is performed through the log processing system, resulting in low coupling with the business system code. For each business system, it is only necessary to simply configure the log output object in the configuration file and configure the target metrics according to different business systems in the metric configuration file of the log processing system. Therefore, in this application, the log monitoring development work is simple, efficient, with low coupling of business code, high robustness, and easy maintenance.

[0106] Figure 5 FIG. shows a schematic structural diagram of the metric log processing device according to an embodiment of this application. Exemplarily, the metric log processing device includes: a receiving module 510, a processing module 520, an object conversion module 530, and a transmission module 540.

[0107] The receiving module 510 is configured to receive the statistical log in a preset format output by the business system;

[0108] The processing module 520 is configured to match the extended metric object according to the statistical log and use the extended metric object to process the statistical log to obtain business metrics;

[0109] The object conversion module 530 is configured to convert the business metrics into target metrics that conform to the format of the third-party monitoring system;

[0110] The transmission module 540 is configured to transmit the target metrics to the third-party monitoring system for the third-party monitoring system to display the target metrics through a corresponding display form.

[0111] It can be understood that the device of this embodiment corresponds to the indicator log processing method of the above embodiment, and the optional items in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0112] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of each module in the above-mentioned indicator log processing method or the above-mentioned indicator log processing device.

[0113] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0114] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0115] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0116] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0118] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0119] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for processing index logs, applicable to a log processing system, characterized in that: include: Receive statistical logs in a preset format output by the business system; Matching the expanded indicator object according to the statistical log, and processing the statistical log using the expanded indicator object to obtain the business indicator; Convert the business indicators into target indicators that conform to the format of the third-party monitoring system; The target indicator is transmitted to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form.

2. The indicator log processing method according to claim 1, characterized in that: The statistical log is output by the business system in the following ways: A standard indicator log is inserted at a preset position in the business code of the business system; and the statistical log is output according to a preset path through the standard indicator log.

3. The indicator log processing method according to claim 1, characterized in that: The matching of the extended indicator object according to the statistical log and processing the statistical log using the extended indicator object to obtain the business indicator includes: Parse the statistical log according to the preset format to obtain an indicator processing type and an indicator parameter set; Matching the extended indicator object according to the indicator processing type; the extended indicator object is obtained by extending the general indicator object based on business logic; wherein the general indicator object is provided by the introduced third-party monitoring system; The extended indicator object performs business processing based on the indicator parameter set to obtain the business indicator.

4. The indicator log processing method according to claim 3 is characterized in that: The method further comprises: Before matching the extended indicator object, determining whether the indicator identifier parsed from the statistical log is in the indicator configuration file; If it exists, the extended indicator object is matched according to the indicator processing type; wherein the indicator configuration file is pre-configured according to the indicator identifier of each target indicator to be received.

5. The indicator log processing method according to claim 4, characterized in that: The statistical log in the preset format is obtained by dividing the indicator identifier, the indicator processing type and the indicator parameter set by a first separator in a preset order, and dividing each indicator parameter in the indicator parameter set by a second separator; The matching of the extended indicator object according to the statistical log and processing the statistical log using the extended indicator object to obtain the business indicator includes: According to the preset order and the first separator, the indicator processing type, the indicator identifier and the indicator parameter set are parsed from the statistical log; According to the second separator, each indicator parameter is parsed from the indicator parameter set of the preset data type.

6. The indicator log processing method according to any one of claims 3 to 5, characterized in that: The converting of the business indicators into target indicators conforming to the format of the three-party monitoring system includes: A conversion component is called to convert the business indicator into the target indicator; wherein the conversion component is a toolkit provided by the third-party monitoring system embedded in the log processing system.

7. The indicator log processing method according to claim 6, characterized in that: The transmitting the target indicator to the third-party monitoring system includes: The target indicator is transmitted to the third-party monitoring system through a crawling function of the third-party monitoring system; wherein the crawling function realizes the crawling indicator based on the monitoring indicator object, crawling frequency, target server and target address configured in the configuration file of the third-party monitoring system.

8. The indicator log processing method according to claim 2, characterized in that: The preset path is the log output path pointing to the log processing system configured in the configuration file of the business system.

9. An indicator log processing device, characterized in that: include: Receiving module, processing module, object conversion module and transmission module; A receiving module, used to receive the statistical log in a preset format output by the business system; A processing module, used to match the expanded indicator object according to the statistical log, and use the expanded indicator object to process the statistical log to obtain the business indicator; An object conversion module, used to convert the business indicators into target indicators that conform to the format of the third-party monitoring system; The transmission module is used to transmit the target indicator to the third-party monitoring system so that the third-party monitoring system can display the target indicator in a corresponding display form.

10. A log processing system, characterized in that: The log processing system is used to communicate with the business system and the three-party monitoring system; The log processing system is used to implement the indicator log processing method according to any one of claims 1-8, obtain the target indicator and transmit it to the third-party monitoring system, so that the third-party monitoring system can display the target indicator in a corresponding display form.

Images