Indicator generation method, system, device and medium based on behavior log
Through the indicator generation method based on behavior logs, the front-end JSON file is generated using the preset configuration interface and the indicator input interface, and converted it into a SPARK executable logical program, which solves the problems of business logic fragmentation and computing logic display of existing SQL script solutions, and realizes more efficient indicator calculation and storage.
Patent Information
- Application Number
- CN202411864174.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-04-15
- Estimated Expiration
- 2044-12-18
AI Technical Summary
The existing SQL script solutions have problems such as fragmentation of business logic, inability to display the computing logic through pages, lack of global optimization capabilities, and complex modification of operation processes.
Through the indicator generation method based on behavior logs, the preset configuration interface and the indicator input interface are used to obtain data, generate or update the front-end JSON file, and convert it into a logical program executable by SPARK to realize indicator calculation and storage.
It solves the problems of fragmentation of business logic and display of computing logic, improves global optimization capabilities, and simplifies the operation process of metric SQL script modification.
Smart Images

Figure CN119311652B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of log data technology, and in particular to a method, system, device and medium for generating indicators based on behavior logs. Background Art
[0002] Log data records detailed information about system operation, including user behavior, system status, abnormal events, etc., providing a valuable source of information for system monitoring, performance optimization, troubleshooting, behavior analysis, etc. Log-based indicator calculation is an effective method proposed to solve this problem. It extracts key information by preprocessing, parsing and calculating log data, and converts it into quantitative indicators. These indicators can intuitively reflect the system's operating status, user behavior characteristics and potential problems, providing strong support for system management and optimization.
[0003] The traditional indicator calculation method is to write a separate SQL script for each indicator, deploy the SQL script to the server, and implement it through scheduled task scheduling. Although SQL is very powerful in relational database management and query, it has some obvious shortcomings when dealing with complex data projects and data warehouses. Specifically, 1. In lengthy SQL scripts, business logic is often scattered in multiple SQL code segments, which leads to fragmentation of business logic; 2. The calculation logic in the SQL script cannot be displayed on the page, and other R&D or operation and maintenance personnel cannot intuitively view the calculation logic of the indicator; 3. SQL queries are usually written for specific data requirements and lack the ability to optimize globally; 4. When modifying the indicator SQL script, upload and download operations are required, and the operation process is relatively complicated. Summary of the invention
[0004] In view of the above-mentioned deficiencies in the prior art, the present application provides an indicator generation method, system, device and medium based on behavior logs to solve the problems of existing SQL script solutions: fragmentation of business logic, inability to display calculation logic through pages, lack of global optimization capability of SQL scripts, and complicated upload and download operations required when modifying indicator SQL scripts.
[0005] In a first aspect, the present application provides a method for generating indicators based on behavior logs, the method comprising:
[0006] Preprocess the initial behavior log within the preset time period to obtain the network behavior log; obtain the filtering conditions and grouping conditions through the preset configuration interface; obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between the indicator formulas through the preset indicator input interface; generate or update the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface; convert the front-end JSON file into a SPARK executable logic program; call the logic program through the SPARK big data engine to obtain the indicator behavior log that meets the filtering conditions and grouping conditions from several network behavior logs; calculate the field results of each field in the indicator behavior log according to the indicator formula corresponding to the indicator formula interface; obtain the indicator result corresponding to the indicator behavior log according to the combination operation formula between the indicator formulas; store the indicator result in the preset database.
[0007] The indicator generation method provided in the embodiment of the present application generates or updates the front-end JSON file according to the data obtained from the configuration interface and the indicator input interface. This file contains all the necessary business logic and calculation logic. The problem of fragmentation of the business logic of the existing SQL script solution is solved; further, the present application can convert the front-end JSON file into a logical program executable by SPARK. This allows the business logic originally scattered in the SQL script to be centralized and optimized. In addition, the acquisition process of the calculation logic (formula) is mainly obtained through the preset indicator input interface. After the configuration is completed, the calculation logic can be displayed on the page through the preset indicator input interface, which solves the problem that the calculation logic cannot be displayed on the page; in addition, since the SQL script and the data are a one-to-one logical processing relationship, it can only be used for the current data, but the indicator formula involved in the present application is an interface, and the indicator formula called by the logic program can be changed by modifying the indicator formula corresponding to the interface, which solves the lack of global optimization ability of the SQL script; in addition, the present application can generate or update the front-end JSON file through the data obtained by the preset configuration interface and the preset indicator input interface, which solves the problem that the upload and download operation process is relatively complicated when the indicator SQL script is modified.
[0008] In one implementation of the present application, the initial behavior log within a preset time period is preprocessed to obtain a network behavior log, specifically including:
[0009] Collect several initial network behavior logs within a preset time period through a preset log collection tool;
[0010] According to the preset filtering conditions, the logs that do not meet the preset filtering conditions are removed from the initial network behavior logs to obtain the network behavior logs;
[0011] Convert unstructured network behavior logs into structured network behavior logs;
[0012] According to the source of the network behavior log, the log type of each network behavior log is identified to obtain the network behavior log.
[0013] In one implementation of the present application, the filtering conditions include at least log type, log time, and preset filtering terminal conditions;
[0014] Obtain indicator behavior logs that meet the filtering and grouping conditions from several network behavior logs, including:
[0015] Obtain network behavior logs that meet the filtering conditions from a number of network behavior logs;
[0016] Then, the network behavior logs meeting the grouping condition are obtained from the network behavior logs meeting the filtering condition.
[0017] In one implementation of the present application, the front-end JSON file is converted into a logic program executable by SPARK, specifically including:
[0018] Read the parameters in the front-end JSON file; automatically fill the parameters into the preset position of the preset logic program template to obtain the SPARK executable logic program; wherein the preset position at least includes: the position of the filter condition, the position of the grouping condition, the position of the indicator formula interface, the position of the combination operation formula between the indicator formulas, and the position of the indicator name.
[0019] In one implementation of the present application, the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between the indicator formulas are obtained through the preset indicator input interface, specifically including:
[0020] Get the indicator name through the name acquisition area of the preset indicator input interface;
[0021] Through the formula acquisition area of the preset indicator input interface, all preset indicator formula interfaces corresponding to each field type are displayed; through external triggering, the selected indicator formula interface is determined from all preset indicator formula interfaces and displayed in the editable area in the formula acquisition area; by triggering the preset operator in the editable area, the combination relationship formula between the indicator formulas is obtained.
[0022] In one implementation of the present application, according to the indicator formula corresponding to the indicator formula interface, the field results of each field in the indicator behavior log are calculated; according to the combined operation formula between the indicator formulas, the indicator results corresponding to the indicator behavior log are obtained, specifically including:
[0023] Based on the indicator formula interface and the field type corresponding to the indicator formula interface, the corresponding indicator formula is called to calculate the field result of the field corresponding to the indicator formula interface in the indicator behavior log;
[0024] Calculate and obtain the field results of each field in the indicator behavior log;
[0025] Substitute all field results into the combination operation formulas between indicator formulas to obtain the indicator results corresponding to the indicator behavior log.
[0026] In a second aspect, the present application provides an indicator generation system based on behavior logs, the system comprising:
[0027] The acquisition module is used to pre-process the initial behavior log within a preset time period to obtain the network behavior log;
[0028] The condition configuration module is used to obtain filtering conditions and grouping conditions through the preset configuration interface;
[0029] The formula configuration module is used to obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between indicator formulas through the preset indicator input interface;
[0030] The file generation module is used to generate or update the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface;
[0031] The program calling module is used to convert the front-end JSON file into a SPARK executable logic program; through the SPARK big data engine, the logic program is called to obtain the indicator behavior logs that meet the filtering conditions and grouping conditions from several network behavior logs; according to the indicator formula corresponding to the indicator formula interface, the field results of each field in the indicator behavior log are calculated; according to the combined operation formula between the indicator formulas, the indicator results corresponding to the indicator behavior log are obtained; and the indicator results are stored in the preset database.
[0032] In one implementation of the present application, the acquisition module includes an acquisition unit.
[0033] Used to collect a number of initial network behavior logs within a preset time period through a preset log collection tool;
[0034] According to the preset filtering conditions, the logs that do not meet the preset filtering conditions are removed from the initial network behavior logs to obtain the network behavior logs;
[0035] Convert unstructured network behavior logs into structured network behavior logs;
[0036] According to the source of the network behavior log, the log type of each network behavior log is identified to obtain the network behavior log.
[0037] In a third aspect, the present application provides an indicator generation device based on a behavior log, the device comprising:
[0038] processor;
[0039] and a memory having executable codes stored thereon, which, when the executable codes are executed, cause the processor to execute any of the above-mentioned methods for generating indicators based on behavior logs.
[0040] In a fourth aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which when executed implement a method for generating an indicator based on a behavior log as described above.
[0041] Those skilled in the art can understand that the present application has at least the following beneficial effects:
[0042] The present application discloses a method, system, device and medium for generating indicators based on behavior logs, which can generate or update a front-end JSON file according to data obtained from a configuration interface and an indicator input interface. This file contains all necessary business logic and calculation logic. It solves the problem of fragmentation of business logic of existing SQL script solutions; further, the present application can convert the front-end JSON file into a logic program executable by SPARK. This allows the business logic originally scattered in the SQL script to be centralized and optimized. In addition, the acquisition process of the calculation logic (formula) of the present application is mainly obtained through the preset indicator input interface. After the configuration is completed, the calculation logic can be displayed on the page through the preset indicator input interface, which solves the problem that the calculation logic cannot be displayed on the page; in addition, since the SQL script and the data are in a one-to-one logical processing relationship, they can only be modified one-to-one, but the indicator formula involved in the present application is an interface, and the indicator formula called by the logic program can be changed by modifying the indicator formula corresponding to the interface, which solves the lack of global optimization capability of the SQL script; in addition, the present application can generate or update the front-end JSON file through the data obtained through the preset configuration interface and the preset indicator input interface, which solves the problem that the upload and download operation process is relatively complicated when modifying the indicator SQL script. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:
[0044] Figure 1 This is a flow chart of a method for generating indicators based on behavior logs provided in an embodiment of the present application.
[0045] Figure 2 It is a schematic diagram of a system structure provided in an embodiment of the present application.
[0046] Figure 3 It is a schematic diagram of a device structure provided in an embodiment of the present application. DETAILED DESCRIPTION
[0047] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.
[0048] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0049] This application abandons the traditional SQL script method of generating indicators, adopts a visual configuration method and uses the big data engine calculation results, which reduces the difficulty of use and improves the calculation efficiency. The front-end JSON file is generated by the visual editing unit, and the front-end JSON file is translated into a SPARK executable logic unit, which is calculated by the SPARK big data calculation engine, and the indicator calculation results are stored in the database. The analyzed log data source, business caliber, calculation logic, etc. are all visualized, which reduces the difficulty of using the indicator function. Business personnel can configure and generate corresponding indicators through simple learning.
[0050] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0051] The embodiment provides a method for generating indicators based on behavior logs, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:
[0052] Step 110: pre-process the initial behavior log within a preset time period to obtain a network behavior log.
[0053] In some embodiments, this step may specifically include:
[0054] Through a preset log collection tool, several initial network behavior logs within a preset time period are collected (for example, log data collection, syslog, traffic and other network behavior logs are collected through log collection tools, traffic monitors, etc.); according to preset filtering conditions, the logs that do not meet the preset filtering conditions are filtered out from the initial network behavior logs to obtain network behavior logs; unstructured network behavior logs are converted into structured network behavior logs; according to the source of the network behavior logs, the log type of each network behavior log is identified to obtain the network behavior log.
[0055] The unstructured network behavior log is converted into a structured network behavior log, which can be specifically:
[0056] First, the format of the structured network behavior log needs to be defined. This usually includes determining which fields are necessary, as well as the data types and meanings of these fields. For example, a structured log may contain the following fields:
[0057] Timestamp: records the time when an event occurs.
[0058] User ID (user_id): The user involved in the recorded event.
[0059] Event type (event_type): records the specific type of event, such as page visits, button clicks, etc.
[0060] Event details (event_details): records detailed information about the event, such as the URL of the page visited, the name of the button clicked, etc.
[0061] IP address (ip_address): The IP address where the event occurred.
[0062] Session ID (session_id): A unique identifier for recording a user session.
[0063] Next, you need to parse the unstructured network behavior logs. This usually involves reading the log files and extracting the necessary information to fill the fields in a structured format. The parsing process may include:
[0064] Text parsing: For logs in text format, you need to use regular expressions or text parsing libraries to extract the required information.
[0065] JSON / XML parsing: For logs in JSON or XML format, you can use the corresponding parsing library to extract data.
[0066] Convert the field to the correct data type, such as converting a string to a timestamp or integer.
[0067] Finally, the structured network behavior logs are stored.
[0068] Step 120: Obtain filtering conditions and grouping conditions through a preset configuration interface.
[0069] The filtering condition is to filter the logs that meet the requirements of the indicator. The filtering can be performed from three aspects: log type, log time and preset filtering terminal conditions. The preset filtering terminal conditions can be in the form of: selecting a specific "terminal IP" from a number of terminal IPs, selecting a specific "terminal IP table" from a number of terminal IP tables, and selecting a specific relationship between "terminal IP" and "terminal IP table" from a number of preset relationships.
[0070] You can select multiple grouping conditions. The first grouping condition is the subject's unique identifier by default, which can be entity ID, IP address, MAC address, etc.
[0071] Step 130: Obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination calculation formula between the indicator formulas through the preset indicator input interface.
[0072] As an example, this step may be specifically as follows:
[0073] Get the indicator name through the name acquisition area of the preset indicator input interface.
[0074] Through the formula acquisition area of the preset indicator input interface, all preset indicator formula interfaces corresponding to each field type are displayed; through external triggering, the selected indicator formula interface is determined from all preset indicator formula interfaces and displayed in the editable area in the formula acquisition area; by triggering the preset operator in the editable area, the combination relationship formula between the indicator formulas is obtained.
[0075] It should be noted that the operator may be specifically four arithmetic operators. For example, the combination relationship formula between the indicator formulas may be the sum of all the indicator formulas.
[0076] Step 140: Generate or update the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface.
[0077] It should be noted that those skilled in the art can update the front-end JSON file by modifying the data in the preset configuration interface and the preset indicator input interface. The front-end JSON file mainly includes the specific data corresponding to the filter conditions, grouping conditions, the combination operation formula between the indicator formula interface and the indicator formula, and the indicator name.
[0078] Step 150, convert the front-end JSON file into a SPARK executable logic program; call the logic program through the SPARK big data engine to obtain the indicator behavior logs that meet the filtering conditions and grouping conditions from a number of network behavior logs; calculate the field results of each field in the indicator behavior log according to the indicator formula corresponding to the indicator formula interface; obtain the indicator result corresponding to the indicator behavior log according to the combination operation formula between the indicator formulas; store the indicator result in a preset database.
[0079] The front-end JSON file is converted into a logical program executable by SPARK, which can be specifically:
[0080] Read the parameters in the front-end JSON file; automatically fill the parameters into the preset position of the preset logic program template to obtain the SPARK executable logic program; wherein the preset position at least includes: the position of the filter condition, the position of the grouping condition, the position of the indicator formula interface, the position of the combination operation formula between the indicator formulas, and the position of the indicator name.
[0081] Among them, obtaining indicator behavior logs that meet the filtering conditions and grouping conditions from a number of network behavior logs may specifically be:
[0082] Obtain network behavior logs that meet the filtering conditions from a plurality of network behavior logs; and then obtain network behavior logs that meet the grouping conditions from the network behavior logs that meet the filtering conditions.
[0083] Among them, according to the indicator formula corresponding to the indicator formula interface, the field results of each field in the indicator behavior log are calculated; according to the combined operation formula between the indicator formulas, the indicator result corresponding to the indicator behavior log is obtained, which can be specifically:
[0084] Based on the indicator formula interface and the field type corresponding to the indicator formula interface, call the corresponding indicator formula, and then calculate the field results of the fields corresponding to the indicator formula interface in the indicator behavior log; calculate and obtain the field results of each field in the indicator behavior log; substitute all field results into the combination operation formula between the indicator formulas to obtain the indicator results corresponding to the indicator behavior log.
[0085] Based on the above description, this application can use page configuration to achieve the generation of indicators. Users only need to know the business scope of the indicators to generate the desired indicators through page configuration. This reduces the difficulty for users and is easy to maintain. The configuration steps of the terminal daily traffic size indicator can be:
[0086] 1. The selection area corresponding to the filter conditions of the preset configuration interface:
[0087] Select "Traffic Log" for Log Type; select "Previous Day" for Log Time; preset filtering terminal condition: "Source IP Address" is included in "Terminal IP Table";
[0088] 2. The selection area corresponding to the grouping conditions of the preset configuration interface:
[0089] Select Source IP Address from the attribute group.
[0090] 3. Preset indicator input interface: enter "daily terminal traffic size" as the indicator name; select the calculation formula interface of "sum of upstream traffic size" and the calculation formula interface of "sum of downstream traffic size" as the indicator formula; the combination calculation formula between indicator formulas is: "sum of upstream traffic size + sum of downstream traffic size".
[0091] In addition, the embodiment of the present application provides an indicator generation system based on behavior logs. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:
[0092] The collection module 210 is used to pre-process the initial behavior log within a preset time period to obtain the network behavior log.
[0093] Among them, the collection module 210 includes a collection unit, which is used to collect a number of initial network behavior logs within a preset time period through a preset log collection tool; according to preset filtering conditions, filter out the logs that do not meet the preset filtering conditions in the initial network behavior logs to obtain network behavior logs; convert unstructured network behavior logs into structured network behavior logs; according to the source of the network behavior log, identify the log type of each network behavior log to obtain the network behavior log.
[0094] The condition configuration module 220 is used to obtain filtering conditions and grouping conditions through a preset configuration interface.
[0095] The formula configuration module 230 is used to obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between the indicator formulas through the preset indicator input interface.
[0096] The file generation module 240 is used to generate or update the front-end JSON file according to the data obtained from the preset configuration interface and the preset indicator input interface.
[0097] The program calling module 250 is used to convert the front-end JSON file into a SPARK executable logic program; through the SPARK big data engine, the logic program is called to obtain the indicator behavior log that meets the filtering conditions and grouping conditions from a number of network behavior logs; according to the indicator formula corresponding to the indicator formula interface, the field results of each field in the indicator behavior log are calculated; according to the combined operation formula between the indicator formulas, the indicator results corresponding to the indicator behavior log are obtained; and the indicator results are stored in a preset database.
[0098] The above is a method embodiment of the present application. Based on the same inventive concept, the present application embodiment also provides an indicator generation device based on a behavior log. Figure 3 As shown, the device includes: a processor; and a memory, on which executable codes are stored. When the executable codes are executed, the processor executes a method for generating indicators based on behavior logs as described in the above embodiment.
[0099] Specifically, the server side preprocesses the initial behavior log within a preset time period to obtain a network behavior log; obtains filtering conditions and grouping conditions through a preset configuration interface; obtains the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between the indicator formulas through a preset indicator input interface; generates or updates the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface; converts the front-end JSON file into a SPARK executable logic program; calls the logic program through the SPARK big data engine to obtain the indicator behavior log that meets the filtering conditions and grouping conditions from several network behavior logs; calculates the field results of each field in the indicator behavior log according to the indicator formula corresponding to the indicator formula interface; obtains the indicator result corresponding to the indicator behavior log according to the combination operation formula between the indicator formulas; and stores the indicator result in a preset database.
[0100] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a method for generating indicators based on behavior logs as described above is implemented.
[0101] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.
Claims
1. A method for generating indicators based on behavior logs, characterized in that: The method comprises: Preprocessing the initial behavior log within a preset time period to obtain a network behavior log; Get filtering and grouping conditions through the preset configuration interface; Through the preset indicator input interface, obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination calculation formula between indicator formulas; Generate or update the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface; Convert the front-end JSON file into a SPARK executable logic program; call the logic program through the SPARK big data engine to obtain the indicator behavior logs that meet the filtering and grouping conditions from several network behavior logs; calculate the field results of each field in the indicator behavior log according to the indicator formula corresponding to the indicator formula interface; obtain the indicator results corresponding to the indicator behavior log according to the combined operation formula between the indicator formulas; store the indicator results in the preset database.
2. The indicator generation method based on behavior log according to claim 1 is characterized in that: Preprocess the initial behavior logs within a preset time period to obtain network behavior logs, including: Collect several initial network behavior logs within a preset time period through a preset log collection tool; According to the preset filtering conditions, the logs that do not meet the preset filtering conditions are removed from the initial network behavior logs to obtain the network behavior logs; Convert unstructured network behavior logs into structured network behavior logs; According to the source of the network behavior log, the log type of each network behavior log is identified to obtain the network behavior log.
3. The indicator generation method based on behavior log according to claim 1 is characterized in that: The filtering conditions at least include log type, log time, and preset filtering terminal conditions; Obtain indicator behavior logs that meet the filtering and grouping conditions from several network behavior logs, including: Obtain network behavior logs that meet the filtering conditions from a number of network behavior logs; Then, the network behavior logs meeting the grouping condition are obtained from the network behavior logs meeting the filtering condition.
4. The indicator generation method based on behavior log according to claim 1 is characterized in that: Convert the front-end JSON file into a logical program executable by SPARK, including: Read the parameters in the front-end JSON file; automatically fill the parameters into the preset position of the preset logic program template to obtain the SPARK executable logic program; wherein the preset position at least includes: the position of the filter condition, the position of the grouping condition, the position of the indicator formula interface, the position of the combination operation formula between the indicator formulas, and the position of the indicator name.
5. The indicator generation method based on behavior log according to claim 1 is characterized in that: Through the preset indicator input interface, obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination calculation formula between indicator formulas, including: Get the indicator name through the name acquisition area of the preset indicator input interface; Through the formula acquisition area of the preset indicator input interface, all preset indicator formula interfaces corresponding to each field type are displayed; through external triggering, the selected indicator formula interface is determined from all preset indicator formula interfaces and displayed in the editable area in the formula acquisition area; by triggering the preset operator in the editable area, the combination relationship formula between the indicator formulas is obtained.
6. The indicator generation method based on behavior log according to claim 1 is characterized in that: According to the indicator formula corresponding to the indicator formula interface, calculate the field results of each field in the indicator behavior log; According to the combined operation formulas between the indicator formulas, the indicator results corresponding to the indicator behavior log are obtained, including: Based on the indicator formula interface and the field type corresponding to the indicator formula interface, the corresponding indicator formula is called to calculate the field result of the field corresponding to the indicator formula interface in the indicator behavior log; Calculate and obtain the field results of each field in the indicator behavior log; Substitute all field results into the combination operation formulas between indicator formulas to obtain the indicator results corresponding to the indicator behavior log.
7. A behavior log-based indicator generation system, characterized in that: The system comprises: The acquisition module is used to pre-process the initial behavior log within a preset time period to obtain the network behavior log; The condition configuration module is used to obtain filtering conditions and grouping conditions through the preset configuration interface; The formula configuration module is used to obtain the indicator name, the indicator formula interface corresponding to each field type, and the combination operation formula between indicator formulas through the preset indicator input interface; The file generation module is used to generate or update the front-end JSON file based on the data obtained from the preset configuration interface and the preset indicator input interface; The program calling module is used to convert the front-end JSON file into a SPARK executable logic program; through the SPARK big data engine, the logic program is called to obtain the indicator behavior logs that meet the filtering conditions and grouping conditions from several network behavior logs; according to the indicator formula corresponding to the indicator formula interface, the field results of each field in the indicator behavior log are calculated; according to the combined operation formula between the indicator formulas, the indicator results corresponding to the indicator behavior log are obtained; and the indicator results are stored in the preset database.
8. The indicator generation system based on behavior log according to claim 7 is characterized in that: The acquisition module includes an acquisition unit, Used to collect a number of initial network behavior logs within a preset time period through a preset log collection tool; According to the preset filtering conditions, the logs that do not meet the preset filtering conditions are removed from the initial network behavior logs to obtain the network behavior logs; Convert unstructured network behavior logs into structured network behavior logs; According to the source of the network behavior log, the log type of each network behavior log is identified to obtain the network behavior log.
9. A device for generating indicators based on behavior logs, characterized in that: The device comprises: processor; and a memory having executable codes stored thereon, which, when executed, causes the processor to execute a method for generating indicators based on behavior logs as described in any one of claims 1 to 6.
10. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the method for generating indicators based on behavior logs as described in any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Business service system oriented to log big-data analysis
CN107577805A
Log data processing method and device
CN111506479A