A dynamic self-adaptive application audit log collection transmission method and device

Abstract

The application discloses a kind of dynamic self-adapting application audit log collection transmission method and device.The method includes: deploying audit agent to target application server, and register to audit server, accept audit center management;In response to console instruction, audit server generates and issues log collection strategy;According to log collection strategy, application audit log is collected by audit agent;Matching the event state attribute recorded by the application audit log collected, determine log transmission mode;For normal state log, according to the recent log package size of audit agent, application priority and current network state, dynamically calculate log transmission parameter and audit agent transmission queue;In turn, application audit log is transmitted to audit server, so that the centralized storage and statistical analysis of application audit log by audit server.The application can realize log collection centralized management scheduling, and determines transmission mode according to the event state attribute recorded by application audit log.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of log collection and transmission, and in particular to a dynamic adaptive application audit log collection and transmission method and device. BACKGROUND

[0002] National standards and industry standards continue to improve the requirements of application security audit, and the real-time transmission pressure of massive log data is also increasing. At present, most Internet companies prefer to use distributed components when building log collection and management systems. The mainstream collection method is to install an agent on the host that generates logs, and each log host pushes log information to a message queue; only when the application server does not support the installation of an agent, will the pull method be used to collect. But these technologies usually lack centralized management of log collection agents and centralized scheduling of log transmission tasks, and lack different processing strategies for different emergency levels of logs.

[0003] Therefore, in the face of the problem that log real-time push technology may cause high network load, and the problem that the log audit center lacks effective management and control of the life state of the agent, a dynamic adaptive application audit log collection and transmission scheme is proposed, which collects application audit logs by using the methods of centralized management and control of audit agents, dynamic adaptive pre-arrangement of log transmission queue and parameters, and hierarchical upload of different types of logs, while meeting the requirements of application security audit, balancing the real-time requirements of audit log collection and transmission, and the network resource utilization efficiency. SUMMARY

[0004] The purpose of the embodiments of the present specification is to solve the above problems, and to provide a dynamic adaptive application audit log collection and transmission method, device, electronic equipment and storage medium.

[0005] To solve the above technical problems, the embodiments of the present specification are implemented as follows:

[0006] In a first aspect, a dynamic adaptive application audit log collection and transmission method is provided, comprising:

[0007] Deploying an audit agent to a target application server and registering it to an audit server to accept management by the audit center;

[0008] In response to the instruction of the console, the audit server generates and publishes a log collection strategy;

[0009] According to the log collection strategy, the audit agent collects the application audit logs of the target application server;

[0010] Matching the state attribute of the event recorded by the collected application audit logs, determining the log transmission mode;

[0011] For the normal state of the log, according to the recent log package size of each audit agent, application priority and current network connection state, the log transmission parameters are dynamically calculated and the audit agent transmission queue is constructed;

[0012] The application audit log is transmitted to the audit server in sequence, so that the audit server centrally stores and statistically analyzes the application audit log.

[0013] Further, the audit agent is deployed to the target application server and registered to the audit server, and accepts the management of the audit center, including:

[0014] The audit agent sends a registration request to the audit server, and the registration request contains the audit agent identifier, the target application server identifier and the identity information of the audit agent;

[0015] After the audit server verifies the identity of the audit agent, the registration is completed by the control console and the public key is allocated to the audit agent;

[0016] The audit agent sends heartbeat information to the audit server at regular intervals, so that the audit server can monitor the survival state of the audit agent and the network connection state between the target server in real time.

[0017] Further, in response to the instruction of the control console, the audit server generates and publishes the log collection strategy, including:

[0018] The control console initiates the audit strategy instruction to the audit server;

[0019] The audit server generates the log collection strategy according to the audit strategy instruction and sends it to the audit agent; wherein the log collection strategy corresponds to the registered audit agent one by one.

[0020] Further, when the state attribute of the event recorded by the application log collected by the audit agent is normal, the log transmission mode is determined to be a timed queue pull, and the audit agent transmits the normal application audit log to the audit server, including:

[0021] The audit agent generates a log digest corresponding to the normal application audit log collected within a specified period; wherein the log digest includes log type, log transmission priority, application priority, log data volume and network connection state;

[0022] The log digest is sent to the audit server, and the audit server sends the digest content of all audit agents to the control console;

[0023] The control console adopts a HyperBand method to calculate log transmission parameters and a transmission queue composed of all audit agents according to recent log packet sizes, application priorities and current network connection states of each audit agent, and to assign a transmission time period including a transmission start time and a maximum transmission duration to each audit agent in the transmission queue.

[0024] The control console sends a log pulling task containing the log transmission parameters and the transmission queue to the audit server.

[0025] When the transmission start time of the transmission time period is reached, the audit server sends a log pulling instruction to the audit agent to receive the normal application audit log and save it to a log database. If the log data volume specified in the log summary has not been completely transmitted after the maximum transmission duration, the audit server sends a transmission end instruction to the audit agent, and the audit agent records the log data that has not been received by the audit server in this transmission and waits for retransmission in the next transmission.

[0026] Further, when the state attribute of the event recorded in the application log collected by the audit agent is an emergency, the log transmission mode is determined to be real-time pushing, and the audit agent installed on the application server transmits the emergency application audit log to the audit server, including:

[0027] The audit agent transmits the emergency application audit log to the audit server in real time.

[0028] After receiving the emergency application audit log, the audit server replies an acknowledgement message to the audit agent, and then generates corresponding alarm information according to the emergency application audit log. If the audit agent does not receive the acknowledgement message, the audit agent retransmits the emergency application audit log.

[0029] Further, the control console can also control the start and / or dormancy of the audit agent through the audit server.

[0030] Further, the control console can also control the start and / or dormancy of the audit agent through the audit server; and / or, a gRPC framework is adopted to establish data and instruction communication connections among the audit server, the audit agent and the control console.

[0031] In a second aspect, a dynamic and adaptive application audit log collection and transmission device is provided, including:

[0032] A first module is configured to deploy an audit agent to a target application server, register to an audit server and accept management of an audit center.

[0033] The second module is configured to generate and publish a log collection strategy by the audit server in response to an instruction of the console;

[0034] The third module is configured to collect application audit logs of the target application server by the audit agent according to the log collection strategy;

[0035] The fourth module is configured to determine a log transmission mode by matching a state attribute of an event recorded by the collected application audit logs;

[0036] The fifth module is configured to dynamically calculate log transmission parameters and build an audit agent transmission queue according to a recent log packet size, an application priority and a current network connection state of each audit agent for logs in a normal state;

[0037] The sixth module is configured to sequentially transmit the application audit logs to the audit server so that the audit server centrally stores and statistically analyzes the application audit logs.

[0038] The fourth aspect provides a computer readable storage medium, characterized in that the computer readable storage medium stores one or more programs, and the one or more programs, when executed by an electronic device including a plurality of application programs, cause the electronic device to perform the dynamic self-adaptive application audit log collection and transmission method of the first aspect.

[0039] The present specification can achieve at least the following technical effects:

[0040] The embodiment of the present application first deploys an audit agent to a target application server and registers the audit agent to an audit server to accept management of the audit center; the audit server generates and publishes a log collection strategy in response to an instruction of a console; the audit agent collects application audit logs according to the log collection strategy; a log transmission mode is determined by matching a state attribute of an event recorded by the collected application audit logs; for logs in a normal state, a HyperBand method is used to dynamically calculate log transmission parameters and an audit agent transmission queue according to a recent log packet size, an application priority and a current network state of the audit agent; and the application audit logs are sequentially transmitted to the audit server so that the audit server centrally stores and statistically analyzes the application audit logs. The embodiment of the present application can realize centralized management of the audit agent and centralized scheduling of log transmission tasks, and different transmission modes are adopted for different state attributes of events recorded by different application audit logs, thereby effectively balancing system network and workloads of the centralized audit server at different times. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Figure 1 One of the dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0043] Figure 2 The second dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0044] Figure 3 The third dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0045] Figure 4 The fourth dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0046] Figure 5 The fifth dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0047] Figure 6 The sixth dynamic adaptive application audit log collection and transmission method schematic diagram provided by the embodiments of the present specification.

[0048] Figure 7 The dynamic adaptive application audit log collection and transmission device schematic diagram provided by the embodiments of the present specification.

[0049] Figure 8 The structure schematic diagram of an electronic device provided by one of the embodiments of the present specification.

[0050] Figure 9 The workflow diagram when the state attribute of the event recorded by the application log collected by the audit agent is an emergency handling.

[0051] Figure 10 The workflow diagram when the original application event recorded by the emergency log is an abnormal event that may affect the running or execution of the application program or normal business function and needs real-time processing.

[0052] Figure 11The structural schematic diagram of a dynamic adaptive application audit log collection and transmission device provided for an embodiment of the present specification.

[0053] Figure 12 The structural schematic diagram of an electronic device provided for an embodiment of the present specification. DETAILED DESCRIPTION

[0054] In order for those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the present specification will be described clearly and completely in the following with reference to the drawings in the embodiments of the present specification. Obviously, the described embodiments are only some of the embodiments of the present specification, not all. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the scope of protection of the present specification.

[0055] As described in the background, with the increasing requirements of national standards and industry standards for application security audit, information systems need reliable and efficient application log collection and transmission methods, especially involving centralized management and unified collection of security logs and business logs of multiple application software in the system. At the same time, the real-time generation of massive logs by application software has brought great pressure to data transmission.

[0056] Currently, the mainstream log collection method in the industry is to install a log collection agent on the host that generates logs, and each agent pushes the collected log records to a message queue, and then an audit node takes out the logs from the message queue and performs audit analysis. Generally, only when the application server does not support the installation of log collection, the pull method is used to collect logs scattered on each server. However, these technologies usually lack centralized management of log collection agents and scheduling of log transmission tasks, and do not consider taking different transmission methods for normal logs and urgent logs.

[0057] Therefore, in order to overcome the problem that the log real-time pushing technology may cause high network load, and the problem that the log audit center lacks effective management control of the agent life state, an application audit log collection and transmission scheme is proposed. The basic idea is to deploy an audit agent on each application server in the information system, and collect normal logs and abnormal logs generated by the target application through centralized management and unified control of the audit agent by the audit server. For normal logs that do not need real-time processing, a fixed time slice transmission method is used to reasonably utilize network idle and audit server idle for centralized transmission. Before centralized transmission, the HyperBand method is used to dynamically calculate log transmission parameters and the transmission queue of the audit agent according to the recent log packet size, application priority and current network connection state of each audit agent; and then the application audit logs are transmitted to the audit server in sequence, so that the audit server can centrally store and statistically analyze the application audit logs. For abnormal logs that need real-time processing, a real-time pushing method is used, and the audit agent reports to the centralized audit server immediately after discovery.

[0058] The application audit log collection and transmission scheme involved in the present specification will be described in detail below through specific examples.

[0059] Embodiment one

[0060] As shown in the figure, a dynamic adaptive application audit log collection and transmission method according to an embodiment of the present application comprises: Figure 1

[0061] S101: Deploy an audit agent to a target application server, and register with an audit server to accept management by the audit center.

[0062] S102: In response to an instruction from a console, the audit server generates and issues a log collection strategy.

[0063] S103: According to the log collection strategy, the audit agent collects application audit logs of the target application server.

[0064] S104: Determine the log transmission method according to the state attribute of the event recorded in the collected application audit logs.

[0065] S105: For normal state logs, use the HyperBand method to dynamically calculate log transmission parameters and build an audit agent transmission queue according to the recent log packet size, application priority and current network connection state of each audit agent.

[0066] S106: Transmit the application audit logs to the audit server in sequence, so that the audit server can centrally store and statistically analyze the application audit logs.

[0067] ​In some embodiments, the gRPC framework is adopted to implement the communication of instructions and data between the audit center and the audit agent, wherein the audit center is composed of the console and the audit server. gRPC is an open-source high-performance remote procedure call (RPC) framework that can run in any environment, efficiently connecting services within and across data centers, supporting load balancing, tracing, health checking, and authentication; it is also suitable for connecting devices, mobile applications, and browsers to backend services in distributed computing. It should be noted that the gRPC framework is only one of the ways to establish the data and instruction communication connection between the audit server, the audit agent, and the console, and any way that can meet the system requirements and network load requirements to establish the above data and instruction communication connection is within the protection scope of the embodiments of the present application.

[0068] In some embodiments, the audit center can also control the startup and / or hibernation of the audit agent. Specifically, the console initiates a startup or hibernation instruction, the audit server receives and analyzes the startup or hibernation instruction initiated by the console, and forwards it to the corresponding audit agent, and the audit agent receives and executes the startup or hibernation operation. The above workflow is shown in Figure 2

[0069] In some embodiments, the audit agent is deployed to the target application server and registered with the audit server to accept the management of the audit center, as shown in Figure 3 , including:

[0070] S311: The audit agent sends a registration request to the audit server, and the registration request contains the audit agent identifier, the target application server identifier, and the identity information of the audit agent.

[0071] S312: After verifying the identity of the audit agent, the audit server completes the registration by the console and allocates a public key for the audit agent.

[0072] S313: The audit agent sends heartbeat information to the audit server at regular intervals, so that the audit server can monitor the survival state of the audit agent and the network connection state between the target server in real time.

[0073] ​Specifically, the audit agent is deployed on the application server, and after installation, it automatically sends a registration request to the audit server to apply to join the system, and the registration request contains the audit agent identifier, the target application server identifier, and the identity information of the audit agent. The audit server checks the agent identity, and if the verification is passed, it registers the audit agent on the console and completes the public key allocation of the agent. After the audit agent joins the system, it sends heartbeat information to the audit server at regular intervals, and each heartbeat information includes agent identifier, heartbeat count, heartbeat sending time, etc. The audit server receives the heartbeat information sent by the agent and records it in real time. When N consecutive heartbeat periods (the value of N is set according to the specific situation of the system and network) cannot receive any message from the audit agent, it is determined that the audit agent has failed, and an alarm is immediately sent. The above workflow is shown in Figure 4 .

[0074] In some embodiments, in response to the instruction of the console, the audit server generates and publishes the log collection strategy, as shown in Figure 5 , which includes:

[0075] S511: The console initiates an audit policy instruction to the audit server.

[0076] S512: The audit server generates the log collection strategy according to the audit policy instruction and sends it to the audit agent; wherein the log collection strategy corresponds to the registered audit agent one by one.

[0077] Specifically, the console initiates the audit policy, the audit server receives and analyzes the audit policy sent by the console, and generates the log collection strategy accordingly; the audit server transmits the log collection strategy to the corresponding audit agent, and the audit agent updates the log collection strategy accordingly. The above workflow is shown in Figure 6 .

[0078] In some embodiments, when the state attribute of the event recorded in the application log collected by the audit agent is normal, the log transmission mode is determined to be a timed queue pull, and the audit agent transmits the normal application audit log to the audit server, as shown in Figure 7 , which includes:

[0079] S711: The audit agent generates a log summary corresponding to the normal application audit log collected within a specified period; wherein the log summary includes log type, log transmission priority, application priority, log data volume, and network connection state.

[0080] S712: Send the log summary to the audit server, and the audit server sends the summary content of all audit agents to the console.

[0081] S713: The control console adopts the HyperBand method to calculate log transmission parameters and a transmission queue composed of all audit agents according to the recent log package size, application priority and current network connection state of each audit agent, and allocates a transmission time period including a transmission start time and a maximum transmission connection duration to each audit agent in the transmission queue.

[0082] S714: The control console sends a log pulling task containing log transmission parameters and a transmission queue to the audit server.

[0083] S715: When the transmission start time of the transmission time period is reached, the audit server sends a log pulling instruction to the audit agent to receive the normal application audit log and save it to the log database. If the specified log data volume in the log summary has not been completely transmitted after the maximum transmission connection duration, the audit server sends a transmission end instruction to the audit agent, and the audit agent records the log data that has not been received by the audit server in this transmission and waits to be retransmitted next time.

[0084] Specifically, for the audit agents in the starting state, each audit agent reports the summary information of the application log collected in the past period of time to the centralized audit server at a fixed time (for example, 3:00 every day), and informs the audit server of the type, priority and data volume of the application log collected by itself. The audit server uploads the log summaries from all audit agents to the control console. The control console adopts the HyperBand method to dynamically calculate log transmission parameters and a transmission queue composed of all audit agents according to the recent log package size, application priority and current network connection state of each audit agent, and allocates a transmission time period including a transmission start time and a maximum transmission connection duration to each audit agent in the transmission queue.

[0085] The task scheduling is performed, and a transmission time period including a transmission start time and a maximum transmission connection duration is allocated to each pulling task. When the transmission start time allocated to each agent is reached, the audit server sends a log pulling instruction to the corresponding agent to start receiving the log data uploaded by the agent and save it to the log database. If the specified log data volume in the log summary has not been completely transmitted after the maximum transmission connection duration, the audit server sends a transmission end instruction to the audit agent, and the audit agent records the log data that has not been received by the audit server in this transmission and waits to be retransmitted next time. The above workflow is shown in Figure 8

[0086] ​In some embodiments, when the state attribute of the event recorded by the application log collected by the audit agent is in need of emergency handling, the log transmission mode is determined to be real-time pushing, and the audit agent installed on the application server transmits the emergency application audit log to the audit server, such as Figure 9 as shown, comprising:

[0087] S911: The audit agent transmits the emergency application audit log to the audit server in real time.

[0088] S912: After the audit server receives the emergency application audit log, the audit server replies to the audit agent with an acknowledgement message, and then generates corresponding alarm information according to the emergency application audit log; if the audit agent does not receive the acknowledgement message, the audit agent retransmits the emergency application audit log.

[0089] Specifically, the original application event recorded by the emergency log is an abnormal event that may affect the running or execution of the application program or normal business functions and needs to be handled in real time. Once the audit agent in the starting state collects such a log, it immediately uploads it to the audit server in a pushing manner. After receiving the emergency log pushed by the audit agent, the audit server replies to the agent with an acknowledgement message, and then forwards it to the alarm module to generate alarm information. If the audit agent does not receive the acknowledgement, the corresponding emergency log should be retransmitted. The above workflow is shown in Figure 10 .

[0090] The embodiment of the present application first deploys the audit agent to the target application server and registers it to the audit server to accept management by the audit center; in response to the instruction of the console, the audit server generates and publishes a log collection strategy; according to the log collection strategy, the audit agent collects the application audit log; matches the state attribute of the event recorded by the collected application audit log to determine the log transmission mode; for normal state logs, the HyperBand method is used to dynamically calculate the log transmission parameters and the audit agent transmission queue according to the recent log packet size of the audit agent, the application priority and the current network state; the application audit log is transmitted to the audit server in sequence, so that the audit server can centrally store and statistically analyze the application audit log. The embodiment of the present application can realize centralized management of the audit agent and centralized scheduling of the log transmission task, and different transmission modes are adopted according to the state attribute of the event recorded by the different application audit logs, effectively balancing the system network and the workload of the centralized audit server at different times.

[0091] Embodiment Two

[0092] Figure 11 The structural schematic diagram of the dynamic adaptive application audit log collection and transmission device 1100 provided for an embodiment of the present application is shown in Figure 11In an embodiment, the dynamic adaptive application audit log collection and transmission device 1100 comprises:

[0093] The first module 1101 is configured to deploy an audit agent to a target application server and register to an audit server, and accept the management of the audit center.

[0094] The second module 1102 is configured to generate and issue a log collection strategy by the audit server in response to the instruction of the console.

[0095] The third module 1103 is configured to collect the application audit log of the target application server by the audit agent according to the log collection strategy.

[0096] The fourth module 1104 is configured to determine the log transmission mode by matching the state attribute of the event recorded in the collected application audit log.

[0097] The fifth module 1105 is configured to dynamically calculate the log transmission parameters and build the audit agent transmission queue according to the recent log packet size, application priority and current network connection state of each audit agent by using the HyperBand method for the log of normal state.

[0098] The sixth module 1106 is configured to transmit the application audit log to the audit server in sequence, so that the audit server centrally stores and statistically analyzes the application audit log.

[0099] It should be understood that the dynamic adaptive application audit log collection and transmission device 1100 of the embodiment of the present specification can also perform the method executed by the dynamic adaptive application audit log collection and transmission device (or equipment) and realize the functions of the dynamic adaptive application audit log collection and transmission device (or equipment) 1100 in the examples shown in the present specification, which will not be described here. Figures 1 to 10 Figures 1 to 10 The dynamic adaptive application audit log collection and transmission device (or equipment) performs the method and realizes the functions of the dynamic adaptive application audit log collection and transmission device (or equipment) 1100 in the examples shown in the present specification, which will not be described here.

[0100] Embodiment Three

[0101] Figure 12 is a structural schematic diagram of an electronic device of an embodiment of the present specification. Please refer to Figure 12 At the hardware level, the electronic device comprises a processor, and optionally further comprises an internal bus, a network interface, and a memory. The memory can include a memory such as a high-speed random-access memory (RAM), and can also include a non-volatile memory such as at least one disk memory. Of course, the electronic device can also include other hardware required by the business.

[0102] ​The processor, the network interface and the memory can be connected with each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 12 Only one bidirectional arrow is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0103] The memory is used to store programs. Specifically, the program can include program code including computer operation instructions. The memory can include an internal memory and a non-volatile memory, and provide instructions and data for the processor.

[0104] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs, and forms a shared resource access control device at a logical level. The processor executes the program stored in the memory, and is specifically used for executing the following operations:

[0105] Deploy the audit agent to the target application server, and register with the audit server to accept the management of the audit center;

[0106] In response to the instruction of the console, the audit server generates and publishes a log collection strategy;

[0107] According to the log collection strategy, the audit agent collects the application audit log of the target application server;

[0108] Determine the log transmission mode according to the state attribute of the event recorded in the collected application audit log;

[0109] For normal state logs, the HyperBand method is used to dynamically calculate log transmission parameters and build an audit agent transmission queue according to the recent log packet size, application priority and current network connection state of each audit agent;

[0110] In succession, the application audit log is transmitted to the audit server, so that the audit server can centrally store and statistically analyze the application audit log.

[0111] The above as described in the specification Figures 1 to 10The dynamic self-adaptive application audit log collection and transmission method disclosed by the embodiment can be applied to a processor or implemented by the processor. The processor can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits in the hardware of the processor or instructions in the form of software. The above processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiments of the present specification can be implemented or executed. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present specification can be directly embodied as a hardware decoding processor for execution, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0112] Of course, in addition to the software implementation, the electronic device of the embodiments of the present specification does not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0113] Embodiment four

[0114] The embodiments of the present specification also propose a computer readable storage medium storing one or more programs, the one or more programs including instructions capable of causing the portable electronic device including a plurality of application programs to execute the following steps when executed by the portable electronic device: Figures 1 to 10 The dynamic self-adaptive application audit log collection and transmission method of the embodiment, and specifically for executing the following steps:

[0115] Deploy the audit agent to the target application server and register to the audit server to accept the management of the audit center;

[0116] In response to the instruction of the console, the audit server generates and publishes a log collection strategy;

[0117] According to the log collection strategy, the audit agent collects the application audit log of the target application server;

[0118] The log transmission mode is determined according to the state attribute of the event recorded in the collected application audit log;

[0119] For the normal state log, the HyperBand method is used to dynamically calculate the log transmission parameters and build the audit agent transmission queue according to the recent log packet size, application priority and current network connection state of each audit agent;

[0120] The application audit log is sequentially transmitted to the audit server for centralized storage and statistical analysis.

[0121] In summary, the above only describes the preferred embodiments of the present specification, and is not intended to limit the protection scope of the present specification. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present specification shall be included in the protection scope of the present specification.

[0122] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an electronic data carrier device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0123] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0124] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0125] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

Claims

1. A dynamic, self-adapting application audit log collection and transmission method, characterized in that, The application comprises the following steps: deploying an audit agent to a target application server and registering to an audit server to accept management of an audit center; in response to an instruction of a console, the audit server generates and issues a log collection strategy; according to the log collection strategy, the audit agent collects application audit logs of the target application server; determining a log transmission mode according to a state attribute of an event recorded in the collected application audit logs; for normal state logs, dynamically calculating log transmission parameters and constructing an audit agent transmission queue according to recent log packet size, application priority and current network connection state of each audit agent; transmitting application audit logs to the audit server in sequence so that the audit server centrally stores and statistically analyzes the application audit logs; when the state attribute of the event recorded in the application audit logs collected by the audit agent is normal, determining that the log transmission mode is a timed queue pull, and the audit agent transmits normal application audit logs to the audit server, comprising: the audit agent generates a log digest corresponding to the normal application audit logs collected within a specified period; wherein the log digest comprises log type, log transmission priority, application priority, log data volume and network connection state; sending the log digest to the audit server, and the audit server sends the digest content of all audit agents to the console; the console uses a HyperBand method to calculate log transmission parameters and a transmission queue composed of all audit agents according to recent log packet size, application priority and current network connection state of each audit agent, and allocates a transmission time period to each audit agent in the transmission queue, the transmission time period comprising a transmission start time and a longest transmission connection duration; the console sends a log pull task containing log transmission parameters and a transmission queue to the audit server; when the transmission start time of the transmission time period is reached, the audit server sends a log pull instruction to the audit agent to receive the normal application audit logs and save them to a log database; if the specified log data volume in the log digest has not been completely transmitted after the longest transmission connection duration, the audit server sends a transmission end instruction to the audit agent, and the audit agent records the log data not received by the audit server in this transmission and waits for retransmission in the next transmission.

2. The dynamic, self-adapting application audit log collection and transfer method of claim 1, wherein, deploying an audit agent to a target application server and registering to an audit server to accept management of an audit center, comprising: the audit agent sends a registration request to the audit server, and the registration request contains an audit agent identifier, a target application server identifier and identity information of the audit agent; after verifying the identity of the audit agent, the audit server completes registration by the console and allocates a public key to the audit agent; the audit agent regularly sends heartbeat information to the audit server to monitor the survival state of the audit agent and the network connection state between the target server and the audit agent in real time.

3. The method of claim 1, wherein, In response to an instruction of the console, the audit server generates and publishes a log collection strategy, comprising: The console initiates an audit strategy instruction to the audit server; The audit server generates the log collection strategy according to the audit strategy instruction and sends it to the audit agent; wherein the log collection strategy corresponds to the registered audit agent one by one.

4. The method of claim 1, wherein, When the state attribute of the event recorded by the application audit log collected by the audit agent is an emergency, the log transmission mode is determined to be real-time pushing, and the audit agent installed on the application server transmits the emergency application audit log to the audit server, comprising: The audit agent sends the emergency application audit log to the audit server in real time; After receiving the emergency application audit log, the audit server replies an acknowledgement message to the audit agent, and then generates corresponding alarm information according to the emergency application audit log; if the audit agent does not receive the acknowledgement message, it re-sends the emergency application audit log.

5. The method of claim 1, wherein, The console can also control the start and / or dormancy of the audit agent through the audit server.

6. The dynamic adaptive application audit log collection and transfer method according to any of claims 1 to 5, characterized in that, The gRPC framework is adopted to establish data and instruction communication connection among the audit server, the audit agent and the console.

7. A dynamic, self-adapting application audit log collection and transmission apparatus, characterized by Comprise: The first module is configured to deploy the audit agent to the target application server and register to the audit server to accept the management of the audit center; The second module is configured to generate and publish the log collection strategy by the audit server in response to the instruction of the console; The third module is configured to collect the application audit log of the target application server by the audit agent according to the log collection strategy; The fourth module is configured to match the state attribute of the event recorded by the collected application audit log to determine the log transmission mode; The fifth module is configured to dynamically calculate the log transmission parameters and build the audit agent transmission queue according to the recent log packet size, application priority and current network connection state of each audit agent for the log of normal state; The sixth module is configured to transmit the application audit log to the audit server in sequence so that the audit server can centrally store and statistically analyze the application audit log.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the dynamic self-adaptive application audit log collection and transmission method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the dynamic self-adaptive application audit log collection and transmission method according to any one of claims 1 to 6.

Images