Log data output method and device, electronic equipment and storage medium

Abstract

The invention discloses a log data output method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining to-be-output log data; analyzing and processing the to-be-output log data, and determining a log level corresponding to the to-be-output log data; determining target output log data from the to-be-output log data according to a log output judgment condition corresponding to the log level; storing the target output log data into a target cache queue meeting a preset queue capacity threshold value; reading the target output log data in the target cache queue according to a preset time sequence; and determining a log output mode of the target output log data according to the current system disk space capacity, and outputting the target output log data according to the log output mode. Automatic management of log output can be achieved, unnecessary log output is reduced, occupation of system resources by log output is reduced, the overall system performance is improved, the system resource utilization rate is increased, and the method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for outputting log data. Background Art

[0002] With the rapid development of information technology, servers have become an important infrastructure for enterprise-level applications and Internet services. Servers generate a large amount of log information during operation, which plays an important role in system monitoring, troubleshooting, performance analysis, security auditing, etc. However, traditional log management technology faces huge challenges and limitations. For example, the storage of a large amount of log data will put a huge pressure on the server's storage resources and affect the overall performance of the server; traditional log management systems usually use fixed log level settings, which may result in excessive unnecessary information output when the system is running normally, and may lose key information when problems occur in the system.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The embodiments of the present application aim to solve at least one of the technical problems in the related art to a certain extent. To this end, the main purpose of the embodiments of the present application is to propose a method, device, electronic device and storage medium for outputting log data, which can realize the automatic management of log output, reduce unnecessary log output, and reduce the occupation of system resources by log output, thereby improving the overall system performance and improving the utilization rate of system resources.

[0005] To achieve the above purpose, an embodiment of the present application provides a method for outputting log data, the method comprising the following steps:

[0006] Get the log data to be output;

[0007] Analyze and process the log data to be output, and determine the log level corresponding to the log data to be output;

[0008] Determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level;

[0009] Storing the target output log data in a target cache queue that meets a preset queue capacity threshold;

[0010] Read the target output log data in the target cache queue in a preset time sequence;

[0011] The log output mode of the target output log data is determined according to the current system disk space capacity, and the target output log data is output according to the log output mode.

[0012] In some embodiments, analyzing and processing the log data to be output to determine the log level corresponding to the log data to be output includes:

[0013] Analyze and process the log data to be output to obtain target log feature data of the log data to be output; wherein the target log feature data includes at least one of the following: the severity of the log information of the log data to be output and the importance of the log information of the log data to be output;

[0014] The log level corresponding to the log data to be output is determined according to the severity of the log information and the importance of the log information of the log data to be output.

[0015] In some embodiments, determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level includes:

[0016] Get current system resource usage information;

[0017] Determining the log output judgment condition corresponding to the log level according to the log level corresponding to the log data to be output;

[0018] According to the current system resource usage information, determining whether the log data to be output meets the log output determination condition corresponding to the log level;

[0019] If the log data to be output meets the log output judgment condition corresponding to the log level, the target output log data is determined from the log data to be output.

[0020] In some embodiments, the method further comprises:

[0021] If the log data to be output does not meet the log output judgment condition corresponding to the log level, the log output process corresponding to the log data to be output is terminated.

[0022] In some embodiments, before storing the target output log data into a target cache queue that meets a preset queue capacity threshold, the method further includes:

[0023] If the current queue usage capacity of the current cache queue is less than the preset queue usage threshold, determining the current cache queue as the target cache queue that meets the preset queue capacity threshold;

[0024] If the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is greater than or equal to the preset system memory capacity threshold, the current cache queue is expanded according to the preset queue expansion threshold, and the expanded current cache queue is determined as the target cache queue that meets the preset queue capacity threshold;

[0025] If the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is less than the preset system memory capacity threshold, the log output process corresponding to the log data to be output is terminated.

[0026] In some embodiments, the method further comprises:

[0027] Get the current queue usage capacity of the current cache queue at a preset time interval;

[0028] If the current queue usage capacity is less than a preset capacity reduction threshold, the queue cache capacity of the current cache queue is reduced.

[0029] In some embodiments, determining the log output mode of the target output log data according to the current system disk space capacity, and outputting the target output log data according to the log output mode, includes:

[0030] If the current system disk space capacity is less than or equal to the first preset system disk space threshold, outputting the target output log data to the console;

[0031] If the current system disk space capacity is greater than the first preset system disk space threshold, and the current system disk space capacity is less than or equal to the second preset system disk space threshold, the target output log data is stored in the system disk file according to a circular overwriting method, and the target output log data in the system disk file is output to the console;

[0032] If the current system disk space capacity is greater than the second preset system disk space threshold, the target output log data is stored in the system disk file according to a sequential append method, and the target output log data in the system disk file is output to the console.

[0033] To achieve the above purpose, another aspect of the embodiment of the present application provides a device for outputting log data, the device comprising the following modules:

[0034] A module for obtaining log data to be output, used for obtaining log data to be output;

[0035] A log level determination module, used to analyze and process the log data to be output, and determine the log level corresponding to the log data to be output;

[0036] A target output log data determination module, used to determine target output log data from the log data to be output according to the log output judgment condition corresponding to the log level;

[0037] A log data cache module, used for storing the target output log data into a target cache queue that meets a preset queue capacity threshold;

[0038] A target output log data reading module, used for reading the target output log data in the target cache queue in a preset time sequence;

[0039] The target output log data output module is used to determine the log output mode of the target output log data according to the current system disk space capacity, and output the target output log data according to the log output mode.

[0040] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0042] The embodiments of the present application include at least the following beneficial effects: the present application provides a method, device, electronic device and storage medium for outputting log data, the scheme acquires log data to be output; analyzes and processes the log data to be output to determine the log level corresponding to the log data to be output; determines target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; stores the target output log data in a target cache queue that meets a preset queue capacity threshold; reads the target output log data in the target cache queue in a preset time sequence; determines the log output mode of the target output log data according to the current system disk space capacity, and outputs the target output log data according to the log output mode. The embodiment of the present application dynamically adjusts the log output level through intelligent analysis, realizes the refined classification of log information, can reduce unnecessary log output, and reduce the occupation of system resources by log output, thereby effectively improving the overall system performance and improving the utilization rate of system resources; at the same time, based on the preset log output judgment strategy, the target output log data that needs to be retained or processed is automatically screened out, effectively reducing the redundancy of log data, enabling the system to record key information at critical moments, and providing convenience for subsequent troubleshooting and analysis work; and, using the cache queue as a temporary storage area for log data, the writing and reading of log data can be effectively managed to avoid system performance problems caused by large amounts of log data; by dynamically evaluating the system disk space capacity, the most appropriate log output method is intelligently selected, which not only ensures the reliable storage of log data, but also avoids system problems caused by insufficient system resources. By automatically managing log output, the log management process is simplified and the log management efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the steps of the method for outputting log data provided by an embodiment of the present application;

[0044] Figure 2 It is a flowchart of a method for outputting log data provided in an embodiment of the present application;

[0045] Figure 3 It is a structural schematic diagram of a log data output device provided in an embodiment of the present application;

[0046] Figure 4 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0048] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0049] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] With the rapid development of information technology, servers have become an important infrastructure for enterprise-level applications and Internet services. Servers generate a large amount of log information during operation, which plays an important role in system monitoring, troubleshooting, performance analysis, security auditing, etc. However, traditional log management technology faces the following challenges and limitations:

[0052] (1) Huge log volume: In a high-concurrency, high-load server environment, logs are generated very quickly and the log volume is huge. If a large amount of log data is directly stored, it will cause huge pressure on the server's storage resources.

[0053] (2) Fixed log level: Traditional log management systems usually use fixed log level settings, such as DEBUG, INFO, WARNING, ERROR, etc. Static log level settings cannot be adjusted dynamically, resulting in excessive unnecessary information being output when the system is operating normally, and key information being lost when problems occur in the system.

[0054] (3) Performance impact: Log output is a resource-consuming process. Especially when a large number of log writing operations are involved, it will significantly increase the burden on the CPU (Central Processing Unit) and I / O (Input / Output), thereby affecting the overall performance of the server.

[0055] (4) Difficult log management: Administrators need to manually intervene to adjust log levels, which is a tedious and error-prone task in large distributed systems.

[0056] In view of this, the embodiments of the present application provide a method, device, electronic device and storage medium for outputting log data. The scheme obtains log data to be output; analyzes and processes the log data to be output to determine the log level corresponding to the log data to be output; determines target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; stores the target output log data in a target cache queue that meets a preset queue capacity threshold; reads the target output log data in the target cache queue in a preset time sequence; determines the log output method of the target output log data according to the current system disk space capacity, and outputs the target output log data according to the log output method. The embodiment of the present application dynamically adjusts the log output level through intelligent analysis, realizes the refined classification of log information, can reduce unnecessary log output, and reduce the occupation of system resources by log output, thereby effectively improving the overall system performance and improving the utilization rate of system resources; at the same time, based on the preset log output judgment strategy, the target output log data that needs to be retained or processed is automatically screened out, effectively reducing the redundancy of log data, enabling the system to record key information at critical moments, and providing convenience for subsequent troubleshooting and analysis work; and, using the cache queue as a temporary storage area for log data, the writing and reading of log data can be effectively managed to avoid system performance problems caused by large amounts of log data; by dynamically evaluating the system disk space capacity, the most appropriate log output method is intelligently selected, which not only ensures the reliable storage of log data, but also avoids system problems caused by insufficient system resources. By automatically managing log output, the log management process is simplified and the log management efficiency is improved.

[0057] The output method of log data provided by the embodiment of the present application relates to the field of data processing technology. The output method of log data provided by the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network) and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms, and the server can also be a node server in a blockchain network; the software can be an application that implements the output method of log data, etc., but is not limited to the above form.

[0058] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs (personal computers), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0059] See also Figure 1 , Figure 1 is an optional step flow chart of the method for outputting log data provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0060] Step S101, obtaining log data to be output;

[0061] The log data to be output is a large amount of log information generated by the server during operation, and the log information plays an important role in system monitoring, troubleshooting, performance analysis, security auditing, etc. However, the log data to be output may contain unnecessary information, so the embodiment of the present application needs to filter the log data to be output to obtain the target output log data in subsequent operations.

[0062] Step S101 shown in the embodiment of the present application provides a data basis for subsequent data analysis by acquiring log data to be output.

[0063] Step S102, analyzing and processing the log data to be output, and determining the log level corresponding to the log data to be output;

[0064] In some embodiments, step S102 may include: analyzing and processing the log data to be output to obtain target log feature data of the log data to be output; wherein the target log feature data includes at least one of the following: the severity of the log information of the log data to be output and the importance of the log information of the log data to be output; determining the log level corresponding to the log data to be output based on the severity of the log information and the importance of the log information of the log data to be output.

[0065] Among them, analysis and processing refers to a series of inspection, parsing and evaluation operations on the original log data to be output, so as to extract useful information or features therefrom, that is, to extract target log feature data. For target log feature data, it refers to the key feature information extracted from the log data to be output through the analysis and processing steps and used for subsequent log level classification. The target log feature data may include at least one of the following contents: the severity of the log information of the log data to be output and the importance of the log information of the log data to be output. The severity of the log information is used to measure the urgency or scope of impact of the problem reflected by the log information. The importance of the log information is similar to the severity of the log information, but it focuses more on the value of the log information for system maintenance, performance tuning or security auditing.

[0066] Optionally, after obtaining the target log characteristic data, the log data can be divided into different categories according to the severity and importance of the log information, that is, the log level corresponding to each log information in the log data to be output.

[0067] As for the log level, it is a mechanism for categorizing log messages. The log level can include but is not limited to the following:

[0068] (1) TRACE level, records every step and variable value of program execution.

[0069] (2) DEBUG level: information recorded when debugging a program, providing detailed information about the program execution process.

[0070] (3) I NFO level, records general information, indicating that the program runs as expected, such as service startup, configuration loading success, etc.

[0071] (4) WARN level, indicating that a potential problem may have occurred, or a situation that should not occur but will not affect the operation of the program has occurred, such as the use of a deprecated API (Application Programming Interface).

[0072] (5) ERROR level: used to record error events, which usually cause some functions of the program to fail.

[0073] Step S102 shown in the embodiment of the present application analyzes and processes the log data to be output, determines the log level corresponding to the log data to be output, and can clearly obtain the operating status and potential problems of the system, so as to formulate maintenance strategies and emergency measures in a targeted manner.

[0074] Step S103, determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level;

[0075] In some embodiments, step S103 may include: obtaining current system resource usage information; determining the log output judgment condition corresponding to the log level based on the log level corresponding to the log data to be output; judging whether the log data to be output meets the log output judgment condition corresponding to the log level based on the current system resource usage information; if the log data to be output meets the log output judgment condition corresponding to the log level, determining the target output log data from the log data to be output.

[0076] In some embodiments, the method may further include: if the log data to be output does not meet the log output judgment condition corresponding to the log level, terminating the log output process corresponding to the log data to be output.

[0077] The current system resource usage information refers to the various resource usage conditions of the server system when executing log output operations, including but not limited to CPU usage, memory usage, disk I / O capacity, network bandwidth and other information.

[0078] Optionally, the log output judgment condition is a rule formulated based on the log level and system resource usage information, and the log output judgment condition is used to determine whether a log of a specific level should be output. In an embodiment of the present application, the log output judgment condition may include the following four points, specifically:

[0079] (1) Log output judgment conditions for Error level logs:

[0080] CPU idle >= 5%;

[0081] Memory free>=10%;

[0082] The average IO duration is less than or equal to 10 milliseconds.

[0083] (2) Log output judgment conditions for Warn level logs:

[0084] The system minimum log level is Warn level;

[0085] CPU idle >= 10%;

[0086] Memory free>=15%;

[0087] The average IO duration is less than or equal to 6 milliseconds.

[0088] (3) Log output judgment conditions for Info-level logs:

[0089] The system minimum log level is Info level;

[0090] CPU idle >= 20%;

[0091] Memory free>=20%;

[0092] The average IO duration is less than or equal to 3 milliseconds.

[0093] (4) Log output judgment conditions for other logs:

[0094] The minimum log level of the system is debug and trace level;

[0095] CPU idle >= 30%;

[0096] Memory free>=25%;

[0097] The average IO duration is less than or equal to 2 milliseconds.

[0098] In a specific implementation, if a certain log data does not meet any of the above-mentioned log output judgment conditions, the log is directly discarded and the log output process ends; if a certain log data meets any of the above-mentioned log output judgment conditions, the next step (cache queue judgment) can be executed. Exemplarily, assuming that a certain log data meets the log output judgment condition of a certain log level (such as the Warn level), the log data can be output normally and the next step is executed; assuming that a certain log data does not meet any of the above-mentioned log output judgment conditions, the log data is directly discarded and the log output process ends. By setting detailed log output judgment conditions, the system can flexibly control the output of logs of different levels while ensuring performance, so as to better monitor system status, locate problems and optimize system performance.

[0099] The target output log data is the log data that needs to be output to the next step after being screened by the log output judgment condition.

[0100] Step S103 shown in the embodiment of the present application determines whether log data of a certain log level can be retained and output by dynamically judging system resources, so that when the system is busy, the normal execution of the program business logic is prioritized, while at the same time, important logs can be retained to the maximum extent and the loss rate of important logs can be reduced.

[0101] Step S104, storing the target output log data into a target cache queue that meets a preset queue capacity threshold;

[0102] In some embodiments, before step S104, it may also include: if the current queue usage capacity of the current cache queue is less than the preset queue usage threshold, then determining the current cache queue as the target cache queue that meets the preset queue capacity threshold; if the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is greater than or equal to the preset system memory capacity threshold, then expanding the current cache queue according to the preset queue expansion threshold, and determining the expanded current cache queue as the target cache queue that meets the preset queue capacity threshold; if the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is less than the preset system memory capacity threshold, then terminating the log output process corresponding to the log data to be output.

[0103] The current cache queue refers to the cache queue currently being used or already existing in the system. The current queue usage capacity is the size of the space already occupied in the current cache queue, that is, the queue usage of the current cache queue.

[0104] The preset queue usage threshold is an upper threshold of the usage capacity set for the current cache queue. When the current queue usage capacity reaches or exceeds this threshold, it means that the current cache queue is close to or has reached full load, and corresponding measures need to be taken to avoid queue overflow. At this time, the embodiment of the present application introduces the current system memory capacity to balance system resources. The current system memory capacity refers to the total amount of memory currently available to the system, and a preset system memory capacity threshold is set for the current system memory capacity. The preset system memory capacity threshold refers to a minimum capacity limit set for the system memory.

[0105] The preset queue expansion threshold is an incremental value used by the system to expand the cache queue capacity when the current cache queue usage capacity reaches or exceeds the preset queue usage threshold and the system memory resources are sufficient. By expanding the queue capacity, the system can continue to receive and store more log data.

[0106] Among them, the preset queue capacity threshold refers to a maximum capacity limit set for the cache queue, and the preset queue capacity threshold is used to ensure that the cache queue will not grow indefinitely, thereby consuming too many system resources. In the embodiment of the present application, the preset queue capacity threshold is divided into two cases. One is when the current queue usage capacity of the current cache queue is less than the preset queue usage threshold, the preset queue capacity threshold at this time can be understood as the value of the available capacity of the current cache queue, and the log data can be received at this time; the other is when the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is greater than or equal to the preset system memory capacity threshold, the current cache queue is expanded according to the preset queue expansion threshold, and the value of the available capacity of the expanded current cache queue is used as the preset queue capacity threshold, and the log data can also be received at this time.

[0107] The target cache queue is a cache queue capable of receiving and storing log data, and has sufficient available queue capacity and system memory resources.

[0108] In a specific implementation, if the log data to be output meets the above log output judgment conditions, the cache queue judgment step can be performed, which can be divided into the following three situations:

[0109] (1) When the cache queue usage is less than 95%, the log data is directly placed in the cache queue;

[0110] (2) When the cache queue usage is greater than or equal to 95% and the system memory is free greater than or equal to 10%, the system automatically expands the cache queue capacity by 50% and puts the log data into the expanded cache queue;

[0111] For example, the initial queue length may be set to 1000, and each expansion may increase the existing length by 50%;

[0112] (3) When the queue usage is greater than or equal to 95% and the system memory idle is less than 10%, it is considered that the system resources are insufficient, and the log data is discarded, ending the process.

[0113] In the specific implementation, the cache queue is adjusted according to the relationship between the current cache queue usage capacity and the system memory capacity, which can ensure the efficient use of system resources, avoid performance degradation or system crash caused by blindly receiving new log data when resources are tight, and reduce the waste caused by idle resources.

[0114] Step S104 shown in the embodiment of the present application can effectively manage the writing and reading of log data by utilizing the cache queue as a temporary storage area for log data, thereby reducing the burden on the CPU and I / O and avoiding system performance issues caused by a large amount of log data.

[0115] In some embodiments, the method may further include: obtaining a current queue usage capacity of a current cache queue at a preset time interval; and reducing the queue cache capacity of the current cache queue if the current queue usage capacity is less than a preset capacity reduction threshold.

[0116] The preset time interval is the time interval for the system to check the current cache queue usage capacity. By regularly checking the queue usage capacity, the system can timely discover and reduce the idle or low-utilization cache queue to release memory resources.

[0117] Optionally, the preset capacity reduction threshold refers to a threshold used by the system to determine whether the current queue usage capacity is too low when executing a queue cache capacity reduction operation. Specifically, when the current queue usage capacity is lower than the preset capacity reduction threshold, it indicates that there is a large amount of free space in the cache queue, and memory resources can be released by reducing the queue cache capacity.

[0118] In the specific implementation, the log cache queue reduction process is: the system program obtains the queue usage at a certain interval (such as 5 minutes). If the cache queue usage capacity is less than 50%, the cache queue is reduced by 30%; if the cache queue usage capacity is greater than or equal to 50%, no operation is performed.

[0119] In a specific implementation, the current queue usage capacity of the current cache queue is obtained at a preset time interval, and the cache capacity of the cache queue is reduced based on the comparison result between the current queue usage capacity and the preset capacity reduction threshold. The cache capacity of the idle or low-utilization cache queue can be reduced in time to release excess memory resources for use by other processes or applications, thereby improving the resource utilization of the entire system.

[0120] Step S105, reading the target output log data in the target cache queue in a preset time sequence;

[0121] The preset time sequence is the time sequence of the log data entering the target cache queue, and the target output log data in the target cache queue is read in a first-in-first-out reading manner.

[0122] Step S105 shown in the embodiment of the present application reduces the burden on the CPU and I / O by reading the log data in the target cache queue in a preset time sequence, while effectively maintaining the time sequence of the log and improving the processing efficiency.

[0123] Step S106, determining a log output mode of the target output log data according to the current system disk space capacity, and outputting the target output log data according to the log output mode.

[0124] In some embodiments, step S106 may include: if the current system disk space capacity is less than or equal to a first preset system disk space threshold, the target output log data is output to the console; if the current system disk space capacity is greater than the first preset system disk space threshold, and the current system disk space capacity is less than or equal to a second preset system disk space threshold, the target output log data is stored in the system disk file according to a circular overwrite method, and the target output log data in the system disk file is output to the console; if the current system disk space capacity is greater than the second preset system disk space threshold, the target output log data is stored in the system disk file according to a sequential append method, and the target output log data in the system disk file is output to the console.

[0125] The current system disk space capacity refers to the total amount of disk space available for storing data in the current system, reflecting the amount of space currently available in the system for storing new files or data.

[0126] The first preset system disk space threshold and the second preset system disk space threshold are both used to determine the free degree of the system disk space.

[0127] Optionally, the log output method may include three situations: one is to directly output the log data to the console when the system disk space is small; the second is to use a circular overwrite method when the system disk space is moderate, output the latest log data to the end of the system disk file, delete the old log data, and finally output the log data in the system disk file to the console; the third is to use a sequential append method when the system disk space is large, that is, a normal log file output method, append the latest log data to the end of the system disk file normally, and finally output the log data in the system disk file to the console.

[0128] Among them, the console is an interface for users to interact with the computer, allowing users to input commands and view the computer's output information.

[0129] For example, assume that the first preset system disk space threshold is 5% and the second preset system disk space threshold is 10%. If the system disk space is <=5%, the system disk space is small at this time, and the log data is directly output to the console; if the system disk space is >5% and <=10%, the system disk space is moderate at this time, and a circular overwrite method is used to output the latest log data to the end of the system disk file, and the old log data is deleted, and finally the log data in the system disk file is output to the console; if the system disk space is >10%, the system disk space is large at this time, and a sequential append method is used to normally append the latest log data to the end of the system disk file, and finally the log data in the system disk file is output to the console.

[0130] Step S106 shown in the embodiment of the present application reduces the burden on the CPU and I / O and improves the utilization rate of the disk space by dynamically selecting the log output method according to the system disk space capacity.

[0131] Steps S101 to S106 shown in the embodiment of the present application are as follows: obtaining log data to be output; analyzing and processing the log data to be output to determine the log level corresponding to the log data to be output; determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; storing the target output log data in a target cache queue that meets a preset queue capacity threshold; reading the target output log data in the target cache queue in a preset time sequence; determining the log output method of the target output log data according to the current system disk space capacity, and outputting the target output log data according to the log output method. The embodiment of the present application dynamically adjusts the log output level through intelligent analysis, realizes the refined classification of log information, can reduce unnecessary log output, reduce the occupation of system resources by log output, thereby effectively improving the overall system performance and improving the utilization rate of system resources; at the same time, based on the preset log output judgment strategy, the target output log data that needs to be retained or processed is automatically screened out, effectively reducing the redundancy of log data, enabling the system to record key information at critical moments, and providing convenience for subsequent troubleshooting and analysis work; and, using the cache queue as a temporary storage area for log data, the writing and reading of log data can be effectively managed to avoid system performance problems caused by a large amount of log data; by dynamically evaluating the system disk space capacity, the most suitable log output mode is intelligently selected, which not only ensures the reliable storage of log data, but also avoids system problems caused by insufficient system resources. By automatically managing log output, the log management process is simplified and the log management efficiency is improved. In addition, the intelligent log management mode provided by the embodiment of the present application also helps the system save storage resources.

[0132] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0133] See also Figure 2 , Figure 2 is a flow chart of the method for outputting log data provided in the embodiment of the present application; Figure 2 As shown, the overall process of the log data output method provided in the embodiment of the present application includes the following steps:

[0134] Step 1: Start the system program and call the log output method.

[0135] The log output method is set by the coder according to the actual situation and is the code reserved in the program during coding. When the system program performs certain operations or processes, it can call the log output method to record and output log information. The log information is used to observe the program running status, business logic running process, etc. The log information can include log level and log content information.

[0136] Step 2: determine the log level of the log data, and determine whether to retain the log output of the log level in combination with the log level and system resource usage.

[0137] like Figure 2 As shown, the log level may include TRACE level, DEBUG level, I NFO level, WARN level and ERROR level, and the log output judgment conditions corresponding to each log level may include the following four contents, specifically:

[0138] (1) Log output judgment conditions for Error level logs:

[0139] CPU idle >= 5%;

[0140] Memory free>=10%;

[0141] The average IO duration is less than or equal to 10 milliseconds.

[0142] (2) Log output judgment conditions for Warn level logs:

[0143] The system minimum log level is Warn level;

[0144] CPU idle >= 10%;

[0145] Memory free>=15%;

[0146] The average IO duration is less than or equal to 6 milliseconds.

[0147] (3) Log output judgment conditions for Info-level logs:

[0148] The system minimum log level is Info level;

[0149] CPU idle >= 20%;

[0150] Memory free>=20%;

[0151] The average IO duration is less than or equal to 3 milliseconds.

[0152] (4) Log output judgment conditions for other logs:

[0153] The minimum log level of the system is debug and trace level;

[0154] CPU idle >= 30%;

[0155] Memory free>=25%;

[0156] The average IO duration is less than or equal to 2 milliseconds.

[0157] In a specific implementation, if a certain log data does not meet any of the above log output judgment conditions, the log is directly discarded and the log output process ends; if a certain log data meets any of the above log output judgment conditions, step three (cache queue judgment) can be executed.

[0158] Step 3: Cache queue determination.

[0159] like Figure 2 As shown, if the log data meets the log output judgment condition in step 2, the cache queue judgment step can be performed. The cache queue judgment process can be specifically divided into the following three cases:

[0160] (1) When the cache queue usage is less than 95%, the log data is directly placed in the cache queue;

[0161] (2) When the cache queue usage is greater than or equal to 95% and the system memory is free greater than or equal to 10%, the system automatically expands the cache queue capacity by 50% and puts the log data into the expanded cache queue;

[0162] (3) When the queue usage is greater than or equal to 95% and the system memory idle is less than 10%, it is considered that the system resources are insufficient, and the log data is discarded, ending the process.

[0163] Step 4: The system program reads the log data in the cache queue in chronological order.

[0164] Step 5: Dynamically select the log output method based on the system disk space capacity and output the log data to the console.

[0165] like Figure 2 As shown, the first preset system disk space threshold is set to 5%, and the second preset system disk space threshold is set to 10%. If the system disk space is <=5%, the log data is directly output to the console; if the system disk space is >5% and <=10%, a circular overwrite method is used to output the latest log data to the end of the system disk file, and the old log data is deleted, and finally the log data in the system disk file is output to the console; if the system disk space is >10%, a sequential append method is used to append the latest log data to the end of the system disk file normally, and finally the log data in the system disk file is output to the console.

[0166] Step 6: After the log data is output to the console, end the process.

[0167] It is worth mentioning that in the log data output method provided in the embodiment of the present application, in addition to the contents of steps 1 to 6 above, the embodiment of the present application also introduces a log cache queue reduction process, specifically, Figure 2 As shown, the log cache queue reduction process is: the system program obtains the queue usage at regular intervals (such as 5 minutes). If the cache queue usage capacity is less than 50%, the cache queue is reduced by 30%; if the cache queue usage capacity is greater than or equal to 50%, no operation is performed. By obtaining the current queue usage capacity of the current cache queue at a preset time interval, and reducing the cache capacity of the cache queue based on the comparison result of the current queue usage capacity and the preset reduction capacity threshold, the cache capacity of the idle or low-utilization cache queue can be reduced in time to release excess memory resources for use by other processes or applications, thereby improving the resource utilization of the entire system.

[0168] It should be pointed out that this embodiment only briefly illustrates the general process of the method for outputting log data. The detailed description of each step can refer to the relevant content in the aforementioned embodiment and will not be repeated here. It can be understood that the present invention is not limited to this.

[0169] The embodiment of the present application obtains log data to be output; analyzes and processes the log data to be output to determine the log level corresponding to the log data to be output; determines target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; stores the target output log data in a target cache queue that meets a preset queue capacity threshold; reads the target output log data in the target cache queue in a preset time sequence; determines the log output method of the target output log data according to the current system disk space capacity, and outputs the target output log data according to the log output method. The embodiment of the present application dynamically adjusts the log output level through intelligent analysis, realizes the refined classification of log information, can reduce unnecessary log output, and reduce the occupation of system resources by log output, thereby effectively improving the overall system performance and improving the utilization rate of system resources; at the same time, based on the preset log output judgment strategy, the target output log data that needs to be retained or processed is automatically screened out, effectively reducing the redundancy of log data, enabling the system to record key information at critical moments, and providing convenience for subsequent troubleshooting and analysis work; and, using the cache queue as a temporary storage area for log data, the writing and reading of log data can be effectively managed to avoid system performance problems caused by large amounts of log data; by dynamically evaluating the system disk space capacity, the most appropriate log output method is intelligently selected, which not only ensures the reliable storage of log data, but also avoids system problems caused by insufficient system resources. By automatically managing log output, the log management process is simplified and the log management efficiency is improved.

[0170] The present application also provides a device for outputting log data. Figure 3 , the above-mentioned log data output method can be implemented, and the device 300 includes the following modules:

[0171] The log data to be outputted acquisition module 301 is used to acquire the log data to be outputted;

[0172] The log level determination module 302 is used to analyze and process the log data to be output, and determine the log level corresponding to the log data to be output;

[0173] The target output log data determination module 303 is used to determine the target output log data from the log data to be output according to the log output judgment condition corresponding to the log level;

[0174] The log data cache module 304 is used to store the target output log data into a target cache queue that meets a preset queue capacity threshold;

[0175] The target output log data reading module 305 is used to read the target output log data in the target cache queue in a preset time sequence;

[0176] The target output log data output module 306 is used to determine the log output mode of the target output log data according to the current system disk space capacity, and output the target output log data according to the log output mode.

[0177] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0178] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned log data output method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0179] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0180] See also Figure 4 , Figure 4 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0181] The processor 401 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0182] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 402, and the processor 401 calls and executes the log data output method of the embodiment of this application;

[0183] Input / output interface 403, used to implement information input and output;

[0184] Communication interface 404, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0185] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0186] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via the bus 405 .

[0187] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned log data output method is implemented.

[0188] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0189] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0190] The log data output method, log data output device, electronic device and storage medium provided in the embodiments of the present application obtain log data to be output; analyze and process the log data to be output to determine the log level corresponding to the log data to be output; determine target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; store the target output log data in a target cache queue that meets a preset queue capacity threshold; read the target output log data in the target cache queue in a preset time sequence; determine the log output mode of the target output log data according to the current system disk space capacity, and output the target output log data according to the log output mode. The embodiment of the present application dynamically adjusts the log output level through intelligent analysis, realizes the refined classification of log information, can reduce unnecessary log output, and reduce the occupation of system resources by log output, thereby effectively improving the overall system performance and improving the utilization rate of system resources; at the same time, based on the preset log output judgment strategy, the target output log data that needs to be retained or processed is automatically screened out, effectively reducing the redundancy of log data, enabling the system to record key information at critical moments, and providing convenience for subsequent troubleshooting and analysis work; and, using the cache queue as a temporary storage area for log data, the writing and reading of log data can be effectively managed to avoid system performance problems caused by large amounts of log data; by dynamically evaluating the system disk space capacity, the most appropriate log output method is intelligently selected, which not only ensures the reliable storage of log data, but also avoids system problems caused by insufficient system resources. By automatically managing log output, the log management process is simplified and the log management efficiency is improved.

[0191] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0192] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0193] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0195] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0196] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0197] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0200] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0201] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for outputting log data, characterized in that: The method comprises the following steps: Get the log data to be output; Analyze and process the log data to be output, and determine the log level corresponding to the log data to be output; Determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; Storing the target output log data in a target cache queue that meets a preset queue capacity threshold; Read the target output log data in the target cache queue in a preset time sequence; The log output mode of the target output log data is determined according to the current system disk space capacity, and the target output log data is output according to the log output mode.

2. The method according to claim 1, characterized in that The analyzing and processing the log data to be output to determine the log level corresponding to the log data to be output includes: Analyze and process the log data to be output to obtain target log feature data of the log data to be output; wherein the target log feature data includes at least one of the following: the severity of the log information of the log data to be output and the importance of the log information of the log data to be output; The log level corresponding to the log data to be output is determined according to the severity of the log information and the importance of the log information of the log data to be output.

3. The method according to claim 1, characterized in that The step of determining target output log data from the log data to be output according to the log output judgment condition corresponding to the log level includes: Get current system resource usage information; Determining the log output judgment condition corresponding to the log level according to the log level corresponding to the log data to be output; According to the current system resource usage information, determining whether the log data to be output meets the log output determination condition corresponding to the log level; If the log data to be output meets the log output judgment condition corresponding to the log level, the target output log data is determined from the log data to be output.

4. The method according to claim 3, characterized in that The method further comprises: If the log data to be output does not meet the log output judgment condition corresponding to the log level, the log output process corresponding to the log data to be output is terminated.

5. The method according to claim 1, characterized in that Before storing the target output log data into a target cache queue that meets a preset queue capacity threshold, the method further includes: If the current queue usage capacity of the current cache queue is less than the preset queue usage threshold, determining the current cache queue as the target cache queue that meets the preset queue capacity threshold; If the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is greater than or equal to the preset system memory capacity threshold, the current cache queue is expanded according to the preset queue expansion threshold, and the expanded current cache queue is determined as the target cache queue that meets the preset queue capacity threshold; If the current queue usage capacity of the current cache queue is greater than or equal to the preset queue usage threshold, and the current system memory capacity is less than the preset system memory capacity threshold, the log output process corresponding to the log data to be output is terminated.

6. The method according to claim 1, characterized in that The method further comprises: Get the current queue usage capacity of the current cache queue at a preset time interval; If the current queue usage capacity is less than a preset capacity reduction threshold, the queue cache capacity of the current cache queue is reduced.

7. The method according to claim 1, characterized in that The step of determining the log output mode of the target output log data according to the current system disk space capacity, and outputting the target output log data according to the log output mode, includes: If the current system disk space capacity is less than or equal to the first preset system disk space threshold, outputting the target output log data to the console; If the current system disk space capacity is greater than the first preset system disk space threshold, and the current system disk space capacity is less than or equal to the second preset system disk space threshold, the target output log data is stored in the system disk file according to a circular overwriting method, and the target output log data in the system disk file is output to the console; If the current system disk space capacity is greater than the second preset system disk space threshold, the target output log data is stored in the system disk file according to a sequential append method, and the target output log data in the system disk file is output to the console.

8. A device for outputting log data, characterized in that: The device comprises the following modules: A module for obtaining log data to be output, used for obtaining log data to be output; A log level determination module, used to analyze and process the log data to be output, and determine the log level corresponding to the log data to be output; A target output log data determination module, used to determine target output log data from the log data to be output according to the log output judgment condition corresponding to the log level; A log data cache module, used for storing the target output log data into a target cache queue that meets a preset queue capacity threshold; A target output log data reading module, used for reading the target output log data in the target cache queue in a preset time sequence; The target output log data output module is used to determine the log output mode of the target output log data according to the current system disk space capacity, and output the target output log data according to the log output mode.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images