Log compression

Abstract

Systems, devices, and methods related to log compression are described. In an example, during execution of an operation using a memory resource, a system log identifying target data can be compiled in that memory resource. The system log can be analyzed using a portion of the memory resource that would otherwise be available for the execution of the operation. The system log can be compressed during the execution of the operation, and the level or timing of this compression can be based on analysis occurring contemporaneously with or as a result of the execution of the operation. In some examples, compressing the system log can include discarding a portion of the system log. Compressing the system log can also include extracting the target data from the system log and converting the extracted target data into structured data when compiling the system log.

Description

Technical Field

[0001] The present disclosure relates generally to semiconductor memories and methods, and more particularly to methods and systems related to log compression. Background Art

[0002] Logs can be compiled while the computing system is executing. For example, as the computing system executes instructions to perform various operations, a log file can be compiled that records events that occur during execution. In some examples, events can be recorded by the operating system and / or by applications executing on the computing system. Logs can be compiled within memory resources used by the computing system to execute the underlying operations. The logs can be compiled in the memory resources until the operations are completed, at which point the logs can be analyzed and / or exported from the memory resources. Summary of the Invention

[0003] In one aspect, the present disclosure provides a method comprising: during execution of an operation from which a system log is compiled in a memory resource, identifying target data within the system log, the memory resource being used for the execution of the operation, wherein a portion of the memory resource includes data of the system log; and during the execution of the operation, compressing the system log within the memory resource based on identifying the target data.

[0004] On the other hand, the present disclosure further provides a system comprising: a processor; and a non-transitory machine-readable storage medium storing instructions executable by the processor to perform the following operations: identifying target data within the system log when compiling a system log from execution of a main program operation; and discarding a portion of the system log selected based on the target data identified within the system log from a portion of a memory resource.

[0005] On the other hand, the present disclosure further provides a non-transitory machine-readable storage medium comprising instructions executable by a processor to perform the following operations: when compiling a system log from the execution of a main program, identifying target data within the system log; when compiling the system log, extracting the target data from the system log from a volatile memory resource used for the execution of the main program; and converting the extracted target data into structured data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An example of a system for log compression consistent with the present disclosure is described.

[0007] Figure 2 An example of a computing device for log compression consistent with the present disclosure is described.

[0008] Figure 3Examples of non-transitory machine-readable memory and processors for log compression consistent with the present disclosure are described.

[0009] Figure 4 An example of a method for log compression consistent with the present disclosure is described. DETAILED DESCRIPTION

[0010] A computing system may include a processor. A processor may include electronic circuitry within the computing system that executes instructions that constitute a computing program. A processor may include a central processing unit (CPU) of the computing system.

[0011] A computing system may include one or more memory resources. For example, a computing system may include a non-transitory machine-readable medium for storing instructions that are executable by a processor to perform various computing operations.

[0012] In some examples, the memory may include memory that can be utilized by an operating system, a computing program, and / or a process when performing various computing operations. This memory may be volatile memory that can temporarily store instructions for execution of the computing program by a processor and data generated from execution of the computing program by the processor. For example, the memory may include random access memory (RAM), which is faster but more expensive than other memory resources available at the computing system, such as non-volatile memory resources or hard drives.

[0013] A computing system may have a computing capacity. The computing capacity may include the ability of the computing system to perform various amounts of computing operations. The computing capacity may include the limits of the computing system to execute various instructions and / or process various amounts of data.

[0014] The computing capacity of a computing system can be defined by a number of factors. One factor can include the speed of the processor. The speed of the processor can refer to the clock speed of the processor cores, which defines the number of operations the processor can perform in a given time.

[0015] However, additional factors may limit the computing capacity of a computing system. For example, as described above, a computing system may utilize volatile memory resources (e.g., RAM) in performing its computing operations. For example, a processor core may receive instructions from a computing operation and, operating at clock speed, rapidly process this information and temporarily store it in a volatile memory resource. Thus, the storage capacity of the volatile memory resource may also serve as a limit on the computing capacity of the computing system.

[0016] For example, as a computing system executes a computing operation, data used to execute the operation, data resulting from the execution, and / or log files associated with and compiled from the execution may begin to fill the computing system's RAM. For example, as described above, logs may be compiled in memory resources until the operation is complete, at which point the logs may be analyzed and / or exported from the memory resources. Thus, the compilation of log files during execution may be a particularly significant contributor to memory resource capacity consumption.

[0017] When RAM capacity is filled, attempts at additional operations or the creation of additional data may result in "out of memory" errors. In these instances, the underlying computing program being executed may become unstable and / or crash. Additionally, the executed computing operations may be paused, interrupted, and / or delayed. Consequently, the execution of computing operations, along with the user experience that depends on the execution of such operations, may be adversely affected.

[0018] In some instances, additional volatile memory may be added to and / or allocated to a computing system to prevent memory exhaustion during the execution of computing operations. However, the volatile memory used to facilitate these operations may impose additional costs on the computing system and / or its use. For example, RAM memory can be thousands of times more expensive per unit of storage than hard drive storage. Thus, adding volatile memory to a computing system may impose a significant cost increase on the acquisition and / or utilization of the computing system.

[0019] In contrast, embodiments consistent with the present disclosure may include mechanisms for preserving the capacity of memory resources for the execution of computing operations. Compared to embodiments that rely on adding supplemental memory resources to execute computing operations, embodiments consistent with the present disclosure may compress data associated with the execution of computing operations within memory resources during the execution of computing operations.

[0020] For example, an example consistent with the present disclosure may include a method comprising, during execution of an operation from which a system log is compiled in a memory resource, analyzing the system log to identify target data within the system log, the memory resource being used for the execution of the operation, wherein the analyzing is performed using a portion of the memory resource that would otherwise be available for the execution of the operation. The method may include, during the execution of the operation, compressing the system log within the memory resource based on the analyzing.

[0021] An example consistent with the present disclosure may include a system comprising: a processor; and a non-transitory machine-readable medium storing instructions executable by the processor to, when compiling a system log from execution of a main program operation, analyze the system log to identify target data within the system log and, based on the target data identified within the system log, discard a portion of the system log from a portion of memory resources used for the execution of the main program.

[0022] An example consistent with the present disclosure may include a non-transitory machine-readable storage medium comprising instructions executable by a processor to, when compiling a system log from execution of a main program, identify target data within the system log. The instructions may be executable to, when compiling the system log, extract the target data identified within the system log from a portion of volatile memory resources used for the execution of the main program and convert the extracted target data into structured data.

[0023] Figure 1 An example of a system 100 for log compression consistent with the present disclosure is described. The system 100 may include a computing system 102. The described components and / or operations of the system 100 may include Figures 2 to 4 The components and / or operations described and / or Figures 2 to 4 The components and / or operations described are interchangeable.

[0024] The computing system 102 may include computing devices such as computers, laptops, desktop computers, smartphones, wearable computers, smart accessories, Internet of Things (IoT) devices, smart appliances, smart environmental control devices, servers, distributed computing systems, cloud-based computing platforms, virtual machines, etc. The computing system 102 may utilize computing resources such as a processor 104 and memory resources 106 to perform computing operations.

[0025] For example, computing system 102 may include a processor 104. Processor 104 may include electronic circuitry for executing instructions constituting a computing program. For example, processor 104 may include electronic circuitry for performing (e.g., executing) arithmetic, logical, control, and / or input / output operations specified by machine-readable instructions.

[0026] The computing system 102 may include memory resources 106. The memory resources 106 may include volatile memory resources. For example, the memory resources 106 may include RAM memory resources. For example, the memory resources 106 may include dynamic random access memory (DRAM) resources or static random access memory (SRAM) resources.

[0027] Memory resources 106 may be memory resources that are solely responsible for storing instructions and / or data associated with executing the computing operations of a computing application while the computing application is executing. When computing system 102 executes the computing operations of a computing program, instructions from the computing program for completing the computing operations may be stored in memory resources 106. Processor 104 may retrieve instructions from memory resources 106. Processor 104 may execute instructions from memory resources 106 to perform the computing operations. Memory resources 106 may be used to store data underlying and / or generated by the computing operations when the instructions are executed.

[0028] For example, computing system 102 may utilize processor 104 to execute data modeling operations of a data modeling application. Memory resources 106 may store, during execution of the data modeling operations, instructions for carrying out the data modeling operations, data modeling algorithms applied to raw data, raw data inserted into the data modeling algorithms, and data output from applying the data modeling algorithms to the raw data. Processor 104 may retrieve this data from memory resources 106 and / or store this data to memory resources 106.

[0029] Additionally, memory resources 106 may include logs 108. Logs 108 may include system logs. Logs 108 may include files used to compile and / or record data regarding events that occur during the execution of an operating system or other computing applications. For example, data regarding device changes, device drivers, system changes, events, operations, errors, warnings, startup messages, abnormal shutdowns, and the like may be compiled in logs 108. In other words, logs 108 may be a compilation location for data regarding software, hardware, system processes, and system components, along with whether the loading process was successful. The data in logs 108 may be used to diagnose the source of problems with computing system 102 and / or predict potential system problems.

[0030] The data compiled in log 108 may be generated by and / or during the execution of computing operations. That is, as instructions for implementing a computing operation of a computing program are retrieved from memory resources 106 and executed by processor 104, the data constituting log 108 may be generated and compiled in log 108 in memory resources 106. Thus, the data compiled in log 108 may consume a portion of memory resources 106 that would otherwise be available for the execution of computing operations (e.g., to store additional instructions associated with the execution of computing operations, to store additional input data associated with the execution of computing operations, to store additional data generated by the execution of computing operations, etc.).

[0031] Log 108 may be compiled until execution of the underlying computing operation is complete. That is, log 108 may be generated from data created by execution of instructions for completing the computing operation. Log 108 may continue to be compiled until execution of the instructions for the computing operation is complete. Log 108 may continue to be compiled as long as instructions for the computing operation are still being executed. In some examples, log 108 may be analyzed and / or exported from memory resources 106 once execution of the instructions for completing the computing operation is complete, but not before.

[0032] Log 108 can contain a large amount of data. That is, the longer or more complex the computing operation, the more data its execution can generate in log 108 for compilation. For example, a data modeling operation for a data modeling computing program can be performed on a large data set. The execution of the data modeling operation can include applying the data model to the large data set, which can take minutes, hours, days, etc. to complete. During this entire cycle, a log 108 of events that occurred during and / or through execution can be compiled in memory resources 106. Log 108 can reach hundreds of millions or even more event logs. As described above, this data compiled into log 108 can consume a portion of memory resources 106 and bring computing system 102 closer to an out-of-memory error.

[0033] Compared to previous mechanisms that continue compiling log 108 until the completion of a single execution of instructions for completing a computing operation, examples consistent with the present disclosure may include a compression manager 110 that can act on data compiled in log 108 during the execution of instructions for completing a computing operation, before the completion of execution of instructions for completing a computing operation, and / or before the completion of compiling data from a computing operation into log 108. Compression manager 110 may include instructions that can be retrieved from memory resources 106 and executed by processor 104 to perform the compression operations described herein. Compression manager 110 may include an execution thread that is smaller than the computing operations of the main computing program executed by processor 104 and therefore consumes less capacity of memory resources 106. Similarly, compression manager 110 may include an execution thread that consumes less processing capacity of processor 104 than the main computing program executed by processor 104.

[0034] The compression manager 110 may include instructions that are executed by the processor 104 concurrently with the execution of instructions for performing the computing operations of the host computing program. That is, the instructions for the compression operations described herein may be executed concurrently with the instructions for the computing operations of the host computing program being executed by the processor 104.

[0035] Compression manager 110 may include instructions executable by processor 104 to analyze log 108. Analyzing log 108 may include analyzing data from log 108 as log 108 is compiled. That is, analyzing log 108 may include analyzing data from the log during and / or concurrently with the execution of the computing operations from which log 108 is compiled. Analyzing log 108 may include analyzing data compiled within log 108 concurrently with the creation and / or compilation of the data into log 108. Analyzing log 108 may include analyzing data compiled within log 108 prior to completion of execution of the computing operations of the main program.

[0036] Analyzing the log 108 may include analyzing data in the log 108 to identify target data within the log 108. For example, analyzing the log 108 may include searching the data compiled in the log 108 for the target data.

[0037] Target data may include data that can be identified as data of interest by a user, administrator, analytics platform, etc. For example, target data may include data indicating a particular event, problem, question, etc. of particular interest to the user, administrator, analytics platform, etc. Target data may include data identified as being particularly useful for conducting an analysis of interest to the user, administrator, analytics platform, etc. Target data may include data generally associated with typical problems or events of particular interest to the user, administrator, analytics platform, etc. Thus, target data may include data indicating that it and / or previous data is a candidate for retention within the compressed log 112, described in detail below.

[0038] Alternatively, the target data may include data indicating normal and / or error-free execution of the computing system 102. Thus, the target data may include data indicating the absence of specific events, problems, issues, etc. of particular interest to a user, administrator, analytics platform, etc. Thus, the target data may include data indicating that it and / or previous data are candidates for removal from the log 108. As with the target data indicating problems described above, the target data indicating error-free execution may be user-specific, administrator-specific, analytics platform-specific, etc., and its definition may be derived from those sources.

[0039] In some examples, the target data may be specified by a user, administrator, analytics platform, etc. That is, the identification of the target data may be configured by and / or unique to a particular user, administrator, analytics platform, etc. of computing system 102. Thus, the target data may differ depending on which user, administrator, analytics platform, etc. is associated with and / or initiates the execution of the computing operation that generated the data compiled into log 108.

[0040] Searching the data in log 108 for target data may include conducting a text search of the data compiled in log 108. The text search of the data may utilize keyword extraction of the text compiled in log 108 to identify the target data. For example, the data compiled in log 108 may be searched for specific text, words, phrases, and / or character sequences that are indicative of target data and / or are target data. The text search of the data may be performed as a regular expression search, a context-free grammar search, or the like that identifies specific text, words, phrases, and / or character sequences within the data compiled in log 108.

[0041] For example, the occurrence of a particular phrase in log 108 may indicate an error and / or events leading up to the error that an administrator is particularly interested in. Thus, searching the data log may include conducting a text search of the data compiled in log 108 for that phrase.

[0042] Alternatively, the occurrence of a particular phrase in log 108 may indicate that previous data and / or events compiled in log 108 are correct and / or erroneous and, thus, of little interest to the administrator. Thus, searching the data log may include conducting a text search of the data compiled in log 108 for that phrase.

[0043] Searching the data in the log 108 for target data may further include searching for flags and / or sequences present in the log 108. For example, searching the data in the log 108 may include searching for a particular text sequence, a particular character sequence, a particular phrase sequence, or a particular event sequence that indicates target data and / or is itself target data. That is, text, words, phrases, and / or characters that appear at a particular time, with a particular frequency, in a particular order, etc., within the data compiled in the log 108 may indicate target data and / or be itself target data.

[0044] For example, the occurrence of a particular event or character sequence in log 108 may indicate an error and / or events leading up to the error that an administrator is particularly interested in. Thus, searching the data log may include conducting a search of the data compiled in log 108 for that particular event or character sequence.

[0045] Alternatively, the occurrence of a particular event or character sequence in log 108 may indicate that previous data and / or events compiled in log 108 are correct and / or erroneous and, thus, of little interest to the administrator. Thus, searching the data log may include conducting a search of the data compiled in log 108 for that particular event or character sequence.

[0046] In addition to the text search described above, searching the data compiled in log 108 may include executing and applying a machine learning model to process the data compiled in the log and identify target data. For example, compression manager 110 may include instructions executable by processor 104 to apply a machine learning model to the data compiled in log 108. For example, a recurrent neural network (RNN) model for identifying target data among the data compiled in log 108 and / or for identifying data that is target data may be applied to the data compiled in log 108 in order to identify the target data.

[0047] Regardless of the data search mechanism used to analyze log 108, analysis of log 108 can be performed during and / or concurrently with the execution of the computing operations from which log 108 was compiled. In some examples, data generated by the execution of the computing operations and compiled into log 108 can be analyzed in real time. Analyzing data in real time can include analyzing the data in real time as it is generated and / or compiled into log 108. That is, analyzing data in real time can include near-instantaneous and / or continuous analysis of the data compiled into log 108. Furthermore, the data can be analyzed before the execution of the computing operations of the main program that generated the log 108 data is completed.

[0048] In some examples, data generated by the execution of computing operations and compiled into log 108 can be analyzed in data frames. For example, the analysis of the data can be performed periodically or in batches. In an example, the analysis of the data can be delayed until a certain amount of data and / or unanalyzed data is compiled into log 108. In some examples, the analysis of the data can be delayed until a certain amount of time (e.g., X fractions of a second, Y seconds, Z minutes, etc.) has passed since the previous analysis of the data compiled into log 108 was performed. For example, the analysis of the data can be scheduled to occur every two seconds during the execution of the computing operations of the main computing program that generated the log 108 data. Furthermore, the data can be analyzed before the execution of the computing operations of the main program that generated the log 108 data is completed.

[0049] As described above, compression manager 110 may include instructions that are retrievable from memory resources 106 and executable by processor 104 to analyze log 108 in order to identify target data within log 108. Thus, analysis of the data in log 108 may be performed utilizing a portion of the processor 104 capacity and / or a portion of the memory resources 106 capacity that would otherwise be available for execution of operations. For example, analysis of the data in log 108 may be performed by retrieving and / or executing compression manager 110 instructions from a portion of memory resources 106 that also includes instructions for executing the computational operations of the main computational program that generated the log 108 data.

[0050] The compression manager 110 may include instructions executable by the processor 104 to compress the log 108 into a compressed log 112. The compressed log 112 may include less data than the log 108. That is, the compressed log 112 may include a version of the log 108 with some data removed. Compressing the log 108 into the compressed log 112 may include compressing the data of the log 108 during execution of a computing operation of the main computing program that generated the log 108 data.

[0051] Compressing the log 108 may include modifying data in the log 108 to transform the log 108 into the compressed log 112. The modifications to the data in the log 108 may be based on the results of the previously described analysis of the system log 108. For example, compressing the log 108 may include removing data from the log 108 to transform the log 108 into the compressed log 112. The data removed from the log 108 to generate the compressed log 112 and / or the data from the log 108 saved in the compressed log 112 may be identified based on the target data identified by the previously described analysis.

[0052] For example, target data identified in log 108 that indicates an event of interest and / or is itself an event of interest may be extracted from log 108 and used to generate compressed log 112. The remaining data from log 108 may be discarded from log 108 and / or excluded from compressed log 112. Similarly, target data identified in log 108 that indicates the absence of an event of interest and / or is itself an event of interest may be discarded from log 108 and / or excluded from compressed log 112. Thus, compressing log 108 may result in generating a compressed log 112 that contains less data than its predecessor log 108 and / or contains only data identified as being of particular interest to a particular user, administrator, analytics platform, etc.

[0053] As described above, analysis of log 108 to identify target data may be performed during the execution of a computing operation from which log 108 is compiled in memory resource 106 and / or before log 108 is completed. Similarly, compression of log 108 into compressed log 112 may occur during the execution of a computing operation from which log 108 is compiled in memory resource 106 and / or before log 108 is completed. That is, compression manager 110 may include instructions executable by processor 104 to compress log 108 into compressed log 112 on the fly as data compiled into log 108 is analyzed. Compressing log 108 into compressed log 112 may be continuous while data is compiled into log 108.

[0054] In some examples, compression manager 110 may include instructions executable by processor 104 to compress log 108 into compressed log 112 in data frames. Compressing log 108 into compressed log 112 may be performed periodically and / or in batches during the compilation of data into log 108. For example, after a certain amount of uncompressed data is reached during the compilation of data into log 108 and / or after a certain period of time has passed since a previous compression operation during the compilation of data into log 108, data in log 108 may be compressed to generate compressed log 112.

[0055] In addition to being compiled and analyzed within memory resources 106, data in log 108 may also be compressed within memory resources 106. That is, a compression operation that removes data from log 108 to generate compressed log 112 may be performed on the data in memory resources 106 during the execution of the underlying computing operations that generated the data. Thus, compressing the data in log 108 may remove data from memory resources 106. As a result, the portion of memory resources 106 previously occupied by the data removed by compression is freed up to accept additional data and / or be used for the execution of the underlying computing operations (e.g., to contain instructions executable to perform the underlying computing operations, to contain data used to perform the underlying computing operations, to contain data resulting from the underlying computing operations, to contain additional log data, etc.). Thus, out-of-memory errors in computing system 102 may be reduced or eliminated by reducing the portion of the capacity of memory resources 106 occupied by log 108 before compilation of log 108 is complete, thereby freeing that capacity for computing operations of the main computing program.

[0056] When data is compiled into log 108, it may be compiled in an unstructured format, a semi-structured format, and / or a first format that is incompatible with or customized for the particular type of analysis being performed on the data. For example, the data may be compiled into log 108 as consecutive rows of raw data in the order in which it was generated. Analysis of log 108 and compression of log 108 may be performed on the unstructured data in log 108. Compression manager 110 may include instructions executable by processor 104 to format and / or reformat the data in log 108 into a different structure in compressed log 112.

[0057] For example, the identified target data within the log 108 may be preserved for inclusion in the compressed log 112. The identified target data from the log 108 may be formatted and / or reformatted from its native format in the log 108 into, for example, a tabular format or a summarized format of the compressed log 112 that is specifically tailored for further analysis (e.g., debugging) that may be performed on the log data after compression.

[0058] To free up additional memory capacity in the memory resource 106, the compression manager 110 may include instructions executable by the processor 104 to move the compressed log 112 out of the memory resource 106 and to another memory resource (not illustrated). For example, the compressed log 112 may be moved on the fly. That is, while data from the log 108 is being compressed, the compressed log 112 may be moved from the memory resource 106 to the other resource. That is, the remaining data in the compressed log 112 may be moved continuously and / or simultaneously with its inclusion in the compressed log 112.

[0059] Before the execution of the main program's computing operations is completed and / or before the compilation of data from that execution into log 108 is completed, compressed log 112 can be moved from memory resource 106 to another resource. Thus, compressed log 112 moved from memory resource 106 to another resource may not be the complete log that would result from the completed execution of the main program's computing operations. Instead, compressed log 112 moved from memory resource 106 to another resource may be the result of analyzing and compressing only a portion of the complete log that would result from the completed execution of the main program's computing operations. That is, the data in log 108 may be analyzed and compressed in a piecemeal manner until the execution of the main program's computing operations is completed. Thus, during the execution of the main program's computing operations, multiple compressed logs 112 may be generated. Thus, each of the multiple compressed logs, each representing a portion of the entire data that would be compressed from a complete and intact execution of the main program's computing operations, can be moved to another memory resource as it is generated.

[0060] Alternatively, the compressed log 112 may be periodically moved out of the memory resource 106 and to another memory resource. For example, the compressed log 112 may be moved out of the memory resource 106 and to another memory resource according to a schedule. For example, during the execution of the computing operation of the main computing program that generates the log 108 data, the compressed log 112 may be scheduled to be moved from the memory resource 106 and to another memory resource every two seconds.

[0061] Moving the compressed log 112 from the memory resource 106 to another memory resource may include moving the compressed log to a different type of memory resource. For example, the memory resource 106 may be a volatile memory resource, while the other memory resource may be a non-volatile memory resource. In some examples, the memory resource 106 may be RAM memory of the computing system 102, while the other memory resource may include a hard drive of the computing system 102. The memory resource 106 may be a dynamic random access memory (DRAM) resource or a static random access memory (SRAM) resource, while the other memory resource may be a solid-state drive (SDD) that utilizes NAND flash memory for storage. Thus, the memory resource 106 may be able to store and retrieve data at a faster rate than the other memory resource.

[0062] By clearing the compressed log 112 from the memory resource 106 during execution of the computing operations of the main program, additional memory capacity of the memory resource 106 can be freed. Thus, the portion of the memory resource 106 previously occupied by the compressed log 112 can be freed to accept additional data and / or be used for execution of the underlying computing operations (e.g., to contain instructions executable to perform the underlying computing operations, to contain data used to perform the underlying computing operations, to contain data resulting from the underlying computing operations, to contain additional log data, etc.). Thus, out-of-memory errors in the computing system 102 can be reduced or eliminated by reducing the portion of the capacity of the memory resource 106 occupied by the compressed log 112 before compilation of the log 108 is complete and / or before execution of the computing operations of the main program is complete. That is, the freed memory capacity in the memory resource 106 can then be used to execute the computing operations of the main computing program, preventing the triggering of out-of-memory errors.

[0063] In addition to the above-described analysis, compression, and movement of log data, the compression manager 110 may also include instructions executable by the processor 104 to analyze non-log data generated by the execution of the main computing operations of the main computing program within the memory resource 106 to identify target data (e.g., specific statistical output from applying a model to a data set, etc.) within the non-log data generated thereby. Similarly, this analysis may be performed before the execution of the main computing operations is completed. Similarly, the compression manager 110 may include instructions executable by the processor 104 to compress the non-log data generated by the execution of the main computing operations of the main computing program within the memory resource 106 based on the analysis before the execution of the main computing operations is completed. Furthermore, the compression manager 110 may include instructions executable by the processor 104 to move the compressed non-log data from the memory resource 106 to another memory resource before the execution of the main computing operations is completed. Removing the compressed non-log data may also free up memory capacity in the memory resource 106, which can then be used to execute the computing operations of the main computing program, thereby preventing the triggering of an out-of-memory error.

[0064] Figure 2 An example of a computing device 230 for log compression consistent with the present disclosure is illustrated. The components and / or operations described with respect to the computing device 230 may include Figure 1 and Figures 3 to 4 The components and / or operations described and / or with respect to Figure 1 and Figures 3 to 4 The components and / or operations described are interchangeable.

[0065] The computing device 230 may include a server, a laptop, a desktop, a tablet, a smartphone, a smart device, an IoT device, etc. The computing device 230 may include a processor 232 and / or a non-transitory memory 234. The non-transitory memory 234 may include instructions (e.g., 236, 238, etc.) that, when executed by the processor 232, cause the computing device 230 to perform the various operations described herein. Although the computing device 230 is illustrated as a single component, it is contemplated that the computing device 230 may be distributed among and / or include multiple such components.

[0066] Computing device 230 may include instructions 236 executable by processor 232 to identify target data within a system log. The system log may include data indicating the health of computing device 230, such as log data and statistical data. The system log may be compiled from data generated by the execution of computing operations by a host computing program. The host computing program may include an operating system and / or computing applications executed by computing device 230. Target data within the system log may be identified by analyzing the system log when the system log is compiled from the execution of computing operations by the host computing program. The system log may be compiled in a memory resource.

[0067] The memory resource may be the same memory resource used to execute the computing operations of the main computing program. For example, the memory resource may be a volatile RAM memory resource, in which data used to execute the computing operations of the main computing program is also stored for access by the processor. In other words, the memory resource may be the same RAM used to execute the computing operations of the main computing program. In other words, a portion of the memory resource used to execute the operations of the main computing program may contain system log data. Thus, the same memory resource can be used for the execution of the operations of the main computing program from which the system log is compiled, for identifying target data within the system log, and for discarding a portion of the system log selected based on the target data identified within the system log.

[0068] To identify target data within the system log, analysis of the system log can be performed within the memory resource as the system log is compiled. That is, the analysis and / or resulting identification can be performed using a portion of the memory resource that would otherwise be available for execution of the main computing operation in order to analyze the system log data compiled in the same memory resource as a result of the execution of the main computing operation.

[0069] Analysis may include analyzing system log data compiled within the system log to identify target data within the system log data. The target data may include data within the system log that is particularly useful for subsequent analysis and should be retained, and / or target data within the system log that is not particularly useful for subsequent analysis and should be discarded. The identification of the target data may be based on definitions and / or definition parameters provided by a user, administrator, analysis platform, etc. In other words, the data of particular interest to the analysis may be defined by a specific user, administrator, analysis platform, etc. and / or may be unique to the specific user, administrator, analysis platform, etc.

[0070] The analysis of system log data and / or the resulting identification of target data within the system log can be performed continuously and / or concurrently with the generation of the system log data and / or the compilation of the system log data into the system log. In some instances, a portion of the system log can be accumulated within the memory resource across a time interval window before the compilation of the system log is complete. For example, during the execution of a primary computing operation, log data can be accumulated in the system log for up to two seconds. After the two seconds have elapsed, the accumulated data in the system log can be analyzed. That is, the system log can be analyzed by searching the accumulated portion of the system log after the compilation of the accumulated portion is complete and before the compilation of the entire system log for the execution of the primary computing operation is complete. This process can be repeated for each consecutive two-second period until the compilation of the entire system log is complete to execute the primary computing operation and / or until the execution of the primary computing operation that generated the log data is fully completed.

[0071] Computing device 230 may include instructions 238 executable by processor 232 to compress system logs. Because system logs are compiled and analyzed within the same memory resources used to execute computing operations of the main computing program, compression can also be achieved by discarding data within the same memory resources. Furthermore, because compilation, analysis, and / or identification of target data within the system logs occurs during and / or before the completion of computing operations, compression can also be achieved during and / or before the completion of computing operations.

[0072] Compressing a system log may include reducing the amount of data stored in the log when the system log is compiled and / or before the execution of the computing operation that generated the system log is completed. For example, compressing the system log may include discarding a portion of the system log stored therein from a portion of a memory resource. As described above, the memory resource in which the system log is compiled may be the same memory resource used to execute the computing operations of the main computing program. Thus, unused capacity of the memory resource is available for execution of the computing operations of the main computing program. Therefore, discarding a portion of the system log from the memory resource may free up additional unused capacity for execution of the computing operations of the main computing program.

[0073] The portion of the system log that is discarded during compression can be selected based on identified target data within the system log. For example, the above-described analysis can identify target data within the system log. The presence of target data within the system log can indicate that the system log data is particularly useful for subsequent analysis and / or is itself particularly useful for subsequent analysis. Thus, that system log data can be selected to be retained from the system log. Conversely, system log data that is not target data and / or is not indicated by the target data as being particularly useful for subsequent analysis can be selected as system log data to be discarded during compression.

[0074] In some examples, non-system log data may also be compressed within the memory resource. For example, non-system log data generated by the execution of computing operations of the main computing program may be compressed before the execution of the computing operations of the main computing program is completed. Compression may include discarding a portion of the non-system log data generated by the execution of computing operations of the main computing program within the memory resource. The non-system log data within the memory resource to be discarded may be selected based on an analysis of the non-system log data when it is stored in the memory resource as identified target data.

[0075] In both the system log data and non-system log data examples described above, the non-discarded and / or target data in the compressed memory resource can be pushed and / or written to a different memory resource. In some examples, the different memory resource can be a non-volatile memory resource. The different memory resource can include a memory resource that does not directly participate in the execution of the computing operations of the main computing program. For example, the different memory resource can include an SSD memory drive of the computing device 230 that is not directly accessed by the processor 232 during the execution of the computing operations of the main computing program.

[0076] Compressing system log data and / or non-system log data within a memory resource and pushing and / or writing the compressed system log data and / or non-system log data from the memory resource to different memory resources frees up capacity in the memory resource. As described above, the unused capacity of the memory resource can be used for the execution of the computing operations of the main computing program. Compressing system log data and / or non-system log data within a memory resource and pushing and / or writing the compressed system log data and / or non-system log data from the memory resource to different memory resources can return the previously occupied capacity to unused capacity that can be used for the execution of the computing operations of the main computing program. Therefore, compressing system log data and / or non-system log data within a memory resource and pushing and / or writing the compressed system log data and / or non-system log data from the memory resource to different memory resources can reduce and / or or eliminate out-of-memory errors in the computing device 230.

[0077] Figure 3 An example of a non-transitory memory 342 and a processor 340 for log compression consistent with the present disclosure is described. Memory resources, such as non-transitory memory 342, may be used to store instructions (e.g., 344, 346, 348, etc.). The instructions may be executed by the processor 340 to perform operations as described herein. The operations are not limited to the specific examples described herein and may include operations related to Figures 1 to 2 and Figure 4 The components and / or operations described and / or with respect to Figures 1 to 2 and Figure 4 The components and / or operations described are interchangeable.

[0078] Non-transitory memory 342 may store instructions 344 executable by processor 340 to identify target data within the system log. Identifying target data within the system log may include analyzing data compiled within the system log as the system log is compiled from execution of computing operations of the host computing program.

[0079] Log data may be compiled in a system log stored in a first portion of a volatile memory resource. Specifically, the volatile memory resource may be a volatile memory resource whose other portion is used for the execution of computing operations of a main computing program that generates the log data. Thus, analysis of the system log may include analysis of the log data within the first portion of the volatile memory resource in which the system log is compiled. That is, a portion of the memory resource used for the execution of operations of the main computing program may contain data from the system log. Thus, the same memory resource may be used for the execution of operations of the main computing program from which the system log is compiled, for identifying target data within the system log, and for extracting the identified target data from the system log within the resource during the execution of the operations.

[0080] Log data compiled in a system log can be analyzed to identify target data within the data in the system log. For example, the data in the system log can be searched for the presence of the target data. The data in the system log can be searched when the system log is compiled.

[0081] The target data may include logs indicating the source of errors that occurred during the execution of computing operations of the main computing program, triggered by system activities. For example, the target data may include a specific sequence of key refreshes, service restarts, database synchronizations, login timestamps, error messages, etc., caused by the execution of computing operations of the main computing program.

[0082] A user interface may be provided that facilitates configuration of an identification code for data targeted for analysis. The identification code for the target data may be determined based on an indication received at this user interface defining characteristics and / or identification codes for the target data.

[0083] The non-transitory memory 342 may store instructions 346 executable by the processor 340 to extract target data from the system log. As described above, analysis of the system log may be used to identify target data within the system log. The system log may be compiled and analyzed in volatile memory resources that are also used for the execution of computing operations of the main computing program. When the system log is compiled, the target data identified within the system log may be extracted from its storage location in the volatile memory resources.

[0084] Extracting the target data may include reducing the size of the system log by deleting portions of the system log before the compilation of the system log is completed. For example, unextracted portions of the system log may be deleted from the system log before the compilation of the system log is completed. Additionally, redundant instances of the target data may be removed from the system log in the volatile memory resource before the compilation of the system log is completed.

[0085] Thus, extracting target data from a system log may result in a compressed system log. A compressed system log may contain less data than the uncompressed system log used as the source of the compressed system log. Consequently, more volatile memory resource capacity may be freed up to handle other data associated with the execution of computing operations of a host computing program that utilizes volatile memory resources. Furthermore, because analysis and / or extraction may occur before the compilation of the system log is complete and / or the execution of computing operations is complete, volatile memory resource capacity may be freed up during the execution of the computing operations that may have been utilized, rather than after the execution ends or fails.

[0086] The non-transitory memory 342 may store instructions 348 executable by the processor 340 to convert the extracted target data into structured data. The extracted target data may be unstructured, semi-structured, and / or have a structure that is incompatible with an analysis protocol subsequently applied to the data. Thus, the extracted target data may be structured and / or restructured into a data structure that is compatible with the subsequently applied analysis protocol. For example, converting the extracted target data into structured data may include compiling a summary statistical report of the extracted target data. That is, converting the extracted data into structured data may include analyzing the extracted data, determining summary statistics of the data, and / or formatting the summary statistics in a tabular format that is compatible with the subsequently applied analysis protocol and / or that can be quickly evaluated and / or understood by a user.

[0087] The extracted target data can be converted into structured data before the compilation of the system log is complete. That is, the conversion can be performed while the system log is still being compiled from the execution of computing operations of the main computing program. In some examples, the conversion can be performed on a portion of the system log data present in the volatile memory at the time of analysis and extraction, but any system log data accumulated in the volatile memory from that time can be the subject of subsequent analysis, extraction, and / or conversion into structured data.

[0088] Additionally, structured data may be sent from the volatile memory resource to the non-volatile memory resource for storage. The structured data may be sent to the non-volatile memory resource for storage before the compilation of the system log is complete. That is, while the system log is still being compiled from the execution of computing operations by the main computing program, the structured data may be sent to the non-volatile memory resource for storage. In some examples, the structured data may be sent to the non-volatile memory resource for storage as part of the system log data present in the volatile memory at the time of analysis, identification, extraction, and conversion, but any system log data accumulated in the volatile memory since then may be the subject of subsequent analysis, identification, extraction, conversion, and / or sending to the non-volatile memory resource for storage.

[0089] After the structured data is sent to the non-volatile memory resource for storage, the parsed portion of the system log can be deleted from the volatile memory resource. That is, once the analysis of a portion of the system log data in the volatile memory has completed to generate the extracted and structured data, the source system log data in the volatile memory can be deleted from the volatile memory. Furthermore, this deletion can occur before the compilation of the system log is complete. In this way, while the compilation of the system log data into the system log proceeds, the portion of the volatile memory storing the parsed, extracted, and converted data can be freed to further participate in the execution of computing operations by the main computing program.

[0090] Figure 4 An example of a method 450 for log compression consistent with the present disclosure is described. The described components and / or operations of the method 450 may include Figures 1 to 3 The components and / or operations described and / or with respect to Figures 1 to 3 The components and / or operations described are interchangeable.

[0091] At 452, method 450 may include identifying target data within a system log during execution of a computing operation from which the system log is compiled. For example, the target data may be identified within the system log by analyzing the system log during execution of the computing operation from which the system log is compiled to search for the target data. The system log may be compiled in a memory resource.

[0092] The memory resource can be the same memory resource used to perform the computing operation. Thus, the identification of target data within the memory resource can be performed when compiling the system log. That is, a portion of the memory resource originally available for the execution of the operation can be utilized to perform analysis of the system log to search for target data. In other words, a portion of the memory resource used for the execution of the operation can contain system log data. Thus, the same memory resource can be used for the execution of the operation from which the system log is compiled, for identifying target data during the execution of the operation, and for compressing the system log within the resource.

[0093] Identifying target data within the system log may include analyzing the system log data compiled within the system log to identify the target data contained therein. The target data may include data within the system log that is particularly useful for subsequent analysis and should be retained, and / or target data within the system log that is not particularly useful for subsequent analysis and should be discarded. The identifier of the target data may be based on definitions and / or definition parameters provided by a user, administrator, analysis platform, etc. In other words, the data of particular interest for analysis may be defined by a specific user, administrator, analysis platform, etc. and / or may be unique to the specific user, administrator, analysis platform, etc.

[0094] Identifying target data within a system log may include applying a machine learning model to data compiled in the system log to identify target data within the system log that is particularly useful for subsequent analysis and should be retained, and / or target data within the system log that is not particularly useful for subsequent analysis and should be discarded. In some examples, that analysis may include analyzing the data compiled in the system log using a recurrent neural network trained to analyze characters within the system log and identify target data within the characters. As described above, the identifier of the target data may be based on a definition and / or definition parameters provided by a user, administrator, analysis platform, etc. Thus, a recurrent neural network may be trained to identify target data within a system log based on the definition and / or definition parameters provided by a user, administrator, analysis platform, etc.

[0095] In some examples, identifying the target data within the system log includes analyzing data compiled in the system log by performing a regular expression pattern search of characters within the system log to identify the target data among the characters within the system log. In some examples, the analyzing may include analyzing the system log by searching the characters within the system log for a flag sequence to identify the target data among the characters within the system log.

[0096] Analysis of the data compiled in the system log and / or the resulting identification of target data may be performed before the completion of execution of the computing operations of the main computing program that generated the data compiled into the system log. That is, analysis of the data and / or the resulting identification of target data may occur concurrently with the compilation of the log data and / or before the completion of the entire log file associated with the execution of the computing operations.

[0097] At 454, method 450 may include compressing the system log within the memory resource. Because the system log is compiled within the memory resource used to execute the computing operations of the main computing program and because analysis of the system log can be performed within the memory resource during execution and / or before execution is completed, compression can also be performed within the memory resource and during execution of the computing operations.

[0098] Compressing the system log can be performed based on the aforementioned identification of target data. Specifically, how to compress data, what data to extract, what data to save, what data to delete, etc. can be determined based on the results of the aforementioned analysis and / or the identified target data. For example, the identified target data within the system log can indicate identified data that should be retained from the system log and / or data that should not be retained in the system log.

[0099] Compressing the system log may include reducing the amount of log data stored in the system log during the execution of the underlying computing operations and while the system log is still being compiled. Reducing the amount of log data stored in the system log may include deleting log data other than target data identified within the system log from the system log in the memory resource. That is, reducing the amount of log data stored in the system log may include deleting all non-target log data and / or all data not indicated by the target data as being particularly useful to preserve (e.g., particularly useful for performing subsequent analysis such as debugging operations). In order to further reduce the amount of log data stored in the system log during the execution of the underlying computing operations and while the system log is still being compiled, the target log data that is exempt from deletion in the system log may be pushed and / or written to a different memory resource. For example, the target log data may be pushed and / or written to an SSD memory resource that is not actively used in the execution of computing operations of the main computing program that generates the system log data.

[0100] Additionally, the compressed data may include reducing the amount of log data stored in the system log by extracting target data from the system log to generate a compressed system log during the execution of the underlying computing operations and while the system log is still being compiled. Remaining data in the system log within the portion of the system log from which the target data was extracted may be deleted. The extracted data may be included in the compressed system log in the memory resource. To further reduce the amount of log data stored in the system log during the execution of the underlying computing operations and while the system log is still being compiled, data representing the compressed system log may be pushed from the memory resource and / or written to a different memory resource. For example, the target log data may be pushed and / or written to an SSD memory resource that is not actively used in the execution of the computing operations of the main computing program that generates the system log data.

[0101] By compressing system logs within a memory resource during the execution of computing operations that generate system log data, additional memory capacity can be freed up within the memory resource. By pushing and / or writing the compressed system log data from the memory resource to a different memory resource, even more memory capacity can be freed up within the memory resource. The free capacity within the memory resource can be used to store additional data associated with the execution of computing operations, including additional system log data, additional computing program data, additional input data, additional output data, and the like. Therefore, compressing system logs within a memory resource and pushing and / or writing the system logs from the memory resource during the execution of computing operations can reduce or eliminate the occurrence of out-of-memory errors. Furthermore, the demand for memory resources during the execution of computing operations can be reduced, thereby allowing computing systems to be constructed with less relatively expensive volatile memory utilized during the execution of computing operations. That is, embodiments consistent with the present disclosure can reduce the incidence of volatile memory capacity acting as a bottleneck in the execution of computing operations.

[0102] In the foregoing detailed description of the present disclosure, reference is made to the accompanying drawings that form a part of the present disclosure, and examples of how the present disclosure may be practiced are shown in the accompanying drawings in an illustrative manner. These examples are described in sufficient detail to enable one of ordinary skill in the art to practice the examples of the present disclosure, and it should be understood that other examples may be utilized and process, electrical and / or structural changes may be made without departing from the scope of the present disclosure. In addition, as used herein, "a plurality of" elements and / or features may refer to more than one such element and / or feature.

[0103] The figures herein follow a numbering convention in which the first digit corresponds to the figure number and the remaining digits identify the element or assembly in the figure. The elements shown in the various figures herein may be added, exchanged, and / or eliminated to provide several additional examples of the present disclosure. In addition, the proportions and relative scales of the elements provided in the figures are intended to illustrate examples of the present disclosure and should not be construed as limiting.

Claims

1. A method for compressing a log, comprising: analyzing a system log during execution of a computing operation to identify target data within the system log, the system log being compiled from the execution of the computing operation into a volatile memory resource utilized in the execution of the operation, wherein the analyzing is performed using a portion of the volatile memory resource that would otherwise be available for utilization in the execution of the computing operation, wherein the portion of the volatile memory resource includes data of the system log; and During the execution of the computing operation, the system log stored within the volatile memory resource is compressed based on identifying the target data to obtain a compressed system log. 2 . The method of claim 1 , comprising extracting, from the system log in the memory resource, the target data identified within the system log. 3 . The method of claim 1 , comprising deleting data from the system log in the memory resource other than the target data identified within the system log.

4. The method of any one of claims 1 to 3, comprising utilizing a recurrent neural network to identify characters within the system log that correspond to the target data.

5. The method of any one of claims 1 to 3, comprising performing a regular expression pattern search to identify characters within the system log that correspond to the target data.

6. The method of any one of claims 1 to 3, comprising searching characters within the system log for flag sequences to identify the target data within the system log.

7. The method of any one of claims 1 to 3, comprising writing data representing the compressed system log to a second memory resource.

8. A system for compressing logs, comprising: processor; and A non-transitory machine-readable storage medium storing instructions executable by the processor to: When instructions for performing main program operations are retrieved from a memory resource and executed by said processor, data constituting a system log is generated and compiled in said memory resource; identifying target data within the system log when the system log is compiled from execution of the main program operation; and A portion of the system log selected based on the target data identified within the system log is discarded from a portion of the memory resource.

9. The system of claim 8, wherein the memory resource is a volatile random access memory resource in which data used for the execution of the main program operation is saved for access by the processor.

10. The system of claim 9, comprising instructions executable by the processor to write the target data to a different memory resource, wherein the different memory resource is a non-volatile memory resource.

11. The system of any one of claims 8 to 10, including instructions executable by the processor to compress non-system log data generated by the execution of the main program operations in the memory resource before the execution of the main program operations is completed.

12. The system of any one of claims 8 to 10, comprising instructions executable by the processor to accumulate a portion of the system log within the memory resource across a time interval window before compilation of the system log is completed to obtain an accumulated portion of the system log.

13. The system of claim 12, comprising instructions executable by the processor to search the accumulated portion of the system log after compilation of the accumulated portion of the system log is completed and before the compilation of the system log is completed.

14. The system of any one of claims 8 to 10, wherein the system log includes log data and statistical data indicative of a health status of the system.

15. A non-transitory machine-readable storage medium comprising instructions executable by a processor to: When instructions for performing main program operations are retrieved from a memory resource and executed by said processor, data constituting a system log is generated and compiled in said memory resource; identifying target data within said system log when compiling said system log from said execution of a main program; extracting the target data from the system log from a volatile memory resource used for the execution of the main program to obtain extracted target data when compiling the system log; and The extracted target data is converted into structured data. 16 . The non-transitory machine-readable storage medium of claim 15 , wherein the target data comprises a log triggered by system activity indicating a source of an error occurring during the execution of the main program. 17 . The non-transitory machine-readable storage medium of claim 15 , comprising instructions executable by the processor to convert the extracted target data into structured data by compiling a summary statistical report of the extracted target data.

18. The non-transitory machine-readable storage medium of any one of claims 15 to 17, comprising instructions executable by the processor to: Before the compilation of the system log is completed, writing the structured data from the volatile memory resource to a non-volatile memory resource for storage; and The analyzed portion of the system log is deleted from the volatile memory resource.

19. The non-transitory machine-readable storage medium of any one of claims 15 to 17, comprising instructions executable by the processor to configure an identification code of the target data based on an indication received at a user interface.

20. The non-transitory machine-readable storage medium of any one of claims 15-17, comprising instructions executable by the processor to remove redundant instances of the target data from the system log before the compilation of the system log in the volatile memory resource is completed.

Images