Method, apparatus and computer program product for processing logs

Abstract

The embodiment of the invention relates to a method and equipment for processing logs and a computer program product. The method comprises the following steps: determining a target log recording component from a group of candidate log recording components based on the duration of a predetermined number of log items recently recorded by the log recording component; the method further includes determining a log group including a plurality of similar log items based on a predetermined number of log items of the recently recorded of the target log recording component. The method also includes determining the same content between the plurality of log items in the log group and a particular content for each log item in the plurality of log items. The method further includes updating the plurality of log entries by deduplicating the same content and preserving particular content for each of the plurality of log entries. Through the method, which log recording component needs to be processed can be quickly and accurately determined, and the same content is deduplicated, so that the stored data volume and the required storage resources are reduced, and the resource utilization rate is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of data processing, and in particular to a method, device, and computer program product for processing logs. Background Art

[0002] In the development of computer technology, in order to understand the operations performed by hardware or software systems, hardware and software systems usually record log files. Many log items are stored in log files to record the operations performed in hardware or software systems. For log items, they are permanently stored in log files as a series of tags and data. When recording log items, some tags / values ​​are usually implicitly added by the log mechanism as trusted fields, while other log item contents are defined by the logging component.

[0003] When a logging component such as a hardware or software system records log items in a log file, some log data recorded in the log file is redundant and / or repeated. In particular, when a problem occurs in the logging component, the recorded log items usually contain a large amount of irrelevant repeated content. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, a device, and a computer program product for processing logs.

[0005] According to a first aspect of the present disclosure, a method for processing logs is provided. The method includes determining a target log recording component from a group of candidate log recording components based on the duration of a predetermined number of log items recently recorded by the log recording component. The method also includes determining a log group including a plurality of similar log items based on a predetermined number of log items recently recorded by the target log recording component. The method also includes determining the same content between the plurality of log items in the log group and the specific content for each of the plurality of log items. The method also includes updating the plurality of log items by deduplicating the same content and retaining the specific content for each of the plurality of log items.

[0006] According to a second aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon, the instructions causing the device to perform actions when executed by the at least one processor, the actions comprising: determining a target log recording component from a set of candidate log recording components based on the duration of a predetermined number of log items recently recorded by the log recording component; determining a log group including a plurality of similar log items based on a predetermined number of log items recently recorded by the target log recording component; determining identical content between a plurality of log items in the log group and specific content for each of the plurality of log items; and updating the plurality of log items by deduplicating identical content and retaining specific content for each of the plurality of log items.

[0007] According to a third aspect of the present disclosure, there is provided a computer program product, which is tangibly stored on a non-volatile computer-readable medium and comprises machine-executable instructions, which when executed cause a machine to perform the steps of the method in the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0009] Figure 1 A schematic diagram illustrating an example environment in which devices and / or methods according to embodiments of the present disclosure may be implemented;

[0010] Figure 2 A flowchart of a method for processing logs according to an embodiment of the present disclosure is illustrated;

[0011] Figure 3 A schematic diagram illustrating multiple log items of a component according to an embodiment of the present disclosure is illustrated;

[0012] Figure 4 A schematic diagram illustrating an example process for triggering a log merging event of a component according to an embodiment of the present disclosure is illustrated;

[0013] Figure 5 A schematic diagram illustrating an example process for grouping a plurality of log items according to an embodiment of the present disclosure is illustrated;

[0014] Figure 6 A schematic diagram illustrating an example of the same content for multiple log items and specific content for each log item according to an embodiment of the present disclosure;

[0015] Figure 7A schematic diagram illustrating an example of a log entry according to an embodiment of the present disclosure;

[0016] Figure 8 A schematic diagram illustrating an example of identical contents of multiple log items according to an embodiment of the present disclosure;

[0017] Fig. 9 A schematic block diagram of an example device suitable for implementing embodiments of the present disclosure is illustrated.

[0018] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] As described above, some log data recorded to files is redundant and / or duplicated and contains irrelevant duplicate content. Since the space for storing logs is limited, in some cases, when analyzing key issues, it is not possible to find key messages due to log overwriting / refreshing. Redundant and repeated recording of some log data will take up extra space and cause log rolling or overlap, especially when the system is operating abnormally. A traditional solution is to adjust the logging components such as software systems to avoid recording duplicate logs. However, for large software systems, it is difficult to address all error handling code paths and ensure that each component does not have an infinite loop to avoid continuous logging. In addition, the large size of log files and the large amount of logs also increase the workload of analysis engineers.

[0022] At least to solve the above and other potential problems, an embodiment of the present disclosure provides a method for processing logs. In this method, a computing device determines a target log recording component from a group of candidate log recording components based on the duration of a predetermined number of log items recently recorded by the log recording component. Then the computing device determines a log group including multiple similar log items based on the predetermined number of log items recently recorded by the target log recording component. The computing device then determines the same content between multiple log items in the log group and the specific content for each log item in the multiple log items. Finally, the computing device updates multiple log items by deduplicating the same content and retaining the specific content for each log item in the multiple log items. Through this method, it is possible to quickly and accurately determine which log recording component needs to be processed, and by deduplicating the same content, the amount of stored data and the required storage resources are reduced, thereby improving resource utilization.

[0023] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. Figure 1 An example environment in which the devices and / or methods of embodiments of the present disclosure may be implemented is shown. Figure 1 As shown, the example environment 100 includes a computing device 104, which is used to find one or more target logging components 106 that may have logging anomalies from a set of candidate logging components 102. Then, the log items of the target logging components 106 are analyzed to process and adjust the log items with higher similarity.

[0024] Example computing devices 104 include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile devices, multiprocessor systems, consumer electronics, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0025] Figure 1 The set of candidate logging components shown in the figure may be any suitable hardware components or software systems, which will record the operations performed in their own log files when performing operations. Figure 1 The set of candidate logging components 102 is shown as being located outside the computing device 104, which is only an example and not a specific limitation of the present disclosure. The set of candidate logging components may also be located inside the computing device 104 or a portion of the set of candidate logging components may be located inside the computing device 104 and another portion of the logging components may be located outside the computing device 104.

[0026] The computing device 104 queries or analyzes a logging component in a set of candidate logging components 102 to determine whether the logging component is a target logging component 106 where a logging anomaly may occur. In some embodiments, the computing device 104 queries and analyzes each logging component in a set of candidate logging components 102 at predetermined intervals. In some embodiments, the computing device 104 may also use different time intervals for analysis of different logging components. In some embodiments, the computing device may query or analyze a logging component in a set of candidate logging components 102 at multiple predetermined times. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure.

[0027] After determining the target log recording component 106, the computing device 104 analyzes the predetermined number of log recording items newly generated by the target log recording component. For example, the K log recording items recently recorded by the target log recording component are obtained and then processed. The computing device 104 analyzes the obtained K recently recorded log recording items based on the similarity between the logs, and then groups them to group the K log recording items into one or more log groups 108. For the log items in the log group 108, they have a high degree of similarity.

[0028] Then, the computing device 108 further processes the log group 108, determines the same content 110 between each log item in the log group 108 and the specific content 112 for each log item in the log group 108. Then, the same content is deduplicated, only one copy of the same content is retained, and the specific content for each log item is retained. Then, the retained same content and the specific content for each log item are used to replace multiple logs in the log group and stored.

[0029] Through this method, it is possible to quickly and accurately determine which log record component needs to be processed, and by deduplicating the same content, the amount of stored data and the required storage resources are reduced, thereby improving resource utilization.

[0030] Combined with the above Figure 1 Describes an example environment in which the devices and / or methods of the embodiments of the present disclosure may be implemented. Figure 2 A flowchart of a method for processing a log according to an embodiment of the present disclosure is described. The method can be Figure 1 The system is executed on computing device 104 or any suitable computing device.

[0031] At block 202, a target logging component is determined from a set of candidate logging components based on the duration of a predetermined number of log items recently recorded by the logging component. For example, the computing device 104 determines whether the candidate logging component is a target logging component based on the duration of a predetermined number of log items recently recorded for each candidate logging component in the set of candidate logging components.

[0032] In some embodiments, computing device 104 selects a candidate logging component from a group of candidate logging components. Then, computing device 104 obtains a predetermined number of log items of the candidate logging component that have been recently recorded. Computing device 104 can determine the duration of the period of time in which the predetermined number of log items are generated based on the predetermined number of log items. Then determine whether the duration is greater than the threshold duration. If it is determined that the duration is greater than the threshold duration, computing device 104 will determine that the candidate logging component is not a target logging component that may generate a logging exception. At this point, the log items recorded by it are not processed, but the log items of the candidate logging component are stored according to the original strategy. If it is determined that the duration is less than or equal to the threshold duration, computing device 104 determines the candidate logging component as a target logging component that may generate a logging exception.

[0033] In some embodiments, when determining the duration of the time period corresponding to the predetermined number of log items, since the log items are arranged in order, the computing device 104 first determines the first log item from the predetermined number of log items, and then obtains the timestamp of the log item. Next, the timestamp of the last log item in the predetermined number of log items is determined. The duration of the time period in which the predetermined number of log items are located is determined by the timestamp of the first log item in the predetermined number of log items and the timestamp of the last log item in the predetermined number of log items.

[0034] For example, Figure 3 FIG. 1 shows a schematic diagram of multiple log items of a log recording component Ci according to an embodiment of the present disclosure. Figure 3 The K time points t before the current time point t are shown in ci,-1 302-1,t ci,-2 302-2,t ci,-3 302-3,…,t ci,-K 302-K, where K is a positive integer. For a logging component Ci, there are corresponding log items recorded by the component Ci at K time points, for example, log item 1 304-1, log item 2 304-2, log item 3 304-3, and log item K 304-K. The duration of the period corresponding to the most recent K log items of the logging component Ci is t ci,-1 -t ci,-K, which can be used to determine whether component Ci is a target log component where logging anomalies may occur.

[0035] Return now Figure 2 Next, it is described that in some embodiments, the computing device 104 usually determines the target logging component from a set of candidate logging components at predetermined intervals. In some embodiments, the computing device 104 can determine the target logging component from a set of candidate logging components at multiple predetermined moments. Alternatively or additionally, when the computing device determines whether each candidate logging component in a set of candidate logging components is the target logging component, the determination moment is different for different logging components. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure.

[0036] At block 204, a log group including a plurality of similar log entries is determined based on a predetermined number of log entries recently recorded by the target logging component. Figure 1 The computing device 104 in the embodiment analyzes and processes a predetermined number of log items recently recorded by the target log recording component 106 to determine a log group including a plurality of similar log items.

[0037] In some embodiments, when determining a log group including a plurality of similar log items, the computing device 104 first obtains a predetermined number of log items of the most recent record of the target log recording component. Afterwards, the computing device 104 can determine the similarity between the logs. Then, the computing device groups the predetermined number of log items according to the similarity between the log items to determine one or more log groups.

[0038] In some embodiments, the predetermined number of log items is determined as a log set. When determining a log group from a predetermined number of log items using the similarity between logs, the computing device may recursively perform the following operations. First, the computing device determines the number of log items in the log set. Then determine whether the number is greater than a threshold number, for example, the threshold number is 2. If the number is determined to be less than the threshold number, it indicates that the number of log items in the log item set is small and no further grouping is required. If the number is determined to be greater than the threshold number, it indicates that there are multiple log items in the log set, so it can be further grouped. At this time, a log item is selected from the log set as a reference log item. Then the reference log item will be used to initialize the log group, and the reference log item will be removed from the log set to update the log set. Then, the computing device 104 determines the similarity between each log item in the updated log set and the reference log item. If it is determined that the similarity exceeds the similarity threshold, the log item in the updated log set is added to the log group, and the log item is removed from the log set to further update the log set. If the similarity does not exceed the similarity threshold, it indicates that the two log items are not similar, and they are not processed, but continue to be retained in the log set. Then, the above operation is repeated on the log set after the above operation is completed until the log set is grouped.

[0039] In some embodiments, when determining the similarity between each log item in the updated log set and the reference log item, the computing device 104 may first remove the timestamps of the log item and the reference log item to generate an adjusted log item and an adjusted reference log item. Then, the computing device 104 determines the similarity between the adjusted log item and the adjusted reference log item as the similarity between the log item and the reference log item.

[0040] At block 206, the same content between the multiple log items in the log group and the specific content for each of the multiple log items are determined. For example, the computing device 104 determines the same content 110 between the multiple log items in the log group 108 and the specific content for each of the multiple log items in the log group.

[0041] In some embodiments, when determining the same content between multiple log items in a log group and the specific content for each log item in the multiple log items, the computing device first compares the multiple log items and determines the same content by comparison. The computing device then determines the content other than the same content in each log item as the specific content for the log item. In some embodiments, the computing device 104 can use various suitable algorithms to process multiple log items in the log group to determine the same content and the specific content for different log items. The above examples are only used to describe the present disclosure, but are not specifically limited to the present disclosure.

[0042] At block 208, multiple log items are updated by deduplicating the same content and retaining the specific content for each of the multiple log items. For example, the computing device 104 updates the multiple log items by deduplicating the same content and retaining the specific content for each of the multiple log items. The deduplicated same content and the specific content for each log item are then stored to replace the multiple log items in the log group.

[0043] In some embodiments, multiple log items can be further displayed by combining the same content of multiple log items in the log group with the specific content of each log item. If it is determined that multiple log items are to be displayed, the computing device combines the same content with the specific content of each log item in the multiple log items to generate multiple log items. Then, the multiple log items are displayed.

[0044] In some embodiments, the computing device can provide a user with a selectable display format, such as a hidden format and other formats. Therefore, when displaying a log, the computing device first determines the display format of multiple log items. If the display format is determined to be a hidden format, the computing device only displays the same content and the specific content for each of the multiple log items. If the display format is determined not to be a hidden format, multiple log items are generated by combining the same content with the specific content for each of the multiple log items. Then, the multiple log items are displayed.

[0045] Through this method, it is possible to quickly and accurately determine which log record component needs to be processed, and by deduplicating the same content, the amount of stored data and the required storage resources are reduced, thereby improving resource utilization.

[0046] Combined with the above Figure 2 and Figure 3 A flow chart of a method for processing logs according to an embodiment of the present disclosure is described. Figure 4 and Figure 5 An example for processing log data according to an embodiment of the present disclosure is described. In this example, an event trigger analysis module and a similarity check and merge module are used. The event trigger analysis module analyzes the log status of each component and identifies abnormal log recording components. Specifically, the event trigger analysis module triggers a component log merge (CLM) event when certain conditions are met by tracking the recorded log items. Next, the similarity check and merge (SCM) module adaptively merges / deduplicates the log items of the logging components.

[0047] The execution process of the event-triggered analysis module can be found in Figure 4 An example process for triggering a log merge event for a component is shown. Figure 4 The process can be found in Figure 1 The system is executed on computing device 104 or any suitable computing device.

[0048] In example 400, starting at block 402 and proceeding to block 404, the computing device performs a predetermined time interval T. Interval , for the i-th component Ci in the component set C, calculate the duration Log of the current predetermined number of log items dur =time_stamp(last_log)-time_stamp(first_log), where T Interval It is used to check whether the time interval for triggering CLM events is met; Log Dur is a variable that represents the duration of the period of the predetermined number of logs in the current log; last_log represents the last log item in the predetermined number of log items; first_log represents the first log item in the predetermined number of log items; the component set C can be represented as a list of log record components, where C = [C1, C2, ..., CN], N represents the number of log record components, 1≤i≤N. Next, at block 406, determine Log dur >Log DurThreshold Is it true, where Log DurThreshold is a duration threshold used to check whether a CLM event is triggered. In some embodiments, Log Dur is an element in D, D = [d1, d2, d3, ..., dN], where di = tt ci，-K is the time interval between the current time point t and the time point t of the last K messages from component Ci ci，-K The time interval between the two, K represents the number of log entries used to calculate the duration D. If component Ci does not have message K, then t ci，-K =0.

[0049] If Log dur Greater than Log DurThreshold , then the operation ends at box 410; if Log dur No greater than Log DurThreshold , then at box 408, a log merge CLM event is triggered for the component Ci that frequently logs items. M is a set of log components that may have log exceptions to be processed. When a CLM event is triggered, a component is added to M. Alternatively or additionally, the component corresponding to the element with the minimum value in the set D can be determined as the target log component that may have log exceptions. For example, M = M.append(Cj), where M.append() represents an add function, Wherein arg.(Min(D)) represents the subscript of the element with the minimum value in the set D; Min() represents taking the minimum value. The above examples are only used to describe the present disclosure, but are not specifically limited to the present disclosure.

[0050] For the target logging component that may fail, considering that similar logs are generated cyclically, a log item merge operation can be created for the latest K log items in the target logging component. The similarity check and merge algorithm is applied to adaptively delete the duplicate content in the K log items. In this process, similar log items are first divided into several groups, and then the same content in each group is deduplicated.

[0051] Reference below Figure 5 Describe the process of grouping. Figure 5 As shown, start at box 502, then determine the log set R to be processed at box 504, where R represents a set of log items to be processed of the target logging component, where R = [r1, r2, ..., rK], ri is the i-th message most recently received when the CLM event is received. The timestamp of each message serves as the fingerprint of the message. At box 506, check whether the length len(R) of the log set R > 2 is true. If len(R) > 2 is not true, end the operation at box 514. If len(R) > 2, go to box 508 and perform operation target str = R.pop(1) and G = [target str ], R.pop() means popping a log item from the set R as the target string target str , which can usually be the latest log entry in R, G = [target str ] indicates the use of the target string target str Initialize G. Then at block 510, for each ri in R, use target str Start similarity checking and group similar logs into G. In this process, first calculate simi ratio =similarity_check(target str , ri), where simi ratio Is a variable that represents the similarity ratio of two message strings; similarity_check() is used to compare two strings and get the similarity. ratio >simi threshold , execute operations G.append(ri) and R.pop(ri), G.append() means adding log items to G, and R.pop() means popping out log items in R, where simi Thresholdis the threshold for determining whether a message is classified into a group. G represents a group of log items with similar content, and the same content in a group of log items will be deduplicated. At block 512, G is added to the group set ∧: ∧.append(G), where Represents a set of groups after grouping similar messages for content deduplication. G stores a set of messages that are similar to target. str The most similar log items. For each log item in each G, the same content is deduplicated, which means that only one copy of the same content is needed in storage, thereby reducing the space required for log messages. After the operation in block 512 is completed, the remaining log items in group G can continue to be grouped.

[0052] In some embodiments, each log item ri can be retrieved by concatenating the same content and specific content of ri. Figure 6 , which shows a schematic diagram of an example of the same content and specific content for a log item according to an embodiment of the present disclosure. Example 600 records that after the above operations, the same content 602 for a log group and specific content for each log item are generated for a group of log items, such as specific content 604 for ri, specific content 606 for rj, and specific content 608 for rv.

[0053] By using this method, the same content can be deduplicated, thereby reducing the amount of log data that needs to be stored and the required storage resources, thereby improving resource utilization.

[0054] After SCM is performed, an optional function "Hide" can be implemented to hide duplicate content. After selecting this function, the same content is displayed only once to simplify the log display. After selecting Hide Display, the size of the displayed log file is reduced because duplicate content is not retrieved. It avoids displaying a large amount of duplicate information. Analysts can easily open the neat log file and search for key information.

[0055] Figure 7 A schematic diagram illustrating an example of a log entry according to an embodiment of the present disclosure is illustrated. Figure 7 A log item includes a log header and log content. The log header is added by the logging mechanism, and the log content 702 is generated by the logging component, which includes some metadata recorded by the logging component, such as timestamp, thread name and code pointer, component identifier, and message text, etc., where the timestamp can be a unique identifier of the log item. For example, the timestamp indicates the time when the logging component records the log, the thread name and code pointer are used to identify the thread and code targeted by the log, the component identifier is used to identify the logging component, and the message text is the specific content of the recorded operation.

[0056] Figure 8 A schematic diagram illustrating an example of identical contents of multiple log items according to an embodiment of the present disclosure. Figure 8 It shows the same content 'R10[log_id:-1][6446(6717nb_truck_20x7fe166be6f80)][mbe_r][INFO]mbe_r_metrics.c:mbe_r_reconfigure_metrics:1017:R_METRICS:Reconfigure metrics:Object with type' determined in the multiple logs 802, 804, 806, 808, 810 and 812, as shown in FIG. Figure 8 Therefore, by deduplicating the identical contents of these log items, the amount of data to be stored can be reduced.

[0057] Fig. 9 A schematic block diagram of an example device 900 that may be used to implement embodiments of the present disclosure is shown. Figure 1 The computing device 104 in the embodiment of the present invention can be implemented by using the device 900. As shown in the figure, the device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or computer program instructions loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0058] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage page 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0059] The various processes and processing described above, such as method 200 and processes 400, 500, may be performed by processing unit 901. For example, in some embodiments, method 200 and processes 400, 500 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by CPU 901, one or more actions of the process or method 200 and processes 400, 500 described above may be performed.

[0060] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0061] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the above. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0062] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0063] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0064] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0065] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0066] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0067] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0068] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing a log, comprising: Determine a target logging component from a set of candidate logging components based on the duration of a predetermined number of log entries recently recorded by the logging component; Determining a log group including a plurality of similar log items based on a predetermined number of log items recently recorded by the target log recording component; Determine the same content among the plurality of log items in the log group and specific content for each of the plurality of log items; as well as The plurality of log entries are updated by deduplicating the identical content and retaining specific content for each of the plurality of log entries.

2. The method according to claim 1, wherein determining a log group including a plurality of similar log entries comprises: Obtaining the predetermined number of log items most recently recorded by the target log recording component; as well as The log group is determined from the predetermined number of log items based on similarities between the log items.

3. The method according to claim 2, wherein the predetermined number of log items are determined as a log set, wherein based on similarities between log items, determining the log group from the predetermined number of log items comprises: Determining the number of log entries in the log set; If it is determined that the number is greater than a threshold number, selecting a log entry from the log set as a reference log entry; Initializing the log group using the reference log item; updating the log set by removing the reference log entry from the log set; Determining a similarity between each log entry in the updated log set and the reference log entry; as well as If it is determined that the similarity exceeds a similarity threshold, the updated log item in the log set is added to the log group.

4. The method according to claim 3, wherein determining the similarity between each log entry in the updated log set and the reference log entry comprises: generating an adjusted log entry and an adjusted reference log entry by removing timestamps of the log entry and the reference log entry; as well as The similarity between the adjusted log entry and the adjusted reference log entry is determined as the similarity between the log entry and the reference log entry.

5. The method according to claim 1, wherein determining the same content among the plurality of log items in the log group and the specific content for each of the plurality of log items comprises: Determining the identical content by comparing the plurality of log items; as well as The content other than the same content in each log item is determined as specific content for the log item.

6. The method according to claim 1, wherein determining a target logging component from a set of candidate logging components comprises: Selecting a candidate logging component from the set of candidate logging components; Obtaining a predetermined number of log items recently recorded by the candidate log recording component; determining a length of a time period corresponding to the predetermined number of log entries; as well as If it is determined that the duration is less than or equal to the threshold duration, the candidate logging component is determined as the target logging component.

7. The method of claim 6, wherein determining the length of the period corresponding to the predetermined number of log entries comprises: The duration is determined based on a timestamp of a first log entry among the predetermined number of log entries and a timestamp of a last log entry among the predetermined number of log entries.

8. The method according to claim 1, wherein determining a target logging component from a set of candidate logging components comprises: The target logging component is determined from the group of candidate logging components at predetermined intervals.

9. The method according to claim 1, further comprising: If it is determined to display the plurality of log items, the plurality of log items are generated by combining the same content with specific content for each of the plurality of log items, respectively; as well as The plurality of log entries are displayed.

10. The method according to claim 1, further comprising: determining a display format of the plurality of log entries; as well as If it is determined that the display format is a hidden format, only the same content and specific content for each of the multiple log items are displayed.

11. The method according to claim 10, further comprising: If it is determined that the display format is not a hidden format, generating the plurality of log items by combining the same content with specific content for each of the plurality of log items, respectively; as well as The plurality of log entries are displayed.

12. An electronic device comprising: at least one processor; as well as a memory coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, causing the device to perform actions, the actions comprising: Determine a target logging component from a set of candidate logging components based on the duration of a predetermined number of log entries recently recorded by the logging component; Determining a log group including a plurality of similar log items based on a predetermined number of log items recently recorded by the target log recording component; Determining the same content among the plurality of log items in the log group and specific content for each of the plurality of log items; and The plurality of log entries are updated by deduplicating the identical content and retaining specific content for each of the plurality of log entries.

13. The electronic device according to claim 12, wherein determining a log group including a plurality of similar log items comprises: Obtaining the predetermined number of log items most recently recorded by the target log recording component; as well as The log group is determined from the predetermined number of log items based on similarities between the log items.

14. The electronic device according to claim 13, wherein the predetermined number of log items are determined as a log set, wherein based on similarities between log items, determining the log group from the predetermined number of log items comprises: Determining the number of log entries in the log set; If it is determined that the number is greater than a threshold number, selecting a log entry from the log set as a reference log entry; Initializing the log group using the reference log item; updating the log set by removing the reference log entry from the log set; Determining a similarity between each log entry in the updated log set and the reference log entry; as well as If it is determined that the similarity exceeds a similarity threshold, the updated log item in the log set is added to the log group.

15. The electronic device of claim 14, wherein determining the similarity between each log item in the updated log set and the reference log item comprises: generating an adjusted log entry and an adjusted reference log entry by removing timestamps of the log entry and the reference log entry; as well as The similarity between the adjusted log entry and the adjusted reference log entry is determined as the similarity between the log entry and the reference log entry.

16. The electronic device according to claim 12, wherein determining the same content among the plurality of log items in the log group and the specific content for each of the plurality of log items comprises: Determining the identical content by comparing the plurality of log items; as well as The content other than the same content in each log item is determined as specific content for the log item.

17. The electronic device according to claim 12, wherein determining a target logging component from a set of candidate logging components comprises: Selecting a candidate logging component from the set of candidate logging components; Obtaining a predetermined number of log items recently recorded by the candidate log recording component; determining a length of a time period corresponding to the predetermined number of log entries; as well as If it is determined that the duration is less than or equal to the threshold duration, the candidate logging component is determined as the target logging component.

18. The electronic device of claim 17, wherein determining the length of the period corresponding to the predetermined number of log entries comprises: The duration is determined based on a timestamp of a first log entry among the predetermined number of log entries and a timestamp of a last log entry among the predetermined number of log entries.

19. The electronic device according to claim 12, wherein determining a target logging component from a set of candidate logging components comprises: The target logging component is determined from the group of candidate logging components at predetermined intervals.

20. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions which, when executed, cause a machine to: Determine a target logging component from a set of candidate logging components based on the duration of a predetermined number of log entries recently recorded by the logging component; Determining a log group including a plurality of similar log items based on a predetermined number of log items recently recorded by the target log recording component; Determine the same content among the plurality of log items in the log group and specific content for each of the plurality of log items; as well as The plurality of log entries are updated by deduplicating the identical content and retaining specific content for each of the plurality of log entries.

Images