System and method for asynchronous log processing and enrichment

Abstract

Comprising a plurality of log records is asynchronously processed to verify the configuration of each log record and the data included in each log record. It is determined that one or more attributes of a particular subset of log records of the plurality of log records correspond to one or more errors. Using the particular subset, one or more enriched log records are generated by augmenting each log record in the particular subset of log records with error information indicating one or more categories corresponding to the one or more errors. A user interface is generated to facilitate correction of the one or more errors, the user interface including a plurality of interactive elements corresponding to a plurality of error metrics for different categories of errors, where the one or more categories of the one or more errors are included in different categories of errors.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 17 / 829,269, filed on May 31, 2022, which is a continuation of U.S. Application No. 17 / 829,213, filed on May 31, 2022, each of which is hereby incorporated by reference in its entirety for all purposes. Technical Field

[0003] The present disclosure relates to systems and methods for enhancing log data processing and enrichment. More specifically, the present disclosure relates to systems and methods for processing (e.g., asynchronously processing) log data to determine errors in log records and enriching the log records with annotations indicating the errors that are viewable on a user interface. Background Art

[0004] Data storage and processing often involve large amounts of log data. Systems are used for log data ingestion, processing, storage, and querying. The computing resources associated with the system are limited. Computing resources must be used efficiently to maintain high log data processing throughput. However, methods for processing log data are often inefficient and inflexible. For example, synchronous processing of large amounts of log data (e.g., processing more than a TB of data in less than a day) is impractical and inefficient. When the collection of log data is large enough, computing resources encounter bottlenecks in synchronous processing.

[0005] In addition, processing log data involves processing one or more log records of log data that include errors. Errors can be caused by the system, such as during processing, or by a data tenant or user (e.g., user errors). User errors can involve incorrect configuration, incorrect data in a log record, or other similar errors. The system may not correctly identify the error, and the system may not report the identity or type of the error to the user or data tenant to correct the error. In addition, a large amount of data may be lost due to errors in the log data that are not identified or otherwise resolved. Summary of the invention

[0006] In some embodiments, a computer-implemented method is provided in which log data including a plurality of log records is received by a computing system. The log data is processed asynchronously to verify the configuration of each of the plurality of log records and the data included in each of the plurality of log records. During the asynchronous processing, it is determined that an attribute of a particular log record in the plurality of log records corresponds to an error. In response to determining that a particular log record in the plurality of log records corresponds to an error: the computing device determines a category of the error and generates an enriched log record by augmenting the particular log record with error information indicating the category of the error. The enriched log record is transmitted via a user interface to facilitate correction of the error included in the particular log record.

[0007] Receiving log data may include: receiving, by a computing device, a first subset of log data via real-time input from a data tenant; and receiving, by the computing device, a second subset of log data via input from a second log data repository, wherein the second subset of log data is received asynchronously from the second log data repository, and wherein the received log data is a union of the first subset of log data and the second subset of log data.

[0008] The category of errors may include at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure, and wherein providing the enriched log record includes generating, by the computing device, a user interface displaying the category of errors.

[0009] The method may also include: in response to determining that a particular log record includes an error: generating, by the computing device, an error metric for transmission to a monitoring service of the computing device, the error metric being used to monitor a set of errors associated with the log data; and generating, by the computing device, a backup of data included in the particular log record among multiple log records, the backup of the data being stored in a third log data repository for allowing the computing device to access the backup of the data for further processing of the particular log record.

[0010] The method may also include determining that the category of the error is payload, wherein generating the enriched log record includes transmitting, by the computing device, the enriched log record to a log data repository to display the enriched log record via a user interface.

[0011] In some cases, the method may include: in response to determining that the category of the error is configured: determining, by the computing device, that an error included in a particular log record prevents the particular log record from being processed; generating, by the computing device, a placeholder log record for storage in a log data repository; generating, by the computing device, an interactive element configured to retrieve the particular log record from a plurality of log records and provide the particular log record for display; augmenting, by the computing device, the placeholder log record with the interactive element; and transmitting, by the computing device, the placeholder log record to the log data repository to provide access to the placeholder log record via a user interface.

[0012] The multiple log records may include a subset of the log records, the subset including log records with errors, wherein the user interface includes a list of selectable elements corresponding to the subset of the log records, and wherein the computer-implemented method further includes: receiving, by the computing device via the user interface, input indicating selection of a first selectable element in the list of selectable elements, wherein the first selectable element corresponds to an enriched log record; and generating, by the computing device, a second user interface for display, the second user interface being used to display details about the enriched log record, wherein the details include error information.

[0013] In some embodiments, a computer-implemented method is provided in which log data including a plurality of log records is received by a computing system, the log data being processed asynchronously to verify the configuration of each of the plurality of log records and the data included in each of the plurality of log records. One or more attributes of log records of a particular subset of the plurality of log records are determined by the computing system during the asynchronous processing to correspond to one or more errors. Using the particular subset of log records, one or more enriched log records are generated by augmenting each log record of the particular subset of log records with error information indicating one or more categories corresponding to the one or more errors. A user interface is generated to facilitate correction of the one or more errors, the user interface including a plurality of interactive elements corresponding to a plurality of error metrics for different categories of errors, wherein the one or more categories of the one or more errors are included in the different categories of errors.

[0014] The method may also include recognizing input received via the first interface, the input indicating a selection of a specific interactive element in the first plurality of interactive elements that corresponds to a specific error metric in the plurality of error metrics; and generating a second user interface, the second user interface including a second plurality of interactive elements representing a subset of one or more enriched log records, wherein the subset of one or more enriched log records includes one or more enriched log records containing one or more errors corresponding to the specific error metric. The method may also include recognizing a second input received via the second interface, the second input indicating a selection of a different specific interactive element in the second plurality of interactive elements that corresponds to a specific enriched log record in the subset of one or more enriched log records; and generating a third user interface, the third user interface including a specific enriched log record, wherein the specific enriched log record includes at least one annotation identifying an error in the one or more errors that corresponds to the specific error metric.

[0015] The method may also include: generating one or more suggestions for correcting the error for display on a third user interface; identifying a third input received via the third interface, the third input indicating that an error in a specific enriched log record has been corrected; and causing a log record in a plurality of log records corresponding to the specific enriched log record to be reprocessed.

[0016] In some cases, the error is a configuration error that prevents processing of a particular enriched log record, and wherein at least one annotation includes a selectable element that links a third user interface to a log data repository that includes a backup version of a log record in a plurality of log records that corresponds to the particular enriched log record.

[0017] In some cases, the method includes: recognizing a third input received via a third interface, the third input indicating a selection of a selectable element; and providing access to a backup version of a log record corresponding to the particular enriched log record to facilitate correction of the log record corresponding to the particular enriched log record.

[0018] The different categories of errors may include at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure.

[0019] In some embodiments, a computer-implemented method is provided, wherein log data including a plurality of log records is received by a computing system. The log data is processed asynchronously to verify the configuration of each of the plurality of log records and the data included in each of the plurality of log records. One or more attributes of log records of a particular subset of the plurality of log records are determined by the computing system during the asynchronous processing to correspond to one or more errors. Using the particular subset of log records, one or more enriched log records are generated by augmenting each log record in the particular subset of log records with error information indicating one or more categories corresponding to the one or more errors. A user interface is generated to facilitate correction of the one or more errors, the user interface comprising a plurality of interactive elements corresponding to a plurality of error metrics of different categories of errors, wherein the one or more categories of the one or more errors are included in the different categories of errors.

[0020] The method may also include: identifying input received via the first interface, the input indicating a selection of a specific interactive element from the first plurality of interactive elements corresponding to a specific error metric from the plurality of error metrics; and generating a second user interface, the second user interface comprising a second plurality of interactive elements representing a subset of one or more enriched log records, wherein the subset of one or more enriched log records comprises one or more enriched log records containing one or more errors corresponding to the specific error metric.

[0021] The method may also include identifying a second input received via a second interface, the second input indicating a different specific interactive element from a second plurality of interactive elements corresponding to a specific enriched log record of a subset of one or more enriched log records; and generating a third user interface, the third user interface including a specific enriched log record, wherein the specific enriched log record includes at least one annotation identifying an error from one or more errors corresponding to a specific error metric.

[0022] The method may also include: generating one or more suggestions for correcting the error for display on a third user interface; identifying a third input received via the third interface, the third input indicating that an error in a specific enriched log record has been corrected; and causing a log record in a plurality of log records corresponding to the specific enriched log record to be reprocessed.

[0023] In some cases, the error is a configuration error that prevents a particular enriched log record from being processed, and wherein at least one annotation includes a selectable element that links a third user interface to a log data repository that includes a backup version of a log record in a plurality of log records that corresponds to the particular enriched log record.

[0024] In some cases, the method includes: recognizing a third input received via a third interface, the third input indicating a selection of a selectable element; and providing access to a backup version of a log record corresponding to the particular enriched log record to facilitate correction of the log record corresponding to the particular enriched log record.

[0025] The different categories of errors may include at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure.

[0026] In some embodiments, a computer-implemented method is provided in which log data including a plurality of log records is received by a computing system. The log data is processed asynchronously to verify the configuration of each of the plurality of log records and the data included in each of the plurality of log records. During the asynchronous processing, it is determined that an attribute of a particular log record in the plurality of log records corresponds to an error. In response to determining that a particular log record in the plurality of log records corresponds to an error: the computing device determines a category of the error and generates an enriched log record by augmenting the particular log record with error information indicating the category of the error. The enriched log record is transmitted via a user interface to facilitate correction of the error included in the particular log record.

[0027] Receiving log data may include: receiving, by a computing device, a first subset of log data via real-time input from a data tenant; and receiving, by the computing device, a second subset of log data via input from a second log data repository, wherein the second subset of log data is received asynchronously from the second log data repository, and wherein the received log data is a union of the first subset of log data and the second subset of log data.

[0028] The category of errors may include at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure, and wherein providing the enriched log record includes generating, by the computing device, a user interface displaying the category of errors.

[0029] The method may also include: in response to determining that a particular log record includes an error: generating, by the computing device, an error metric for transmission to a monitoring service of the computing device, the error metric being used to monitor a set of errors associated with the log data; and generating, by the computing device, a backup of data included in a particular log record among a plurality of log records, the backup of the data being stored in a third log data repository for allowing the computing device to access the backup of the data for further processing of the particular log record.

[0030] The method may also include determining that the category of the error is payload, wherein generating the enriched log record includes transmitting, by the computing device, the enriched log record to a log data repository to display the enriched log record via a user interface.

[0031] In some cases, the method may include: in response to determining that the category of the error is configured: determining, by the computing device, that an error included in a particular log record prevents the particular log record from being processed; generating, by the computing device, a placeholder log record for storage in a log data repository; generating, by the computing device, an interactive element configured to retrieve a particular log record from a plurality of log records and provide the particular log record for display; augmenting, by the computing device, the placeholder log record with the interactive element; and transmitting, by the computing device, the placeholder log record to the log data repository to provide access to the placeholder log record via a user interface.

[0032] The multiple log records may include a subset of the log records, the subset including log records with errors, wherein the user interface includes a list of selectable elements corresponding to the subset of log records, and wherein the computer-implemented method further includes: receiving, by the computing device via the user interface, input indicating selection of a first selectable element in the list of selectable elements, wherein the first selectable element corresponds to an enriched log record; and generating, by the computing device, a second user interface for display, the second user interface being used to display details about the enriched log record, wherein the details include error information.

[0033] In some embodiments, a system is provided that includes one or more data processors and a computer-readable storage medium (e.g., a non-transitory computer-readable medium) that contains instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of one or more methods disclosed herein.

[0034] In some embodiments, a computer program product is provided, which is tangibly embodied in a machine-readable storage medium (eg, a non-transitory machine-readable storage medium) and includes instructions configured to cause one or more data processors to perform part or all of one or more methods disclosed herein.

[0035] The terms and expressions employed are used as terms of description and not of limitation, and the use of such terms and expressions is not intended to exclude any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications may be made within the scope of the claimed invention. Therefore, it should be understood that although the claimed invention has been specifically disclosed by embodiments and optional features, modifications and variations of the concepts disclosed herein may be made by those skilled in the art, and such modifications and variations are considered to be within the scope of the invention as defined by the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] This description refers to the following drawings, in which like reference numerals are used in different figures to indicate like or analogous components.

[0037] Figure 1 is a block diagram illustrating an example of a data processing environment according to an embodiment.

[0038] Figure 2 is a flow diagram of a process for asynchronously processing and enriching log data, according to an embodiment.

[0039] Figure 3 is a flow chart of a process for enhancing a set of generated user interfaces for log data according to an embodiment.

[0040] Figure 4 is an example of a user interface for displaying error metrics according to an embodiment.

[0041] Figure 5 is an example of a user interface for displaying enriched log records according to an embodiment.

[0042] Figure 6 are examples of two user interfaces for displaying annotations of a selected enriched journal record, according to an embodiment.

[0043] Figure 7 is an example of a user interface for displaying different log records according to an embodiment, wherein similar errors are highlighted in corresponding log records.

[0044] Figure 8 is another example of a user interface for displaying different log records according to an embodiment, wherein similar errors are highlighted in corresponding log records.

[0045] Fig. 9 is a simplified diagram illustrating a distributed system for implementing one of the embodiments.

[0046] Fig.10 is a simplified block diagram illustrating one or more components of a system environment according to an embodiment.

[0047] Fig.11 An exemplary computer system is illustrated in which various embodiments of the invention may be implemented. DETAILED DESCRIPTION

[0048] In the following description, specific details are set forth for purposes of explanation in order to provide a comprehensive understanding of certain embodiments. However, it is apparent that various embodiments can be practiced without these specific details. The drawings and descriptions are not intended to be limiting. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as superior or advantageous over other embodiments or designs.

[0049] Overview

[0050] Certain aspects and features of the present disclosure relate to enhancing log data processing, storage, querying, etc. by processing (e.g., asynchronously processing) log data to enrich the log data. Asynchronous processing of log data may involve processing received log data as it is received, rather than, for example, synchronously (in batches) processing received log data. In addition, one or more log records including errors may be detected. Errors may include configuration errors, data errors, or other suitable errors related to log data. Configuration errors may indicate that a particular log record is not configured as expected by the processing system. Data errors may indicate that the data included in a particular log record is incorrect or otherwise unprocessable. Enriching log data may involve generating an enriched log record by augmenting a particular log record with one or more annotations indicating the type of error determined about a particular log record. The enriched log record may be stored in a repository accessible by a computing system, which may receive queries for the enriched log record and may generate one or more user interfaces in response to receiving one or more of the queries for providing the enriched log record to correct errors in the log data.

[0051] As used herein, a "log record" refers to a small piece of data and a corresponding key-value pair. A log record can be structured or unstructured data.

[0052] As used herein, an "enriched log record" refers to a particular log record that includes at least one annotation indicating an error for the particular log record.

[0053] Data tenants or other suitable users often send their data to data services for storage and processing. Data tenants are sending large and exponentially growing amounts of log data for processing and storage. Due to the huge amount of log data, synchronous processing of the data is impractical and may lead to bottlenecks in processing the log data. In addition, errors may exist in the log data. For example, due to incorrect data included in one or more log records, one or more log records may not be processed or may otherwise encounter errors during processing. When processing large amounts of data, identifying and presenting errors may be difficult. In addition, the inability to identify and / or present errors may result in the loss of data in the log records that include errors.

[0054] In order to at least solve the above-mentioned problems related to log data processing, log data can be processed and / or enriched asynchronously. Asynchronous processing of log data, such as processing log data after receiving log data instead of batch processing log data (or as a supplement thereto) can increase the maximum amount of log data that can be processed within a period of time. For example, synchronous processing of data can introduce increased latency by maintaining user processing threads for a long time, which can prevent or otherwise hinder the generation of synchronous processing results with a large amount of log data. For example, compared with asynchronous processing of log data, the efficiency of processing log data is low, and the amount of time required to process log data is greater. Thus, asynchronous processing of log data can reduce the amount of time to process log data and improve efficiency. In addition, compared with synchronous processing of log data, various computing resources can be reduced, such as the amount of processor usage for processing similar amounts of log data. In some cases, asynchronous processing can include (for example) processing a log message when it is received. In some cases, asynchronous processing can include periodic processing (for example, no matter how many messages have been received since the last processing). Asynchronous processing can provide benefits, such as supporting iterative updates, thereby providing a smoother update transition. Additionally, this approach can support faster attainment of accurate results, as processing can be iteratively built over time in a manner that values ​​time series data and / or learning.

[0055] In addition, one or more log records including errors can be enriched. Enriching one or more log records can facilitate correction of one or more log records. For example, enriching one or more log records can involve annotating the log record with an identification of the error of the log record. The enriched log record with the error annotation can be presented to the data tenant or other suitable user to correct the log record associated with the enriched log record. Additionally or alternatively, a representation of the enriched log message and the error annotation can be presented (e.g., it includes the error annotation and includes a count or identifier of the log message associated with the annotated error). Thus, the log record can be corrected and data loss can be avoided.

[0056] The user interface can display various error representations and / or metrics, which can improve the efficiency obtained via user input and intelligent data presentation. One or more enriched log records can be used to generate the user interface. One or more enriched records can include one or more different categories of errors, and error metrics can track data related to enriched log records, different categories of errors, other suitable data related to log records, or any suitable combination thereof. For example, an example of error metrics can include the number of log records with configuration errors. Other examples of error metrics (e.g., data errors in log records, etc.) can also be used. The user interface can provide error metrics, enriched log records, and any suitable data included therein. In some examples, more than one user interface can be used or generated to provide error metrics, enriched log records, data or annotations included therein, or any suitable combination thereof.

[0057] Example environment for dynamic log indexing and processing

[0058] Figure 1 1 is a block diagram illustrating an example of a data processing environment 100 according to some embodiments. The processing environment 100 can be used to process log data (e.g., asynchronously) and enrich log records. In some embodiments, input log data 102 can be input into the processing environment 100 by a user 101, who can be a data tenant or other suitable user of the processing environment 100. The processing environment 100 can perform one or more authorization verifications 104 on the input log data 102. For example, the user 101 who inputs the input log data 102 can be verified by requesting credentials or otherwise authenticating the user 101 to transmit the input log data 102 for asynchronous processing and enrichment attempts. In some examples, the authorization verification 104 can be processed synchronously or otherwise batched. In response to a failure in the authorization verification, an error message can be returned to the user 101, and in this example, the input log data 102 may not be processed or enriched. In response to a successful authorization verification, the input log data 102 can be transmitted to a data ingestion service 106.

[0059] The data ingestion service 106 may ingest the input log data 102, may pre-process the input log data 102, or otherwise begin processing the input log data 102, or a combination thereof. For example, the input log data 102 may be pre-processed via one or more synchronization basic validations 108. The synchronization basic validations 108 may include basic validations related to configuration and data. For example, if a particular log record is empty, includes a configuration that the processing environment 100 cannot receive, or other suitable validation failures, then a particular log record of the input log data 102 may cause the synchronization basic validation 108 to fail. In some embodiments, the synchronization basic validation 108 may be a quick and shallow validation compared to the processing validations (e.g., configuration validation 112 and data validation 114) of the processing environment 100. In response to the failure to validate the log data during the synchronization basic validation 108, the processing environment 100 or other suitable computing device may transmit an error message to the user 101, and at least a subset of the input log data 102 may not be further transmitted to the processing environment 100.

[0060] In response to successfully validating the input log data 102 during the synchronous basic validation 108, the data ingestion service 106 or other suitable component of the processing environment 100 transmits the input log data 102 to the asynchronous process 110. The asynchronous process 110 can initiate asynchronous processing of the input log data 102. For example, the asynchronous process 110 can execute or otherwise transmit the input log data 102 for performing configuration validation 112, data validation 114, other suitable validations, or any suitable combination thereof. The asynchronous process 110 can asynchronously process or transmit the input log data 102 for processing, and, in contrast to batch processing or synchronous processing of the input log data 102, asynchronous processing of the input log data 102 involves processing the input log data 102 or transmitting the input log data 102 for processing after receiving the input log data 102. In some embodiments, asynchronous processing of the input log data 102 involves processing the input log data 102 via parallel computation within the input log data 102 or with other suitable input data. In some embodiments, asynchronously processing the input log data 102 increases the computational power of the processing environment 100 and may increase the rate at which the input log data 102 is processed and / or enriched.

[0061] In some embodiments, the input log data 102 and the additional data from the log data repository 116 can be transmitted to the asynchronous processing 110. The additional data can include a previous log record. For example, the previous log record may have included one or more errors that prevented the previous log record from being processed, and the one or more errors may have been corrected before the previous log record was transmitted to the asynchronous processing 110. The additional data can include other suitable log data that can be stored in the log data repository 116. The asynchronous processing 110 receives and asynchronously processes or transmits the input log data 102 and the additional data for processing. In some embodiments, the asynchronous processing 110 processes the input log data 102 and the additional data in parallel.

[0062] Configuration verification 112 may include verification or other suitable verification related to the configuration of each log record and / or additional data included in the input log data 102. For example, a particular log record of the input log data 102 may be expected to be configured as a JSON file, but may not be a JSON file or may be a damaged or incorrectly configured JSON file. Thus, the processing environment 100 may be unable to process a particular log record. In response to one or more failures in the configuration verification 112, the particular log record or any of its data or metadata may be transmitted to the monitoring service 118 for recording the failed configuration verification of the particular log record. In addition, the data included in the particular log record may be copied or otherwise backed up and saved to the log data repository 120.

[0063] Data validation 114 may include validation or other appropriate verification related to data included in each log record and / or additional data in input log data 102. For example, a particular log record may include incorrect data, such as an incorrect pointer, an incorrect data value, an incorrect function, other suitable incorrect data, or any suitable combination thereof. Thus, processing environment 100 may not be able to correctly or successfully process a particular log record. In response to one or more failures in data validation 114, the particular log record or any data or metadata thereof may be transmitted to monitoring service 118 for recording the failed configuration validation of the particular log record.

[0064] The monitoring service 118 tracks or otherwise monitors errors detected or otherwise determined during the asynchronous processing 110, the configuration validation 112, the data validation 114, etc. The monitoring service 118 can receive an instance of each error detected during the processing of the input log data 102 and the additional data, and the monitoring service 118 can generate or otherwise receive error metrics related to the processing of the input log data 102 and / or the additional data. The error metrics include the category of errors detected, the number of errors, unrecognized log records, and unprocessed log records, etc. The monitoring service 118 can provide the error metrics and any other suitable data for correcting one or more log records.

[0065] After performing configuration validation 112, data validation 114, and any other suitable asynchronous validation or processing, the processing environment 100 can transmit the input log data 102 and / or additional data to the core processing 122. The core processing 122 can process the input log data 102 and / or additional data. Processing the input log data 102 and / or additional data involves performing tasks requested by the user 101 or other suitable entity to be performed on the input log data 102 and / or additional data. In addition, processing the input log data 102 and / or additional data can involve processing the input log data 102 and / or additional data for storage and querying. The core processing 122 can involve storing processed log records from the input log data 102 and / or additional data in a log analysis repository 124, and the processed log records stored in the log analysis repository 124 can be accessed via one or more user interfaces 126, for example, through one or more queries.

[0066] [The core processing 122 involves processing a log record that includes one or more errors. In some embodiments, the core processing 122 receives an enriched log record, while in other embodiments, the core processing 122 generates an enriched log record based on one or more errors. The enriched log record includes an annotation that identifies an error in a log record corresponding to the enriched log record. For example, the core processing 122 may receive a particular log record and may identify an error associated with the particular log record. In some embodiments, the core processing 122 may receive an indication of an error indicating a particular log record from the configuration validation 112, from the data validation 114, from other suitable components, or from any suitable combination thereof. The core processing 122 may use the indication of the error to generate an annotation and augment the particular log record with the annotation to generate an enriched log record. The annotation includes information that identifies the error included in the particular log record. For example, the annotation may include the category of the error, the location of the error in the particular log record, one or more suggestions on how to fix the error, whether the error prevented or prevented the particular log record from being processed, other suitable annotation information, or any suitable combination thereof. In some embodiments, one or more other components (such as asynchronous processing 110, etc.) can generate enriched log records and transmit the enriched log records to core processing 122 for processing, storage, and providing (e.g., via queries and (one or more) user interfaces 126) the enriched log records.

[0067] In some embodiments, a particular enriched log record corresponds to a log record that includes a configuration error or other similar error that prevents the log record from being processed. The log record can be copied or otherwise backed up to the log data repository 120 to prevent data loss during the processing of the log record. In addition, the core processing 122 or any other suitable component of the processing environment 100 can generate a placeholder log record as the particular enriched log record. For example, the placeholder log record includes a link or other suitable interactive element for accessing the backup log record stored in the log data repository.

[0068] The processing environment 100 can generate one or more user interfaces 126 for accessing the log analysis repository 124. In some embodiments, the log analysis repository 124 is or otherwise includes a database that stores processed log records and enriched log records. The log analysis repository 124 is configured to provide access to the processed log records (which may not include detected errors) and the enriched log records via one or more user interfaces 126.

[0069] The one or more user interfaces 126 may include one user interface, two user interfaces, three user interfaces, or more user interfaces for providing access to enriched log records, for correcting or facilitating correction of enriched log records, and for other suitable purposes. For example, the processing environment 100 may generate a first user interface in the one or more user interfaces 126, the first user interface including a set of interactive elements corresponding to the error metric. The error metric may be tracked or otherwise accumulated via the monitoring service 118 or other suitable components of the processing environment 100. The set of interactive elements may display the error metric via the first user interface, and the set of interactive elements may correspond to the error metric. For example, a first interactive element may correspond to a first error metric (such as an unprocessed log record), and a second interactive element may correspond to a second error metric (such as a log record with an error), etc.

[0070] In response to receiving input indicating a selection of at least one interactive element included in the first user interface (e.g., from user 101), processing environment 100 or other suitable component thereof generates a second user interface in one or more user interfaces 126. The second user interface includes a set of enriched log records corresponding to error metrics, where the error metrics correspond to the selected interactive elements of the first user interface. For example, the selected interactive elements may include interactive elements corresponding to unprocessed log records. Thus, the second user interface may include a second set of interactive elements corresponding to a list or other suitable set of enriched log records, where the enriched log records correspond to unprocessed log record error metrics.

[0071] In response to receiving input indicating a selection of at least one interactive element included in the second user interface (e.g., from user 101), processing environment 100 or other suitable components thereof generates a third user interface in one or more user interfaces 126. The third user interface includes details of a specific enriched log record corresponding to the selected interactive element of the second user interface. For example, the third user interface can display details about a specific enriched log record. The details can include metadata for a specific enriched log record, values ​​or other data included in a specific enriched log record, annotations of errors associated with a specific enriched log record, other suitable details, or any suitable combination thereof. Processing environment 100 or any suitable component thereof can generate other suitable user interfaces 126 for facilitating correction of log records corresponding to the enriched log records.

[0072] For asynchronous log data processing and rich sample processing

[0073] Figure 22 is a flow chart illustrating an example of a process 200 for asynchronously processing and enriching log data. Process 200 may be performed, at least in part, by any component described in the figures herein, for example, by any component of processing environment 100 or by processing environment 100 itself. Process 200 may begin at block 210 when log data including a collection of log records is received. The log data may be received by a user, such as a data tenant, or from a first log data repository including previous log records that may have been previously unprocessed due to configuration or other suitable errors. The received log data may be subjected to synchronous preprocessing that may prevent processing of some faulty or unauthorized log records. In some embodiments, the received log data includes one or more errors, such as configuration errors and payload errors.

[0074] At block 220, the received log data is processed asynchronously. Asynchronously processing the received log data involves processing the received log data after the log data is received, and optionally in parallel with the additional data. Asynchronously processing the log data can prevent bottlenecks or other processing difficulties associated with synchronization or batch processing of the log data. In some embodiments, asynchronously processing the received log data involves performing one or more validations on the log records included in the received log data. For example, the one or more validations include configuration validation, data or payload validation, other suitable asynchronous validation operations, or any suitable combination thereof.

[0075] Configuration validation involves validating the configuration of a particular log record or otherwise determining whether the configuration of a particular log record includes an error. For example, a log record may be configured as an XML file, but the processing environment 100 or other computing entity that performs asynchronous processing of received log data may expect the log record to be configured as a JSON file. Thus, the processing environment 100 or other computing entity that performs asynchronous processing of received log data may not be able to process the log record. Other suitable configuration validations may be performed via asynchronous processing.

[0076] Data or payload verification involves verifying the payload or other suitable data included in the log record. For example, the log record may include various types of payloads or other suitable data. The processing environment 100 may expect the log record to include a first type of payload that is different from the second type of payload included in the log record. In addition, the payload may include typographical errors or other errors, such as incorrect pointers, function calls, etc. Thus, although the processing environment 100 or other computing entity that performs asynchronous processing of received log data may be able to process the log record, payload or data errors may prevent accurate or complete processing of the log record. Other suitable data or payload verifications may be performed via asynchronous processing.

[0077] At box 230, the processing environment 100 or other computing entity that performs asynchronous processing of the received log data determines whether one or more log records of the received log data include errors. Errors can be identified during the asynchronous processing of the received log data, such as during or after configuration verification, data or payload verification, or a combination thereof. At box 235, and in response to determining that no log record includes an error in the received log data, the asynchronously processed log data is stored in a log data repository. The log data repository can be a second log data repository that is the same or different from the first log data repository. In addition, the log data repository can store the processed log data and provide the processed log data for display, query, or other suitable tasks, such as via one or more user interfaces.

[0078] At block 240, and in response to determining that at least one log record of the received log data includes an error, a category of the error is determined. The category of the error may include configuration, data or payload, other suitable error categories, or any suitable combination thereof. In some embodiments, the category of the error may be any of: (i) invalid configuration, (ii) invalid payload or data, (iii) structure mismatch, (iv) not authorized or not found, and (v) parser failure. Other suitable error categories may be determined for one or more log records.

[0079] The invalid configuration error category involves an invalid or incorrect configuration or file type for a particular log record. The invalid payload or data error category involves an invalid format for a particular log record, the size of a particular log record exceeds a maximum size limit, an invalid archive format for a particular log record, etc. The structure mismatch error category involves a mismatch between the data included in a particular log record and the entry start expression of a parsed entity that parses the particular log record. Thus, for the structure mismatch error category, at least a subset of log records with a structure mismatch will not produce processed log entries because parsing cannot begin due to the structure mismatch. The unauthorized or not found error category involves an authorization error or a missing log record in the first log data repository. For example, a log record with an unauthorized or not found error may include an object call that may be inaccurate or may otherwise prevent the log record from being retrieved from the object storage device. The parser failure error category involves a mismatch when applying a basic parser on a log record to extract fields of the log record.

[0080] At block 250, an enriched log record is generated. The enriched log record is generated based on the determined category of the identified error and based on the log record including the identified error. In some embodiments, the processing environment 100 or any suitable component thereof generates a note that includes at least the category of the error. For example, the note can include the category of the error, the identification of the error, the processing failure experienced due to the error, the location of the error, one or more suggestions for correcting the error, other suitable information related to the error, or any suitable combination thereof. The note is expanded to include the specific log record of the error to generate an enriched log record. For example, the note can be expanded to any suitable part (such as an introduction or top part) of the specific log record to generate an enriched log record. Thus, when viewing the enriched log record, the user can quickly determine the category of the error, determine the identity of the error, determine how to correct the error, etc.

[0081] In embodiments where an error (such as a configuration error) with a particular log record prevents the particular log record from being processed, the annotation may additionally or alternatively include a link or other suitable interactive element. Also, the annotation may be expanded to a placeholder log record representing the particular log record to generate an enriched log record. The link or other suitable interactive element may connect the enriched log record with a backup or duplicate version of the particular log record to prevent data loss during asynchronous processing of log data.

[0082] At box 260, the enriched log records are stored in a log data repository. The log data repository may provide access to the enriched log records, for example, through one or more user interfaces and in response to one or more queries. In some embodiments, the log data repository may receive one or more queries for accessing the enriched log records. For example, a user may submit a query to the log data repository via one or more user interfaces or other suitable media, and the log data repository provides the enriched log records. The enriched log records are provided for display, analysis, correction of one or more errors of the associated log records, and for any other suitable purpose.

[0083] Exemplary Processing for Providing Enriched Log Data Via a User Interface

[0084] Figure 3300 is a flowchart illustrating an example of a process 300 for providing enriched log data via a user interface. The process 300 may be performed at least in part by any component described in the figure herein, for example, by any component of the processing environment 100 or by the processing environment 100 itself. When the log data is received, the process 300 may start at block 310. The received log data includes a collection of log records, and in some embodiments, the processing environment 100 or any other suitable computing entity processes the log records asynchronously. For example, the processing environment 100 or any suitable component thereof performs configuration validation, payload or data validation or other suitable validation on the log records of the received log data. One or more errors may be detected or otherwise determined with respect to the log record, and an enriched log record may be generated based on the log record and one or more detected errors. For example, a note including error information (such as the category of the error) may be extended to one or more log records with errors to generate an enriched log record. The enriched log record may be stored in a data repository for providing an enriched log record via one or more user interfaces to correct the associated log record.

[0085] At block 320, a first user interface having a first set of interactive elements is generated. The first set of interactive elements corresponds to one or more error metrics. The error metrics may include one error metric, two error metrics, three error metrics, statistics corresponding to one or more error metrics, other suitable information, or any suitable combination thereof. In some embodiments, the error metrics are tracked and accumulated by the monitoring service 118 of the processing environment 100. Examples of error metrics may include unprocessed log records, unidentified log records, processed log records with errors, and any other suitable type of error metrics. Therefore, the first set of interactive elements may include multiple interactive elements corresponding to multiple error metrics. For example, if there are three error metrics, then the first set of interactive elements may include at least three interactive elements, so that each of the at least three interactive elements represents a different error metric in the three error metrics. In some embodiments, more than three interactive elements may be included in the first set of interactive elements. For example, three interactive elements corresponding to the three error metrics may be generated and included in the first user interface, and one or more additional interactive elements corresponding to the statistics of the error metrics may be generated and included in the first user interface. The statistics may include timestamps of the errors included in the error metrics or any other suitable comparisons or statistics related to the errors included in the error metrics.

[0086] In some embodiments, the first set of interactive elements can be arranged in an array on the first user interface. For example, the first set of interactive elements can be arranged in a rectangular shape or in any other suitable arrangement for displaying error metrics on the first user interface. In some embodiments, each interactive element displays a corresponding error metric or statistical data. In one such example, a particular interactive element corresponding to an unprocessed log record can display the number of unprocessed log records in a given time.

[0087] At block 330, an input is received via the first user interface indicating a selection of a first particular interactive element in the first set of interactive elements. For example, a user (such as a data tenant or other suitable user of the processing environment 100) may input a selection of the first particular interactive element. The input is received via the first user interface and may be input using a computer mouse, a computer keyboard, or any other suitable input device. The first user interface may receive any other suitable input related to the error metric and the first set of interactive elements.

[0088] At block 340, a second user interface having a second set of interactive elements is generated. In response to receiving input indicating selection of the first specific interactive element, the processing environment 100 or other suitable computing entity generates a second user interface. The second user interface includes or otherwise displays a second set of interactive elements corresponding to the error metric represented by the first specific interactive element. For example, if the first specific interactive element represents a processed log record with an error, then the second set of interactive elements may include interactive elements corresponding to enriched log records with data or payload errors (which produce processed log records with errors). In other examples, if the first specific interactive element represents an unprocessed log record, then the second set of interactive elements may include interactive elements corresponding to enriched log records (which represent log records with configuration errors that prevent the log record from being processed). The second set of interactive elements may include any other suitable interactive elements corresponding to enriched log records of different types.

[0089] In some embodiments, the second set of interactive elements is arranged as a list on the second user interface. For example, the second set of interactive elements may include a list of enriched log records, and the list of enriched log records may be displayed on the second user interface. Each enriched log record in the list of enriched log records may be interactive, so that selecting a specific enriched log record in the list of enriched log records causes the specific enriched log record and any data or metadata thereof to be displayed on the second user interface. The second set of interactive elements may be otherwise suitably arranged on the second user interface to provide a list of enriched log records to a user or other suitable entity to select a specific enriched log record from.

[0090] At block 350, an input is received via a second user interface indicating a selection of a second particular interactive element in the second set of interactive elements. For example, a user (such as a data tenant or other suitable user of processing environment 100) may input a selection of the second particular interactive element. The input is received via the second user interface and may be input using a computer mouse, a computer keyboard, or any other suitable input device. The second user interface may receive any other suitable input related to the enriched log record represented by the second set of interactive elements.

[0091] At box 360, a third user interface with an enriched log record corresponding to the second specific interactive element is generated. In response to receiving an input indicating the selection of the second specific interactive element, the processing environment 100 or other suitable computing entity generates a third user interface. The third user interface includes or otherwise displays a specific enriched log record corresponding to the second specific interactive element or otherwise represented by the second specific interactive element. In some embodiments, the third user interface includes or otherwise displays data or metadata about a specific enriched log record that corresponds to a log record with an error. For example, the third user interface displays data or payload included in the log record, suitable metadata about the log record, annotations expanded into the log record, other suitable information related to the log record or a specific enriched log record, or any suitable combination thereof.

[0092] The annotation can be generated by the processing environment 100, any suitable component thereof, or any other suitable computing entity. The annotation includes information about the error associated with the log record. For example, the annotation can include the category of the error, the identification of the error, the string or other payload that may cause the error, the position of the error in the log record, one or more suggestions on how to correct the error, other suitable information or any suitable combination thereof. The annotation is expanded into the log record to generate a specific enriched log record, and the annotation can be displayed at the top or introduction section of the displayed specific enriched log record or relative to other suitable positions of the specific enriched log record. In some embodiments, the annotation is expanded into the log record and displayed on the third user interface so that the annotation is highlighted on the third user interface or is otherwise convenient for the user to view on the third user interface.

[0093] Exemplary User Interface

[0094] Figure 4 4 is an example of a user interface 400 for displaying error metrics according to an embodiment. The user interface 400 may be or otherwise include, for example, the first user interface described with respect to block 320 of the process 300. The user interface 400 includes a first set 402 of interactive elements. As shown, the first set 402 of interactive elements includes six interactive elements 402a-f, but any other suitable number of interactive elements (such as less than six or more than six) may be included in the user interface 400 or otherwise displayed on the user interface 400. The first set 402 of interactive elements is shown as being arranged in an array on the user interface 400, but other suitable arrangements of the first set 402 of interactive elements are also possible. In some embodiments, the first set 402 of interactive elements corresponds to error metrics, which may be tracked or otherwise accumulated by the processing environment 100 or any suitable component thereof (such as the monitoring service 118). The error metrics include unprocessed log records, unrecognized log records, processed log records with errors, and any other suitable error metrics.

[0095] Each interactive element 402 in the first set 402 of interactive elements corresponds to a different error metric or statistic based on an error metric. Interactive element 402a corresponds to an unidentified log record error metric, interactive element 402b corresponds to an unprocessed log record error metric, and interactive element 402c corresponds to a processed log record with an incorrect error metric. An unidentified log record error metric may correspond to an unidentified log record (such as a log record with an invalid source or authorization). An unprocessed log record error metric may correspond to a log record that was not processed due to a configuration error or other similar error. A processed log record with an incorrect error metric may correspond to a processed log record that includes one or more data or payload errors.

[0096] Interactive elements 402d-f correspond to statistics about error metrics. For example, interactive element 402d corresponds to the source of log records corresponding to unidentified log record error metrics, interactive element 402e corresponds to the origin of log records corresponding to unprocessed log record error metrics, and interactive element 402f corresponds to the distribution of error identifications that caused errors when log records were processed, as indicated by processed log records with incorrect error metrics. Any other suitable number or type of interactive elements 402 may be generated and displayed on user interface 400 to provide error metrics to a user or other suitable entity.

[0097] In some embodiments, a user or other suitable entity provides input via user interface 400, such as using a computer mouse or other suitable computer input device. The input may include a selection of at least one interactive element 402a-f in the first set 402 of interactive elements. Processing environment 100, any suitable component thereof, or any other suitable computing entity may generate a subsequent user interface in response to receiving an input indicating the selection of at least one interactive element 402. The subsequent user interface is generated based on the selection of at least one interactive element 402. For example, if interactive element 402a is selected, a subsequent user interface may be generated to include a list of one or more enriched log records corresponding to an unidentified logging error metric. In another example, if interactive element 402b is selected, a subsequent user interface may be generated to include a list of one or more enriched log records corresponding to an unprocessed logging error metric, etc.

[0098] Figure 55 is an example of a user interface 500 for displaying an enriched log record according to an embodiment. The user interface 500 may be or otherwise include, for example, the second user interface described with respect to block 340 of process 300. The user interface 500 includes a second set 502 of interactive elements, an element tracker 504, a graph 506, display options 508, and a graph controller 510. As shown, the second set 502 of interactive elements includes four interactive elements 502a-d, but any other suitable number (such as less than four or more than four) of interactive elements 502 may be included in the user interface 500 or otherwise displayed on the user interface 500. The second set 502 of interactive elements is shown as being arranged as a list on the user interface 500, but other suitable arrangements of the second set 502 of interactive elements are possible. In some embodiments, the second set 502 of interactive elements corresponds to an enriched log record including errors corresponding to the selected error metric.

[0099] Each interactive element 502 in the second set 502 of interactive elements corresponds to a different enriched log record having an error associated with the error metric. Interactive element 502a corresponds to a first enriched log record, interactive element 502b corresponds to a second enriched log record, interactive element 502c corresponds to a third enriched log record, and interactive element 502d corresponds to a fourth enriched log record. Each of the first, second, third, and fourth enriched log records can represent a different log record having a similar error associated with the selected error metric. For example, the enriched log record representation displayed on the user interface 500 includes a log record with a configuration or identification error that renders each displayed log record unrecognizable.

[0100] Each interactive element 502 in the second set of interactive elements 502 is selectable. For example, a user can provide input to the user interface 500 for selecting a particular interactive element 502, such as using a computer mouse. In addition, the user can scroll using a computer input device to find the desired interactive element 502 to select. Selecting a particular interactive element 502 can cause data about the corresponding enriched log record to be displayed (e.g., via a subsequent user interface).

[0101] The user interface 500 may include an element tracker 504 that may track the number of enriched log records generated for display on the user interface 500. For example, as shown, the element tracker 504 indicates that 25 enriched log records were generated on the user interface 500 and based on the selection of a particular interactive element in the first set 402 of interactive elements, a total of 162 enriched log records are available for selection. In addition, the user interface 500 may include a graph 506 that may illustrate the number of log records having errors corresponding to the selected error metric over a selected time period. The graph 506 includes a horizontal axis 512 representing time and a vertical axis 514 representing the number of log records having errors corresponding to the selected error metric. The graph 506 may be controlled by a graph controller 510 that may allow adjustment of the horizontal axis 512 of the graph 506 and other suitable adjustments to the graph 506.

[0102] In some embodiments, the second set 502 of interactive elements can display a subset of information about the corresponding enriched log record. For example, interactive element 502b displays the time of processing or storage of the enriched log record, the source of the log record, and other suitable information, such as the notes included in the enriched log record, on the user interface 500. As shown, interactive element 502b displays a note that includes at least a subset of error information related to the error included in the log record. For example, the log record includes a configuration error and a log analysis process or a payload error, and interactive element 502b displays the category or type of the error. Each interactive element in the second set 502 of interactive elements can display or otherwise include any other suitable information related to the enriched log record.

[0103] Figure 6600a-b for displaying the selected annotations of the enriched log record according to an embodiment. The user interface 600a and the user interface 600b can each be an example of a third user interface or otherwise include a third user interface, for example, as described in block 360 of the process 300. The user interface 600a includes the selected interactive element 602. In some embodiments, the selected interactive element 602 corresponds to the interactive element 502a displayed on the user interface 500, and the interactive element 502a is selected via input to the user interface 500 so that the user interface 600a is generated to display the selected interactive element 602. In addition, the user interface 600 includes interactive elements 502b-d, an organization column 604, and an information column 606. The interactive elements 502b-d can be displayed on both the user interface 500 and the user interface 600a. The user can select one or more of the interactive elements 502b-d (e.g., after viewing the information about the selected interactive element 602) via input to the user interface 600a for viewing information about one or more of the interactive elements 502b-d.

[0104] Organization column 604 displays one or more metadata about the enriched log record corresponding to the selected interactive element 602. For example, and as shown, user interface 600a displays the timestamp of the corresponding interactive element corresponding to the enriched log record in organization column 604. Thus, the interactive elements displayed on user interface 600a (and user interface 500) can be organized by time or by any other suitable organizational data (such as resource ID, error type, etc.).

[0105] Information column 606 shows one or more pieces of information about the enriched log record corresponding to the selected interactive element 602 or included in the enriched log record. For example, and as shown in the figure, user interface 600a displays error type, processing error fields (such as source, note, log group, log origin) and other suitable information about the enriched log record corresponding to the selected interactive element via information column 606. In addition, as shown in the figure, the notes of the enriched log record are displayed at the top of the enriched log record or other easily identifiable parts to provide notes to the user. The notes include the error information related to the error included in the log record corresponding to the selected interactive element 602.

[0106] In some embodiments, the error information includes the category of the error, the identification of the error, the location of the error in the log record, the source of the error, other suitable error information or any suitable combination thereof. Additionally or alternatively, in the example where the log record includes an error that blocks the processing of the log record, a user interface 600b can be generated. The user interface 600b includes the selected interactive element 602, the organization column 604, the information column 606 and any other suitable components. The user interface 600b includes a note, which is expanded into the enriched log record represented by the selected interactive element 602 displayed via the user interface 600b. The note includes a pointer 608 or other suitable interactive elements, which links the enriched log record to the backup version of the log record. The backup version of the log record can be stored or otherwise provided from a separated log data repository to prevent data loss during asynchronous processing of log data. The pointer 608 in the note can provide access to the backup version of the log record to facilitate the correction of the log record. In some embodiments, the pointer 608 in the selection note can be generated to display or otherwise provide the backup version of the log record to the user for correction of the log record.

[0107] Figure 7 700 is an example of a user interface 700 that displays different log records that are detected to include similar errors according to an embodiment. As shown, the user interface 700 includes log record A 702 and log record B 704. Each of log record A 702 and log record B 704 includes corresponding log data. For example, the log data of log record A 702 includes identification 706a, interaction type 708a, other information 710a, source 712a and error type 714a. In addition, the log data of log record B 704 includes identification 706b, interaction type 708b, other information 710b, source 712b and error type 714b. In the example shown, the log data of log record A 702 is different from the log data of log record B 704. For example, and as shown, identification 706a-b is different, interaction type 708a-b is different, and other information 710a-b is different.

[0108] Other log data of log record A 702 can be similar or identical to other log data of log record B 704. For example, in the illustrated example, log record A 702 and log record B 704 are not processed due to a timeout error or some other unprocessed suitable error. As shown, log record A 702 includes an error type 714a, which states "log data was not processed within the specified time", and log record B 704 includes an error type 714b, which states "server timeout error". Although not identical, error types 714a-b are similar. Thus, user interface 700 can include annotation box 716 or other suitable metadata, which can highlight or otherwise provide guidance for users or other suitable entities viewing user interface 700. For example, annotation box 716 is shown as stating "log record with timeout error", but other suitable information can be displayed via annotation box 716 to inform users of common or similar error types between log record A 702, log record B 704 and any other suitable log record.

[0109] Error types 714a-b can be adjusted or otherwise enriched. For example, the text of error types 714a-b can be adjusted to highlight or otherwise attract attention to error types 714a-b. As shown, the text of error types 714a-b is bold, italicized, and characterized by a font different from other parts of the log data. In other undepicted examples, the text of the error type can be highlighted in different colors, and the text of error types 714a-b can be different colors, etc., to highlight error types 714a-b. Additionally or alternatively, the source or cause of the error in the corresponding log record can be adjusted, highlighted, or otherwise enriched (in the same page or other pages of the interface). For example, in the example shown, each of the sources 712a-b includes a typographical error (one of the sources includes an extra "i" and one source includes an extra "w"), which may cause a corresponding timeout error. Thus, the text of source 712a-b is underlined to highlight or otherwise attract attention to the source of the error in the corresponding log record.

[0110] Figure 8It is an example of a user interface 800 that is detected as including different log records of similar errors according to the display of an embodiment. As shown, the user interface 800 includes log record C 802 and log record D 804. Log record C 802 and log record D 804 each include corresponding log data. For example, the log data of log record C 802 includes identification 806a, interaction type 808a, other information 810a, source 812a and error type 814a. In addition, the log data of log record D 804 includes identification 806b, interaction type 808b, other information 810b, source 812b and error type 814b. In some embodiments, the log data of log record C 802 can be different from the log data of log record D 804. For example, and as shown, identification 806a-b is different, interaction type 808a-b is different, and other information 810a-b is different.

[0111] Other log data of log record C802 can be similar or identical with other log data of log record D804.For example, due to configuration error or some other suitable unprocessed errors, log record C802 and log record D804 are not processed.As shown in the figure, log record C802 includes error type 814a, which states "log data configuration is incorrect", and log record D804 includes error type 814b, which states "configuration is incorrect".Although not identical, error type 814a-b is similar.Thus, user interface 800 includes annotation box 816 or other suitable metadata, which can highlight or otherwise provide guidance for the user or other suitable entity viewing user interface 800.For example, annotation box 816 is shown as stating "log record with configuration error", but other suitable information can be displayed via annotation box 816, to inform the user of common or similar error types between log record C802, log record D804 and any other suitable log record.

[0112] In some embodiments, error type 814a-b can be adjusted or otherwise enriched. For example, the text of error type 814a-b can be adjusted to highlight or otherwise attract attention to error type 814a-b. As shown, the text of error type 814a is enlarged, and characterized by a font different from other parts of the log data, and the text of error type 814b is bolded, italicized, and characterized by a font different from other parts of the log data. In other embodiments, the text of error type 814a-b can be highlighted with different colors, and the text of error type 814a-b can be different colors, etc., to highlight error type 814a-b. Additionally or alternatively, the source or cause of the error in the corresponding log record can be adjusted, highlighted or otherwise enriched. For example, the log data included in the corresponding log record is scattered in the entire log record with the arrangement of the corresponding configuration error identified by the asynchronous processing system. Thus, text of log data that may be arranged in a manner that is not recognizable is bolded, italicized, highlighted in a different color, etc., to highlight or otherwise draw attention to the source of the error in the corresponding log record.

[0113] Descriptive System

[0114] Fig. 9 A simplified diagram of a distributed system 900 for implementing one of the embodiments is depicted. In the illustrated embodiment, the distributed system 900 includes one or more client computing devices 902, 904, 906, and 908 that execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., over one or more networks 910. A server 912 can be communicatively coupled to the remote client computing devices 902, 904, 906, and 908 via the network(s) 910.

[0115] In various embodiments, server 912 may be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services may be provided to users of client computing devices 902, 904, 906, and / or 908 as web-based services or cloud services or under a software as a service (SaaS) model. Users operating client computing devices 902, 904, 906, and / or 908 may in turn utilize one or more client applications to interact with server 912 to utilize the services provided by these components.

[0116] In the configuration depicted in the figure, software components 918, 920 and 922 of distributed system 900 are shown as being implemented on server 912. In other embodiments, one or more components of distributed system 900 and / or the services provided by these components may also be implemented by one or more of client computing devices 902, 904, 906 and / or 908. Then, users operating client computing devices can use the services provided by these components using one or more client applications. These components can be implemented with hardware, firmware, software or a combination thereof. It should be appreciated that various different system configurations are possible, which may be different from distributed system 900. The embodiment shown in the figure is therefore an example of a distributed system for implementing an embodiment system, and is not intended to be restrictive.

[0117] The client computing devices 902, 904, 906, and / or 908 may be portable handheld devices (e.g., iPhone®, cellular phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass® head mounted displays), running software such as Microsoft Windows Mobile® and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc.), and enabling Internet, email, short message service (SMS), Blackberry®, or other communication protocols. The client computing devices may be general purpose personal computers, including, by way of example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing devices may be workstation computers running any of a variety of commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively or additionally, client computing devices 902, 904, 906, and 908 may be any other electronic devices capable of communicating over network(s) 910, such as thin client computers, Internet-enabled gaming systems (e.g., a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and / or personal messaging devices.

[0118] Although an exemplary distributed system 900 with four client computing devices is shown, any number of client computing devices may be supported. Other devices, such as devices with sensors, etc., may interact with the server 912 .

[0119] The network(s) 910 in the distributed system 900 may be any type of network familiar to those skilled in the art that may support data communications using any of a variety of commercially available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. Merely by way of example, the network(s) 910 may be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, etc. The network(s) 910 may be a wide area network and the Internet. It may include virtual networks, including but not limited to virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (e.g., networks operating in accordance with any of the protocols of the Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suite, Bluetooth®, and / or any other wireless protocol); and / or any combination of these networks and / or other networks.

[0120] The server 912 may be composed of one or more general-purpose computers, special-purpose server computers (including, by way of example, PC (personal computer) servers, UNIX® servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. In various embodiments, the server 912 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 912 may correspond to a server for performing the processing described above according to an embodiment of the present disclosure.

[0121] The server 912 may run an operating system including any of the operating systems discussed above, as well as any commercially available server operating system. The server 912 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA® servers, database servers, etc. Exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

[0122] In some embodiments, the server 912 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of the client computing devices 902, 904, 906, and 908. As examples, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party information sources and continuous data streams, which may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. The server 912 may also include one or more applications to display the data feeds and / or real-time events via one or more display devices of the client computing devices 902, 904, 906, and 908.

[0123] Distributed system 900 may also include one or more databases 914 and 916. Databases 914 and 916 may reside in various locations. As an example, one or more of databases 914 and 916 may reside on a non-transient storage medium local to server 912 (and / or residing in server 912). Alternatively, databases 914 and 916 may be remote from server 912 and communicate with server 912 via a network-based connection or a dedicated connection. In one set of examples, databases 914 and 916 may reside in a storage area network (SAN). Similarly, any necessary files for executing the functions that server 912 has may be appropriately stored locally on server 912 and / or stored remotely. In one set of embodiments, databases 914 and 916 may include a relational database suitable for storing, updating and retrieving data in response to commands in SQL format, such as a database provided by Oracle.

[0124] Fig.10 1 is a simplified block diagram of one or more components of a system environment 1000 that can provide services provided by one or more components of an embodiment system as cloud services according to an embodiment of the present disclosure. In the illustrated embodiment, the system environment 1000 includes one or more client computing devices 1004, 1006, and 1008 that can be used by a user to interact with a cloud infrastructure system 1002 that provides cloud services. The client computing device can be configured to operate a client application, such as a web browser, a proprietary client application (e.g., Oracle Forms), or some other application that can be used by a user of the client computing device to interact with the cloud infrastructure system 1002 to use the services provided by the cloud infrastructure system 1002.

[0125] It should be appreciated that the cloud infrastructure system 1002 depicted in the figure may have other components than those depicted. In addition, the embodiment shown in the figure is merely one example of a cloud infrastructure system that may incorporate embodiments of the present invention. In some other embodiments, the cloud infrastructure system 1002 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0126] Client computing devices 1004 , 1006 , and 1008 may be similar devices to those described above with respect to 902 , 904 , 906 , and 908 .

[0127] Although the exemplary system environment 1000 is shown with three client computing devices, any number of client computing devices may be supported. Other devices (such as devices with sensors, etc.) may interact with the cloud infrastructure system 1002 .

[0128] The network(s) 1010 may facilitate the communication and exchange of data between clients 1004, 1006, and 1008 and cloud infrastructure system 1002. Each network may be any type of network familiar to those skilled in the art that may support data communications using any of a variety of commercially available protocols, including those described above for network(s) 910.

[0129] Cloud infrastructure system 1002 may include one or more computers and / or servers, which may include those described above with respect to server 912 .

[0130] In some embodiments, the services provided by the cloud infrastructure system may include many services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. The services provided by the cloud infrastructure system can be dynamically expanded to meet the needs of users of the cloud infrastructure system. The specific instantiation of the services provided by the cloud infrastructure system is referred to as a "service instance" in this article. In general, any service available to users from a cloud service provider's system via a communication network (such as the Internet) is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own local servers and systems. For example, a cloud service provider's system can host applications, and users can order and use applications on demand via a communication network such as the Internet.

[0131] In some examples, services in a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided by a cloud provider to users, or other means as known in the art. For example, a service may include password-protected access to remote storage devices on the cloud over the Internet. As another example, a service may include a hosted relational database and scripting language middleware engine based on web services for private use by networked developers. As another example, a service may include access to an email software application hosted on a cloud provider's website.

[0132] In certain embodiments, the cloud infrastructure system 1002 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle public cloud provided by the present assignee.

[0133] In various embodiments, the cloud infrastructure system 1002 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by the cloud infrastructure system 1002. The cloud infrastructure system 1002 may provide cloud services via different deployment models. For example, services may be provided in accordance with a public cloud model, where the cloud infrastructure system 1002 is owned by an organization that sells cloud services (e.g., owned by Oracle), and the services are available to the general public or businesses in different industries. As another example, services may be provided in accordance with a private cloud model, where the cloud infrastructure system 1002 operates only for a single organization and may provide services to one or more entities within the organization. Cloud services may also be provided in accordance with a community cloud model, where the cloud infrastructure system 1002 and the services provided by the cloud infrastructure system 1002 are shared by several organizations in a related community. Cloud services may also be provided in accordance with a hybrid cloud model, which is a combination of two or more different models.

[0134] In some embodiments, the services provided by the cloud infrastructure system 1002 may include one or more services provided under a software as a service (SaaS) category, a platform as a service (PaaS) category, an infrastructure as a service (IaaS) category, or other service categories including hybrid services. A customer may subscribe to one or more services provided by the cloud infrastructure system 1002 via a subscription order. The cloud infrastructure system 1002 then performs processing to provide the services in the customer's subscription order.

[0135] In some embodiments, the services provided by the cloud infrastructure system 1002 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, a SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, customers may utilize applications executed on the cloud infrastructure system. Customers may obtain application services without the need for customers to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and flexibility to large organizations.

[0136] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared public architecture and enable the ability to build new applications by fully utilizing the shared services provided by the platform. The PaaS platform can manage and control the underlying software and infrastructure used to provide PaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without the need for customers to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), and the like.

[0137] By utilizing the services provided by the PaaS platform, customers can adopt programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model that enables organizations to pool database resources and supply database as a service to customers in the form of a database cloud. The middleware cloud service may provide customers with a platform for developing and deploying various cloud applications, and the Java cloud service may provide customers with a platform for deploying Java applications in a cloud infrastructure system.

[0138] A variety of different infrastructure services can be provided by IaaS platforms in cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources (such as storage, networks, and other basic computing resources) for customers to utilize the services provided by SaaS platforms and PaaS platforms.

[0139] In some embodiments, the cloud infrastructure system 1002 may also include infrastructure resources 1030 for providing resources for providing various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1030 may include a combination of pre-integrated and optimized hardware (such as servers, storage devices, and networking resources) to perform services provided by the PaaS platform and the SaaS platform.

[0140] In some embodiments, resources in cloud infrastructure system 1002 can be shared by multiple users and dynamically reallocated as needed. In addition, resources can be allocated to users in different time zones. For example, cloud infrastructure system 1030 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours, and then enable the same resources to be reallocated to another group of users in a different time zone, thereby maximizing the utilization of the resources.

[0141] In certain embodiments, a plurality of internal shared services 1032 may be provided that are shared by different components or modules of the cloud infrastructure system 1002 and services provided by the cloud infrastructure system 1002. These internal shared services may include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, and the like.

[0142] In some embodiments, the cloud infrastructure system 1002 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, the cloud management function can include capabilities for provisioning, managing, and tracking customer subscriptions received by the cloud infrastructure system 1002, etc.

[0143] In one embodiment, as depicted in the figure, cloud management functionality may be provided by one or more modules, such as order management module 1020, order orchestration module 1022, order supply module 1024, order management and monitoring module 1026, and identity management module 1028. These modules may include or be provided using one or more computers and / or servers, which may be general purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0144] In example operation 1034, a customer using a client device, such as client devices 1004, 1006, or 1008, may interact with cloud infrastructure system 1002 by requesting one or more services provided by cloud infrastructure system 1002 and placing an order to subscribe to one or more services offered by cloud infrastructure system 1002. In some examples, the customer may access a cloud user interface (UI), cloud UI 1012, cloud UI 1014, and / or cloud UI 1016, and place a subscription order via these UIs. Order information received by cloud infrastructure system 1002 in response to a customer placing an order may include information identifying the customer and one or more services offered by cloud infrastructure system 1002 to which the customer wants to subscribe.

[0145] After a customer places an order, order information is received via cloud UIs 1012 , 1014 , and / or 1016 .

[0146] At operation 1036, the order is stored in order database 1018. Order database 1018 may be one of several databases operated by cloud infrastructure system 1018 and in conjunction with other system elements.

[0147] At operation 1038, the order information is forwarded to order management module 1020. In some cases, order management module 1020 may be configured to perform billing and accounting functions related to the order, such as validating the order and, upon validation, booking the order.

[0148] At operation 1040, information about the order is transmitted to order orchestration module 1022. Order orchestration module 1022 can use the order information to orchestrate the provision of services and resources for the order placed by the customer. In some cases, order orchestration module 1022 can use the services of order provisioning module 1024 to orchestrate the provision of resources to support the subscribed services.

[0149] In certain embodiments, order orchestration module 1022 enables management of processing associated with each order and applies logic to determine whether an order should proceed to provisioning. At operation 1042, upon receiving an order for a new subscription, order orchestration module 1022 sends a request to order provisioning module 1024 to allocate resources and configure those resources required to fulfill the subscription order. Order provisioning module 1024 enables allocation of resources for services ordered by a customer. Order provisioning module 1024 provides an abstraction layer between the cloud services provided by cloud infrastructure system 1000 and the physical implementation layer for provisioning resources for providing the requested services. Thus, order orchestration module 1022 can be isolated from implementation details, such as whether services and resources are actually provisioned on the fly or pre-provisioned and allocated / assigned only upon request.

[0150] At operation 1044 , once the services and resources are provisioned, notifications of the provided services may be sent to customers on client devices 1004 , 1006 , and / or 1008 via the order provisioning module 1024 of the cloud infrastructure system 1002 .

[0151] At operation 1046, the order management and monitoring module 1026 can manage and track the customer's subscription order. In some cases, the order management and monitoring module 1026 can be configured to collect usage statistics for the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0152] In certain embodiments, cloud infrastructure system 1000 may include identity management module 1028. Identity management module 1028 may be configured to provide identity services, such as access management and authorization services in cloud infrastructure system 1000. In some embodiments, identity management module 1028 may control information about clients that wish to utilize services provided by cloud infrastructure system 1002. Such information may include information authenticating the identities of these clients and information describing which actions these clients are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). Identity management module 1028 may also include management of descriptive information about each client and how and by whom such descriptive information may be accessed and modified.

[0153] Fig.11 An exemplary computer system 1100 in which various embodiments of the present invention may be implemented is illustrated. System 1100 may be used to implement any of the above-described computer systems. As shown, computer system 1100 includes a processing unit 1104 that communicates with multiple peripheral subsystems via a bus subsystem 1102. These peripheral subsystems may include a processing acceleration unit 1106, an I / O subsystem 1108, a storage subsystem 1118, and a communication subsystem 1124. Storage subsystem 1118 includes a tangible computer-readable storage medium 1122 and a system memory 1110.

[0154] The bus subsystem 1102 provides a mechanism for the various components and subsystems of the computer system 1100 to communicate with each other as intended. Although the bus subsystem 1102 is schematically shown as a single bus, an alternative embodiment of the bus subsystem can utilize multiple buses. The bus subsystem 1102 can be any type of bus structure of several types, including a memory bus or a memory controller, a peripheral bus, and a local bus using any architecture of a variety of bus architectures. For example, this architecture can include an industry standard architecture (ISA) bus, a microchannel architecture (MCA) bus, an enhanced ISA (EISA) bus, a video electronics standard association (VESA) local bus, and a peripheral component interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured by the IEEE P1386.1 standard.

[0155] The processing unit 1104, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1100. One or more processors may be included in the processing unit 1104. These processors may include a single-core processor or a multi-core processor. In some embodiments, the processing unit 1104 may be implemented as one or more independent processing units 1132 and / or 1134, wherein a single-core processor or a multi-core processor is included in each processing unit. In other embodiments, the processing unit 1104 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0156] In various embodiments, the processing unit 1104 can execute various programs in response to the program code and can maintain multiple concurrently executed programs or processes. At any given time, some or all of the program code to be executed may reside in (one or more) processors 1104 and / or in the storage subsystem 1118. Through appropriate programming, (one or more) processors 1104 can provide the various functions described above. The computer system 1100 may additionally include a processing acceleration unit 1106, which may include a digital signal processor (DSP), a dedicated processor, etc.

[0157] I / O subsystem 1108 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touch pads or touch screens incorporated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as Microsoft Kinect® motion sensors, which enable users to control input devices such as Microsoft Xbox® 360 game controllers and interact with input devices by using natural user interfaces of gestures and voice commands. User interface input devices may also include eye gesture recognition devices, such as Google Glass® blink detectors that detect eye activity from users (e.g., "blinking" when taking photos and / or making menu selections) and transform eye gestures into inputs into input devices (e.g., Google Glass®). In addition, user interface input devices may include voice recognition sensing devices that enable users to interact with voice recognition systems (e.g., Siri® navigators) through voice commands.

[0158] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game panels and drawing tablets, and audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser rangefinders, and line-of-sight tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, and the like.

[0159] The user interface output device may include a display subsystem, indicator lights, or a non-visual display such as an audio output device. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD) or plasma display, a projection device, a touch screen, etc. In general, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 1100 to a user or other computers. For example, the user interface output device may include, but is not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0160] Computer system 1100 may include a storage subsystem 1118 containing software elements, shown as presently located within system memory 1110. System memory 1110 may store program instructions that may be loaded and executed on processing unit 1104, as well as data generated during execution of these programs.

[0161] Depending on the configuration and type of computer system 1100, system memory 1110 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or currently being operated and executed by processing unit 1104. In some embodiments, system memory 1110 may include a variety of different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some embodiments, a basic input / output system (BIOS) containing basic routines that help transfer information between elements within computer system 1100, such as during startup, may typically be stored in ROM. By way of example and not limitation, system memory 1110 also illustrates application programs 1112, which may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc., program data 1114, and operating system 1116. As examples, operating system 1116 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0162] Storage subsystem 1118 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provide the above-described functionality when executed by the processor may be stored in storage subsystem 1118. These software modules or instructions may be executed by processing unit 1104. Storage subsystem 1118 may also provide a repository for storing data used in accordance with the present invention.

[0163] The storage subsystem 1100 may also include a computer-readable storage media reader 1120 that may be further connected to a computer-readable storage medium 1122. Together with and optionally in conjunction with the system memory 1110, the computer-readable storage medium 1122 may comprehensively represent remote, local, fixed, and / or removable storage devices plus storage media for temporarily and / or more permanently containing, storing, transmitting, and retrieving computer-readable information.

[0164] The computer-readable storage medium 1122 containing the code or a portion of the code may also include any suitable media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and / or transmission of information. This may include tangible computer-readable storage media, such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media, such as data signals, data transmissions, or any other media that can be used to send the desired information and can be accessed by the computing system 1100.

[0165] For example, the computer-readable storage medium 1122 may include a hard disk drive that reads from or writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads from or writes to a removable non-volatile magnetic disk, and an optical disk drive that reads from or writes to a removable non-volatile optical disk (such as a CD ROM, a DVD, and a Blu-Ray® (Blu-ray) disk or other optical media). The computer-readable storage medium 1122 may include, but is not limited to: a Zip® drive, a flash memory card, a universal serial bus (USB) flash drive, a secure digital (SD) card, a DVD disk, a digital audio tape, etc. The computer-readable storage medium 1122 may also include: a solid-state drive (SSD) based on non-volatile memory (such as an SSD based on flash memory, an enterprise flash drive, a solid-state ROM, etc.), an SSD based on volatile memory (such as a solid-state RAM, a dynamic RAM, a static RAM, a DRAM-based SSD, a magnetoresistive RAM (MRAM) SSD), and a hybrid SSD using a combination of SSDs based on DRAM and flash memory. The disk drives and their associated computer-readable media may provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the computer system 1100 .

[0166] The communication subsystem 1124 provides an interface to other computer systems and networks. The communication subsystem 1124 is used as an interface for receiving data from other systems and sending data from the computer system 1100 to other systems. For example, the communication subsystem 1124 can enable the computer system 1100 to be connected to one or more devices via the Internet. In some embodiments, the communication subsystem 1124 may include a radio frequency (RF) transceiver component for accessing a wireless voice and / or data network (e.g., using cellular phone technology, advanced data network technology, such as 3G, 4G or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 1202.11 series standards) or other mobile communication technologies, or any combination thereof), a global positioning system (GPS) receiver component and / or other components. In some embodiments, as an addition or alternative to a wireless interface, the communication subsystem 1124 can provide a wired network connection (e.g., Ethernet).

[0167] In some embodiments, the communication subsystem 1124 may also receive input communications in the form of structured and / or unstructured data feeds 1126 , event streams 1128 , event updates 1130 , and the like on behalf of one or more users who may use the computer system 1100 .

[0168] For example, the communication subsystem 1124 can be configured to receive data feeds 1126 in real time from users of a social network and / or other communication service, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0169] Additionally, the communication subsystem 1124 may also be configured to receive data in the form of a continuous data stream, which may include event streams 1128 and / or event updates 1130, which may be continuous or unbounded in nature, real-time events without explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial price tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, and the like.

[0170] The communication subsystem 1124 can also be configured to output structured and / or unstructured data feeds 1126, event streams 1128, event updates 1130, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1100.

[0171] Computer system 1100 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® cellular phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

[0172] Due to the ever-changing nature of computers and networks, the description of the computer system 1100 depicted in the figure is intended only as a specific example. Many other configurations with more or fewer components than the system depicted in the figure are possible. For example, customized hardware may also be used and / or specific elements may be implemented in hardware, firmware, software (including applets) or a combination thereof. In addition, connections to other computing devices such as network input / output devices may be adopted. Based on the disclosure and teachings provided herein, those of ordinary skill in the art will recognize other ways and / or methods of implementing various embodiments.

[0173] In the foregoing description, various aspects of the present invention are described with reference to their specific embodiments, but those skilled in the art will recognize that the present invention is not limited thereto. The various features and aspects of the foregoing invention may be used alone or in combination. In addition, without departing from the broader spirit and scope of this specification, the embodiments may be used in any number of environments and applications other than those described herein. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A computer-implemented method comprising: receiving, by a computing device, log data comprising a plurality of log records; asynchronously processing, by a computing device, the log data to verify a configuration of each of the plurality of log records and data included in each of the plurality of log records; During asynchronous processing, determining that an attribute of a particular log record in the plurality of log records corresponds to an error; In response to determining that the particular log record in the plurality of log records corresponds to an error: determining, by the computing device, the category of the error; as well as generating, by the computing device, an enriched log record using the particular log record by augmenting the particular log record with error information indicating a category of error; as well as The enriched log record is transmitted by the computing device via a user interface to facilitate correction of errors included in the particular log record.

2. The computer-implemented method of claim 1 , wherein receiving log data comprises: receiving, by a computing device via real-time input from a data tenant, a first subset of the log data; as well as A second subset of log data is received by the computing device via input from a second log data repository, wherein the second subset of log data is received asynchronously from the second log data repository, and wherein the received log data is a union of the first subset of log data and the second subset of log data.

3. A computer-implemented method as described in claim 1, wherein the category of errors includes at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure, and wherein providing enriched logging includes generating, by the computing device, a user interface that displays the category of errors.

4. The computer-implemented method of claim 1 , further comprising, in response to determining that a particular log record includes an error: generating, by the computing device, an error metric for transmission to a monitoring service of the computing device, the error metric being used to monitor a set of errors associated with the log data; and A backup of the data included in the particular log record in the plurality of log records is generated by the computing device, and the backup of the data is stored in a third log data repository for allowing the computing device to access the backup of the data for further processing of the particular log record.

5. The computer-implemented method of claim 4, further comprising determining that the category of the error is payload, wherein generating the enriched log record comprises transmitting, by the computing device, the enriched log record to a log data repository for displaying the enriched log record via a user interface.

6. The computer-implemented method of claim 4, further comprising, in response to determining that the category of error is configuration: determining, by the computing device, that an error included in the particular log record prevents the particular log record from being processed; generating, by the computing device, a placeholder log record for storage in a log data repository; generating, by a computing device, an interactive element configured to retrieve the particular log record from the plurality of log records and provide the particular log record for display; augmenting the placeholder log record with the interactive element by the computing device; as well as The placeholder log record is transmitted, by the computing device, to a log data repository to provide access to the placeholder log record via a user interface.

7. The computer-implemented method of claim 1 , wherein the plurality of log records comprises a subset of log records, the subset comprising log records having errors, wherein the user interface comprises a list of selectable elements corresponding to the subset of log records, and wherein the computer-implemented method further comprises: receiving, by the computing device via the user interface, input indicating selection of a first selectable element in the list of selectable elements, wherein the first selectable element corresponds to the enriched log record; as well as A second user interface is generated by the computing device for display, the second user interface being used to display details about the enriched log record, wherein the details include error information.

8. A non-transitory machine-readable storage medium comprising a computer program product, the computer program product comprising instructions configured to cause a data processing apparatus to perform operations comprising: receiving log data comprising a plurality of log records; asynchronously processing log data to verify a configuration of each of the plurality of log records and data included in each of the plurality of log records; During asynchronous processing, determining that an attribute of a particular log record in the plurality of log records corresponds to an error; In response to determining that the particular log record in the plurality of log records corresponds to an error: Determine the category of error; as well as generating an enriched log record using the specific log record by augmenting the specific log record with error information indicating a category of the error; as well as The enriched log record is transmitted via a user interface to facilitate correction of errors included in a particular log record.

9. The non-transitory machine-readable storage medium of claim 8, wherein the operation of receiving log data comprises: receiving a first subset of log data via a real-time input from a data tenant; as well as A second subset of log data is received via input from a second log data repository, wherein the second subset of log data is received asynchronously from the second log data repository, and wherein the received log data is a union of the first subset of log data and the second subset of log data.

10. The non-transitory machine-readable storage medium of claim 8, wherein the category of errors comprises at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure, and wherein providing enriched logging comprises generating a user interface displaying the category of errors.

11. The non-transitory machine-readable storage medium of claim 8, wherein the operations further comprise, in response to determining that the particular log record includes an error: generating an error metric for transmission to a monitoring service of the computing device, the error metric being capable of being used to monitor a set of errors associated with the log data; and A backup of the data included in the particular log record in the plurality of log records is generated, the backup of the data being storable in a third log data repository for allowing the computing device to access the backup of the data for further processing of the particular log record.

12. The non-transitory machine-readable storage medium of claim 11, wherein the operation further comprises determining that the category of the error is a payload, wherein the operation of generating an enriched log record comprises transmitting the enriched log record to a log data repository to display the enriched log record via a user interface.

13. The non-transitory machine-readable storage medium of claim 11, wherein the operations further comprise, in response to determining that the category of the error is configuration: determining that an error included in a particular log record prevents the particular log record from being processed; generating placeholder log records for storage in a log data repository; generating an interactive element configured to retrieve the particular log record from the plurality of log records and provide the particular log record for display; Augment placeholder logging with interactive elements; as well as The placeholder log record is transmitted to a log data repository to provide access to the placeholder log record via a user interface.

14. The non-transitory machine-readable storage medium of claim 8, wherein the plurality of log records comprises a subset of log records, the subset comprising log records having errors, wherein the user interface comprises a list of selectable elements corresponding to the subset of log records, and wherein the operations further comprise: receiving, via a user interface, input indicating selection of a first selectable element in the list of selectable elements, wherein the first selectable element corresponds to an enriched log record; and A second user interface is generated for display, the second user interface being used to display details about the enriched log record, wherein the details include error information.

15. A system comprising: one or more data processors; as well as A non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform operations comprising: receiving log data comprising a plurality of log records; asynchronously processing log data to verify a configuration of each of the plurality of log records and data included in each of the plurality of log records; During asynchronous processing, determining that an attribute of a particular log record in the plurality of log records corresponds to an error; In response to determining that the particular log record in the plurality of log records corresponds to an error: Determine the category of error; and Using the specific log record, generating an enriched log record by augmenting the specific log record with error information indicating a category of the error; and The enriched log record is transmitted via a user interface to facilitate correction of errors included in that particular log record.

16. The system of claim 15, wherein the operation of receiving log data comprises: receiving a first subset of log data via a real-time input from a data tenant; as well as A second subset of log data is received via input from a second log data repository, wherein the second subset of log data is received asynchronously from the second log data repository, and wherein the received log data is a union of the first subset of log data and the second subset of log data.

17. The system of claim 15, wherein the category of errors comprises at least one of invalid configuration, invalid payload, structure mismatch, unauthorized, not found, or parsing failure, and wherein providing enriched logging comprises generating a user interface displaying the category of errors.

18. The system of claim 15, wherein the operations further comprise, in response to determining that the particular log record includes an error: generating an error metric for transmission to a monitoring service of the computing device, the error metric being capable of being used to monitor a set of errors associated with the log data; generating a backup of data included in the particular log record in the plurality of log records, the backup of the data being storable in a third log data repository for allowing a computing device to access the backup of the data for further processing of the particular log record; as well as The category of the error is determined to be a payload, wherein the operation of generating the enriched log record includes transmitting the enriched log record to a log data repository to display the enriched log record via a user interface.

19. The system of claim 15, wherein the operations further comprise, in response to determining that the particular log record includes an error: generating an error metric for transmission to a monitoring service of the computing device, the error metric being capable of being used to monitor a set of errors associated with the log data; generating a backup of data included in the particular log record in the plurality of log records, the backup of the data being storable in a third log data repository for allowing a computing device to access the backup of the data for further processing of the particular log record; determining that an error included in the particular log record prevents the particular log record from being processed; generating placeholder log records for storage in a log data repository; generating an interactive element configured to retrieve the particular log record from the plurality of log records and provide the particular log record for display; Augment placeholder logging with interactive elements; as well as The placeholder log records are transmitted to a log data repository to provide access to the placeholder log records via a user interface.

20. The system of claim 15, wherein the plurality of log records comprises a subset of log records, the subset comprising log records having errors, wherein the user interface comprises a list of selectable elements corresponding to the subset of log records, and wherein the operations further comprise: receiving, via a user interface, input indicating selection of a first selectable element in the list of selectable elements, wherein the first selectable element corresponds to an enriched log record; and A second user interface is generated for display, the second user interface for displaying details about the enriched log record, wherein the details include error information.

Images