Log field anomaly analysis method and device, equipment and storage medium

Abstract

The invention discloses a log field anomaly analysis method and device, equipment and a storage medium. Comprising the steps of obtaining a time span log, and determining a tracking identifier corresponding to the time span log; acquiring a span log set related to the tracking identifier from a log memory; and obtaining a page index, calculating index data corresponding to each field in the span log set according to the page index, and calculating an abnormal score of each field according to the index data. By determining the tracking identifier, an accurate initial basis is provided for subsequent acquisition of a complete call link log, and pertinence and accuracy of an analysis starting point are ensured. By collecting all the logs of the same calling link, the situation that key logs are omitted due to the fact that the time range is only limited to the time range selected by the user is avoided, and the integrity of log data is guaranteed. By generating the exception score, the exception degree of the field can be quantified, the exception condition of the field is intuitively reflected, the user is helped to quickly recognize the field needing to be focused on, and the exception troubleshooting efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of log analysis, and in particular to a method, apparatus, device, and storage medium for log field anomaly analysis. Background Technology

[0002] In distributed systems, a single user request often involves multiple service calls, forming a complete call chain and generating a large amount of logs. To ensure stable system operation, logs need to be analyzed to troubleshoot anomalies. However, log data is scattered and complexly correlated. How to efficiently locate the fields that cause problems from massive amounts of logs is a significant challenge for operations and development teams.

[0003] In existing log analysis solutions, some methods collect logs within a user-specified time range, extract trace identifiers to correlate related logs, and then analyze log fields based on preset metrics. For example, after collecting logs for a specific time period, trace identifiers are used to summarize logs from the same call chain, and basic data such as the total number of entries and the number of errors involved in a field are calculated to help users identify anomalies.

[0004] However, existing technologies often only acquire logs within a user-specified time range, ignoring logs outside that time range in the same call chain. This results in incomplete analysis data and affects the accuracy of anomaly detection. Furthermore, existing technologies cannot quantify the degree of field anomalies, making it difficult to meet the analysis needs of different scenarios and hindering users from quickly locating the key fields that cause problems. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for log field anomaly analysis. By collecting logs over a specific time span and obtaining corresponding tracking identifiers, it acquires all relevant logs in the same call chain. Then, it calculates the index data of the field based on page metrics and generates anomaly scores by combining custom weights. This solves the technical problem in the prior art where log analysis is incomplete and lacks scientific quantitative anomaly assessment methods, making it difficult for users to quickly locate the cause of the problem.

[0006] According to one aspect of the present invention, a method for anomaly analysis of log fields is provided, the method comprising:

[0007] Obtain time-span logs and determine the tracking identifier corresponding to the time-span logs;

[0008] Retrieve the span log set associated with the trace identifier from the log store, wherein the span log set includes logs that exceed the time span in the same call chain;

[0009] Obtain page metrics, calculate the corresponding metric data for each field in the span log collection based on the page metrics, and calculate the anomaly score for each field based on the metric data.

[0010] Optionally, the process involves acquiring time span logs and determining the corresponding tracing identifiers for the time span logs. This includes: acquiring the time range input by the user; collecting log data within the time range in real time using a collector to obtain time span logs; and extracting fields from the time span logs used to identify the call chain as tracing identifiers. The tracing identifiers are used to associate all logs within the same call chain.

[0011] Optionally, obtaining page metrics includes: using each field as a target field and the target as an input parameter, establishing a default field function to generate a general variable representing the target field; obtaining the user-input search processing statement, binding the general variable to the search processing statement through the default field function; and executing the bound search processing statement to generate page metrics for each target field.

[0012] Optional, the indicator data includes total number, number of errors, error rate, root error, percentage of erroneous samples, percentage of normal samples, and difference.

[0013] Optionally, the abnormal score of each field is calculated based on the indicator data, including: obtaining the indicator weights corresponding to each user-defined indicator data; taking each field as the field to be analyzed and determining the target indicator data corresponding to the field to be analyzed; and performing weighted calculation on each target indicator data based on the indicator weights to determine the abnormal score of the field to be analyzed.

[0014] Optionally, the method also includes: generating a display page based on the anomaly scores of each field, and displaying the display page to the user; obtaining the user's selected fields of interest based on the display page, and determining the field type of the fields of interest; when the field type is an entity, retrieving the alarms and performance metrics of the corresponding entity, generating a performance view, and feeding the performance view back to the user.

[0015] Optionally, determining the field type of the field of interest includes: obtaining an entity configuration list, matching the field of interest against the entity configuration list, wherein the entity configuration list includes each entity field; determining whether the field of interest is located in the entity configuration list, and if so, determining that the field type is entity; otherwise, determining that the field type is non-entity.

[0016] According to another aspect of the present invention, a log field anomaly analysis apparatus is provided, the apparatus comprising:

[0017] The log and tracking identifier acquisition module is used to acquire logs over a time span and determine the tracking identifiers corresponding to the logs over that time span.

[0018] The span log collection module is used to retrieve the span log collection associated with the tracking identifier from the log storage. The span log collection includes logs that exceed the time span in the same call chain.

[0019] The Field Anomaly Score Determination Module is used to obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score for each field based on the metric data.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] At least one processor;

[0022] and a memory communicatively connected to the at least one processor;

[0023] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform a log field anomaly analysis method according to any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a log field anomaly analysis method according to any embodiment of the present invention.

[0025] The technical solution of this invention provides an accurate initial basis for obtaining complete call chain logs by determining the tracking identifier, ensuring the relevance and accuracy of the analysis starting point. By collecting all logs from the same call chain, it avoids missing key logs due to limiting the time range to the user, thus ensuring the integrity of the log data. By generating anomaly scores, it quantifies the degree of anomaly of fields, intuitively reflecting the anomaly situation of the fields, helping users quickly identify fields that require key attention, and improving the efficiency and accuracy of anomaly investigation.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a log field anomaly analysis method provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is a flowchart of another log field anomaly analysis method provided in Embodiment 2 of the present invention;

[0030] Figure 3 This is a schematic diagram of the structure of a log field anomaly analysis device provided in Embodiment 3 of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of an electronic device that implements a log field anomaly analysis method according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1 This is a flowchart illustrating a log field anomaly analysis method according to Embodiment 1 of the present invention. This embodiment is applicable to log analysis scenarios. The method can be executed by a log field anomaly analysis device, which can be implemented in hardware and / or software and can be configured in a computer controller. Figure 1 As shown, the method includes:

[0036] S110. Obtain the time span log and determine the tracking identifier corresponding to the time span log.

[0037] Time span logs refer to log data generated within a specific time range selected by the user on the page. Time span logs form the basis for subsequent analysis; for example, if a user selects logs from the most recent 30 minutes, then the logs within that period constitute the time span log. A trace identifier is an identifier used to identify a complete call chain. In a distributed system, a single user request may involve multiple service calls, forming a complete call chain that shares the same trace identifier (traceid). The trace identifier allows disparate logs to be linked together, reconstructing the entire request processing flow.

[0038] Optionally, the process involves acquiring time span logs and determining the corresponding tracing identifiers for the time span logs. This includes: acquiring the time range input by the user; collecting log data within the time range in real time using a collector to obtain time span logs; and extracting fields from the time span logs used to identify the call chain as tracing identifiers. The tracing identifiers are used to associate all logs within the same call chain.

[0039] The collector is the component in the controller responsible for collecting log data. It collects logs generated within a user-selected time range. The user-input time range refers to a specific time interval chosen by the user on the system page; for example, the user might select the most recent 30 minutes. This time range becomes the time range for subsequent analysis. The collector then performs real-time collection of log data generated within this 30-minute period. The logs collected within this time range constitute the time-span log.

[0040] Furthermore, after obtaining the logs spanning the time span, it's necessary to extract the field used to identify the call chain from these logs; this field is the trace ID. Because in a distributed system, a single user request may involve calls between multiple services, these calls together constitute a complete call chain, and all logs belonging to the same call chain will contain the same trace ID. Therefore, the trace ID allows all logs within the same call chain to be associated. The trace ID is the key identifier for achieving log association within the same call chain.

[0041] S120. Obtain a span log set associated with the trace identifier from the log storage, wherein the span log set includes logs that exceed the time span in the same call chain.

[0042] In this context, a log storage unit refers to a storage component used to store various logs generated by the system. It can query and return relevant log data based on a tracing identifier, serving as the data source for obtaining a complete log set. Span logs refer to the logs generated by each service call or processing step in distributed tracing, where each span is considered a span. The span log set is the sum of all span logs associated with the same tracing identifier. Because span logs for the same call chain may be distributed across different points in time, some span logs may exceed the user-selected time span but still belong to the call chain. Therefore, the span log set will include logs that exceed the selected time span.

[0043] S130. Obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score for each field based on the metric data.

[0044] Page metrics refer to parameters set by users on the page or provided by the system by default to measure the degree of anomalies in fields. These include total number, number of errors, error rate, number of root errors, percentage of error samples, percentage of normal samples, and difference. Fields refer to specific attribute information contained in the span logs, such as IP address and service name, and are the basic unit of analysis. Each field has multiple possible values. Metric data refers to the specific numerical values ​​obtained by calculating different values ​​for each field in the span log set based on the page metrics. For example, for the IP field with values ​​10, 22, 34, and 5, the corresponding metric data can be calculated as: total number 100, number of errors 80, and error rate 80%. Anomaly score is a numerical value used to characterize the degree of anomalies in a field, calculated based on the metric data of each field and user-defined metric weights. The calculation formula is: Anomaly Score = Weight 1 × Metric 1 value + Weight 2 × Metric 2 value + ... + Weight n × Metric n value. The higher the score, the higher the degree of anomaly in that field, and the more attention it requires.

[0045] Optionally, obtaining page metrics includes: using each field as a target field and the target as an input parameter, establishing a default field function to generate a general variable representing the target field; obtaining the user-input search processing statement, binding the general variable to the search processing statement through the default field function; and executing the bound search processing statement to generate page metrics for each target field.

[0046] The default field function is a pre-defined function that receives a target field as input and generates a generic variable representing that target field, enabling unified processing of different target fields. The generic variable is generated by the default field function and used to uniformly represent each target field. The Search Processing Language (SPL) is a user-input command used to process and analyze log data. Binding refers to the process of associating the generic variable with the search processing language using the default field function. Page metrics are parameters used to measure the degree of anomalies in a field. After executing the bound search processing language, a specific parameter value is calculated for each target field, which is the corresponding page metric. For example, for the IP target field, calculating the total number of different IP addresses, the number of errors, etc., constitutes the page metric for the IP field.

[0047] Specifically, when retrieving page metrics, the controller treats each field as a target field and uses it as an input parameter to create a default field function. This function generates a generic variable representing each target field, such as ${field}. Next, the controller receives the SPL statement input by the user and binds the generic variable to the user-input SPL statement using the default field function. Finally, by executing the bound SPL statement, the corresponding page metric is generated for each target field.

[0048] Optional, the indicator data includes total number, number of errors, error rate, root error, percentage of erroneous samples, percentage of normal samples, and difference.

[0049] The total number of records in this field refers to the total number of log entries across the spans associated with a given value. For example, if an IP address has 100 log entries across its spans, then the total number of records for that IP field is 100. The error count is the number of erroneous log entries across the spans associated with that field. If 80 out of 100 log entries for an IP address are erroneous, then the error count for that IP is 80. The error rate is the proportion of errors in the log entries across the spans associated with that field. It is calculated by dividing the error count by the total number of records. For example, 80 errors divided by 100 results in an error rate of 80%. Field values ​​with high error rates require close monitoring. The root error count refers to the number of errors for which this field is considered a root error. A root error occurs when a span error in the last span log of a tracing chain causes errors in other span logs within that chain. For example, if an IP address experiences 70 root errors, its root error count is 70. A higher root error count indicates more traces might be reporting errors due to that field value, making it a likely root cause. The error sample percentage is the proportion of each field value among all error samples. For instance, if there are 260 error samples and a certain IP has 80 errors, its error sample percentage is 80 divided by 260, approximately 30.76%. The normal sample percentage is the proportion of each field value among all normal samples. Assuming there are 140 normal samples and a certain IP has 20 normal span logs, its normal sample percentage is 20 divided by 140, approximately 14.28%. The difference is the error sample percentage minus the normal sample percentage. For example, if an IP has an error sample percentage of 30.76% and a normal sample percentage of 14.28%, the difference is 16.48%. If the difference is large, it means that the field value accounts for a large proportion of incorrect samples and a small proportion of normal samples, and most of them are incorrect, so it needs to be paid close attention to.

[0050] Optionally, the abnormal score of each field is calculated based on the indicator data, including: obtaining the indicator weights corresponding to each user-defined indicator data; taking each field as the field to be analyzed and determining the target indicator data corresponding to the field to be analyzed; and performing weighted calculation on each target indicator data based on the indicator weights to determine the abnormal score of the field to be analyzed.

[0051] Specifically, the controller retrieves the weights of each metric data that the user customizes on the page. This means users can set different weights for each metric, such as total count, number of errors, and error rate, to reflect the importance of different metrics in anomaly detection. The controller treats each field as a field to be analyzed and determines the corresponding target metric data for that field, including the previously calculated total count, number of errors, error rate, root error count, percentage of erroneous samples, percentage of normal samples, and difference. Then, based on the user-defined weights, the controller performs a weighted calculation on the target metric data for that field. This calculation involves multiplying each metric data by its corresponding weight and summing all the products. The result is the anomaly score for that field. For example, if the user sets a weight of 0.5 for the error rate, the error rate of the field to be analyzed is 80%, the weight set for the root error count is 0.3, the root error count is 70, and there are other indicators and their corresponding weights, then the anomaly score is 80% × 0.5 + 70 × 0.3 + other indicator data × corresponding weights. The final calculation result is the anomaly score of each field to be analyzed. The higher the score, the higher the degree of anomaly of the field, and the more attention it needs to pay.

[0052] The technical solution of this invention provides an accurate initial basis for obtaining complete call chain logs by determining the tracking identifier, ensuring the relevance and accuracy of the analysis starting point. By collecting all logs from the same call chain, it avoids missing key logs due to limiting the time range to the user, thus ensuring the integrity of the log data. By generating anomaly scores, it quantifies the degree of anomaly of fields, intuitively reflecting the anomaly situation of the fields, helping users quickly identify fields that require key attention, and improving the efficiency and accuracy of anomaly investigation.

[0053] Example 2

[0054] Figure 2 This is a flowchart of a log field anomaly analysis method provided in Embodiment 2 of the present invention. This embodiment adds an entity performance view binding process based on Embodiment 1. The specific content of steps S210-S230 is largely the same as steps S110-S130 in Embodiment 1, therefore, it will not be described again in this embodiment. Figure 2 As shown, the method includes:

[0055] S210. Obtain the time span log and determine the tracking identifier corresponding to the time span log.

[0056] Optionally, the process involves acquiring time span logs and determining the corresponding tracing identifiers for the time span logs. This includes: acquiring the time range input by the user; collecting log data within the time range in real time using a collector to obtain time span logs; and extracting fields from the time span logs used to identify the call chain as tracing identifiers. The tracing identifiers are used to associate all logs within the same call chain.

[0057] S220. Obtain a span log set associated with the trace identifier from the log storage, wherein the span log set includes logs that exceed the time span in the same call chain.

[0058] S230. Obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score for each field based on the metric data.

[0059] Optionally, obtaining page metrics includes: using each field as a target field and the target as an input parameter, establishing a default field function to generate a general variable representing the target field; obtaining the user-input search processing statement, binding the general variable to the search processing statement through the default field function; and executing the bound search processing statement to generate page metrics for each target field.

[0060] Optional, the indicator data includes total number, number of errors, error rate, root error, percentage of erroneous samples, percentage of normal samples, and difference.

[0061] Optionally, the abnormal score of each field is calculated based on the indicator data, including: obtaining the indicator weights corresponding to each user-defined indicator data; taking each field as the field to be analyzed and determining the target indicator data corresponding to the field to be analyzed; and performing weighted calculation on each target indicator data based on the indicator weights to determine the abnormal score of the field to be analyzed.

[0062] S240. Generate a display page based on the abnormal scores of each field and display the display page to the user.

[0063] Specifically, when generating the display page based on the anomaly scores of each field, the controller will organize and present the anomaly scores of each field and the corresponding indicator data. The displayed page will list information such as the total number of different IPs, services, etc., the number of errors, the error rate, the number of root errors, and the anomaly score. Users can intuitively see the anomalies of each field.

[0064] S250. Obtain the user's selected fields of interest based on the display page, and determine the field type of the fields of interest.

[0065] The display page refers to the page generated by the controller based on the anomaly scores and corresponding metrics for each field. The page clearly lists relevant information for each field, allowing users to intuitively understand the anomalies in each field. "Fields of interest" refers to the fields selected by the user on the display page that require focused attention. By viewing the anomaly scores and metrics for each field on the display page, users can select fields with high anomaly levels or those they believe require in-depth analysis as fields of interest. Field types are divided into entity and non-entity types. Entities typically refer to types with specific corresponding objects such as hosts and services, while non-entity fields are field types that do not correspond to a specific entity.

[0066] Specifically, based on the displayed page, users can select the fields they want to focus on, i.e., the fields of interest. Then, the controller will determine the field type of the field of interest and determine whether it belongs to an entity, such as the host corresponding to an IP address or the service corresponding to a service name.

[0067] Optionally, determining the field type of the field of interest includes: obtaining an entity configuration list, matching the field of interest against the entity configuration list, wherein the entity configuration list includes each entity field; determining whether the field of interest is located in the entity configuration list, and if so, determining that the field type is entity; otherwise, determining that the field type is non-entity.

[0068] S260. When the field type is entity, retrieve the alarms and performance metrics of the corresponding entity, generate a performance view, and feed the performance view back to the user.

[0069] An entity refers to an object represented by a field with a specific corresponding object, such as a host corresponding to an IP address or a service corresponding to a service name. Entities have bound performance views that can be associated with relevant alarms and performance metrics. Alarms are abnormal notifications generated by an entity within a user-selected time range, reflecting problems or anomalies that occurred within that time period. Performance metrics are key data used to measure the operational status of an entity. A performance view is a view generated by the controller after processing the entity's alarm information and performance metrics.

[0070] Specifically, when the type of the field of interest is determined to be an entity, the controller will retrieve the alarm information and related performance metrics for that entity within the user-selected time range. These performance metrics include CPU utilization, memory utilization, and CPU load. The controller will then compile the alarms and performance metrics into a performance view and provide it to the user, allowing them to view the specific status of the entity, verify whether it is indeed abnormal, and provide a basis for further root cause analysis.

[0071] The technical solution of this invention generates a display page, allowing users to quickly understand the anomalies of each field, providing a clear basis for subsequently selecting fields of interest. By focusing on fields of key interest to users, it clarifies whether they belong to an entity, laying the foundation for targeted information retrieval. Users can intuitively view entity alerts and performance status, verifying whether the entity is abnormal, thus improving the efficiency of analyzing abnormal log fields.

[0072] Example 3

[0073] Figure 3 This is a schematic diagram of a log field anomaly analysis device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a log and tracking identifier acquisition module 310, used to acquire time span logs and determine the tracking identifiers corresponding to the time span logs;

[0074] The span log collection module 320 is used to obtain a span log collection related to the tracking identifier from the log storage, wherein the span log collection includes logs that exceed the time span in the same call chain;

[0075] The Field Anomaly Score Determination Module 330 is used to obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score of each field based on the metric data.

[0076] Optionally, the log and trace identifier acquisition module 310 is specifically used for: acquiring the time range input by the user, collecting log data within the time range in real time through a collector to obtain time span logs; extracting the fields used to identify the call chain from the time span logs as trace identifiers, wherein the trace identifiers are used to associate all logs in the same call chain.

[0077] Optionally, the field anomaly score determination module 330 specifically includes: a page metric acquisition unit, used to: take each field as a target field and the target as an input parameter, establish a default field function, and generate a general variable representing the target field; acquire the user-input search processing statement, and bind the general variable to the search processing statement through the default field function; execute the bound search processing statement to generate page metrics for each target field.

[0078] Optional, optional, the field anomaly score determination module 330 specifically includes: an anomaly score calculation unit, used to: obtain the indicator weights corresponding to each user-defined indicator data; take each field as the field to be analyzed and determine the target indicator data corresponding to the field to be analyzed; perform weighted calculation on each target indicator data based on the indicator weights to determine the anomaly score of the field to be analyzed.

[0079] Optionally, the device also includes a performance view feedback module, used to: generate a display page based on the abnormal scores of each field, and display the display page to the user; obtain the user's selected fields of interest based on the display page, and determine the field type of the fields of interest; when the field type is an entity, retrieve the alarms and performance indicators of the corresponding entity, generate a performance view, and feed the performance view back to the user.

[0080] Optionally, the performance view feedback module specifically includes: a field type determination unit, used to: obtain an entity configuration list, match the fields of interest through the entity configuration list, wherein the entity configuration list includes each entity field; determine whether the field of interest is located in the entity configuration list, if so, determine that the field type is entity; otherwise, determine that the field type is non-entity.

[0081] The technical solution of this invention provides an accurate initial basis for obtaining complete call chain logs by determining the tracking identifier, ensuring the relevance and accuracy of the analysis starting point. By collecting all logs from the same call chain, it avoids missing key logs due to limiting the time range to the user, thus ensuring the integrity of the log data. By generating anomaly scores, it quantifies the degree of anomaly of fields, intuitively reflecting the anomaly situation of the fields, helping users quickly identify fields that require key attention, and improving the efficiency and accuracy of anomaly investigation.

[0082] The log field anomaly analysis device provided in this embodiment of the invention can execute a log field anomaly analysis method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0083] Example 4

[0084] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a log field anomaly analysis method.

[0088] In some embodiments, a log field anomaly analysis method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the log field anomaly analysis method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a log field anomaly analysis method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for anomaly analysis of log fields, characterized in that, include: Obtain time-span logs and determine the tracking identifier corresponding to the time-span logs; Retrieve a span log set associated with the tracking identifier from the log storage, wherein the span log set includes logs from the same call chain that exceed the time span; Obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score for each field based on the metric data.

2. The method according to claim 1, characterized in that, The step of acquiring time-span logs and determining the tracking identifier corresponding to the time-span logs includes: The system obtains the time range input by the user and collects log data within the time range in real time through a collector to obtain time-span logs. Extract the field used to identify the call chain from the time span log, and use it as the tracing identifier, wherein the tracing identifier is used to associate all logs in the same call chain.

3. The method according to claim 1, characterized in that, The acquisition of page metrics includes: Each field is used as the target field, and the target is used as the input parameter to create a default field function to generate a general variable representing the target field; Obtain the search processing statement input by the user, and bind general variables to the search processing statement through default field functions; Execute the bound search processing statements to generate page metrics for each target field.

4. The method according to claim 1, characterized in that, The indicator data includes total number, number of errors, error rate, root error number, percentage of erroneous samples, percentage of normal samples, and difference.

5. The method according to claim 4, characterized in that, The calculation of the anomaly score for each field based on the indicator data includes: Obtain the weights of each user-defined metric. Each field is selected as the field to be analyzed, and the target indicator data corresponding to the field to be analyzed is determined. The target indicator data are weighted and calculated based on the weights of each indicator to determine the anomaly score of the field to be analyzed.

6. The method according to claim 1, characterized in that, The method further includes: A display page is generated based on the abnormal scores of each field, and the display page is shown to the user. Based on the displayed page, obtain the fields of interest selected by the user and determine the field type of the fields of interest; When the field type is an entity, retrieve the corresponding entity's alarms and performance metrics, generate a performance view, and then feed the performance view back to the user.

7. The method according to claim 6, characterized in that, The field types for determining the fields of interest include: Obtain an entity configuration list, and match the fields of interest using the entity configuration list, wherein the entity configuration list includes each entity field; Determine whether the field of interest is in the entity configuration list; if so, determine that the field type is an entity. Otherwise, determine that the field type is non-entity.

8. A log field anomaly analysis device, characterized in that, include: The log and tracking identifier acquisition module is used to acquire time-span logs and determine the tracking identifiers corresponding to the time-span logs. The span log set module is used to retrieve a span log set related to the tracking identifier from the log storage, wherein the span log set includes logs in the same call chain that exceed the time span; The field anomaly score determination module is used to obtain page metrics, calculate the metric data corresponding to each field in the span log set based on the page metrics, and calculate the anomaly score of each field based on the metric data.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-7.

Images