A method and system for reporting abnormal logs in wearable devices

Abstract

The present invention discloses a method and system for reporting abnormal logs in a wearable device, the method comprising: monitoring and collecting log data of the wearable device; preprocessing the log data, grouping and clustering based on the preprocessed log data to extract a log template; obtaining a log sequence set according to the log template, and extracting log key features based on the log sequence set; performing abnormality detection on the log data according to the log key features, performing abnormal response to the wearable device according to the abnormality detection result, and reporting abnormal events to a background server. The present invention groups and clusters the log data of the wearable device and extracts the log template, obtains the log sequence set and extracts the log key features, performs abnormality detection on the log data according to the log key features, performs abnormal response and abnormal event reporting in a timely manner, can process a large amount of log data, effectively utilizes the various key features of the log data, and improves the accuracy of abnormality detection of the wearable device log.

Description

Technical Field

[0001] The present invention relates to the technical field of log reporting, and in particular to a method and system for reporting abnormal logs in a wearable device. Background Art

[0002] With the rapid development of technology, wearable devices such as smart bracelets and phone watches have become indispensable electronic devices in people's daily lives. During the use of wearable devices, system errors or application errors often occur, which will generate abnormal logs.

[0003] In the prior art, when a wearable device application fails, the wearable device's abnormal log is usually detected in a rule-based manner, and then the problem reflected in the abnormal log is handled through manual review. However, as wearable devices become more and more intelligent, device logs usually contain massive amounts of data, with many types of failures and fast changes, and a large number of meaningless suspected abnormal logs, which poses a huge challenge to log anomaly detection and leads to a high false alarm rate. In addition, the existing anomaly detection method loses a lot of key information in the process of extracting the original log information, and the utilization of log information is not high. Summary of the invention

[0004] In view of the defects in the prior art, the present invention provides a method and system for reporting abnormal logs in a wearable device.

[0005] In a first aspect, a method for reporting abnormal logs in a wearable device includes:

[0006] Monitor and collect log data from wearable devices;

[0007] Preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates;

[0008] Acquire a log sequence set according to the log template, and extract key features of the log based on the log sequence set;

[0009] The log data is detected for anomalies according to the key features of the log, an abnormal response is made to the wearable device according to the abnormal detection result, and the abnormal event is reported to the background server.

[0010] Furthermore, the log data is preprocessed as follows:

[0011] Traversing all log contents of the log data and performing regular expression matching to locate variable characters in the log data;

[0012] Separating and removing the variable characters from the log content to obtain a non-variable log;

[0013] The non-variable log is standardized to reduce character string spaces in the log content and separate to obtain a plurality of non-variable words;

[0014] Construct corresponding log statements according to the non-variable words to obtain preprocessed log data.

[0015] Furthermore, the log data after preprocessing is grouped and clustered to extract log templates, specifically:

[0016] Constructing a log parsing tree, and generating a plurality of log groups based on the log parsing tree according to the preprocessed log data;

[0017] Calculating similarities between log statements in the multiple log groups, obtaining a similar statement group whose similarity is greater than a threshold, and generating an initial statement template according to the log statements in the similar statement group;

[0018] Obtain the number of occurrences of the same non-variable word in each log sentence of the initial sentence template respectively, and calculate the weight of each non-variable word;

[0019] The non-variable words are converted into semantic vectors according to a preset rule, and the non-variable words in the log sentence corresponding to the initial sentence template are weighted according to the weight to obtain a corresponding semantic vector set;

[0020] The initial sentence templates are clustered based on the semantic vector set to generate a log template with semantic information.

[0021] Furthermore, the step of obtaining a log sequence set according to the log template and extracting log key features based on the log sequence set is specifically as follows:

[0022] Dividing the log data of the wearable device into a plurality of log sequences according to the log template, and generating a log sequence set;

[0023] Extracting key log features in the log sequence based on the log sequence set;

[0024] The key features of the log include time series features, invariant features and statistical features.

[0025] Furthermore, the log data is detected for anomalies according to the key features of the log, an abnormal response is made to the wearable device according to the abnormal detection result, and the abnormal event is reported to the background server, specifically:

[0026] Pre-building a neural network model, wherein the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model;

[0027] Input the time series features into the AT-CNN-BiGRU model, and input the invariant features and statistical features into the GBDT model to perform anomaly detection on the log data;

[0028] An abnormal log is identified according to the abnormal detection result, an abnormal response is made to the wearable device according to the abnormal log, and the abnormal event is reported to the background server.

[0029] In a second aspect, a system for reporting abnormal logs in a wearable device includes:

[0030] Data monitoring module: used to monitor and collect log data of wearable devices;

[0031] Template extraction module: used for preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates;

[0032] Feature extraction module: used to obtain a log sequence set according to the log template, and extract key log features based on the log sequence set;

[0033] Abnormal reporting module: used to detect abnormalities in log data according to the key features of the log, respond to abnormalities to the wearable device according to the abnormality detection results, and report abnormal events to the background server.

[0034] Furthermore, the template extraction module is specifically used for:

[0035] Traversing all log contents of the log data and performing regular expression matching to locate variable characters in the log data;

[0036] Separating and removing the variable characters from the log content to obtain a non-variable log;

[0037] The non-variable log is standardized to reduce character string spaces in the log content and separate to obtain a plurality of non-variable words;

[0038] Construct corresponding log statements according to the non-variable words to obtain preprocessed log data.

[0039] Furthermore, the template extraction module is also used for:

[0040] Constructing a log parsing tree, and generating a plurality of log groups based on the log parsing tree according to the preprocessed log data;

[0041] Calculating similarities between log statements in the multiple log groups, obtaining a similar statement group whose similarity is greater than a threshold, and generating an initial statement template according to the log statements in the similar statement group;

[0042] Obtain the number of occurrences of the same non-variable word in each log sentence of the initial sentence template respectively, and calculate the weight of each non-variable word;

[0043] The non-variable words are converted into semantic vectors according to a preset rule, and the non-variable words in the log sentence corresponding to the initial sentence template are weighted according to the weight to obtain a corresponding semantic vector set;

[0044] The initial sentence templates are clustered based on the semantic vector set to generate a log template with semantic information.

[0045] Furthermore, the feature extraction module is specifically used for:

[0046] Dividing the log data of the wearable device into a plurality of log sequences according to the log template, and generating a log sequence set;

[0047] Extracting key log features in the log sequence based on the log sequence set;

[0048] The key features of the log include time series features, invariant features and statistical features.

[0049] Furthermore, the abnormality reporting module is specifically used for:

[0050] Pre-building a neural network model, wherein the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model;

[0051] Input the time series features into the AT-CNN-BiGRU model, and input the invariant features and statistical features into the GBDT model to perform anomaly detection on the log data;

[0052] An abnormal log is identified according to the abnormal detection result, an abnormal response is made to the wearable device according to the abnormal log, and the abnormal event is reported to the background server.

[0053] The beneficial effects of the present invention are as follows: grouping and clustering the monitored wearable device log data and extracting log templates, obtaining log sequence sets based on the log templates and extracting log key features, and then performing anomaly detection on the log data based on a neural network model according to the log key features, making timely anomaly responses and reporting of abnormal events, being able to process a large amount of log data, effectively utilizing the key features of the log data, and improving the accuracy of anomaly detection in wearable device logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0055] Figure 1 A flowchart of a method for reporting abnormal logs in a wearable device provided in Embodiment 1 of the present invention;

[0056] Figure 2 A module block diagram of a system for reporting abnormal logs in a wearable device provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0057] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0058] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0059] Embodiment 1

[0060] like Figure 1 As shown, a method for reporting abnormal logs in a wearable device includes:

[0061] S1: monitors and collects log data from wearable devices;

[0062] Specifically, the log data of the wearable device is monitored and collected in real time.

[0063] S2: preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates;

[0064] Specifically, all log contents in the log data are traversed and regular expression matching is performed to locate variable characters in the log data, and the variable characters are separated and removed from the log content to obtain non-variable logs. The non-variable logs are standardized to reduce the string of the processed log data, including reducing multiple consecutive spaces to one space, removing redundant spaces in the string, and separating it into multiple non-variable words. Finally, the corresponding log statements are constructed according to the non-variable words to obtain the preprocessed log data.

[0065] Preferably, the standardization process includes but is not limited to using the isalpha() function of char to remove variables containing numbers, using tuples to remove repeated characters, etc.

[0066] Furthermore, a log parsing tree is constructed. After the log statements in the log data are preprocessed, log statements with the same structure generally have the same log length. Therefore, the root node of the log parsing tree uses the same length corresponding to the same type of log data to classify the log statements by length, and efficiently processes each classified subtree in a multi-threaded concurrent execution mode. Since the log statement begins with a constant, the constant at the beginning of the sentence is often a landmark word of this type of log statement and has a representative role. Therefore, each subtree is reclassified by constant prefix matching. After length and multiple constant prefix matching, most of the log classification work can be completed and log groups can be generated.

[0067] A similarity function between log statements in multiple log groups is obtained, and similarity values ​​are calculated. Log groups whose similarity values ​​exceed a preset similarity threshold are obtained, and all log statements contained therein are extracted as similar statement groups. Log statements in similar statement groups are clustered to generate multiple initial statement templates.

[0068] The log sentence in the initial sentence template is obtained, and several independent non-variable words in the log sentence are extracted, the number of occurrences of each non-variable word in the log sentence is counted, and the weight of each non-variable word is calculated based on the number of occurrences, and the weight reflects the importance of each non-variable word to the log sentence. The non-variable words are converted into semantic vectors according to preset rules, and the non-variable words of the corresponding sentence of the initial sentence template are weighted based on the semantic vector according to the calculated weight to obtain the corresponding semantic vector set.

[0069] Furthermore, after obtaining the semantic vector set, clustering compression is performed on the acquired initial sentence template to generate a log template with semantic information. Preferably, the clustering compression method includes but is not limited to using a DBSCAN method.

[0070] S3: acquiring a log sequence set according to the log template, and extracting key features of the log based on the log sequence set;

[0071] Specifically, after obtaining the log template, the log data of the wearable device is segmented into multiple log sequences based on the log template, and a log sequence set is generated. The log segmentation method includes but is not limited to subject-based independent sequence segmentation and sliding window segmentation.

[0072] After obtaining the log sequence set, we extract key features such as time series features, invariant features, and statistical features from the log sequence using the log sequence as the basic unit. Since logs are recorded in the order in which log events occur, the processed logs are also traversed in the order of the timestamps generated by the logs during the log sequence segmentation process. The sequence itself already represents the time sequence relationship, so the time series feature is the time sequence of the log events in the sequence. We only need to directly abstract the sequence content to represent it as a time series feature.

[0073] S4: performing anomaly detection on the log data according to the key features of the log, responding to the wearable device according to the anomaly detection result, and reporting the abnormal event to the background server;

[0074] Specifically, a neural network model is pre-constructed, and the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model.

[0075] The AT-CNN-BiGRU model includes a BiGRU bidirectional gated recurrent unit layer, an attention mechanism AT layer, and a convolutional neural network CNN layer. Among them, the BiGRU bidirectional gated recurrent unit layer is a further optimization and improvement based on the gated recurrent unit. The gated recurrent neural network can better capture the dependencies on sequences with long time step distances. The reset gate helps capture short-term dependencies in the sequence, and the update gate helps capture long-term dependencies in the sequence. When the reset gate is open, the gated recurrent unit contains the basic recurrent neural network; when the update gate is open, the gated recurrent unit can skip subsequences. BiGRU technology can better mine data and derive the correlation in the data. It can complete training and testing with a small number of parameters. The use of bidirectional gated recurrent units can not only capture historical data but also predict future data to monitor the charging status of electric vehicles. It has stronger memory than other technologies, and the output results are more accurate and faster.

[0076] Attention Mechanism The AT layer compares the importance of the acquired information through the attention mechanism to achieve selective learning of information, that is, to filter out the most core and important data information from a large amount of data information, and focus on these important information, ignoring most of the unimportant information. The larger the weight, the more focus is on its corresponding Value, that is, the weight represents the importance of the information, and the Value is its corresponding information.

[0077] The convolutional neural network (CNN) layer is a type of feedforward neural network with deep structure and convolution calculations. It is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities, strong feature extraction capabilities, and can classify input information in a translation-invariant manner according to their hierarchical structure. Therefore, they are also called "translation-invariant artificial neural networks." CNN can solve the three major vulnerabilities of fully connected neural networks. When CNN expands an image into a vector, it will not lose data information and maintain the original data characteristics. When processing too many parameters, it can effectively reduce the number of parameters, improve efficiency, and reduce the difficulty and complexity of the network calculation process. When CNN faces a large number of parameter processing, it can also avoid overfitting of the network.

[0078] In order to make different features be effectively utilized and improve the accuracy of anomaly detection in log data, the time series features are input into the AT-CNN-BiGRU model, and the invariant features and statistical features are input into the GBDT model to perform anomaly detection on log data.

[0079] Furthermore, an abnormal log is identified based on the abnormal detection result, and an abnormal response is made to the wearable device based on the abnormal log to remind the wearer to check the abnormal information. At the same time, the abnormal event is reported to the background server based on the abnormal log, and the background server is notified in time to check and handle the problems reflected in the abnormal log.

[0080] Embodiment 2

[0081] like Figure 2 As shown, a reporting system for abnormal logs in a wearable device includes:

[0082] Data monitoring module: used to monitor and collect log data of wearable devices;

[0083] Template extraction module: used for preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates;

[0084] Feature extraction module: used to obtain a log sequence set according to the log template, and extract key log features based on the log sequence set;

[0085] Abnormal reporting module: used to detect abnormalities in log data according to the key features of the log, respond to abnormalities to the wearable device according to the abnormality detection results, and report abnormal events to the background server.

[0086] Furthermore, the template extraction module is specifically used for:

[0087] Traversing all log contents of the log data and performing regular expression matching to locate variable characters in the log data;

[0088] Separating and removing the variable characters from the log content to obtain a non-variable log;

[0089] The non-variable log is standardized to reduce character string spaces in the log content and separate to obtain a plurality of non-variable words;

[0090] Construct corresponding log statements according to the non-variable words to obtain preprocessed log data.

[0091] Furthermore, the template extraction module is also used for:

[0092] Constructing a log parsing tree, and generating a plurality of log groups based on the log parsing tree according to the preprocessed log data;

[0093] Calculating similarities between log statements in the multiple log groups, obtaining a similar statement group whose similarity is greater than a threshold, and generating an initial statement template according to the log statements in the similar statement group;

[0094] Obtain the number of occurrences of the same non-variable word in each log sentence of the initial sentence template respectively, and calculate the weight of each non-variable word;

[0095] The non-variable words are converted into semantic vectors according to a preset rule, and the non-variable words in the log sentence corresponding to the initial sentence template are weighted according to the weight to obtain a corresponding semantic vector set;

[0096] The initial sentence templates are clustered based on the semantic vector set to generate a log template with semantic information.

[0097] Furthermore, the feature extraction module is specifically used for:

[0098] Dividing the log data of the wearable device into a plurality of log sequences according to the log template, and generating a log sequence set;

[0099] Extracting key log features in the log sequence based on the log sequence set;

[0100] The key features of the log include time series features, invariant features and statistical features.

[0101] Furthermore, the abnormality reporting module is specifically used for:

[0102] Pre-building a neural network model, wherein the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model;

[0103] Input the time series features into the AT-CNN-BiGRU model, and input the invariant features and statistical features into the GBDT model to perform anomaly detection on the log data;

[0104] An abnormal log is identified according to the abnormal detection result, an abnormal response is made to the wearable device according to the abnormal log, and the abnormal event is reported to the background server.

[0105] It should be noted that, for a more specific workflow of a reporting system for abnormal logs in a wearable device, please refer to the aforementioned method embodiment section, which will not be repeated here.

[0106] The present invention groups and clusters the monitored wearable device log data and extracts log templates, obtains log sequence sets based on the log templates and extracts key features of the logs, and then performs anomaly detection on the log data based on a neural network model according to the key features of the logs, performs anomaly response and reports abnormal events in a timely manner, can process a large amount of log data, effectively utilizes the key features of the log data, and improves the accuracy of anomaly detection in the wearable device logs.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A method for reporting abnormal logs in a wearable device, characterized in that: include: Monitor and collect log data from wearable devices; Preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates; Acquire a log sequence set according to the log template, and extract key log features based on the log sequence set; Performing anomaly detection on the log data according to the key features of the log, responding to the wearable device according to the anomaly detection result, and reporting the abnormal event to the background server; The preprocessing of the log data is specifically as follows: Traversing all log contents of the log data and performing regular expression matching to locate variable characters in the log data; Separating and removing the variable characters from the log content to obtain a non-variable log; The non-variable log is standardized to reduce character string spaces in the log content and separate multiple non-variable words; the standardization includes using the isalpha() function of char to remove variables containing numbers and using a tuple to remove repeated characters; Constructing corresponding log statements according to the non-variable words to obtain preprocessed log data; The step of obtaining a log sequence set according to the log template and extracting log key features based on the log sequence set is specifically as follows: Dividing the log data of the wearable device into a plurality of log sequences according to the log template, and generating a log sequence set; Extracting key log features in the log sequence based on the log sequence set; The key features of the log include time series features, invariant features and statistical features; The abnormality detection is performed on the log data according to the key features of the log, an abnormal response is made to the wearable device according to the abnormality detection result, and the abnormal event is reported to the background server, specifically: Pre-building a neural network model, wherein the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model; The time series features are input into the AT-CNN-BiGRU model, and the invariant features and statistical features are input into the GBDT model to perform anomaly detection on the log data; the AT-CNN-BiGRU model includes a BiGRU bidirectional gated recurrent unit layer, an attention mechanism AT layer, and a convolutional neural network CNN layer; Attention Mechanism The AT layer compares the importance of the acquired information through the attention mechanism to achieve selective learning of information, filter out the most important data information from a large amount of data information, and focus on these important information, ignoring most unimportant information; An abnormal log is identified according to the abnormal detection result, an abnormal response is made to the wearable device according to the abnormal log, and the abnormal event is reported to the background server.

2. The method for reporting abnormal logs in a wearable device according to claim 1, characterized in that: The log data after preprocessing is grouped and clustered to extract log templates, specifically: Constructing a log parsing tree, and generating a plurality of log groups based on the log parsing tree according to the preprocessed log data; Calculating similarities between log statements in the multiple log groups, obtaining a similar statement group whose similarity is greater than a threshold, and generating an initial statement template according to the log statements in the similar statement group; Obtain the number of occurrences of the same non-variable word in each log sentence of the initial sentence template respectively, and calculate the weight of each non-variable word; The non-variable words are converted into semantic vectors according to a preset rule, and the non-variable words in the log sentence corresponding to the initial sentence template are weighted according to the weight to obtain a corresponding semantic vector set; The initial sentence templates are clustered based on the semantic vector set to generate a log template with semantic information.

3. A reporting system for abnormal logs in a wearable device, characterized in that: include: Data monitoring module: used to monitor and collect log data of wearable devices; Template extraction module: used for preprocessing the log data, and performing grouping and clustering based on the preprocessed log data to extract log templates; Feature extraction module: used to obtain a log sequence set according to the log template, and extract key log features based on the log sequence set; Abnormal reporting module: used to detect abnormalities in log data according to the key features of the log, respond to abnormalities to the wearable device according to the abnormality detection results, and report abnormal events to the background server; The template extraction module is specifically used for: Traversing all log contents of the log data and performing regular expression matching to locate variable characters in the log data; Separating and removing the variable characters from the log content to obtain a non-variable log; The non-variable log is standardized to reduce character string spaces in the log content and separate multiple non-variable words; the standardization includes using the isalpha() function of char to remove variables containing numbers and using a tuple to remove repeated characters; Constructing corresponding log statements according to the non-variable words to obtain preprocessed log data; The feature extraction module is specifically used for: Dividing the log data of the wearable device into a plurality of log sequences according to the log template, and generating a log sequence set; Extracting key log features in the log sequence based on the log sequence set; The key features of the log include time series features, invariant features and statistical features; The abnormality reporting module is specifically used for: Pre-building a neural network model, wherein the neural network model is composed of an AT-CNN-BiGRU model and a GBDT model; The time series features are input into the AT-CNN-BiGRU model, and the invariant features and statistical features are input into the GBDT model to perform anomaly detection on the log data; the AT-CNN-BiGRU model includes a BiGRU bidirectional gated recurrent unit layer, an attention mechanism AT layer, and a convolutional neural network CNN layer; Attention Mechanism The AT layer compares the importance of the acquired information through the attention mechanism to achieve selective learning of information, filter out the most important data information from a large amount of data information, and focus on these important information, ignoring most unimportant information; An abnormal log is identified according to the abnormal detection result, an abnormal response is made to the wearable device according to the abnormal log, and the abnormal event is reported to the background server.

4. The system for reporting abnormal logs in a wearable device according to claim 3, characterized in that: The template extraction module is also used for: Constructing a log parsing tree, and generating a plurality of log groups based on the log parsing tree according to the preprocessed log data; Calculating similarities between log statements in the multiple log groups, obtaining a similar statement group whose similarity is greater than a threshold, and generating an initial statement template according to the log statements in the similar statement group; Obtain the number of occurrences of the same non-variable word in each log sentence of the initial sentence template respectively, and calculate the weight of each non-variable word; The non-variable words are converted into semantic vectors according to a preset rule, and the non-variable words in the log sentence corresponding to the initial sentence template are weighted according to the weight to obtain a corresponding semantic vector set; The initial sentence templates are clustered based on the semantic vector set to generate a log template with semantic information.

Images