Log detection method, device and computer readable storage medium

Abstract

Embodiments of the present application disclose a log detection method and device and a computer readable storage medium, which can be applied to various scenes of artificial intelligence. The system command log is obtained. The historical command information and the historical path information are spliced to obtain command path information, and based on a preset detection window, the to-be-detected command path information is identified in the command path information. The to-be-detected command path information is subjected to feature extraction to obtain the to-be-detected command path feature corresponding to the preset detection window. The to-be-detected command path feature is subjected to feature dimension conversion to obtain the command path feature of a target dimension, and based on the command path feature of the target dimension, the abnormal command log is detected in the system command log. In this way, the features of the system command log based on the command information and the path information are extracted, and based on the extracted features, the abnormal command log is detected in the system command log, thereby improving the accuracy of log detection and further improving the efficiency of log detection.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a log detection method, device, and computer-readable storage medium. Background Art

[0002] In recent years, with the rapid development of internet technology, the scale of computer systems has continued to grow. Consequently, it is difficult to avoid the occurrence of abnormal behavior and other problems during computer system operation. These abnormalities can lead to performance degradation, data corruption, and other issues, resulting in significant information and economic losses. Therefore, it is necessary to detect abnormal information in computer systems. In existing technologies, abnormal information is often identified by detecting computer system log data.

[0003] During the research and practice of existing technologies, it was found that existing log detection methods detect abnormal information based on log data in natural language form, requiring the log information to have a certain degree of discrimination. However, not all log data of computer systems are in natural language form. For other forms of log data, existing log detection methods cannot accurately detect abnormal information of computer systems, resulting in low accuracy of log detection, and thus low efficiency of log detection. Summary of the Invention

[0004] The embodiments of the present application provide a log detection method, device, and computer-readable storage medium, which can improve the accuracy of system log detection and thereby improve the efficiency of log detection.

[0005] This embodiment of the present application provides a log detection method, including:

[0006] Obtaining a system command log, wherein the system command log includes historical command information and historical path information corresponding to the historical command information;

[0007] splicing the historical command information and the historical path information to obtain command path information, and identifying at least one piece of command path information to be detected in the command path information based on a preset detection window;

[0008] Extracting features of the command path information to be detected to obtain features of the command path to be detected corresponding to the preset detection window;

[0009] The command path feature to be detected is converted into a feature dimension to obtain a command path feature of a target dimension, and abnormal command logs are detected in the system command log based on the command path feature of the target dimension.

[0010] Accordingly, an embodiment of the present application provides a log detection device, including:

[0011] an acquiring unit, configured to acquire a system command log, wherein the system command log includes historical command information and historical path information corresponding to the historical command information;

[0012] an identification unit, configured to combine the historical command information and the historical path information to obtain command path information, and identify at least one piece of command path information to be detected in the command path information based on a preset detection window;

[0013] An extraction unit, configured to extract features from the path information of the command to be detected to obtain features of the path of the command to be detected corresponding to the preset detection window;

[0014] The detection unit is used to convert the feature dimension of the command path feature to be detected to obtain the command path feature of the target dimension, and detect abnormal command logs in the system command log based on the command path feature of the target dimension.

[0015] In one embodiment, the extraction unit includes:

[0016] a splicing subunit, configured to splice the command path information to be detected corresponding to the preset detection window to obtain spliced ​​command path information;

[0017] A multi-dimensional feature extraction subunit, configured to perform multi-dimensional feature extraction on the spliced ​​command path information to obtain command path features corresponding to each dimension;

[0018] The feature fusion subunit is used to perform feature fusion on the command path features of each dimension to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0019] In one embodiment, the multi-dimensional feature extraction subunit includes:

[0020] a word processing module, configured to perform word segmentation processing on the spliced ​​command path information to obtain command path words, and perform feature extraction on the command path words to obtain word features and position features;

[0021] a fragment processing module, configured to segment the spliced ​​command path information to obtain at least one command path information fragment, and perform feature extraction on the command path information fragment to obtain log fragment features;

[0022] An assignment module is used to use the word feature, position feature and log segment feature as the command log feature corresponding to the spliced ​​command path information.

[0023] In one embodiment, the feature fusion subunit includes:

[0024] a correlation feature extraction module, configured to extract features from the command path features to obtain correlation features corresponding to the command path features of each dimension, wherein the correlation features are feature information representing the correlation relationship between the command path features of each dimension and the command path features of other dimensions;

[0025] an association weight determination module, configured to determine, based on the association feature, an association weight corresponding to the command path feature of each dimension;

[0026] The feature fusion module is used to fuse the command path features of each dimension according to the association weight to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0027] In one embodiment, the detection unit includes:

[0028] an abnormality probability calculation subunit, configured to calculate an abnormality probability of the spliced ​​command path information based on the command path characteristics of the target dimension;

[0029] a target command path information screening subunit, configured to screen target command path information from the spliced ​​command path information according to the abnormality probability;

[0030] The abnormal command log identification subunit is used to identify abnormal command logs in the system command log based on the target command path information.

[0031] In one embodiment, the abnormal command log identification subunit includes:

[0032] a module for extracting command path information to be identified, configured to extract at least one piece of command path information to be identified from the target command path information;

[0033] an exception occurrence count module, configured to count the number of command path information to be identified in the target command path information, and obtain the exception occurrence count corresponding to each piece of command path information to be identified;

[0034] The abnormal command log screening module is used to screen abnormal command logs in the system command log based on the number of occurrences of the abnormality.

[0035] In one embodiment, the abnormal command log screening module includes:

[0036] A total number of times counting submodule is used to count the total number of times each command path information to be identified appears in the system command log;

[0037] an anomaly score scoring submodule, configured to score the command path information to be identified based on the total number of occurrences and the number of anomaly occurrences corresponding to the command path information to be identified, and obtain an anomaly score corresponding to the command path information to be identified;

[0038] The abnormal command log determining submodule is configured to filter out abnormal command path information from the command path information to be identified according to the abnormality score, and determine the system command log corresponding to the abnormal command path information as the abnormal command log.

[0039] In one embodiment, the log detection device includes:

[0040] A receiving unit, configured to receive command information to be detected;

[0041] A search unit, configured to search the historical command information corresponding to the abnormal command log for target command information that matches the command information to be detected;

[0042] The shielding unit is used to shield the execution of the command information to be detected when the target command information is searched.

[0043] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for loading by a processor to execute the steps in any log detection method provided in the embodiment of the present application.

[0044] In addition, an embodiment of the present application also provides a computer device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the log detection method provided in the embodiment of the present application.

[0045] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the log detection method provided in the present application.

[0046] The embodiment of the present application obtains the system command log; splices the historical command information and the historical path information to obtain the command path information, and identifies at least one command path information to be detected in the command path information based on the preset detection window; extracts features of the command path information to be detected to obtain the command path features to be detected corresponding to the preset detection window; converts the feature dimension of the command path features to be detected to obtain the command path features of the target dimension, and detects abnormal command logs in the system command log based on the command path features of the target dimension. In this way, the features of the system command log are characterized in a fine-grained manner by extracting the features of the command information and the corresponding path information, thereby detecting abnormal command logs in the system command log based on the extracted features, so as to accurately detect abnormal problems of the operating system whose logs are in the form of commands and paths. At the same time, the accuracy of log detection can be adjusted by using the preset detection window to extract the features of the command information and the corresponding path information, thereby improving the accuracy of log detection and thus improving the efficiency of log detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a schematic diagram of an implementation scenario of a log detection method provided in an embodiment of the present application;

[0049] Figure 2 This is a flow chart of a log detection method provided in an embodiment of the present application;

[0050] Figure 3 This is a feature extraction diagram of a log detection method provided by an embodiment of the present application;

[0051] Figure 4 This is another flow chart of a log detection method provided by an embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the structure of the log detection device provided in an embodiment of the present application;

[0053] Figure 6 It is a structural diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0055] The embodiments of the present application provide a log detection method, device, and computer-readable storage medium. The log detection device can be integrated into a computer device, which can be a server or a terminal.

[0056] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as basic cloud computing services such as big data and artificial intelligence platforms. Terminals may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. Terminals and servers can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0057] See also Figure 1 , taking the log detection device integrated into the computer equipment as an example, Figure 1 A schematic diagram of an implementation scenario of the log detection method provided in an embodiment of the present application, wherein the computer device can be a server or a terminal, and the computer device can obtain a system command log; splice historical command information and historical path information to obtain command path information, and based on a preset detection window, identify at least one command path information to be detected in the command path information; perform feature extraction on the command path information to be detected to obtain the command path feature to be detected corresponding to the preset detection window; perform feature dimension conversion on the command path feature to be detected to obtain the command path feature of the target dimension, and detect abnormal command logs in the system command log based on the command path feature of the target dimension.

[0058] It should be noted that the embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Figure 1The illustrated schematic diagram of the implementation environment scenario of the log detection method is merely an example. The implementation environment scenario of the log detection method described in the embodiments of this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation of the technical solutions provided in the embodiments of this application. Persons skilled in the art will appreciate that with the evolution of log detection and the emergence of new business scenarios, the technical solutions provided in this application will be equally applicable to similar technical problems.

[0059] The solutions provided in the embodiments of this application involve technologies such as machine learning of artificial intelligence, which are specifically illustrated by the following embodiments. It should be noted that the order of description of the following embodiments does not limit the preferred order of the embodiments.

[0060] This embodiment will be described from the perspective of a log detection device. The log detection device may be integrated into a computer device, which may be a server. This application does not limit this.

[0061] See also Figure 2 , Figure 2 : is a flow chart of the log detection method provided in the embodiment of the present application. The log detection method includes:

[0062] In step 101, a system command log is obtained.

[0063] The system command log may be a system log, which may be a computer system such as an operating system. The system command log may include historical command information and historical path information corresponding to the historical command information. The historical command information may include information about commands executed by the system in a historical process, and the historical path information may include information about file paths corresponding to the historical command information. One command may correspond to multiple file paths.

[0064] In order to facilitate the management of data in large-scale operating systems, the system logs of large-scale operating systems generally consist of simple command content and corresponding file paths. However, existing methods for detecting abnormal information based on log data in natural language form require that the log information has a certain degree of differentiation. For log data composed of commands and paths in the operating system, existing log detection methods cannot accurately detect abnormal information of the system based on the system log, resulting in low accuracy of log detection, which in turn leads to low efficiency of log detection. To this end, an embodiment of the present application provides a log detection method that characterizes the features of the system command log in a fine-grained manner by extracting features of command information and corresponding path information, so as to accurately detect abnormal command logs of the system in the system log. The log detection method provided by the present application is described in detail below.

[0065] In step 102, historical command information and historical path information are spliced ​​to obtain command path information, and at least one piece of command path information to be detected is identified in the command path information based on a preset detection window.

[0066] Wherein, the command path information can be the information obtained by splicing the historical command information and the corresponding historical path information, the preset detection window can be a pre-set window for detecting the system command log, and the size of the preset detection window can be set according to actual needs. For example, when the accuracy requirement of log detection is large, a smaller preset detection window can be set; when the accuracy requirement of log detection is small, a larger preset detection window can be set; when the computing power of the log detection device is strong, a larger preset detection window can be set; when the computing power of the log detection device is weak, a smaller preset detection window can be set. For example, each preset detection window can be set to detect and process two system command logs, or each preset detection window can be set to detect and process 100 system command logs, etc. In this way, the accuracy of log detection can be adjusted by controlling the window size, thereby improving the efficiency of log detection. The command path information to be detected can be at least one command path information that each preset detection window is responsible for processing.

[0067] Among them, based on the preset detection window, there can be multiple ways to identify at least one command path information to be detected in the command path information. For example, the detection parameters corresponding to the preset detection window and the attribute information corresponding to each command path information can be obtained, and according to the detection parameters and attribute information, at least one command path information to be detected can be screened out in the command path information.

[0068] The detection parameter may be a parameter used when detecting command path information in a preset detection window, for example, the number of command path information that can be processed simultaneously by the preset detection window. The attribute information may be information that characterizes the nature of the command path information and its relationship to other command path information, for example, the time information of the command path information, which may include information about the time when the system command log corresponding to the command path information was generated.

[0069] Among them, according to the detection parameter and attribute information, there can be multiple ways to filter out at least one command path information to be detected in the command path information. For example, taking the attribute information as time information and the detection parameter as the number of command path information as an example, the command path information can be sorted according to the time information corresponding to each command path information to obtain the sorted command path information, so that, in the sorted command path information, at least one command path information with a ranking corresponding to the number of command path information can be determined as the command path information to be detected. For example. Assuming that the preset detection window can process 5 command path information at the same time, the preset detection window can be used to filter out the top 5 command path information in the sorted command path information to obtain the command path information to be detected.

[0070] Optionally, the historical command information and the historical path information may be concatenated to obtain the command path information, and then at least one piece of command path information to be detected may be identified from the command path information based on a preset detection window. Alternatively, at least one system command log to be detected may be identified from the system command log based on a preset detection window, and then the historical command information and the corresponding historical path information in the system command log may be concatenated to obtain the command path information to be detected. Specific configurations may be made based on actual needs and are not limited here.

[0071] In step 103, feature extraction is performed on the path information of the command to be detected to obtain the path features of the command to be detected corresponding to the preset detection window.

[0072] The feature of the command path to be detected may be feature information representing information of the command path to be detected.

[0073] Among them, there can be multiple ways to perform feature extraction on the command path information to be detected and obtain the command path features to be detected corresponding to the preset detection window. For example, the command path information to be detected corresponding to the preset detection window can be spliced ​​to obtain the spliced ​​command path information, and multi-dimensional feature extraction can be performed on the spliced ​​command path information to obtain the command path features corresponding to each dimension. The command path features of each dimension can be feature fused to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0074] The spliced ​​command path information may be information obtained by splicing each piece of command path information to be detected corresponding to a preset detection window, that is, information obtained by splicing each piece of command path information to be detected processed in the preset detection window. The command path feature may be a feature of each dimension obtained by performing multi-dimensional feature extraction on the spliced ​​command path information, and is used to characterize the information of multiple dimensions in the spliced ​​command path information.

[0075] There are multiple ways to splice the command path information to be detected corresponding to the preset detection window. For example, each command path information to be detected in the preset detection window can be spliced ​​front to back to obtain spliced ​​command path information. Alternatively, each command path information to be detected in the preset detection window can be spliced ​​front to back in different orders to obtain multiple spliced ​​command path information.

[0076] After the command path information to be detected corresponding to the preset detection window is spliced, multi-dimensional feature extraction can be performed on the spliced ​​command path information to obtain the command path features corresponding to each dimension. There are many ways to perform multi-dimensional feature extraction on the spliced ​​command path information. For example, the spliced ​​command path information can be segmented to obtain command path words, and feature extraction can be performed on the command path words to obtain word features and position features. The spliced ​​command path information can be segmented to obtain at least one command path information segment, and feature extraction can be performed on the command path information segment to obtain log segment features. The word features, position features, and log segment features can be used as command log features corresponding to the spliced ​​command path information.

[0077] Among them, the command path word can be the result obtained after word segmentation processing of the spliced ​​command path information, and can include each word in the spliced ​​command path information. The word feature can be the feature information that characterizes the command path word, the position feature can be the feature information that characterizes the position of each word in the spliced ​​command path information, and the command path information segment can be the segment obtained after segmenting the spliced ​​command path information. For example, each spliced ​​command path information to be detected in the spliced ​​command path information can be segmented to obtain multiple command path information segments. Optionally, each command path information to be detected obtained after segmentation can also be segmented to obtain command information segments and path information segments corresponding to each command path information to be detected obtained after segmentation.

[0078] After the command path information to be detected corresponding to the preset detection window is spliced, the command path features of each dimension can be fused to obtain at least one command path feature to be detected corresponding to the preset detection window. There are multiple ways to fuse the command path features of each dimension. For example, feature extraction can be performed on the command path features to obtain the associated features corresponding to the command path features of each dimension. Based on the associated features, the associated weights corresponding to the command path features of each dimension are determined. According to the associated weights, the command path features of each dimension are fused to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0079] The correlation feature may be feature information representing the correlation between the command path feature of each dimension and the command path features of other dimensions, and the correlation weight may represent the importance of the command path feature of each dimension in the command path features.

[0080] Among them, feature extraction is performed on the command path feature to obtain the associated features corresponding to the command path features of each dimension. Based on the associated features, there can be multiple ways to determine the associated weights corresponding to the command path features of each dimension. For example, an attention network (Attention) can be used to extract features from the command path features of each dimension to obtain the associated features corresponding to the command path features of each dimension. For example, the command path features of each dimension can be converted into a three-dimensional spatial vector, including a query vector (query, abbreviated as q), a key vector (key, abbreviated as k) and a value vector (value, abbreviated as v). The specific conversion method can be understood as the fusion of the command path features of each dimension with the conversion parameters of the three dimensions, and the query vector, key vector and value vector are used as the associated features corresponding to the command path features of each dimension.

[0081] After extracting the features of the command path features of each dimension, the associated weight corresponding to the command path features of each dimension can be determined based on the associated features. There are many ways to determine the associated weight corresponding to the command path features of each dimension based on the associated features. For example, an attention network can be used to perform a dot product between the query vector corresponding to the command path features of each dimension and the key vectors of other command path features. The attention score corresponding to the command path features of each dimension can be obtained, and then the associated weight corresponding to the command path features of each dimension can be calculated based on the attention score corresponding to each visual feature.

[0082] Among them, in addition to using the attention network to extract features of the command path features of each dimension, other networks can also be used to capture the correlation between the command path features of each dimension and the command path features of other dimensions, and then determine the weight of the command path features of each dimension in the command path features of all dimensions.

[0083] In one embodiment, a trained log detection model can be used to detect system command logs. Specifically, the trained log detection model may include an encoder of a deep self-attention transformer network (Transformer), which extracts features of the command path information to be detected through the encoder of the Transformer to obtain features of the command path to be detected corresponding to the preset detection window. For example, please refer to Figure 3 , Figure 3This is a feature extraction diagram of a log detection method provided by an embodiment of the present application. Assuming that the predicted detection window includes the command path information to be detected (createFile, C: / ctfmon.exe) and (createProce, C: / Program Files / msedge.exe), the command path information to be detected is spliced ​​to obtain the spliced ​​command path information createFile-C: / ctfmon.exe and createProce-C: / Program Files / msedge.exe. Among them, createFile and createProce are historical command information, and the C: / ctfmon.exe and C: / Program Files / msedge.exe can be the corresponding historical path information. The spliced ​​command path information can be input into the encoding layer of the Transformer encoder, and the spliced ​​command path information is word-encoded by the encoding layer to obtain word features, and the spliced ​​command path information is position-encoded to obtain position features corresponding to the spliced ​​command path information. The spliced ​​command path information is command-encoded to obtain log fragment features corresponding to the spliced ​​command path information. Then, the multi-head attention mechanism (Multi-HeadAttention) in Transformer can be used to perform feature fusion on the command path features of each dimension. After the multi-head attention layer is used to perform feature fusion on the command path features of each dimension, the fully connected feedforward network layer (FeedForward) can be used to perform nonlinear transformation on the fused features, and then the residual connection and intra-layer regularization layer (Add&Norm) are used to perform residual connection and intra-layer regularization on the results after the nonlinear transformation to obtain at least one command path feature to be detected corresponding to the preset detection window. Optionally, in order to obtain more accurate features, a one-head multi-head attention mechanism can be used to perform feature fusion on the command path features of each dimension. In addition, the log detection model can be pre-trained using a bidirectional encoder representation (Bidirectional Encoder Representation from Transformers, abbreviated as BERT) based on a deep self-attention transform network to obtain a trained log detection module.

[0084] In step 104, the command path features to be detected are converted into feature dimensions to obtain command path features of a target dimension, and abnormal command logs are detected in the system command logs based on the command path features of the target dimension.

[0085] The target dimension command path feature may be a feature obtained by converting the target dimension command path feature to be detected. The target dimension may be a dimension of a pre-set low-dimensional feature space, wherein the number of dimensions of the low-dimensional feature space is smaller than the number of dimensions of the feature space in which the target dimension command path feature is currently located. That is, the target dimension command path feature may be a feature obtained by reducing the dimension of the target dimension command path feature. The abnormal command log may be an abnormal log in the system command log, which may cause an abnormal problem in the system when the system executes the command corresponding to the log.

[0086] Among them, there are many ways to convert the feature dimension of the command path features to be detected. For example, a linear transformation layer can be used to extract the features of the command path features to be detected, so as to transform the features of the command path to be detected in the current feature space into a low-dimensional feature space to obtain the command path features of the target dimension, so as to facilitate the abnormal classification of the command path information to be detected based on the low-dimensional target dimension command path features.

[0087] After converting the command path features to be detected into the target dimension and obtaining the command path features of the target dimension, abnormal command logs can be detected in the system command log based on the command path features of the target dimension. There are multiple ways to detect abnormal command logs in the system command log based on the command path features of the target dimension. For example, based on the command path features of the target dimension, the abnormality probability of the spliced ​​command path information can be calculated. Based on the abnormality probability, the target command path information can be filtered out from the spliced ​​command path information. Based on the target command path information, abnormal command logs can be identified in the system command log.

[0088] Among them, the abnormality probability can be the probability that the spliced ​​command path information is abnormal spliced ​​command path information, the abnormal spliced ​​command path information can be the spliced ​​command path information corresponding to the command that can cause the system to have an abnormality, and the target command path information can be the abnormal spliced ​​command path information in the spliced ​​command path information corresponding to the preset detection window.

[0089] Among them, there are many ways to calculate the abnormality probability of the spliced ​​command path information based on the command path characteristics of the target dimension. For example, a multilayer perceptron (MLP) can be used to calculate the abnormality probability of the command path information based on the command path characteristics of the target dimension.

[0090] After calculating the abnormal probability of the spliced ​​command path information based on the command path characteristics of the target dimension, the target command path information can be filtered out from the spliced ​​command path information according to the abnormal probability. There are multiple ways to filter out the target command path information from the spliced ​​command path information according to the abnormal probability. For example, an abnormal probability threshold can be obtained, and the abnormal probability of each spliced ​​command path information is compared with the abnormal probability threshold. According to the comparison result, the spliced ​​command path information corresponding to the abnormal probability greater than the abnormal probability threshold is determined as the target command path information. The abnormal probability threshold can be a critical value. When the abnormal probability of the command path information is greater than the critical value, the spliced ​​command path information can be determined to be the target command path information. When the abnormal probability of the spliced ​​command path information is not greater than the critical value, the spliced ​​command path information can be determined not to be the target command path information.

[0091] After the target command path information is screened out from the spliced ​​command path information based on the abnormality probability, abnormal command logs can be identified in the system command log based on the target command path information. There are multiple ways to identify abnormal command logs in the system command log based on the target command path information. For example, at least one command path information to be identified can be extracted from the target command path information, and the number of command path information to be identified can be counted in the target command path information to obtain the number of abnormal occurrences corresponding to each command path information to be identified. Based on the number of abnormal occurrences, abnormal command logs can be screened out from the system command log.

[0092] Among them, the command path information to be identified can be the command path information to be detected extracted from the target command path information, and the number of times the exception occurs can be the number of times the command path information to be identified is extracted from the target command path information, that is, the number of times the command path information to be identified exists in the target command path information.

[0093] Among them, based on the number of occurrences of the exception, there can be multiple ways to filter out abnormal command logs in the system command log. For example, the total number of occurrences of each command path information to be identified in the system command log can be counted, and based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, the command path information to be identified is scored to obtain the abnormality score corresponding to the command path information to be identified. According to the abnormality score, the abnormal command path information is filtered out in the command path information to be identified, and the system command log corresponding to the abnormal command path information is determined as the abnormal command log.

[0094] The total number of occurrences may be the number of times the command path information to be identified appears in the system command log, and the anomaly score may be a score obtained by scoring the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, and is used to indicate the likelihood that the system command log corresponding to the identified command path information is a log generated when an abnormality occurs in the system execution command. The abnormal command path information may be the command path information corresponding to the system command log generated when an abnormality occurs in the system execution command in the command path information to be identified.

[0095] There are various ways to score the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified. For example, the ratio between the number of abnormal occurrences and the total number of occurrences corresponding to the command path information to be identified can be calculated, and the abnormality score can be determined based on the ratio. For example, the ratio can be directly determined as the abnormality score corresponding to the command path information to be identified, or the ratio can be transformed to obtain the abnormality score corresponding to the command path information to be identified. The specific calculation method can be set according to actual needs and is not limited here.

[0096] After scoring the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, the abnormal command path information can be screened out in the command path information to be identified according to the abnormality score. Among them, there can be multiple ways to screen out abnormal command path information in the command path information to be identified according to the abnormality score. For example, an abnormality score threshold can be set to screen out the command path information to be identified whose abnormality score is greater than the abnormality score threshold to obtain abnormal command path information. The abnormality score threshold can be two critical values. When the abnormality score is greater than the critical value, the command path information to be identified corresponding to the abnormal score can be determined as abnormal command path information. When the abnormality score is not greater than the critical value, the command path information to be identified corresponding to the abnormal score can be determined as non-abnormal command path information.

[0097] In one embodiment, a trained log detection model can be used to perform anomaly detection on system command logs. Specifically, the log detection model can also include a discriminator network, which can include a linear transformation layer and an output activation layer. Optionally, the linear transformation layer can include three linear transformation layers, and the output activation layer can be a sigmoid function layer. The discriminator network can then be used to transform the feature dimension of the command path features to be detected to obtain the command path features of the target dimension. Based on the command path features of the target dimension, abnormal command logs can be detected in the system command log. An activation function can be used to classify the command path features of the target dimension to determine whether there is abnormal command path information in the command path information to be detected corresponding to each preset detection window. Thus, based on the abnormal command path information corresponding to the preset detection window, abnormal command logs can be identified in the system command log. This can improve the speed of the log detection model in predicting abnormal command logs. At the same time, the log detection model can learn the long-term dependencies between log sequence data, improve the accuracy of log detection, and further improve the efficiency of log detection.

[0098] There are many ways to train the log detection model to obtain the trained log detection model. For example, a binary cross entropy (BCE) loss function can be used to train the log detection model. The formula of the binary cross entropy loss function can be expressed as:

[0099]

[0100] Among them, Y t Is a real value, which can be used to indicate whether the system command log is an abnormal command log. To predict the probability value, the probability that the system command log is an abnormal command log can be predicted for the log detection module. n represents the number of training samples, ∑ represents the summation symbol, and log represents the logarithmic function symbol. The input data can be forward-passed until an error is generated in the output. Backpropagation can then be combined with the gradient descent method to calculate the gradient of the loss for all weights in the network of the log detection model. The gradient descent method is then used to update the weights to minimize the loss function. In addition, during the training process, the learning rate can be dynamically adjusted to dynamically change the learning rate at different optimization stages to obtain better training results. Optionally, the learning rate can be controlled by exponential decay. In the early stages of network training, the learning rate decreases rapidly. As training continues, the network approaches the convergence position, and the rate of decrease of the learning rate also decreases, so that the log detection model can achieve the best learning effect.

[0101] In one embodiment, the abnormal command log obtained by the log detection method provided in the embodiment of the present application can be used to predict whether the commands executed by the system in the future contain malicious behavior, that is, it can be predicted whether the command to be executed is an abnormal command, thereby improving the efficiency of system abnormality detection. Specifically, after detecting the abnormal command log in the system command log based on the command path characteristics of the target dimension, it is also possible to receive the command information to be detected, search the historical command information corresponding to the abnormal command log for the target command information that matches the command information to be detected, and when the target command information is found, block the execution of the command information to be detected.

[0102] The command information to be detected can be a command waiting to be executed that the system has received, and the target command information can be the command information that matches the command information to be detected in the historical command information corresponding to the abnormal command log. For example, it can be the same command information as the command information to be detected. In this way, when a command to be executed is received, the command information that matches the command information to be detected can be searched in the historical command information corresponding to the abnormal command log, so that it can be determined whether the command is an abnormal command based on the search result. When the target command information is found, the execution of the command information to be detected is shielded, that is, the command information to be detected is not executed. In this way, it can be accurately predicted whether the commands received in the future contain malicious behavior, thereby avoiding the situation where the system has abnormal problems due to the execution of the command, further ensuring the safe operation of the system and improving the efficiency of system abnormality detection.

[0103] As can be seen from the above, the embodiment of the present application obtains the system command log; splices the historical command information and the historical path information to obtain the command path information, and based on the preset detection window, identifies at least one command path information to be detected in the command path information; extracts features of the command path information to be detected to obtain the command path features to be detected corresponding to the preset detection window; converts the feature dimension of the command path features to be detected to obtain the command path features of the target dimension, and detects abnormal command logs in the system command log based on the command path features of the target dimension. In this way, the features of the system command log are characterized in a fine-grained manner by extracting the features of the command information and the corresponding path information, thereby detecting abnormal command logs in the system command log based on the extracted features, so as to accurately detect abnormal problems of the operating system whose logs are in the form of commands and paths. At the same time, the accuracy of log detection can be adjusted by using the preset detection window to extract the features of the command information and the corresponding path information, thereby improving the accuracy of log detection and thus improving the efficiency of log detection.

[0104] The method described in the above embodiment will be further described in detail below with examples.

[0105] In this embodiment, the log detection device will be specifically integrated into a computer device as an example for description. Specifically, the log detection method will be described by taking a server as the execution subject and the system command log as the operating system log as an example.

[0106] To better describe the embodiments of this application, please refer to Figure 4 , Figure 4 This is another flow chart of the log detection method provided in the embodiment of the present application. The specific process is as follows:

[0107] In step 201, the server obtains a system command log, combines the historical command information with the historical path information to obtain command path information, and identifies at least one command path information to be detected in the command path information based on a preset detection window.

[0108] Among them, the server can have multiple ways to identify at least one command path information to be detected in the command path information based on a preset detection window. For example, the server can obtain the detection parameters corresponding to the preset detection window and the attribute information corresponding to each command path information, and filter out at least one command path information to be detected in the command path information based on the detection parameters and the attribute information.

[0109] Among them, there are many ways for the server to filter out at least one command path information to be detected from the command path information based on the detection parameter and attribute information. For example, taking the attribute information as time information and the detection parameter as the number of command path information as an example, the server can sort the command path information according to the time information corresponding to each command path information to obtain the sorted command path information, so that at least one command path information with a ranking corresponding to the number of command path information can be determined as the command path information to be detected in the sorted command path information. For example. Assuming that the preset detection window can process 5 command path information at the same time, the preset detection window can be used to filter out the top 5 command path information in the sorted command path information to obtain the command path information to be detected.

[0110] Optionally, the server may concatenate historical command information and historical path information to obtain command path information, and then identify at least one piece of command path information to be detected from the command path information based on a preset detection window. Alternatively, the server may identify at least one system command log to be detected from the system command log based on a preset detection window, and then concatenate the historical command information and corresponding historical path information in the system command log to obtain the command path information to be detected. Specific configurations may be made based on actual needs and are not limited here.

[0111] In step 202, the server splices the command path information to be detected corresponding to the preset detection window to obtain the spliced ​​command path information, performs word segmentation processing on the spliced ​​command path information to obtain command path words, and performs feature extraction on the command path words to obtain word features and position features.

[0112] The server may splice the command path information to be detected corresponding to the preset detection window in various ways. For example, the server may splice each command path information to be detected in the preset detection window front to back to obtain spliced ​​command path information. Alternatively, the server may splice each command path information to be detected in the preset detection window front to back in different orders to obtain multiple spliced ​​command path information.

[0113] In step 203, the server segments the spliced ​​command path information to obtain at least one command path information segment, and performs feature extraction on the command path information segment to obtain log segment features, and uses the word features, position features, and log segment features as command log features corresponding to the spliced ​​command path information.

[0114] The command path information segments may be segments obtained by segmenting the spliced ​​command path information. For example, the server may segment each spliced ​​command path information to be detected in the spliced ​​command path information, thereby obtaining multiple command path information segments. Optionally, the server may further segment each segmented command path information to be detected to obtain command information segments and path information segments corresponding to each segmented command path information to be detected.

[0115] In step 204, the server performs feature extraction on the command path feature to obtain the associated features corresponding to the command path features of each dimension, determines the associated weights corresponding to the command path features of each dimension based on the associated features, and fuses the command path features of each dimension according to the associated weights to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0116] Among them, the server performs feature extraction on the command path feature to obtain the associated features corresponding to the command path feature of each dimension. Based on the associated features, there can be multiple ways to determine the associated weights corresponding to the command path features of each dimension. For example, the server can use an attention network to perform feature extraction on the command path feature of each dimension to obtain the associated features corresponding to the command path features of each dimension. For example, the command path features of each dimension can be converted into a three-dimensional spatial vector, including a query vector, a key vector and a value vector. The specific conversion method can be understood as the fusion of the command path features of each dimension with the conversion parameters of the three dimensions, and the query vector, key vector and value vector are used as the associated features corresponding to the command path features of each dimension.

[0117] After the server extracts the features of the command path features of each dimension, it can determine the associated weight corresponding to the command path features of each dimension based on the associated features. There are many ways for the server to determine the associated weight corresponding to the command path features of each dimension based on the associated features. For example, the server can use an attention network to perform a dot product between the query vector corresponding to the command path features of each dimension and the key vectors of other command path features, and obtain the attention score corresponding to the command path features of each dimension. Then, based on the attention score corresponding to each visual feature, the server can calculate the associated weight corresponding to the command path features of each dimension.

[0118] Among them, in addition to using the attention network to extract features of the command path features of each dimension, other networks can also be used to capture the correlation between the command path features of each dimension and the command path features of other dimensions, and then determine the weight of the command path features of each dimension in the command path features of all dimensions.

[0119] In one embodiment, the server can use a trained log detection model to detect the system command log. Specifically, the trained log detection model can include a Transformer encoder, which extracts features of the command path information to be detected through the Transformer encoder to obtain the command path features to be detected corresponding to the preset detection window. For example, please refer to Figure 3Assuming that the predicted detection window includes the command path information to be detected (createFile, C: / ctfmon.exe) and (createProce, C: / Program Files / msedge.exe), the command path information to be detected is spliced ​​to obtain the spliced ​​command path information createFile-C: / ctfmon.exe and createProce-C: / Program Files / msedge.exe. Among them, createFile and createProce can be historical command information, and C: / ctfmon.exe and C: / Program Files / msedge.exe can be the corresponding historical path information. The spliced ​​command path information can be input into the encoding layer of the Transformer encoder. The encoding layer performs word element encoding on the spliced ​​command path information to obtain word features, performs position encoding on the spliced ​​command path information to obtain position features corresponding to the spliced ​​command path information, and performs command encoding on the spliced ​​command path information to obtain log segment features corresponding to the spliced ​​command path information. Then, the multi-head attention mechanism (Multi-HeadAttention) in Transformer can be used to fuse the command path features of each dimension. After the multi-head attention layer is used to fuse the command path features of each dimension, the fully connected feedforward network layer (FeedForward) can be used to perform nonlinear transformation on the fused features, and then the residual connection and intra-layer regularization layer (Add&Norm) can be used to perform residual connection and intra-layer regularization on the results after the nonlinear transformation to obtain at least one command path feature to be detected corresponding to the preset detection window. Optionally, in order to obtain more accurate features, a one-head multi-head attention mechanism can be used to fuse the command path features of each dimension. In addition, BERT can be used to pre-train the log detection model to obtain a trained log detection module.

[0120] In step 205, the server converts the feature dimension of the command path feature to be detected to obtain the command path feature of the target dimension, calculates the abnormality probability of the spliced ​​command path information based on the command path feature of the target dimension, and filters out the target command path information from the spliced ​​command path information based on the abnormality probability.

[0121] Among them, there are many ways for the server to convert the feature dimensions of the command path features to be detected. For example, the server can use a linear transformation layer to extract the feature of the command path features to be detected, so as to transform the command path features to be detected in the current feature space into a low-dimensional feature space to obtain the command path features of the target dimension, so as to facilitate the abnormal classification of the command path information to be detected based on the low-dimensional target dimension command path features.

[0122] Among them, there are many ways for the server to calculate the abnormal probability of the spliced ​​command path information based on the command path characteristics of the target dimension. For example, the server can use a multi-layer perceptron to calculate the abnormal probability of the command path information based on the command path characteristics of the target dimension.

[0123] After the server calculates the abnormal probability of the spliced ​​command path information based on the command path characteristics of the target dimension, it can filter out the target command path information from the spliced ​​command path information based on the abnormal probability. There are many ways for the server to filter out the target command path information from the spliced ​​command path information based on the abnormal probability. For example, an abnormal probability threshold can be obtained, and the abnormal probability of each spliced ​​command path information is compared with the abnormal probability threshold. According to the comparison result, the spliced ​​command path information corresponding to the abnormal probability greater than the abnormal probability threshold is determined as the target command path information. The abnormal probability threshold can be a critical value. When the abnormal probability of the command path information is greater than the critical value, the spliced ​​command path information can be determined to be the target command path information. When the abnormal probability of the spliced ​​command path information is not greater than the critical value, the spliced ​​command path information can be determined not to be the target command path information.

[0124] In step 206, the server extracts at least one command path information to be identified from the target command path information, counts the number of command path information to be identified in the target command path information, obtains the number of abnormal occurrences corresponding to each command path information to be identified, and counts the total number of occurrences of each command path information to be identified in the system command log.

[0125] The total number of occurrences may be the number of times the command path information to be identified appears in the system command log, and the anomaly score may be a score obtained by scoring the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, and is used to indicate the likelihood that the system command log corresponding to the identified command path information is a log generated when an operating system executes a command abnormally. The abnormal command path information may be the command path information corresponding to the system command log generated when an operating system executes a command abnormally in the command path information to be identified.

[0126] In step 207, the server scores the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, and obtains an abnormality score corresponding to the command path information to be identified. According to the abnormality score, the server filters out the abnormal command path information in the command path information to be identified, and determines the system command log corresponding to the abnormal command path information as the abnormal command log.

[0127] Among them, the server can score the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified in various ways. For example, the server can calculate the ratio between the number of abnormal occurrences and the total number of occurrences corresponding to the command path information to be identified, and determine the abnormality score based on the ratio. For example, the server can directly determine the ratio as the abnormality score corresponding to the command path information to be identified, or it can perform a transformation based on the ratio to obtain the abnormality score corresponding to the command path information to be identified, etc. The specific calculation method can be set according to actual needs and is not limited here.

[0128] After the server scores the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified, it can filter out abnormal command path information in the command path information to be identified based on the abnormality score. There are many ways for the server to filter out abnormal command path information in the command path information to be identified based on the abnormality score. For example, the server can set an abnormality score threshold to filter out the command path information to be identified whose abnormality score is greater than the abnormality score threshold to obtain abnormal command path information. The abnormality score threshold can be two critical values. When the abnormality score is greater than the critical value, the command path information to be identified corresponding to the abnormal score can be determined as abnormal command path information. When the abnormality score is not greater than the critical value, the command path information to be identified corresponding to the abnormal score can be determined as non-abnormal command path information.

[0129] In one embodiment, the server can use a trained log detection model to perform anomaly detection on the system command log. Specifically, the log detection model can also include a discriminator network, which can include a linear transformation layer and an output activation layer. Optionally, the linear transformation layer can include three linear transformation layers, and the output activation layer can be a sigmoid function layer. The discriminator network can then be used to transform the feature dimension of the command path features to be detected to obtain the command path features of the target dimension. Based on the command path features of the target dimension, abnormal command logs can be detected in the system command log. The activation function can be used to classify the command path features of the target dimension to determine whether there is abnormal command path information in the command path information to be detected corresponding to each preset detection window. Thus, based on the abnormal command path information corresponding to the preset detection window, abnormal command logs can be identified in the system command log. This can improve the speed of the log detection model in predicting abnormal command logs. At the same time, the log detection model can learn the long-term dependencies between log sequence data, improve the accuracy of log detection, and further improve the efficiency of log detection.

[0130] In step 208, the server receives the command information to be detected, searches for target command information matching the command information to be detected in the historical command information corresponding to the abnormal command log, and blocks the execution of the command information to be detected when the target command information is found.

[0131] The command information to be detected can be a command waiting to be executed received by the operating system, and the target command information can be command information that matches the command information to be detected in the historical command information corresponding to the abnormal command log, for example, it can be the same command information as the command information to be detected. In this way, when a command to be executed is received, the command information that matches the command information to be detected can be searched in the historical command information corresponding to the abnormal command log, so that it can be determined whether the command is an abnormal command based on the search result. When the target command information is searched, the execution of the command information to be detected is shielded, that is, the command information to be detected is not executed. In this way, it can be accurately predicted whether the commands received in the future contain malicious behavior, thereby avoiding the situation where the operating system has abnormal problems due to the execution of the command, further ensuring the safe operation of the operating system, and improving the efficiency of system abnormality detection.

[0132] As can be seen from the above, the embodiment of the present application obtains the system command log through the server, splices the historical command information and the historical path information to obtain command path information, and identifies at least one command path information to be detected in the command path information based on a preset detection window; the server splices the command path information to be detected corresponding to the preset detection window to obtain spliced ​​command path information, performs word segmentation processing on the spliced ​​command path information to obtain command path words, and performs feature extraction on the command path words to obtain word features and position features; the server divides the spliced ​​command path information to obtain at least one command path information segment, and performs feature extraction on the command path information segment to obtain log segment features, and uses the word features, position features and log segment features as the command log features corresponding to the spliced ​​command path information; the server performs feature extraction on the command path features to obtain the associated features corresponding to the command path features of each dimension, and based on the associated features, determines the associated weights corresponding to the command path features of each dimension, and according to the associated weights, fuses the command path features of each dimension to obtain at least one command path feature to be detected corresponding to the preset detection window; the server The feature dimension of the command path to be detected is converted to obtain the command path feature of the target dimension. Based on the command path feature of the target dimension, the abnormal probability of the spliced ​​command path information is calculated. According to the abnormal probability, the target command path information is screened out from the spliced ​​command path information. The server extracts at least one command path information to be identified from the target command path information, counts the number of command path information to be identified in the target command path information, obtains the number of abnormal occurrences corresponding to each command path information to be identified, and counts the total number of occurrences of each command path information to be identified in the system command log. The server scores the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified to obtain the abnormal score corresponding to the command path information to be identified. According to the abnormal score, the abnormal command path information is screened out from the command path information to be identified, and the system command log corresponding to the abnormal command path information is determined as the abnormal command log. The server receives the command information to be detected, searches for the target command information matching the command information to be detected in the historical command information corresponding to the abnormal command log, and blocks the execution of the command information to be detected when the target command information is found. In this way, the features of the system command log are characterized in a fine-grained manner by extracting the features of the command information and the corresponding path information, so that abnormal command logs can be detected in the system command log based on the extracted features, so as to accurately detect abnormal problems of the operating system whose logs are in the form of commands and paths. At the same time, using a preset detection window to extract the features of the command information and the corresponding path information can adjust the accuracy of log detection, thereby improving the accuracy of log detection and thus improving the efficiency of log detection.At the same time, when receiving the command information to be detected, the command information matching the command information to be detected can be searched in the historical command information corresponding to the abnormal command log, so that it can be determined whether the command is an abnormal command based on the search result. When the target command information is searched, the execution of the command information to be detected is shielded, that is, the command information to be detected is not executed. In this way, it is possible to accurately predict whether the commands received in the future contain malicious behavior, thereby avoiding the situation where the system has abnormal problems due to the execution of the command, further ensuring the safe operation of the system and improving the efficiency of system abnormality detection.

[0133] In order to better implement the above method, an embodiment of the present invention further provides a log detection device, which can be integrated into a computer device, and the computer device can be a server.

[0134] For example, Figure 5 , which is a schematic diagram of the structure of a log detection device provided in an embodiment of the present application, the log detection device may include an acquisition unit 301, an identification unit 302, an extraction unit 303, and a detection unit 304, as follows:

[0135] An acquiring unit 301 is configured to acquire a system command log, where the system command log includes historical command information and historical path information corresponding to the historical command information.

[0136] an identification unit 302 configured to combine the historical command information and the historical path information to obtain command path information, and identify at least one piece of command path information to be detected in the command path information based on a preset detection window;

[0137] The extraction unit 303 is configured to extract features from the path information of the command to be detected to obtain features of the path of the command to be detected corresponding to the preset detection window;

[0138] The detection unit 304 is configured to perform feature dimension conversion on the command path feature to be detected to obtain a command path feature of a target dimension, and detect abnormal command logs in the system command logs based on the command path feature of the target dimension.

[0139] In one embodiment, the extraction unit 303 includes:

[0140] a splicing subunit, configured to splice the command path information to be detected corresponding to the preset detection window to obtain spliced ​​command path information;

[0141] A multi-dimensional feature extraction subunit is used to perform multi-dimensional feature extraction on the spliced ​​command path information to obtain the command path feature corresponding to each dimension;

[0142] The feature fusion subunit is used to perform feature fusion on the command path features of each dimension to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0143] In one embodiment, the multi-dimensional feature extraction subunit includes:

[0144] A word processing module is used to perform word segmentation processing on the spliced ​​command path information to obtain command path words, and perform feature extraction on the command path words to obtain word features and position features;

[0145] a fragment processing module, configured to segment the spliced ​​command path information to obtain at least one command path information fragment, and perform feature extraction on the command path information fragment to obtain log fragment features;

[0146] The assignment module is used to use the word feature, the position feature and the log segment feature as the command log feature corresponding to the spliced ​​command path information.

[0147] In one embodiment, the feature fusion subunit includes:

[0148] A correlation feature extraction module is used to extract features from the command path feature to obtain correlation features corresponding to the command path feature of each dimension, wherein the correlation features are feature information representing the correlation relationship between the command path feature of each dimension and the command path features of other dimensions;

[0149] An association weight determination module, configured to determine an association weight corresponding to the command path feature of each dimension based on the association feature;

[0150] The feature fusion module is used to fuse the command path features of each dimension according to the association weight to obtain at least one command path feature to be detected corresponding to the preset detection window.

[0151] In one embodiment, the detection unit 304 includes:

[0152] An abnormality probability calculation subunit, configured to calculate an abnormality probability of the spliced ​​command path information based on the command path characteristics of the target dimension;

[0153] a target command path information screening subunit, configured to screen target command path information from the spliced ​​command path information according to the abnormality probability;

[0154] The abnormal command log identification subunit is used to identify the abnormal command log in the system command log based on the target command path information.

[0155] In one embodiment, the abnormal command log identification subunit includes:

[0156] A command path information extraction module to be identified is used to extract at least one command path information to be identified from the target command path information;

[0157] An exception occurrence count module is used to count the number of command path information to be identified in the target command path information to obtain the exception occurrence count corresponding to each command path information to be identified;

[0158] The abnormal command log screening module is used to screen out abnormal command logs in the system command log based on the number of occurrences of the abnormality.

[0159] In one embodiment, the abnormal command log screening module includes:

[0160] The total number of times statistics submodule is used to count the total number of times each command path information to be identified appears in the system command log;

[0161] An anomaly score scoring submodule is used to score the command path information to be identified based on the total number of occurrences and the number of anomaly occurrences corresponding to the command path information to be identified, and obtain an anomaly score corresponding to the command path information to be identified;

[0162] The abnormal command log determining submodule is configured to filter out abnormal command path information from the command path information to be identified according to the abnormal score, and determine the system command log corresponding to the abnormal command path information as the abnormal command log.

[0163] In one embodiment, the log detection device includes:

[0164] A receiving unit, configured to receive command information to be detected;

[0165] A search unit, configured to search the historical command information corresponding to the abnormal command log for target command information that matches the command information to be detected;

[0166] The shielding unit is used to shield the execution of the command information to be detected when the target command information is searched.

[0167] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0168] As can be seen from the above, the embodiment of the present application obtains the system command log through the acquisition unit 301; the identification unit 302 splices the historical command information and the historical path information to obtain the command path information, and based on the preset detection window, identifies at least one command path information to be detected in the command path information; the extraction unit 303 extracts features of the command path information to be detected to obtain the command path features to be detected corresponding to the preset detection window; the detection unit 304 converts the feature dimension of the command path features to be detected to obtain the command path features of the target dimension, and detects abnormal command logs in the system command log based on the command path features of the target dimension. In this way, the features of the system command log are characterized in a fine-grained manner by extracting the features of the command information and the corresponding path information, so that the abnormal command log is detected in the system command log according to the extracted features, so as to accurately detect abnormal problems of the operating system whose logs are in the form of commands and paths. At the same time, the accuracy of the log detection can be adjusted by using the preset detection window to extract the features of the command information and the corresponding path information, thereby improving the accuracy of the log detection and thus improving the efficiency of the log detection.

[0169] The present application also provides a computer device, such as Figure 6 , which shows a schematic diagram of the structure of a computer device involved in an embodiment of the present application. The computer device may be a server. Specifically:

[0170] The computer device may include one or more processing core processors 401, one or more computer readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0171] Processor 401 is the control center of the computer device. It connects all components of the computer device using various interfaces and circuits. It executes software programs and / or modules stored in memory 402 and accesses data stored in memory 402 to perform various computer functions and process data. Optionally, processor 401 may include one or more processing cores. Preferably, processor 401 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 401.

[0172] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and log detection by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0173] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0174] The computer device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0175] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device will load the executable files corresponding to one or more application processes into the memory 402 according to the following instructions, and the processor 401 will run the application stored in the memory 402 to implement various functions as follows:

[0176] Obtain system command logs; concatenate historical command information and historical path information to obtain command path information, and based on a preset detection window, identify at least one command path information to be detected in the command path information; perform feature extraction on the command path information to be detected to obtain the command path features to be detected corresponding to the preset detection window; perform feature dimension conversion on the command path features to be detected to obtain the command path features of the target dimension, and detect abnormal command logs in the system command logs based on the command path features of the target dimension.

[0177] The specific implementation of each of the above operations can be found in the previous embodiments and will not be described in detail here. It should be noted that the computer device provided in the embodiment of the present application and the method for log detection in the above embodiment are of the same concept, and the specific implementation process is detailed in the above method embodiment and will not be described in detail here.

[0178] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0179] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any log detection method provided in an embodiment of the present application. For example, the instructions can execute the following steps:

[0180] Obtain system command logs; concatenate historical command information and historical path information to obtain command path information, and based on a preset detection window, identify at least one command path information to be detected in the command path information; perform feature extraction on the command path information to be detected to obtain the command path features to be detected corresponding to the preset detection window; perform feature dimension conversion on the command path features to be detected to obtain the command path features of the target dimension, and detect abnormal command logs in the system command logs based on the command path features of the target dimension.

[0181] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0182] Since the instructions stored in the computer-readable storage medium can execute the steps of any log detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any log detection method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0183] According to one aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations provided in the above embodiments.

[0184] The above is a detailed introduction to a log detection method, device, and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A log detection method, characterized in that: include: Obtaining a system command log, wherein the system command log includes historical command information and historical path information corresponding to the historical command information; splicing the historical command information and the historical path information to obtain command path information, and identifying at least one piece of command path information to be detected in the command path information based on a preset detection window; Extracting features of the command path information to be detected to obtain features of the command path to be detected corresponding to the preset detection window; The command path feature to be detected is converted into a feature dimension to obtain a command path feature of a target dimension, and abnormal command logs are detected in the system command log based on the command path feature of the target dimension.

2. The log detection method according to claim 1, wherein: The extracting features of the command path information to be detected to obtain features of the command path to be detected corresponding to the preset detection window includes: splicing the command path information to be detected corresponding to the preset detection window to obtain spliced ​​command path information; Performing multi-dimensional feature extraction on the spliced ​​command path information to obtain command path features corresponding to each dimension; The command path features of each dimension are subjected to feature fusion to obtain at least one command path feature to be detected corresponding to the preset detection window.

3. The log detection method according to claim 2, wherein: The multi-dimensional feature extraction is performed on the spliced ​​command path information to obtain the command path feature corresponding to each dimension, including: Performing word segmentation processing on the spliced ​​command path information to obtain command path words, and performing feature extraction on the command path words to obtain word features and position features; Segmenting the spliced ​​command path information to obtain at least one command path information segment, and performing feature extraction on the command path information segment to obtain log segment features; The word feature, position feature, and log segment feature are used as command log features corresponding to the spliced ​​command path information.

4. The log detection method according to claim 2, wherein: The step of fusing the command path features of each dimension to obtain at least one command path feature to be detected corresponding to the preset detection window includes: Performing feature extraction on the command path feature to obtain correlation features corresponding to the command path feature of each dimension, wherein the correlation features are feature information representing the correlation relationship between the command path feature of each dimension and the command path features of other dimensions; Determining, based on the association features, an association weight corresponding to the command path features of each dimension; The command path features of each dimension are fused according to the association weight to obtain at least one command path feature to be detected corresponding to the preset detection window.

5. The log detection method according to claim 2, wherein: The detecting an abnormal command log in the system command log according to the command path feature of the target dimension includes: Calculating an abnormality probability of the spliced ​​command path information based on the command path characteristics of the target dimension; Filtering target command path information from the spliced ​​command path information according to the abnormality probability; Based on the target command path information, an abnormal command log is identified in the system command log.

6. The log detection method according to claim 5, wherein: The identifying an abnormal command log in the system command log based on the target command path information includes: Extracting at least one command path information to be identified from the target command path information; Counting the number of command path information to be identified in the target command path information to obtain the number of abnormal occurrences corresponding to each piece of command path information to be identified; Based on the number of occurrences of the exception, abnormal command logs are screened out from the system command logs.

7. The log detection method according to claim 6, wherein: The filtering out abnormal command logs from the system command logs based on the number of occurrences of the abnormality includes: Counting the total number of occurrences of each command path information to be identified in the system command log; Scoring the command path information to be identified based on the total number of occurrences and the number of abnormal occurrences corresponding to the command path information to be identified to obtain an abnormality score corresponding to the command path information to be identified; According to the abnormality score, abnormal command path information is screened out from the command path information to be identified, and the system command log corresponding to the abnormal command path information is determined as the abnormal command log.

8. The log detection method according to claim 1, wherein: After detecting an abnormal command log in the system command log based on the command path feature of the target dimension, the method further includes: receiving command information to be detected; Searching the historical command information corresponding to the abnormal command log for target command information that matches the command information to be detected; When the target command information is found, the execution of the command information to be detected is shielded.

9. A log detection device, characterized in that: include: an acquiring unit, configured to acquire a system command log, wherein the system command log includes historical command information and historical path information corresponding to the historical command information; an identification unit, configured to combine the historical command information and the historical path information to obtain command path information, and identify at least one piece of command path information to be detected in the command path information based on a preset detection window; An extraction unit, configured to extract features from the path information of the command to be detected to obtain features of the path of the command to be detected corresponding to the preset detection window; The detection unit is used to convert the feature dimension of the command path feature to be detected to obtain the command path feature of the target dimension, and detect abnormal command logs in the system command log based on the command path feature of the target dimension.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor to execute the steps in the log detection method according to any one of claims 1 to 8.

11. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the log detection method according to any one of claims 1 to 8 when executing the computer program.

12. A computer program product, characterized in that The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps in the log detection method according to any one of claims 1 to 8 are implemented.

Images