A network attack identification method and device based on log data, a terminal device and a storage medium

Abstract

The application discloses a network attack identification method and device based on log data, a terminal device and a storage medium. First, the log data of the power grid system is converted into structured text, and the structured text is input into a log classification model, so that the log classification model automatically classifies the log data as normal access or abnormal access based on captured context information and calculated correlation between words. When it is determined that the access is normal, first prompt information is generated to represent that no network attack has occurred. When it is determined that the access is abnormal, second prompt information is generated to represent that a network attack has occurred. Furthermore, the type of the network attack can be identified based on a preset attack classification model. Therefore, the application can automatically identify whether a network attack exists in the power grid system and automatically classify the corresponding network attack type, thereby improving the processing efficiency of the log data and ensuring the reliability of the overall operation of the power grid system.

Description

Technical Field

[0001] This invention relates to the field of attack identification technology in power grid operation, and in particular to a network attack identification method, apparatus, terminal device and storage medium based on log data. Background Technology

[0002] Smart grid systems organically integrate advanced information technology with traditional power grid infrastructure, enabling timely prediction and detection of faults. However, the introduction of communication technologies exposes power grid systems to the risk of cyberattacks. Several types of cyberattacks commonly occur in power grid operation, including probe attacks, DoS attacks, and R2L attacks. All of these types of attacks can threaten the stable operation of the power grid, thus requiring timely and accurate detection and response.

[0003] However, traditional methods mainly rely on manual screening and judgment of power grid system event logs. For example, it requires manual review of each log entry to identify normal and abnormal access events based on experience, thereby determining whether a network attack has occurred. Given the massive volume of event logs in power grid systems, existing methods of reviewing and identifying them are inefficient and prone to omissions and misinterpretations, leading to a decline in the overall reliability of the power grid system. Summary of the Invention

[0004] This invention provides a network attack identification method, apparatus, terminal device, and storage medium based on log data. It utilizes a preset log classification model to classify normal or abnormal access, achieving automated processing of log data, improving processing efficiency, and effectively solving the problems of low efficiency and easy omissions and misinterpretations caused by relying on manual screening and judgment of event logs of power grid systems in the prior art.

[0005] An embodiment of the present invention provides a network attack identification method based on log data, comprising:

[0006] Obtain log data of the power grid system within a certain time period;

[0007] The unstructured event logs in the log data are converted into structured text, and then the structured text is input into a preset log classification model so that the log classification model can capture the context information in the structured text and calculate the correlation between each word in the structured text. Then, based on the context information and the correlation, the classification result corresponding to the structured text is generated; wherein, the classification result includes: normal access or abnormal access.

[0008] When the classification result is determined to be normal access, a first prompt message is generated to indicate that no network attack has occurred.

[0009] When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred;

[0010] The training of the preset log classification model includes:

[0011] The log classification model to be trained is iteratively trained by taking structured text samples corresponding to several log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

[0012] Preferably, the step of converting unstructured event logs in the log data into structured text includes:

[0013] For each unstructured event log in the log data, the unstructured event log data is segmented based on the space delimiter to generate a first set containing multiple elements; wherein, the elements are used to represent independent information fragments in the log data;

[0014] For each first set corresponding to each unstructured event log, a set of regular expressions containing several preset variables is matched with each element in the first set. Then, the variables in the regular expression set replace the elements in the first set that successfully match the variables, generating a second set.

[0015] Based on the length of each second set, second sets with the same length are grouped together to generate several log groups with different lengths;

[0016] The second set that has not been classified once is marked as the third set. The starting position of each element in the third set is compared with the starting position of the elements in each set in each log group. Then, the third set that has the same starting position of the elements as a log group is classified into the corresponding log group for a second time.

[0017] The third set that has not been reclassified is marked as the fourth set. Each fourth set is traversed one by one, and the Manhattan distance between the fourth set and each log group set is calculated. It is determined whether the largest Manhattan distance is greater than a preset distance threshold. If it is, the fourth set is reclassified three times into the log group corresponding to the largest Manhattan distance. Otherwise, the fourth set is divided into a new log group. When the fourth set has been traversed, several updated log groups are generated. Each updated log group corresponds to a log template.

[0018] For each updated log group, the elements or variables corresponding to each set in the updated log group are filled into the corresponding log template to generate the corresponding fill data;

[0019] Based on the populated data corresponding to each updated log group, a structured text corresponding to the log data is generated.

[0020] Preferably, the step of inputting the structured text into a preset log classification model, so that the log classification model captures the contextual information in the structured text and calculates the correlation between various words in the structured text, includes:

[0021] The structured text is converted into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid; wherein, the preset vocabulary includes multiple preset word vectors, and different preset word vectors correspond to different sequence numbers; the sentence includes several information fragments;

[0022] The integer sequence, ID sequence, and attention mask sequence are simultaneously input into a preset log classification model. The log classification model generates context information corresponding to the structured text based on the preset values ​​corresponding to each information fragment in the integer sequence and ID sequence, as well as the position of each information fragment in the text. Then, the log classification model performs validity identification on each preset word vector in the structured text based on the attention mask sequence, and calculates the relevance between each word based on the context information corresponding to the structured text and the validity of each preset word vector.

[0023] Preferably, the step of converting the structured text into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid includes:

[0024] Each element or variable in the structured text is treated as an information segment, and a first identifier and a second identifier are added to the beginning and end of the information segment, respectively; wherein the first identifier and the second identifier are different.

[0025] Based on the first identifier and the second identifier, each information fragment is mapped to a preset word vector in the preset vocabulary, and each information fragment is assigned a corresponding sequence number based on the sequence number of each preset word vector in the preset vocabulary.

[0026] The sequence numbers corresponding to each information fragment are concatenated sequentially to generate an integer sequence corresponding to the structured text, which is used to map the relationship between each information fragment and the preset vocabulary.

[0027] Each sentence in the structured text is sequentially encoded, and then the codes corresponding to each sentence are concatenated to generate an ID sequence for distinguishing the positions of different sentences.

[0028] The integer sequence and the ID sequence are respectively filled with preset values ​​until the lengths of the integer sequence and the ID sequence are both equal to the preset length value; wherein, the preset length value is: the length of the sequence extracted from the log data and consisting of integers;

[0029] The positions corresponding to each information segment in the integer sequence are assigned a value of 1, and the positions corresponding to each preset value in the integer sequence are assigned a value of 0, thereby generating an attention mask sequence for distinguishing whether information segments are valid; wherein, the segment corresponding to a position assigned a value of 1 is a valid segment, and the segment corresponding to a position assigned a value of 0 is an invalid segment.

[0030] Preferably, the second notification information further includes: the attack type corresponding to the network attack when a network attack occurs;

[0031] When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred, including:

[0032] When the classification result is determined to be an abnormal access, each unstructured event log in the log data is converted into a corresponding word vector set to generate a final word vector set list; wherein, the word vector set list contains several word vector sets, each word vector set contains several target word vectors, and each target word vector corresponds to each word in the unstructured event log;

[0033] The word vector set list is input into a preset attack classification model so that the attack classification model generates a corresponding feature vector based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vector;

[0034] Based on the type of network attack, generate a second notification message that characterizes a network attack that has occurred and includes the type of network attack.

[0035] The generation process of the preset attack classification model includes:

[0036] The attack classification model is iteratively trained by taking several log data points with abnormal access and the actual network attack type corresponding to each log data point with abnormal access as input, and the predicted network attack type of each log data point with abnormal access as output, until the model converges, thus generating the preset attack classification model.

[0037] Preferably, the step of converting each unstructured event log in the log data into a corresponding word vector set to generate a final word vector set list includes:

[0038] For each unstructured event log in the log data, the unstructured event log is segmented to generate a first word set containing several words;

[0039] For each first word set, after performing a stop word removal operation on each word in the first word set, a second word set is generated;

[0040] Remove duplicate words from each of the second word sets to generate several third word sets; there are no duplicate words among the third word sets.

[0041] For each third word set, each word in the third word set is mapped to a high-dimensional vector space to generate a word vector set containing several target word vectors; wherein each target word vector in the word vector set corresponds to a word in the third word set.

[0042] The individual word vector sets are aggregated to generate a list of word vector sets containing several word vector sets.

[0043] Preferably, the attack classification model includes: an input layer, a GRU layer, a CNN layer, and a fully connected layer;

[0044] The attack classification model generates corresponding feature vectors based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vectors, including:

[0045] The input layer of the attack classification model is used to receive a list of word vector sets;

[0046] The GRU layer of the attack classification model is used to capture the long-term dependencies between the target word vectors in the word vector set list.

[0047] The CNN layer of the attack classification model is used to extract local and global features from the word vector set list based on the long-term dependency relationship.

[0048] The fully connected layer of the attack classification model is used to perform nonlinear transformations on the local features and global features respectively to generate corresponding feature vectors, and then generate the network attack type corresponding to the log data based on the feature vectors.

[0049] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0050] One embodiment of the present invention provides a network attack identification device based on log data, including: a log data acquisition module, a classification result generation module, and a prompting module;

[0051] The log data acquisition module is used to acquire log data corresponding to the power grid system within a certain time period;

[0052] The classification result generation module is used to convert unstructured event logs in the log data into structured text, and then input the structured text into a preset log classification model so that the log classification model can capture the context information in the structured text and calculate the correlation between each word in the structured text. Then, based on the context information and the correlation, the module generates the classification result corresponding to the structured text. The classification result includes: normal access or abnormal access.

[0053] The prompting module is used to generate a first prompt message indicating that no network attack has occurred when the classification result is determined to be normal access.

[0054] When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred;

[0055] The training of the preset log classification model includes:

[0056] The log classification model to be trained is iteratively trained by taking structured text samples corresponding to several log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

[0057] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.

[0058] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the network attack identification method based on log data described in the above-described embodiment of the invention.

[0059] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0060] Another embodiment of the present invention provides a storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the network attack identification method based on log data described in the above-described embodiment of the invention.

[0061] The following benefits can be obtained by implementing the present invention:

[0062] This invention provides a method, apparatus, terminal device, and storage medium for identifying network attacks based on log data. The invention first acquires log data corresponding to a power grid system within a certain time period. Then, it converts the unstructured event logs in the log data into structured text, and inputs this structured text into a preset log classification model. The log classification model automatically classifies the structured text as normal or abnormal access. During the classification process, the log classification model can capture contextual information in the structured text and calculate the correlation between various words in the structured text, enabling the model to more accurately understand the log content and determine whether it is normal or abnormal access. Compared to manual judgment, the machine learning-based classification method of this invention is more objective and accurate. Finally, when the classification result is determined to be normal access, a first prompt message indicating that no network attack has occurred is generated; when the classification result is determined to be abnormal access, a second prompt message indicating that a network attack has occurred is generated. Therefore, this invention can not only identify network attacks in the power grid system in real time but also automatically generate corresponding classification results. When the classification result is abnormal access, it can also generate a second prompt message indicating that a network attack has occurred, helping maintenance personnel to take timely measures to deal with the attack. Compared with existing technologies, this invention can use a preset log classification model to classify normal or abnormal access, thereby automating the processing of log data, improving processing efficiency, reducing manual intervention, reducing the risk of omissions and misreading, and further ensuring the overall reliability of the power grid system. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating a network attack identification method based on log data provided in an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of a network attack type identification process provided in an embodiment of the present invention.

[0065] Figure 3 This is a schematic diagram of a network attack identification device based on log data provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] like Figure 1 The diagram shown is a flowchart illustrating a network attack identification method based on log data according to an embodiment of the present invention. The network attack identification method based on log data includes:

[0068] Step S1: Obtain the log data of the power grid system within a certain time period;

[0069] Step S2: Convert the unstructured event logs in the log data into structured text, and then input the structured text into a preset log classification model so that the log classification model can capture the context information in the structured text and calculate the correlation between each word in the structured text. Then, based on the context information and the correlation, generate the classification result corresponding to the structured text; wherein, the classification result includes: normal access or abnormal access.

[0070] The training of the preset log classification model includes:

[0071] The log classification model to be trained is iteratively trained by taking a number of structured text samples corresponding to log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

[0072] Step S3: When the classification result is determined to be normal access, generate a first prompt message to indicate that no network attack has occurred;

[0073] When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred.

[0074] In a preferred embodiment, for step S1, the present invention can identify normal or abnormal access based on the log data of the power grid system within a certain time period.

[0075] When acquiring event log data from the power grid system, the event log data can be cleaned and filtered. The cleaning operation mainly includes removing duplicate log entries and log data with incorrect formatting. The filtering operation mainly includes noise filtering, removing log text that is irrelevant to normal network operation or network attacks, and obtaining data documents containing only network access event logs.

[0076] Then, after converting the unstructured event logs in the log data document into structured text, the structured text is input into a preset log classification model so that the log classification model can automatically classify the structured text as normal or abnormal access.

[0077] For step S2, in a preferred embodiment, converting the unstructured event logs in the log data into structured text includes:

[0078] For each unstructured event log in the log data, the unstructured event log data is segmented based on the space delimiter to generate a first set containing multiple elements; wherein, the elements are used to represent independent information fragments in the log data;

[0079] For each first set corresponding to each unstructured event log, a set of regular expressions containing several preset variables is matched with each element in the first set. Then, the variables in the regular expression set replace the elements in the first set that successfully match the variables, generating a second set.

[0080] Based on the length of each second set, second sets with the same length are grouped together to generate several log groups with different lengths;

[0081] The second set that has not been classified once is marked as the third set. The starting position of each element in the third set is compared with the starting position of the elements in each set in each log group. Then, the third set that has the same starting position of the elements as a log group is classified into the corresponding log group for a second time.

[0082] The third set that has not been reclassified is marked as the fourth set. Each fourth set is traversed one by one, and the Manhattan distance between the fourth set and each log group set is calculated. It is determined whether the largest Manhattan distance is greater than a preset distance threshold. If it is, the fourth set is reclassified three times into the log group corresponding to the largest Manhattan distance. Otherwise, the fourth set is divided into a new log group. When the fourth set has been traversed, several updated log groups are generated. Each updated log group corresponds to a log template.

[0083] For each updated log group, the elements or variables corresponding to each set in the updated log group are filled into the corresponding log template to generate the corresponding fill data;

[0084] Based on the populated data corresponding to each updated log group, a structured text corresponding to the log data is generated.

[0085] Specifically, in a preferred embodiment, the present invention can divide the event log based on the space delimiter. i Convert to a token collection (tokens) i ; Match tokens using regular expressions i Simplification is performed by using a set of regular expressions for commonly used variables such as IP addresses to simplify the tokens. i Tokens that successfully match the regular expression set are deleted. Finally, all numeric tokens are replaced with placeholders "*", resulting in the simplified set tokens_new. i ;

[0086] Assuming logs representing the same event are more likely to have the same log length, therefore, based on tokens_new i The length is initially searched in the parse tree, looking for a match with tokens_new. i The next node that matches the length; if it does not exist, a new node is created with that length and used as the node that matched successfully.

[0087] If log start position tokens representing the same event are more likely to be the same, then use tokens_new. i The token at the starting position is searched a second time on the parse tree to find the next node that matches the token at the starting position. If it does not exist, a new node is created using the token at the starting position and it is used as the node that has successfully matched.

[0088] Finally, for the remaining nodes, use tokens_new i The final search is completed by calculating the distance between the tokens_new and the log group set of previously found nodes. This is then done using TF-IDF. i The log is converted into a set of word frequency vectors, and its Manhattan distance to the word frequency vector sets of each log group is calculated. If the distance is greater than a preset threshold, the log is removed from the list. i Add it to this group; otherwise, add tokens_new i Create a new log group.

[0089] Repeat the above classification operation for all logs in the data document to obtain a complete parse tree. The parse tree contains multiple log groups, each of which is a separate log template. After filling the template with the initial data of all logs in the data document, a structured log data document can be obtained. The structured data document is used as the structured text corresponding to the log data.

[0090] Furthermore, after obtaining the structured text, it can be converted into an integer sequence, an ID sequence, and an attention mask sequence.

[0091] Then, when the structured text is input into a preset log classification model, so that the log classification model can capture the contextual information in the structured text and calculate the correlation between the words in the structured text, the specific steps include:

[0092] First, the structured text is converted into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid; wherein, the preset vocabulary includes multiple preset word vectors, and different preset word vectors correspond to different sequence numbers; the sentence includes several information fragments;

[0093] Then, the integer sequence, ID sequence, and attention mask sequence are simultaneously input into a preset log classification model, so that the log classification model generates context information corresponding to the structured text based on the preset information corresponding to each information fragment in the integer sequence and ID sequence and the position of each information fragment in the text.

[0094] Finally, the log classification model identifies the validity of each preset word vector in the structured text based on the attention mask sequence. Based on the context information corresponding to the structured text and the validity of each preset word vector, it calculates the correlation between each word. Then, based on the context information and the correlation, it generates the classification result corresponding to the structured text, that is, identifies the classification result corresponding to the log data; for example, normal access or abnormal access.

[0095] Specifically, when converting the structured text into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid, the process is as follows:

[0096] Each element or variable in the structured text is treated as an information segment, and a first identifier and a second identifier are added to the beginning and end of the information segment, respectively; wherein the first identifier and the second identifier are different.

[0097] Based on the first identifier and the second identifier, each information fragment is mapped to a preset word vector in the preset vocabulary, and each information fragment is assigned a corresponding sequence number based on the sequence number of each preset word vector in the preset vocabulary.

[0098] The sequence numbers corresponding to each information fragment are concatenated sequentially to generate an integer sequence corresponding to the structured text, which is used to map the relationship between each information fragment and the preset vocabulary.

[0099] Each sentence in the structured text is sequentially encoded, and then the codes corresponding to each sentence are concatenated to generate an ID sequence for distinguishing the positions of different sentences.

[0100] The integer sequence and the ID sequence are respectively filled with preset values ​​until the lengths of the integer sequence and the ID sequence are both equal to the preset length value; wherein, the preset length value is: the length of the sequence extracted from the log data and consisting of integers;

[0101] The positions corresponding to each information segment in the integer sequence are assigned a value of 1, and the positions corresponding to each preset value in the integer sequence are assigned a value of 0, thereby generating an attention mask sequence for distinguishing whether information segments are valid; wherein, the segment corresponding to a position assigned a value of 1 is a valid segment, and the segment corresponding to a position assigned a value of 0 is an invalid segment.

[0102] Indicatively, this invention can convert structured logs in a data document into a token sequence. After adding [CLS] and [SEP] to the beginning and end of each sentence respectively, an integer sequence and an ID sequence are obtained sequentially. The sequences are then padded to add an attention mask sequence to distinguish the original and padded parts. The integer sequence, ID sequence, and attention mask sequence are then sequentially input into a trained log classification model to classify access types. In a preferred embodiment, the log classification model is the BERT model.

[0103] Specifically, using a structured log i As an example, this structured log i Each word, symbol, and number slice in the text is a token (which can be understood as an information fragment). [CLS] and [SEP] are added to the beginning and end of each sentence, respectively. Then, the BERT vocabulary is used to map the tokens to obtain the integer IDs corresponding to each token. After concatenation, the integer sequence corresponding to the log text is obtained.

[0104] Since the log contains multiple statements, each sentence can be encoded to distinguish sentence segments at different positions. For example, a sequence with the same length as the tokrn sequence after adding [SEP] and [CLS] to the j-th sentence and containing only the number j-1 can be used as its ID sequence. Thus, the ID sequences of all sentences in the log can be concatenated to obtain the ID sequence of the log.

[0105] Add zeros to the integer sequence until its length is g, which is illustrative and represents the longest initial integer sequence length of all logs. Similarly, add f to the ID sequence until its length is g, where f is the log number. i Total number of sentences in the text;

[0106] To distinguish the padding sequence from the initial sequence, prepare a sequence of length g, where the positions of valid tokens are set to 1 and the padding positions are set to 0, thus obtaining the final attention mask sequence.

[0107] Finally, the corresponding integer sequence, ID sequence, and attention mask sequence are obtained and input into the trained BERT model in sequence to obtain the category probability of each log. Logs with a probability less than the threshold p are marked as normal access logs, and logs with a probability not less than the threshold p are marked as abnormal access logs.

[0108] Therefore, embodiments of the present invention convert text into a sequence of integers, enabling text data to be processed and stored efficiently by a computer. Compared to the original text string, integers occupy less storage space and are calculated faster.

[0109] By combining ID sequences and integer sequences, and using attention masks, the model can capture contextual information in the text, enabling it to process long sequence data, capture complex language structures and semantic relationships, and provide more accurate access classification results.

[0110] For step S3, in a preferred embodiment, the present invention can distinguish the classification result to generate different prompt information. For example, when the classification result is determined to be normal access, a first prompt information is generated to indicate that no network attack has occurred; when the classification result is determined to be abnormal access, a second prompt information is generated to indicate that a network attack has occurred. Schematic, the second prompt information also includes: the attack type corresponding to the network attack when a network attack occurs.

[0111] In a preferred embodiment, when the classification result is determined to be abnormal access, the generation of a second prompt message to indicate that a network attack has occurred specifically includes:

[0112] When the classification result is determined to be an abnormal access, each unstructured event log in the log data is converted into a corresponding word vector set to generate a final word vector set list; wherein, the word vector set list contains several word vector sets, each word vector set contains several target word vectors, and each target word vector corresponds to each word in the unstructured event log;

[0113] The word vector set list is input into a preset attack classification model so that the attack classification model generates a corresponding feature vector based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vector;

[0114] Based on the type of network attack, generate a second notification message that characterizes a network attack that has occurred and includes the type of network attack.

[0115] The generation process of the preset attack classification model includes:

[0116] The attack classification model is iteratively trained by taking several log data points with abnormal access and the actual network attack type corresponding to each log data point with abnormal access as input, and the predicted network attack type of each log data point with abnormal access as output, until the model converges, thus generating the preset attack classification model.

[0117] Therefore, this invention can not only automatically identify whether there is a network attack in the power grid system, but also automatically classify the corresponding network attack type based on the attack classification model.

[0118] Specifically, the process of converting each unstructured event log in the log data into a corresponding word vector set to generate the final word vector set list includes:

[0119] For each unstructured event log in the log data, the unstructured event log is segmented to generate a first word set containing several words;

[0120] For each first word set, after performing a stop word removal operation on each word in the first word set, a second word set is generated;

[0121] Remove duplicate words from each of the second word sets to generate several third word sets; there are no duplicate words among the third word sets.

[0122] For each third word set, each word in the third word set is mapped to a high-dimensional vector space to generate a word vector set containing several target word vectors; wherein each target word vector in the word vector set corresponds to a word in the third word set.

[0123] The individual word vector sets are aggregated to generate a list of word vector sets containing several word vector sets.

[0124] Specifically, in this embodiment of the invention, before inputting log data containing abnormal access into a preset attack classification model, each unstructured event log in the log data is first converted into a list of word vector sets. This converts the log data into a numerical form that the model can understand and process. Therefore, converting unstructured event logs into word vector sets allows the model to better understand and utilize the semantic information in the logs, thereby improving the accuracy of attack classification.

[0125] Specifically, the log data exhibiting abnormal access is sequentially segmented, stop words are removed, and word vectors are trained to obtain the corresponding word vector set. First, the log data... i To perform word segmentation, input the log text into NLTK's `word_tokenize` method to obtain the word set {w i1 ,w i2 ,…,w im ,…,w in}; where w im It refers to l i The m-th word after word segmentation, where n is l i The total number of words contained;

[0126] Remove logs i Stop words in the segmented word set can be removed from the NLTK's built-in stop word list, resulting in a shortened word set {w}. i1 ,w i2 ,…,w iq}, q represents l after removing stop words i The number of words retained;

[0127] Repeat the stop word removal process to obtain a word set. Input the word set into the Word2vec model for CBOW training to obtain the word vectors {v1, v2, ..., v} for all words. z}, and assign each word vector {v1, v2, ..., v} to the following vectors: z Aggregate the data to obtain a list of word vector sets {V1, V2, ..., V} corresponding to the log data. N}, where N is the total number of unstructured event logs.

[0128] In a preferred embodiment, the attack classification model includes: an input layer, a GRU layer, a CNN layer, and a fully connected layer;

[0129] The attack classification model generates corresponding feature vectors based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vectors, including:

[0130] The input layer of the attack classification model is used to receive a list of word vector sets;

[0131] The GRU layer of the attack classification model is used to capture the long-term dependencies between the target word vectors in the word vector set list.

[0132] The CNN layer of the attack classification model is used to extract local and global features from the word vector set list based on the long-term dependency relationship.

[0133] The fully connected layer of the attack classification model is used to perform nonlinear transformations on the local and global features respectively to generate corresponding feature vectors, and then generate the network attack type corresponding to the log data based on the feature vectors. The fully connected layer also includes an output layer.

[0134] Specifically, the process of constructing an attack classification model based on a GRU-CNN neural network includes:

[0135] Add an input layer of size d_w×i, where d_w is the dimension of a single word vector and i is a list of word vector sets {V1, V2, ..., V}. N The maximum length of the word vector set in};

[0136] Add a GRU layer of size d_g. At each time step, the GRU unit receives the current input and the hidden state from the previous time step. The data passes through a reset gate R. t Update Gate Z t After processing, candidate hidden states are generated and the hidden states are updated. The new hidden states are then passed to the next time step. The Sigmoid activation function and the tanh activation function are used in the process.

[0137] Add a CNN layer and transform the output data of the GRU layer into The matrix is ​​designed to meet the input requirements of the CNN layer. The convolutional layer slides the convolutional kernel across the input data, calculating weighted sums at each position to capture important features. Let the kernel size of the model's convolutional layer be k×k, the number of channels be 1, and the number of kernels be c. The pooling layer performs downsampling to reduce the feature dimensionality, using max pooling. The pooling window size is set to s×s, with a stride equal to the window's length and width, reducing the data volume to a fraction of the previous amount.

[0138] Add a fully connected layer of size d_c×d_c using the ReLU activation function, where d_c is the height and width of the CNN layer output vector; add an output layer of size t using the Softmax activation function, where t is the total number of abnormal access event categories, to achieve a multi-class classification task. The model outputs the probability of belonging to each category, and the category with the highest probability is taken as the final output result.

[0139] The attack classification model is trained using cross-entropy as the loss function, and the model parameters are adjusted using gradient descent. The model is optimized through multiple iterations of training to obtain the optimal GRU-CNN classification model, which is the attack classification model.

[0140] Then, the word vectors in {V1,V2,…,V} can be used as input vectors and sequentially input into the trained attack classification model to obtain the probability that the log data belongs to each anomaly category. The category corresponding to the highest probability is taken as the network attack type corresponding to the log data.

[0141] Therefore, in this embodiment of the invention, unstructured event logs are converted into a list of word vector sets and input into an attack classification model based on a GRU-CNN neural network. Since the GRU layer can capture long-term dependencies and the CNN layer can extract local and global features, the combination of the two enables the model to better understand the log data, thus enabling the model to more accurately capture the semantic information and contextual relationships in the logs, thereby improving the accuracy of classification.

[0142] Furthermore, converting log data into a list of word vector sets significantly reduces the dimensionality of the data, while enabling the model to efficiently process this numerical data. In addition, the combination of GRU and CNN makes the model more efficient at processing long sequence data, thereby improving overall processing efficiency.

[0143] The output layer of the attack classification model uses the Softmax activation function, which can output the probability of log data belonging to each anomaly category and supports multi-classification tasks. Thus, the model can identify multiple network attack types at the same time, improving the model's practicality.

[0144] The entire classification process of this invention is highly automated. From the preprocessing of log data, the classification of normal or abnormal access, to the final output of the network attack type judgment result corresponding to the log data, no manual intervention is required, which improves work efficiency and makes the classification process more objective and reliable.

[0145] In a preferred embodiment, such as Figure 2The flowchart illustrating network attack type identification includes the following steps: First, data preprocessing is performed, including structured parsing of event logs, which converts the power grid system's log data into structured text. Then, normal access event logs are filtered to obtain abnormal access event logs, and the attack types of these abnormal access event logs are further identified. Word vectors are trained based on the log data, and a multi-classification model based on a GRU-CNN neural network is constructed to further classify and identify the attack types of the abnormal access event logs. This invention can automatically complete all processes of data preprocessing, model training, evaluation and optimization, and network event log identification, greatly improving the automation level of the process. Furthermore, the identification and application of the GRU-CNN neural network-based multi-classification model improves the accuracy of classification.

[0146] like Figure 3 As shown, based on the above embodiments of various network attack identification methods based on log data, the present invention provides corresponding device embodiments;

[0147] One embodiment of the present invention provides a network attack identification device based on log data, including: a log data acquisition module, a classification result generation module, and a prompting module;

[0148] The log data acquisition module is used to acquire log data corresponding to the power grid system within a certain time period;

[0149] The classification result generation module is used to convert unstructured event logs in the log data into structured text, and then input the structured text into a preset log classification model so that the log classification model can capture the context information in the structured text and calculate the correlation between each word in the structured text. Then, based on the context information and the correlation, the module generates the classification result corresponding to the structured text. The classification result includes: normal access or abnormal access.

[0150] The prompting module is used to generate a first prompt message indicating that no network attack has occurred when the classification result is determined to be normal access.

[0151] When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred;

[0152] The training of the preset log classification model includes:

[0153] The log classification model to be trained is iteratively trained by taking structured text samples corresponding to several log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

[0154] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0155] Those skilled in the art will clearly understand that, for convenience and simplicity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0156] Based on the above embodiments of various network attack identification methods based on log data, the present invention provides corresponding embodiments for terminal devices.

[0157] One embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a network attack identification method based on log data as described in any embodiment of the present invention.

[0158] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0159] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0160] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart memory card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device or other volatile solid-state storage device.

[0161] Based on the above embodiments of various network attack identification methods based on log data, the present invention provides corresponding embodiments for storage media.

[0162] One embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a network attack identification method based on log data as described in any embodiment of the present invention.

[0163] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0164] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A network attack identification method based on log data, characterized in that, include: Obtain log data of the power grid system within a certain time period; For each unstructured event log in the log data, the unstructured event log data is segmented based on the space delimiter to generate a first set containing multiple elements; wherein, the elements are used to represent independent information fragments in the log data; For each first set corresponding to each unstructured event log, a set of regular expressions containing several preset variables is matched with each element in the first set. Then, the variables in the regular expression set replace the elements in the first set that successfully match the variables, generating a second set. Based on the length of each second set, second sets with the same length are grouped together to generate several log groups with different lengths; The second set that has not been classified once is marked as the third set. The starting position of each element in the third set is compared with the starting position of the elements in each set in each log group. Then, the third set that has the same starting position of the elements as a log group is classified into the corresponding log group for a second time. The third set that has not been reclassified is marked as the fourth set. Each fourth set is traversed one by one, and the Manhattan distance between the fourth set and each log group set is calculated. It is determined whether the largest Manhattan distance is greater than a preset distance threshold. If it is, the fourth set is reclassified three times into the log group corresponding to the largest Manhattan distance. Otherwise, the fourth set is divided into a new log group. When the fourth set has been traversed, several updated log groups are generated. Each updated log group corresponds to a log template. For each updated log group, the elements or variables corresponding to each set in the updated log group are filled into the corresponding log template to generate the corresponding fill data; Based on the populated data corresponding to each updated log group, generate the structured text corresponding to the log data; The structured text is input into a preset log classification model so that the log classification model can capture the contextual information in the structured text and calculate the correlation between each word in the structured text. Then, based on the contextual information and the correlation, the model generates a classification result corresponding to the structured text. The classification result includes: normal access or abnormal access. When the classification result is determined to be normal access, a first prompt message is generated to indicate that no network attack has occurred. When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred; The training of the preset log classification model includes: The log classification model to be trained is iteratively trained by taking structured text samples corresponding to several log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

2. The network attack identification method based on log data as described in claim 1, characterized in that, The step of inputting the structured text into a preset log classification model, so that the log classification model can capture the contextual information in the structured text and calculate the correlation between the words in the structured text, includes: The structured text is converted into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid; wherein, the preset vocabulary includes multiple preset word vectors, and different preset word vectors correspond to different sequence numbers; the sentence includes several information fragments; The integer sequence, ID sequence, and attention mask sequence are simultaneously input into a preset log classification model. The log classification model generates context information corresponding to the structured text based on the preset values ​​corresponding to each information fragment in the integer sequence and ID sequence, as well as the position of each information fragment in the text. Then, the log classification model performs validity identification on each preset word vector in the structured text based on the attention mask sequence, and calculates the relevance between each word based on the context information corresponding to the structured text and the validity of each preset word vector.

3. The network attack identification method based on log data as described in claim 2, characterized in that, The process of converting the structured text into an integer sequence for mapping the relationship between each information fragment and a preset vocabulary, an ID sequence for distinguishing the positions of different sentences, and an attention mask sequence for distinguishing whether an information fragment is valid includes: Each element or variable in the structured text is treated as an information segment, and a first identifier and a second identifier are added to the beginning and end of the information segment, respectively; wherein the first identifier and the second identifier are different. Based on the first identifier and the second identifier, each information fragment is mapped to a preset word vector in the preset vocabulary, and each information fragment is assigned a corresponding sequence number based on the sequence number of each preset word vector in the preset vocabulary. The sequence numbers corresponding to each information fragment are concatenated sequentially to generate an integer sequence corresponding to the structured text, which is used to map the relationship between each information fragment and the preset vocabulary. Each sentence in the structured text is sequentially encoded, and then the codes corresponding to each sentence are concatenated to generate an ID sequence for distinguishing the positions of different sentences. The integer sequence and the ID sequence are respectively filled with preset values ​​until the lengths of the integer sequence and the ID sequence are both equal to the preset length value; wherein, the preset length value is: the length of the sequence extracted from the log data and consisting of integers; The positions corresponding to each information segment in the integer sequence are assigned a value of 1, and the positions corresponding to each preset value in the integer sequence are assigned a value of 0, thereby generating an attention mask sequence for distinguishing whether information segments are valid; wherein, the segment corresponding to a position assigned a value of 1 is a valid segment, and the segment corresponding to a position assigned a value of 0 is an invalid segment.

4. The network attack identification method based on log data as described in claim 3, characterized in that, The second notification information also includes: the attack type corresponding to the network attack when the network attack occurs; When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred, including: When the classification result is determined to be an abnormal access, each unstructured event log in the log data is converted into a corresponding word vector set to generate a final word vector set list; wherein, the word vector set list contains several word vector sets, each word vector set contains several target word vectors, and each target word vector corresponds to each word in the unstructured event log; The word vector set list is input into a preset attack classification model so that the attack classification model generates a corresponding feature vector based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vector; Based on the type of network attack, generate a second notification message that characterizes a network attack that has occurred and includes the type of network attack. The generation process of the preset attack classification model includes: The attack classification model is iteratively trained by taking several log data points with abnormal access and the actual network attack type corresponding to each log data point with abnormal access as input, and the predicted network attack type of each log data point with abnormal access as output, until the model converges, thus generating the preset attack classification model.

5. The network attack identification method based on log data as described in claim 4, characterized in that, The step of converting each unstructured event log in the log data into a corresponding word vector set to generate a final word vector set list includes: For each unstructured event log in the log data, the unstructured event log is segmented to generate a first word set containing several words; For each first word set, after performing a stop word removal operation on each word in the first word set, a second word set is generated; Remove duplicate words from each of the second word sets to generate several third word sets; there are no duplicate words among the third word sets. For each third word set, each word in the third word set is mapped to a high-dimensional vector space to generate a word vector set containing several target word vectors; wherein each target word vector in the word vector set corresponds to a word in the third word set. The individual word vector sets are aggregated to generate a list of word vector sets containing several word vector sets.

6. The network attack identification method based on log data as described in claim 5, characterized in that, The attack classification model includes: an input layer, a GRU layer, a CNN layer, and a fully connected layer; The attack classification model generates corresponding feature vectors based on each target word vector in the word vector set list, and then generates the network attack type corresponding to the log data based on the feature vectors, including: The input layer of the attack classification model is used to receive a list of word vector sets; The GRU layer of the attack classification model is used to capture the long-term dependencies between the target word vectors in the word vector set list. The CNN layer of the attack classification model is used to extract local and global features from the word vector set list based on the long-term dependency relationship. The fully connected layer of the attack classification model is used to perform nonlinear transformations on the local features and global features respectively to generate corresponding feature vectors, and then generate the network attack type corresponding to the log data based on the feature vectors.

7. A network attack identification device based on log data, characterized in that, include: Log data acquisition module, classification result generation module, and prompt module; The log data acquisition module is used to acquire log data corresponding to the power grid system within a certain time period; The classification result generation module is used to convert unstructured event logs in the log data into structured text, and then input the structured text into a preset log classification model so that the log classification model can capture the context information in the structured text and calculate the correlation between each word in the structured text. Then, based on the context information and the correlation, the module generates the classification result corresponding to the structured text. The classification result includes: normal access or abnormal access. The prompting module is used to generate a first prompt message indicating that no network attack has occurred when the classification result is determined to be normal access. When the classification result is determined to be an abnormal access, a second prompt message is generated to indicate that a network attack has occurred; The training of the preset log classification model includes: The log classification model to be trained is iteratively trained by taking structured text samples corresponding to several log data and the actual classification result corresponding to each structured text sample as input, and the predicted classification result of each structured text sample as output, until the model converges and the preset log classification model is generated.

8. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a network attack identification method based on log data as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform a network attack identification method based on log data as described in any one of claims 1 to 6.

Images