Security audit method, device, electronic device and computer readable medium
By collecting and preprocessing a variety of log types and generating security incident warning information in combination with the security audit model, the security risk problem of incomplete audit rules is solved and efficient and accurate security audit is achieved.
Patent Information
- Application Number
- CN202410546609.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-06
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-05-06
AI Technical Summary
When conducting security audits, a large number of audit rules need to be set up in advance, resulting in error audits that are prone to abnormal behavior when the audit rules are not comprehensive, which increases security risks.
Collect target log collections, including network traffic log types, system operation log types, and application operation log types, perform log preprocessing and feature extraction, and generate security event warning information in combination with the pre-trained security audit model, without setting up complex audit rules .
By automatically generating security incident warning information, security risks are reduced, audit efficiency and accuracy are improved, and false audits are avoided due to incomplete rules.
Smart Images

Figure CN118473726B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a security audit method, device, electronic device, and computer-readable medium. Background Art
[0002] With the development of Internet-related technologies, they are widely integrated into people's production and life. Abnormal behaviors (such as traffic attack behaviors, program attack behaviors based on program vulnerabilities, etc.) pose a huge threat to the operation of Internet-based networked devices. At present, when conducting security audits, the commonly used method is to conduct behavioral audits by setting audit rules.
[0003] However, when the above method is adopted, the following technical problems often occur:
[0004] When conducting behavioral audits, it is often necessary to set a large number of audit rules in advance. When the audit rules are not comprehensive, it is very easy to make erroneous audits of abnormal behaviors, thereby increasing security risks.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose security audit methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a security audit method, the method comprising: collecting a target log set, wherein the log types of the target logs in the target log set include: a network traffic log type, a system operation record log type, and an application operation log type; for each target log in the target log set, executing the following security event warning information generation steps: performing log preprocessing on the target log to generate a preprocessed log record group; in response to the log type corresponding to the target log being a system operation record log type or an application operation log type, performing log record feature extraction on each preprocessed log record in the preprocessed log record group to generate a log record Features, obtain a log record feature group; generate a security event warning information set for the target log according to the pre-trained security audit model and the log record feature group, wherein the security event warning information in the security event warning information set includes: log record identification, security event level and security event description information; in response to the log type corresponding to the target log being a network traffic log type, generate a network traffic flow graph for the target log according to the pre-processed log record group; perform graph feature extraction on the network traffic flow graph to generate network traffic flow graph features; generate a security event warning information set for the target log according to the security audit model and the network traffic flow graph features.
[0009] In a second aspect, some embodiments of the present disclosure provide a security audit device, which includes: a collection unit, configured to collect a target log set, wherein the log types of the target logs in the above target log set include: a network traffic log type, a system operation record log type, and an application running log type; an execution unit, configured to, for each target log in the above target log set, perform the following security event warning information generation steps: perform log preprocessing on the above target log to generate a preprocessed log record group; in response to the log type corresponding to the above target log being a system operation record log type or an application running log type, perform log record feature extraction on each preprocessed log record in the preprocessed log record group; Take, to generate log record features, to obtain a log record feature group; based on the pre-trained security audit model and the above-mentioned log record feature group, generate a security event warning information set for the above-mentioned target log, wherein the security event warning information in the above-mentioned security event warning information set includes: log record identification, security event level and security event description information; in response to the log type corresponding to the above-mentioned target log being a network traffic log type, based on the above-mentioned preprocessed log record group, generate a network traffic flow graph for the above-mentioned target log; perform graph feature extraction on the above-mentioned network traffic flow graph to generate network traffic flow graph features; based on the above-mentioned security audit model and the network traffic flow graph features, generate a security event warning information set for the above-mentioned target log.
[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.
[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.
[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the security audit method of some embodiments of the present disclosure, the security risk is reduced. Specifically, the reason for the high security risk is that when conducting a behavior audit, a large number of audit rules often need to be set in advance. When the audit rules are not comprehensive, it is very easy to have an erroneous audit of abnormal behavior, thereby increasing the security risk. Based on this, the security audit method of some embodiments of the present disclosure, first, collects a target log set, wherein the log type of the target log in the above target log set includes: a network traffic log type, a system operation record log type, and an application running log type. Secondly, for each target log in the above target log set, the following security event warning information generation step is performed: the first step is to perform log preprocessing on the above target log to generate a preprocessed log record group. In practice, there may be log records in the form of dirty data during the log collection process, so it is necessary to perform log preprocessing on the target log. In the second step, in response to the log type corresponding to the above target log being a system operation record log type or an application running log type, log record feature extraction is performed on each preprocessed log record in the above preprocessed log record group to generate a log record feature and obtain a log record feature group. For the target logs of the system operation record log type and the application operation log type, since there are complete operation steps or operation steps corresponding to them, the present disclosure performs feature extraction with log records as the granularity. The third step is to generate a security event warning information set for the target log according to the pre-trained security audit model and the above-mentioned log record feature group, wherein the security event warning information in the above-mentioned security event warning information set includes: log record identification, security event level and security event description information. By combining the security audit model, security event warning information for log records is automatically generated. The fourth step is to respond to the log type corresponding to the above-mentioned target log as a network traffic log type, and generate a network traffic flow graph for the above-mentioned target log according to the above-mentioned pre-processed log record group. The fifth step is to perform graph feature extraction on the above-mentioned network traffic flow graph to generate network traffic flow graph features. The target log of the network traffic log type corresponds to a relatively obvious traffic flow feature, so the present disclosure performs feature extraction in the form of a graph. The sixth step is to generate a security event warning information set for the above-mentioned target log according to the above-mentioned security audit model and the network traffic flow graph features. This method eliminates the need to set complex audit rules, while ensuring the efficiency and accuracy of security audits and reducing security risks. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0014] Figure 1 is a flow chart of some embodiments of the security audit method according to the present disclosure;
[0015] Figure 2 is a schematic diagram of the structure of some embodiments of the security audit device according to the present disclosure;
[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0023] refer to Figure 1, shows a process 100 of some embodiments of the security audit method according to the present disclosure. The security audit method comprises the following steps:
[0024] Step 101: Collect a target log set.
[0025] In some embodiments, the execution subject (e.g., computing device) of the security audit method can collect the target log set by wired connection or wireless connection. Among them, the log types of the target logs in the target log set include: network traffic log type, system operation record log type and application operation log type. Among them, the target log of the network traffic log type can be a log for monitoring network traffic. The target log of the system operation record log type can be a log for recording operation records for the operating system. The target log of the application operation log type can be a log for recording the running state of the application during operation. In practice, for the target log of the network traffic log type, the above-mentioned execution subject can collect the target log of the network traffic log type from the Nginx server. Specifically, the Nginx server can be set between the client and the server end for providing services to forward requests, so the above-mentioned execution subject can directly collect the target log of the network traffic log type through the Nginx server. For the target log of the system operation record log type, the above-mentioned execution subject can directly pull the target log of the system operation record log type from the server running the operating system. For the target log of the application running log type, the above-mentioned execution subject can directly pull the target log of the application running log type from the server and / or client running the application. Specifically, since the log sources of target logs of different log types are different, the above-mentioned execution subject can adopt an asynchronous collection method to collect the above-mentioned target log set in real time. For the target log of the application running log type, before collecting, the present disclosure has fulfilled the obligations of informing the subject of personal information and obtaining the authorization and consent of the subject of personal information in advance.
[0026] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future.
[0027] It should be noted that the above-mentioned computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here. For example, when the above-mentioned execution subject is software, it can be a comprehensive log audit platform. It should be understood that the number of computing devices can be any number according to the implementation needs.
[0028] Step 102: for each target log in the target log set, perform the following security event warning information generation steps:
[0029] Step 1021: perform log preprocessing on the target log to generate a preprocessed log record group.
[0030] In some embodiments, the above-mentioned execution subject may perform log preprocessing on the target log to generate a preprocessed log record group. Among them, the preprocessed log record is a log record in the target log that has undergone log preprocessing. In practice, multiple log records are stored in the target log. For example, for a target log of a network traffic log type, the log record may be a record that records network requests. For another example, for a target log of a system operation log type, the log record may be a record that records system operation steps. For another example, for a target log of an application running log type, the log record may be a record that records the running status of the application. Specifically, the above-mentioned execution subject may perform data cleaning on the target log by combining the Hadoop or Spark big data processing framework to generate a preprocessed log record group.
[0031] In some optional implementations of some embodiments, the execution subject performs log preprocessing on the target log to generate a preprocessed log record group, which may include the following steps:
[0032] The first step is to parse the target logs to generate an initial log record group.
[0033] In practice, the above-mentioned execution subject can read the log records in the target log one by one through streaming reading to obtain an initial log record group. Among them, the initial log record in the initial log record group includes a set of record items. The record item represents the field in the initial log record. The record item corresponds to a record item default identifier. The record item default identifier indicates whether the corresponding record item is allowed to be empty. Specifically, the initial log record often contains multiple record items, and different record items correspond to different data requirements. For example, some record items are required to be filled in, that is, they are not allowed to be empty.
[0034] In the second step, for each initial log record in the above initial log record group, perform the following filtering steps:
[0035] The first sub-step is to determine the logical OR result of the record item state value corresponding to each record item in the record item set included in the initial log record and the record item default identifier corresponding to the record item, and obtain a logical OR result sequence.
[0036] The record item status value indicates whether the record item is empty.
[0037] In practice, the above-mentioned execution subject can determine the record item status value corresponding to the record item through the isNotEmpty() function. Among them, the isNotEmpty() function is used to determine whether the record item is empty, and the return value of the isNotEmpty() function, that is, the record item status value, is of Boolean type. Specifically, when the record item status value is True, the record item default mark is True, and the corresponding logical or result is True. When the record item status value is True, the record item default mark is False, and the corresponding logical or result is True. When the record item status value is False, the record item default mark is False, and the corresponding logical or result is False. The record item status value is True, indicating that the record item is not empty. The record item status value is False, indicating that the record item is empty. The record item default mark is True, indicating that the record item is allowed to be empty. The record item default mark is False, indicating that the record item is not allowed to be empty.
[0038] The second sub-step is to perform a progressive logical AND operation on the logic or results in the above logic or result sequence to obtain a logical AND result.
[0039] Among them, when there is a False logical OR result in the logical OR result sequence, the obtained logical AND result is False. When all logical OR results in the logical OR result sequence are True, the obtained logical AND result is True.
[0040] The third sub-step is, in response to the logical AND result being 0, filtering the initial log record from the initial log record group.
[0041] The third step is to format each filtered log record in the obtained filtered log record group to generate a preprocessed log record, thereby obtaining the above-mentioned preprocessed log record set.
[0042] In practice, the execution entity may perform JSON formatting on each filtered log record in the obtained filtered log record group to generate a preprocessed log record, thereby obtaining the preprocessed log record set.
[0043] Step 1022, in response to the log type corresponding to the target log being a system operation record log type or an application running log type, log record feature extraction is performed on each preprocessed log record in the preprocessed log record group to generate a log record feature to obtain a log record feature group.
[0044] In some embodiments, in response to the log type corresponding to the target log being a system operation record log type or an application operation log type, the above-mentioned execution subject may perform log record feature extraction on each preprocessed log record in the preprocessed log record group to generate log record features and obtain a log record feature group. In practice, for the target log of the system operation record log type or the application operation log type, the stored log records often correspond to independent log contents, so the above-mentioned execution subject may perform log record feature extraction with the preprocessed log record as the granularity. In practice, the above-mentioned execution subject may perform log record feature extraction on the preprocessed log record through the Word2Vec model to generate log record features.
[0045] In some optional implementations of some embodiments, the execution subject extracts log record features from each preprocessed log record in the preprocessed log record group to generate log record features, which may include the following steps:
[0046] In the first step, for each record item in the record item set included in the preprocessed log record, the following initial encoding steps are performed:
[0047] In a first sub-step, in response to the record item type of the record item being a numerical type, binary encoding is performed on the record item to obtain an encoded record item corresponding to the record item.
[0048] In practice, the execution subject may convert the record item into a binary representation to obtain an encoded record item corresponding to the record item.
[0049] In a second sub-step, in response to the record item type of the record item being a fixed-length character type, one-hot encoding is performed on the record item to obtain an encoded record item corresponding to the record item.
[0050] In practice, the fixed-length character type may be a record item with a pre-set character length. For example, the data type of a record item may be char
[20] , which indicates that the maximum character length of the record item is 20. Therefore, the corresponding record item type is a fixed-length character type. Since the record item type is a fixed-length character type, the length of the encoded record item is fixed.
[0051] In a third sub-step, in response to the record item type of the record item being a variable-length character type, word embedding processing is performed on the record item to obtain an encoded record item corresponding to the record item.
[0052] In practice, the character length of the variable-length character type representation record item is not fixed. For example, the data type of the record item can be a String type. Since the record item type is a variable-length character type, the use of one-hot encoding will result in a longer character length of the encoded record item. Therefore, the word embedding processing method is used to control the length of the encoded record item. Specifically, the above-mentioned execution subject can perform word embedding processing on the above-mentioned record item through the Word2Vec model to obtain the encoded record item corresponding to the above-mentioned record item.
[0053] The second step is to concatenate the encoded record items in the obtained encoded record item sequence to generate a concatenated record item.
[0054] As an example, the encoded record item sequence may include: encoded record item A, encoded record item B, and encoded record item C. Therefore, the above execution entity may concatenate the encoded record item A, the encoded record item B, and the encoded record item C, and the concatenated record item obtained is "encoded record item A+encoded record item B+encoded record item C".
[0055] The third step is to perform hash feature processing on the above-mentioned concatenated record items to generate log record features corresponding to the above-mentioned preprocessed log records.
[0056] The hash length corresponding to the log record feature is 512 bits. In practice, the execution subject may perform hash feature processing on the concatenated record items through the SHA-512 hash algorithm to generate the log record feature corresponding to the pre-processed log record.
[0057] Step 1023, generating a security event warning information set for the target log based on the pre-trained security audit model and the log record feature group.
[0058] In some embodiments, the above-mentioned execution subject can generate a security event warning information set for the target log based on the pre-trained security audit model and the log record feature group. Among them, the security event warning information in the security event warning information set includes: log record identification, security event level and security event description information. The log record identification represents the log record (pre-processed log record) that needs to be warned of a security event. The security event level represents the risk level corresponding to the log record corresponding to the log record identification. The security event description information represents the risk description corresponding to the log record corresponding to the log record identification. For example, the above-mentioned execution subject can generate a security event warning information set for the target log based on the pre-trained security audit model and the log record feature group through a generative pre-trained model, such as a GPT model. Among them, the log record features in the log record feature group and the security event warning information in the security event warning information set correspond one to one.
[0059] Optionally, the security audit model includes: a first security audit sub-model and a second security audit sub-model. The first security audit sub-model and the second security audit sub-model are set in parallel. Specifically, the first security audit sub-model is used to generate security event warning information corresponding to the target log of the system operation record log type and the application operation log type. The above-mentioned first security audit sub-model includes: a non-graph feature extraction model and a first warning information generation block. The above-mentioned second security audit sub-model includes: a graph feature extraction model and a second warning information generation block. The above-mentioned first warning information generation block and the above-mentioned second warning information generation block both include: a security event level classifier and a description information generator. Specifically, the non-graph feature extraction model adopts an encoder structure, for example, the non-graph feature extraction model adopts the encoder part in the BERT model. The graph feature extraction model adopts a GCN (Graph Convolutional Networks, graph convolutional network) model. The description information generator in the first warning information generation block includes: a decoder structure and a word segmenter, and the decoder structure is symmetrical with the structure of the non-graph feature extraction model. The word segmenter predicts the probability distribution of words to determine the security event description information. The security incident level classifier in the first warning information generation block is connected to the decoder structure and is used to classify the security incident level. The description information generation model included in the second warning information generation block adopts a Transformer structure. The security incident level classifier included in the second warning information generation block is connected to the graph feature extraction model and is used to classify the security incident level. Specifically, since the level category of the security incident level is fixed, and the input size of the security incident level classifier included in the first warning information generation block and the security incident level classifier included in the second warning information generation block are inconsistent, the security incident level classifier included in the first warning information generation block and the security incident level classifier included in the second warning information generation block can both adopt a fully connected layer + softmax classifier structure. Due to the different input sizes, there is a difference in the input size of the fully connected layer included in the security incident level classifier in the first warning information generation block and the fully connected layer included in the security incident level classifier in the second warning information generation block.
[0060] In some optional implementations of some embodiments, a security event warning information set for the target log is generated based on the pre-trained security audit model and the log record feature group, including:
[0061] In the first step, for each logging feature in the above logging feature group, perform the following generation steps:
[0062] In the first sub-step, log feature extraction is performed on the log record feature group through the non-graph feature extraction model to generate target log record features.
[0063] In the second sub-step, based on the security event level classifier included in the first warning information generation block and the target log record feature, a security event level included in the security event warning information corresponding to the log record feature is generated.
[0064] Specifically, the target log record features can be feature mapped through the fully connected layer included in the security event level classifier, and then the target log record features after feature mapping are input into the softmax classifier to generate the security event level.
[0065] The third sub-step is, in response to the security event level being less than or equal to a preset level, initializing the security event description information included in the security event warning information corresponding to the log record feature to empty.
[0066] In practice, when the security incident level is less than or equal to the preset level, it indicates that there is no security incident, and therefore, the security incident description information included in the security incident warning information is empty.
[0067] The fourth sub-step, in response to the security event level being greater than a preset level, generates security event description information included in the security event warning information corresponding to the log record feature according to the description information generator included in the first warning information generation block and the target log record feature.
[0068] Specifically, the execution subject may input the target log record feature into the description information generator included in the first warning information generation block to generate the security event description information included in the security event warning information corresponding to the log record feature.
[0069] Step 1024 , in response to the log type corresponding to the target log being a network traffic log type, a network traffic flow graph for the target log is generated according to the preprocessed log record group.
[0070] In some embodiments, in response to the log type corresponding to the target log being a network traffic log type, the above-mentioned execution subject may generate a network traffic flow graph for the target log based on the preprocessed log record group. In practice, when the log type is a network traffic log type, the preprocessed log record in the preprocessed log record group may include: a sending source address, a receiving source address, a message type and a message, etc. Therefore, the above-mentioned execution subject may use the sending source address and the receiving source address included in the preprocessed log record as graph nodes and the message flow corresponding to the preprocessed log record as a directed edge to generate a network traffic flow graph.
[0071] In some optional implementations of some embodiments, in response to the log type corresponding to the target log being a network traffic log type, the execution subject generates a network traffic flow graph for the target log according to the preprocessed log record group, which may include the following steps:
[0072] In the first step, an initial network traffic flow diagram is generated according to the sending source and receiving source corresponding to the preprocessed log records in the preprocessed log record group.
[0073] In practice, the execution subject can use the sending source address and receiving source address included in the preprocessed log record as graph nodes to obtain the initial network traffic flow graph. Specifically, the initial network traffic flow can be represented in the form of an adjacency matrix. Since the graph nodes have been determined, the size of the adjacency matrix can be fixed.
[0074] In the second step, based on the preprocessed log record group, the network traffic edges and network traffic intensity of the initial network traffic flow graph are updated to generate the network traffic flow graph.
[0075] In practice, at different times, network requests may be initiated from the same sending source to the same receiving source. In order to preserve the traffic characteristics corresponding to the same network request at different times, a network traffic table is used to characterize the flow direction of network requests, and network traffic intensity is used to characterize the network traffic between the same and the same receiving source. Specifically, the message flow direction corresponding to the preprocessed log record is used as a directed edge, and the number of network requests initiated by the same sending source to the same receiving source is counted as the network traffic intensity, and the above initial traffic flow graph is updated. Specifically, the matrix value of the adjacency matrix can be updated to the corresponding network traffic intensity. When the matrix value of the adjacency matrix is not 0, it indicates the existence of directed edges and network traffic intensity. When the matrix value of the adjacency matrix is 0, it indicates the absence of directed edges.
[0076] Step 1025, extracting graph features from the network traffic flow graph to generate network traffic flow graph features.
[0077] In some embodiments, the above-mentioned execution entity may perform graph feature extraction on the network traffic flow graph to generate network traffic flow graph features. In practice, since the network traffic flow graph is a directed graph structure, the above-mentioned execution entity may perform graph feature extraction on the network traffic flow graph through the GraphSAGE model to generate network traffic flow graph features. The above-mentioned execution entity may also perform graph feature extraction on the network traffic flow graph through the GCN (Graph Convolutional Networks) model to generate network traffic flow graph features. When the GCN model is used, the graph feature extraction model included in the second security audit sub-model may be directly used to avoid repeated feature extraction.
[0078] Step 1026, generating a security event warning information set for the target log based on the security audit model and the network traffic flow graph characteristics.
[0079] In some embodiments, the above-mentioned execution entity can generate a security event warning information set for the target log in various ways according to the security audit model and the network traffic flow graph characteristics.
[0080] In some optional implementations of some embodiments, the execution subject generates a security event warning information set for a target log according to a security audit model and network traffic flow graph characteristics, which may include the following steps:
[0081] In the first step, the graph feature extraction model included in the second security audit sub-model is used to extract the graph feature of the network traffic flow graph to generate the target network traffic flow graph feature.
[0082] In practice, when step 1025 adopts the graph feature extraction model included in the second security audit sub-model, the Mulberry Tree execution entity can directly use the network traffic flow graph feature as the target network traffic flow graph feature.
[0083] The second step is to generate the above-mentioned security event warning information set according to the second warning information generation block included in the above-mentioned second security audit sub-model and the above-mentioned target network traffic flow graph characteristics.
[0084] In practice, the second warning information generation block includes: a security event level classifier and a description information generator. The description information generation model included in the second warning information generation block adopts a Transformer structure. The security event level classifier included in the second warning information generation block is connected to the graph feature extraction model to classify the security event level. The security event level classifier included in the second warning information generation block can adopt a fully connected layer + softmax classifier structure.
[0085] Optionally, after generating security event warning information for the target log according to the security audit model and the network traffic flow graph characteristics, the method further includes:
[0086] In response to the existence of target security event warning information in the security event warning information set, the following processing steps are performed:
[0087] The first sub-step is to forward the above-mentioned target security event warning information to the target terminal.
[0088] The target terminal has event handling authority for the above security event level. In practice, the target terminal can be bound to a user account that has the authority to handle the corresponding security event level. The above target security event warning information is security event warning information that includes security event level trigger warning configuration information. The security event level trigger warning configuration information may include multiple warning triggers to automatically forward the target security event warning information to the target terminal.
[0089] The second sub-step is to archive the log record corresponding to the log record identifier included in the target security event warning information and the target security event warning information.
[0090] In practice, the above-mentioned execution entity may store the log record corresponding to the log record identifier included in the target security event warning information and the above-mentioned target security event warning information in a data warehouse to realize information archiving.
[0091] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the security audit method of some embodiments of the present disclosure, the security risk is reduced. Specifically, the reason for the high security risk is that when conducting a behavior audit, a large number of audit rules often need to be set in advance. When the audit rules are not comprehensive, it is very easy to have an erroneous audit of abnormal behavior, thereby increasing the security risk. Based on this, the security audit method of some embodiments of the present disclosure, first, collects a target log set, wherein the log type of the target log in the above target log set includes: a network traffic log type, a system operation record log type, and an application running log type. Secondly, for each target log in the above target log set, the following security event warning information generation step is performed: the first step is to perform log preprocessing on the above target log to generate a preprocessed log record group. In practice, there may be log records in the form of dirty data during the log collection process, so it is necessary to perform log preprocessing on the target log. In the second step, in response to the log type corresponding to the above target log being a system operation record log type or an application running log type, log record feature extraction is performed on each preprocessed log record in the above preprocessed log record group to generate a log record feature and obtain a log record feature group. For the target logs of the system operation record log type and the application operation log type, since there are complete operation steps or operation steps corresponding to them, the present disclosure performs feature extraction with log records as the granularity. The third step is to generate a security event warning information set for the target log according to the pre-trained security audit model and the above-mentioned log record feature group, wherein the security event warning information in the above-mentioned security event warning information set includes: log record identification, security event level and security event description information. By combining the security audit model, security event warning information for log records is automatically generated. The fourth step is to respond to the log type corresponding to the above-mentioned target log as a network traffic log type, and generate a network traffic flow graph for the above-mentioned target log according to the above-mentioned pre-processed log record group. The fifth step is to perform graph feature extraction on the above-mentioned network traffic flow graph to generate network traffic flow graph features. The target log of the network traffic log type corresponds to a relatively obvious traffic flow feature, so the present disclosure performs feature extraction in the form of a graph. The sixth step is to generate a security event warning information set for the above-mentioned target log according to the above-mentioned security audit model and the network traffic flow graph features. This method eliminates the need to set complex audit rules, while ensuring the efficiency and accuracy of security audits and reducing security risks.
[0092] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a security audit device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the security audit device can be specifically applied to various electronic devices.
[0093] like Figure 2 As shown, the security audit device 200 of some embodiments includes: a collection unit 201 and an execution unit 202. The collection unit 201 is configured to collect a target log set, wherein the log types of the target logs in the target log set include: a network traffic log type, a system operation record log type, and an application running log type; the execution unit 202 is configured to perform the following security event warning information generation steps for each target log in the target log set: perform log preprocessing on the target log to generate a preprocessed log record group; in response to the log type corresponding to the target log being a system operation record log type or an application running log type, perform log record feature extraction on each preprocessed log record in the preprocessed log record group to generate a log record feature , obtain a log record feature group; generate a security event warning information set for the target log according to the pre-trained security audit model and the log record feature group, wherein the security event warning information in the security event warning information set includes: log record identification, security event level and security event description information; in response to the log type corresponding to the target log being a network traffic log type, generate a network traffic flow graph for the target log according to the pre-processed log record group; perform graph feature extraction on the network traffic flow graph to generate network traffic flow graph features; generate a security event warning information set for the target log according to the security audit model and the network traffic flow graph features.
[0094] It is understandable that the units recorded in the security audit device 200 are similar to those in the reference Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features and beneficial effects described above for the method are also applicable to the security audit device 200 and the units included therein, and will not be repeated here.
[0095] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0096] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.
[0097] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0098] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0099] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0100] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0101] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: collects a target log set, wherein the log types of the target logs in the above-mentioned target log set include: network traffic log type, system operation record log type and application running log type; for each target log in the above-mentioned target log set, executes the following security event warning information generation steps: performs log preprocessing on the above-mentioned target log to generate a preprocessed log record group; in response to the log type corresponding to the above-mentioned target log being a system operation record log type or an application running log type, performs log record feature extraction on each preprocessed log record in the above-mentioned preprocessed log record group; Take, to generate log record features, to obtain a log record feature group; based on the pre-trained security audit model and the above-mentioned log record feature group, generate a security event warning information set for the above-mentioned target log, wherein the security event warning information in the above-mentioned security event warning information set includes: log record identification, security event level and security event description information; in response to the log type corresponding to the above-mentioned target log being a network traffic log type, based on the above-mentioned preprocessed log record group, generate a network traffic flow graph for the above-mentioned target log; perform graph feature extraction on the above-mentioned network traffic flow graph to generate network traffic flow graph features; based on the above-mentioned security audit model and the network traffic flow graph features, generate a security event warning information set for the above-mentioned target log.
[0102] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0104] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising an acquisition unit and an execution unit. The names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the acquisition unit may also be described as "a unit for collecting a target log set, wherein the log types of the target logs in the above target log set include: a network traffic log type, a system operation record log type, and an application running log type".
[0105] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0106] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A security audit method comprising: Collecting a target log set, wherein the log types of the target logs in the target log set include: a network traffic log type, a system operation record log type, and an application running log type; For each target log in the target log set, the following security event warning information generation steps are performed: Performing log preprocessing on the target log to generate a preprocessed log record group; In response to the log type corresponding to the target log being a system operation record log type or an application running log type, performing log record feature extraction on each preprocessed log record in the preprocessed log record group to generate a log record feature, thereby obtaining a log record feature group; Generate a security event warning information set for the target log according to the pre-trained security audit model and the log record feature group, wherein the security event warning information in the security event warning information set includes: log record identifier, security event level and security event description information; In response to the log type corresponding to the target log being a network traffic log type, generating a network traffic flow graph for the target log according to the preprocessed log record group; Extracting graph features from the network traffic flow graph to generate network traffic flow graph features; According to the security audit model and the network traffic flow graph characteristics, a security event warning information set for the target log is generated, wherein: The extracting log record features from each preprocessed log record in the preprocessed log record group to generate log record features includes: For each record item in the record item set included in the preprocessed log record, the following initial encoding steps are performed: In response to the record item type of the record item being a numerical type, binary encoding the record item to obtain an encoded record item corresponding to the record item; In response to the record item type of the record item being a fixed-length character type, performing one-hot encoding on the record item to obtain an encoded record item corresponding to the record item; In response to the record item type of the record item being a variable-length character type, performing word embedding processing on the record item to obtain an encoded record item corresponding to the record item; splicing the encoded record items in the obtained encoded record item sequence to generate a spliced record item; Hash feature processing is performed on the concatenated record item to generate a log record feature corresponding to the preprocessed log record, wherein the hash length corresponding to the log record feature is 512 bits.
2. The method according to claim 1, wherein: After generating security event warning information for the target log according to the security audit model and the network traffic flow graph characteristics, the method further includes: In response to the existence of target security event warning information in the security event warning information set, wherein the target security event warning information is security event warning information containing security event level triggering warning configuration information, the following processing steps are performed: Forwarding the target security event warning information to a target terminal, wherein the target terminal has event handling authority corresponding to the security event level; The log record corresponding to the log record identifier included in the target security event warning information and the target security event warning information are archived.
3. The method according to claim 2, wherein: The performing log preprocessing on the target log to generate a preprocessed log record group includes: Parsing the target log to generate an initial log record group, wherein the initial log record in the initial log record group includes a set of record items, each record item has a corresponding record item default identifier, and the record item default identifier indicates whether the corresponding record item is allowed to be empty; For each initial log record in the initial log record group, the following filtering steps are performed: Determine a logical OR result of a record item status value corresponding to each record item in the record item set included in the initial log record and a record item default identifier corresponding to the record item, to obtain a logical OR result sequence, wherein the record item status value indicates whether the record item is empty; Performing a progressive logical AND operation on the logic or results in the logic or result sequence to obtain a logical AND result; In response to the logical AND result being 0, filtering the initial log record from the initial log record group; Log record formatting is performed on each filtered log record in the obtained filtered log record group to generate a preprocessed log record, thereby obtaining the preprocessed log record set.
4. The method according to claim 3, wherein: The security audit model includes: a first security audit sub-model and a second security audit sub-model, the first security audit sub-model includes: a non-graph feature extraction model and a first warning information generation block, the second security audit sub-model includes: a graph feature extraction model and a second warning information generation block, wherein the first warning information generation block and the second warning information generation block both include: a security event level classifier and a description information generator; and The generating a security event warning information set for the target log according to the pre-trained security audit model and the log record feature group includes: For each logging feature in the logging feature group, the following generation steps are performed: Performing log feature extraction on the log record feature group by using the non-graph feature extraction model to generate target log record features; Generate the security event level included in the security event warning information corresponding to the log record feature according to the security event level classifier included in the first warning information generation block and the target log record feature; In response to the security event level being less than or equal to a preset level, initializing the security event description information included in the security event warning information corresponding to the log record feature to be empty; In response to the security event level being greater than a preset level, security event description information included in the security event warning information corresponding to the log record feature is generated according to the description information generator included in the first warning information generation block and the target log record feature.
5. The method according to claim 4, wherein: In response to the log type corresponding to the target log being a network traffic log type, generating a network traffic flow graph for the target log according to the preprocessed log record group includes: Generate an initial network traffic flow graph according to the sending source and receiving source corresponding to the preprocessed log records in the preprocessed log record group; According to the preprocessed log record group, the network traffic edges and network traffic intensity of the initial network traffic flow graph are updated to generate the network traffic flow graph.
6. The method according to claim 5, wherein: The generating a security event warning information set for the target log according to the security audit model and the network traffic flow graph characteristics includes: By using the graph feature extraction model included in the second security audit sub-model, the graph feature extraction is performed on the network traffic flow graph feature to generate a target network traffic flow graph feature; The security event warning information set is generated according to the second warning information generation block included in the second security audit sub-model and the target network traffic flow graph characteristics.
7. A security audit device, comprising: The collecting unit is configured to collect a target log set, wherein the log types of the target logs in the target log set include: a network traffic log type, a system operation record log type, and an application running log type; The execution unit is configured to perform the following security event warning information generation steps for each target log in the target log set: perform log preprocessing on the target log to generate a preprocessed log record group; in response to the log type corresponding to the target log being a system operation record log type or an application running log type, perform log record feature extraction on each preprocessed log record in the preprocessed log record group to generate a log record feature and obtain a log record feature group; generate a security event warning information set for the target log based on a pretrained security audit model and the log record feature group, wherein the security event warning information in the security event warning information set includes: a log record identifier, a security event level, and a security event description information; in response to the log type corresponding to the target log being a network traffic log type, generate a network traffic flow graph for the target log based on the preprocessed log record group; perform graph feature extraction on the network traffic flow graph to generate a network traffic flow graph feature; generate a security event warning information set for the target log based on the security audit model and the network traffic flow graph feature, wherein The extracting log record features from each preprocessed log record in the preprocessed log record group to generate log record features includes: For each record item in the record item set included in the preprocessed log record, the following initial encoding steps are performed: In response to the record item type of the record item being a numerical type, binary encoding the record item to obtain an encoded record item corresponding to the record item; In response to the record item type of the record item being a fixed-length character type, performing one-hot encoding on the record item to obtain an encoded record item corresponding to the record item; In response to the record item type of the record item being a variable-length character type, performing word embedding processing on the record item to obtain an encoded record item corresponding to the record item; splicing the encoded record items in the obtained encoded record item sequence to generate a spliced record item; Hash feature processing is performed on the concatenated record item to generate a log record feature corresponding to the preprocessed log record, wherein the hash length corresponding to the log record feature is 512 bits.
8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Network node flow anomaly analysis method and system
CN107070952A
Event auditing method and device, terminal equipment and storage medium
CN111045847A
Log-based early warning method and device, server and storage medium
CN112636957A
Log auditing method, system and equipment based on multivariate log data analysis and medium
CN114020726A
Log auditing method and system based on multivariate log data analysis
CN116010368A
