A Network Intrusion Detection Method and System Based on Log Audit

Abstract

The present invention provides a network intrusion detection method and system based on log auditing, which relates to the field of network security technology. The method includes: obtaining multi-source log data from different log sources; performing data parsing on the log data to obtain the source IP address and destination IP address of the log data; storing the source IP address and destination IP address in the form of a summary data structure; calculating the signal deviation value between the current summary data structure and the historical summary data structure; reordering the signal deviation values in ascending order of numerical value to obtain the target signal deviation value; calculating the actual energy value of the target signal deviation value through wavelet transform; calculating the estimated energy value and variance estimated value of the target signal deviation value through exponentially weighted moving average; dynamically determining the energy threshold according to the estimated energy value and variance estimated value of the target signal deviation value; comparing the actual energy value of the target signal deviation value with the energy threshold to perform network intrusion detection.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a network intrusion detection method and system based on log auditing. Background Art

[0002] With the rapid development of Internet technology, network attacks and intrusions have become a major challenge in the field of information security. Various forms of network attacks, such as denial of service attacks, data leaks, and malware intrusions, have posed a serious threat to the global network environment. In order to ensure the security and stability of network systems, timely detection and prevention of network intrusions has become one of the core goals in the field of network security.

[0003] With the continuous expansion of network scale and the dramatic increase of data volume, more and more modern technologies are being applied to network intrusion detection.

[0004] Patented technologies such as CN109522716B perform network intrusion detection based on temporal neural networks, which effectively solves the shortcomings of RNN and is closer to the real-time requirements of the network than LSTM. However, this type of network intrusion detection method has low computational efficiency and insufficient real-time performance when processing massive data.

[0005] In addition, existing network intrusion detection methods usually rely on detection thresholds pre-set by experts. When faced with complex and changing network environments, the detection rate is low and the false alarm rate is high. Summary of the invention

[0006] In order to solve the technical problems in the prior art of low computing efficiency and insufficient real-time performance when processing massive data, relying on detection thresholds pre-set by experts, and having low detection rate and high false alarm rate when facing a complex and changeable network environment, the present invention provides a network intrusion detection method and system based on log auditing.

[0007] The technical solution provided by the embodiment of the present invention is as follows:

[0008] First aspect:

[0009] An embodiment of the present invention provides a network intrusion detection method based on log auditing, comprising:

[0010] S1: Obtain multi-source log data from different log sources;

[0011] S2: parsing the log data to obtain the source IP address and destination IP address of the log data;

[0012] S3: storing the source IP address and the destination IP address in the form of a summary data structure;

[0013] S4: Calculate the signal deviation value between the current summary data structure and the historical summary data structure;

[0014] S5: reordering the signal deviation values ​​in ascending order to obtain a target signal deviation value;

[0015] S6: Calculate the actual energy value of the target signal deviation value through wavelet transform;

[0016] S7: Calculate the energy estimation value and variance estimation value of the target signal deviation value by exponentially weighted moving average;

[0017] S8: dynamically determining an energy threshold according to the energy estimation value and the variance estimation value of the target signal deviation value;

[0018] S9: Compare the actual energy value of the target signal deviation value with the energy threshold to perform network intrusion detection.

[0019] Second aspect:

[0020] An embodiment of the present invention provides a network intrusion detection system based on log auditing, comprising:

[0021] processor;

[0022] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the network intrusion detection method based on log auditing as described in the first aspect is implemented.

[0023] The third aspect:

[0024] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the network intrusion detection method based on log auditing as described in the first aspect is implemented.

[0025] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0026] By storing the source IP address and the destination IP address in the form of a summary data structure, the calculation efficiency is high and the real-time performance is strong when processing massive data. By calculating the signal deviation value between the current summary data structure and the historical summary data structure, the signal deviation values ​​are reordered in order from small to large to obtain the target signal deviation value. The actual energy value of the target signal deviation value is calculated through wavelet transform, and the energy estimation value and variance estimation value of the target signal deviation value are calculated through exponentially weighted moving average. According to the energy estimation value and variance estimation value of the target signal deviation value, the energy threshold is dynamically determined, and it no longer depends on the detection threshold set in advance by experts. When facing a complex and changeable network environment, the detection rate is high and the false alarm rate is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A flow chart of a network intrusion detection method based on log auditing provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the structure of a network intrusion detection system based on log auditing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0031] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0032] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0033] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0034] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0035] Reference Manual Attached Figure 1 , shows a flow chart of a network intrusion detection method based on log auditing provided by an embodiment of the present invention.

[0036] The embodiment of the present invention provides a network intrusion detection method based on log auditing, which can be implemented by a network intrusion detection device based on log auditing, and the network intrusion detection device based on log auditing can be a terminal or a server. The processing flow of the network intrusion detection method based on log auditing may include the following steps:

[0037] S1: Obtain multi-source log data from different log sources;

[0038] S2: parse the log data to obtain the source IP address and destination IP address of the log data;

[0039] S3: The source IP address and the destination IP address are stored in the form of a summary data structure.

[0040] It should be noted that Sketch is a type of approximate data structure used to efficiently process large-scale data streams. It can compress and represent data features through methods such as hash functions and counters under limited memory and computing resources. Sketch data structures are usually used for fast frequency statistics, change detection or pattern matching. They do not need to store all the original data, thus having significant advantages in time and space. Common applications include network traffic analysis, frequency estimation, and anomaly detection.

[0041] In the present invention, the summary data structure represents data features through compression, does not need to store all the original data, and can process large-scale network log data using only limited memory and counters, effectively avoiding the excessive reliance on memory of traditional storage methods.

[0042] In a possible implementation, the summary data structure includes multiple hash tables, the hash tables include an independent hash function and multiple counters, and S3 specifically includes sub-steps S301 and S302:

[0043] S301: Calculate the hash values ​​of the source IP address and the destination IP address respectively through the hash function:

[0044] ;

[0045] in, h i ( ) indicates that through i The hash value calculated by the hash function is SIP Indicates the source IP address. a i Indicates i The first random number of the hash function, b i Indicates i The second random number of the hash function,p represents a Mersenne prime, K Indicates the size of the hash table, DIP Indicates the destination IP address.

[0046] It should be noted that a hash function is a mathematical function that maps input data of any length to an output of a fixed length, and its output is usually a hash value (hash value). It has the characteristics of fast calculation, determinism and anti-collision, that is, the same input can always generate the same output, but different inputs can generate different outputs as much as possible. Hash functions are widely used in data encryption, information verification, data storage (such as hash tables) and distributed systems to achieve efficient data search and verification.

[0047] S302: According to the hash values ​​of the source IP address and the destination IP address, the source IP address and the destination IP address are stored in respective counters:

[0048] ;

[0049] in, S 1 Indicates the counter value after storing the hash value of the source IP address. S 2 Indicates the counter value that stores the hash value of the destination IP address.

[0050] Specifically, the summary data structure uses multiple hash tables to store the information of the source IP address (SIP) and the destination IP address (DIP). Each hash table contains an independent hash function and several counters to record the data distribution after hash mapping. First, the hash values ​​of the source IP address and the destination IP address are calculated by the hash function respectively. These hash values ​​are used to determine the corresponding counter positions in the hash table. Then, the calculated hash values ​​are mapped to the corresponding counters, where the hash value of the source IP address is stored in a counter structure, and the hash value of the destination IP address is stored in another counter structure. Whenever a new data stream arrives, the value of the relevant counter is automatically incremented to accurately reflect the traffic contribution of the corresponding IP address.

[0051] In the present invention, the IP address can be quickly mapped to a fixed-size hash table through a hash function, which greatly reduces the time for direct search and comparison. The design of the hash table ensures fast access and update of large-scale log data, which is suitable for real-time processing scenarios. Whenever a new data stream arrives, the value of the counter is automatically incremented, which can reflect the traffic characteristics of each IP address in real time. Using the counter value stored in the hash table, the traffic changes at different time points can be quickly compared, providing a data basis for anomaly detection. The frequency of the hash value is recorded by the counter instead of saving the actual data, which further reduces the storage space requirement.

[0052] S4: Calculate the signal deviation value between the current summary data structure and the historical summary data structure.

[0053] In the present invention, by calculating the signal deviation value between the current and historical summary data structures, significant changes or abnormal fluctuations in traffic can be effectively discovered. By calculating the signal deviation value, the difference between network traffic and historical data can be quickly obtained, reducing the computational burden of traditional intrusion detection methods that need to check a large amount of raw data, and improving the speed of detection and response.

[0054] In a possible implementation manner, S4 is specifically:

[0055] The signal deviation value between the current summary data structure and the historical summary data structure is calculated according to the following formula:

[0056] ;

[0057] in, S dev represents the signal deviation value, | | represents the absolute value, S t,i Indicated in t The summary data structure at time i The counter value in the hash table, S t-1,i Indicated in t -1 in the summary data structure at time i The counter value of the hash table, n i,j Indicates the current summary data structure. i The first j The value of the counter, m i,j The structure of the history summary data i The first j The value of the counter, J Represents the total number of counters in the hash table.

[0058] Specifically, the signal deviation value is:

[0059] .

[0060] In the present invention, by calculating the signal deviation value of the current summary data structure and the historical summary data structure, the slight changes in data traffic can be accurately captured. The calculation of the signal deviation value reflects the difference between the two, so that abnormal fluctuations in network traffic can be efficiently detected. The signal deviation value provides a dynamic reference for the system. By comparing historical data with current data, the detection algorithm can adaptively adjust the detection rules and strategies according to the characteristics and changes of the traffic, thereby improving the detection effect. The signal deviation value can reduce the impact of single-dimensional anomalies on the overall detection by integrating the differences of multiple counters, thereby enhancing the system's anti-interference ability against network attacks.

[0061] S5: Reorder the signal deviation values ​​in ascending order to obtain a target signal deviation value.

[0062] In a possible implementation, S5 is specifically:

[0063] According to the following formula, the signal deviation values ​​are reordered in ascending order to obtain the target signal deviation value:

[0064] ;

[0065] in, Indicates the target signal deviation value, sort ( ) indicates a sort operation. ascending Indicates ascending order, that is, sorting from small to large.

[0066] For example, if the unsorted signal deviation value S dev =<5,2,8,3>, then the target signal deviation value obtained after reordering =<2,3,5,8>.

[0067] In the present invention, by reordering the signal deviation values ​​in ascending order, the target signal deviation value can be obtained, which can effectively put the most significant abnormal changes in a higher priority position, thereby improving the accuracy and response speed of abnormal detection. This sorting method helps the system highlight those traffic patterns that may represent intrusion or abnormal activities, reduce interference with unimportant and small fluctuations, and make the detection process more efficient, sensitive and accurate.

[0068] S6: Calculate the actual energy value of the target signal deviation value through wavelet transform.

[0069] It should be noted that wavelet transform is a signal processing technology that analyzes the time-frequency characteristics of a signal by decomposing it into different frequency components and locating each component in time. Unlike Fourier transform, wavelet transform uses a window of variable size and can provide information in both the time domain and the frequency domain, making it very suitable for analyzing non-stationary signals. It is widely used in image processing, signal denoising, compression, and anomaly detection, and can effectively capture the local characteristics and detailed changes of a signal.

[0070] In the present invention, wavelet transform can decompose the signal into different frequency components and locate them in the time domain, which enables it to provide information about the time variation and frequency components of the signal at the same time. Unlike the traditional Fourier transform, wavelet transform uses a window of variable size and can capture the instantaneous changes and local characteristics of the signal more accurately.

[0071] In a possible implementation, S6 specifically includes sub-steps S601 and S602:

[0072] S601: Decompose the target signal deviation value into an approximate signal and a detail signal.

[0073] It should be noted that the approximate signal is a low-frequency component extracted from the original signal by wavelet transform, which represents the overall trend or global characteristics of the signal. It captures the slower and smoother changing part of the signal, and is usually used to describe the main structure or basic mode of the signal, and is suitable for long-term change analysis.

[0074] It should be noted that the detail signal is a high-frequency component extracted from the original signal by wavelet transform, which represents the subtle fluctuations or local characteristics of the signal. It reflects the rapidly changing parts of the signal, such as edge or mutation information, and is often used to analyze short-term changes in the signal or detect anomalies.

[0075] Among them, the approximate signal is specifically:

[0076] ;

[0077] in, c i The trend signal i elements, V ij Represents the scaling factor matrix i Line j The elements of the column, Indicates the target signal deviation value. j counter value.

[0078] Among them, the scaling factor matrix is ​​specifically:

[0079] ;

[0080] in, v 1 represents the first scaling factor in the scaling factor matrix, v 2 represents the second scaling factor in the scaling factor matrix, v 3 represents the third scaling factor in the scaling factor matrix, v 4 Represents the fourth scaling factor in the scaling factor matrix.

[0081] The detailed signals are as follows:

[0082] ;

[0083] in, d i The fluctuation signal i elements, W ij Represents the first i Line j Elements of a column.

[0084] Among them, the wavelet coefficient matrix is ​​specifically:

[0085] ;

[0086] in, w 1 represents the first wavelet coefficient in the wavelet coefficient matrix, w 2 represents the second wavelet coefficient in the wavelet coefficient matrix, w 3 represents the third wavelet coefficient in the wavelet coefficient matrix, w 4 Represents the fourth wavelet coefficient in the wavelet coefficient matrix.

[0087] S602: Calculate the actual energy value of the target signal deviation value according to the approximate signal and the detail signal:

[0088] ;

[0089] in, exist t The actual energy value at the moment, D j Indicates the detail signal j elements, A j Represents the approximate signal j elements.

[0090] In the present invention, the target signal deviation value is decomposed into an approximate signal and a detail signal by wavelet transform, and the low-frequency and high-frequency components of the signal are captured respectively, thereby realizing multi-scale analysis of network traffic. The approximate signal reflects the overall trend and long-term changes of the traffic, while the detail signal captures sudden fluctuations and local anomalies. By calculating the actual energy values ​​of the two signals respectively, the overall characteristics and short-term fluctuations of the traffic can be more accurately evaluated, effectively improving the sensitivity and accuracy of anomaly detection. In addition, this decomposition method can adapt to non-stationary signals, reduce noise interference, and enhance the ability to recognize complex and diverse attack patterns, thereby providing stronger support for network intrusion detection.

[0091] S7: Calculate the energy estimate and variance estimate of the target signal deviation value by exponentially weighted moving average.

[0092] It should be noted that the exponentially weighted moving average (EWMA) is a statistical method for calculating weighted averages. It smoothes the data by giving higher weights to recent data points to capture recent trends in the data. The weights of EWMA decay exponentially, and the influence of historical data on the average gradually decreases over time. This method is widely used in time series analysis, trend forecasting, and anomaly detection because it provides a good balance between real-time and smoothness.

[0093] In a possible implementation manner, S7 specifically includes:

[0094] The energy estimate and variance estimate of the target signal deviation value are calculated by exponentially weighted moving average according to the following formula:

[0095] ;

[0096] in, Indicated in t The estimated energy value at time, α represents the weight coefficient of the energy estimate, Indicated in t -1 The actual energy value at time Indicated in t The estimated energy value at time -1, Indicated in t The estimated variance at time , β represents the weight coefficient of the variance estimate, e t-1 Indicated in t Energy deviation value at time -1, Indicated in t The variance estimate at time -1.

[0097] In the present invention, through exponential weighting, EWMA can not only retain recent trend information, but also avoid excessive influence of historical data on current estimation, thereby improving the accuracy of energy and variance estimation. EWMA reduces the impact of accidental and unimportant fluctuations on the final result through smoothing, which helps to improve the accuracy of anomaly detection. Variance estimation is an important indicator for judging data volatility. EWMA provides a smooth way to estimate variance, so that in the case of large traffic fluctuations, it can accurately reflect the fluctuation range of traffic, thereby enhancing the detection system's ability to identify abnormal fluctuations.

[0098] S8: Dynamically determine the energy threshold value according to the energy estimation value and the variance estimation value of the target signal deviation value.

[0099] In a possible implementation manner, S8 specifically includes:

[0100] The energy threshold is determined dynamically according to the following formula:

[0101] ;

[0102] in, Th t represents the energy threshold, λ represents the weight coefficient of the energy threshold, sketch . attribute Represents a property identifier.

[0103] In the present invention, by dynamically determining the energy threshold based on the real-time energy estimation value and variance estimation value, the system can adapt to changes in network traffic. When the pattern of network traffic changes, the threshold will also be automatically adjusted without relying on fixed threshold settings, which enhances the flexibility and adaptability of the system. The dynamic adjustment of the energy threshold can automatically adjust the sensitivity according to different network environments and time periods, ensuring that higher detection sensitivity is provided when traffic is abnormal, while avoiding excessive false alarms under normal circumstances.

[0104] S9: Compare the actual energy value of the target signal deviation value with the energy threshold to perform network intrusion detection.

[0105] In the present invention, by comparing the actual energy value of the target signal deviation value with the dynamically calculated energy threshold, the system can sensitively detect abnormal behaviors in the traffic that deviate greatly from the normal mode. By comparing the actual energy value with the dynamic threshold, the system can more effectively distinguish normal fluctuations from potential intrusion behaviors, thereby reducing false positives and false negatives. This makes the detection results more reliable, especially in an environment where the network traffic is relatively complex, and can filter out the interference of normal fluctuations and noise.

[0106] In a possible implementation, S9 specifically includes sub-steps S901 and S902:

[0107] S901: Compare the actual energy value of the target signal deviation value with the energy threshold, and set a network intrusion detection rule. The network intrusion detection rule includes detection rules for source IP addresses and destination IP addresses:

[0108] ;

[0109] in, δ t Indicated in t The source IP address detection rule at the moment, when δ t =1, the source IP address is detected as abnormal, that is, the network is invaded. δ t =0, the source IP address is detected as normal, that is, the network has not been invaded. Indicated in t The detection rule of the destination IP address at the moment, when =1, the destination IP address is detected as abnormal, that is, the network is invaded. =0, the destination IP address is detected as normal, that is, the network has not been invaded;

[0110] S902: Perform network intrusion detection according to detection rules of source IP addresses and destination IP addresses.

[0111] Specifically, when the attribute identifier is the source IP address (DIP), the energy threshold If the energy threshold is greater than the actual energy value of the target signal deviation value, it means that the source IP address is detected as abnormal. When the attribute identifier is the destination IP address (SIP), the energy threshold ,If at this time the energy threshold is less than the actual energy value of the target signal deviation value, it indicates that the destination IP address is detected as abnormal.

[0112] Further, when the source IP address and / or the destination IP address is detected as abnormal, it is determined that the network is invaded.

[0113] In the present invention, by independently detecting the source IP and the destination IP, the system can accurately identify the source and target of abnormal traffic in the network. Based on the comparison between the actual energy value and the dynamic threshold, the detection rules of the source IP and the destination IP can be dynamically adjusted according to the change of real-time traffic, further improving the accuracy of intrusion detection and reducing the occurrence of false positives and false negatives. By detecting the source IP and the destination IP separately, network intrusion can be identified more accurately, the attack target can be discovered in time, and the spread of the attack source can be helped to effectively limit the impact range of the attack.

[0114] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0115] By storing the source IP address and the destination IP address in the form of a summary data structure, the calculation efficiency is high and the real-time performance is strong when processing massive data. By calculating the signal deviation value between the current summary data structure and the historical summary data structure, the signal deviation values ​​are reordered in order from small to large to obtain the target signal deviation value. The actual energy value of the target signal deviation value is calculated through wavelet transform, and the energy estimation value and variance estimation value of the target signal deviation value are calculated through exponentially weighted moving average. According to the energy estimation value and variance estimation value of the target signal deviation value, the energy threshold is dynamically determined, and it no longer depends on the detection threshold set in advance by experts. When facing a complex and changeable network environment, the detection rate is high and the false alarm rate is low.

[0116] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a network intrusion detection system based on log auditing provided by the present invention.

[0117] The present invention further provides a network intrusion detection system 20 based on log auditing, which is applied to the above-mentioned network intrusion detection method based on log auditing, and comprises:

[0118] Processor 201.

[0119] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the network intrusion detection method based on log auditing as in the method embodiment is implemented.

[0120] The network intrusion detection system 20 based on log auditing provided by the present invention can execute the above-mentioned network intrusion detection method based on log auditing and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0121] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0122] By storing the source IP address and the destination IP address in the form of a summary data structure, the calculation efficiency is high and the real-time performance is strong when processing massive data. By calculating the signal deviation value between the current summary data structure and the historical summary data structure, the signal deviation values ​​are reordered in order from small to large to obtain the target signal deviation value. The actual energy value of the target signal deviation value is calculated through wavelet transform, and the energy estimation value and variance estimation value of the target signal deviation value are calculated through exponentially weighted moving average. According to the energy estimation value and variance estimation value of the target signal deviation value, the energy threshold is dynamically determined, and it no longer depends on the detection threshold set in advance by experts. When facing a complex and changeable network environment, the detection rate is high and the false alarm rate is low.

[0123] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0124] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0125] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0126] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0127] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0128] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0129] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0131] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0134] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0135] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the network intrusion detection method based on log auditing as described in the method embodiment is implemented.

[0136] A computer-readable storage medium provided by the present invention can implement the steps and effects of the network intrusion detection method based on log auditing of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0137] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0138] By storing the source IP address and the destination IP address in the form of a summary data structure, the calculation efficiency is high and the real-time performance is strong when processing massive data. By calculating the signal deviation value between the current summary data structure and the historical summary data structure, the signal deviation values ​​are reordered in order from small to large to obtain the target signal deviation value. The actual energy value of the target signal deviation value is calculated through wavelet transform, and the energy estimation value and variance estimation value of the target signal deviation value are calculated through exponentially weighted moving average. According to the energy estimation value and variance estimation value of the target signal deviation value, the energy threshold is dynamically determined, and it no longer depends on the detection threshold set in advance by experts. When facing a complex and changeable network environment, the detection rate is high and the false alarm rate is low.

[0139] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0140] There are a few points to note:

[0141] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0142] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0143] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0144] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A network intrusion detection method based on log auditing, characterized in that: include: S1: Obtain multi-source log data from different log sources; S2: parsing the log data to obtain the source IP address and destination IP address of the log data; S3: storing the source IP address and the destination IP address in the form of a summary data structure; S4: Calculate the signal deviation value between the current summary data structure and the historical summary data structure; S5: reordering the signal deviation values ​​in ascending order to obtain a target signal deviation value; S6: Calculate the actual energy value of the target signal deviation value through wavelet transform; S7: Calculate the energy estimation value and variance estimation value of the target signal deviation value by exponentially weighted moving average; S8: dynamically determining an energy threshold according to the energy estimation value and the variance estimation value of the target signal deviation value; S9: Compare the actual energy value of the target signal deviation value with the energy threshold to perform network intrusion detection; Wherein, the S6 specifically includes: S601: Decomposing the target signal deviation value into an approximate signal and a detail signal; S602: Calculate the actual energy value of the target signal deviation value according to the approximate signal and the detail signal: Among them, P t d The actual value of energy at time t, D j represents the jth element in the detail signal, A j represents the jth element in the approximated signal.

2. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The summary data structure includes multiple hash tables, each of which includes an independent hash function and multiple counters. S3 specifically includes: S301: Calculate the hash values ​​of the source IP address and the destination IP address respectively through the hash function: h i (SIP)=(a i SIP+b i )modpmodK h i (DIP)=(a i DIP+b i )modpmodK a,b∈[1,p-1] Among them, h i () represents the hash value calculated by the i-th hash function, SIP represents the source IP address, a i represents the first random number of the i-th hash function, b i represents the second random number of the i-th hash function, p represents a Mersenne prime number, K represents the size of the hash table, and DIP represents the destination IP address; S302: According to the hash values ​​of the source IP address and the destination IP address, store the source IP address and the destination IP address in each of the counters: S1[i,h i (SIP)]+=1,h i (SIP)∈[1,K] S2[i,h i (DIP)]+=1,h i (DIP)∈[1,K] Among them, S1 represents the counter value after storing the hash value of the source IP address, and S2 represents the counter value after storing the hash value of the destination IP address.

3. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The S4 is specifically: The signal deviation value between the current summary data structure and the historical summary data structure is calculated according to the following formula: S dev =|St ,i -St -1,i | St. ,i =<in ,1 ,in ,2 ,...,in ,J > St. -1,i =<my ,1 ,my ,2 ,...,my ,J > Among them, S dev represents the signal deviation value, || represents the absolute value, S t,i represents the counter value in the i-th hash table of the summary data structure at time t, S t-1,i represents the counter value of the i-th hash table in the summary data structure at time t-1, n i,j Represents the value of the jth counter in the i-th hash table of the current summary data structure, m i,j Represents the value of the jth counter in the i-th hash table of the history summary data structure, and J represents the total number of counters in the hash table.

4. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The S5 is specifically: According to the following formula, the signal deviation values ​​are reordered in ascending order to obtain the target signal deviation value: S' dev =sort(S dev ,ascending) Among them, S' dev It represents the target signal deviation value, sort() represents the sorting operation, and ascending represents ascending order, that is, sorting in ascending order.

5. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The approximate signal is specifically: Among them, c i Represents the i-th element of the trend signal, V ij Represents the element in the i-th row and j-th column of the scaling factor matrix, S' dev,j represents the jth counter value in the target signal deviation value; The detail signal is specifically: Among them, d i represents the i-th element of the fluctuation signal, W ij Represents the element in the i-th row and j-th column of the wavelet coefficient matrix.

6. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The S7 is specifically: The energy estimation value and variance estimation value of the target signal deviation value are calculated by exponentially weighted moving average according to the following formula: in, represents the energy estimation value at time t, α represents the weight coefficient of the energy estimation value, represents the actual value of energy at time t-1, represents the estimated energy value at time t-1, represents the variance estimate at time t, β represents the weight coefficient of the variance estimate, e t-1 represents the energy deviation value at time t-1, Represents the estimated variance at time t-1.

7. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The S8 is specifically: The energy threshold is determined dynamically according to the following formula: Among them, Th t represents the energy threshold, λ represents the weight coefficient of the energy threshold, and sketch.attribute represents the attribute identifier.

8. The network intrusion detection method based on log auditing according to claim 1 is characterized in that: The S9 specifically includes: S901: Compare the actual energy value of the target signal deviation value with the energy threshold, and set a network intrusion detection rule, wherein the network intrusion detection rule includes a detection rule for a source IP address and a destination IP address: Among them, δ t Represents the detection rule of the source IP address at time t, when δ t =1, the source IP address is detected as abnormal, that is, the network is invaded. t = 0, the source IP address is detected as normal, that is, the network has not been invaded, t ' represents the detection rule of the destination IP address at time t, when δ t When '=1, the destination IP address is detected as abnormal, that is, the network is invaded. t When '=0, the destination IP address is detected as normal, that is, the network has not been invaded; S902: Perform network intrusion detection according to the detection rules of the source IP address and the destination IP address.

9. A network intrusion detection system based on log auditing, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the network intrusion detection method based on log auditing as described in any one of claims 1 to 7 is implemented.

Images