A network feature analysis method and device, electronic equipment and storage medium

Abstract

The application discloses a network feature analysis method and device, electronic equipment and storage medium, the method comprises the following steps: analyzing network abnormal traffic through an exponential distribution function to obtain a preliminary analysis result; performing regression prediction on the network abnormal traffic to obtain a fitting value; determining information entropy according to the preliminary analysis result; adjusting the exponential distribution function according to the comparison result of the fitting value and the information entropy, and obtaining a second analysis result; marking the multivariate data in the second analysis result to obtain an information entropy vector of the multivariate data, and determining abnormal flow information entropy features based on the information entropy vector; the application adopts a regression algorithm combined with an information entropy algorithm to accurately filter and analyze abnormal data, thereby completing network flow space-time feature analysis, and can be widely applied to the technical field of network data processing.

Description

Technical Field

[0001] This invention relates to the field of network data processing technology, and in particular to a network feature analysis method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous expansion of network applications, the requirements for network security are becoming increasingly stringent. Network information system security is facing severe challenges. Traditional network security analysis relies on existing network security products, whose detection performance is clearly insufficient in the face of the rapid increase in network data volume and the rapid increase in attack types. The method of using them to analyze large amounts of log information to discover network anomalies is no longer effective. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, this invention proposes a network feature analysis method, apparatus, electronic device, and storage medium, capable of performing network feature analysis efficiently and accurately.

[0004] On one hand, embodiments of the present invention provide a network feature analysis method, including:

[0005] Preliminary analysis results were obtained by analyzing abnormal network traffic using the exponential distribution function.

[0006] Regression prediction is performed on abnormal network traffic to obtain fitted values;

[0007] Based on the preliminary analysis results, the information entropy is determined;

[0008] Based on the comparison results between the fitted values ​​and the information entropy, the exponential distribution function is adjusted, and a second analysis result is obtained;

[0009] The tuple data in the second analysis results are labeled to obtain the information entropy vector of the tuple data, and the information entropy characteristics of the abnormal flow are determined based on the information entropy vector.

[0010] Optionally, regression prediction is performed on abnormal network traffic to obtain fitted values, including:

[0011] A regression prediction model is set up based on ridge regression;

[0012] Abnormal network traffic is input into the regression prediction model, and the fitted value is obtained by combining the preliminary analysis results and the regularization term; the regularization term is determined based on the fitted hyperparameters and weight constants.

[0013] Optionally, based on the preliminary analysis results, the information entropy is determined, including:

[0014] Obtain the prior probabilities of each piece of information or event in the preliminary analysis results;

[0015] Based on prior probabilities, the information content of each piece of information or event is obtained through a logarithmic function;

[0016] The information entropy is obtained by summing the expected information content of each piece of information or event.

[0017] Information entropy is negatively correlated with the accuracy of the preliminary analysis results.

[0018] Optionally, the exponential distribution function is adjusted based on the comparison between the fitted values ​​and the information entropy, including:

[0019] When the difference between the fitted value and the information entropy is greater than the first preset ratio, the absolute value of the difference between the fitted value and the information entropy is obtained, and then the differential data is determined.

[0020] The differentiated data are weighted and averaged within a second preset ratio to obtain a reference value; the reference value is used to adjust the loss function of the exponential distribution function.

[0021] Optionally, the method further includes:

[0022] Differential data outside the second preset ratio are removed, and the remaining data are weighted and averaged.

[0023] Specifically, when the ratio of the removed data to the remaining data is greater than the third preset ratio, the differential data within the preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data.

[0024] Regression predictions are performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference between the actual fitted value and the simulated fitted value is determined.

[0025] The difference data within the first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

[0026] Optionally, the tuple data in the second analysis results are labeled to obtain the information entropy vector of the tuple data, including:

[0027] Each network flow in the tuple data in the second analysis result is labeled; wherein, the network flow of the tuple data includes source address, destination address, source port, destination port, number of bytes per packet in the source flow, and number of bytes per packet in the destination flow;

[0028] Based on the information entropy of each network flow in the tuple data, the information entropy vector of the tuple data is obtained.

[0029] The information entropy vector represents the network state within a preset time window.

[0030] Optionally, the information entropy features of the abnormal flow are determined based on the information entropy vector, including:

[0031] Obtain the information entropy within the target time window as the baseline entropy;

[0032] By cross-processing the information entropy vector and the baseline entropy, the degree of network state deviation is determined, and the information entropy characteristics of abnormal flows are determined based on the degree of network state deviation.

[0033] On the other hand, embodiments of the present invention provide a network feature analysis device, comprising:

[0034] The first module is used to analyze abnormal network traffic using the exponential distribution function to obtain preliminary analysis results;

[0035] The second module is used to perform regression prediction on abnormal network traffic and obtain fitted values.

[0036] The third module is used to determine the information entropy based on the preliminary analysis results;

[0037] The fourth module is used to adjust the exponential distribution function based on the comparison results of the fitted values ​​and information entropy, and to obtain the second analysis results;

[0038] The fifth module is used to label the tuple data in the second analysis results, obtain the information entropy vector of the tuple data, and determine the information entropy characteristics of the abnormal flow based on the information entropy vector.

[0039] Optionally, the second module is specifically used for:

[0040] A regression prediction model is set up based on ridge regression;

[0041] Abnormal network traffic is input into the regression prediction model, and the fitted value is obtained by combining the preliminary analysis results and the regularization term; the regularization term is determined based on the fitted hyperparameters and weight constants.

[0042] Optionally, the third module is specifically used for:

[0043] Obtain the prior probabilities of each piece of information or event in the preliminary analysis results;

[0044] Based on prior probabilities, the information content of each piece of information or event is obtained through a logarithmic function;

[0045] The information entropy is obtained by summing the expected information content of each piece of information or event.

[0046] Information entropy is negatively correlated with the accuracy of the preliminary analysis results.

[0047] Optionally, the step in the fourth module, which adjusts the exponential distribution function based on the comparison results of the fitted values ​​and the information entropy, is specifically used for:

[0048] When the difference between the fitted value and the information entropy is greater than the first preset ratio, the absolute value of the difference between the fitted value and the information entropy is obtained, and then the differential data is determined.

[0049] The differentiated data are weighted and averaged within a second preset ratio to obtain a reference value; the reference value is used to adjust the loss function of the exponential distribution function.

[0050] Optionally, the device further includes:

[0051] The sixth module is used to remove differential data outside the second preset ratio and to perform a weighted average on the remaining data;

[0052] Specifically, when the ratio of the removed data to the remaining data is greater than the third preset ratio, the differential data within the preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data.

[0053] Regression predictions are performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference between the actual fitted value and the simulated fitted value is determined.

[0054] The difference data within the first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

[0055] Optionally, the step in the fifth module of labeling the tuple data in the second analysis results to obtain the information entropy vector of the tuple data is specifically used for:

[0056] Each network flow in the tuple data in the second analysis result is labeled; wherein, the network flow of the tuple data includes source address, destination address, source port, destination port, number of bytes per packet in the source flow, and number of bytes per packet in the destination flow;

[0057] Based on the information entropy of each network flow in the tuple data, the information entropy vector of the tuple data is obtained.

[0058] The information entropy vector represents the network state within a preset time window.

[0059] Optionally, in the fifth module, the information entropy features of the abnormal flow are determined based on the information entropy vector, specifically for:

[0060] Obtain the information entropy within the target time window as the baseline entropy;

[0061] By cross-processing the information entropy vector and the baseline entropy, the degree of network state deviation is determined, and the information entropy characteristics of abnormal flows are determined based on the degree of network state deviation.

[0062] On the other hand, embodiments of the present invention provide an electronic device, including: a processor and a memory; the memory is used to store a program; the processor executes the program to implement the above-described network feature analysis method.

[0063] On the other hand, embodiments of the present invention provide a computer storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described network feature analysis method.

[0064] This invention first analyzes abnormal network traffic using an exponential distribution function to obtain preliminary analysis results; then, it performs regression prediction on the abnormal network traffic to obtain fitted values; based on the preliminary analysis results, it determines the information entropy; this invention comprehensively analyzes the relevant characteristics of network anomalies through the combined application of fitted values ​​and information entropy, facilitating subsequent comprehensive comparative analysis; based on the comparison results of fitted values ​​and information entropy, it adjusts the exponential distribution function to obtain second analysis results; this invention optimizes the exponential distribution function through a comprehensive comparison of fitted values ​​and information entropy, and performs secondary analysis on the data related to abnormal network traffic, accurately filtering and analyzing the abnormal data to obtain more accurate analysis data; it labels the tuple data in the second analysis results to obtain the information entropy vector of the tuple data, and determines the information entropy characteristics of abnormal flow based on the information entropy vector; this invention locates and captures the information entropy characteristics of abnormal flow by labeling the tuples, so as to accurately reflect the changing trend of network flow information. Attached Figure Description

[0065] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0066] Figure 1 This is a schematic diagram of an implementation environment for network feature analysis provided in an embodiment of the present invention;

[0067] Figure 2 This is a flowchart illustrating a network feature analysis method provided in an embodiment of the present invention;

[0068] Figure 3 A schematic diagram of the regression prediction process provided in an embodiment of the present invention;

[0069] Figure 4 A schematic diagram of the information entropy quantization process provided for embodiments of the present invention;

[0070] Figure 5 This is a flowchart illustrating the process of adjusting the exponential distribution function according to an embodiment of the present invention.

[0071] Figure 6This is a schematic diagram of the process for obtaining the information entropy vector provided in an embodiment of the present invention;

[0072] Figure 7 A flowchart illustrating the process of determining the entropy characteristics of abnormal flow information provided in an embodiment of the present invention;

[0073] Figure 8 This is a schematic diagram of the structure of a network feature analysis device provided in an embodiment of the present invention;

[0074] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0075] Figure 10 A computer system architecture block diagram suitable for implementing electronic devices according to embodiments of the present invention is provided. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0077] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100," "second / S200," etc., in the specification, claims, and the aforementioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0078] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0079] It is understood that the network feature analysis method provided in this embodiment of the invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet computer, laptop computer, or desktop computer, but it is not limited to these.

[0080] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the invention. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0081] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0082] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0083] Terminal 102 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0084] Exemplary based on Figure 1 The implementation environment shown in this embodiment of the invention provides a network feature analysis method. The following description uses the application of this network feature analysis method in server 101 as an example. It can be understood that this network feature analysis method can also be applied to terminal 102.

[0085] Reference Figure 2 , Figure 2 This is a flowchart illustrating a network feature analysis method applied to a server, provided in an embodiment of the present invention. The execution subject of this network feature analysis method can be any of the aforementioned computer devices. (Refer to...) Figure 2 The method includes the following steps:

[0086] S100. Analyze abnormal network traffic using the exponential distribution function to obtain preliminary analysis results;

[0087] In some specific embodiments, the exponential distribution function is used to analyze data exhibiting abnormal flow characteristics. This function can represent the time interval between independent random events, such as the time interval between passengers entering an airport, the reference to an internet webpage link, and the time difference before / after the link is referenced. After analyzing and measuring the traffic load, it can be found that the network flow conforms to an exponential distribution. The exponential distribution function is described below:

[0088]

[0089] Where x is the time interval for statistical network traffic, and λ e This represents the network load traffic value within a time period. When x <= 0, the network load traffic value is zero; when x > 0, the time interval and the network load traffic value follow an exponential function distribution.

[0090] S200. Perform regression prediction on abnormal network traffic to obtain fitted values;

[0091] It should be noted that in some embodiments, such as Figure 3 As shown, regression prediction of abnormal network traffic to obtain fitted values ​​may include: S201, setting a regression prediction model based on ridge regression; S202, inputting abnormal network traffic into the regression prediction model, and combining preliminary analysis results and regularization terms to obtain fitted values; wherein, the regularization term is determined based on the fitting hyperparameters and weight constants.

[0092] In some specific implementations, a regression prediction model can be constructed, and abnormal network traffic can be fed into this model for computation to obtain a fitted value (loss function). Specifically, the ridge regression method can be used to detect the amount of data not involved in the model training process during the current exponential distribution function training. The larger the fitted value, the smaller the amount of data not involved in the effective training.

[0093] Ridge regression model formula: ||Xθ-y|| 2 +||Γθ|| 2

[0094] In the formula, X = input; y = output (prediction result); || = regularization.

[0095] Where Γ is defined as aI (objective training result = fitted value),

[0096] The process to prevent overfitting is as follows: θ(a) = (X T X+aI) -1 X T y and I are identity matrices.

[0097] Where X represents the input; y represents the predicted output; || represents the regularization operation; I represents the identity matrix; θ is the fitting hyperparameter; Γ is the weight constant; a is the weight of the identity matrix; and θ(a) represents the value to be calculated under certain conditions.

[0098] S300. Based on the preliminary analysis results, determine the information entropy;

[0099] It should be noted that in some embodiments, such as Figure 4 As shown, based on the preliminary analysis results, the information entropy is determined, including: S301, obtaining the prior probability of each piece of information or event in the preliminary analysis results; S302, obtaining the information content of each piece of information or event based on the prior probability through a logarithmic function; S303, summing the expected information content of each piece of information or event to obtain the information entropy; wherein, the information entropy is negatively correlated with the accuracy of the preliminary analysis results.

[0100] It should also be noted that information entropy, also known as Shannon entropy, represents the mathematical probability of a specific piece of information or event. In human-computer systems that process data, it is used to represent the uncertainty of the information itself. Because there are many uncertainties in data streams, information entropy is very suitable as a research object for network flows. The more certain an piece of information or event is, the lower its information entropy; conversely, the more uncertain it is, the higher its information entropy.

[0101] For example: if the probability of a certain piece of information or event occurring is 100%, then the information content is zero. If the probability of a certain piece of information or event occurring is zero, then the information content is maximized. Let the piece of information or event be x. i Its prior probability is p(x) i If ), then its information content is:

[0102] H(X i ) = -log e p(x i )

[0103] Where e is the base of the logarithm, typically 2. The above formula represents the information content of a single event. However, when describing the average information content of a set of information or events, information entropy is needed.

[0104] Let the event set X = (x1, x2, x3, ..., xn) nThe probabilities of occurrence of each event are P = (p(x1), p(x2), p(x3), ..., p(x4)). n Then the information entropy H(X) is defined as:

[0105]

[0106] S400. Based on the comparison results of the fitted values ​​and information entropy, adjust the exponential distribution function and obtain the second analysis result;

[0107] It should be noted that in some embodiments, such as Figure 5 As shown, the exponential distribution function is adjusted based on the comparison results of the fitted value and the information entropy, including: S401, when the difference between the fitted value and the information entropy is greater than the first preset ratio, the absolute value of the difference between the fitted value and the information entropy is obtained, and then the differential data is determined; S402, the differential data is weighted and averaged within the second preset ratio to obtain a reference value; wherein, the reference value is used to adjust the loss function of the exponential distribution function.

[0108] In some embodiments, the method may further include: removing differential data outside a second preset ratio and performing a weighted average on the remaining data; wherein, when the ratio of the removed data to the remaining data is greater than a third preset ratio, differential data within a preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data; regression prediction is performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference data between the actual fitted value and the simulated fitted value is determined; the difference data within a first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

[0109] In some embodiments, a second analysis result is obtained based on the excluded data using an adjusted exponential distribution function.

[0110] In some specific embodiments, the fitted value calculated by the regression prediction model (the smaller the fitted value, the more accurate the calculation result of the exponential distribution function) is innovatively compared with the information entropy value through the aforementioned steps (the lower the information entropy value, the more accurate the calculation result of the exponential distribution function).

[0111] Finally, the results of the exponential distribution function calculation fall into three categories:

[0112] Scenario 1: This is the optimal result calculated using the exponential distribution function. That is, both the fitted value and the information entropy are relatively small.

[0113] Scenario 2: When there is a large difference between the fitted value and the information entropy, the following processing steps can be performed:

[0114] First, obtain the absolute value of the difference between the two values, and then take a weighted average within 10% as the loss function and a reference value for the accuracy of the calculation result of the exponential distribution function.

[0115] Secondly, the data representing the majority of the differences are removed and then recalculated, and processed using a weighted average method.

[0116] Then, if the proportion of data to be calculated based on significant differences is too large, for example, |10%-40%|=30%, use 30% of the data as the actual dataset, and at the same time, use historical data of the same time period and type as the simulated dataset.

[0117] Finally, the two datasets are fed into the regression prediction model to obtain actual and simulated fitted values. Data with a difference of less than 10% between the fitted values ​​are used in the actual calculation, thereby increasing the accuracy of the exponential distribution function calculation results by reducing the amount of data processed. Otherwise, outlier data is discarded.

[0118] Scenario 3: This is the worst-case scenario for the exponential distribution function calculation, where both the fitted value and the information entropy are relatively large. In this case, a weighted average of the fitted value and the information entropy can be used as the loss function for the exponential distribution function calculation.

[0119] S500: Label the tuple data in the second analysis result, obtain the information entropy vector of the tuple data, and determine the information entropy characteristics of the abnormal flow based on the information entropy vector;

[0120] It should be noted that in some embodiments, such as Figure 6 As shown, the data in the second analysis result is labeled to obtain the information entropy vector of the data in ...

[0121] In some embodiments, such as Figure 7 As shown, the abnormal flow information entropy characteristics are determined based on the information entropy vector, including: S521, obtaining the information entropy within the target time window as the baseline entropy; S522, determining the degree of network state deviation by entropy cross-processing the information entropy vector and the baseline entropy, and determining the abnormal flow information entropy characteristics based on the degree of network state deviation.

[0122] In some specific embodiments, network traffic analysis using a six-tuple (i.e., a tuple including source address, destination address, source port, destination port, number of bytes per packet in the source flow, and number of bytes per packet in the destination flow) combined with an information entropy algorithm can be used to label the six-tuple data, and then perform time series graph temporal feature analysis on the labeled six-tuple data. The implementation steps are as follows:

[0123] S1. Select six time-series dimensions that best reflect the changing trends of network flow to represent network security trends:

[0124] Source address (SrcIP);

[0125] Destination IP address;

[0126] Source port (SrcPort);

[0127] Destination Port;

[0128] Number of bytes per packet in the source stream (SrcBpp);

[0129] Number of bytes per packet in the target stream (DestBpp).

[0130] It should be noted that information entropy has the following mathematical properties:

[0131] Nonnegativity, H(X)≥0. H(X)=0 if and only if the probability of a single event P(X) is 1.

[0132] When all events have the same probability of occurrence, that is, when P(X) = 1 / n, the information entropy reaches its maximum value log2n.

[0133] The amount of information about multiple events in entropy (x1, x2, x3, ..., x n The probability components p(x1), p(x2), p(x3), ..., p(x4) satisfy the commutative law and therefore have the following probability components: p(x5), p(x6), p(x7), p(x8), p(x9), p(x1), p(x2), p(x3), ..., p(x9), p(x1), p(x2), p(x3), p(x4), p(x5), p(x6), p(x7), p(x8), n The swapping of positions does not affect the value of information entropy, which indicates that information entropy focuses on the overall distribution of the probability of an event occurring, rather than on a specific event.

[0134] P and Q are discrete distributions of the event set, p i q i Let P and Q be the distribution functions of probabilities. To measure the degree to which distribution P differs from distribution Q in a statistically discriminative sense, the entropy crossover algorithm is defined as:

[0135]

[0136] The entropy crossover algorithm has the following mathematical properties:

[0137] La(P,Q)≤0.

[0138] If P = Q or , then La(P,Q) = 0.

[0139] The smaller La(P,Q) is, the more information is needed to distinguish between P and Q.

[0140] If we directly use the statistical values ​​of 6 network flows, we cannot accurately reflect the manifestation pattern of abnormal network behavior in the traffic. Therefore, we use information entropy as a metric. For ease of calculation, we take the logarithm base to be 2, as shown in the following formula:

[0141]

[0142] If data is concentrated at a single point, meaning all data has the same value, the information entropy is 0; conversely, if the data is widely distributed, the information entropy is large. For example, if malware scans all ports of a host, the target port information entropy is large; if it scans the same port across the entire network, the target port information entropy is small. The network flow information entropy vector is defined as follows:

[0143] E(t)=[H i (SrcIP),H i (DestIP),H i (SrcPort),H i (DestPort),H i (SrcBpp),H i (DestBpp)]

[0144] This vector reflects the network state within the time window t. The time series of the information entropy vector, E(1), E(2), E(3), E(4), E(5), etc., can reflect the network state over a period of time. A period of stable network operation can be taken as the baseline entropy, as shown in the following formula, where n is a sequence of multiple normal time windows of the network flow, which can be obtained by learning the information entropy values ​​within the normal time windows of the network.

[0145]

[0146] To distinguish whether the network status is normal within a certain time window, the entropy crossover algorithm can be used for judgment. To simplify the calculation, α is set to 0.5, as shown in the following formula:

[0147]

[0148] By calculating the relative information between the current observation point and the normal point, we can determine the degree to which the current state deviates from the normal state and judge whether the current state is normal, as shown in the following formula, where Current is the network flow distribution function in the current time window, Normal is the network flow distribution function in the normal time window, and Threshold is the basic threshold (the value can be determined by experimental methods).

[0149] |L 0.5 (Current,Normal)|>Threshold

[0150] The design of information entropy uses colors of the same color scheme to represent the same data dimension, with IP addresses in red, ports in green, the number of bytes per packet in blue, gray vertical lines representing time windows, and red vertical lines representing network anomalies.

[0151] S2. By analyzing the tree diagram, the spatial characteristics of network flow within a certain time window can be discovered based on its features. However, for a longer period of network flow security situation, the tree diagram is not as intuitive as the time series diagram. The following section uses time series diagram features for qualitative analysis, as shown in Table 1 (Features of Abnormal Flow Information Entropy). In the table, "↓" indicates a decreasing trend in the network flow information entropy curve, "↑" indicates an increasing curve, and "—" indicates no significant change.

[0152] Table 1

[0153]

[0154]

[0155] For example, if malware or a hacker scans the same port on all hosts in a network (multi-target host single-port scan), the source network addresses within the window period will be relatively concentrated on the hacker's host, and the destination host will have a large number of identical scanned ports. Because the target network addresses are scanned in an orderly manner across all hosts, the SrcIP entropy will decrease, the DestPort entropy will be very small, while the DestP entropy will increase. If malware or a host scans different ports on all hosts in a network (multi-target host multi-port scan), the target ports and target IPs become very wide and ordered, resulting in a decrease in SrcIP entropy and an increase in DestPort and DestIP entropy. If a single-source denial-of-service attack occurs, the attacking source host will generate a large number of connection request packets sent to the destination host, which will inevitably cause a decrease in the SrcIP, DestP, and Destport entropy. If the network flow is normal, all entropy values ​​change relatively smoothly.

[0156] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0157] First, it's important to note that existing network security products are proving insufficient in their detection capabilities due to the dramatic increase in network data volume and attack types. Analyzing massive amounts of log information to detect network anomalies is no longer effective. In this context, network security visualization technology has emerged to meet network security requirements. Network security visualization is an emerging, multidisciplinary research field. It combines network security situational awareness with visualization techniques. Leveraging human vision's ability to perceive models and structures, it presents abstract network and system data graphically, helping analysts quickly and accurately analyze log files to identify network status and anomalies, and predict network security trends. It is a crucial tool for addressing network information security issues.

[0158] This invention targets abnormal network traffic by employing an exponential distribution function combined with an information entropy algorithm to predict abnormal network flows. Simultaneously, it uses a network traffic analysis six-tuple approach combined with the information entropy algorithm to label the six-tuple data, and then performs time series graph temporal feature analysis on the labeled six-tuple data. It innovatively uses a regression algorithm combined with the information entropy algorithm to accurately filter and analyze abnormal data, thus providing a method for analyzing the spatiotemporal characteristics of network flows. Furthermore, the embodiments of this invention can accurately reflect the network feature analysis results by combining network security visualization technology.

[0159] In some specific embodiments, network feature analysis can be achieved through the following process steps:

[0160] Step 1: For abnormal network traffic, an exponential distribution function combined with an information entropy algorithm is used to predict abnormal network flows. Simultaneously, an innovative regression prediction model is employed to calculate a fitted value. The smaller the fitted value, the more accurate the exponential distribution function calculation result, which is then compared with the information entropy value. When there is a significant difference between the fitted value and the information entropy, the data with the larger difference and historical simulated data of the same type are again fed into the regression prediction model to obtain data with smaller differences in the fitted values, which are then used in the actual calculation. This increases the accuracy of the exponential distribution function calculation result by reducing the amount of data processed. Conversely, abnormal data is discarded.

[0161] Step 2: By combining network traffic analysis with the information entropy algorithm, the six-tuple data is labeled, and the labeled six-tuple data is subjected to time series graph temporal feature analysis.

[0162] In summary, this invention innovatively employs a regression algorithm combined with an information entropy algorithm to accurately filter and analyze abnormal data, thereby completing a method for spatiotemporal feature analysis of network flows. This invention can perform network feature analysis efficiently and accurately.

[0163] On the other hand, such as Figure 8 As shown, this embodiment of the invention provides a network feature analysis device 800, comprising: a first module 810, used to analyze abnormal network traffic using an exponential distribution function to obtain preliminary analysis results; a second module 820, used to perform regression prediction on the abnormal network traffic to obtain fitted values; a third module 830, used to determine information entropy based on the preliminary analysis results; a fourth module 840, used to adjust the exponential distribution function based on the comparison results of the fitted values ​​and information entropy, and obtain second analysis results; and a fifth module 850, used to label the tuple data in the second analysis results to obtain the information entropy vector of the tuple data, and determine the abnormal flow information entropy features based on the information entropy vector.

[0164] It should be noted that, in some embodiments, the device further includes the following modules:

[0165] The sixth module is used to remove differential data outside the second preset ratio and to perform a weighted average on the remaining data;

[0166] Specifically, when the ratio of the removed data to the remaining data is greater than the third preset ratio, the differential data within the preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data.

[0167] Regression predictions are performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference between the actual fitted value and the simulated fitted value is determined.

[0168] The difference data within the first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

[0169] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0170] On the other hand, such as Figure 9 As shown, this embodiment of the invention also provides an electronic device 900, which includes at least one processor 910 and at least one memory 920 for storing at least one program; taking one processor 910 and one memory 920 as an example.

[0171] The processor 910 and memory 920 can be connected via a bus or other means.

[0172] Memory 920, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 920 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 920 may optionally include memory remotely located relative to the processor, and this remote memory can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0173] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] Specifically, Figure 10 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present invention is shown.

[0175] It should be noted that, Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0176] like Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage section 1008 into random access memory (RAM). The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output interface 1005 (I / O interface) is also connected to the bus 1004.

[0177] The following components are connected to the input / output interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a local area network card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0178] In particular, according to embodiments of the present invention, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit 1001, it performs various functions defined in the system of the present invention.

[0179] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein computer-readable program code is carried. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0180] The content of the method embodiments of the present invention is applicable to the system embodiments. The specific functions implemented in the system embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0181] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.

[0182] The content of the method embodiments of the present invention is applicable to the computer-readable storage medium embodiments. The specific functions implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0183] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0184] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0185] It should be noted that although several modules for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0186] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0187] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0188] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0189] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution means, apparatus, or device (such as a computer-based device, a processor-including device, or other means that can fetch and execute instructions from, or in conjunction with, an instruction execution means, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution means, apparatus, or device.

[0191] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0192] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0193] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0194] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0195] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A network feature analysis method, characterized in that, include: Preliminary analysis results were obtained by analyzing abnormal network traffic using the exponential distribution function. Regression prediction is performed on the abnormal network traffic to obtain fitted values; Based on the preliminary analysis results, the information entropy is determined; Based on the comparison results between the fitted value and the information entropy, the exponential distribution function is adjusted, and a second analysis result is obtained; The tuple data in the second analysis result are labeled to obtain the information entropy vector of the tuple data, and the information entropy characteristics of the abnormal flow are determined based on the information entropy vector. The step of adjusting the exponential distribution function based on the comparison result of the fitted value and the information entropy includes: When the difference between the fitted value and the information entropy is greater than a first preset ratio, the absolute value of the difference between the fitted value and the information entropy is obtained, thereby determining the differentiated data; The differentiated data are weighted and averaged within a second preset ratio to obtain a reference value; wherein the reference value is used to adjust the loss function of the exponential distribution function; The method further includes: Differential data outside the second preset ratio are removed, and the remaining data are weighted and averaged. Specifically, when the ratio of the removed data to the remaining data is greater than a third preset ratio, differential data within a preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data. Regression predictions are performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference data between the actual fitted value and the simulated fitted value is determined. The difference data within the first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

2. The network feature analysis method according to claim 1, characterized in that, The regression prediction of the abnormal network traffic to obtain fitted values ​​includes: A regression prediction model is set up based on ridge regression; The abnormal network traffic is input into the regression prediction model, and the fitted value is obtained by combining the preliminary analysis results and the regularization term; wherein, the regularization term is determined based on the fitting hyperparameters and weight constants.

3. The network feature analysis method according to claim 1, characterized in that, The step of determining information entropy based on the preliminary analysis results includes: Obtain the prior probability of each piece of information or event in the preliminary analysis results; Based on the prior probabilities, the information content of each piece of information or event is obtained through a logarithmic function; The information entropy is obtained by summing the expected information content of each piece of information or event. The information entropy is negatively correlated with the accuracy of the preliminary analysis results.

4. The network feature analysis method according to claim 1, characterized in that, The step of labeling the tuple data in the second analysis result to obtain the information entropy vector of the tuple data includes: Each network flow in the tuple data in the second analysis result is labeled; wherein, the network flow in the tuple data includes source address, destination address, source port, destination port, number of bytes per packet in the source flow, and number of bytes per packet in the destination flow; Based on the information entropy of each network flow in the tuple data, the information entropy vector of the tuple data is obtained. The information entropy vector represents the network state within a preset time window.

5. The network feature analysis method according to claim 1, characterized in that, The step of determining the abnormal flow information entropy features based on the information entropy vector includes: Obtain the information entropy within the target time window as the baseline entropy; By performing entropy cross-processing on the information entropy vector and the baseline entropy, the degree of network state deviation is determined, and the abnormal flow information entropy characteristics are determined based on the degree of network state deviation.

6. A network feature analysis device, characterized in that, include: The first module is used to analyze abnormal network traffic using the exponential distribution function to obtain preliminary analysis results; The second module is used to perform regression prediction on the abnormal network traffic to obtain a fitted value; The third module is used to determine the information entropy based on the preliminary analysis results; The fourth module is used to adjust the exponential distribution function based on the comparison result between the fitted value and the information entropy, and to obtain the second analysis result; The fifth module is used to label the tuple data in the second analysis result, obtain the information entropy vector of the tuple data, and determine the information entropy characteristics of the abnormal flow based on the information entropy vector. Specifically, the step of adjusting the exponential distribution function based on the comparison result of the fitted value and the information entropy in the fourth module is used for: When the difference between the fitted value and the information entropy is greater than the first preset ratio, the absolute value of the difference between the fitted value and the information entropy is obtained, and then the differential data is determined. The differentiated data are weighted and averaged within a second preset ratio to obtain a reference value; the reference value is used to adjust the loss function of the exponential distribution function. The device also includes a sixth module, used for: Differential data outside the second preset ratio are removed, and the remaining data are weighted and averaged. Specifically, when the ratio of the removed data to the remaining data is greater than the third preset ratio, the differential data within the preset range is extracted as the actual dataset; and a simulated dataset is obtained based on historical network data. Regression predictions are performed on the actual dataset and the simulated dataset respectively to obtain the actual fitted value and the simulated fitted value, and the difference between the actual fitted value and the simulated fitted value is determined. The difference data within the first preset ratio is used for calculation and restoration, and the difference data outside the first preset ratio is removed.

7. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 5.

8. A computer storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1 to 5.

Images