A data intelligent backup method and system based on data security analysis

Abstract

The present invention relates to the field of data processing technology, and in particular to a data intelligent backup method and system based on data security analysis. The method obtains a sliding window with multidimensional network flow data as the last data in real time, and uses an incremental PCA method to obtain multidimensional reconstruction data corresponding to each multidimensional network flow data in the sliding window; according to the multidimensional reconstruction data, an adaptive weight and a reconstruction error threshold corresponding to each dimension in the multidimensional network flow data are obtained, and a comprehensive anomaly score of the multidimensional network flow data is obtained according to the adaptive weight, the reconstruction error threshold and a maximum value function; according to the comprehensive anomaly score of the multidimensional network flow data, anomaly detection is performed on the multidimensional network flow data to obtain anomaly detection results, and intelligent data backup of the network data is performed according to the anomaly detection results, thereby ensuring that key data can be protected in time when potential security threats are detected.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data intelligent backup method and system based on data security analysis. Background Art

[0002] As an effective dimensionality reduction and feature extraction technology, the principal component analysis (PCA) method has been widely used in the field of network traffic anomaly detection. The PCA method can extract the main features from high-dimensional network traffic data, which helps to identify network anomaly patterns. At present, the method of using the PCA method for network traffic anomaly detection is anomaly detection based on reconstruction error. This method first performs incremental PCA dimensionality reduction on the network traffic data, then reprojects the reduced data into the original space, calculates the error between the original data and the reconstructed data, and if the error exceeds the preset threshold, it is determined to be abnormal data. This method can effectively detect abnormal traffic data that is significantly different from normal network behavior patterns.

[0003] However, the traditional incremental PCA reconstruction error calculation is to calculate the overall error through the sum of squares. In the cloud environment, some attacks or failures sometimes only show abnormalities in a short time or on specific network features. The traditional incremental PCA will treat all features equally, and thus it is difficult to effectively capture these instantaneous or local anomalies, and will smooth out some local abnormal signals.

[0004] Therefore, how to optimize the use of traditional incremental PCA's insufficient sensitivity to short-term local anomalies in cloud computing environments and its inability to distinguish the importance differences of different network features to improve the accuracy of network traffic anomaly detection has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a data intelligent backup method and system based on data security analysis to solve the problem that traditional incremental PCA is insufficiently sensitive to transient local anomalies in cloud computing environments and cannot distinguish the importance differences of different network features, so as to improve the accuracy of network traffic anomaly detection.

[0006] In a first aspect, an embodiment of the present invention provides a data intelligent backup method based on data security analysis, the method comprising the following steps:

[0007] Acquire multidimensional network flow data at any sampling time in real time, obtain a sliding window with the multidimensional network flow data as the last data, and use the incremental PCA method to obtain multidimensional reconstruction data corresponding to each multidimensional network flow data in the sliding window;

[0008] For the data corresponding to any dimension in the multidimensional network traffic data, a reconstruction error threshold of the data corresponding to the dimension is obtained according to the data corresponding to the dimension and the multidimensional reconstruction data corresponding to each multidimensional network traffic data in the sliding window, and an adaptive weight of the data corresponding to the dimension is obtained according to the data corresponding to the dimension of each multidimensional network traffic data in the sliding window;

[0009] According to the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data, and the corresponding multidimensional reconstruction data, the reconstruction error of the multidimensional network traffic data is optimized using a maximum function to obtain a comprehensive anomaly score of the multidimensional network traffic data;

[0010] According to the comprehensive anomaly score of the multi-dimensional network traffic data, anomaly detection is performed on the multi-dimensional network traffic data to obtain anomaly detection results, and intelligent data backup of the network data is performed according to the anomaly detection results.

[0011] Preferably, obtaining the reconstruction error threshold of the data corresponding to the dimension according to the data corresponding to the dimension and the multidimensional reconstructed data corresponding to each multidimensional network traffic data in the sliding window includes:

[0012] Respectively obtain the first difference absolute value of each multidimensional network traffic data in the sliding window and the corresponding multidimensional reconstructed data under the dimension, and obtain the first median, the first quartile and the third quartile of all the first difference absolute values;

[0013] Obtaining in the sliding window respectively the second difference absolute values ​​of the data of two adjacent multi-dimensional network flow data under the dimension, obtaining the second median of all the second difference absolute values, and calculating the first ratio between the second difference absolute value corresponding to the multi-dimensional network flow data and the second median;

[0014] The sum of the constant 1 and the first ratio is used as the independent variable of the preset logarithmic function to obtain the time continuity index corresponding to the dimension, the difference between the first quartile and the third quartile is calculated, the product of the difference and the time continuity index is obtained, and the reconstruction error threshold of the data corresponding to the dimension is obtained according to the product and the first median.

[0015] Preferably, obtaining the adaptive weight of the data corresponding to the dimension according to the data corresponding to the dimension of each multi-dimensional network traffic data in the sliding window includes:

[0016] Obtain the data variance corresponding to each dimension in the sliding window respectively, obtain the third median of all data variances, calculate the absolute value of the difference between the data variance corresponding to each dimension and the third median respectively, and obtain the fourth median of all the absolute values ​​of the difference;

[0017] A second ratio between the absolute value of the difference of the data corresponding to the dimension and the fourth median is calculated, and the second ratio is positively mapped using a preset exponential function to obtain an adaptive weight of the data corresponding to the dimension.

[0018] Preferably, the method of optimizing the reconstruction error of the multidimensional network traffic data using a maximum function according to the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data, and the corresponding multidimensional reconstruction data, to obtain a comprehensive anomaly score of the multidimensional network traffic data includes:

[0019] According to the absolute value of the data difference between the multidimensional network traffic data and the corresponding multidimensional reconstructed data in any dimension, a reconstruction error of the data corresponding to the dimension is obtained, the difference between the reconstruction error and a reconstruction error threshold of the data corresponding to the dimension is obtained, and a maximum value function is used to obtain the maximum value between the difference and a preset value as the optimized reconstruction error of the data corresponding to the dimension;

[0020] The optimized reconstruction error of the data corresponding to each dimension in the multidimensional network traffic data is obtained, and according to the adaptive weight of the data corresponding to each dimension in the multidimensional network traffic data, all the optimized reconstruction errors are weighted summed to obtain a comprehensive anomaly score of the multidimensional network traffic data.

[0021] Preferably, performing anomaly detection on the multi-dimensional network traffic data according to the comprehensive anomaly score of the multi-dimensional network traffic data to obtain anomaly detection results includes:

[0022] A maximum anomaly score threshold is obtained according to the updated anomaly score threshold. If the comprehensive anomaly score of the multi-dimensional network traffic data is greater than the maximum anomaly score threshold, a network anomaly is marked at the sampling moment of the multi-dimensional network traffic data.

[0023] Preferably, performing anomaly detection on the multi-dimensional network traffic data according to the comprehensive anomaly score of the multi-dimensional network traffic data to obtain anomaly detection results further includes:

[0024] A new sliding window is constructed with the multidimensional network traffic data as the last data, the new sliding window is smaller than the sliding window, and an average comprehensive anomaly score is obtained according to the comprehensive anomaly score of each multidimensional network traffic data in the new sliding window. If the average comprehensive anomaly score is greater than the updated anomaly score threshold, it is marked that a network anomaly occurs in the time period corresponding to the new sliding window.

[0025] Preferably, the method for obtaining the updated anomaly score threshold comprises:

[0026] Obtain a comprehensive anomaly score for each multidimensional network traffic data within a preset threshold update duration, obtain a preset percentile of all comprehensive anomaly scores, obtain an anomaly score threshold before updating, perform weighted summation on the preset percentile and the anomaly score threshold before updating, and obtain an updated anomaly score threshold.

[0027] Preferably, the intelligent data backup of network data according to the abnormality detection result includes:

[0028] If the anomaly detection result is that a network anomaly occurs at the sampling moment of the multi-dimensional network traffic data, data backup is performed on the network traffic data of all dimensions within a first time period formed before and after the sampling moment of the multi-dimensional network traffic data.

[0029] Preferably, the intelligent data backup of network data according to the abnormality detection result includes:

[0030] If the anomaly detection result is that a network anomaly occurs in the time period corresponding to the new sliding window, data backup is performed on network traffic data of all dimensions within a second time period formed before and after the sampling moment of the multi-dimensional network traffic data, wherein the first time period is smaller than the second time period.

[0031] In a second aspect, an embodiment of the present invention provides a data intelligent backup system based on data security analysis, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned data intelligent backup method based on data security analysis.

[0032] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0033] The present invention acquires multidimensional network flow data at any sampling time in real time, obtains a sliding window with the multidimensional network flow data as the last data, and uses an incremental PCA method to acquire multidimensional reconstruction data corresponding to each multidimensional network flow data in the sliding window; for data corresponding to any dimension in the multidimensional network flow data, according to the data corresponding to the dimension and the multidimensional reconstruction data corresponding to each multidimensional network flow data in the sliding window, acquires a reconstruction error threshold of the data corresponding to the dimension, and according to the data corresponding to the dimension of each multidimensional network flow data in the sliding window, acquires an adaptive weight of the data corresponding to the dimension; according to the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network flow data, and the corresponding multidimensional reconstruction data, optimizes the reconstruction error of the multidimensional network flow data by using a maximum function to acquire a comprehensive anomaly score of the multidimensional network flow data; according to the comprehensive anomaly score of the multidimensional network flow data, performs anomaly detection on the multidimensional network flow data to obtain anomaly detection results, and performs intelligent data backup of network data according to the anomaly detection results. Among them, when reconstructing the error of multi-dimensional network traffic data, the reconstruction error of each dimension is optimized by introducing a dynamic reconstruction error threshold and a maximum value function. At the same time, by introducing the adaptive weight of dynamic features, the importance of the data of each dimension in anomaly detection based on reconstruction error is adjusted, so that the comprehensive anomaly score of the multi-dimensional network traffic data can capture local significant changes while retaining the sensitivity to global anomalies, thereby enhancing the accuracy of anomaly detection using the comprehensive anomaly score of the multi-dimensional network traffic data, thereby performing intelligent data backup operations according to the anomaly detection results, ensuring that critical data can be protected in time when potential security threats are detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 It is a method flow chart of a data intelligent backup method based on data security analysis provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0037] It should be noted that the terms "first", "second", etc. in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0038] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0039] See also Figure 1 , is a method flow chart of a data intelligent backup method based on data security analysis provided by the first embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0040] Step S101, acquiring multidimensional network traffic data at any sampling time in real time, obtaining a sliding window with the multidimensional network traffic data as the last data, and using the incremental PCA method to obtain multidimensional reconstructed data corresponding to each multidimensional network traffic data in the sliding window.

[0041] Data collection probes are deployed at key nodes in the cloud computing environment, and the sampling frequency is set to 1 second, 2 seconds, and 5 seconds. There is no restriction here. For example, network traffic data is collected in real time through network traffic mirroring, NetFlow and other technologies. The network traffic data includes network traffic statistics, connection information, data packet statistics, session information, protocol distribution, error statistics, system resource usage and other multi-dimensional features. Therefore, each sampling moment can collect network traffic data of multiple dimensions, which is recorded as multi-dimensional network traffic data.

[0042] PCA (Principal Component Analysis) is the most common data dimensionality reduction method. PCA linearly transforms the original data, finds the principal component with the largest information content of the original data, removes the components with lower information content, thereby reducing redundancy and noise. Therefore, in the prior art, the reconstruction error of incremental PCA is usually used for anomaly detection. Among them, the basic principle of incremental PCA is to gradually build a principal component analysis (PCA) model, process one data point or a batch of data points each time, and update the model parameters. This method does not need to recalculate the covariance matrix of the entire data set, thereby improving computational efficiency. During each update process, incremental PCA calculates the projection of each new data point on the principal component and calculates the reconstruction error, which represents the degree of difference between the data point and the principal component model.

[0043] A reconstruction error threshold is set. When the reconstruction error exceeds the reconstruction error threshold, the multidimensional data is considered to be abnormal data. Therefore, after obtaining the multidimensional network flow data at the i-th sampling moment in real time, a sliding window with the multidimensional network flow data at the i-th sampling moment as the last data is constructed. The sliding window contains the multidimensional network flow data collected within 5 minutes. After obtaining the sliding window, the data in the sliding window is preprocessed, including data cleaning or standardization. Then, the incremental PCA method is used to reconstruct the data of the preprocessed sliding window, and the multidimensional reconstructed data corresponding to each multidimensional network flow data in the sliding window can be obtained, thereby obtaining the multidimensional reconstructed data of the multidimensional network flow data at the i-th sampling moment. It should be noted that the incremental PCA method for obtaining reconstructed data belongs to the prior art and will not be repeated here.

[0044] Step S102, for the data corresponding to any dimension in the multidimensional network traffic data, obtain the reconstruction error threshold of the data corresponding to the dimension according to the data corresponding to the dimension and the multidimensional reconstruction data corresponding to each multidimensional network traffic data in the sliding window, and obtain the adaptive weight of the data corresponding to the dimension according to the data corresponding to the dimension of each multidimensional network traffic data in the sliding window.

[0045] It is known that when using incremental PCA based on reconstruction error for anomaly detection, the traditional calculation formula for reconstruction error is: ,in, is the original data, It is the reconstructed data, but the traditional reconstruction error is not sensitive enough to local anomalies in the cloud computing environment, and cannot effectively distinguish the importance differences of different network features. That is, the traditional reconstruction error calculation formula uses the sum of squares to calculate the overall error, resulting in a smoothing effect that will mask some local anomalies. Secondly, all network features will be treated equally, ignoring the importance differences of different network features in anomaly detection.

[0046] Therefore, in order to solve the above problems, an embodiment of the present invention optimizes the reconstruction error of the multidimensional network traffic data at the i-th sampling moment based on the data in the sliding window and the multidimensional reconstruction data of the multidimensional network traffic data at the i-th sampling moment, so as to improve the analysis accuracy of the reconstruction error.

[0047] First, considering that in actual network traffic data, most network features may remain relatively stable, and only a few network features experience short-term significant changes, and the calculation of the sum of squares in the traditional reconstruction error will "smooth" such local significant changes into the overall error, resulting in the masking of short-term but important anomalies. Suppose that 100 different network features are monitored, 99 of which remain normal, but one network feature (such as the traffic of a specific port) suddenly surges 10 times. In the calculation of the sum of squares, this significant local anomaly will be masked by the small fluctuations of the other 99 normal network features. Therefore, in the embodiment of the present invention, a local reconstruction error threshold is introduced, that is, a reconstruction error threshold is obtained for the data corresponding to each dimension in the multi-dimensional network traffic data at the i-th sampling moment, which can increase the sensitivity of the local anomaly amplitude and effectively distinguish normal small fluctuations from real abnormal surges. Therefore, in the abnormal analysis of network traffic data, more attention can be paid to significant feature deviations beyond the normal range, rather than all small changes.

[0048] Specifically, taking the data corresponding to any dimension in the multi-dimensional network traffic data at the i-th sampling moment as an example, the method for obtaining the reconstruction error threshold of the data corresponding to the dimension is:

[0049] Respectively obtain the first difference absolute value of each multidimensional network traffic data in the sliding window and the corresponding multidimensional reconstructed data under the dimension, and obtain the first median, the first quartile and the third quartile of all the first difference absolute values;

[0050] Obtaining in the sliding window respectively the second difference absolute values ​​of the data of two adjacent multi-dimensional network flow data under the dimension, obtaining the second median of all the second difference absolute values, and calculating the first ratio between the second difference absolute value corresponding to the multi-dimensional network flow data and the second median;

[0051] The sum of the constant 1 and the first ratio is used as the independent variable of the preset logarithmic function to obtain the time continuity index corresponding to the dimension, the difference between the first quartile and the third quartile is calculated, the product of the difference and the time continuity index is obtained, and the reconstruction error threshold of the data corresponding to the dimension is obtained according to the product and the first median.

[0052] In one embodiment, for the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment, the incremental PCA method is first used to obtain the multidimensional reconstructed data corresponding to each multidimensional network traffic data in the sliding window, and the first absolute value of the difference between the data corresponding to the kth dimension in each multidimensional network traffic data in the sliding window and the data corresponding to the kth dimension in the corresponding multidimensional reconstructed data is obtained, which is recorded as the first absolute value of the difference corresponding to the j-th multidimensional network traffic data in the sliding window on the kth dimension. ,in, Represents the data corresponding to the kth dimension in the jth multidimensional network traffic data in the sliding window, Represent the data corresponding to the kth dimension in the multidimensional reconstructed data corresponding to the jth multidimensional network traffic data, obtain the median median1, the first quartile Q1 and the third quartile Q3 of all the first difference absolute values, and at the same time, according to the data corresponding to the kth dimension in each multidimensional network traffic data in the sliding window, obtain the absolute value of the difference between the data corresponding to the kth dimension in two adjacent multidimensional network traffic data, and record the data corresponding to the kth dimension in the jth multidimensional network traffic data in the sliding window. and the data corresponding to the kth dimension in the j-1th multidimensional network traffic data in the sliding window The second absolute value of the difference between , calculate the median2 of the absolute values ​​of all second differences, then the reconstruction error threshold of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling time is:

[0053]

[0054] in, represents the reconstruction error threshold of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling time, median1 represents the median of the absolute values ​​of all first differences, IQR(|Q1-Q3|) represents the interquartile range between the first quartile and the third quartile, log() represents the logarithmic function with a constant of 2 as the base, 1 represents the constant, represents the second absolute value of the difference between the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment and the data corresponding to the kth dimension in the multidimensional network traffic data at the i-1-th sampling moment, and median2 represents the median of all the second absolute values ​​of the difference.

[0055] It should be noted that by setting the reconstruction error threshold of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment through the median median1 and the interquartile range IQR (|Q1-Q3|), the sensitivity to different network features can be adaptively adjusted. It is used to characterize the relative change rate of the data corresponding to the kth dimension at the ith sampling moment relative to the data corresponding to the kth dimension at the historical sampling moment, so that the interquartile range IQR (|Q1-Q3|) can more accurately capture the mutation of the data of the kth dimension in continuous time, thereby enhancing the local sensitivity of the kth dimension.

[0056] Similarly, the reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data at the i-th sampling moment can be obtained, and then, the adaptive weights of different network features are further introduced. By assigning appropriate weights to different network features, the importance of each network feature in anomaly detection can be automatically adjusted, and the sensitivity of changes in each network feature is improved. Therefore, in an embodiment of the present invention, an adaptive weight is obtained for the data corresponding to each dimension in the multidimensional network traffic data at the i-th sampling moment to enhance network features with large fluctuations or abnormal fluctuations and reduce network features with large long-term fluctuations, so that when the reconstruction error is subsequently calculated, a reconstruction error result that is more in line with the scenario can be obtained.

[0057] Specifically, taking the data corresponding to any dimension in the multi-dimensional network traffic data at the i-th sampling moment as an example, the method for obtaining the adaptive weight of the data corresponding to the dimension is:

[0058] Obtain the data variance corresponding to each dimension in the sliding window respectively, obtain the third median of all data variances, calculate the absolute value of the difference between the data variance corresponding to each dimension and the third median respectively, and obtain the fourth median of all the absolute values ​​of the difference;

[0059] A second ratio between the absolute value of the difference of the data corresponding to the dimension and the fourth median is calculated, and the second ratio is positively mapped using a preset exponential function to obtain an adaptive weight of the data corresponding to the dimension.

[0060] In one embodiment, for the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment, the data variance var in each dimension is calculated according to all the multidimensional network traffic data in the sliding window, and the median median3 of all data variances is obtained, and then the absolute value of the difference between the data variance in each dimension and the median median3 is calculated. The absolute value of the difference between the data variance in the kth dimension in the sliding window and the median media3 is recorded as , we get the median4 of all the absolute values ​​of the differences, then the adaptive weight of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling time is:

[0061]

[0062] in, It represents the adaptive weight of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment. 1 represents a constant. exp() represents an exponential function with a natural constant as the base. represents the data variance in the kth dimension in the sliding window, || represents the absolute value of the difference, median3 represents the median of all data variances, and median4 represents the median of all absolute values ​​of the differences.

[0063] It should be noted that the feature importance of the corresponding dimension is characterized by analyzing the deviation of the data variance of each dimension in the sliding window relative to the median of the data variance of all dimensions. The larger the value is, the larger the characteristic fluctuation of the k-th dimension in the sliding window is, and the larger the adaptive weight of the data corresponding to the k-th dimension in the multidimensional network traffic data at the i-th sampling moment is.

[0064] Similarly, the adaptive weight of the data corresponding to each dimension in the multi-dimensional network traffic data at the i-th sampling moment is obtained.

[0065] Step S103, based on the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data, and the corresponding multidimensional reconstructed data, the reconstruction error of the multidimensional network traffic data is optimized using the maximum value function to obtain a comprehensive anomaly score of the multidimensional network traffic data.

[0066] After obtaining the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data at the i-th sampling moment, the local reconstruction error of the data corresponding to each dimension in the multidimensional network traffic data at the i-th sampling moment can be optimized, so as to obtain the reconstruction error of the multidimensional network traffic data at the i-th sampling moment as the comprehensive anomaly score of the multidimensional network traffic data at the i-th sampling moment. The larger the reconstruction error, the more abnormal the corresponding multidimensional network traffic data.

[0067] Among them, the method for obtaining the comprehensive anomaly score of the multi-dimensional network traffic data at the i-th sampling moment is:

[0068] According to the absolute value of the data difference between the multidimensional network traffic data and the corresponding multidimensional reconstructed data in any dimension, a reconstruction error of the data corresponding to the dimension is obtained, the difference between the reconstruction error and a reconstruction error threshold of the data corresponding to the dimension is obtained, and a maximum value function is used to obtain the maximum value between the difference and a preset value as the optimized reconstruction error of the data corresponding to the dimension;

[0069] The optimized reconstruction error of the data corresponding to each dimension in the multidimensional network traffic data is obtained, and according to the adaptive weight of the data corresponding to each dimension in the multidimensional network traffic data, all the optimized reconstruction errors are weighted summed to obtain a comprehensive anomaly score of the multidimensional network traffic data.

[0070] In one embodiment, the calculation expression of the comprehensive anomaly score of the multi-dimensional network traffic data at the i-th sampling time is:

[0071]

[0072] Where E represents the comprehensive anomaly score of the multidimensional network traffic data at the i-th sampling moment, N represents the number of dimensions of the multidimensional network traffic data, Represents the adaptive weight of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment, Represents the reconstruction error threshold of the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment, Represents the data corresponding to the kth dimension in the multidimensional network traffic data at the i-th sampling moment, Represents the data corresponding to the kth dimension in the multidimensional reconstructed data corresponding to the multidimensional network traffic data at the i-th sampling moment, | | represents the absolute value symbol, max() represents the maximum value function, and 0 represents the preset value.

[0073] It should be noted that the use of It replaces the square difference in the traditional reconstruction error, which can directly reflect the deviation amplitude of the corresponding network feature, avoid the transition smoothing effect brought by the square operation, and can more sensitively capture sudden surges in network traffic; using the preset value 0 and the maximum value function max(), it directly focuses on the abnormal range to be screened, effectively avoiding fluctuations within the normal error range, and thus ensuring that only errors exceeding the reconstruction error threshold are included in the calculation; by introducing adaptive weights, it can automatically adjust the importance according to the statistical characteristics of each network feature. For example: if a network feature (such as the traffic on a specific port) shows a large fluctuation, it will get a higher weight and pay more attention to the changes in the network feature, thus improving the ability of the reconstruction error calculation process to identify important anomalies, automatically adjusting the attention to different network features, and adapting to the dynamic changes in network behavior, so as to discover potential security threats earlier and more accurately, and help administrators more effectively respond to the evolving network security challenges.

[0074] Step S104, performing anomaly detection on the multi-dimensional network traffic data according to the comprehensive anomaly score of the multi-dimensional network traffic data, obtaining anomaly detection results, and performing intelligent data backup of the network data according to the anomaly detection results.

[0075] It is known that when using incremental PCA to perform anomaly detection on multi-dimensional network traffic data, only by comparing the comprehensive anomaly score with the anomaly score threshold can it be finally determined whether the corresponding original data is abnormal data. Therefore, the anomaly score threshold is an important parameter. In the embodiment of the present invention, the anomaly score threshold is set to be a dynamic threshold. By setting a threshold update time of, for example, 5 minutes, the anomaly score threshold is updated in real time, that is, updated every 5 minutes. Before performing anomaly detection on the multi-dimensional network traffic data at the i-th sampling moment, the updated anomaly score threshold is first obtained, wherein the method for obtaining the updated anomaly score threshold is:

[0076] Obtain a comprehensive anomaly score for each multidimensional network traffic data within a preset threshold update duration, obtain a preset percentile of all comprehensive anomaly scores, obtain an anomaly score threshold before updating, perform weighted summation on the preset percentile and the anomaly score threshold before updating, and obtain an updated anomaly score threshold.

[0077] In one embodiment, the comprehensive anomaly score of each multidimensional network traffic data collected within 5 minutes is obtained to obtain a comprehensive anomaly score set, and the 95th percentile of the comprehensive anomaly score set is obtained, which is recorded as , and then according to And the anomaly score threshold before updating, get the updated anomaly score threshold:

[0078]

[0079] in, represents the updated anomaly score threshold, represents the anomaly score threshold before updating, is the smoothing factor, its value is 0.9-0.99, 1 represents a constant, Represents the 95th percentile of the combined anomaly score set.

[0080] After obtaining the updated anomaly score threshold, anomaly detection is performed on the multidimensional network traffic data at the i-th sampling moment according to the comprehensive anomaly score of the multidimensional network traffic data at the i-th sampling moment and the updated anomaly score threshold. The anomaly detection in the embodiment of the present invention includes a two-layer detection strategy, wherein the first layer detection strategy is: according to the updated anomaly score threshold, a maximum anomaly score threshold is obtained, and if the comprehensive anomaly score of the multidimensional network traffic data is greater than the maximum anomaly score threshold, then it is marked that a network anomaly occurs at the sampling moment of the multidimensional network traffic data.

[0081] In one embodiment, a maximum anomaly score threshold is set. , if the comprehensive anomaly score of the multidimensional network traffic data at the i-th sampling time , it means that the multi-dimensional network traffic data collected at the i-th sampling moment is abnormal data, that is, there is a potential security threat at the i-th sampling moment, and then the network anomaly occurs at the i-th sampling moment.

[0082] The second-layer detection strategy is: construct a new sliding window with the multi-dimensional network traffic data as the last data, the new sliding window is smaller than the sliding window, and the average comprehensive anomaly score is obtained according to the comprehensive anomaly score of each multi-dimensional network traffic data in the new sliding window. If the average comprehensive anomaly score is greater than the updated anomaly score threshold, it is marked that a network anomaly occurs in the time period corresponding to the new sliding window, otherwise, no network anomaly occurs.

[0083] In one embodiment, a new sliding window is set to include multidimensional network traffic data collected within 30 seconds, and the multidimensional network traffic data at the i-th sampling moment is the last one in the new sliding window, and then according to the above-mentioned comprehensive anomaly score acquisition method, the comprehensive anomaly score of each multidimensional network traffic data in the new sliding window is obtained to obtain the average comprehensive anomaly score corresponding to the new sliding window ,like , it indicates that there is a high probability of network security threats in the time period corresponding to the new sliding window, and then marks the time period corresponding to the new sliding window as having a network anomaly. Otherwise, there is no network anomaly.

[0084] After determining the anomaly detection result of the multi-dimensional network traffic data at the i-th sampling moment, the intelligent data backup of the network data in the cloud computing environment can be performed according to the anomaly detection result, so as to protect the key data in time when a potential security threat is detected. Correspondingly, the above two-layer detection strategy corresponds to two-layer intelligent data backup methods respectively. Specifically, the intelligent data backup method for the first-layer detection strategy is:

[0085] If the anomaly detection result is that a network anomaly occurs at the sampling moment of the multi-dimensional network traffic data, data backup is performed on the network traffic data of all dimensions within a first time period formed before and after the sampling moment of the multi-dimensional network traffic data.

[0086] In one embodiment, when it is determined that a network anomaly occurs at the i-th sampling moment, the network traffic data of all dimensions within 1 minute before and 30 seconds after the i-th sampling moment is immediately backed up, which belongs to high-priority data backup, and the high-priority data backup is stored in a high-speed, high-availability storage system to ensure that these key data can be quickly accessed and restored when needed. It should be noted that the period of 1 minute before and 30 seconds after the i-th sampling moment is the first period.

[0087] The intelligent data backup method for the second-tier detection strategy is:

[0088] If the anomaly detection result is that a network anomaly occurs in the time period corresponding to the new sliding window, data backup is performed on network traffic data of all dimensions within a second time period formed before and after the sampling moment of the multi-dimensional network traffic data, wherein the first time period is smaller than the second time period.

[0089] In one embodiment, when it is determined that a network anomaly occurs in the time period corresponding to the new sliding window, the network traffic data of all dimensions within 5 minutes before and 2 minutes after the i-th sampling moment is backed up, which belongs to regular data backup, and the regular data backup is stored in the standard storage system. It should be noted that the time period constructed 5 minutes before and 2 minutes after the i-th sampling moment is the second time period.

[0090] After the known data backup is completed, the backup data needs to be encrypted and stored and strictly access controlled to prevent unauthorized access. Specifically, after each data backup is completed, the system automatically performs backup verification, including verification verification and random sampling recovery testing, to ensure the integrity and availability of the backup data.

[0091] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a data intelligent backup system based on data security analysis, including a memory and a processor, and the processor executes a computer program stored in the memory to implement the steps of any one of the above-mentioned data intelligent backup methods based on data security analysis.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A data intelligent backup method based on data security analysis, characterized in that: The data intelligent backup method based on data security analysis includes: Acquire multidimensional network flow data at any sampling time in real time, obtain a sliding window with the multidimensional network flow data as the last data, and use the incremental PCA method to obtain multidimensional reconstruction data corresponding to each multidimensional network flow data in the sliding window; For the data corresponding to any dimension in the multidimensional network traffic data, a reconstruction error threshold of the data corresponding to the dimension is obtained according to the data corresponding to the dimension and the multidimensional reconstruction data corresponding to each multidimensional network traffic data in the sliding window, and an adaptive weight of the data corresponding to the dimension is obtained according to the data corresponding to the dimension of each multidimensional network traffic data in the sliding window; According to the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data, and the corresponding multidimensional reconstruction data, the reconstruction error of the multidimensional network traffic data is optimized using a maximum function to obtain a comprehensive anomaly score of the multidimensional network traffic data; According to the comprehensive anomaly score of the multi-dimensional network traffic data, anomaly detection is performed on the multi-dimensional network traffic data to obtain anomaly detection results, and intelligent data backup of the network data is performed according to the anomaly detection results.

2. The data intelligent backup method based on data security analysis according to claim 1 is characterized in that: The step of obtaining a reconstruction error threshold of the data corresponding to the dimension according to the data corresponding to the dimension and the multidimensional reconstructed data corresponding to each multidimensional network traffic data in the sliding window comprises: Respectively obtain the first difference absolute value of each multidimensional network traffic data in the sliding window and the corresponding multidimensional reconstructed data under the dimension, and obtain the first median, the first quartile and the third quartile of all the first difference absolute values; Obtaining in the sliding window respectively the second difference absolute values ​​of the data of two adjacent multi-dimensional network flow data under the dimension, obtaining the second median of all the second difference absolute values, and calculating the first ratio between the second difference absolute value corresponding to the multi-dimensional network flow data and the second median; The sum of the constant 1 and the first ratio is used as the independent variable of the preset logarithmic function to obtain the time continuity index corresponding to the dimension, the difference between the first quartile and the third quartile is calculated, the product of the difference and the time continuity index is obtained, and the reconstruction error threshold of the data corresponding to the dimension is obtained according to the product and the first median.

3. The data intelligent backup method based on data security analysis according to claim 1 is characterized in that: The step of obtaining the adaptive weight of the data corresponding to the dimension according to the data corresponding to the dimension of each multi-dimensional network traffic data in the sliding window comprises: Obtain the data variance corresponding to each dimension in the sliding window respectively, obtain the third median of all data variances, calculate the absolute value of the difference between the data variance corresponding to each dimension and the third median respectively, and obtain the fourth median of all the absolute values ​​of the difference; A second ratio between the absolute value of the difference of the data corresponding to the dimension and the fourth median is calculated, and the second ratio is positively mapped using a preset exponential function to obtain an adaptive weight of the data corresponding to the dimension.

4. The data intelligent backup method based on data security analysis according to claim 1 is characterized in that: The method optimizes the reconstruction error of the multidimensional network traffic data using a maximum function according to the adaptive weight and reconstruction error threshold of the data corresponding to each dimension in the multidimensional network traffic data, and the corresponding multidimensional reconstruction data, to obtain a comprehensive anomaly score of the multidimensional network traffic data, including: According to the absolute value of the data difference between the multidimensional network traffic data and the corresponding multidimensional reconstructed data in any dimension, a reconstruction error of the data corresponding to the dimension is obtained, the difference between the reconstruction error and a reconstruction error threshold of the data corresponding to the dimension is obtained, and a maximum value function is used to obtain the maximum value between the difference and a preset value as the optimized reconstruction error of the data corresponding to the dimension; The optimized reconstruction error of the data corresponding to each dimension in the multidimensional network traffic data is obtained, and according to the adaptive weight of the data corresponding to each dimension in the multidimensional network traffic data, all the optimized reconstruction errors are weighted summed to obtain a comprehensive anomaly score of the multidimensional network traffic data.

5. The data intelligent backup method based on data security analysis according to claim 1 is characterized in that: The performing anomaly detection on the multi-dimensional network traffic data according to the comprehensive anomaly score of the multi-dimensional network traffic data to obtain an anomaly detection result includes: A maximum anomaly score threshold is obtained according to the updated anomaly score threshold. If the comprehensive anomaly score of the multi-dimensional network traffic data is greater than the maximum anomaly score threshold, a network anomaly is marked at the sampling moment of the multi-dimensional network traffic data.

6. The data intelligent backup method based on data security analysis according to claim 5 is characterized in that: The performing anomaly detection on the multi-dimensional network traffic data according to the comprehensive anomaly score of the multi-dimensional network traffic data to obtain an anomaly detection result also includes: A new sliding window is constructed with the multidimensional network traffic data as the last data, the new sliding window is smaller than the sliding window, and an average comprehensive anomaly score is obtained according to the comprehensive anomaly score of each multidimensional network traffic data in the new sliding window. If the average comprehensive anomaly score is greater than the updated anomaly score threshold, it is marked that a network anomaly occurs in the time period corresponding to the new sliding window.

7. The data intelligent backup method based on data security analysis according to claim 5 is characterized in that: The method for obtaining the updated anomaly score threshold comprises: Obtain a comprehensive anomaly score for each multidimensional network traffic data within a preset threshold update duration, obtain a preset percentile of all comprehensive anomaly scores, obtain an anomaly score threshold before updating, perform weighted summation on the preset percentile and the anomaly score threshold before updating, and obtain an updated anomaly score threshold.

8. The data intelligent backup method based on data security analysis according to claim 6 is characterized in that: The intelligent data backup of network data according to the abnormality detection result includes: If the anomaly detection result is that a network anomaly occurs at the sampling moment of the multi-dimensional network traffic data, a first time period consisting of the time before and after the sampling moment of the multi-dimensional network traffic data is obtained, and network traffic data of all dimensions within the first time period are backed up.

9. The data intelligent backup method based on data security analysis according to claim 8, characterized in that: The intelligent data backup of network data according to the abnormality detection result includes: If the anomaly detection result is that a network anomaly occurs in the time period corresponding to the new sliding window, then a second time period consisting of the time before and after the sampling moment of the multi-dimensional network traffic data is obtained, and the network traffic data of all dimensions within the second time period is backed up, wherein the first time period is smaller than the second time period.

10. A data intelligent backup system based on data security analysis, comprising a memory and a processor, characterized in that: The processor executes the computer program stored in the memory to implement a data intelligent backup method based on data security analysis as described in any one of claims 1 to 9.

Images