Network security analysis method and system based on big data

Abstract

The invention relates to the technical field of network risk analysis, and particularly discloses a network security analysis method and system based on big data, and the method comprises the steps: obtaining a behavior log and a data transmission speed of each network attack behavior in an attack time period based on a big data technology, and taking the behavior log and the data transmission speed as attack features; obtaining the influence degree of each network attack behavior, and taking the influence degree as an attack label; and counting attack features and attack tags as a sample set to train a neural network model. According to the method, a large number of attack samples are obtained through the big data technology, the behavior-influence degree mapping relation is established, when the user behavior is detected, the behavior is analyzed in real time, then the influence degree is output, and whether a risk exists or not is judged; an original single subject identification process is expanded, and a more comprehensive and flexible network security identification scheme is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of network risk analysis, and in particular to a network security analysis method and system based on big data. Background Art

[0002] With the development of computer equipment and transmission technology, the Internet has gradually become the infrastructure for most production and life activities. While it brings great convenience, there are also some risks. Cyber ​​attacks are one of them. The behavior of a third party maliciously obtaining or processing data without authorization is a cyber attack, which can lead to information leakage or loss. Therefore, it is very important to carry out network security management.

[0003] Most of the existing network security management methods adopt the black and white list method. Any behavior carried out by unauthorized entities is identified as a risk. This method actually only detects the data transmission subject. If a third party borrows the identity of an authorized subject, it is easy to carry out a network attack. How to perform real-time network analysis during data transmission and improve the level of network security management are the technical problems that the technical solution of the present invention aims to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a network security analysis method and system based on big data to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A network security analysis method based on big data, the method comprising:

[0007] Based on big data technology, the behavior log and data transmission speed of each network attack behavior during the attack period are obtained, and the behavior log and data transmission speed are used as attack features;

[0008] Obtaining the impact of each network attack behavior, and using the impact as an attack label; the impact is represented by the module occupancy rate of the device;

[0009] Count the attack features and attack labels as sample sets to train the neural network model;

[0010] When the network connection demand is obtained, the behavior log and data transmission speed are monitored in real time. Taking the current time as the starting point, the monitored data before the preset period is packaged and input into the trained neural network model to obtain the first impact;

[0011] Matching the packaged data with the attack signature regularly, and determining the second impact degree according to the matching result;

[0012] Compare the first impact and the second impact, select the larger value as the final impact and provide real-time feedback.

[0013] As a further solution of the present invention: the step of obtaining the influence of each network attack behavior and using the influence as an attack label includes:

[0014] Get the CPU usage, GPU usage, and memory usage after each network attack;

[0015] Inserting a target file in advance under each disk, and obtaining the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different;

[0016] Determine the impact based on CPU usage, GPU usage, memory usage, and the reading speed of each target file;

[0017] The impact degree is used as an attack label.

[0018] As a further solution of the present invention: the step of periodically matching the packaged data with the attack signature and determining the second impact according to the matching result comprises:

[0019] Read the behavior logs and data transmission speed monitored before the preset time period at a preset frequency;

[0020] Calculate the mean and standard deviation of the data transmission speed, perform a preliminary screening of attack features based on the mean and standard deviation, and select a preset number of attack features as reference features;

[0021] Based on the data transmission speed, the packaged data and the reference features are matched in the time domain, and the time domain distance is calculated synchronously;

[0022] When the time domain distance is less than a preset distance threshold, performing a logical operation on the behavior log to determine the log distance;

[0023] A feature is selected from the reference features according to the time domain distance and the log distance as the target feature, and the influence degree corresponding to the target feature is read as the second influence degree.

[0024] As a further solution of the present invention: the calculation process of the time domain distance is:

[0025] Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence;

[0026] Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence;

[0027] generating a distance matrix based on the first sequence and the second sequence;

[0028] Determining a temporal distance based on the distance matrix;

[0029] The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,y j ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0030] The objective function for determining the time domain distance based on the distance matrix is:

[0031] Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths;

[0032] The calculation process of the log distance is:

[0033] Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

[0034] As a further solution of the present invention: the step of selecting a feature from the reference features according to the time domain distance and the log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree includes:

[0035] The temporal distance and log distance are summed according to the preset weights to obtain the comprehensive distance;

[0036] Select the reference feature with the smallest comprehensive distance as the target feature;

[0037] The influence degree corresponding to the target feature is read from the sample set as the second influence degree.

[0038] As a further solution of the present invention: the step of comparing the first influence degree and the second influence degree, selecting the larger value as the final influence degree and providing real-time feedback comprises:

[0039] When the first influence exceeds the second influence, the number of times the first influence is greater than the second influence is accumulated as the correct number;

[0040] Calculating a ratio of the number of correct answers to the total number of determinations of the second influence, and adjusting a frequency of determination of the second influence according to the ratio; the frequency is inversely proportional to the ratio;

[0041] When the first influence does not exceed the second influence, the monitored behavior log and data transmission speed are used as features, the second influence is inserted into the sample set as a label, and the neural network model is updated.

[0042] The technical solution of the present invention also provides a network security analysis system based on big data, the system comprising:

[0043] An attack feature generation module is used to obtain the behavior log and data transmission speed of each network attack behavior during the attack period based on big data technology, and use the behavior log and data transmission speed as attack features;

[0044] An attack label generation module, used to obtain the impact of each network attack behavior and use the impact as an attack label; the impact is represented by the module occupancy rate of the device;

[0045] The sample application module is used to collect statistics on attack features and attack labels as sample sets to train the neural network model;

[0046] The model application module is used to monitor the behavior log and data transmission speed in real time when the network connection demand is obtained, and to package the monitored data before the preset time period with the current time as the starting point, and input the trained neural network model to obtain the first influence;

[0047] A matching determination module, used for periodically matching the packaged data with the attack signature, and determining the second impact degree according to the matching result;

[0048] The data feedback module is used to compare the first influence degree and the second influence degree, select the larger value as the final influence degree and provide real-time feedback.

[0049] As a further solution of the present invention: the attack label generation module includes:

[0050] An occupancy acquisition unit is used to obtain the CPU occupancy, GPU occupancy and memory occupancy after each network attack behavior;

[0051] A disk speed query unit is used to pre-insert a target file under each disk and obtain the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different;

[0052] An impact calculation unit, used to determine the impact according to the CPU occupancy rate, GPU occupancy rate, memory occupancy rate and the reading speed of each target file;

[0053] The impact application unit is used to use the impact as an attack label.

[0054] As a further solution of the present invention: the matching determination module includes:

[0055] A data packaging unit, used to periodically read the behavior logs and data transmission speed monitored before the preset time period for packaging at a preset frequency;

[0056] A preliminary screening unit, used to calculate the mean and standard deviation of the data transmission speed, perform preliminary screening on the attack features based on the mean and standard deviation, and select a preset number of attack features as reference features;

[0057] A time domain distance calculation unit, used for performing time domain matching on the packaged data and the reference features based on the data transmission speed, and synchronously calculating the time domain distance;

[0058] A log distance calculation unit, configured to perform a logical operation on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold;

[0059] The influence reading unit is used to select a feature from the reference features according to the time domain distance and the log distance as the target feature, and read the influence corresponding to the target feature as the second influence.

[0060] As a further solution of the present invention: the calculation process of the time domain distance is:

[0061] Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence;

[0062] Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence;

[0063] generating a distance matrix based on the first sequence and the second sequence;

[0064] Determining a temporal distance based on the distance matrix;

[0065] The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,y j ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0066] The objective function for determining the time domain distance based on the distance matrix is:

[0067] Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths;

[0068] The calculation process of the log distance is:

[0069] Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

[0070] Compared with the prior art, the beneficial effects of the present invention are: the present invention obtains a large number of attack samples through big data technology, establishes a mapping relationship between behavior and impact, analyzes the behavior in real time when user behavior is detected, and then outputs the impact to determine whether there is a risk; the original single subject identification process is expanded to provide a more comprehensive and flexible network security identification solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0072] Figure 1 The figure is a flowchart of the network security analysis method based on big data.

[0073] Figure 2 This is a structural block diagram of the network security analysis system based on big data. DETAILED DESCRIPTION

[0074] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0075] Figure 1 The flowchart of the network security analysis method based on big data is as follows. In an embodiment of the present invention, a network security analysis method based on big data includes:

[0076] Step S100: Obtaining the behavior log and data transmission speed of each network attack behavior during the attack period based on big data technology, and using the behavior log and data transmission speed as attack features;

[0077] Big data technology in the technical solution of this application refers to data acquisition and storage technology. Due to the large amount of data, it is called big data. This application needs to obtain a large amount of data generated by network attack behaviors during the attack process, specifically behavior logs and data transmission speeds. During the network interaction process, this is data that computer devices can easily obtain and will continue to obtain. In other words, as long as there is network interaction, computer devices will obtain behavior logs and data transmission speeds, which will generally be deleted within a day or a week.

[0078] The data generated by a large amount of network attack behaviors during the attack process obtained by this application is essentially a kind of big data. Therefore, it is necessary to use the data acquisition and storage technology in the existing big data technology to process these data, obtain the behavior log and data transmission speed of each network attack behavior, as the characteristics of an attack behavior, called attack characteristics.

[0079] It should be noted that the behavior log and data transmission speed obtained in this application are the behavior log and data transmission speed within a period of time, that is, the duration of an attack behavior.

[0080] Step S200: Obtain the impact of each network attack behavior and use the impact as an attack label; the impact is represented by the module occupancy rate of the device;

[0081] Each network attack will have an impact on computer devices. This application uses the parameter of impact to describe the impact of each network attack on computer devices, and uses the impact as an attack label. The impact on computer devices refers to the comprehensive impact on components such as CPU, GPU, memory and hard disk in computer devices.

[0082] Step S300: Count the attack features and attack labels as a sample set to train a neural network model;

[0083] Count the attack features and attack labels, construct feature-label samples, and then create a sample set, which is used to train a neural network model; the input of the neural network model is the attack features (behavior logs and data transmission speed), and the output is the attack label (impact degree).

[0084] Step S400: When the network connection requirement is obtained, the behavior log and data transmission speed are monitored in real time, and the monitored data before the preset time period is packaged with the current time as the starting point, and input into the trained neural network model to obtain the first influence degree;

[0085] Steps S100 to S300 are the neural network model generation stage, and steps S400 to S600 are the specific application stage. When a network connection requirement is obtained, the computer device monitors the behavior log and data transmission speed in real time (the same as the data structure of the attack feature), and inputs the behavior log and data transmission speed monitored within a period of time (starting from the current moment and before the preset time period) into the trained neural network model to obtain the impact degree, which is called the first impact degree. The first impact degree is output by the neural network model, and reflects the numerical value of the impact degree on the computer device, which can be directly fed back to the management end as a risk score as the output of this method.

[0086] Step S500: regularly matching the packaged data with the attack signature, and determining the second impact degree according to the matching result;

[0087] The neural network model is applied in real time, and the above matching process is applied periodically. The matching process is a specific comparison process, which compares the behavior logs and data transmission speeds monitored over a period of time with the data in the sample set. This comparison is time-consuming, but it can query the most similar samples and read the influence of the most similar samples as the second influence; the accuracy of the second influence is higher than that of the first influence.

[0088] Step S600: Compare the first influence and the second influence, select the larger value as the final influence and provide real-time feedback;

[0089] Compare the first influence and the second influence. When the first influence exceeds the second influence, accumulate the number of times the first influence is greater than the second influence as the correct number.

[0090] Calculating a ratio of the number of correct answers to the total number of determinations of the second influence, and adjusting a frequency of determination of the second influence according to the ratio; the frequency is inversely proportional to the ratio;

[0091] The meaning of this process is that if the first influence degree output by the neural network model exceeds the second influence degree obtained by the comparison process, it means that the comparison process regards it as a dangerous behavior, and the neural network model also regards it as dangerous. At this time, it can be considered that the accuracy of the neural network model is relatively high, and the application frequency of the comparison process is continuously reduced; from a mathematical point of view, when the first influence degree exceeds the second influence degree, the cumulative number of times the first influence degree is greater than the second influence degree is taken as the correct number, and the correct number is divided by the total number of applications of the comparison process. The larger the ratio, the higher the accuracy, the corresponding frequency must be reduced, and the number of applications of the comparison process is smaller. Therefore, the frequency is inversely proportional to the ratio.

[0092] When the first influence does not exceed the second influence, the monitored behavior log and data transmission speed are used as features, the second influence is inserted into the sample set as a label, and the neural network model is updated.

[0093] Furthermore, when the first influence degree does not exceed the second influence degree, it means that when the comparison process considers it dangerous, the neural network model does not consider it dangerous. At this time, it means that the accuracy of the neural network model is not enough. The current behavior and the second influence degree obtained by the comparison process are used as samples, inserted into the sample set, and the neural network model is updated.

[0094] The above process is repeated continuously until the accuracy of the neural network model is high enough. Finally, the application frequency of the comparison process will be very low. Since the comparison process itself consumes a lot of resources, the process of reducing the frequency greatly reduces the cost consumption.

[0095] The specific description of step S200 in the above content is as follows:

[0096] The step of obtaining the impact of each network attack behavior and using the impact as an attack label includes:

[0097] Get the CPU usage, GPU usage, and memory usage after each network attack;

[0098] Inserting a target file in advance under each disk, and obtaining the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different;

[0099] Determine the impact based on CPU usage, GPU usage, memory usage, and the reading speed of each target file;

[0100] The impact degree is used as an attack label.

[0101] The above content specifically defines the concept of impact, which only considers four parts, namely CPU, GPU, memory and disk; the CPU, GPU and memory can directly collect occupancy, and the computer device has its own monitoring model; however, for the disk occupancy, the present application provides a specific solution, which pre-sets some files at different disks, called target files. In actual applications, by adjusting these target files, the most accurate disk reading speed can be measured. Finally, a value reflecting the impact on the computer device can be determined by combining the CPU occupancy, GPU occupancy, memory occupancy and the reading speed of each target file, which is called the impact; the impact is inversely proportional to the CPU occupancy, GPU occupancy and memory occupancy, and inversely proportional to the reading speed of each target file.

[0102] The specific description of step S500 in the above content is as follows:

[0103] The step of periodically matching the packaged data with the attack signature and determining the second impact according to the matching result comprises:

[0104] Read the behavior logs and data transmission speed monitored before the preset time period at a preset frequency;

[0105] Calculate the mean and standard deviation of the data transmission speed, perform a preliminary screening of attack features based on the mean and standard deviation, and select a preset number of attack features as reference features;

[0106] Based on the data transmission speed, the packaged data and the reference features are matched in the time domain, and the time domain distance is calculated synchronously;

[0107] When the time domain distance is less than a preset distance threshold, performing a logical operation on the behavior log to determine the log distance;

[0108] A feature is selected from the reference features according to the time domain distance and the log distance as the target feature, and the influence degree corresponding to the target feature is read as the second influence degree.

[0109] The above content provides a specific comparison process, which is described as follows:

[0110] At a preset frequency (dynamic frequency, related to the accuracy of the neural network model, the higher the accuracy, the smaller the frequency), the behavior logs and data transmission speeds monitored before the preset time period of packaging are read at regular intervals, the mean and standard deviation of the data transmission speed are calculated, and the big data samples (attack features) are preliminarily screened by the mean and standard deviation to obtain some samples that meet the requirements, which are called reference features; the specific preliminary screening process is: calculate the mean of the data transmission speed in the big data sample, calculate the difference ratio between the mean and the mean of the currently packaged data, calculate the standard deviation of the data transmission speed in the big data sample, calculate the difference ratio between the standard deviation and the standard deviation of the currently packaged data, and when both difference ratios meet the preset ratio conditions, select the big data sample as the reference feature.

[0111] Furthermore, by performing time domain matching on the packaged data and the reference features based on the data transmission speed, the time domain distance can be obtained. When the time domain distance is small enough, logical operations are performed on the behavior logs to determine the log distance. Finally, the time domain distance and the log distance are combined to jointly determine the final feature, called the target feature, and the influence degree corresponding to the target feature is read in the sample set as the second influence degree.

[0112] Specifically, the calculation process of the time domain distance is:

[0113] Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence;

[0114] Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence;

[0115] generating a distance matrix based on the first sequence and the second sequence;

[0116] Determining a temporal distance based on the distance matrix;

[0117] The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,y j ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0118] The objective function for determining the time domain distance based on the distance matrix is:

[0119] Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths;

[0120] The calculation process of the log distance is:

[0121] Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

[0122] The above content limits the specific calculation process of the two distances, among which the time domain distance adopts the DTW distance, which is used to calculate the distance between two sequences. The two sequences do not need to be of equal length, which is consistent with the actual situation of the present application. The duration of the data transmission process cannot be determined in advance, so the DTW distance is adopted; the core idea of ​​the DTW distance is to construct a matrix with the two sequences as rows and columns respectively, fill the distance in each row and column position in the matrix, and then connect from the upper left corner of the matrix to the lower right corner to obtain multiple paths, and select the path with the shortest length. The length of the path is the DTW distance.

[0123] The calculation process of log distance is relatively simple. Each process in the behavior log has a unique label. After converting it into a set, there is no need to consider the order. The difference of the set is calculated. The more elements in the difference set, the greater the difference between the two behavior logs and the greater the distance. Therefore, the number of elements in the difference set can be directly used as the log distance.

[0124] Furthermore, the step of selecting a feature from the reference features according to the time domain distance and the log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree includes:

[0125] The temporal distance and log distance are summed according to the preset weights to obtain the comprehensive distance;

[0126] Select the reference feature with the smallest comprehensive distance as the target feature;

[0127] The influence degree corresponding to the target feature is read from the sample set as the second influence degree.

[0128] The above content describes the application process of time domain distance and log distance in detail. The time domain distance and log distance are summed according to the preset weights to obtain the comprehensive distance, and the reference feature with the smallest comprehensive distance is selected as the target feature; the influence degree corresponding to the target feature is read in the sample set as the second influence degree.

[0129] Figure 2 1 is a structural block diagram of a network security analysis system based on big data. In an embodiment of the present invention, a network security analysis system based on big data, the system 10 includes:

[0130] An attack feature generation module 11 is used to obtain the behavior log and data transmission speed of each network attack behavior within the attack period based on big data technology, and use the behavior log and data transmission speed as attack features;

[0131] An attack label generation module 12 is used to obtain the impact of each network attack behavior and use the impact as an attack label; the impact is represented by the module occupancy rate of the device;

[0132] The sample application module 13 is used to collect statistics of attack features and attack labels as sample sets to train the neural network model;

[0133] The model application module 14 is used to monitor the behavior log and data transmission speed in real time when the network connection demand is obtained, and to package the monitored data before the preset time period with the current time as the starting point, and input the trained neural network model to obtain the first influence degree;

[0134] A matching determination module 15, used for periodically matching the packaged data with the attack signature, and determining a second impact degree according to the matching result;

[0135] The data feedback module 16 is used to compare the first influence degree and the second influence degree, select the larger value as the final influence degree and provide real-time feedback.

[0136] Furthermore, the attack label generating module 12 includes:

[0137] An occupancy acquisition unit is used to obtain the CPU occupancy, GPU occupancy and memory occupancy after each network attack behavior;

[0138] A disk speed query unit is used to pre-insert a target file under each disk and obtain the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different;

[0139] An impact calculation unit, used to determine the impact according to the CPU occupancy rate, GPU occupancy rate, memory occupancy rate and the reading speed of each target file;

[0140] The impact application unit is used to use the impact as an attack label.

[0141] Specifically, the matching determination module 15 includes:

[0142] A data packaging unit, used to periodically read the behavior logs and data transmission speed monitored before the preset time period for packaging at a preset frequency;

[0143] A preliminary screening unit, used to calculate the mean and standard deviation of the data transmission speed, perform preliminary screening on the attack features based on the mean and standard deviation, and select a preset number of attack features as reference features;

[0144] A time domain distance calculation unit, used for performing time domain matching on the packaged data and the reference features based on the data transmission speed, and synchronously calculating the time domain distance;

[0145] A log distance calculation unit, configured to perform a logical operation on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold;

[0146] The influence reading unit is used to select a feature from the reference features according to the time domain distance and the log distance as the target feature, and read the influence corresponding to the target feature as the second influence.

[0147] Furthermore, the calculation process of the time domain distance is:

[0148] Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence;

[0149] Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence;

[0150] generating a distance matrix based on the first sequence and the second sequence;

[0151] Determining a temporal distance based on the distance matrix;

[0152] The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,yj ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0153] The objective function for determining the time domain distance based on the distance matrix is:

[0154] Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths;

[0155] The calculation process of the log distance is:

[0156] Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A network security analysis method based on big data, characterized in that: The method comprises: Based on big data technology, the behavior log and data transmission speed of each network attack behavior during the attack period are obtained, and the behavior log and data transmission speed are used as attack features; Obtaining the impact of each network attack behavior, and using the impact as an attack label; the impact is represented by the module occupancy rate of the device; Count the attack features and attack labels as sample sets to train the neural network model; When the network connection demand is obtained, the behavior log and data transmission speed are monitored in real time. Taking the current time as the starting point, the monitored data before the preset period is packaged and input into the trained neural network model to obtain the first impact; Matching the packaged data with the attack signature regularly, and determining the second impact degree according to the matching result; Compare the first impact and the second impact, select the larger value as the final impact and provide real-time feedback.

2. The network security analysis method based on big data according to claim 1 is characterized in that: The step of obtaining the impact of each network attack behavior and using the impact as an attack label includes: Get the CPU usage, GPU usage, and memory usage after each network attack; Inserting a target file in advance under each disk, and obtaining the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different; Determine the impact based on CPU usage, GPU usage, memory usage, and the reading speed of each target file; The impact degree is used as an attack label.

3. The network security analysis method based on big data according to claim 1 is characterized in that: The step of periodically matching the packaged data with the attack signature and determining the second impact according to the matching result comprises: Read the behavior logs and data transmission speed monitored before the preset time period at a preset frequency; Calculate the mean and standard deviation of the data transmission speed, perform a preliminary screening of attack features based on the mean and standard deviation, and select a preset number of attack features as reference features; Based on the data transmission speed, the packaged data and the reference features are matched in the time domain, and the time domain distance is calculated synchronously; When the time domain distance is less than a preset distance threshold, performing a logical operation on the behavior log to determine the log distance; A feature is selected from the reference features according to the time domain distance and the log distance as the target feature, and the influence degree corresponding to the target feature is read as the second influence degree.

4. The network security analysis method based on big data according to claim 3 is characterized in that: The calculation process of the time domain distance is: Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence; Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence; generating a distance matrix based on the first sequence and the second sequence; Determining a temporal distance based on the distance matrix; The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,y j ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y; The objective function for determining the time domain distance based on the distance matrix is: Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths; The calculation process of the log distance is: Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

5. The network security analysis method based on big data according to claim 4 is characterized in that: The step of selecting a feature from the reference features according to the time domain distance and the log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree comprises: The temporal distance and log distance are summed according to the preset weights to obtain the comprehensive distance; Select the reference feature with the smallest comprehensive distance as the target feature; The influence degree corresponding to the target feature is read from the sample set as the second influence degree.

6. The network security analysis method based on big data according to claim 1 is characterized in that: The step of comparing the first influence and the second influence, selecting the larger value as the final influence and providing real-time feedback comprises: When the first influence exceeds the second influence, the number of times the first influence is greater than the second influence is accumulated as the correct number; Calculating a ratio of the number of correct answers to the total number of determinations of the second influence, and adjusting a frequency of determination of the second influence according to the ratio; the frequency is inversely proportional to the ratio; When the first influence does not exceed the second influence, the monitored behavior log and data transmission speed are used as features, the second influence is inserted into the sample set as a label, and the neural network model is updated.

7. A network security analysis system based on big data, characterized in that: The system comprises: An attack feature generation module is used to obtain the behavior log and data transmission speed of each network attack behavior during the attack period based on big data technology, and use the behavior log and data transmission speed as attack features; An attack label generation module, used to obtain the impact of each network attack behavior and use the impact as an attack label; the impact is represented by the module occupancy rate of the device; The sample application module is used to collect statistics on attack features and attack labels as sample sets to train the neural network model; The model application module is used to monitor the behavior log and data transmission speed in real time when the network connection demand is obtained, and to package the monitored data before the preset time period with the current time as the starting point, and input the trained neural network model to obtain the first influence; A matching determination module, used for periodically matching the packaged data with the attack signature, and determining the second impact degree according to the matching result; The data feedback module is used to compare the first influence degree and the second influence degree, select the larger value as the final influence degree and provide real-time feedback.

8. The network security analysis system based on big data according to claim 7 is characterized in that: The attack label generation module includes: An occupancy acquisition unit is used to obtain the CPU occupancy, GPU occupancy and memory occupancy after each network attack behavior; A disk speed query unit is used to pre-insert a target file under each disk and obtain the reading speed of each target file after each network attack behavior; the target file is at least one, and the depths of different target files are different; An impact calculation unit, used to determine the impact according to the CPU occupancy rate, GPU occupancy rate, memory occupancy rate and the reading speed of each target file; The impact application unit is used to use the impact as an attack label.

9. The network security analysis system based on big data according to claim 7 is characterized in that: The matching determination module comprises: A data packaging unit, used to periodically read the behavior logs and data transmission speed monitored before the preset time period for packaging at a preset frequency; A preliminary screening unit, used to calculate the mean and standard deviation of the data transmission speed, perform preliminary screening on the attack features based on the mean and standard deviation, and select a preset number of attack features as reference features; A time domain distance calculation unit, used for performing time domain matching on the packaged data and the reference features based on the data transmission speed, and synchronously calculating the time domain distance; A log distance calculation unit, configured to perform a logical operation on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold; The influence reading unit is used to select a feature from the reference features according to the time domain distance and the log distance as the target feature, and read the influence corresponding to the target feature as the second influence.

10. The network security analysis system based on big data according to claim 9 is characterized in that: The calculation process of the time domain distance is: Arrange the data transmission speeds in the packaged data in time order to obtain a first sequence; Selecting reference features in sequence, and arranging the data transmission speeds in the selected reference features in chronological order to obtain a second sequence; generating a distance matrix based on the first sequence and the second sequence; Determining a temporal distance based on the distance matrix; The element D(i,j) in the distance matrix = dis(x i ,y j ); x i For the first sequence X=(x1,x2,...,x n ) is the i-th data transmission speed, y j The second sequence Y = (y1, y2, ..., y m ) is the jth data transmission speed; dis(x i ,y j ) is x i and j The distance between them is calculated by a linear norm and a quadratic norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y; The objective function for determining the time domain distance based on the distance matrix is: Where SD is the time domain distance, W = (w1,w2,...,w K ) represents a path from position (1,1) to position (n,m) in the distance matrix, w k is the kth element D(i,j) on the path, D(i,j) represents the first sequence X=(x1,x2,...,x n ) and the second sequence Y = (y1, y2, ..., y m ) is the distance between the jth data transmission speed in the path; K is the total number of all elements included in the path; Indicates selecting the minimum length among all paths; The calculation process of the log distance is: Convert the two behavior logs to be calculated into sets, calculate the difference of the sets, and use the number of elements in the difference as the log distance.

Images