A network security analysis method and system based on big data

Abstract

This invention relates to the field of network risk analysis technology, specifically disclosing a network security analysis method and system based on big data. The method includes acquiring behavior logs and data transmission speeds of each network attack during the attack period using big data technology, and using these logs and speeds as attack features; acquiring the impact of each network attack, and using this impact as an attack label; statistically analyzing the attack features and labels as a sample set to train a neural network model. This invention acquires a large number of attack samples using big data technology, establishes a behavior-impact mapping relationship, and analyzes user behavior in real time upon detection, thereby outputting the impact and determining whether a risk exists. It expands upon the original single-subject identification process, providing a more comprehensive and flexible network security identification solution.

Description

Technical Field

[0001] This invention relates to the field of network risk analysis technology, specifically a network security analysis method and system based on big data. Background Technology

[0002] With the development of computer equipment and transmission technology, the network has gradually become the infrastructure for most production and life activities. While bringing great convenience, it also poses some risks, one of which is network attack. The act of a third party maliciously obtaining or processing data without authorization constitutes a network attack, which can lead to information leakage or loss. Therefore, network security management is very important.

[0003] Most existing network security management methods use blacklists and whitelists, identifying any unauthorized actions as risks. This approach only detects the data transmission entity. If a third party uses the identity of an authorized entity, it can easily launch a network attack. How to perform real-time network analysis during data transmission to improve network security management is the technical problem that this invention aims to solve. Summary of the Invention

[0004] The purpose of this invention is to provide a network security analysis method and system based on big data to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A big data-based network security analysis method, the method comprising:

[0007] Based on big data technology, we obtain the behavior logs and data transmission speed of each network attack during the attack period, and use the behavior logs and data transmission speed as attack characteristics.

[0008] The impact of each network attack is obtained and used as an attack label; the impact is represented by the module utilization rate of the device.

[0009] Statistical attack features and attack labels are used as a sample set to train a neural network model;

[0010] When a network connection request is received, the behavior logs and data transmission speed are monitored in real time. Starting from the current moment, the monitored data from a preset time period is packaged and input into the trained neural network model to obtain the first degree of influence.

[0011] The packaged data is periodically matched with attack characteristics, and the second degree of influence is determined based on the matching results;

[0012] Compare the first and second impact values, select the larger value as the final impact value, and provide feedback in real time.

[0013] As a further aspect of the present invention: the step of obtaining the impact of each network attack behavior and using the impact as an attack label includes:

[0014] Obtain CPU usage, GPU usage, and memory usage after each network attack.

[0015] Target files are pre-inserted into each disk, and the read speed of each target file is obtained after each network attack; there is at least one target file, and the depth of different target files is different.

[0016] The degree of impact is determined based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file;

[0017] The impact level is used as an attack label.

[0018] As a further aspect of the present invention: the step of periodically matching the packaged data with attack characteristics and determining the second influence degree based on the matching result includes:

[0019] At a preset frequency, periodically read the behavior logs and data transmission speed monitored before the preset time period of the package;

[0020] Calculate the mean and standard deviation of data transmission speed, and perform preliminary screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features;

[0021] Based on the data transmission speed, perform temporal matching between packaged data and reference features, and simultaneously calculate the temporal distance;

[0022] When the time-domain distance is less than a preset distance threshold, logical operations are performed on the behavior log to determine the log distance;

[0023] Based on the temporal distance and log distance, select a feature from the reference features as the target feature, and read the influence degree corresponding to the target feature as the second influence degree.

[0024] As a further aspect of the present invention: the calculation process of the time-domain distance is as follows:

[0025] The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence;

[0026] Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence;

[0027] Generate a distance matrix based on the first sequence and the second sequence;

[0028] The time-domain distance is determined based on the distance matrix;

[0029] The element D(i,j) in the distance matrix is ​​dis(x) i ,y j );x i The first sequence is X = (x1, x2, ..., x...). n The data transmission speed of the i-th data in ) is y j For the second sequence Y = (y1, y2, ..., y m The j-th data transmission speed in ); dis(x i ,y j ) is x i and y j The distance between them, wherein the distance is calculated using the first norm and the second norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0030] The objective function for determining the time-domain distance based on the distance matrix is:

[0031] In the formula, SD represents the time-domain distance, and W = (w1, w2, ..., w K ) represents a path in the distance matrix from position (1,1) to position (n,m), w k Let D(i,j) be the k-th element on the path, where D(i,j) represents the first sequence X = (x1, x2, ..., xj). n The data transmission speed of the i-th data in the sequence Y = (y1, y2, ..., y) is related to the second sequence Y = (y1, y2, ..., y) m The distance between the j-th data transmission speeds in the path; K is the total number of all elements included in the path; This indicates that the path with the minimum length is selected from all paths.

[0032] The calculation process for the log distance is as follows:

[0033] Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

[0034] As a further aspect of the present invention: the step of selecting a feature from the reference features based on the temporal distance and log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree includes:

[0035] The temporal distance and log distance are summed according to preset weights to obtain the comprehensive distance;

[0036] The reference feature with the smallest comprehensive distance is selected as the target feature;

[0037] The influence degree corresponding to the target feature is read from the sample set and used as the second influence degree.

[0038] As a further aspect of the present invention: the step of comparing the first influence degree and the second influence degree, selecting the larger value as the final influence degree, and providing real-time feedback includes:

[0039] When the first degree of influence exceeds the second degree of influence, the number of times the first degree of influence is greater than the second degree of influence is accumulated and counted as the number of correct results;

[0040] Calculate the ratio of the number of correct answers to the total number of determinations of the second influence, and adjust the determination frequency of the second influence based on the ratio; the frequency is inversely proportional to the ratio.

[0041] When the first degree of influence does not exceed the second degree of influence, the monitored behavior logs and data transmission speed are used as features, the second degree of influence is used as a label and inserted into the sample set to update the neural network model.

[0042] The present invention also provides a network security analysis system based on big data, the system comprising:

[0043] The attack signature generation module is used to obtain the behavior logs and data transmission speed of each network attack behavior during the attack period based on big data technology, and use the behavior logs and data transmission speed as attack signatures.

[0044] An attack tag generation module is used to obtain the impact of each network attack behavior and use the impact as an attack tag; the impact is represented by the module utilization rate of the device.

[0045] The sample application module is used to statistically analyze attack features and attack labels, which serve as a sample set for training neural network models.

[0046] The model application module is used to monitor behavior logs and data transmission speed in real time when a network connection requirement is obtained. Starting from the current moment, it packages the monitored data from a preset time period and inputs it into the trained neural network model to obtain the first influence degree.

[0047] The matching and determination module is used to periodically match the packaged data with attack characteristics and determine the second degree of influence based on the matching results;

[0048] The data feedback module is used to compare the first and second impact values, select the larger value as the final impact value, and provide feedback in real time.

[0049] As a further aspect of the present invention: the attack tag generation module includes:

[0050] The utilization rate acquisition unit is used to acquire the CPU utilization rate, GPU utilization rate, and memory utilization rate after each network attack.

[0051] The disk speed query unit is used to pre-insert target files on each disk and obtain the read speed of each target file after each network attack; the target file is at least one, and the depth of different target files is different.

[0052] The impact calculation unit is used to determine the impact based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file;

[0053] The influence application unit is used to use the influence as an attack label.

[0054] As a further aspect of the present invention: the matching determination module includes:

[0055] The data packaging unit is used to periodically read the behavior logs and data transmission speed monitored before the preset time period of the package at a preset frequency;

[0056] The initial screening unit is used to calculate the mean and standard deviation of the data transmission speed, and to perform initial screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features.

[0057] The temporal distance calculation unit is used to perform temporal matching between packaged data and reference features based on data transmission speed, and to simultaneously calculate the temporal distance.

[0058] The log distance calculation unit is used to perform logical operations on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold.

[0059] The influence reading unit is used to select a feature from the reference features based on the temporal distance and log distance as the target feature, and read the influence of the target feature as the second influence.

[0060] As a further aspect of the present invention: the calculation process of the time-domain distance is as follows:

[0061] The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence;

[0062] Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence;

[0063] Generate a distance matrix based on the first sequence and the second sequence;

[0064] The time-domain distance is determined based on the distance matrix;

[0065] The element D(i,j) in the distance matrix is ​​dis(x) i ,y j );x i The first sequence is X = (x1, x2, ..., x...). n The data transmission speed of the i-th data in ) is y j For the second sequence Y = (y1, y2, ..., y m The j-th data transmission speed in ); dis(x i ,y j ) is x i and y j The distance between them, wherein the distance is calculated using the first norm and the second norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0066] The objective function for determining the time-domain distance based on the distance matrix is:

[0067] In the formula, SD represents the time-domain distance, and W = (w1, w2, ..., w K ) represents a path in the distance matrix from position (1,1) to position (n,m), w k Let D(i,j) be the k-th element on the path, where D(i,j) represents the first sequence X = (x1, x2, ..., xj). n The data transmission speed of the i-th data in the sequence Y = (y1, y2, ..., y) is related to the second sequence Y = (y1, y2, ..., y) m The distance between the j-th data transmission speeds in the path; K is the total number of all elements included in the path; This indicates that the path with the minimum length is selected from all paths.

[0068] The calculation process for the log distance is as follows:

[0069] Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

[0070] Compared with existing technologies, the beneficial effects of this invention are: this invention obtains a large number of attack samples through big data technology, establishes a mapping relationship between behavior and impact, analyzes user behavior in real time when it is detected, and then outputs the impact to determine whether there is a risk; it expands the original single subject identification process and provides a more comprehensive and flexible network security identification solution. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0072] Figure 1 This is a flowchart of a big data-based network security analysis method.

[0073] Figure 2 This is a block diagram of the structure of a big data-based network security analysis system. Detailed Implementation

[0074] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0075] Figure 1 This is a flowchart illustrating a big data-based network security analysis method. In this embodiment of the invention, a big data-based network security analysis method includes:

[0076] Step S100: Based on big data technology, obtain the behavior logs and data transmission speed of each network attack behavior during the attack period, and use the behavior logs and data transmission speed as attack characteristics;

[0077] In this application, "big data technology" refers to data acquisition and storage technology. Due to the large amount of data, it is called big data. This application needs to acquire a large amount of data generated during network attack activities, specifically behavior logs and data transmission speed. This data is very easy for computer devices to acquire and will be acquired continuously during network interaction. In other words, as long as there is network interaction, computer devices will acquire behavior logs and data transmission speed, which will generally be deleted within one day or one week.

[0078] The large amount of data generated during the network attack activities obtained in this application is essentially big data. Therefore, it is necessary to use existing big data technologies for data acquisition and storage to process this data and obtain the behavior logs and data transmission speed of each network attack activity as characteristics of an attack activity, which are called attack features.

[0079] It should be noted that the behavior logs and data transmission speed obtained in this application are behavior logs and data transmission speeds within a time period, that is, the duration of an attack.

[0080] Step S200: Obtain the impact of each network attack behavior and use the impact as an attack label; the impact is represented by the module utilization rate of the device;

[0081] Every cyberattack impacts computer equipment. This application uses the parameter of "impact level" to describe the degree of impact of each cyberattack on computer equipment, and uses the impact level as an attack label. The degree of impact on computer equipment refers to the comprehensive impact on components such as the CPU, GPU, memory, and hard drive.

[0082] Step S300: Collect statistical attack features and attack labels as a sample set to train a neural network model;

[0083] Statistical attack features and attack labels are collected to construct feature-label samples, thereby creating a sample set. The neural network model is trained using the sample set. The input of the neural network model is the attack features (behavioral logs and data transmission speed), and the output is the attack label (impact).

[0084] Step S400: When a network connection request is received, monitor the behavior log and data transmission speed in real time. Starting from the current time, package the data monitored before the preset time period, input it into the trained neural network model, and obtain the first influence degree.

[0085] Steps S100 to S300 are the neural network model generation stage, and steps S400 to S600 are the specific application stage. When a network connection requirement is obtained, the computer device monitors the behavior logs and data transmission speed in real time (with the same data structure as the attack characteristics). The behavior logs and data transmission speed monitored within a certain period of time (starting from the current time and before the preset time period) are input into the trained neural network model to obtain the impact degree. This impact degree is called the first impact degree. The first impact degree is a numerical value output by the neural network model, reflecting the degree of impact on the computer device. It can be directly fed back to the management end as a risk score as the output of this method.

[0086] Step S500: Periodically match the packaged data with attack characteristics, and determine the second degree of influence based on the matching results;

[0087] The neural network model is applied in real time, while the matching process described above is applied periodically. The matching process is a specific comparison process that compares the behavior logs and data transmission speed monitored over a period of time with the data in the sample set. This comparison is time-consuming, but it can find the most similar sample and read the influence of the most similar sample as the second influence. The accuracy of the second influence is higher than that of the first influence.

[0088] Step S600: Compare the first and second impact values, select the larger value as the final impact value, and provide real-time feedback.

[0089] Compare the first and second impact values. When the first impact value exceeds the second impact value, accumulate the number of times the first impact value is greater than the second impact value, and use this as the number of correct results.

[0090] Calculate the ratio of the number of correct answers to the total number of determinations of the second influence, and adjust the determination frequency of the second influence based on the ratio; the frequency is inversely proportional to the ratio.

[0091] This process means that if the first influence value output by the neural network model exceeds the second influence value obtained from the comparison process, it indicates that the comparison process is considered a dangerous behavior, and the neural network model is also considered dangerous. In this case, the accuracy of the neural network model can be considered high, and the application frequency of the comparison process will be continuously reduced. From a mathematical point of view, when the first influence value exceeds the second influence value, the cumulative number of times the first influence value is greater than the second influence value is taken as the correct number. The correct number is divided by the total number of times the comparison process is applied. The larger the ratio, the higher the accuracy, and the corresponding frequency should be reduced, and the number of times the comparison process is applied is smaller. Therefore, the frequency is inversely proportional to the ratio.

[0092] When the first degree of influence does not exceed the second degree of influence, the monitored behavior logs and data transmission speed are used as features, the second degree of influence is used as a label and inserted into the sample set to update the neural network model.

[0093] Furthermore, when the first degree of influence does not exceed the second degree of influence, it means that the neural network model does not consider the comparison process dangerous. In this case, it means that the accuracy of the neural network model is insufficient. Therefore, the current behavior and the second degree of influence obtained from the comparison process are used as samples and inserted into the sample set to update the neural network model.

[0094] The above process is repeated until the accuracy of the neural network model is high enough. Eventually, the frequency of the comparison process will be very low. Since the comparison process itself consumes a lot of resources, reducing the frequency greatly reduces the cost.

[0095] The specific explanation of step S200 in the above content is as follows:

[0096] The step of obtaining the impact of each network attack and using the impact as an attack label includes:

[0097] Obtain CPU usage, GPU usage, and memory usage after each network attack.

[0098] Target files are pre-inserted into each disk, and the read speed of each target file is obtained after each network attack; there is at least one target file, and the depth of different target files is different.

[0099] The degree of impact is determined based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file;

[0100] The impact level is used as an attack label.

[0101] The above content provides a specific definition of the concept of impact, which only considers four components: CPU, GPU, memory, and disk. CPU, GPU, and memory usage can be directly collected, as computer devices have built-in monitoring models. However, for disk usage, this application provides a specific solution: pre-setting some files on different disks, called target files. In practical applications, adjusting these target files allows for the most accurate measurement of disk read speeds. Finally, combining CPU usage, GPU usage, memory usage, and the read speed of each target file, a value reflecting the impact on the computer device can be determined, called the impact degree. The impact degree is inversely proportional to CPU usage, GPU usage, and memory usage, and also inversely proportional to the read speed of each target file.

[0102] The specific explanation of step S500 in the above content is as follows:

[0103] The step of periodically matching the packaged data with attack characteristics and determining the second influence based on the matching results includes:

[0104] At a preset frequency, periodically read the behavior logs and data transmission speed monitored before the preset time period of the package;

[0105] Calculate the mean and standard deviation of data transmission speed, and perform preliminary screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features;

[0106] Based on the data transmission speed, perform temporal matching between packaged data and reference features, and simultaneously calculate the temporal distance;

[0107] When the time-domain distance is less than a preset distance threshold, logical operations are performed on the behavior log to determine the log distance;

[0108] Based on the temporal distance and log distance, select a feature from the reference features as the target feature, and read the influence degree corresponding to the target feature as the second influence degree.

[0109] The above content provides a specific comparison process, which is explained below:

[0110] At a preset frequency (dynamic frequency, related to the accuracy of the neural network model; the higher the accuracy, the lower the frequency), the behavior logs and data transmission speed monitored before the preset time period are read at regular intervals. The mean and standard deviation of the data transmission speed are calculated. The mean and standard deviation are used to initially screen the big data samples (attack features) to obtain some samples that meet the requirements, which are called reference features. The specific initial screening process is as follows: calculate the mean of data transmission speed in the big data samples, calculate the difference ratio between the mean and the mean of the currently packaged data, calculate the standard deviation of data transmission speed in the big data samples, calculate the difference ratio between the standard deviation and the standard deviation of the currently packaged data, and when both difference ratios meet the preset ratio conditions, the big data samples are selected as reference features.

[0111] Furthermore, by performing time-domain matching on the packaged data and reference features based on the data transmission speed, the time-domain distance can be obtained. When the time-domain distance is small enough, logical operations are performed on the behavior logs to determine the log distance. Finally, the time-domain distance and the log distance are combined to determine the final feature, called the target feature. The influence degree corresponding to the target feature is read in the sample set as the second influence degree.

[0112] Specifically, the calculation process for the time-domain distance is as follows:

[0113] The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence;

[0114] Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence;

[0115] Generate a distance matrix based on the first sequence and the second sequence;

[0116] The time-domain distance is determined based on the distance matrix;

[0117] The element D(i,j) in the distance matrix is ​​dis(x) i ,y j );x i The first sequence is X = (x1, x2, ..., x...). n The data transmission speed of the i-th data in ) is y j For the second sequence Y = (y1, y2, ..., y m The j-th data transmission speed in ); dis(x i ,y j ) is x i and y j The distance between them, wherein the distance is calculated using the first norm and the second norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0118] The objective function for determining the time-domain distance based on the distance matrix is:

[0119] In the formula, SD represents the time-domain distance, and W = (w1, w2, ..., w K ) represents a path in the distance matrix from position (1,1) to position (n,m), w k Let D(i,j) be the k-th element on the path, where D(i,j) represents the first sequence X = (x1, x2, ..., xj). n The data transmission speed of the i-th data in the sequence Y = (y1, y2, ..., y) is related to the second sequence Y = (y1, y2, ..., y) m The distance between the j-th data transmission speeds in the path; K is the total number of all elements included in the path; This indicates that the path with the minimum length is selected from all paths.

[0120] The calculation process for the log distance is as follows:

[0121] Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

[0122] The above content specifies the detailed calculation process of the two distances. The time-domain distance uses the DTW distance, which is used to calculate the distance between two sequences. These two sequences do not need to be of equal length, which is consistent with the actual situation of this application. The duration of data transmission cannot be determined in advance, so the DTW distance is used. The core idea of ​​the DTW distance is to construct a matrix with the two sequences as rows and columns respectively, fill each row and column position in the matrix with distance, and then connect the upper left corner to the lower right corner of the matrix to obtain multiple paths. The path with the shortest length is selected, and the length of this path is the DTW distance.

[0123] The calculation process of log distance is relatively simple. Each process in the behavior log has a unique label. After converting it into a set, there is no need to consider the order. Calculate the difference between the sets. The more elements in the difference set, the greater the difference between the two behavior logs and the farther the distance. Therefore, the number of elements in the difference set can be directly used as the log distance.

[0124] Furthermore, the step of selecting a feature from the reference features based on the temporal distance and log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree includes:

[0125] The temporal distance and log distance are summed according to preset weights to obtain the comprehensive distance;

[0126] The reference feature with the smallest comprehensive distance is selected as the target feature;

[0127] The influence degree corresponding to the target feature is read from the sample set and used as the second influence degree.

[0128] The above content describes in detail the application process of temporal distance and log distance. The temporal distance and log distance are summed according to the preset weights to obtain the comprehensive distance. The reference feature with the smallest comprehensive distance is selected as the target feature. The influence degree corresponding to the target feature is read in the sample set as the second influence degree.

[0129] Figure 2 This is a block diagram of the composition structure of a big data-based network security analysis system. In this embodiment of the invention, a big data-based network security analysis system, system 10, includes:

[0130] The attack signature generation module 11 is used to obtain the behavior logs and data transmission speed of each network attack behavior during the attack period based on big data technology, and use the behavior logs and data transmission speed as attack signatures.

[0131] The attack tag generation module 12 is used to obtain the impact of each network attack behavior and use the impact as an attack tag; the impact is represented by the module utilization rate of the device.

[0132] Sample application module 13 is used to statistically analyze attack features and attack labels, which serve as a sample set for training neural network models.

[0133] Model application module 14 is used to monitor behavior logs and data transmission speed in real time when a network connection requirement is obtained. Starting from the current time, it packages the monitored data from a preset time period and inputs it into the trained neural network model to obtain the first influence degree.

[0134] The matching and determination module 15 is used to periodically match the packaged data with attack characteristics and determine the second degree of influence based on the matching results;

[0135] The data feedback module 16 is used to compare the first and second impact values, select the larger value as the final impact value, and provide feedback in real time.

[0136] Furthermore, the attack tag generation module 12 includes:

[0137] The utilization rate acquisition unit is used to acquire the CPU utilization rate, GPU utilization rate, and memory utilization rate after each network attack.

[0138] The disk speed query unit is used to pre-insert target files on each disk and obtain the read speed of each target file after each network attack; the target file is at least one, and the depth of different target files is different.

[0139] The impact calculation unit is used to determine the impact based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file;

[0140] The influence application unit is used to use the influence as an attack label.

[0141] Specifically, the matching determination module 15 includes:

[0142] The data packaging unit is used to periodically read the behavior logs and data transmission speed monitored before the preset time period of the package at a preset frequency;

[0143] The initial screening unit is used to calculate the mean and standard deviation of the data transmission speed, and to perform initial screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features.

[0144] The temporal distance calculation unit is used to perform temporal matching between packaged data and reference features based on data transmission speed, and to simultaneously calculate the temporal distance.

[0145] The log distance calculation unit is used to perform logical operations on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold.

[0146] The influence reading unit is used to select a feature from the reference features based on the temporal distance and log distance as the target feature, and read the influence of the target feature as the second influence.

[0147] Furthermore, the calculation process for the time-domain distance is as follows:

[0148] The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence;

[0149] Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence;

[0150] Generate a distance matrix based on the first sequence and the second sequence;

[0151] The time-domain distance is determined based on the distance matrix;

[0152] The element D(i,j) in the distance matrix is ​​dis(x) i ,y j );x i The first sequence is X = (x1, x2, ..., x...). n The data transmission speed of the i-th data in ) is y j For the second sequence Y = (y1, y2, ..., y m The j-th data transmission speed in ); dis(x i ,yj ) is x i and y j The distance between them, wherein the distance is calculated using the first norm and the second norm; n is the total number of elements in the first sequence X, and m is the total number of elements in the second sequence Y;

[0153] The objective function for determining the time-domain distance based on the distance matrix is:

[0154] In the formula, SD represents the time-domain distance, and W = (w1, w2, ..., w K ) represents a path in the distance matrix from position (1,1) to position (n,m), w k Let D(i,j) be the k-th element on the path, where D(i,j) represents the first sequence X = (x1, x2, ..., xj). n The data transmission speed of the i-th data in the sequence Y = (y1, y2, ..., y) is related to the second sequence Y = (y1, y2, ..., y) m The distance between the j-th data transmission speeds in the path; K is the total number of all elements included in the path; This indicates that the path with the minimum length is selected from all paths.

[0155] The calculation process for the log distance is as follows:

[0156] Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A network security analysis method based on big data, characterized in that, The method includes: Based on big data technology, we obtain the behavior logs and data transmission speed of each network attack during the attack period, and use the behavior logs and data transmission speed as attack characteristics. The impact of each network attack is obtained and used as an attack label; the impact is represented by the module utilization rate of the device. Statistical attack features and attack labels are used as a sample set to train a neural network model; When a network connection request is received, the behavior logs and data transmission speed are monitored in real time. Starting from the current moment, the data monitored before the preset time period is packaged and input into the trained neural network model to obtain the first degree of influence. The packaged data is periodically matched with attack characteristics, and the second degree of influence is determined based on the matching results; Compare the first and second impact values, select the larger value as the final impact value, and provide feedback in real time. The step of obtaining the impact of each network attack and using the impact as an attack label includes: Obtain CPU usage, GPU usage, and memory usage after each network attack. Target files are pre-inserted into each disk, and the read speed of each target file is obtained after each network attack; the number of target files is at least one, and the depth of different target files is different. The degree of impact is determined based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file; The impact level is used as an attack label.

2. The network security analysis method based on big data according to claim 1, characterized in that, The step of periodically matching the packaged data with attack characteristics and determining the second influence based on the matching results includes: At a preset frequency, periodically read the behavior logs and data transmission speed monitored before the preset time period of the package; Calculate the mean and standard deviation of data transmission speed, and perform preliminary screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features; Based on the data transmission speed, perform temporal matching between packaged data and reference features, and simultaneously calculate the temporal distance; When the time-domain distance is less than a preset distance threshold, logical operations are performed on the behavior log to determine the log distance; Based on the temporal distance and log distance, select a feature from the reference features as the target feature, and read the influence degree corresponding to the target feature as the second influence degree.

3. The network security analysis method based on big data according to claim 2, characterized in that, The calculation process for the time-domain distance is as follows: The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence; Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence; Generate a distance matrix based on the first sequence and the second sequence; The time-domain distance is determined based on the distance matrix; Elements in the distance matrix ; For the first sequence The first in Data transmission speed, For the second sequence The first in Data transmission speed; for and The element distance between them, wherein the element distance is calculated using the first norm and the second norm; For the first sequence The total number of elements in For the second sequence The total number of elements in; The objective function for determining the time-domain distance based on the distance matrix is: In the formula, For time-domain distance, Represents a line in the distance matrix from Location to The path to the location, For the first on the path k element , Represents the first sequence The data transmission speed of the i-th data in the second sequence The distance between the j-th data transmission speeds; This represents the total number of all elements included in the path. This indicates that the path with the minimum length is selected from all paths. The calculation process for the log distance is as follows: Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

4. The network security analysis method based on big data according to claim 3, characterized in that, The step of selecting a feature from the reference features based on the temporal distance and log distance as the target feature, and reading the influence degree corresponding to the target feature as the second influence degree includes: The temporal distance and log distance are summed according to preset weights to obtain the comprehensive distance; The reference feature with the smallest comprehensive distance is selected as the target feature; The influence degree corresponding to the target feature is read from the sample set and used as the second influence degree.

5. The network security analysis method based on big data according to claim 1, characterized in that, The step of comparing the first and second impact values, selecting the larger value as the final impact value, and providing real-time feedback includes: When the first degree of influence exceeds the second degree of influence, the number of times the first degree of influence is greater than the second degree of influence is accumulated and counted as the number of correct results; Calculate the ratio of the number of correct answers to the total number of second influence values, and adjust the determination frequency of the second influence value according to the ratio; the frequency is inversely proportional to the ratio. When the first degree of influence does not exceed the second degree of influence, the monitored behavior logs and data transmission speed are used as features, the second degree of influence is used as a label and inserted into the sample set to update the neural network model.

6. A network security analysis system based on big data, characterized in that, The system includes: The attack signature generation module is used to obtain the behavior logs and data transmission speed of each network attack behavior during the attack period based on big data technology, and use the behavior logs and data transmission speed as attack signatures. An attack tag generation module is used to obtain the impact of each network attack behavior and use the impact as an attack tag; the impact is represented by the module utilization rate of the device. The sample application module is used to statistically analyze attack features and attack labels, which serve as a sample set for training neural network models. The model application module is used to monitor behavior logs and data transmission speed in real time when a network connection requirement is obtained. Starting from the current moment, it packages the data monitored before a preset time period and inputs it into the trained neural network model to obtain the first influence degree. The matching and determination module is used to periodically match the packaged data with attack characteristics and determine the second degree of influence based on the matching results; The data feedback module is used to compare the first and second impact values, select the larger value as the final impact value, and provide real-time feedback. The attack tag generation module includes: The utilization rate acquisition unit is used to acquire the CPU utilization rate, GPU utilization rate, and memory utilization rate after each network attack. The disk speed query unit is used to pre-insert target files on each disk and obtain the read speed of each target file after each network attack; the number of target files is at least one, and the depth of different target files is different. The impact calculation unit is used to determine the impact based on CPU utilization, GPU utilization, memory utilization, and the reading speed of each target file; The influence application unit is used to use the influence as an attack label.

7. The big data-based network security analysis system according to claim 6, characterized in that, The matching determination module includes: The data packaging unit is used to periodically read the behavior logs and data transmission speed monitored before the preset time period of the package at a preset frequency; The initial screening unit is used to calculate the mean and standard deviation of the data transmission speed, and to perform initial screening of attack features based on the mean and standard deviation, selecting a preset number of attack features as reference features. The temporal distance calculation unit is used to perform temporal matching between packaged data and reference features based on data transmission speed, and to simultaneously calculate the temporal distance. The log distance calculation unit is used to perform logical operations on the behavior log to determine the log distance when the time domain distance is less than a preset distance threshold. The influence reading unit is used to select a feature from the reference features based on the temporal distance and log distance as the target feature, and read the influence of the target feature as the second influence.

8. The big data-based network security analysis system according to claim 7, characterized in that, The calculation process for the time-domain distance is as follows: The data transmission speeds in the packaged data are arranged chronologically to obtain the first sequence; Reference features are selected sequentially, and the data transmission speeds of the selected reference features are arranged in chronological order to obtain the second sequence; Generate a distance matrix based on the first sequence and the second sequence; The time-domain distance is determined based on the distance matrix; Elements in the distance matrix ; For the first sequence The first in Data transmission speed, For the second sequence The first in Data transmission speed; for and The element distance between them, wherein the element distance is calculated using the first norm and the second norm; For the first sequence The total number of elements in For the second sequence The total number of elements in; The objective function for determining the time-domain distance based on the distance matrix is: In the formula, For time-domain distance, Represents a line in the distance matrix from Location to The path to the location, For the first on the path k element , Represents the first sequence The data transmission speed of the i-th data in the second sequence The distance between the j-th data transmission speeds; This represents the total number of all elements included in the path. This indicates that the path with the minimum length is selected from all paths. The calculation process for the log distance is as follows: Convert the two behavior logs to be calculated into sets, calculate the difference between the sets, and use the number of elements in the difference set as the log distance.

Images