Network vulnerability detection method based on Spark multi-source data fusion and feature analysis

Abstract

The invention discloses a network vulnerability detection method based on Spark multi-source data fusion and feature analysis, which comprises the following steps of: acquiring multi-source data in real time, efficiently processing Spark clusters (including data cleaning, processing and feature correlation calculation), constructing and training a vulnerability model, and finally establishing an early warning mechanism. All links are organically combined to realize omnibearing perception and accurate analysis. According to the method, the Spark and the Pearson correlation coefficient are combined, a filtering type feature selection algorithm is applied, and an early warning mechanism is coordinated with each technology, so that the vulnerability detection accuracy and efficiency are improved together, and the network security is better guaranteed; the serious defects of an existing traditional vulnerability detection method in the aspects of processing capacity, response speed, adaptability to novel attacks and the like are effectively overcome.

Description

Technical Field

[0001] The invention belongs to the field of network technology and security technology, and relates to a network vulnerability detection method. Background Art

[0002] Traditional network vulnerability detection methods usually rely on regular scanning and known vulnerability databases to identify security vulnerabilities in the system through static and dynamic analysis. These methods are generally run on a single machine, have limited processing power, focus mainly on specific data sources, are susceptible to rule restrictions, have slow response speeds, and usually require manual intervention to review and process scan results. This makes traditional methods inflexible and inefficient in dealing with new attacks and complex environments, so they fail to fully perceive the network security situation of massive data and issue early warnings when detecting potential vulnerabilities.

[0003] After reading and analyzing the existing literature, it is found that in the existing patent literature:

[0004] Document 1 (CN118233174A) proposes a network security anomaly information monitoring method and system, which first collects the traffic data and user behavior data of the network port, wherein the traffic data includes traffic size data, traffic rate, packet size, packet time interval, protocol type and source address and destination address of the traffic data packet, and the user behavior data includes user login behavior data and user access behavior data, and extracts features from the traffic data and user behavior data; models are trained on the features extracted from the traffic data and user behavior data; the network environment is detected in real time through the model, and model parameters are updated and adjusted; data feedback is performed on the model to optimize the model. This patent document mainly collects the traffic data and user behavior data of the network port, and extracts features from the traffic data and user behavior data; however, this method only adopts a specific data processing method, and does not use cluster computing resources to process different source data in parallel, and cannot be applied to comprehensive feature extraction of massive network data.

[0005] Document 2 (CN118626982A) proposes a multimodal anomaly detection method and system for big data network traffic, the method comprising: obtaining traffic data generated by an intranet, and performing flow analysis on the traffic data to obtain deep packet detection features of the traffic data; performing abnormal behavior detection on the traffic data based on the deep packet detection features through a machine learning model to obtain multiple suspected abnormal behavior traffic in the traffic data; inputting each of the suspected abnormal behavior traffic into a deep learning model, extracting the multimodal features of each of the suspected abnormal behavior traffic and performing feature fusion to obtain a representation vector of each of the suspected abnormal behavior traffic; performing detection and classification based on the representation vector of each of the suspected abnormal behavior traffic to obtain an abnormal behavior classification result for each of the suspected abnormal behavior traffic. This patent document mainly uses Spark streaming to encapsulate multiple batches of traffic data and extract features; however, it does not involve various correlation indicators such as the Pearson correlation coefficient and mutual information. It simply extracts features from the traffic data and does not select the most valuable features for vulnerability detection based on a filtering feature selection algorithm combined with multiple correlation indicators. It is unable to screen out features that are highly correlated with vulnerability detection to reduce interference from irrelevant features, resulting in low accuracy and efficiency of vulnerability detection.

[0006] Document 3 (CN202010200957) proposes a security analysis method for open source software, which uses a distributed index library and a pre-built Spark-based machine learning model to analyze and identify the source code and obtain vulnerabilities in the source software; the shortcomings of this patent document are that the data source is single, and only the parameters of the calculation model are compared. A filtering feature selection algorithm is not used, which is easily restricted by rules and lacks a corresponding early warning feedback module. Therefore, this solution also has the problems of low vulnerability detection accuracy and low efficiency.

[0007] In summary, the existing vulnerability detection technology does not utilize Spark's distributed computing capabilities to achieve multi-source data fusion and large-scale data processing in data processing. The technology has a single data source and can only process limited specific data, making it difficult to cope with large-scale data in a complex network environment. In terms of data feature extraction and selection, the existing technology often only extracts some common features, while ignoring other important features, and cannot screen out features with high correlation with vulnerability detection, causing the model to be disturbed by irrelevant features, and the accuracy and efficiency of vulnerability detection are low. In terms of early warning mechanism, there is a lack of a perfect early warning mechanism like the present invention. When a potential vulnerability or abnormal situation is detected, an early warning cannot be issued in time, and relevant personnel cannot obtain information and take measures in time. Summary of the invention

[0008] The purpose of the present invention is to provide a network vulnerability detection method based on Spark multi-source data fusion and feature analysis to address the deficiencies in the prior art. The present invention effectively solves the serious defects of the existing traditional vulnerability detection methods in terms of processing capacity, response speed, and adaptability to new attacks; it realizes comprehensive perception and analysis of network security situation, quickly processes massive data, identifies potential vulnerabilities and complex attack patterns with high precision, adapts to changing attack environments, improves detection real-time and accuracy, protects information system security, and ensures data integrity and business continuity.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A network vulnerability detection method based on Spark multi-source data fusion and feature analysis includes the following steps:

[0011] S1: collects data information from different sources in real time and transmits the data information to the Spark cluster for processing;

[0012] S2: Data cleaning and real-time data processing for heterogeneous data sources in the Spark cluster;

[0013] S3: Using a filtering feature selection algorithm on the data processed in step S2 to calculate the correlation between the features and the vulnerability detection results;

[0014] The filtering feature selection algorithm is implemented using the Pearson correlation coefficient and / or mutual information, so as to obtain the most valuable feature data for vulnerability detection;

[0015] S4: Vulnerability detection model construction and training, by continuously optimizing the model to adapt to emerging attack patterns;

[0016] S5: Establish an early warning mechanism. When the vulnerability detection model determines that there is a potential vulnerability or abnormal situation, the early warning system is triggered. If an early warning signal needs to be issued, the early warning information sending module sends the early warning information to relevant personnel or systems.

[0017] The present invention combines multiple sources of data in real time, efficiently processes them in Spark clusters (including data cleaning, processing, and feature correlation calculation), builds and trains vulnerability models, and finally establishes an early warning mechanism. All aspects are organically combined to achieve all-round perception and accurate analysis. The present invention combines Spark with the Pearson correlation coefficient, applies a filtering feature selection algorithm, and coordinates the early warning mechanism with various technologies to jointly improve the accuracy and efficiency of vulnerability detection and better ensure network security.

[0018] In the network vulnerability detection system of the present invention, the distributed computing capability of Spark is combined with the Pearson correlation coefficient to show its advantages. The distributed capability of Spar enables the system to efficiently process large-scale data from multiple sources such as network traffic, system logs, system monitoring, etc.; the distributed storage and calculation of data avoids the problem of limited processing capability of a traditional single machine, and provides a basis for comprehensive feature extraction; by processing different data sources, it can extract rich features such as packet size, protocol type, IP address, etc. in network traffic data, as well as event type, timestamp, etc. in system log data. The Pearson correlation coefficient is used to measure the linear correlation between features and vulnerability detection results, and its value is between -1 and 1. The closer the absolute value is to 1, the stronger the correlation. Based on the large number of features extracted by Spark, the Pearson correlation coefficient can accurately screen out features that are highly correlated with vulnerability detection. The distributed capability of Spark and the Pearson correlation coefficient cooperate with each other to jointly improve the performance of the entire network vulnerability detection system.

[0019] The present invention further illustrates that when the filtering feature selection algorithm in step S3 is implemented using the Pearson correlation coefficient, step S3 includes:

[0020] S301: feature extraction;

[0021] Extracting data features from the data processed in step S2;

[0022] S302: Correlation calculation;

[0023] ①Assume that the characteristic vector X of one set of data is 1 ,x 2 ,…,x N} and vulnerability detection result vector Y = {y 1 ,y 2 ,…,y M}; where N is the number of extracted features and M is the number of elements in the vulnerability detection result vector;

[0024] ②For feature x i Pearson correlation coefficient with vulnerability detection result vector Y Use the following formula to calculate:

[0025]

[0026] Among them, x ij The feature x represents the jth sample (i.e., the data instance used for vulnerability detection analysis) i The value of x i Represents feature x i The mean value of y j represents the vulnerability detection result of the jth sample, It represents the mean of vulnerability detection results, and m represents the number of indicators related to vulnerability detection.

[0027] By calculating the Pearson correlation coefficient between each feature and the vulnerability detection result, the larger the absolute value of the Pearson correlation coefficient, the stronger the correlation.

[0028] The present invention further illustrates that when the filtering feature selection algorithm in step S3 is implemented using mutual information, step S3 includes:

[0029] ①Define the feature vector X and the vulnerability detection result vector Y;

[0030] ② Calculate the joint probability distribution and marginal probability distribution;

[0031] ③ Use the following formula to calculate feature x i The mutual information I(x i ,Y):

[0032]

[0033] Where x is an element in the feature vector X, X is the feature vector, y is an element in the vulnerability detection result vector, and Y is the vulnerability detection result vector; p(x, y) is the joint probability distribution, which indicates the probability of the feature (such as the source IP address) and the detection result (such as whether the vulnerability is detected) appearing at the same time; p(x) is the marginal probability distribution, which indicates the probability of the feature appearing alone; p(y) is the marginal probability distribution, which indicates the probability of the detection result appearing alone. ；

[0034] The larger the calculated mutual information value between a certain feature and the vulnerability detection result is, the higher the correlation between the feature and the vulnerability detection result is.

[0035] The present invention further illustrates that in step S4, the vulnerability detection model is constructed and trained by continuously optimizing the model, using the features of multi-source data to construct the model, training and evaluating with labeled data, adjusting parameters based on the results, and continuously updating the model in combination with new attack-related new data to continuously adapt to new attack patterns.

[0036] The present invention further illustrates that in step S2, the data cleaning is to use Spark's DataFrame API to perform a cleaning operation on the data, including removing duplicate data, processing missing values ​​and outliers, etc.

[0037] The present invention further illustrates that in step S2, the real-time data processing is to process the real-time data stream in a micro-batch processing manner using the Spark Streaming component to achieve real-time conversion and preprocessing of the data.

[0038] The present invention further illustrates that, in step S1, the different source data information includes network flow data, system log data, system monitoring data, and the like.

[0039] In order to better understand the present invention, some technical terms are explained as follows:

[0040] Spark: (Apache Spark) An open source distributed computing system for large-scale data processing.

[0041] Feature extraction: The process of extracting representative and distinguishing features from raw data. For example, in network security vulnerability detection, for network traffic data, extract features such as packet size, protocol type, IP address, etc., so that they can be used for model training and analysis to help identify vulnerability patterns.

[0042] Pearson correlation coefficient: A statistical indicator used to measure the linear correlation between two variables. In vulnerability detection feature selection, it is used to calculate the linear correlation between features and vulnerability detection results. The value range is between -1 and 1. The closer the absolute value is to 1, the stronger the correlation is.

[0043] Mutual Information: An information theory metric that measures the correlation between two random variables. In feature selection, it is used to evaluate the degree of correlation between features and vulnerability detection results. The larger the mutual information value, the higher the correlation between the feature and the result.

[0044] In the present invention, Spark is combined with the Pearson correlation coefficient: Spark's distributed computing capability provides a wider data basis for the application of the Pearson correlation coefficient. Spark can process large-scale multi-source data, so that the Pearson correlation coefficient can be calculated on more feature data. At the same time, the Pearson correlation coefficient uses the features extracted by Spark to accurately screen out features that are highly correlated with vulnerability detection. This combination improves the accuracy and efficiency of vulnerability detection, which is an important innovation of the present invention.

[0045] In the present invention, a filtering feature selection algorithm is applied: the present invention adopts a filtering feature selection algorithm, combined with multiple correlation indicators such as the Pearson correlation coefficient and mutual information to evaluate the correlation between each feature and the vulnerability detection result. By comprehensively considering the results of different correlation indicators, it is possible to screen out features with high correlation with vulnerability detection, reduce the interference of irrelevant features, and make the model more focused on features that are valuable for vulnerability detection. Compared with the prior art, the application of this filtering feature selection algorithm can improve the accuracy and efficiency of vulnerability detection, which is another important innovation of the present invention.

[0046] In the present invention, the early warning mechanism works in coordination with other technologies: the early warning mechanism works in coordination with the previous multi-source data collection, Spark cluster processing, feature correlation calculation, and vulnerability model construction and training. When the vulnerability detection model determines that there is a potential vulnerability or an abnormal situation, the early warning mechanism can be triggered in time and send early warning information. This collaborative working mode makes the entire network vulnerability detection system more complete and efficient, and can better ensure network security.

[0047] Advantages of the present invention:

[0048] 1. The present invention utilizes Spark's powerful distributed computing capabilities, deeply integrates multi-dimensional data sources, uses a filtering feature selection algorithm, and combines multiple correlation indicators such as the Pearson correlation coefficient and mutual information to select the most valuable features for vulnerability detection, thereby achieving all-round perception and accurate analysis of network security situation, thereby identifying potential vulnerabilities and complex attack patterns with high accuracy.

[0049] 2. The present invention continuously optimizes model parameters to enable it to flexibly adapt to the ever-changing attack environment, greatly improve the real-time and accuracy of detection, build a solid and reliable security line for the information system, and effectively ensure data integrity and business continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of a system architecture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described below in conjunction with the accompanying drawings.

[0052] Embodiment: A network vulnerability detection method based on Spark multi-source data fusion and feature analysis, comprising the following steps:

[0053] S1: collects network traffic data, system log data, system monitoring data and other source data in real time, and transmits the above data information to the Spark cluster for processing;

[0054] S2: Data cleaning and real-time data processing for heterogeneous data sources in the Spark cluster;

[0055] S3: Using a filtering feature selection algorithm on the data processed in step S2 to calculate the correlation between the features and the vulnerability detection results;

[0056] The filtering feature selection algorithm is provided with a Pearson correlation coefficient and / or mutual information, so as to obtain the most valuable feature data for vulnerability detection;

[0057] S4: Vulnerability detection model construction and training, by continuously optimizing the model to adapt to emerging attack patterns;

[0058] S5: Establish an early warning mechanism. When the vulnerability detection model determines that there is a potential vulnerability or abnormal situation, the early warning system is triggered. If an early warning signal needs to be issued, the early warning information sending module sends the early warning information to relevant personnel or systems.

[0059] Application Example 1:

[0060] Taking the operator's network security system as an example, combined with the attached Figure 1 , wherein the filtering feature selection algorithm is implemented using the Pearson correlation coefficient. The process (method) of building a network vulnerability detection system based on Spark multi-source data fusion and feature analysis includes the following steps:

[0061] 1. Data source layer

[0062] Data sources: The data sources in the operator's network security system include network traffic data sources, system log data sources, and system monitoring data sources. Network traffic data contains a large amount of network connection information, system logs record various operations and events of the system, and system monitoring data reflects the operating status of the system.

[0063] 2. Data Collection Layer

[0064] Real-time collection: Responsible for collecting data information from different sources in real time. For example, collecting network traffic data through network monitoring tools, reading system log data from system log files, and obtaining system monitoring data from system monitoring tools; then transmitting this data to the Spark cluster for processing.

[0065] 3.Spark cluster layer

[0066] Data cleaning: Use Spark's DataFrame API to clean the data. For network traffic data, remove duplicate data packet records, process missing field values ​​and abnormal values ​​in the data packet (such as unreasonable data packet size, etc.). For system logs and system monitoring data, also remove duplicate records, process missing values ​​and abnormal values ​​to ensure data quality.

[0067] Real-time data processing: Spark Streaming components are used to process real-time data streams. Taking network traffic data as an example, it can convert and preprocess the real-time inflowing network traffic data in a micro-batch manner, such as parsing data packets and extracting relevant features.

[0068] Feature correlation calculation (Pearson correlation coefficient):

[0069] Feature extraction: For network traffic data, features such as packet size, protocol type, source IP address, and destination IP address can be extracted. Assuming there are n packets in the network traffic data and each packet has m features, the features of the network traffic data can be represented by an n*m matrix. For system log and application log data, features such as event type, timestamp, user ID, and operation type can be extracted and represented by corresponding matrices.

[0070] Correlation calculation:

[0071] ① A set of network traffic data is used for vulnerability detection, where a set of feature vectors X = {x 1 ,x 2 ,…,x N} (where N is the number of extracted network traffic features) and vulnerability detection result vector Y = {y 1 ,y 2 ,…,y M}(Assume that y in Y 1 Indicates whether there is a vulnerability, y 2 ~y M Other indicators related to vulnerability detection, such as the risk level of the vulnerability, the scope of possible impact, etc.).

[0072] ②For x i The Pearson correlation coefficient with the vulnerability detection result vector Y is calculated using the following formula: i Correlation with vulnerability detection result Y

[0073]

[0074] Among them, x ij Represents the feature x of the jth sample i The value of Represents feature x i The mean value of y j represents the vulnerability detection result of the jth sample, It represents the mean of vulnerability detection results, and m represents the number of indicators related to vulnerability detection.

[0075] By calculating the Pearson correlation coefficient between each feature and the vulnerability detection result, the larger the absolute value, the stronger the correlation. The feature with stronger correlation can be selected. For example, if the calculation finds that the packet size feature x i The high Pearson correlation coefficient with the vulnerability detection results indicates that the packet size is correlated with the vulnerability to some extent, and perhaps larger or smaller packet sizes are more easily associated with certain types of vulnerabilities.

[0076] 4. Vulnerability Detection Layer

[0077] Vulnerability model construction and training: Based on the features extracted from various data sources and the key features screened by the Pearson correlation coefficient, the decision tree algorithm is used as an example to take the features as nodes of the decision tree and determine whether there is a vulnerability based on the different branching conditions of the feature values. The constructed model is trained using a large amount of labeled data. During the training process, the model parameters are continuously adjusted so that the model can better adapt to the characteristics and rules of the data.

[0078] Vulnerability detection: Detect whether there are potential vulnerabilities or anomalies. By testing the processed and analyzed data, determine whether there are situations that meet the characteristics of vulnerabilities.

[0079] 5. Early warning layer

[0080] Warning trigger: When the vulnerability detection model determines that there is a potential vulnerability or abnormal situation, the warning system is triggered.

[0081] Warning information sending: The warning information sending module sends warning information to relevant personnel or systems. For example, the warning information is sent to the network security operation and maintenance personnel of the operator so that they can take timely measures to deal with potential vulnerabilities or abnormal situations.

[0082] Application Example 2:

[0083] The difference between this application example and application example 1 is that the mutual information algorithm is used for calculation. Taking the network firewall system as an example, the algorithm steps are as follows:

[0084] ① Define the feature vector X and the result vector Y, where the feature vector may contain the source IP address of the network connection (x 1 )、Destination IP address(x 2 ), protocol type (x 3 )、Port Number(x 4 ), connection duration (x 5 ) and other network connection related features; the result vector Y contains whether the vulnerability is detected (y 1 ), the severity of the vulnerability (y 2 ) and other test result related features.

[0085] ② Calculate the joint probability distribution and marginal probability distribution, for example: for the source IP address (x 1 ) and whether a vulnerability is detected (y 1 ):

[0086] Count the number of times vulnerabilities are detected in network connections with different source IP addresses. Assume that there are 100 network connections with a source IP address of 192.168.1.10, and vulnerabilities are detected in 20 of them;

[0087] Calculate the joint probability distribution p(x1 ,y 1 )) and the marginal probability distribution p(x 1 )、p(y 1 ):

[0088]

[0089]

[0090]

[0091] Similar statistics and probability distribution calculations are performed for other feature combinations such as destination IP address and vulnerability severity;

[0092] ③ Use the formula to calculate the mutual information between each feature and the detection result:

[0093]

[0094] Where x is an element in the feature vector X, X is the feature vector, y is an element in the vulnerability detection result vector, and Y is the vulnerability detection result vector; p(x,y) is the joint probability distribution, which indicates the probability of the feature (such as the source IP address) and the detection result (such as whether the vulnerability is detected) appearing at the same time; p(x) is the marginal probability distribution, which indicates the probability of the feature appearing alone; p(y) is the marginal probability distribution, which indicates the probability of the detection result appearing alone;

[0095] If the calculated mutual information value between a certain feature and the detection result is large, it means that the feature has a high correlation with the detection result. If the mutual information value between the source IP address and whether a vulnerability is detected is large, it means that the source IP address is a factor that has an important impact on the detection result. When optimizing the vulnerability detection function of the network firewall system, this feature can be considered as a priority.

[0096] Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation of the present invention. For ordinary technicians in the relevant technical field, other different forms of changes or modifications can be made based on the above description; it is not necessary and impossible to list all the implementation methods here; and the obvious changes or modifications derived from them are still within the protection scope of the present invention.

Claims

1. A network vulnerability detection method based on Spark multi-source data fusion and feature analysis, characterized in that The following steps are involved: S1: collects data information from different sources in real time and transmits the data information to the Spark cluster for processing; S2: Data cleaning and real-time data processing for heterogeneous data sources in the Spark cluster; S3: Using a filtering feature selection algorithm on the data processed in step S2 to calculate the correlation between the features and the vulnerability detection results; The filtering feature selection algorithm is implemented using the Pearson correlation coefficient and / or mutual information, so as to obtain the most valuable feature data for vulnerability detection; S4: Vulnerability detection model construction and training, by continuously optimizing the model to adapt to emerging attack patterns; S5: Establish an early warning mechanism. When the vulnerability detection model determines that there is a potential vulnerability or abnormal situation, the early warning system is triggered.

2. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: When the filtering feature selection algorithm in step S3 is implemented by using the Pearson correlation coefficient, step S3 includes: S301: feature extraction; Extracting data features from the data processed in step S2; S302: Correlation calculation; ①Assume that the eigenvector of one set of data is X={x1,x2,…,x N } and vulnerability detection result vector Y = {y1,y2,…,y M }; where N is the number of extracted features and M is the number of elements in the vulnerability detection result vector; ②For feature x i Pearson correlation coefficient with vulnerability detection result vector Y Use the following formula to calculate: Among them, x ij Represents the feature x of the jth sample i The value of Represents feature x i The mean value of y j represents the vulnerability detection result of the jth sample, represents the mean of vulnerability detection results, and m represents the number of indicators related to vulnerability detection; By calculating the Pearson correlation coefficient between each feature and the vulnerability detection result, the larger the absolute value of the Pearson correlation coefficient, the stronger the correlation.

3. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: When the filtering feature selection algorithm in step S3 is implemented using mutual information, step S3 includes: ①Define the feature vector X and the vulnerability detection result vector Y; ② Calculate the joint probability distribution and marginal probability distribution; ③ Use the following formula to calculate feature x i The mutual information I(x i ,Y): Where x is an element in the feature vector X, X is the feature vector, y is an element in the vulnerability detection result vector, and Y is the vulnerability detection result vector; p(x,y) is the joint probability distribution, which indicates the probability of the feature and the detection result appearing at the same time; p(x) is the marginal probability distribution, which indicates the probability of the feature appearing alone; p(y) is the marginal probability distribution, which indicates the probability of the detection result appearing alone; The larger the calculated mutual information value between a certain feature and the vulnerability detection result is, the higher the correlation between the feature and the vulnerability detection result is.

4. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: In step S4, the vulnerability detection model is constructed and trained by continuously optimizing the model, using the features of multi-source data to construct the model, training and evaluating with labeled data, adjusting parameters based on the results, and continuously updating the model in combination with new attack-related data to continuously adapt to new attack patterns.

5. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: In step S2, the data cleaning is to use Spark's DataFrame API to perform data cleaning operations, including removing duplicate data, processing missing values ​​and outliers.

6. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: In step S2, the real-time data processing is to process the real-time data stream in a micro-batch processing manner using the Spark Streaming component to achieve real-time conversion and preprocessing of data.

7. The network vulnerability detection method based on Spark multi-source data fusion and feature analysis according to claim 1 is characterized in that: In step S1, the different source data information includes network traffic data, system log data and system monitoring data.

Images