A system intrusion detection method based on SAE and FSVM
By employing SAE and FSVM intrusion detection methods on recruitment platforms, monitoring network traffic, and using stacked autoencoders and support vector machines for feature learning and classification, the security issues of recruitment platforms are resolved, efficient abnormal data identification and response are achieved, and system security is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG WANYOUMALI NETWORK TECHNOLOGY CO LTD SHANGHAI BRANCH
- Filing Date
- 2024-12-05
- Publication Date
- 2026-06-05
AI Technical Summary
Due to their large scale and openness, recruitment platforms have become a prime target for cyberattacks, leading to security issues such as denial-of-service attacks, data breaches, and missing information.
A system intrusion detection method based on SAE and FSVM is adopted. By monitoring network traffic and generating real-time data, preprocessing and dimensionality reduction are performed. Stacked autoencoders and support vector machine models are used for feature learning and classification to identify abnormal data and take response measures.
It improves the efficiency and accuracy of intrusion detection, enabling rapid identification and response to abnormal behavior, and reduces the platform's security risks.
Smart Images
Figure CN122160075A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of system security technology, specifically to a system intrusion detection method based on SAE and FSVM. Background Technology
[0002] As a platform for personal and corporate information, recruitment platforms bring convenience to job seekers and companies. However, due to their large scale, openness, and complexity, these platforms also pose greater challenges to personal information security and privacy protection. Their centralized management makes them a prime target for cyberattacks, potentially leading to security issues such as denial of service, information leaks, and data loss or corruption.
[0003] Therefore, a new technical solution needs to be designed to address this issue. Summary of the Invention
[0004] The purpose of this invention is to provide a system intrusion detection method based on SAE and FSVM, which solves the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a system intrusion detection method based on SAE and FSVM, comprising the following method steps:
[0006] Step 1: Set up a monitoring system at the network port of the recruitment platform to monitor and intercept network traffic entering the platform system and generate real-time data;
[0007] Step two involves processing the historical data and using the processed data for rule learning in the intrusion detection model to learn the characteristics of normal and abnormal data.
[0008] Step 3: Preprocess the real-time data intercepted in Step 1, and use the autoencoder model obtained in Step 2 to perform dimensionality reduction on the real-time data. The processed data is then classified using the model generated in Step 2, and the characteristics of the data are labeled.
[0009] Step four: Put the abnormal data and some normal data from step three into the historical database to facilitate the relearning of the rules of the intrusion detection model and improve the model.
[0010] Step 5: Based on the annotations in Step 3, the system separates and filters the real-time data and provides response measures for abnormal data, such as warnings and blocking.
[0011] In a preferred embodiment of the present invention, in step two, a stacked autoencoder is constructed using historical data. The structure and parameters of the neural network model are optimized according to the scale and dimension of the actual data. When the dimension of the data is too high, the model complexity can be increased to ensure good working results. The trained stacked autoencoder is used to perform feature processing on these historical data.
[0012] The data after feature processing needs to be reduced in size to obtain the support vector set from the historical data. The accuracy of the algorithm and the size of the reduced data are adjusted by setting the cluster radius r and the number of similar clusters n. The parameter r is inversely proportional to the algorithm time and directly proportional to the size of the reduced data; the parameter n is directly proportional to the time and directly proportional to the size of the reduced data.
[0013] The scaled-down dataset is used as training samples and input into the SVM model to obtain a model structure for identifying normal and abnormal data.
[0014] As a preferred embodiment of the present invention, the specific steps of the model training phase are as follows:
[0015] Step 1: Data preprocessing. For non-numerical data in the network data, feature extraction is used to convert them into numerical data. Then, the model uses data standardization to make the network data dimensionless.
[0016] Step 2, Feature processing: The stacked autoencoder model learns rules from the sample data through a layer-by-layer greedy algorithm, and then extracts features from the sample data, continuously abstracting the original data to a low-dimensional space.
[0017] Step 3: Sample reduction. The size of the sample data is reduced to obtain a support vector dataset, which improves the training speed of the SVM model.
[0018] Step four: SVM classifier training. The processed sample data is used to perform feature learning on the SVM classifier model to obtain a classifier model based on normal and abnormal data.
[0019] In a preferred embodiment of the present invention, the data detection stage is as follows:
[0020] Step 1: Data preprocessing, including feature extraction and standardization of network data;
[0021] Step 2, Feature processing: Use the stacked autoencoder model obtained during the training phase to extract features from the network data and reduce the dimensionality of the network data.
[0022] Step 3: SVM classifier processing. The SVM model obtained during the training phase is used to classify the processed network data and identify normal and abnormal data.
[0023] As a preferred embodiment of the present invention, the specific steps of sample reduction are as follows:
[0024] Step 1: Perform the following clustering operation on the normal data in the sample data. Randomly select a data point from the normal sample set as the cluster center and put it into the set Ncore. Set the value of the cluster radius r. Traverse the normal sample set and calculate the similarity between the sample data and the cluster centers in the set Ncore. Put the samples with similarity less than R into the corresponding clusters. When a data sample that does not belong to the cluster appears, set the sample as a new cluster center. Continue to traverse the normal sample set until all sample data are in clusters, and obtain the cluster set N, where the i-th element is represented as ni, which is a cluster, and its corresponding cluster center is nc i.
[0025] Step 2: Repeat the operation of Step 1 to process the abnormal data in the sample data in the same way to obtain a cluster set A, where the j-th element is represented by aj, which is a cluster, and its corresponding cluster center is acj.
[0026] Step 3: Perform Cartesian product operation (N*A) on N and A. For each element (ni, aj) in set N*A, calculate the similarity between the cluster centers of ni and aj.
[0027] Step 4: Classify the elements in N*A based on ni. Sort the elements in each category according to the similarity obtained in the previous step from small to large. Set the number of similar clusters n. Select the first n elements and take aj elements from these n elements to form a new sample set AA.
[0028] Step 5: Repeat the operation of step 4. Classify the elements in N*A according to aj. Sort the elements in each category from small to large according to the similarity obtained in step 4. Select the first n elements and take aj elements from these n elements to form a new sample set NN.
[0029] Step 6: Merge the sample sets AA and NN to generate a new sample set D, and use D as the original input sample for SVM.
[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0031] This invention, while performing feature processing on network data using SAE, employs a scaling strategy to reduce the size of sample data, which effectively improves the learning speed and learning rate of the SVM classifier, thereby enhancing the efficiency of intrusion detection. Attached Figure Description
[0032] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0033] Figure 1 This is a flowchart of a system intrusion detection method based on SAE and FSVM according to the present invention. Detailed Implementation
[0034] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0035] This invention provides an intrusion detection method that combines stacked autoencoders and fast support vector machines. It uses stacked autoencoders and sample reduction strategies to improve the speed of detection model construction and improve model efficiency. Before detection data, a certain amount of historical data is required to provide rule learning for the model, which includes normal data and abnormal data.
[0036] The following is a further description of the present invention:
[0037] Step 1: Set up a monitoring system at the network port of the recruitment platform to monitor and intercept network traffic entering the platform system and generate real-time data;
[0038] Step 2: Process the historical data and use the processed data for rule learning of the intrusion detection model to learn the characteristics of normal and abnormal data;
[0039] During the process, a stacked autoencoder is constructed using historical data. The structure and parameters of the neural network model are optimized according to the scale and dimensionality of the actual data. When the dimensionality of the data is too high, the model complexity can be increased to ensure good performance. The trained stacked autoencoder is then used to perform feature processing on this historical data.
[0040] The data after feature processing needs to be reduced in size to obtain the support vector set from historical data. The accuracy of the algorithm and the size of the reduced data are adjusted by setting the cluster radius r and the number of similar clusters n. The parameter r is inversely proportional to the algorithm's time and directly proportional to the size of the reduced data; the parameter n is directly proportional to its time and directly proportional to the size of the reduced data.
[0041] The scaled-down dataset is used as training samples and input into the SVM model to obtain a model structure for identifying normal and abnormal data.
[0042] Step 3: Preprocess the real-time data intercepted in Step 1, and use the autoencoder model obtained in Step 2 to perform dimensionality reduction on the real-time data. The processed data is then classified using the model generated in Step 2, and the characteristics of the data (normal or abnormal) are labeled.
[0043] Step 4: Put the abnormal data and some normal data from Step 3 into the historical database to facilitate the relearning of the rules of the intrusion detection model and improve the model.
[0044] Step 5: Based on the annotations in Step 3, the system separates and filters the real-time data and provides response measures for abnormal data, such as warnings and blocking.
[0045] To further clarify:
[0046] The model is divided into a training phase and a recognition phase, and the overall process is as follows: Figure 1 As shown.
[0047] The specific steps in the model training phase are as follows:
[0048] Data preprocessing involves converting non-numerical data in the network data into numerical data through feature extraction. Next, the model standardizes the network data to make it dimensionless.
[0049] Feature processing: The stacked autoencoder model learns rules from sample data through a layer-by-layer greedy algorithm, and then extracts features from the sample data, continuously abstracting the original data to a low-dimensional space.
[0050] Sample reduction involves reducing the size of the sample data to obtain a support vector dataset, thereby improving the training speed of the SVM model.
[0051] SVM classifier training involves using processed sample data to learn features from the SVM classifier model, resulting in a classifier model based on both normal and abnormal data.
[0052] Data detection phase:
[0053] Data preprocessing includes feature extraction and standardization of network data;
[0054] Feature processing involves using the stacked autoencoder model obtained during the training phase to extract features from the network data, thereby reducing the dimensionality of the network data.
[0055] The SVM classifier process uses the SVM model obtained during the training phase to classify the processed network data, identifying normal and abnormal data.
[0056] The specific steps for sample reduction are as follows:
[0057] The following clustering operation is performed on the normal data in the sample data: A data point is randomly selected from the normal sample set as the cluster center and placed into the set Ncore. The cluster radius r is set. The normal sample set is traversed, and the similarity between the sample data and the cluster centers in set Ncore is calculated sequentially. Samples with a similarity less than R are placed into the corresponding clusters. When a sample does not belong to a given cluster, it is set as a new cluster center. The process continues traversing the normal sample set until all sample data are in clusters, resulting in a cluster set N. The i-th element is denoted as ni, which represents a cluster, and its corresponding cluster center is nc i.
[0058] Repeat step 1 to process the abnormal data in the sample data in the same way to obtain a cluster set A, where the j-th element is represented by aj, which is a cluster and its corresponding cluster center is acj.
[0059] Perform a Cartesian product operation (N*A) on N and A, and for each element (ni, aj) in the set N*A, calculate the similarity between the cluster centers of ni and aj;
[0060] Classify the elements in N*A based on ni, sort the elements in each category according to the similarity obtained in the previous step from small to large, set the number of similar clusters n, select the first n elements, and take aj elements from these n elements to form a new sample set AA;
[0061] Repeat step 4 to classify the elements in N*A based on aj. Sort the elements in each category according to the similarity obtained in step 4 from small to large, select the first n elements, and take aj elements from these n elements to form a new sample set NN.
[0062] Merge the sample sets AA and NN to generate a new sample set D, and use D as the original input sample for SVM.
[0063] During this process, the optimal values of cluster radius r and the number of similar clusters n in clustering will be different for different experimental samples and samples after different preprocessing schemes. They need to be optimized reasonably so that the set D is close to the SV dataset and the training speed of the SVM model can be improved. However, this also carries the risk of reducing the accuracy of the model.
[0064] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A system intrusion detection method based on SAE and FSVM, characterized in that, The methods and steps include the following: Step 1: Set up a monitoring system at the network port of the recruitment platform to monitor and intercept network traffic entering the platform system and generate real-time data; Step two involves processing the historical data and using the processed data for rule learning in the intrusion detection model to learn the characteristics of normal and abnormal data. Step 3: Preprocess the real-time data intercepted in Step 1, and use the autoencoder model obtained in Step 2 to perform dimensionality reduction on the real-time data. The processed data is then classified using the model generated in Step 2, and the characteristics of the data are labeled. Step four: Put the abnormal data and some normal data from step three into the historical database to facilitate the relearning of the rules of the intrusion detection model and improve the model. Step 5: Based on the annotations in Step 3, the system separates and filters the real-time data and provides response measures for abnormal data.
2. The system intrusion detection method based on SAE and FSVM according to claim 1, characterized in that: In step two, a stacked autoencoder is constructed using historical data. The structure and parameters of the neural network model are optimized according to the scale and dimension of the actual data. When the dimension of the data is too high, the complexity of the model can be increased to ensure good working results. The trained stacked autoencoder is used to perform feature processing on these historical data. The data after feature processing needs to be reduced in size to obtain the support vector set from the historical data. The accuracy of the algorithm and the size of the reduced data are adjusted by setting the cluster radius r and the number of similar clusters n. The parameter r is inversely proportional to the algorithm time and directly proportional to the size of the reduced data; the parameter n is directly proportional to the time and directly proportional to the size of the reduced data. The scaled-down dataset is used as training samples and input into the SVM model to obtain a model structure for identifying normal and abnormal data.
3. The system intrusion detection method based on SAE and FSVM according to claim 1, characterized in that: The specific steps in the model training phase are as follows: Step 1: Data preprocessing. For non-numerical data in the network data, feature extraction is used to convert them into numerical data. Then, the model uses data standardization to make the network data dimensionless. Step 2, Feature processing: The stacked autoencoder model learns rules from the sample data through a layer-by-layer greedy algorithm, and then extracts features from the sample data, continuously abstracting the original data to a low-dimensional space. Step 3: Sample reduction. The size of the sample data is reduced to obtain a support vector dataset, which improves the training speed of the SVM model. Step four: SVM classifier training. The processed sample data is used to perform feature learning on the SVM classifier model to obtain a classifier model based on normal and abnormal data.
4. The system intrusion detection method based on SAE and FSVM according to claim 1, characterized in that: Data detection phase: Step 1: Data preprocessing, including feature extraction and standardization of network data; Step 2, Feature processing: Use the stacked autoencoder model obtained during the training phase to extract features from the network data and reduce the dimensionality of the network data. Step 3: SVM classifier processing. The SVM model obtained during the training phase is used to classify the processed network data and identify normal and abnormal data.
5. A system intrusion detection method based on SAE and FSVM according to claim 3, characterized in that: The specific steps for sample reduction are as follows: Step 1: Perform the following clustering operation on the normal data in the sample data. Randomly select a data point from the normal sample set as the cluster center and put it into the set Ncore. Set the value of the cluster radius r. Traverse the normal sample set and calculate the similarity between the sample data and the cluster centers in the set Ncore. Put the samples with similarity less than R into the corresponding cluster. When a data sample does not belong to the cluster, set the sample as a new cluster center. Continue to traverse the normal sample set until all sample data are in the cluster, and obtain the cluster set N, where the i-th element is represented as ni, which is a cluster, and its corresponding cluster center is nci. Step 2: Repeat the operation of Step 1 to process the abnormal data in the sample data in the same way to obtain a cluster set A, where the j-th element is represented by aj, which is a cluster, and its corresponding cluster center is acj. Step 3: Perform Cartesian product operation (N*A) on N and A. For each element (ni, aj) in set N*A, calculate the similarity between the cluster centers of ni and aj. Step 4: Classify the elements in N*A based on ni. Sort the elements in each category according to the similarity obtained in the previous step from small to large. Set the number of similar clusters n. Select the first n elements and take aj elements from these n elements to form a new sample set AA. Step 5: Repeat the operation of step 4. Classify the elements in N*A according to aj. Sort the elements in each category from small to large according to the similarity obtained in step 4. Select the first n elements and take aj elements from these n elements to form a new sample set NN. Step 6: Merge the sample sets AA and NN to generate a new sample set D, and use D as the original input sample for SVM.