A network security defense method, system and storage medium

Abstract

The present application provides a network security defense method, system and storage medium, including: obtaining garbage collection parameter settings of a log structure file system, including a recycling threshold, a log segment size, a data validity period and a metadata management rule, evaluating the garbage collection efficiency and resource utilization of the system under the current settings, and identifying potential optimization points; according to the sensitive data definition and classification rules, using a decision tree model to identify and classify sensitive data in log structure files, obtaining data sets of different sensitivity levels, and setting different migration frequency thresholds and encryption strengths for sensitive data of different sensitivity levels; establishing a comprehensive system monitoring and log analysis platform, tracking garbage collection efficiency, sensitive data processing and system performance indicators in real time, and adaptively optimizing and dynamically adjusting garbage collection parameters and sensitive data processing mechanisms according to monitoring data and preset rules.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a network security defense method, system and storage medium. Background Art

[0002] In a log-structured file system, the garbage collection mechanism reclaims invalid space by identifying and migrating valid data blocks. However, this process may leave traces of sensitive data in the original location on the physical storage medium. These data are usually closely related to network applications, including user credentials, access logs, business records, etc. Once leaked, criminals may steal information to carry out cyber attacks, identity theft and other cyber crimes. Even if the file is deleted, attackers can still directly read the storage medium through malware or physical means to steal the remaining sensitive information. Therefore, a defective garbage collection mechanism is tantamount to indirectly increasing the network security risks. In addition, considering the important role of system operation logs in post-audit and traceability, the garbage collection process itself should also meet the requirements of security reinforcement and be able to resist interference and infiltration by malicious users. The garbage collection threshold setting directly affects data migration. If the threshold is too high, the migration cost can be reduced, but the deleted data cannot be cleaned up in time, increasing the risk of information leakage; if the threshold is too low, it will affect system performance, and frequent overwriting may deepen the data traces. The order of data migration also needs to be paid attention to. Simple linear migration is easy to implement, but it may store sensitive data fragments in a centralized manner, and the consequences of destructive reading are difficult to estimate. There is an urgent need to design a new data migration method that can minimize the risk of data residue while meeting the garbage collection performance. This requires careful trade-offs in both time and space, avoiding unnecessary increases in migration overhead, considering the impact of fragmentation on system availability, and taking into account legal data recovery and forensic analysis. In other words, how to ensure data security while ensuring system performance and space utilization in log-structured file systems is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a network security defense method, which mainly includes:

[0004] Obtain the garbage collection parameter settings of the log-structured file system, including the recycling threshold, log segment size, data validity period, and metadata management rules, evaluate the system's garbage collection efficiency and resource utilization under the current settings, and identify potential optimization points; based on the evaluation results, adjust the garbage collection parameter settings, balance the recycling frequency and system performance, optimize the distribution of data between different storage levels, establish a sound monitoring and logging mechanism, track the garbage collection process and system operation status in real time, and formulate sensitive data definitions and classification rules based on data access patterns and business needs; based on the sensitive data definitions and classification rules, use a decision tree model to identify and classify sensitive data in log-structured files, obtain data sets of different sensitivity levels, and set different migration frequency thresholds and encryption strengths for sensitive data of different sensitivity levels; during the garbage collection process, introduce a data destruction verification mechanism to dynamically adjust sensitive data of different sensitivity levels The migration frequency threshold of sensitive data is set to minimize the residual time of sensitive data in log structure files and storage media, and sensitive data is processed in order of migration frequency threshold from low to high. At the same time, access permission control and data encryption methods are used to encrypt and access sensitive data in order of encryption strength from high to low, and used as a sensitive data processing mechanism; the sensitive data processing mechanism is integrated into the garbage collection process, and the sensitive data processing mechanism is gradually optimized in an incremental manner to avoid impacting the system stability, and a fallback mechanism is established to determine in real time whether anomalies occur during the optimization process. If so, the optimized sensitive data processing mechanism is restored to the previous step before the anomaly occurs; a comprehensive system monitoring and log analysis platform is established to track garbage collection efficiency, sensitive data processing status and system performance indicators in real time, and adaptively optimize and dynamically adjust garbage collection parameters and sensitive data processing mechanisms based on monitoring data and preset rules.

[0005] The present invention provides a network security defense system, which mainly includes: a garbage collection parameter evaluation module, which is used to obtain and evaluate the garbage collection parameter settings of a log structure file system and identify potential optimization points; a parameter optimization and adjustment module, which is used to adjust the garbage collection parameters according to the evaluation results, optimize data distribution, and establish a monitoring mechanism; a sensitive data identification and classification module, which is used to formulate sensitive data rules and adopt a decision tree model to identify and classify sensitive data; a sensitive data processing mechanism module, which is used to set a migration frequency threshold and encryption strength, introduce data destruction verification, and dynamically adjust the processing mechanism; a system integration and optimization module, which is used to integrate the sensitive data processing mechanism into the garbage collection process, gradually optimize and establish a fallback mechanism; a monitoring analysis and adaptive adjustment module, which is used to establish a monitoring and analysis platform, and adaptively optimize and dynamically adjust parameters and mechanisms according to data.

[0006] The present invention provides an electronic device, comprising: one or more processors, and a memory; the processor and the memory are coupled; the memory is used to store computer program code, the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes the network security defense method as described above. The present invention provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions or programs, and when the computer instructions or programs are executed on a computer, the computer executes the network security defense method as described above.

[0007] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0008] The present invention discloses a network security defense method, system and storage medium. The method solves the problem of residual sensitive data and security in a log-structured file system by optimizing a garbage collection mechanism and introducing a sensitive data processing mechanism. The present invention first evaluates and adjusts garbage collection parameters, and then automatically identifies and classifies sensitive data using a decision tree model. Differentiated migration frequency thresholds and encryption strengths are set for data of different sensitivity levels, and a data destruction verification mechanism is introduced during the garbage collection process. By dynamically adjusting the migration frequency threshold, the residual time of sensitive data is minimized, and access permission control and data encryption technology are used to protect sensitive data. The present invention also establishes a comprehensive monitoring and log analysis platform, and realizes adaptive optimization of garbage collection parameters and sensitive data processing mechanisms. This method significantly improves the data security and performance of a log-structured file system, and provides an effective solution for sensitive data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 The present invention is a flowchart of a network security defense method. DETAILED DESCRIPTION

[0010] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0011] like Figure 1 In this embodiment, a network security defense method, system, and storage medium may specifically include:

[0012] Step S101, obtain the garbage collection parameter settings of the log structure file system, including the recycling threshold, log segment size, data validity period and metadata management rules. Evaluate the garbage collection efficiency and resource utilization of the system under the current settings, and identify potential optimization points.

[0013] Obtain usage data of each log segment in the log-structured file system, and perform descriptive statistical analysis on the log segment size, data validity period, and metadata management rules based on the usage data; based on the results of the descriptive statistical analysis, use the linear regression method to calculate the garbage collection threshold of the current system, and compare the garbage collection threshold with the preset threshold; perform load testing and obtain system operation data, which includes storage space utilization, write latency, and read performance; use the multivariate linear regression method to establish a correlation model between garbage collection efficiency and various parameters, determine the key factors affecting garbage collection efficiency through the correlation model, and dynamically optimize the log segment size.

[0014] Exemplarily, the usage of each log segment in the log structure file system is obtained, descriptive statistics are performed on the log segment size, data validity period and metadata management rules, and the average value and standard deviation of the log segment usage rate are calculated. According to the statistical results, the garbage collection threshold of the current system is set by using the linear regression method, and compared with the preset threshold to obtain the initial configuration value of the garbage collection parameter. For the garbage collection parameters of the initial configuration, a load test is performed and system operation data is collected, including indicators such as storage space utilization, write latency, and read performance. A correlation model between garbage collection efficiency and various operating parameters is established using multivariate linear regression to determine the key factors affecting garbage collection efficiency. According to the regression analysis results, a parameter adjustment strategy is formulated to dynamically optimize the log segment size. A dynamic threshold is set based on data access frequency and time interval, and the data validity period determination rule is adjusted. At the same time, the metadata management strategy is optimized to reduce the degree of fragmentation and improve the storage space utilization and system throughput. Key performance indicators and alarm thresholds for monitoring are set, including garbage collection frequency, compression ratio, etc. These indicators are continuously monitored during system operation, and the parameter automatic tuning process is triggered when the monitoring indicators exceed the preset range. The gradient descent method is used to continuously optimize the garbage collection parameters to achieve efficient use of system resources while ensuring data consistency. The usage of each log segment in the log structure file system is obtained, and the average log segment size is calculated using statistical tools. The average value is 256MB, and the standard deviation is 32MB; the average value of the data validity period is 7 days, and the standard deviation is 2 days; the average index node density in the metadata management rule is 100 per 1MB of data. The linear regression model is used to calculate the current system's garbage collection threshold of 75%. After comparing it with the preset threshold of 80%, the initial garbage collection parameter is configured to 70%. Perform a load test and collect system operation data: the storage space utilization rate is 65%, the average write latency is 5ms, and the read performance is 100MB / s. Use multiple linear regression to establish an association model to determine that the garbage collection threshold, log segment size, and data validity period are key influencing factors. According to the regression results, the parameter adjustment strategy is formulated, including dynamically adjusting the log segment size to 192MB, and the data validity period determination rule sets a dynamic threshold based on the access frequency. The data validity period with a frequency lower than 0.1 times / day is shortened to 3 days. Optimize metadata management strategy, increase index node density to 150 per 1MB of data, and reduce fragmentation. Set key performance indicators to garbage collection frequency no more than once per hour, and maintain compression ratio between 1.5:1 and 2:1. When monitoring indicators are out of range, such as garbage collection frequency reaching 1.2 times per hour, trigger the automatic parameter tuning process. Use gradient descent method to iteratively optimize the garbage collection threshold with a learning rate of 0.01, and finally adjust the threshold to 72%, so that the system resource utilization rate is increased to 75%, while maintaining data consistency.

[0015] Step S102, according to the evaluation results, adjust the garbage collection parameter settings, balance the recycling frequency and system performance, optimize the distribution of data among different storage levels, establish a sound monitoring and logging mechanism, track the garbage collection process and system operation status in real time, and formulate sensitive data definitions and classification rules based on data access patterns and business needs.

[0016] Get the current garbage collection parameter settings, including the collection threshold, log segment size, and data validity period; extract CPU usage, memory usage, and storage I / O latency from the system performance monitoring tool to obtain system performance indicators. Based on the garbage collection parameter settings and system performance indicators, a multivariate linear regression method is used to establish a correlation model between garbage collection frequency and system performance. The garbage collection parameter configuration that maximizes system throughput is determined through the correlation model. According to the garbage collection parameter configuration, the data is stored in layers, specifically: if the data access frequency is higher than the preset threshold, the data is migrated to the solid-state drive; if the data access frequency is lower than the preset threshold, the data is migrated to the mechanical hard drive. Deploy a distributed log collector and obtain key event information and system operation status information during the garbage collection process from the distributed log collector. Based on the extracted data access pattern information, a clustering algorithm is used to analyze the data access pattern to obtain the data access pattern clustering results. Based on the data access pattern clustering results, a sensitive data definition rule base and classification strategy are established.

[0017] Exemplarily, the current garbage collection parameter settings are obtained, including the recycling threshold, log segment size, and data validity period. Indicators such as CPU usage, memory usage, and storage I / O latency are extracted from the system performance monitoring tool. A correlation model between garbage collection frequency and system performance is established through multivariate linear regression. The optimal garbage collection parameter configuration is obtained with the maximization of system throughput as the optimization goal. According to the optimized parameter configuration, combined with the read and write speed and capacity characteristics of the storage layer, a tiered strategy based on access frequency and data size is used to classify the data. Hot data with an access frequency higher than 100 times per hour is migrated to the solid-state drive, and cold data with an access frequency lower than once a day is migrated to the mechanical hard drive. The overall storage performance and resource utilization are optimized by dynamically adjusting the distribution ratio of data between different storage layers. A distributed log collector is deployed to collect key events and system operation status information during the garbage collection process, including the collection start time, duration, amount of recycled data, and resource consumption. The collected log data is transmitted in real time to the central log analysis platform, which has data storage, query, and visualization functions, and cleans, parses, and extracts indicators from the log data to generate a garbage collection performance report. The data access pattern information extracted from the log analysis platform is combined with the business sensitivity scoring standard to automatically classify and mark the data. The business sensitivity score is calculated based on three dimensions: data importance, access frequency, and security level. The time series clustering algorithm is used to analyze the data access pattern, classify similar access patterns, establish a sensitive data definition rule base and classification strategy, and achieve refined data management and differentiated protection. The data of the last 24 hours is extracted from the system performance monitoring tool, including garbage collection parameter settings, including a recycling threshold of 70%, a log segment size of 256MB, a data validity period of 7 days, and system performance indicators, including an average CPU usage of 60%, a memory usage of 85%, and a storage I / O delay of 5ms. The association model is established using multivariate linear regression, with system throughput as the dependent variable and garbage collection frequency and system performance indicators as independent variables. The optimal configuration is to adjust the recycling threshold to 75%, the log segment size to 192MB, and the data validity period remains unchanged. According to the new configuration, a tiering strategy based on access frequency and data size is adopted to divide data into three categories: hot data, which is accessed more than 100 times per hour, warm data, which is accessed 1-100 times per day, and cold data, which is accessed less than once a day. Hot data, accounting for 20% of the total, is migrated to the solid-state drive, cold data, accounting for 50% of the total, is migrated to the mechanical hard disk, and warm data, accounting for 30% of the total, is retained in the original storage layer. Deploy a distributed log collector to collect garbage collection events and system status every 5 minutes, including the start time of recycling, the average duration of 2 minutes, the average amount of recycled data of 500MB, and the resource consumption CPU peak utilization rate increased by 10%. The central log analysis platform uses a time series database to store logs, supports SQL queries and chart visualization, and generates a garbage collection performance report every hour.Use time series clustering algorithms, such as DTW distance + hierarchical clustering method, to analyze data access patterns and group patterns with a similarity of more than 85% into one category. Combined with the business sensitivity score, the data importance weight is 0.5, the access frequency weight is 0.3, and the security level weight is 0.2, and the data is automatically classified and labeled. Establish a sensitive data definition rule library containing 10 rules, such as "data containing personal identity information and accessed more than 50 times per hour is judged to be highly sensitive", to achieve refined data management.

[0018] Step S103, based on the sensitive data definition and classification rules, a decision tree model is used to identify and classify the sensitive data in the log structure file to obtain data sets of different sensitivity levels. Different migration frequency thresholds and encryption strengths are set for sensitive data of different sensitivity levels.

[0019] Obtain a pre-established log structure file and extract data features from the log structure file. The data features include data content, access frequency, data size, and metadata information. Perform feature conversion according to the type of data features. Standardize numerical features, perform unique hot encoding on categorical features, and convert text data into numerical vectors using the TF-IDF algorithm. After the data feature conversion is completed, use the decision tree algorithm to train the training data set. The decision tree algorithm uses the Gini coefficient as the node splitting criterion. Use the trained decision tree model to predict the test data set to obtain the classification accuracy, precision, and recall rate, draw the ROC curve, and calculate the AUC value. If the AUC value is lower than the preset threshold, increase the number of training samples or adjust the feature selection. Select the optimal classification threshold according to the ROC curve, and map the probability value output by the decision tree to three sensitivity levels: high, medium, and low. The probability corresponding to the high sensitivity level is greater than the first threshold, the probability corresponding to the medium sensitivity level is between the second threshold and the first threshold, and the probability corresponding to the low sensitivity level is less than the second threshold.

[0020] Exemplarily, data features are extracted from log structure files, including data content, access frequency, data size and metadata information, and the features are converted into numerical and categorical types according to predefined sensitive data rules. Numerical features are standardized, categorical features are encoded with one-hot encoding, and text data are converted into numerical vectors using TF-IDF. A training data set and a test data set containing features and sensitivity level labels are constructed. The training data set is trained using a decision tree algorithm, the Gini coefficient is used as the node splitting criterion, the maximum tree depth is set to 10, and the minimum number of sample splits is set to 50. The model parameters are optimized through 5-fold cross validation, with 80% of the data used for training and 20% of the data used for validation in each fold, and the parameter combination with the best performance of the validation set is selected. The test data set is predicted using the trained decision tree model, the confusion matrix is ​​calculated, the classification accuracy, precision and recall are obtained, the ROC curve is drawn and the AUC value is calculated, including traversing the features of each sample in the test data set according to the trained decision tree model, predicting the category of the sample through the branch structure of the decision tree, and obtaining the prediction result. The prediction results and true labels are counted to construct a confusion matrix. The diagonal elements of the confusion matrix represent the number of samples predicted correctly, and the off-diagonal elements represent the number of samples predicted incorrectly. The confusion matrix is ​​used to calculate evaluation indicators such as classification accuracy, precision, and recall. Accuracy is the ratio of the number of samples predicted to be positive and actually positive to the number of samples predicted to be positive, and recall is the ratio of the number of samples predicted to be positive and actually positive to the number of samples predicted to be positive. According to the prediction results and true labels, the false positive rate and true positive rate under different thresholds are calculated, and the ROC curve is drawn. The horizontal axis of the ROC curve is the false positive rate, and the vertical axis is the true positive rate. The larger the area under the curve AUC value, the better the model performance. If the AUC value is greater than the preset threshold, it is judged that the decision tree model has good performance and can be used for actual business prediction tasks; if the AUC value is less than the preset threshold, the decision tree model needs to be optimized, such as adjusting the tree depth, the minimum number of samples of leaf nodes and other parameters, retraining the model and evaluating the performance. The ROC curve is a method that graphically represents the relationship between the true positive rate (i.e., recall rate) and the false positive rate (i.e., FP / (FP+TN)) at different thresholds. TN (True Negative) is the number of samples that are actually negative and correctly predicted as negative, and FP (False Positive) is the number of samples that are actually negative but incorrectly predicted as positive. The AUC value is the area under the ROC curve, and its value range is from 0 to 1. The larger the AUC value, the better the performance of the classifier. If the AUC value is lower than the preset threshold of 0.9, increase the number of training samples or adjust the feature selection, and retrain the model until the accuracy requirements are met. Select the optimal classification threshold based on the ROC curve, and map the probability value output by the decision tree to three sensitivity levels: high, medium, and low.The probability corresponding to the high sensitivity level is greater than 0.8, the probability corresponding to the medium sensitivity level is between 0.5 and 0.8, and the probability corresponding to the low sensitivity level is less than 0.5. For high sensitivity level data, the migration frequency threshold is set to once an hour, using the AES-256 encryption algorithm; the migration frequency of medium sensitivity level data is once a day, using the AES-128 encryption algorithm; the migration frequency of low sensitivity level data is once a week, using the basic symmetric encryption algorithm. Extract data features from log structure files, including data content, such as ID card number, bank account number, access frequency is the average number of accesses per hour, data size (KB) and metadata information, including creation time and modification time. According to predefined rules, the existence of sensitive information such as ID card number and bank account number is converted into Boolean features 0 or 1. Standardize numerical features such as access frequency and data size, and map the values ​​to the range of 0-1. Text data is converted into numerical vectors using the TF-IDF method, and the top 1000 words are taken as features. Construct 10,000 training samples and 2,000 test samples containing the above features and sensitivity level labels high, medium, and low. The decision tree algorithm was used for training, and the Gini coefficient was used as the node splitting criterion. The maximum tree depth was set to 10, and the minimum number of sample splits was set to 50. The model parameters were optimized through 5-fold cross validation, with 8,000 data points used for training and 2,000 data points used for validation in each fold. The parameter combination with the highest accuracy of the validation set was selected. The test data set was predicted and the confusion matrix was calculated. The classification accuracy was 0.92, the precision was 0.89, and the recall was 0.93. The ROC curve was drawn and the AUC value was calculated to be 0.95. The best classification threshold of 0.7 was selected according to the ROC curve, and the probability value output by the decision tree was mapped to the sensitivity level: the probability greater than 0.8 was highly sensitive, between 0.5 and 0.8 was medium sensitive, and less than 0.5 was low sensitive. The high-sensitivity data accounting for 15% was migrated once an hour and encrypted using AES-256; the medium-sensitivity data accounting for 35% was migrated once a day and encrypted using AES-128; the low-sensitivity data accounting for 50% was migrated once a week and encrypted using a basic symmetric encryption algorithm.

[0021] Step S104, during the garbage collection process, a data destruction verification mechanism is introduced to dynamically adjust the migration frequency thresholds of sensitive data of different sensitivity levels, minimize the residual time of sensitive data in log structure files and storage media, and process sensitive data in order of migration frequency thresholds from low to high. At the same time, access permission control and data encryption methods are used to encrypt and access sensitive data in order of encryption strength from high to low, and serve as a sensitive data processing mechanism.

[0022] Obtain the sensitivity level information of the data block to be recovered, and determine the number of overwrites based on the sensitivity level information; overwrite the data block to be recovered multiple times, and the number of overwrites is dynamically adjusted by the sensitivity level information. Use the sliding window method to calculate the data residual time, and the sliding window size is set to the preset time unit; according to the data residual time, the average life cycle of data of different sensitivity levels is calculated by the exponentially weighted moving average algorithm. Determine the migration frequency threshold based on the average life cycle, and the migration frequency threshold is inversely proportional to the sensitivity level information; establish a priority queue, and the priority queue is sorted from high to low according to the migration frequency threshold. Obtain data access requests, which contain user role information; determine access rights based on user role information, and if the access rights meet the preset conditions, obtain the corresponding key from the hierarchical key system. Receive access logs of data access requests and encrypt and store the access logs.

[0023] Exemplarily, a secure erase algorithm is used to overwrite the data blocks to be recovered in the log structure file multiple times, and each overwrite uses a different random data pattern. The number of overwrites is dynamically adjusted according to the sensitivity level. High-sensitivity data is overwritten 7 times, medium-sensitivity data is overwritten 3 times, and low-sensitivity data is overwritten once. The SHA-256 hash check is used to ensure that the data is completely cleared. For solid-state drives, the TRIM command is used to clear data; for mechanical hard drives, the secure erase instruction is used for physical erasure. The data residual time is calculated using the sliding window method, the window size is set to 24 hours, and the residual time statistics are updated every hour. According to the sensitive data residual minimization method, the exponentially weighted moving average algorithm is used to calculate the average life cycle of data of different sensitivity levels, and the weight factor is set to 0.2. The migration frequency threshold is dynamically adjusted, and the migration frequency threshold of high-sensitivity data is set to 1 / 4 of the average life cycle, medium-sensitivity data is set to 1 / 2, and low-sensitivity data is set to 3 / 4. The data access pattern is monitored in real time using a time series-based anomaly detection algorithm, and data migration is triggered when the access frequency exceeds the threshold. Sensitive data is processed in order of migration frequency threshold from low to high, and a priority queue is established. High-sensitivity data is placed at the head of the queue, and low-sensitivity data is placed at the tail of the queue. Data is taken from the head of the queue for migration and encryption each time it is processed, and the data status is updated after the processing is completed. When recalculating the priority, the weighted sum of the data sensitivity weight of 0.5, the access frequency weight of 0.3, and the last processing time weight of 0.2 is considered. A role-based access control mechanism is used to protect sensitive data in combination with multi-layer encryption technology. Key management adopts a hierarchical key system, the root key is protected by a hardware security module, and the derived key is generated by a key derivation function. High-sensitivity data is encrypted using the AES-256 algorithm and assigned the highest permission level, medium-sensitivity data is encrypted using the AES-128 algorithm and assigned a medium permission level, and low-sensitivity data uses a basic symmetric encryption algorithm and is assigned a normal permission level. Permission verification and decryption operations are performed when accessing data, and access logs are encrypted, stored, and integrity protected. In the log-structured file system, a 100MB block of highly sensitive data is securely erased by overwriting it with a random data pattern seven times. After each overwrite, the SHA-256 hash value is calculated and compared with the original data hash to ensure that the data is completely cleared. For the 2GB of sensitive data in the 256GB solid-state drive, the TRIM command is sent to clear it; for the 5GB of sensitive data in the 500GB mechanical hard drive, the ATA secure erase command is used for physical erasure. Using a 24-hour sliding window, the data residual time statistics are updated once an hour, and the average residual time of highly sensitive data is calculated to be 4 hours. Using the exponentially weighted moving average algorithm with a weight factor of 0.2, the average life cycle of highly sensitive data is calculated to be 48 hours, and the migration frequency threshold is set to 12 hours accordingly.Use a time series-based anomaly detection algorithm, set the detection window to 1 hour, and trigger migration when the data access frequency exceeds 10 times per minute. Establish a priority queue containing 1,000 data records, including 300 highly sensitive data, 400 medium sensitive data, and 300 low sensitive data. Each time processing, take out 50 data from the head of the queue for migration and encryption, and update the status after processing. When recalculating the priority, for a data with a sensitivity of 0.9, an access frequency of 5 times per hour, and the last processing time 24 hours ago, its priority is calculated as 0.9×0.5+5 / 10×0.3+24 / 48×0.2=0.7. A three-layer key system is adopted, the root key is protected by a hardware security module certified by FIPS140-2Level3, and the derived key is generated using the PBKDF2 algorithm. High-sensitivity data is encrypted using the AES-256 algorithm with a 256-bit key length; medium-sensitivity data is encrypted using the AES-128 algorithm with a 128-bit key length; low-sensitivity data is encrypted using the 3DES algorithm with a 168-bit key length. When accessing data, permission verification is performed through the RBAC model, and the access log is protected for integrity using the HMAC-SHA256 algorithm.

[0024] Step S105, integrate the sensitive data processing mechanism into the garbage collection process, and gradually optimize the sensitive data processing mechanism in an incremental manner to avoid impacting the system stability, and establish a fallback mechanism to determine in real time whether an abnormality occurs during the optimization process. If so, restore the optimized sensitive data processing mechanism to the previous step where the abnormality occurs.

[0025] Receive a plug-in request message carrying a sensitive data processing plug-in identifier, the plug-in request message is sent by the garbage collection system; send a loading instruction to the sensitive data processing plug-in according to the plug-in request message, the sensitive data processing plug-in sends a configuration signal of processing rules and optimization parameters according to the loading instruction; receive a gray test report with processing results and performance indicators, the gray test report is generated by the garbage collection system after the test is executed; determine whether the performance indicator exceeds the preset threshold, including: querying the system indicator database; wherein the system indicator database includes historical performance data with moving average and standard deviation; determine whether there are abnormal values ​​related to the performance indicator in the system indicator database; if there are abnormal values ​​related to the performance indicator in the system indicator database, trigger an alarm of the corresponding level; if the performance indicator does not trigger an alarm, allow the sensitive data processing plug-in to pass the optimization verification. Obtain parameter changes and performance indicator information in the optimization history database; perform time series analysis on parameter changes and performance indicator information; predict the impact of the next step of optimization through a random forest regression model; and determine whether to adjust the optimization strategy according to the impact.

[0026] Exemplarily, a plug-in architecture is used to integrate the sensitive data processing mechanism into the garbage collection process, and a unified plug-in interface is designed, including methods such as initialization, data processing, and resource release. Processing rules and optimization parameters are defined through configuration files, and sensitive data processing plug-ins are dynamically loaded during the garbage collection process using the reflection mechanism. The corresponding processing method is selected according to the data sensitivity level, and the processing log and performance indicators are recorded at the same time. An incremental optimization algorithm is designed to divide the optimization process of the sensitive data processing mechanism into multiple small steps, each of which modifies a parameter or adds a new function. A parameter priority queue is established to select parameters to be optimized based on historical performance impact. A gray release strategy is used for A / B testing, 1% of the traffic is imported into the new version, and the system performance before and after optimization is compared, including indicators such as garbage collection time, CPU usage, memory usage, and throughput. If the performance improvement exceeds the preset threshold, the gray scale ratio is gradually expanded, otherwise it is rolled back. A real-time monitoring module is implemented, and a sliding window algorithm is used to calculate the moving average and standard deviation of the key indicators of the system, and three alarm levels are set: warning, serious, and emergency. The warning level is when the indicator exceeds 1.5 times the standard deviation, the serious level is 2 times the standard deviation, and the emergency level is 3 times the standard deviation. When an indicator is abnormal, an alarm of the corresponding level is triggered and the optimization process is suspended. At the same time, the current system status and configuration information are recorded as a recovery point. If the abnormality lasts longer than the preset value, it will automatically roll back to the previous stable version, generate a recovery patch through the binary difference algorithm, and quickly apply it to the production environment. Build an optimization history database to store parameter changes, performance indicators, and system logs for each optimization. Use the ARIMA model for time series analysis to identify optimization trends and potential risks. Train a random forest regression model based on historical data to predict the impact of the next optimization. When the prediction result shows a high risk, automatically adjust the optimization strategy or suspend the optimization process. Real-time monitoring data is regularly written to the historical database to update the ARIMA model parameters and retrain the random forest model to ensure prediction accuracy. In a 10TB distributed storage system, a plug-in architecture is designed to integrate the sensitive data processing mechanism into the garbage collection process. Define a unified interface including initialize(), processData(), and release() methods, and set processing rules and optimization parameters through XML configuration files. When garbage collection is triggered, use the Java reflection mechanism to dynamically load the plug-in to process 100GB of sensitive data. The incremental optimization algorithm divides the process into 10 steps, and adjusts one parameter in each step, such as increasing the encryption strength from AES-128 to AES-256. The parameter priority queue is sorted according to the historical performance impact, and the parameters with the greatest impact are selected for priority adjustment. A grayscale release strategy is adopted, and 1%, or 100GB of data, is initially imported into the new version. The garbage collection time is compared, and it is reduced from an average of 30 minutes to 28 minutes, and the CPU usage rate is reduced from 70% to 65%. After the performance improvement exceeds the 5% threshold, the grayscale ratio doubles every hour until it reaches 100%.The real-time monitoring module uses a 1-hour sliding window to calculate the moving average and standard deviation of system indicators every minute. Set the three-level alarms as warning (CPU usage exceeds 75%), severe (memory usage exceeds 90%), and emergency (disk I / O latency exceeds 100ms). When the alarm is triggered, save the current configuration as a recovery point. If the anomaly lasts for more than 10 minutes, roll back. Use the rsync algorithm to generate binary differential patches for fast rollback. The optimization history database records each parameter adjustment and the corresponding performance changes, and stores the latest 1,000 optimization records. Use the ARIMA (2,1,2) model to analyze time series data and predict the 2.3% performance improvement that may be brought by the next optimization. Train a random forest regression model containing 100 decision trees, with input features including the current system load, optimization parameter values, etc., to predict the risk probability of the next optimization. If the risk exceeds 30%, suspend the optimization. Write the monitoring data to the history database every hour, update the ARIMA model parameters, and retrain the random forest model to ensure that the prediction accuracy remains above 90%.

[0027] Step S106, establish a comprehensive system monitoring and log analysis platform to track garbage collection efficiency, sensitive data processing and system performance indicators in real time, and adaptively optimize and dynamically adjust garbage collection parameters and sensitive data processing mechanisms based on monitoring data and preset rules.

[0028] A data collection request carrying a system node identifier is obtained, and the data collection request is issued by the system node during operation; a sampling instruction is sent to the system node according to the data collection request, and the system node collects original data related to garbage collection, sensitive data processing and performance according to the sampling instruction; a collection confirmation message with a node identifier and original data is received, and the collection confirmation message is generated by the system node after completing data collection; whether the original data meets the adaptive sampling condition, including: querying a system load database; wherein the system load database includes historical data with a load threshold; judging whether there are data records exceeding the load threshold in the system load database; if there are data records exceeding the load threshold in the system load database, reducing the frequency of the sampling instruction; cleaning, aggregating and normalizing the original data to obtain key performance indicators; performing trend analysis on the key performance indicators to obtain trend data; performing anomaly detection on the trend data through a random forest algorithm to obtain anomaly indicators; determining an adjustment plan for system configuration parameters according to the anomaly indicators and preset optimization rules; optimizing the adjustment plan using a PID control algorithm to obtain optimized system configuration parameters.

[0029] For example, a distributed data collection agent is deployed to collect raw data related to garbage collection, sensitive data processing and performance at each node of the system, including CPU usage, memory usage, disk I / O, network traffic, garbage collection time and frequency, sensitive data processing volume and time consumption, etc. Adaptive sampling technology is used to dynamically adjust the collection frequency according to the system load, and reduce the collection frequency when the load is high to ensure that the impact of data collection on system performance is minimized. The collected data is transmitted to the central data processing unit in real time through the message queue. The raw data is cleaned, aggregated and normalized in the central data processing unit to calculate a series of key performance indicators, such as garbage collection efficiency, sensitive data processing rate, system throughput, etc. The processed indicator data is stored in the InfluxDB time series database, and a multidimensional index is established to improve query efficiency. Use Grafana to build a visual dashboard to display real-time and historical performance indicators, and set alarm thresholds. Based on the historical data in InfluxDB, the ARIMA model is used to perform trend analysis on various indicators, and the IsolationForest random forest algorithm is used for anomaly detection to calculate the change rate and fluctuation range of the indicators. Combined with the preset optimization rules, including CPU usage not exceeding 80%, memory usage not exceeding 90%, and garbage collection time not exceeding 10% of the total running time, the adjustment suggestions for garbage collection parameters and sensitive data processing mechanism are generated. A mapping relationship table between adjustment suggestions and configuration parameters is established to convert the suggestions into specific configuration items. An adaptive optimization controller is designed, and the PID control algorithm is used to automatically modify the system configuration parameters according to the adjustment suggestions, including garbage collection trigger threshold, number of recycling threads, sensitive data processing batch size, etc. The PID parameters are continuously adjusted through feedback control until the system performance reaches the optimal state. At the same time, the results of each adjustment are recorded to optimize the parameters of the control algorithm. In a distributed storage system, 100 data collection agent nodes are deployed, and each node initially collects data at an interval of 5 seconds. Adaptive sampling algorithm is used. When the CPU usage exceeds 85%, the collection interval is increased to 10 seconds; when it is less than 50%, it is restored to a 5-second interval. About 10MB of data is transmitted to the central processing unit per second through the Kafka message queue. The data processing unit uses Apache Spark for real-time stream processing to calculate the average and standard deviation within a 60-second sliding window. The processed indicator data is written to InfluxDB, and a 1-hour data retention policy is set, with automatic compression every 5 minutes. The Grafana dashboard displays 6 key indicators: CPU usage, memory usage, disk I / O, network traffic, garbage collection time ratio, and sensitive data processing rate. Multi-level alarm thresholds are set, such as a severe alarm when the CPU usage exceeds 90%. The ARIMA (2,1,2) model is used to predict the performance trend in the next hour, and the IsolationForest algorithm detects anomalies with a contamination rate of 0.1.The preset rules include a CPU usage target of 75%, a memory usage target of 85%, and a garbage collection time ratio of no more than 8%. A mapping relationship table is established. If the CPU usage exceeds the target by 5%, the garbage collection trigger threshold is increased by 10%. The initial parameters of the PID controller are set to P=0.5, I=0.2, and D=0.1. The PID parameters are adjusted every 10 minutes based on the system response. The optimization history is recorded. When three consecutive adjustments fail to improve performance, roll back to the last stable configuration. The system can automatically adjust when the load changes to maintain the best performance.

[0030] The present invention provides a network security defense system, which mainly includes: a garbage collection parameter evaluation module, which is used to obtain and evaluate the garbage collection parameter settings of a log structure file system and identify potential optimization points; a parameter optimization and adjustment module, which is used to adjust the garbage collection parameters according to the evaluation results, optimize data distribution, and establish a monitoring mechanism; a sensitive data identification and classification module, which is used to formulate sensitive data rules and adopt a decision tree model to identify and classify sensitive data; a sensitive data processing mechanism module, which is used to set a migration frequency threshold and encryption strength, introduce data destruction verification, and dynamically adjust the processing mechanism; a system integration and optimization module, which is used to integrate the sensitive data processing mechanism into the garbage collection process, gradually optimize and establish a fallback mechanism; a monitoring analysis and adaptive adjustment module, which is used to establish a monitoring and analysis platform, and adaptively optimize and dynamically adjust parameters and mechanisms according to data.

[0031] The present invention provides an electronic device, comprising: one or more processors, and a memory; the processor and the memory are coupled; the memory is used to store computer program code, the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes the network security defense method as described above. The present invention provides a computer-readable storage medium, the computer-readable storage medium stores computer instructions or programs, and when the computer instructions or programs are executed on a computer, the computer executes the network security defense method as described above.

[0032] The above only lists some preferred embodiments of the present invention, but the present invention is not limited thereto, and many improvements and changes can be made. As long as the improvements and changes are made on the basis of the basic principles of the present invention, they should be regarded as falling within the protection scope of the present invention.

Claims

1. A network security defense method, characterized in that: The method comprises: Obtain the garbage collection parameter settings of the log-structured file system, including the recycling threshold, log segment size, data validity period, and metadata management rules, evaluate the system's garbage collection efficiency and resource utilization under the current settings, and identify potential optimization points; based on the evaluation results, adjust the garbage collection parameter settings, balance the recycling frequency and system performance, optimize the distribution of data between different storage levels, establish a sound monitoring and logging mechanism, track the garbage collection process and system operation status in real time, and formulate sensitive data definitions and classification rules based on data access patterns and business needs; based on the sensitive data definitions and classification rules, use a decision tree model to identify and classify sensitive data in log-structured files to obtain data sets of different sensitivity levels, set different migration frequency thresholds and encryption strengths for sensitive data of different sensitivity levels, and the migration frequency threshold is inversely proportional to the sensitivity level information; during the garbage collection process, introduce a data destruction verification mechanism to dynamically adjust the garbage collection process. The migration frequency threshold of sensitive data with the same sensitivity level minimizes the residual time of sensitive data in log structure files and storage media, processes sensitive data in order of migration frequency threshold from low to high, and uses access control and data encryption methods to encrypt and access sensitive data in order of encryption strength from high to low, and use it as a sensitive data processing mechanism; integrate the sensitive data processing mechanism into the garbage collection process, and gradually optimize the sensitive data processing mechanism in an incremental manner to avoid impacting system stability, and establish a fallback mechanism to determine in real time whether anomalies occur during the optimization process. If so, restore the optimized sensitive data processing mechanism to the previous step where the anomaly occurred; establish a comprehensive system monitoring and log analysis platform to track garbage collection efficiency, sensitive data processing status and system performance indicators in real time, and adaptively optimize and dynamically adjust garbage collection parameters and sensitive data processing mechanisms based on monitoring data and preset rules.

2. The method according to claim 1, characterized in that The method of obtaining the garbage collection parameter settings of the log structure file system, including the recycling threshold, log segment size, data validity period and metadata management rules, evaluates the garbage collection efficiency and resource utilization of the system under the current settings, and identifies potential optimization points, including: Obtain usage data of each log segment in the log structure file system, and perform descriptive statistical analysis on the log segment size, data validity period, and metadata management rules based on the usage data; According to the descriptive statistical analysis results, the garbage collection threshold of the current system is calculated using the linear regression method, and the garbage collection threshold is compared with the preset threshold; Perform load testing and obtain system operation data, including storage space utilization, write latency, and read performance; The multivariate linear regression method is used to establish the correlation model between garbage collection efficiency and various parameters. The key factors affecting garbage collection efficiency are determined through the correlation model, and the log segment size is dynamically optimized.

3. The method according to claim 1, characterized in that According to the evaluation results, adjust the garbage collection parameter settings, balance the collection frequency and system performance, optimize the distribution of data among different storage tiers, establish a sound monitoring and logging mechanism, track the garbage collection process and system operation status in real time, and formulate sensitive data definitions and classification rules based on data access patterns and business needs, including: Get the current garbage collection parameter settings, including the collection threshold, log segment size, and data validity period; Extract CPU usage, memory usage, and storage I / O latency from system performance monitoring tools to obtain system performance indicators; According to the garbage collection parameter settings and system performance indicators, a multivariate linear regression method is used to establish the correlation model between garbage collection frequency and system performance; Determine the garbage collection parameter configuration that maximizes system throughput through the association model; According to the garbage collection parameter configuration, the data is stored in layers, specifically: if the data access frequency is higher than the preset threshold, the data is migrated to the solid-state drive; If the data access frequency is lower than the preset threshold, the data will be migrated to the mechanical hard disk; Deploy a distributed log collector and obtain key event information and system operation status information during the garbage collection process from the distributed log collector; According to the extracted data access pattern information, a clustering algorithm is used to analyze the data access pattern to obtain the data access pattern clustering result; Establish a sensitive data definition rule base and classification strategy based on the data access pattern clustering results.

4. The method according to claim 1, characterized in that According to the definition and classification rules of sensitive data, a decision tree model is used to identify and classify sensitive data in log structure files to obtain data sets of different sensitivity levels. Different migration frequency thresholds and encryption strengths are set for sensitive data of different sensitivity levels, including: Obtain a pre-established log structure file, and extract data features from the log structure file, the data features including data content, access frequency, data size, and metadata information; Perform feature conversion based on the type of data features, standardize numerical features, perform one-hot encoding on categorical features, and convert text data into numerical vectors using the TF-IDF algorithm; After the data feature conversion is completed, the training data set is trained using the decision tree algorithm, which uses the Gini coefficient as the node splitting criterion; The trained decision tree model is used to predict the test data set, obtain the classification accuracy, precision and recall, draw the ROC curve and calculate the AUC value; If the AUC value is lower than the preset threshold, increase the number of training samples or adjust the feature selection; The optimal classification threshold is selected according to the ROC curve, and the probability value output by the decision tree is mapped to three sensitivity levels: high, medium and low. The probability corresponding to the high sensitivity level is greater than the first threshold, the probability corresponding to the medium sensitivity level is between the second threshold and the first threshold, and the probability corresponding to the low sensitivity level is less than the second threshold.

5. The method according to claim 1, characterized in that In the garbage collection process, a data destruction verification mechanism is introduced to dynamically adjust the migration frequency thresholds of sensitive data of different sensitivity levels, minimize the residual time of sensitive data in log structure files and storage media, process sensitive data in order of migration frequency thresholds from low to high, and use access control and data encryption methods to encrypt and access sensitive data in order of encryption strength from high to low, as a sensitive data processing mechanism, including: Obtain the sensitivity level information of the data block to be recovered, and determine the number of overwrite times according to the sensitivity level information; Overwrite the data blocks to be recycled multiple times, and the number of overwrites is dynamically adjusted based on the sensitivity level information; The data residual time is calculated using the sliding window method, and the sliding window size is set to a preset time unit; According to the data retention time, the average life cycle of data of different sensitivity levels is calculated using the exponentially weighted moving average algorithm; Determine the migration frequency threshold based on the average life cycle, and the migration frequency threshold is inversely proportional to the sensitivity level of information; Establish a priority queue, and sort the priority queue from low to high according to the migration frequency threshold; Get a data access request, which contains user role information; Determine access rights based on user role information. If the access rights meet the preset conditions, obtain the corresponding key from the hierarchical key system. Receive access logs of data access requests and encrypt and store the access logs.

6. The method according to claim 1, characterized in that The sensitive data processing mechanism is integrated into the garbage collection process, and the sensitive data processing mechanism is gradually optimized in an incremental manner to avoid impact on system stability, and a fallback mechanism is established to determine in real time whether an abnormality occurs during the optimization process. If so, the optimized sensitive data processing mechanism is restored to the previous step where the abnormality occurs, including: receiving a plug-in request message carrying a sensitive data processing plug-in identifier, the plug-in request message being sent by a garbage collection system; Send a loading instruction to the sensitive data processing plug-in according to the plug-in request information, and the sensitive data processing plug-in sends a configuration signal of processing rules and optimization parameters according to the loading instruction; Receive a grayscale test report with processing results and performance indicators. The grayscale test report is generated by the garbage collection system after executing the test; Determining whether the performance indicator exceeds a preset threshold includes: querying a system indicator database; Among them, the system indicator database includes historical performance data with moving averages and standard deviations; Determine whether there are abnormal values ​​related to performance indicators in the system indicator database; If there are abnormal values ​​related to performance indicators in the system indicator database, an alarm of the corresponding level will be triggered; If the performance indicator does not trigger an alarm, the sensitive data processing plug-in is allowed to pass the optimization verification; Obtain parameter changes and performance indicator information in the optimization history database; Conduct time series analysis of parameter changes and performance indicator information; The impact of the next step of optimization is predicted through the random forest regression model; Determine whether to adjust the optimization strategy based on the degree of impact.

7. The method according to claim 1, characterized in that The above-mentioned comprehensive system monitoring and log analysis platform is established to track garbage collection efficiency, sensitive data processing and system performance indicators in real time, and adaptively optimize and dynamically adjust garbage collection parameters and sensitive data processing mechanisms based on monitoring data and preset rules, including: Obtaining a data collection request carrying a system node identifier, where the data collection request is issued by the system node during operation; Send sampling instructions to system nodes according to data collection requests, and system nodes collect raw data related to garbage collection and sensitive data processing and performance according to the sampling instructions; Receive the collection confirmation information with the node identification and the original data, which is generated by the system node after completing the data collection; Determining whether the original data meets the adaptive sampling condition includes: querying the system load database; Wherein, the system load database includes historical data with load thresholds; Determine whether there are data records exceeding the load threshold in the system load database; If there are data records in the system load database that exceed the load threshold, the frequency of sampling instructions is reduced; Clean, aggregate and normalize the raw data to obtain key performance indicators; Conduct trend analysis on key performance indicators to obtain trend data; Use the random forest algorithm to detect anomalies in trend data and obtain anomaly indicators; Determine the adjustment plan for system configuration parameters based on abnormal indicators and preset optimization rules; The PID control algorithm is used to optimize the adjustment plan and obtain the optimized system configuration parameters.

8. An electronic device, characterized in that: include: One or more processors, and a memory; the processor and the memory are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes the network security defense method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions or programs. When the computer instructions or programs are executed on a computer, the computer executes the network security defense method according to any one of claims 1 to 7.

Images