Establishment method of data leakage prevention system

Abstract

The invention provides a data leakage prevention system building method, which comprises the steps of building a data leakage risk prediction model according to principal component characteristics, training the model by adopting a support vector machine algorithm to obtain a data leakage risk prediction model, inputting to-be-predicted multi-source heterogeneous data into the risk prediction model, and obtaining a data leakage risk prediction result. Calculating a leakage risk score of the data combination; a sensitivity threshold value is preset, whether the leakage risk score exceeds the preset sensitivity threshold value or not is judged, if the leakage risk score exceeds the preset sensitivity threshold value, the data combination is marked as highly sensitive data, an early warning mechanism is triggered, and desensitization and encryption are conducted on the data combination according to a preset desensitization rule and an encryption method; and continuously monitoring data updating conditions of different data sources, when newly added data is detected, automatically triggering a data fusion and leakage risk prediction process, obtaining sensitivity characteristic representation of the newly added data, updating a data leakage risk prediction model, and dynamically adjusting the sensitivity degree and protection measures of data combination.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method for building a data leakage prevention system. Background Art

[0002] In the era of big data, enterprises and institutions often need to integrate and analyze heterogeneous data from multiple sources to explore the potential value contained therein. However, the combination of different data sets may generate unexpected risks of sensitive information leakage. Even if each individual data set has been desensitized, seemingly unrelated data combinations may still reveal highly sensitive information such as personal identity and privacy. This requires identifying which data combinations may produce high sensitivity before data fusion, and taking corresponding protection measures accordingly. The current technical contradiction is that the value of data fusion is often based on the full combination of multi-source heterogeneous data, but this combination may bring unpredictable privacy leakage risks. How to fully explore the value of data while identifying the sensitivity of various data combinations in advance and taking preventive measures is an urgent problem to be solved. This requires in-depth analysis of data characteristics and business rules in different industry scenarios, abstracting a set of scientific sensitivity assessment methods, and building a prediction model based on this to achieve early warning and precise prevention and control of high-risk data combinations. At the same time, this set of methods must be able to adapt to the dynamic changes in the data environment, continuously optimize the prediction effect when new data is continuously added, and ultimately achieve the full release of data value and strong protection of privacy security. Summary of the invention

[0003] The present invention provides a method for building a data leakage prevention system, which mainly includes:

[0004] Obtain multiple data sets from different data sources, perform data preprocessing on each data set, use data mining methods to extract features, obtain sensitivity features of multiple data sets, and screen out several key features that have a significant impact on data leakage risks based on preset sensitivity assessment criteria;

[0005] Aiming at several key characteristics, a sensitivity fusion model of multi-source heterogeneous data is constructed. By setting weight parameters, the key features of different data sources are weighted and fused to obtain the fused target sensitivity feature set. The principal component analysis method is used to reduce the dimension of the target sensitivity feature set and extract the principal component features that affect the contribution of data leakage risk.

[0006] According to the main component characteristics, a data leakage risk prediction model is constructed, and the support vector machine algorithm is used to train the model and obtain the data leakage risk prediction model. The multi-source heterogeneous data to be predicted is input into the risk prediction model, and the leakage risk score of the data combination is calculated;

[0007] Preset a sensitivity threshold to determine whether the leakage risk score exceeds the preset sensitivity threshold. If it exceeds the threshold, the data combination is marked as highly sensitive data, triggering an early warning mechanism to desensitize and encrypt the data combination according to the preset desensitization rules and encryption methods;

[0008] Continuously monitor the data updates of different data sources. When new data is detected, automatically trigger the data fusion and leakage risk prediction process and obtain the sensitivity feature representation of the new data, update the data leakage risk prediction model, and dynamically adjust the sensitivity and protection measures of the data combination;

[0009] Based on business needs and data usage scenarios, formulate data access and sharing strategies. For data combinations with low leakage risk but high sensitivity, strictly limit data access and circulation to ensure data security. For data combinations with high leakage risk but low sensitivity, authorized users are allowed to use and analyze data within a certain scope based on user permissions and access approval rules.

[0010] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0011] The present invention discloses a method for building a data leakage prevention system. The method obtains multiple data sets, performs data preprocessing and feature extraction, and identifies key sensitivity features. Using association analysis technology, the potential association between data sets is discovered, and the risk after combination is evaluated. A sensitivity fusion model is constructed, and the features are weighted and fused and reduced in dimension to extract the principal component features. Based on this, a data leakage risk prediction model is constructed to calculate the leakage risk score of the data combination. When the preset threshold is exceeded, an early warning is triggered and desensitization encryption is implemented. The present invention can also continuously monitor data updates, dynamically adjust the risk prediction model, and formulate differentiated data access strategies according to business needs. This method can effectively identify and prevent potential leakage risks brought about by multi-source heterogeneous data combinations, improve data security, and ensure the rational use of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The present invention is a flowchart of a method for building a data leakage prevention system.

[0013] Figure 2 A schematic diagram of a method for building a data leakage prevention system according to the present invention.

[0014] Figure 3 This is another schematic diagram of a method for building a data leakage prevention system according to the present invention. DETAILED DESCRIPTION

[0015] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0016] like Figure 1-3 In this embodiment, a method for building a data leakage prevention system may specifically include:

[0017] S101. Obtain multiple data sets from different data sources, perform data preprocessing on each data set, use data mining methods to extract features, obtain sensitivity features of the multiple data sets, and screen out several key features that have a significant impact on data leakage risks based on preset sensitivity assessment standards.

[0018] The data in the relational database is obtained according to the data source interface adapter, and the data files on the file server are read through the FTP protocol; the data extraction, conversion and loading process is executed to obtain the original data set; data cleaning, standardization and structural conversion are performed on the original data set to obtain the preprocessed data set; the preprocessed data set is reduced in dimension to extract the main features, and the influence of each feature on the data leakage risk is calculated; if the information gain ratio exceeds the preset threshold, the feature is included in the candidate feature set; a feature importance evaluation model is constructed according to the candidate feature set, and the features with the lowest importance are iteratively eliminated through the recursive feature elimination method until the number of remaining features reaches the preset target; the remaining features are normalized to obtain a standardized key feature data set; the Apriori algorithm is used to mine association rules on the standardized key feature data set, and the support and confidence between feature pairs are calculated; the feature association network diagram is drawn according to the support and confidence, and the Dijkstra shortest path algorithm is used to identify the risk propagation path.

[0019] Exemplarily, a data source interface adapter is constructed, JDBC technology is used to connect to a relational database, the FTP protocol is used to read data files on a file server, the Kettle tool is used to execute the data extraction, conversion and loading process to extract raw data, and data cleaning, standardization and structural conversion are performed according to the data type and format to generate a preprocessed data set in a standard format. For the preprocessed data set, the principal component analysis algorithm is used to reduce the dimension and extract the main features, and the information gain calculation formula in the ID3 algorithm is used to calculate the degree of influence of each feature on the risk of data leakage. The information gain ratio threshold is set to 0.1 to screen out candidate feature sets with significant impact on the risk. A feature importance evaluation model based on a decision tree is constructed, and the candidate feature set and the preset sensitivity evaluation index are input. The redundant features are iteratively eliminated by the recursive feature elimination method, and the 10% features with the lowest importance are eliminated each iteration until the number of remaining features reaches the preset target, and the final key feature subset is obtained. The selected key features are normalized, and the feature values ​​are mapped to the [0,1] interval using the Min-Max normalization method to obtain a standardized key feature data set. The Apriori algorithm is used to mine association rules for the standardized key feature data set. The minimum support is set to 0.05 and the minimum confidence is set to 0.6. The support and confidence between feature pairs are calculated, and the feature association network diagram is drawn. The Dijkstra shortest path algorithm is used to identify the risk propagation path. The PageRank algorithm is used to calculate the node centrality, determine the key nodes, and form the data leakage risk assessment results. When building the data source interface adapter, the Oracle database is connected through JDBC technology, and the connection string "jdbc:oracle:thin:@localhost:1521:orcl" and the user name "admin" and password "password123" are used to establish a connection. At the same time, the FTP protocol is used to connect to the file server, and the host address "ftp.example.com", port number 21, user name "ftpuser" and password "ftppass456" are used to access the " / data" directory on the server. Use Kettle to create a data extraction, transformation and loading job, set the data source connection parameters, define the transformation steps including table input, text file input, field selection, filter and table output, and convert the data from different sources into UTF-8 encoded CSV format and store it in the preprocessed dataset directory " / preprocessed_data". Apply the principal component analysis algorithm to the preprocessed dataset, set the number of principal components to 10, and the cumulative explained variance ratio reaches 95%. Use the ID3 algorithm to calculate the information gain. For feature A, calculate the information gain G(D,A)=H(D)-H(D|A), where H(D) is the entropy of dataset D and H(D|A) is the conditional entropy under feature A.The information gain ratio threshold was set to 0.1, and the features with information gain ratio greater than the threshold were selected as candidate feature sets. A feature importance evaluation model based on decision tree was constructed, and the decision tree was generated by CART algorithm. The maximum depth of the tree was set to 10, and the minimum number of leaf node samples was 5. Redundant features were iteratively eliminated by recursive feature elimination method, and the 10% features with the lowest importance score were removed in each iteration until the number of remaining features was reduced to 50. The selected key features were Min-Max standardized, and the feature value x was mapped to the interval [0,1]. The calculation formula was the normalized value x'=(x-min) / (max-min), where min and max were the minimum and maximum values ​​of the feature, respectively. The Apriori algorithm was used to mine association rules on the standardized key feature data set, and the minimum support was set to 0.05 and the minimum confidence was set to 0.6 to generate an association rule set. A feature association network diagram was constructed based on association rules, with nodes representing features, edges representing association relationships, and the weight of the edge being the confidence of the corresponding rule. The Dijkstra algorithm was used to calculate the shortest path between nodes and identify the risk propagation path. The PageRank algorithm is used to calculate node centrality, with the number of iterations set to 100 and the damping factor set to 0.85. The 10 nodes with the highest scores are identified as key nodes. The final data leakage risk assessment results include a list of key features, a feature association network diagram, a risk propagation path, and key node information.

[0020] Analyze the data type, structure and content of each data set, cross-compare the fields and information in different data sets, identify related data items, use association rule mining or graphical analysis data association analysis methods to find out the potential associations between different data sets, evaluate the new information or inferences generated after the combination of data sets, and identify potential risk points, including personal privacy leakage and sensitive information inference.

[0021] Obtain metadata information of each data set, extract data type, structure and field information from the metadata information; calculate the Jaccard similarity of field names based on field information, and calculate the Pearson correlation coefficient of data features for numeric fields; if the Jaccard similarity or Pearson correlation coefficient is greater than the preset similarity threshold, determine the corresponding field as a potential associated field pair; apply the Apriori algorithm to mine frequent item sets between data sets; calculate the support and confidence of association rules for frequent item sets, and if the support is greater than the preset minimum support and the confidence is greater than the preset confidence threshold, determine it as a strongly associated data item combination; construct a data association network using strongly associated data item combinations, and use the Louvain algorithm to detect closely associated data subsets in the data association network; based on the data association network, predict the risk of information leakage and obtain a risk score; if the risk score is higher than the preset risk threshold, mark the corresponding node as a high-risk node.

[0022] Exemplarily, the data type, structure and field information of each data set are extracted through the metadata parsing tool, and the field distribution characteristics, including mean, variance and distribution type, are calculated using Python's pandas library to generate a data set structure description file. The Jaccard similarity is used to calculate the field name similarity, and the Pearson correlation coefficient is used to calculate the data feature similarity of the numerical field. The similarity threshold is set to 0.8, and potential associated field pairs are screened out to construct a field association graph between data sets. According to the field association graph, the Apriori algorithm is used to mine frequent item sets and association rules between data sets, and the minimum support is set to 0.05 and the confidence threshold is set to 0.6 to identify strongly associated data item combinations. Semantic analysis is performed on the identified strongly associated data item combinations, and the Word2Vec model is used to calculate the semantic similarity between fields to verify the rationality of the association. The semantic similarity threshold is set to 0.7. A data association network is constructed based on the verified associated data items, and the Louvain algorithm is used to detect closely associated data subsets. The sensitive information vocabulary automatically expanded by the industry standard personal information classification directory and the naive Bayes classifier is combined to mark nodes containing personal privacy and sensitive information. Using the GraphSAGE graph neural network model, based on the constructed data association network, potential information leakage risks are predicted, risk scores and warning reports are generated, with a score range of 0-100. Nodes with scores above 80 are marked as high risk, and the final list of potential risk points is output. During the implementation process, the automated metadata parsing tool is first used to scan multiple data sets to extract field information including user ID, name, age, address, etc., and then the pandas library is used to calculate the statistical characteristics of these fields, such as the number of unique values ​​of user ID, the average length of the name is 10 characters, the mean age is 35 years old, and the variance is 15. Then the fields in different data sets are calculated for similarity. For example, the Jaccard similarity of "customer name" and "user name" is 0.85, and the Pearson correlation coefficient of "annual income" and "monthly salary" is 0.92, both of which are higher than the set threshold of 0.8, so they are identified as potential association fields. Then, the Apriori algorithm found frequent item sets in these associated fields, such as {user ID, mobile phone number, home address}, with a support of 0.06 and a confidence of 0.75, exceeding the preset threshold. Further semantic analysis of these associated items was performed, and the pre-trained Word2Vec model was used to calculate the semantic similarity of the fields. For example, the similarity of "address" and "location" was 0.85, which verified the rationality of their association. Subsequently, a data association network was constructed, in which nodes represent data fields and edges represent association relationships. The Louvain algorithm was applied to detect communities and identify highly associated subgraphs, such as the closely associated subset containing {name, ID number, home address, work unit}. Potential sensitive information nodes were marked by combining the industry standard personal information classification directory and the sensitive information vocabulary trained by the naive Bayes classifier.Finally, the GraphSAGE model is used to analyze this association network and predict the risk score of each node. For example, a node containing detailed consumption records of users received a high risk score of 92 points and was included in the final list of potential risk points, generating a detailed risk assessment report, including potential information leakage channels and possible impacts.

[0023] S102. Construct a sensitivity fusion model for multi-source heterogeneous data based on several key characteristics. By setting weight parameters, perform weighted fusion on the key features of different data sources to obtain a fused target sensitivity feature set. Use principal component analysis to perform dimensionality reduction on the target sensitivity feature set to extract the principal component features that affect the contribution to data leakage risk.

[0024] The preset scoring criteria for multiple heterogeneous data sources are obtained, and the preset scoring criteria include data update frequency, data integrity and outlier ratio; the reliability score of each data source is calculated according to the preset scoring criteria to obtain the data source weight vector; the random forest algorithm is used to calculate the importance score of the key features in each data source in combination with the data source reliability score, and a weighted feature fusion matrix is ​​constructed by the product of the data source weight and the feature importance score; the weighted feature fusion matrix is ​​used to perform weighted sum operation on the key features of different data sources. If there are features with the same name, the weighted average method is used for fusion to obtain the fused target sensitivity feature set; the fused target sensitivity feature set is subjected to Z-score normalization to eliminate the dimensional differences between different features; for the standardized target sensitivity feature set, the feature covariance matrix is ​​calculated and the eigenvalues ​​and eigenvectors are solved, and the first several principal components whose cumulative contribution rate reaches the preset threshold are selected, and they are arranged in descending order according to the size of the eigenvalues ​​to obtain the principal component feature set after dimensionality reduction.

[0025] Exemplarily, according to the preset data source reliability scoring criteria, including data update frequency, data integrity and outlier ratio, multiple heterogeneous data sources are evaluated, the reliability score of each data source is calculated, and the data source weight vector is generated. The random forest algorithm is used, combined with the data source reliability score, to calculate the importance score of the key features in each data source, and the weighted feature fusion matrix is ​​constructed by multiplying the data source weight and the feature importance score. Using the weighted feature fusion matrix, the key features of different data sources are weighted and summed. For the features with the same name, the weighted average method is used for fusion to generate the fused target sensitivity feature set. The fused feature set is Z-score standardized to eliminate the dimensional differences between different features. The principal component analysis algorithm is applied to the standardized target sensitivity feature set, the feature covariance matrix is ​​calculated, the eigenvalues ​​and eigenvectors are solved, and the first several principal components with a cumulative contribution rate of 85% are selected, and the eigenvalues ​​are arranged in descending order to obtain the principal component feature set after dimensionality reduction. In the actual implementation process, the reliability of three heterogeneous data sources (relational database, log file and API data stream) is first evaluated. The frequency of data updates is calculated as the number of updates per hour, the data integrity is expressed as the proportion of non-empty fields, and the proportion of outliers is identified by the 3-times standard deviation method. For example, the update frequency of data source A is 24 times / day, the integrity is 98%, the proportion of outliers is 0.5%, and the final reliability score is 0.9; the scores of data sources B and C are 0.8 and 0.7 respectively. Subsequently, the random forest algorithm is used to calculate the importance of features. Assuming that there are 10 key features, the importance score of the "user behavior" feature in data source A is 0.8, which is multiplied by the data source weight of 0.9 to obtain a weighted score of 0.72. Repeat this process for all features and data sources to construct a 10x3 weighted feature fusion matrix. Next, the weighted average of the features with the same name in the matrix is ​​performed, such as the weighted scores of "user ID" in the three data sources are 0.72, 0.64 and 0.56, and the score after fusion is 0.64. This operation is performed for all features to obtain a 10-dimensional fusion feature vector. Then, the 10 features are Z-score standardized, and the mean value of each eigenvalue is subtracted and divided by the standard deviation, so that the mean of all features is 0 and the standard deviation is 1. Finally, principal component analysis is applied to the standardized feature set to calculate the feature covariance matrix and solve the eigenvalues ​​and eigenvectors. Assuming that the cumulative contribution rate of the first four principal components reaches 86%, which exceeds the preset threshold of 85%, these four principal components are selected as the final dimensionality reduction result to form a 4-dimensional principal component feature set, which effectively extracts the main factors affecting the risk of data leakage.

[0026] S103. According to the principal component characteristics, a data leakage risk prediction model is constructed, and the model is trained using a support vector machine algorithm to obtain a data leakage risk prediction model. The multi-source heterogeneous data to be predicted is input into the risk prediction model, and the leakage risk score of the data combination is calculated.

[0027] Obtain samples of known risk levels from the historical sample database and log files, and perform missing value filling and outlier detection processing on the sample data; determine the principal component characteristics as input variables and the risk level as output variables based on the processed sample data to obtain a standardized training data matrix; use the grid search method to optimize the kernel function type, penalty parameter C and kernel function parameter γ of the support vector machine. If the search range of the penalty parameter C is from the preset first value to the second value, and the search range of the kernel function parameter γ is from the preset third value to the fourth value, then determine the optimal parameter combination through cross-validation; according to the optimal parameter combination, train the support vector machine model, and use the sequential minimum optimization algorithm to solve the dual problem to obtain the support vector and decision function; for the multi-source heterogeneous data to be predicted, obtain the principal component feature representation in the same format as the training data through feature extraction and principal component transformation, input the principal component feature representation into the trained support vector machine model, and judge the leakage risk score of the data combination.

[0028] Exemplarily, a training data set is constructed based on the principal component features, historical samples with known risk levels are extracted from databases and log files, missing values ​​are filled and outliers are detected for the data, the principal component features are used as input variables, and the risk level is used as the output variable to form a standardized training data matrix. The grid search method is used to optimize the kernel function type, penalty parameter C and kernel function parameter γ of the support vector machine. The search range of C is set to 0.1 to 100, and the search range of γ is set to 0.001 to 1, both of which grow exponentially. The optimal parameter combination is determined by 5-fold cross validation. The support vector machine model is trained using the optimal parameter configuration, and the sequential minimum optimization algorithm is used to solve the dual problem. The specific implementation includes selecting variable pairs that violate the KKT condition, updating the Lagrange multiplier and bias term, iterating until convergence, obtaining the support vector and decision function, and constructing a data leakage risk prediction model. The performance of the trained model is evaluated, and the accuracy, precision, recall and other indicators are calculated using an independent test set, and the risk warning threshold is set to 0.8. The multi-source heterogeneous data to be predicted are converted into principal component feature representation in the same format as the training data through feature extraction and principal component transformation, and then input into the trained support vector machine model. The leakage risk score of the data combination is calculated through the decision function. In the actual implementation process, 10,000 historical data samples are first extracted from the enterprise database and log files, including 20 principal component features and corresponding risk level labels. The data is preprocessed, 5% of missing values ​​are processed by mean filling, and 2% of outliers are detected and removed by the 3 times standard deviation method, and finally 9,800 valid samples are obtained. Then, the support vector machine parameters are optimized using grid search, the search range of C is set to [0.1, 1, 10, 100], and the search range of γ is [0.001, 0.01, 0.1, 1]. The performance of each set of parameters is evaluated by 5-fold cross validation. After 200 iterations, the optimal parameter combination is determined to be C = 10, γ = 0.01, and the kernel function is selected as RBF kernel. The support vector machine model was trained using the sequential minimum optimization algorithm. Two samples that most violated the KKT condition were selected in each iteration, and their Lagrange multipliers and model bias terms were updated. After 5,000 iterations, the model converged and 1,500 support vectors were obtained. Subsequently, 2,000 independent test samples were used to evaluate the model performance, and the accuracy, precision, and recall rates were 94%, 92%, and 95%, respectively. The risk warning threshold was set to 0.8, that is, a risk score exceeding 0.8 was considered high risk. Finally, a new set of multi-source heterogeneous data was predicted. First, the data from different sources were uniformly converted into a 20-dimensional principal component feature representation, and then input into the trained model to obtain a risk score of 0.75, which was lower than the warning threshold and was judged to be a medium risk level.

[0029] S104. Preset a sensitivity threshold to determine whether the leakage risk score exceeds the preset sensitivity threshold. If it exceeds the threshold, the data combination is marked as highly sensitive data, triggering an early warning mechanism, and desensitizing and encrypting the data combination according to preset desensitization rules and encryption methods.

[0030] Obtain the leakage risk score of the data combination to be evaluated from the database, and determine whether the risk score exceeds the sensitivity threshold; if the risk score exceeds the sensitivity threshold, mark the data combination as highly sensitive data, and generate warning information containing data ID, risk score and trigger time; select the corresponding desensitizing algorithm from the preset desensitizing rule library to desensitize the data combination according to the sensitive information type and highly sensitive data mark in the data combination; for the desensitized data combination, select an encryption algorithm from AES, RSA or homomorphic encryption algorithm according to the data sensitivity and processing performance requirements for encryption processing to obtain the encrypted data combination; store the encrypted data combination in the security audit database, and record the desensitization and encryption information.

[0031] Exemplarily, according to historical data statistics, ROC curve analysis is used to select the best balance point, the sensitivity threshold is set to 0.8, the leakage risk score of the data combination to be evaluated is read from the database, and the comparator is used to determine whether the risk score exceeds the preset threshold. If the risk score exceeds the threshold, the data marking module is called to mark the data combination as highly sensitive data, and the warning signal is triggered at the same time, and the warning information containing the data ID, risk score, and trigger time is generated, written into the log system and pushed to relevant personnel. According to the different types of sensitive information and highly sensitive data tags in the data combination, the corresponding desensitization algorithm is automatically selected from the preset desensitization rule library, such as using K-anonymity processing for the ID card number and using the L-diversity algorithm to partially hide the mobile phone number. Call the encryption module, select the appropriate encryption algorithm from AES, RSA and homomorphic encryption algorithms according to the data sensitivity and processing performance requirements, encrypt the desensitized data, generate the encrypted data combination and store it. Record the data processing log, including detailed information on desensitization and encryption operations, and store it in the security audit database for subsequent data processing tracking and security auditing. In the actual implementation process, 10,000 historical data were first used to perform ROC curve analysis, and the true positive rate and false positive rate under different thresholds were calculated. The point 0.8 that maximized the Youden index (true positive rate-false positive rate) was selected as the sensitivity threshold. Subsequently, the data combination to be evaluated was read from the Oracle database, and its leakage risk score was 0.85. The comparator judged that 0.85 exceeded the preset threshold of 0.8, and automatically triggered the data marking module, marking the data combination as "highly sensitive". At the same time, an early warning message {data ID: DT20240415001, risk score: 0.85, trigger time: 2024-04-1514:30:25} was generated, written into the Elasticsearch log system and pushed to the security administrator through WeChat for Enterprise. Then, the system detected that the data combination contained two types of sensitive information, ID card number and mobile phone number, and automatically selected the K-anonymity algorithm (K=5) from the desensitization rule library to process the ID card number, and used the L-diversity algorithm (L=3) to partially hide the mobile phone number. Then, according to the high sensitivity of the data and the real-time processing requirements, the AES-256 encryption algorithm is selected to encrypt the desensitized data, and the preset key "3F4528482B4D6251655468576D5A7134" is used for encryption. Finally, the system generates a processing log of {operation ID: OP20240415001, desensitization algorithm: [K-anonymity, L-diversity], encryption algorithm: AES-256, processing time: 2024-04-1514:30:26} and stores it in the MongoDB security audit database.

[0032] S105. Continuously monitor the data update status of different data sources. When new data is detected, automatically trigger the data fusion and leakage risk prediction process and obtain the sensitivity feature representation of the new data, update the data leakage risk prediction model, and dynamically adjust the sensitivity and protection measures of the data combination.

[0033] Receive the update status information sent by the data source, where the update status information includes the timestamp and data source identifier of the newly added data; call the data fusion module according to the update status information, where the data fusion module adopts an incremental update strategy based on timestamp comparison to integrate the newly added data with the existing data; perform feature extraction on the fused data set generated by the data fusion module, and use the principal component analysis method to reduce the dimension of the features to obtain a new sensitivity feature representation; input the sensitivity feature representation into the data leakage risk prediction model to obtain an updated risk score; if the updated risk score deviates from the preset threshold, use the stochastic gradient descent algorithm to perform online learning and incremental training on the data leakage risk prediction model to update the parameters of the data leakage risk prediction model; adjust the protection measures of the corresponding data combination according to the updated risk score, including encryption strength or access control level; record the parameter update and protection measures adjustment information of the data leakage risk prediction model in the security audit database.

[0034] Exemplarily, a data source monitoring program is deployed to detect the update status of each data source in real time through scheduled polling every 5 minutes or using the ApacheKafka message queue mechanism. When new data is found, the update timestamp and data source identifier are recorded. According to the detected new data, the data fusion module is called to integrate the new data with the existing data. The incremental update strategy based on timestamp comparison is adopted to update only the new or changed data to generate a fused data set. The fused data set is subjected to feature extraction, and the principal component analysis is used for dimensionality reduction. The principal component that explains 95% of the variance is retained to obtain a new sensitivity feature representation. The feature vector is input into the existing data leakage risk prediction model to calculate the updated risk score. Based on the new risk score, the parameters of the data leakage risk prediction model are updated, and the stochastic gradient descent algorithm is used for online learning and incremental training to dynamically adjust the model weights. At the same time, according to the new sensitivity, the protection measures of the corresponding data combination are automatically updated, such as adjusting the encryption strength or modifying the access control level. Detailed log records are made for model updates and protection measures adjustments, including update time, adjustment content and operator, and the logs are stored in the security audit database to ensure the traceability of the entire process. In the actual implementation process, the system deployed a data source monitoring program, used the Quartz scheduling framework to set a scheduled task with a 5-minute interval, and configured the Apache Kafka message queue with the Topic "data_update". When 100 new records were detected in the database table "user_info", a message {source:"user_db",timestamp:"2024-04-1515:30:00",count:100} was immediately sent to Kafka. After receiving the message, the data fusion module used JDBC to connect to the database and executed the SQL query "SELECT * FROM user_info WHERE update_time>'2024-04-1515:25:00'" to obtain the newly added data. Subsequently, the merge function of the pandas library was used to merge the new data with the existing data set based on the "user_id" field to generate a fused data set containing 10,100 records. The PCA algorithm in the scikit-learn library was applied to the fusion data set for dimensionality reduction. The n_components parameter was set to 0.95, and the original 50 features were reduced to 15 principal components. The new feature vector was input into the pre-trained support vector machine model, and the updated average risk score was calculated to be 0.72. Based on the new risk score, the stochastic gradient descent algorithm was implemented using SGDClassifier, and the parameters loss = 'hinge', alpha = 0.0001, and max_iter = 1000 were set to perform incremental training on the model.At the same time, according to the change in risk score, the data encryption algorithm is upgraded from AES-128 to AES-256, and the access control level is adjusted from "internally visible" to "confidential". Finally, the system generates an audit log {operation:"model_update",time:"2024-04-1515:35:00",risk_score:0.72,encryption:"AES-256",access_level:"Confidential"} and stores it in the "audit_log" collection of the MongoDB database.

[0035] S106. Develop data access and sharing strategies based on business needs and data usage scenarios. For data combinations with low leakage risk but high sensitivity, strictly restrict data access and circulation to ensure data security. For data combinations with high leakage risk but low sensitivity, allow authorized users to use and analyze data within a certain scope based on user permissions and access approval rules.

[0036] Obtain data leakage risk and sensitivity information, and construct a data risk matrix. The data risk matrix includes a horizontal axis representing leakage risk and a vertical axis representing sensitivity; map the data combination to several quadrants according to the data risk matrix, and determine access and sharing strategies for the quadrants, wherein temporary authorization and full-process monitoring strategies are adopted for the high-risk and low-sensitivity quadrant; if it is judged that the data combination belongs to the quadrant with low leakage risk but high sensitivity, implement a strong access control mechanism, which includes multi-factor authentication and fine-grained authorization; if it is judged that the data combination belongs to the quadrant with high leakage risk but low sensitivity, design a dynamic authorization mechanism, which includes granting temporary access rights after approval by department heads, data security officers, and system administrators at three levels; deploy a comprehensive log audit and anomaly monitoring system, which uses the isolation forest algorithm to analyze user behavior patterns in real time to obtain abnormal access detection results.

[0037] Exemplarily, a data risk matrix is ​​constructed, with the horizontal axis representing the risk of leakage (low, medium, high) and the vertical axis representing the sensitivity (low, medium, high). The data combination is mapped to 9 quadrants, and corresponding access and sharing strategies are defined for each quadrant, such as the high-risk and low-sensitivity quadrant adopting temporary authorization and full-process monitoring strategies. According to business needs and data usage scenarios, data access templates are preset for different departments and roles. For example, the marketing department can access basic customer information but cannot access transaction records. For data combinations with low leakage risk but high sensitivity, a strong access control mechanism is implemented, using multi-factor authentication, such as fingerprint recognition combined with dynamic passwords and fine-grained authorization, to limit the scope of data access and usage duration. For data combinations with high leakage risk but low sensitivity, a dynamic authorization mechanism is designed to automatically generate access application workflows based on user roles, business needs and access history, and grant temporary access rights after approval by department heads, data security officers and system administrators. Deploy a comprehensive log audit and anomaly monitoring system to record all data access and operation behaviors, use the isolation forest algorithm to analyze user behavior patterns in real time, detect abnormal access and potential data abuse, and automatically trigger an alarm and suspend the access rights of relevant users when abnormal behavior is detected. In the actual implementation process, a 3x3 data risk matrix was first constructed to map 100 data combinations to 9 quadrants. Among them, the high-risk and low-sensitivity quadrant, such as customer browsing records, adopts a 24-hour temporary authorization and full-process behavior monitoring strategy. Subsequently, according to the business needs of the company's five core departments, 15 data access templates were preset, such as the marketing department can access basic customer information but is restricted to viewing transaction records in the last three months. For data combinations with a risk score lower than 0.3 but a sensitivity higher than 0.8, such as employee salary information, a two-factor authentication mechanism was implemented, requiring users to provide fingerprint recognition and a 6-digit dynamic password, and limiting the access time to 30 minutes. For data combinations with a risk score higher than 0.7 but a sensitivity lower than 0.4, such as anonymized user behavior data, a dynamic authorization process is designed. After the user submits an application, a workflow containing data description, purpose of use, and time range is automatically generated. The application is approved by the department head within 4 hours, the data security officer within 12 hours, and the system administrator within 24 hours. After passing the three-level approval, a temporary access right for 72 hours is granted. Finally, a log audit system based on the ELK (Elasticsearch, Logstash, Kibana) stack is deployed, which processes 1,000 access logs per second and uses the isolation forest algorithm (with a pollution rate of 0.1) to analyze user behavior in real time. When abnormal behavior is detected, such as downloading a large amount of data in a short period of time, the system automatically triggers an alarm within 5 seconds and immediately suspends the access rights of the relevant users.

[0038] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the present application. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

Claims

1. A method for building a data leakage prevention system, characterized in that: The method comprises: Obtain multiple data sets from different data sources, perform data preprocessing on each data set, use data mining methods to extract features, obtain sensitivity features of multiple data sets, and screen out several key features that have a significant impact on data leakage risks based on preset sensitivity assessment criteria; Aiming at several key characteristics, a sensitivity fusion model of multi-source heterogeneous data is constructed. By setting weight parameters, the key features of different data sources are weighted and fused to obtain the fused target sensitivity feature set. The principal component analysis method is used to reduce the dimension of the target sensitivity feature set and extract the principal component features that affect the contribution of data leakage risk. According to the main component characteristics, a data leakage risk prediction model is constructed, and the support vector machine algorithm is used to train the model and obtain the data leakage risk prediction model. The multi-source heterogeneous data to be predicted is input into the risk prediction model, and the leakage risk score of the data combination is calculated; Preset a sensitivity threshold to determine whether the leakage risk score exceeds the preset sensitivity threshold. If it exceeds the threshold, the data combination is marked as highly sensitive data, triggering an early warning mechanism to desensitize and encrypt the data combination according to the preset desensitization rules and encryption methods; Continuously monitor the data updates of different data sources. When new data is detected, automatically trigger the data fusion and leakage risk prediction process and obtain the sensitivity feature representation of the new data, update the data leakage risk prediction model, and dynamically adjust the sensitivity and protection measures of the data combination; Based on business needs and data usage scenarios, formulate data access and sharing strategies. For data combinations with low leakage risk but high sensitivity, strictly limit data access and circulation to ensure data security. For data combinations with high leakage risk but low sensitivity, authorized users are allowed to use and analyze data within a certain scope based on user permissions and access approval rules.

2. The method according to claim 1, characterized in that The method obtains multiple data sets from different data sources, performs data preprocessing on each data set, and uses data mining methods to extract features to obtain sensitivity features of multiple data sets. According to preset sensitivity evaluation criteria, several key features that have a significant impact on data leakage risks are screened out, including: Acquire data from relational databases using data source interface adapters, and read data files on file servers using FTP protocol; Execute the data extraction, transformation and loading process to obtain the original data set; Perform data cleaning, standardization, and structural transformation on the original data set to obtain the preprocessed data set; Reduce the dimension of the preprocessed data set to extract the main features, and calculate the impact of each feature on the risk of data leakage; If the information gain ratio exceeds the preset threshold, the feature is included in the candidate feature set; A feature importance evaluation model is constructed based on the candidate feature set, and the features with the lowest importance are iteratively eliminated through a recursive feature elimination method until the number of remaining features reaches the preset target; Normalize the remaining features to obtain a standardized key feature data set; Use the Apriori algorithm to mine association rules on the standardized key feature data set and calculate the support and confidence between feature pairs; Draw a feature association network diagram based on support and confidence, and use the Dijkstra shortest path algorithm to identify the risk propagation path; It also includes: analyzing the data type, structure and content of each data set, cross-comparing the fields and information in different data sets, identifying associated data items, using association rule mining or graphical analysis data association analysis methods to find potential associations between different data sets, evaluating new information or inferences generated after combining data sets, and identifying potential risk points, including personal privacy leakage and sensitive information inference.

3. The method according to claim 2, characterized in that The data type, structure and content of each data set are analyzed, the fields and information in different data sets are cross-compared, the associated data items are identified, the potential associations between different data sets are found by using association rule mining or graphical analysis data association analysis methods, the new information or inferences generated after the combination of data sets are evaluated, and potential risk points are identified. Potential risk points include personal privacy leakage and sensitive information inference, including: Obtain metadata information of each data set, and extract data type, structure, and field information from the metadata information; Calculate the Jaccard similarity of field names based on field information, and calculate the Pearson correlation coefficient of data features for numeric fields; If the Jaccard similarity or the Pearson correlation coefficient is greater than the preset similarity threshold, the corresponding fields are determined to be potential associated field pairs; Apply Apriori algorithm to mine frequent itemsets between data sets; Calculate the support and confidence of the association rules for the frequent item sets. If the support is greater than the preset minimum support and the confidence is greater than the preset confidence threshold, it is determined to be a strongly associated data item combination. The data association network is constructed by combining strongly associated data items, and the Louvain algorithm is used to detect the closely associated data subsets in the data association network; Based on the data association network, predict the risk of information leakage and obtain the risk score; If the risk score is higher than the preset risk threshold, the corresponding node will be marked as a high-risk node.

4. The method according to claim 1, characterized in that: According to the above, a sensitivity fusion model of multi-source heterogeneous data is constructed for several key characteristics. By setting weight parameters, the key features of different data sources are weighted and fused to obtain a fused target sensitivity feature set. The principal component analysis method is used to reduce the dimension of the target sensitivity feature set and extract the principal component features that affect the contribution of data leakage risk, including: Obtain preset scoring criteria for multiple heterogeneous data sources, including data update frequency, data completeness, and outlier ratio; Calculate the reliability score of each data source according to the preset scoring criteria to obtain a data source weight vector; The random forest algorithm is used to calculate the importance scores of key features in each data source in combination with the data source reliability scores, and a weighted feature fusion matrix is ​​constructed by multiplying the data source weight and the feature importance score. The weighted feature fusion matrix is ​​used to perform weighted sum operation on the key features of different data sources. If there are features with the same name, the weighted average method is used for fusion to obtain the fused target sensitivity feature set. Perform Z-score normalization on the fused target sensitivity feature set to eliminate the dimensional differences between different features; For the standardized target sensitivity feature set, the feature covariance matrix is ​​calculated and the eigenvalues ​​and eigenvectors are solved. The first several principal components whose cumulative contribution rate reaches the preset threshold are selected and arranged in descending order according to the eigenvalue size to obtain the principal component feature set after dimensionality reduction.

5. The method according to claim 1, characterized in that The data leakage risk prediction model is constructed according to the principal component characteristics, the support vector machine algorithm is used to train the model and obtain the data leakage risk prediction model, the multi-source heterogeneous data to be predicted is input into the risk prediction model, and the leakage risk score of the data combination is calculated, including: Obtain samples of known risk levels from historical sample databases and log files, and perform missing value filling and outlier detection processing on sample data; According to the processed sample data, the principal component characteristics are determined as input variables and the risk level is determined as the output variable to obtain a standardized training data matrix; The kernel function type, penalty parameter C and kernel function parameter γ of the support vector machine are optimized by using a grid search method. If the search range of the penalty parameter C is from a preset first value to a second value, and the search range of the kernel function parameter γ is from a preset third value to a fourth value, the optimal parameter combination is determined by cross-validation. According to the optimal parameter combination, the support vector machine model is trained, and the sequential minimum optimization algorithm is used to solve the dual problem to obtain the support vector and decision function; For the multi-source heterogeneous data to be predicted, the principal component feature representation in the same format as the training data is obtained through feature extraction and principal component transformation. The principal component feature representation is input into the trained support vector machine model to determine the leakage risk score of the data combination.

6. The method according to claim 1, characterized in that The preset sensitivity threshold determines whether the leakage risk score exceeds the preset sensitivity threshold. If it exceeds the threshold, the data combination is marked as highly sensitive data, triggering the early warning mechanism, and desensitizing and encrypting the data combination according to the preset desensitization rules and encryption methods, including: Obtain the leakage risk score of the data combination to be evaluated from the database, and determine whether the risk score exceeds the sensitivity threshold; If the risk score exceeds the sensitivity threshold, the data combination will be marked as highly sensitive data, and an early warning message containing the data ID, risk score, and trigger time will be generated; According to the sensitive information type and highly sensitive data mark in the data combination, select the corresponding desensitization algorithm from the preset desensitization rule library to desensitize the data combination; For the data combination after desensitization processing, according to the data sensitivity and processing performance requirements, an encryption algorithm is selected from AES and RSA or homomorphic encryption algorithm to perform encryption processing to obtain the encrypted data combination; The encrypted data combination is stored in the security audit database, and the information of the desensitization and encryption processing is recorded.

7. The method according to claim 1, characterized in that The continuous monitoring of data updates from different data sources, when new data is detected, automatically triggers the data fusion and leakage risk prediction process and obtains the sensitivity feature representation of the new data, updates the data leakage risk prediction model, and dynamically adjusts the sensitivity of the data combination and the protection measures, including: Receive update status information sent by the data source, the update status information includes the timestamp of the newly added data and the data source identifier; The data fusion module is called according to the update status information. The data fusion module adopts an incremental update strategy based on timestamp comparison to integrate the newly added data with the existing data. Extract features from the fused data set generated by the data fusion module, use the principal component analysis method to reduce the dimension of the features, and obtain a new sensitivity feature representation; Input the sensitivity feature representation into the data leakage risk prediction model to obtain an updated risk score; If there is a deviation between the updated risk score and the preset threshold, the stochastic gradient descent algorithm is used to perform online learning and incremental training on the data leakage risk prediction model to update the parameters of the data leakage risk prediction model; Adjust the protection measures for the corresponding data portfolio based on the updated risk score, including encryption strength or access control level; The parameter updates and protection measures adjustment information of the data leakage risk prediction model are recorded in the security audit database.

8. The method according to claim 1, characterized in that Based on business needs and data usage scenarios, data access and sharing strategies are formulated. For data combinations with low leakage risk but high sensitivity, data access and circulation are strictly restricted to ensure data security. For data combinations with high leakage risk but low sensitivity, authorized users are allowed to use and analyze data within a certain scope based on user permissions and access approval rules, including: Obtain data leakage risk and sensitivity information and construct a data risk matrix. The data risk matrix includes a horizontal axis representing leakage risk and a vertical axis representing sensitivity; Mapping data combinations to several quadrants according to the data risk matrix, defining access and sharing strategies for the quadrants, wherein a temporary authorization and full-process monitoring strategy is adopted for the high-risk and low-sensitivity quadrant; If the data combination is judged to be in the quadrant of low leakage risk but high sensitivity, a strong access control mechanism is implemented, which includes multi-factor authentication and fine-grained authorization; If the data combination is judged to belong to the quadrant with high leakage risk but low sensitivity, a dynamic authorization mechanism is designed. The dynamic authorization mechanism includes granting temporary access rights after approval by the department head, data security officer and system administrator at three levels; Deploy a comprehensive log audit and anomaly monitoring system. The comprehensive log audit and anomaly monitoring system uses the isolation forest algorithm to analyze user behavior patterns in real time and obtain abnormal access detection results.

Images