Cross-network information data security desensitization and access control method and system

Through SMOTomek technology and machine learning algorithms, improve the class imbalance problem of data sets, enhance the generalization ability of intrusion detection models, and combine fine-grained access control and desensitization mechanisms to solve the problem that traditional network security measures are difficult to prevent complex and variable attacks. Improved data security.

CN119094147AInactive Publication Date: 2024-12-06BEIJING HUADANG ZHIYUAN TECH CO LTD

Patent Information

Application Number
CN202410500200.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-03-13
Filing Date
2024-04-24
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional network security measures are difficult to adapt to complex and changeable security attacks and cannot effectively defend against new threats, especially in the generalization ability of intrusion detection models is insufficient under the problem of data class imbalance.

Method used

SMOTomek technology combined with machine learning algorithms is used to enhance the generalization ability of intrusion detection models by oversampling and preprocessing of data sets, and ensure the security of information and data through fine-grained access control and desensitization mechanisms.

Benefits of technology

It improves the defense capabilities of complex and variable security attacks, enhances the identification capabilities of intrusion detection models, reduces the risks of data leakage and abuse, and ensures the security of data throughout the entire life cycle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094147A_ABST
    Figure CN119094147A_ABST
Patent Text Reader

Abstract

The invention discloses a cross-network information data security desensitization and access control method and system, and the method comprises the steps: obtaining cross-network information data, and carrying out the desensitization of the cross-network information data; setting user permissions, and performing permission verification on the user request based on different user permissions; and after the authority verification succeeds, intrusion detection is carried out on the process that the user accesses the desensitized cross-network information data based on an oversampling and preprocessing SMOTETomek technology and a machine learning model. According to the method, the SMOTETomek technology can be combined with a machine learning algorithm to improve the class imbalance problem of a data set and enhance the generalization ability of an intrusion detection model, and the security of information data is ensured through a desensitization mechanism and fine-grained access control, so that the effective defense of complex and variable security attacks is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a method and system for cross-network information data security desensitization and access control. Background Art

[0002] With the rapid development of information technology, the network environment is becoming more and more complex, and enterprises and organizations are increasingly relying on cross-network systems to process and store data. Data security has become an issue that cannot be ignored. Traditional network security measures include the following:

[0003] Data encryption: Convert data into ciphertext through mathematical algorithms, so that even if others obtain the data, they cannot decipher the content except the authorized user.

[0004] Firewalls and intrusion detection systems: Block unauthorized access and monitor potential malicious activities through certain rules or patterns.

[0005] Access control: Limit user access to system resources by establishing access permissions and rules.

[0006] In the era of big data, enterprises and organizations have a large amount of personal sensitive information and key operational data. At this time, data desensitization is particularly critical. Data desensitization refers to the use of technical means to convert sensitive information into a data copy without sensitive content, so as to retain the use value of the data while preventing the leakage of sensitive information.

[0007] However, although existing data desensitization technologies can protect personal privacy and corporate information to a certain extent, in reality, due to the continuous updating of network intrusion methods, traditional intrusion detection systems are difficult to adapt to new threats and cannot effectively defend against complex and changing security attacks. Summary of the invention

[0008] In view of this, the present application provides a cross-network information data security desensitization and access control method and system, which can use SMOTETomek technology combined with machine learning algorithms to improve the class imbalance problem of data sets, enhance the generalization ability of intrusion detection models, and ensure the security of information data through desensitization mechanisms and fine-grained access control, thereby achieving effective defense against complex and changeable security attacks.

[0009] The present application provides a cross-network information data security desensitization and access control method, including:

[0010] Obtaining cross-network information data, and performing desensitization processing on the cross-network information data;

[0011] Set user permissions and verify user requests based on different user permissions;

[0012] After the permission verification is successful, intrusion detection is performed on the user's access to the desensitized cross-network information data process based on oversampling and preprocessing SMOTETomek technology and machine learning models.

[0013] Optionally, intrusion detection is performed on the process of a user accessing the desensitized cross-network information data based on oversampling and preprocessing SMOTETomek technology and a machine learning model, including:

[0014] Cleaning the desensitized cross-network information data;

[0015] Using SMOTETomek to oversample the cleaned cross-network information data;

[0016] The characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of users accessing the desensitized cross-network information data through a machine learning model.

[0017] Optionally, using SMOTETomek to oversample the cleaned cross-network information data includes:

[0018] Analyzing the desensitized cross-network information data to identify the distribution of normal and abnormal samples in the desensitized cross-network information data, wherein the data set of normal samples is larger than the data set of abnormal samples;

[0019] Using SMOTE to oversample the data set of the abnormal sample to synthesize a second abnormal sample, where the second abnormal sample is a new abnormal sample generated after oversampling;

[0020] Tomek links are used to identify and remove pairs of samples to strengthen the decision boundary of the classifier.

[0021] Optionally, using SMOTE to oversample the dataset of the abnormal samples includes:

[0022] Finding k nearest neighbors of the abnormal sample in the feature space, where k is a preset number of neighbors;

[0023] For each abnormal sample and its nearest neighbor, a new sample point is generated between the two using a linear interpolation method, wherein the set of the new sample points is the second abnormal sample;

[0024] The second abnormal sample is added to the desensitized cross-network information data.

[0025] Optionally, identify and remove sample pairs via Tomek links, including:

[0026] Obtaining oversampled abnormal samples, and identifying all nearest neighbor sample pairs Tomeklinks in the abnormal samples;

[0027] Remove pairs of samples that are identified as the majority class in Tomek links.

[0028] Optionally, obtaining features of the cross-network information data after oversampling, and performing intrusion detection on a process in which a user accesses the desensitized cross-network information data through a machine learning model based on the features of the cross-network information data, includes:

[0029] Using an information gain method to select features of the oversampled cross-network information data;

[0030] Using the selected features of the cross-network information data to construct a training data set;

[0031] Constructing a decision tree model, and training the decision tree model using the training data set;

[0032] The trained decision tree model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0033] Optionally, obtaining features of the cross-network information data after oversampling, and performing intrusion detection on a process in which a user accesses the desensitized cross-network information data through a machine learning model based on the features of the cross-network information data, includes:

[0034] Using a chi-square test method to select features of the cross-network information data after oversampling;

[0035] Using the selected features of the cross-network information data to construct a training data set;

[0036] Constructing a random forest model, and training the random forest model using the training data set;

[0037] Using regularization techniques to prevent overfitting of the random forest model and evaluate the performance of the model;

[0038] The trained random forest model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0039] Optionally, obtaining cross-network information data and performing desensitization processing on the cross-network information data includes:

[0040] Classify and process the cross-network information data to identify sensitive information that needs to be desensitized;

[0041] Determine the corresponding desensitization strategy based on the sensitivity level of different cross-network information data;

[0042] The sensitive information is desensitized according to the corresponding desensitization strategy.

[0043] Optionally, based on different user permissions, perform permission verification on user requests, including:

[0044] Use the access control list (ACL) mechanism to verify user access requests.

[0045] The disclosed embodiment also provides a cross-network information data security desensitization and access control system, including:

[0046] A desensitization module, used to obtain cross-network information data and perform desensitization processing on the cross-network information data;

[0047] The verification module is used to set user permissions and verify user requests based on different user permissions;

[0048] The detection module is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data based on oversampling and preprocessing SMOTETomek technology and machine learning models after the permission verification is successful.

[0049] This application provides a cross-network information data security desensitization and access control method and system, which uses SMOTETomek technology combined with machine learning algorithms to improve the class imbalance problem of data sets, enhance the generalization ability of intrusion detection models, and ensure the security of information data through desensitization mechanisms and fine-grained access control. It uses the ability of machine learning to identify and respond to unknown threats in real time, and uses more scientific data desensitization technology to ensure the security of data throughout its life cycle, becoming a key measure to improve network data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the disclosed embodiments in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram of the process of cross-network information data security desensitization and access control method provided in an embodiment of the present application;

[0052] Figure 2 It is a schematic diagram of the structure of a cross-network information data security desensitization and access control system provided in one embodiment. DETAILED DESCRIPTION

[0053] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0054] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0055] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0056] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0057] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0058] Embodiment 1

[0059] The cross-network information data security desensitization and access control method is mainly applicable to network systems that need to process and store a large amount of sensitive information. For example, in the financial services industry, banks and other financial institutions process a large amount of customer data, such as transaction records and personal identification information. In such an application scenario, desensitizing sensitive data is a basic requirement for maintaining data security. At the same time, it is also necessary to have the ability to dynamically monitor and respond to potential network threats to ensure that the integrity and privacy of the data are not compromised.

[0060] In addition to its application in the financial industry, the disclosed embodiments can also be applied to the fields of healthcare, e-commerce, social networking, etc., and can be used in any system that processes a large amount of sensitive information. For example, in the healthcare field, it can protect the patient's private data from unauthorized access and misuse, and at the same time has a better ability to identify potential network intrusions.

[0061] Exemplarily, after adopting the solution of the embodiment of the present disclosure, when the insurance company processes the claims data, some sensitive personal information is desensitized and cannot be directly identified, but the relevant processing procedures (such as claims review) can still operate normally. An online retail platform desensitizes the user's shopping records so that data analysis is not affected and the security of user privacy is improved. A social networking platform applies the cross-network information data security desensitization and access control mechanism, which not only protects user privacy, but also identifies and prevents a network attack on user data in advance through an intrusion detection model.

[0062] By integrating data desensitization, access control and machine learning intrusion detection technology, this system can effectively respond to emerging network threats on the basis of comprehensive data security, thereby achieving multiple guarantees for sensitive information protection and intrusion defense across network environments. The SMOTETomek technology is used to optimize the balance of data sets, enhance the model's ability to detect rare attack patterns, and reduce the risk of data leakage and abuse through fine-grained access control and effective data desensitization strategies.

[0063] Figure 1 A flow chart of a cross-network information data security desensitization and access control method provided in an embodiment of the present disclosure, the cross-network information data security desensitization and access control method comprises steps S1 to S3.

[0064] S1. Obtain cross-network information data, and perform desensitization processing on the cross-network information data;

[0065] Desensitizing means converting sensitive data through technical means, greatly reducing the risk of illegal data recovery while retaining the validity of the data as much as possible.

[0066] S1 specifically includes:

[0067] S11. Classify and process the cross-network information data to identify sensitive information that needs to be desensitized;

[0068] First, information data is obtained from different data sources through a network interface or data access layer. Then, the data is analyzed using a data classification algorithm and classified according to predefined sensitive data types such as personal identification, financial information, etc. Specifically, the data classification algorithm can be a rule engine that automatically identifies and marks sensitive information through natural language processing (NLP) technology and / or machine learning models to ensure that all sensitive data is accurately classified. It accurately identifies sensitive information and ensures that effective desensitization operations can be performed in subsequent processing, while reducing the impact on non-sensitive data.

[0069] S12. Determine the corresponding desensitization strategy according to the sensitivity level of different cross-network information data;

[0070] Specifically, for different types and levels of sensitive data, corresponding desensitization strategies such as masking, hashing, and assignment are adopted according to the data usage environment and security requirements. In addition, it should be noted that a policy decision engine is used, which will refer to industry standards, regulatory requirements, and best practices to select the most suitable desensitization method. The purpose of this is to effectively balance the value of data utilization and security protection requirements and reduce the risk of data leakage.

[0071] S13. Desensitize the sensitive information according to the corresponding desensitization strategy.

[0072] In S13, after applying the determined desensitization strategy, the sensitive data can be desensitized, for example, using an irreversible hash function to process information that needs to be kept completely confidential. In order to ensure the efficiency and effectiveness of desensitization, different data blocks can be processed in parallel, and the data quality can be tested after processing to ensure that no errors are introduced during the desensitization process. This protects sensitive information while maintaining the availability of data at the non-sensitive level.

[0073] Exemplarily, the specific process of S11-S13 can be implemented through the following sub-steps:

[0074] Sub-step 1: Use regular expressions and keyword matching to mark sensitive words in the extracted data.

[0075] Sub-step 2: Use the database's built-in classification function or a custom algorithm model to classify the data items into security levels.

[0076] Sub-step 3: Dynamically assign desensitization strategies based on data access frequency and sensitivity, for example, adding noise to commonly used data.

[0077] Sub-step 4: Use fake data instead of original data to anonymize information such as ID card number and mobile phone number.

[0078] Sub-step 5: Use advanced hash algorithms such as SHA-256 to perform hash desensitization on the identified highly sensitive data.

[0079] In the above steps, sensitive words in the data have been marked by regular expressions and keyword matching. At this point, it is necessary to filter out highly sensitive data, which is usually directly related to personal privacy or core business secrets of the enterprise, such as ID card numbers, bank account numbers, etc.

[0080] SHA-256 belongs to the SHA-2 (Secure Hash Algorithm 2) family and is an irreversible encryption algorithm that can convert a data string of any length into a unique data string (hash value) of fixed length (256 bits). Due to the high security of SHA-256, its output is extremely sensitive to accidental or intentional modification of the original data. Even a small change in the original data will result in a significant change in the output hash value.

[0081] Specifically, the SHA-256 algorithm is applied to the selected highly sensitive data. The algorithm performs multiple rounds of complex operations on the data content to generate a unique hash value. This ensures that even if the data is intercepted, without the corresponding reverse algorithm and sufficient computing power, the attacker cannot recover the original data from the hash value. For example:

[0082] For each selected piece of highly sensitive data, generate a random salt value and attach it to the original data to enhance the randomness and uniqueness of the hash algorithm. Apply the SHA-256 algorithm to the data with the added salt value to generate the corresponding hash value. Store the generated hash value and replace the original sensitive data to ensure that no form of original sensitive data is retained in the database. Regularly audit and check the hash process and results to ensure the accuracy and security of the desensitization mechanism. Finally, perform normalization or standardization on the desensitized data to ensure that the data remains consistent and correct in subsequent analysis.

[0083] S2. Set user permissions and verify user requests based on different user permissions;

[0084] For example, the access control list (ACL) mechanism can be used to verify the user access request. Specifically, S2 includes: S21: permission configuration

[0085] Define different access levels for different user roles to ensure the principle of least privilege. S22: Access request verification

[0086] Use the access control list (ACL) mechanism to strictly verify user access requests.

[0087] After confirming the user's permission configuration, the next step is to verify the user's access request to ensure that each access is legal and in compliance with the permission regulations. The verification process mainly includes the following sub-steps:

[0088] S221: Access request received

[0089] The system first needs to be able to capture all access requests to sensitive data and record the requests to provide a basis for subsequent verification and auditing.

[0090] S222: Authentication

[0091] Identify the user who requests access to ensure that the user is authorized. Identity verification can be done in a variety of ways, such as passwords, biometrics, and electronic certificates, to improve security levels.

[0092] S223: Permission verification

[0093] Based on the user's identity, check their access rights to the requested resources. Usually this step involves mechanisms such as access control list (ACL), role-based access control (RBAC) or attribute-based access control (ABAC).

[0094] S224: Request context analysis

[0095] Consider the context of the request, such as access time, location, device, and other factors, to further filter and restrict abnormal or non-policy-compliant access behaviors.

[0096] S225: Behavior pattern detection

[0097] The system can detect unusual behaviors such as frequent login attempts, abnormal data access patterns, etc. by analyzing users' historical access patterns, which helps identify potential security threats.

[0098] S226: Access decisions and records

[0099] Based on all the verification information, the system makes access decisions, allowing or denying access requests, and records the verification process and results in the security log to facilitate security audits.

[0100] Through the above verification steps, a strong access request verification mechanism can be established to effectively control access activities that may bring security risks and ensure the security of information resources.

[0101] S3. After the permission verification is successful, intrusion detection is performed on the user's access to the desensitized cross-network information data process based on oversampling and preprocessing SMOTETomek technology and machine learning model.

[0102] SMOTETomek means: combining the techniques of SMOTE (interpolating minority class samples to generate new samples) and Tomek (eliminating noise samples by identifying nearest neighbor sample pairs) to solve the class imbalance problem.

[0103] Among them, based on the oversampling and preprocessing SMOTETomek technology and the machine learning model, the intrusion detection is performed on the process of the user accessing the desensitized cross-network information data, including steps S31-S33:

[0104] S31. Cleaning the desensitized cross-network information data;

[0105] Data cleaning is an important step in data preprocessing, which involves checking, correcting, and deleting errors and irrelevant data in the data set. The purpose of data cleaning is to improve data quality and accuracy, laying a good foundation for data analysis and modeling. During the data cleaning process, it is necessary to remove duplicate records, correct errors, fill in missing values, and exclude outliers, etc., to ensure that the desensitized cross-network information data is accurate and reliable, and eliminate quality issues that may affect subsequent oversampling and machine learning model establishment.

[0106] Exemplarily, in the disclosed embodiments, duplicate data records are identified and deleted to ensure that each piece of data is unique. For missing values ​​in the data, appropriate strategies are adopted to fill them according to the statistical distribution of the attributes in which they are located, for example, using the mean, median, or using a prediction model as the fill value. Outlier detection is performed on numerical data to exclude outliers in the data set, which can be performed using Tukey's boxplot method or a standard deviation-based method. Correct format errors and inconsistent encoding problems in the data set, such as unifying date and time formats, text encoding, etc. Finally, perform data validation to detect and process records that do not conform to logic or data criteria through verification rules or data quality control algorithms.

[0107] S32. Oversampling the cleaned cross-network information data using SMOTETomek;

[0108] In S32, SMOTETomek is used to oversample the cleaned cross-network information data, which specifically includes steps S321-S323:

[0109] S321. Analyze the desensitized cross-network information data to identify the distribution of normal and abnormal samples in the desensitized cross-network information data, wherein the data set of normal samples is larger than the data set of abnormal samples;

[0110] After desensitization, the sample distribution in the data set needs to be analyzed to identify the proportion and distribution characteristics of normal and abnormal samples. This analysis is crucial for determining the subsequent oversampling strategy. Specifically, the number of each type of samples can be calculated through data analysis tools or custom algorithms, and the sample distribution can be displayed through visualization, so as to have a clear understanding of the class balance problem in the data set. In general, the proportion of abnormal samples is much lower than that of normal samples. This class imbalance may have an adverse effect on subsequent machine learning modeling.

[0111] Exemplarily, in S321, an Exploratory Data Analysis (EDA) method is used to analyze the characteristics and distribution of normal and abnormal samples in the data set. In addition, graphical analysis, such as a bar chart, a pie chart, or a scatter plot, can be used to visualize the distribution of samples of different categories. After identifying the distribution of normal and abnormal samples, the percentage of abnormal samples in the total samples is calculated, thereby providing a quantitative basis for the formulation of an oversampling strategy. Finally, it is necessary to confirm whether the abnormal samples in the data set are statistically significant through statistical tests, such as a chi-square test or ANOVA (analysis of variance), in order to determine the rationality and necessity of oversampling.

[0112] S322. Use SMOTE to oversample the data set of the abnormal sample to synthesize a second abnormal sample, where the second abnormal sample is a new abnormal sample generated after oversampling;

[0113] SMOTE (Synthetic Minority Over-sampling Technique) is a synthetic minority oversampling technique. This technique is a method of synthesizing new minority samples by interpolating between minority samples, thereby expanding the dataset of abnormal samples.

[0114] Specifically, SMOTE randomly selects k nearest neighbor samples for each abnormal sample, and then randomly generates new data points between each abnormal sample and its nearest neighbor samples, thereby increasing the number of abnormal samples and improving the class imbalance problem. SMOTE can enhance the classification model's ability to identify minority classes during training and improve the model's prediction performance by increasing the number of abnormal samples (minority classes).

[0115] In S322, the data set of the abnormal samples is oversampled using SMOTE, including the following steps A1-A3:

[0116] A1. Find the k nearest neighbors of the abnormal sample in the feature space, where k is a preset number of neighbors;

[0117] For each abnormal sample in the feature space, the nearest k samples are found by calculating the distance between other abnormal samples and it. The distance is usually calculated using Euclidean distance or other distance measurement methods.

[0118] For each abnormal sample in the data set, the k-nearest neighbor algorithm (k-NN) is used to find the nearest k samples in the abnormal sample set. The k nearest neighbor sample information of each abnormal sample is stored for subsequent interpolation.

[0119] A2. For each abnormal sample and its nearest neighbor, a new sample point is generated between the two using linear interpolation, wherein the set of the new sample points is the second abnormal sample;

[0120] The line segment between the abnormal sample and its nearest neighbor is used as the baseline, and new data points are randomly generated on the baseline through linear interpolation to form a new abnormal sample. For example, a neighboring point can be randomly selected, and then a point on the line between the target minority sample and this neighboring point can be randomly selected as the newly synthesized sample point.

[0121] Specifically, a neighboring sample of each abnormal sample is randomly selected, and new data points are generated between the selected neighboring samples and the abnormal samples in random proportions to ensure that the newly generated sample points contain the characteristic attributes of the original sample points and are between the characteristic values ​​of the abnormal sample and its selected neighboring samples.

[0122] A3. Add the second abnormal sample to the desensitized cross-network information data.

[0123] The new samples generated by the interpolation method are added to the original data set to increase the number of abnormal samples and achieve the purpose of oversampling.

[0124] Specifically, all newly generated abnormal samples are integrated into the original dataset to form a new augmented dataset. The sample distribution of the dataset is recalculated to ensure that the class distribution of the oversampled dataset is more balanced.

[0125] S323. Identify and remove sample pairs through Tomek links to strengthen the decision boundary of the classifier.

[0126] After completing the generation of synthetic samples, the key task is to clearly define the boundary between the minority class (such as intrusion behavior) and the majority class (such as normal behavior). In SMOTETomek, this step is mainly achieved by using Tomek Links.

[0127] Among them, identifying and removing sample pairs through Tomek links includes the following steps B1-B2:

[0128] B1. Obtain oversampled abnormal samples, and identify all nearest neighbor sample pairs Tomek links in the abnormal samples;

[0129] Tomek links exist in the nearest neighbor pairs of different categories. If two samples of different categories are each other's nearest neighbors, this pair of samples constitutes a Tomek link. Tomek links usually indicate that there may be noise or critical points at the boundary decision. In addition, for the data set that has been oversampled by SMOTE, all Tomek Links are identified. TomekLinks belong to different categories. These sample pairs in the data set are often located near the classification decision boundary.

[0130] In the oversampled data set, find the nearest neighbor for each sample. Check whether each pair of nearest neighbor samples belongs to different categories. If so, mark them as Tomek link.

[0131] B2. Remove the majority class sample pairs identified as Tomek links.

[0132] Removing the majority class samples identified as Tomek Links helps make the classification boundary clearer, thereby reducing the uncertainty of the classifier on the boundary between the minority class and the majority class. In other words, removing the labeled majority class samples will help the classifier better learn the distinction between the two classes and improve the performance of the classifier.

[0133] After removing Tomek Links, the dataset was reintegrated to ensure that the classification boundaries in the training dataset were more accurate.

[0134] Among the discovered Tomek links, remove the samples belonging to the majority class, which can make the class boundaries clearer. Repeat the above process until no new Tomek links are identified, ensuring that the dataset no longer contains any Tomek links.

[0135] Through sub-steps B1 and B2, the embodiment of the present application can improve the clarity of the decision boundary of the classifier, thereby enhancing the generalization ability of the model and the accuracy of minority class recognition. This is crucial to improving the class imbalance problem of the data set and helps to improve the performance of subsequent machine learning models.

[0136] It should be noted that in machine learning, "unclear classification boundaries" is usually a qualitative description, but the embodiments of the present disclosure can use some quantitative methods to evaluate the clarity of classification boundaries. The following criteria may be included:

[0137] Confusion Matrix:

[0138] The confusion matrix is ​​a basic tool for measuring classification performance. It shows the relationship between the actual class and the class predicted by the model. The off-diagonal elements (false positives and false negatives) in the confusion matrix can indicate how unclear the classification boundaries are. The clearer the boundaries, the lower the number of false positives and false negatives.

[0139] Precision and Recall:

[0140] Precision refers to the proportion of samples correctly predicted as positive to samples predicted as positive, while recall refers to the proportion of samples correctly predicted as positive to samples actually positive. Generally, when the boundaries are clear, the values ​​of these two indicators will be higher.

[0141] F1 Score:

[0142] The F1 score is the harmonic mean of precision and recall, which can be used to evaluate the overall performance of a classifier. When the classification boundary is not clear, it is difficult for the model to maintain high precision and high recall at the same time, resulting in a decrease in the F1 score.

[0143] Receiver Operating Characteristic Curve (ROC Curve) and Area Under the Curve (AUC): The ROC curve reflects the relationship between the true positive rate (TPR) and the false positive rate (FPR) of the classifier under different threshold settings, and the AUC measures the area under the ROC curve. For a classifier with clearer boundaries, its ROC curve tends to the upper left corner and the AUC value is close to 1.

[0144] Classification boundary visualization:

[0145] Visualization techniques can be used to visually evaluate classification boundaries. The classification boundaries of some algorithms (such as decision trees) can be directly visualized, while for complex algorithms such as deep learning models, dimensionality reduction techniques (such as t-SNE) can be used to display data points and classification boundaries in two or three-dimensional space.

[0146] Through the above-mentioned quantitative method, the embodiments of the present disclosure can evaluate whether the classification boundaries of the model are clear from different perspectives, and optimize the model performance accordingly. However, the final evaluation of the model usually depends on the needs of the specific application scenario and the trade-offs between different performance indicators. In practice, the selection of appropriate evaluation criteria is determined by the nature of the problem and actual business needs.

[0147] S33. Obtain the characteristics of the cross-network information data after oversampling, and based on the characteristics of the cross-network information data, perform intrusion detection on the process of users accessing the desensitized cross-network information data through a machine learning model.

[0148] In S33, the performance of the machine learning model depends largely on the quality of the input features. Feature acquisition is to extract information that can represent the characteristics of the data from the raw data. Therefore, according to the different types of data and the needs of the model, a suitable feature extraction method can be selected to extract features that help distinguish normal behavior from intrusion behavior.

[0149] Regarding the machine learning model in the embodiment of the present application, in the embodiment of the present application, it is necessary to train a machine learning model (such as a random forest, decision tree, etc.) that can identify potential intrusion behaviors based on data features. The model can classify and predict new requests by learning the characteristic patterns of normal and abnormal data. Specifically, the model can be trained using an oversampled and cleaned data set to ensure that the model has good generalization capabilities for the detection of minority class samples.

[0150] After training, the machine learning model can make judgments and early warnings. That is, after the model is established, new user requests will be converted into a form that the model can understand by extracting the same feature set. The model will judge the request and output whether it is an abnormal behavior. If the model determines that it is an abnormal behavior, the system will trigger an early warning and take corresponding measures, such as restricting user permissions, recording logs, and notifying administrators.

[0151] Exemplarily, S33 includes: screening and evaluating features that contribute significantly to distinguishing normal behavior from abnormal behavior; using these features to construct a training data set to ensure the diversity and representativeness of the data set; selecting a suitable machine learning algorithm according to the actual scenario, for example, a decision tree, random forest, support vector machine (SVM) or neural network, etc.; using the selected algorithm to learn the training data set and establish an intrusion detection model; evaluating and optimizing the model through methods such as cross-validation to ensure the stability and accuracy of the model; deploying the trained model to a real-time monitoring system to detect and warn of intrusion behavior.

[0152] In one embodiment, the characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of the user accessing the desensitized cross-network information data through a machine learning model (decision tree model), including the following steps C1-C4:

[0153] C1. Using the information gain method to select the characteristics of the cross-network information data after oversampling;

[0154] Information gain is a method of evaluating feature importance, based on the concept of entropy, which measures the degree to which data uncertainty is reduced after using the feature for classification.

[0155] Specifically, in C1, the information gain value of each feature for the classification target is calculated. Features with higher information gain are selected because they can provide more information about the classification target.

[0156] C2. constructing a training data set using the selected features of the cross-network information data;

[0157] Building a training dataset involves selecting representative samples and the features selected in the previous step to train the decision tree model.

[0158] The features selected by information gain are used to extract corresponding data from the original data set to form a training data set. Exemplarily, the data set can be divided into a training set, a validation set, etc.

[0159] C3. Construct a decision tree model and train the decision tree model using the training data set;

[0160] Decision tree is a popular classification algorithm that builds a tree structure by learning the relationship between data features and classification results.

[0161] Train the decision tree classifier using the training dataset.

[0162] For each node, the best segmentation feature is selected and the tree is recursively constructed based on the principle of maximizing the purity increment of the segmentation.

[0163] In addition, model parameters can be adjusted through techniques such as cross-validation. Adjust the parameters to find the model with the best evaluation indicators (such as precision, recall, F1 score, etc.).

[0164] C4. Use the trained decision tree model to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0165] The trained decision tree can be used to monitor data streams in real time and classify each data instance to determine whether it is an intrusion behavior.

[0166] Specifically, the real-time data stream features are classified and judged through a decision tree. Based on the judgment result of the decision tree, it is determined whether the output is likely to be an abnormal behavior, and appropriate security responses are made accordingly.

[0167] In another embodiment, the characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of the user accessing the desensitized cross-network information data through a machine learning model (random forest model), including the following steps D1-D5:

[0168] D1. Using the chi-square detection method to select the characteristics of the cross-network information data after oversampling;

[0169] Chi-square test is a statistical method used to test whether the observed categorical data conforms to the expected distribution. It is used in feature selection to evaluate the correlation between a single feature and the target variable.

[0170] Specifically, in D1, for each feature, calculate the chi-square statistic value between it and the target category. The larger this value is, the lower the independence between the feature and the target category is, and the stronger the correlation is. Set the significance level (such as 0.05) and use the chi-square distribution table or calculation program to determine the critical value of the chi-square statistic. Retain features with statistic values ​​higher than the critical value. These features have a strong dependence on the target category and are more likely to be useful predictors. Use the selected features to construct a feature set and use it to train a machine learning model for subsequent intrusion detection.

[0171] The above steps complete the process of feature selection using the chi-square test, which identifies and retains the most informative features for classification and excludes irrelevant features from the feature set, thereby reducing model complexity and improving the model's predictive power and generalization.

[0172] D2. Constructing a training data set using the selected features of the cross-network information data;

[0173] D3. Construct a random forest model and train the random forest model using the training data set;

[0174] It should be noted that decision tree models are prone to overfitting due to excessive tree depth, that is, the model learns the training data too much in detail, but has poor generalization ability. Therefore, a maximum depth limit is set during the training process to control the growth of the tree and prevent an overly complex decision tree structure.

[0175] D4. Use regularization techniques to prevent overfitting of the random forest model and evaluate the performance of the model;

[0176] In addition, the following factors need to be considered:

[0177] 1) Set the minimum number of samples for a node:

[0178] Ensuring that each decision tree node has a sufficient number of samples can effectively reduce the impact of noise on the model and the risk of overfitting. Set the minimum number of samples before the decision node is split, and the minimum number of samples for a leaf node.

[0179] 2) Use K-fold cross validation to adjust parameters:

[0180] K-fold cross validation can evaluate the model's ability to generalize to unknown data and find the optimal model parameters through multiple rounds of training and validation.

[0181] Divide the dataset into K parts, use K-1 parts for training each time, and use the remaining 1 part for verification. After K cycles, average the results.

[0182] 3) Pruning:

[0183] Pruning is a technique that removes unnecessary nodes after the decision tree is fully grown to avoid overfitting.

[0184] Use cost-complexity pruning, or use a validation set to decide where to prune.

[0185] 4) Ensemble learning:

[0186] By combining the prediction results of multiple models, the overall prediction accuracy can be improved and the risk of overfitting can be reduced.

[0187] Implement ensemble learning algorithms such as random forest or gradient boosted decision tree (GBDT) to integrate multiple base models to improve performance.

[0188] D5. Use the trained random forest model to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0189] Embodiment 2

[0190] Figure 2 A schematic diagram of a cross-network information data security desensitization and access control system provided in an embodiment of the present disclosure, the cross-network information data security desensitization and access control system 300 includes:

[0191] The desensitization module 301 is used to obtain cross-network information data and perform desensitization processing on the cross-network information data;

[0192] Verification module 302, used to set user permissions and perform permission verification on user requests based on different user permissions;

[0193] The detection module 303 is used to perform intrusion detection on the process of the user accessing the desensitized cross-network information data based on oversampling and preprocessing SMOTETomek technology and machine learning model after the permission verification is successful.

[0194] The detection module 303 performs intrusion detection on the process of the user accessing the desensitized cross-network information data based on the oversampling and preprocessing SMOTETomek technology and the machine learning model, including:

[0195] Cleaning the desensitized cross-network information data;

[0196] Using SMOTETomek to oversample the cleaned cross-network information data;

[0197] The characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of users accessing the desensitized cross-network information data through a machine learning model.

[0198] The cross-network information data after cleaning is oversampled using SMOTETomek, including:

[0199] Analyzing the desensitized cross-network information data to identify the distribution of normal and abnormal samples in the desensitized cross-network information data, wherein the data set of normal samples is larger than the data set of abnormal samples;

[0200] Using SMOTE to oversample the data set of the abnormal sample to synthesize a second abnormal sample, where the second abnormal sample is a new abnormal sample generated after oversampling;

[0201] Tomek links are used to identify and remove pairs of samples to strengthen the decision boundary of the classifier.

[0202] In one embodiment, oversampling the dataset of the abnormal samples using SMOTE includes:

[0203] Finding k nearest neighbors of the abnormal sample in the feature space, where k is a preset number of neighbors;

[0204] For each abnormal sample and its nearest neighbor, a new sample point is generated between the two using a linear interpolation method, wherein the set of the new sample points is the second abnormal sample;

[0205] The second abnormal sample is added to the desensitized cross-network information data.

[0206] In addition, sample pairs were identified and removed via Tomek links, including:

[0207] Obtaining oversampled abnormal samples, and identifying all nearest neighbor sample pairs Tomeklinks in the abnormal samples;

[0208] Remove pairs of samples that are identified as the majority class in Tomek links.

[0209] Specifically, in one example, the characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of the user accessing the desensitized cross-network information data through a machine learning model, including:

[0210] Using an information gain method to select features of the oversampled cross-network information data;

[0211] Using the selected features of the cross-network information data to construct a training data set;

[0212] Constructing a decision tree model, and training the decision tree model using the training data set;

[0213] The trained decision tree model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0214] In another example, obtaining features of the oversampled cross-network information data, and performing intrusion detection on a process in which a user accesses the desensitized cross-network information data through a machine learning model based on the features of the cross-network information data, includes:

[0215] Using a chi-square test method to select features of the cross-network information data after oversampling;

[0216] Using the selected features of the cross-network information data to construct a training data set;

[0217] Constructing a random forest model, and training the random forest model using the training data set;

[0218] Using regularization techniques to prevent overfitting of the random forest model and evaluate the performance of the model;

[0219] The trained random forest model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

[0220] Optionally, obtaining cross-network information data and performing desensitization processing on the cross-network information data includes:

[0221] Classify and process the cross-network information data to identify sensitive information that needs to be desensitized;

[0222] Determine the corresponding desensitization strategy based on the sensitivity level of different cross-network information data;

[0223] The sensitive information is desensitized according to the corresponding desensitization strategy.

[0224] Optionally, based on different user permissions, perform permission verification on user requests, including:

[0225] Use the access control list (ACL) mechanism to verify user access requests.

[0226] This application provides a cross-network information data security desensitization and access control method and system, which uses SMOTETomek technology combined with machine learning algorithms to improve the class imbalance problem of data sets, enhance the generalization ability of intrusion detection models, and ensure the security of information data through desensitization mechanisms and fine-grained access control. It uses the ability of machine learning to identify and respond to unknown threats in real time, and uses more scientific data desensitization technology to ensure the security of data throughout its life cycle, becoming a key measure to improve network data security.

[0227] The system of the embodiments of the present disclosure can execute the method provided by the embodiments of the present disclosure, and the implementation principles are similar. The actions performed by each module in the system of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For the detailed functional description of each module of the system, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0228] The above is only an optional implementation method for some implementation scenarios of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present disclosure, other similar implementation methods based on the technical ideas of the present disclosure are also within the protection scope of the embodiments of the present disclosure.

Claims

1. A method for cross-network information data security desensitization and access control, characterized in that: The method comprises: Obtaining cross-network information data, and performing desensitization processing on the cross-network information data; Set user permissions and verify user requests based on different user permissions; After the permission verification is successful, intrusion detection is performed on the user's access to the desensitized cross-network information data process based on oversampling and preprocessing SMOTETomek technology and machine learning models.

2. The method according to claim 1, characterized in that Based on the oversampling and preprocessing SMOTETomek technology and machine learning model, intrusion detection is performed on the process of users accessing the desensitized cross-network information data, including: Cleaning the desensitized cross-network information data; Using SMOTETomek to oversample the cleaned cross-network information data; The characteristics of the cross-network information data after oversampling are obtained, and based on the characteristics of the cross-network information data, intrusion detection is performed on the process of users accessing the desensitized cross-network information data through a machine learning model.

3. The method according to claim 2, characterized in that SMOTETomek is used to oversample the cleaned cross-network information data, including: Analyzing the desensitized cross-network information data to identify the distribution of normal and abnormal samples in the desensitized cross-network information data, wherein the data set of normal samples is larger than the data set of abnormal samples; Using SMOTE to oversample the data set of the abnormal sample to synthesize a second abnormal sample, where the second abnormal sample is a new abnormal sample generated after oversampling; Tomek links are used to identify and remove pairs of samples to strengthen the decision boundary of the classifier.

4. The method according to claim 3, characterized in that: SMOTE is used to oversample the dataset of the abnormal samples, including: Finding k nearest neighbors of the abnormal sample in the feature space, where k is a preset number of neighbors; For each abnormal sample and its nearest neighbor, a new sample point is generated between the two using a linear interpolation method, wherein the set of the new sample points is the second abnormal sample; The second abnormal sample is added to the desensitized cross-network information data.

5. The method according to claim 3, characterized in that: Identify and remove sample pairs through Tomek links, including: Obtaining oversampled abnormal samples, and identifying all nearest neighbor sample pairs Tomeklinks in the abnormal samples; Remove pairs of samples that are identified as the majority class in Tomek links.

6. The method according to any one of claims 2 to 5, characterized in that: Acquiring the features of the cross-network information data after oversampling, and performing intrusion detection on the process of a user accessing the desensitized cross-network information data through a machine learning model based on the features of the cross-network information data, including: Using an information gain method to select features of the oversampled cross-network information data; Using the selected features of the cross-network information data to construct a training data set; Constructing a decision tree model, and training the decision tree model using the training data set; The trained decision tree model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

7. The method according to any one of claims 2 to 5, characterized in that: Acquiring the features of the cross-network information data after oversampling, and performing intrusion detection on the process of a user accessing the desensitized cross-network information data through a machine learning model based on the features of the cross-network information data, including: Using a chi-square test method to select features of the cross-network information data after oversampling; Using the selected features of the cross-network information data to construct a training data set; Constructing a random forest model, and training the random forest model using the training data set; Using regularization techniques to prevent overfitting of the random forest model and evaluate the performance of the model; The trained random forest model is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data.

8. The method according to claim 1, characterized in that Obtaining cross-network information data and performing desensitization processing on the cross-network information data includes: Classify and process the cross-network information data to identify sensitive information that needs to be desensitized; Determine the corresponding desensitization strategy based on the sensitivity level of different cross-network information data; The sensitive information is desensitized according to the corresponding desensitization strategy.

9. The method according to claim 1, characterized in that: Based on different user permissions, user requests are verified, including: Use the access control list (ACL) mechanism to verify user access requests.

10. A cross-network information data security desensitization and access control system, characterized in that: include: A desensitizing module, used to obtain cross-network information data and perform desensitization processing on the cross-network information data; The verification module is used to set user permissions and verify user requests based on different user permissions; The detection module is used to perform intrusion detection on the process of users accessing the desensitized cross-network information data based on oversampling and preprocessing SMOTETomek technology and machine learning models after the permission verification is successful.

Citation Information

Patent Citations

  • Data desensitization method based on authority

    CN108418676A

  • Unified authorized access method and system for multiple data sources

    CN110909386A

  • Automatic desensitization data acquisition and distribution system and method

    CN111460512A

  • Invasion detecting system and method, and recording medium

    JP2005130399A

Cited By

  • Large model gateway security protection system and security protection method

    CN120951390A