A data asset security dynamic monitoring method and system based on machine learning

Abstract

The present invention belongs to the field of data asset security technology, and discloses a data asset security dynamic monitoring method and system based on machine learning. The present invention extracts user behavior pattern characteristics, data access characteristics and network traffic characteristics of data assets, and then respectively calculates the user behavior pattern security factor, data access security factor and network traffic security factor of the data assets in the current cycle, and then constructs an asset security monitoring model based on machine learning, and obtains the threshold of each security factor according to the asset security monitoring model. Finally, based on the comparison of each security factor in the current cycle with the corresponding threshold, it is evaluated whether there are security risks in the data assets in the current cycle, which can realize real-time and dynamic monitoring of the security status of data assets, overcome the lag and limitations of traditional static security protection measures, and greatly improve the timeliness and effectiveness of data asset security protection.

Description

Technical Field

[0001] The present invention belongs to the technical field of data asset security, and specifically relates to a data asset security dynamic monitoring method and system based on machine learning. Background Art

[0002] With the rapid development of information technology, enterprises and organizations have accumulated a large amount of data assets, which are of vital value to their business operations and decision-making. However, data assets face many security threats, such as network attacks, data leakage, and illegal operations by internal personnel. Traditional static security protection measures can no longer meet the increasingly complex and changing data security needs. Therefore, a method that can monitor the security status of data assets in real time and dynamically is needed to timely discover potential security risks and take corresponding countermeasures. Summary of the invention

[0003] The purpose of the present invention is to provide a data asset security dynamic monitoring method and system based on machine learning to solve the problems faced in the above-mentioned background technology.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] A data asset security dynamic monitoring method based on machine learning, the method comprising the following steps:

[0006] Step S1: Deploy collection points in the generation, storage, transmission and use of data assets, collect relevant information and perform preprocessing;

[0007] Step S2, extracting user behavior pattern characteristics, data access characteristics and network traffic characteristics of data assets from the preprocessed data;

[0008] Step S3: Analyze the newly collected data that has been preprocessed and feature extracted to obtain the user behavior pattern security factor, data access security factor, and network traffic security factor of the data asset;

[0009] Step S4: construct an asset security monitoring model based on machine learning, train the model through historical data, and optimize the performance of the model so that it can accurately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of data assets;

[0010] Step S5: Compare the user behavior pattern security factor, data access security factor and network traffic security factor of the data asset with the corresponding standard values ​​to determine whether the data asset is in a safe state.

[0011] As a further description of the solution of the present invention, the user behavior pattern characteristics include login duration and operation frequency; the data access characteristics include access source IP, access duration and access frequency; and the network traffic characteristics include traffic size.

[0012] As a further description of the solution of the present invention, the specific working process of step S3 includes:

[0013] Collect user behavior parameters at regular intervals to obtain the user's login duration and operation frequency in the current cycle, and then build a mathematical calculation model for the safety factor of the user behavior pattern. The expression is:

[0014] ;

[0015] Where n is the total number of cycles, The user's login time in the current cycle. is the user's login duration in the i-th cycle, is the maximum login duration of users in all cycles. The minimum login duration of the user in all cycles; The user login frequency in the current period. is the user login frequency in the i-th period, is the maximum login frequency of users in all cycles, is the minimum login frequency of users in all cycles; where i belongs to n, is the safety factor of the user behavior pattern in the current cycle; and They are the weight coefficients corresponding to login duration and login frequency respectively.

[0016] As a further description of the solution of the present invention, the specific working process of step S3 also includes:

[0017] Collect data access parameters at regular intervals, obtain the number of access source IPs in the current cycle, obtain the access duration of each access source IP in the current cycle, obtain the access frequency of each access source IP in the current cycle, and then build a mathematical calculation model for the data access security factor. The expression is:

[0018] ;

[0019] Where n is the total number of cycles, is the access duration of the jth IP in the current cycle, is the access duration of the jth IP in the i-th period, is the maximum access duration of the jth IP in all cycles, is the minimum access duration of the jth IP in all cycles; is the access frequency of the jth IP in the current period, is the access frequency of the jth IP in the i-th period, is the maximum access frequency of the jth IP in all cycles, is the minimum access frequency of the jth IP in all cycles; where i belongs to n, is the data access security factor of the jth IP in the current period; and are the weight coefficients corresponding to the access duration and access frequency respectively; m is the number of access source IPs in the current period, It is the data access safety factor of the current cycle.

[0020] As a further description of the solution of the present invention, the specific working process of step S3 also includes:

[0021] Collect network traffic parameters at regular intervals to obtain the amount of data transmitted during the peak period of the current cycle, and the amount of data transmitted during the valley period of the current cycle. Then, construct a mathematical calculation model for the network traffic safety factor. The expression is:

[0022] ;

[0023] Where n is the total number of cycles, is the amount of data transmitted during the peak period of the current cycle, is the amount of data transmitted during the peak time period in the i-th cycle, is the maximum amount of data transmitted during the peak time period in all cycles, The minimum amount of data transmitted during the peak time period in all cycles; is the amount of data transmitted during the low period of the current cycle. is the amount of data transmitted during the low period of the i-th cycle, is the maximum amount of data transmitted during the low period of all cycles, is the minimum amount of data transmitted during the low period of all cycles; where i belongs to n, is the network traffic safety factor of the current period; and are the weight coefficients corresponding to the peak time period and the trough time period respectively.

[0024] As a further description of the solution of the present invention, the working process of step S4 includes:

[0025] The historical data after feature selection is divided into a training set and a validation set in a ratio of 80:20. The training set is used for model training, and the validation set is used for model parameter tuning, so as to ultimately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of the data assets.

[0026] As a further description of the solution of the present invention, the working process of step S5 includes:

[0027] The user behavior pattern security factor, data access security factor and network traffic security factor of the current period are compared with the corresponding thresholds respectively. If any of the security factors is greater than or equal to the corresponding threshold, it means that there are security risks in the data assets in the current period. If none of the security factors is greater than or equal to the corresponding threshold, the probability of security risks in the data assets in the current period is further judged.

[0028] As a further description of the solution of the present invention, the specific process of further determining the probability that the data assets in the current period have security risks includes:

[0029] The current cycle data asset safety volatility coefficient is calculated using the following formula:

[0030] ;

[0031] In the formula, , and are the thresholds corresponding to the user behavior pattern security factor, data access security factor, and network traffic security factor, respectively. , and are the weight coefficients corresponding to the user behavior pattern security factor, data access security factor and network traffic security factor, respectively. It is the safety volatility coefficient of data assets in the current cycle;

[0032] Similarly, calculate the security volatility coefficient of data assets in the previous cycle , and then by the formula Calculate the probability that data assets in the current cycle have security risks.

[0033] A data asset security dynamic monitoring system based on machine learning, the system comprising:

[0034] Data collection module, used to collect dynamic parameter information of data assets in real time;

[0035] The data processing module is used to ensure the quality and consistency of data through data cleaning, denoising, format conversion and standardization operations for subsequent analysis and processing;

[0036] A feature extraction module is used to extract features related to data asset security from preprocessed data;

[0037] The security assessment module is used to assess the security status of data assets in the current cycle based on data asset security-related features.

[0038] Beneficial effects of the present invention:

[0039] The present invention collects data in real time at regular intervals, and after preprocessing the collected data, extracts user behavior pattern characteristics, data access characteristics and network traffic characteristics of the data assets, and then respectively calculates the user behavior pattern safety factor, data access safety factor and network traffic safety factor of the data assets in the current period based on the parameter information corresponding to the user behavior pattern characteristics, data access characteristics and network traffic characteristics of the data assets, and then constructs an asset security monitoring model based on machine learning, and obtains the thresholds of the user behavior pattern safety factor, data access safety factor and network traffic safety factor of the data assets according to the asset security monitoring model; finally, by comparing the user behavior pattern safety factor, data access safety factor and network traffic safety factor in the current period with the corresponding thresholds, it is evaluated whether there are security risks in the data assets in the current period, and can realize real-time and dynamic monitoring of the security status of data assets, overcome the lag and limitations of traditional static security protection measures, and greatly improve the timeliness and effectiveness of data asset security protection.

[0040] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0042] Figure 1 This is a partial flow chart of the method for dynamic monitoring of data asset security based on machine learning of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 As shown, a data asset security dynamic monitoring method based on machine learning is disclosed, and the method includes the following steps:

[0045] Step S1: Deploy collection points in the generation, storage, transmission and use of data assets, collect relevant information and perform preprocessing;

[0046] Step S2, extracting user behavior pattern characteristics, data access characteristics and network traffic characteristics of data assets from the preprocessed data;

[0047] Step S3: Analyze the newly collected data after preprocessing and feature extraction to obtain the user behavior pattern security factor, data access security factor and network traffic security factor of the data asset;

[0048] Step S4: construct an asset security monitoring model based on machine learning, train the model through historical data, and optimize the performance of the model so that it can accurately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of data assets;

[0049] Step S5: Compare the user behavior pattern security factor, data access security factor and network traffic security factor of the data asset with the corresponding standard values ​​to determine whether the data asset is in a safe state.

[0050] Through the above technical scheme, the present invention collects data in real time at regular intervals, and after preprocessing the collected data, extracts the user behavior pattern characteristics, data access characteristics and network traffic characteristics of the data assets, and then respectively calculates the user behavior pattern security factor, data access security factor and network traffic security factor of the data assets in the current period based on the parameter information corresponding to the user behavior pattern characteristics, data access characteristics and network traffic characteristics of the data assets, and then constructs an asset security monitoring model based on machine learning, and obtains the thresholds of the user behavior pattern security factor, data access security factor and network traffic security factor of the data assets according to the asset security monitoring model; finally, by comparing the user behavior pattern security factor, data access security factor and network traffic security factor in the current period with the corresponding thresholds, it is evaluated whether there are security risks in the data assets in the current period, and can realize real-time and dynamic monitoring of the security status of data assets, overcome the lag and limitations of traditional static security protection measures, and greatly improve the timeliness and effectiveness of data asset security protection.

[0051] As a further description of the solution of the present invention, the user behavior pattern characteristics include login duration and operation frequency; the data access characteristics include access source IP, access duration and access frequency; and the network traffic characteristics include traffic size.

[0052] As a further description of the solution of the present invention, the specific working process of step S3 includes:

[0053] Collect user behavior parameters at regular intervals to obtain the user's login duration and operation frequency in the current cycle, and then build a mathematical calculation model for the safety factor of the user behavior pattern. The expression is:

[0054] ;

[0055] Where n is the total number of cycles, The user's login time in the current cycle. is the user's login duration in the i-th cycle, is the maximum login duration of users in all cycles. The minimum login duration of the user in all cycles; The user login frequency in the current period. is the user login frequency in the i-th period, is the maximum login frequency of users in all cycles, is the minimum login frequency of users in all cycles; where i belongs to n, is the safety factor of the user behavior pattern in the current cycle; and They are the weight coefficients corresponding to login duration and login frequency respectively.

[0056] Through the above technical solution, this embodiment provides a method for calculating the safety factor of user behavior pattern, firstly obtaining the user login duration and frequency in the current cycle, as well as the user's maximum login duration, user's minimum login duration, user's maximum login frequency and user's minimum login frequency in all cycles, and then combining the user's average login duration of all cycles and the average login frequency of all cycles, through the formula Calculate the safety factor of the user behavior pattern in the current cycle, where: and They respectively reflect the fluctuations in user login duration and frequency during all periods.

[0057] As a further description of the solution of the present invention, the specific working process of step S3 also includes:

[0058] Collect data access parameters at regular intervals, obtain the number of access source IPs in the current cycle, obtain the access duration of each access source IP in the current cycle, obtain the access frequency of each access source IP in the current cycle, and then build a mathematical calculation model for the data access security factor. The expression is:

[0059] ;

[0060] Where n is the total number of cycles, is the access duration of the jth IP in the current cycle, is the access duration of the jth IP in the i-th period, is the maximum access duration of the jth IP in all cycles, is the minimum access duration of the jth IP in all cycles; is the access frequency of the jth IP in the current period, is the access frequency of the jth IP in the i-th period, is the maximum access frequency of the jth IP in all cycles, is the minimum access frequency of the jth IP in all cycles; where i belongs to n, is the data access security factor of the jth IP in the current period; and are the weight coefficients corresponding to the access duration and access frequency respectively; m is the number of access source IPs in the current cycle, It is the data access safety factor of the current cycle.

[0061] Through the above technical solution, this embodiment provides a method for calculating the data access security factor. First, the access duration and frequency of the j-th IP in the current cycle, as well as the maximum login duration of the j-th IP, the minimum login duration of the j-th IP, the maximum login frequency of the j-th IP and the minimum login frequency of the j-th IP in all cycles are obtained. Then, the average login duration of the j-th IP in all cycles and the average login frequency of all cycles are combined to calculate the data access security factor through the formula: Calculate the j-th IP data access security factor in the current cycle, where: and They respectively reflect the fluctuation of the access duration and frequency of the jth IP in all cycles. Finally, through the formula The data access safety factor of the current cycle is calculated.

[0062] As a further description of the solution of the present invention, the specific working process of step S3 also includes:

[0063] Collect network traffic parameters at regular intervals to obtain the amount of data transmitted during the peak period of the current cycle, and the amount of data transmitted during the valley period of the current cycle. Then, construct a mathematical calculation model for the network traffic safety factor. The expression is:

[0064] ;

[0065] Where n is the total number of cycles, is the amount of data transmitted during the peak period of the current cycle, is the amount of data transmitted during the peak time period in the i-th cycle, is the maximum amount of data transmitted during the peak time period in all cycles, The minimum amount of data transmitted during the peak time period in all cycles; is the amount of data transmitted during the low period of the current cycle. is the amount of data transmitted during the low period of the i-th cycle, is the maximum amount of data transmitted during the low period of all cycles, is the minimum amount of data transmitted during the low period of all cycles; where i belongs to n, is the network traffic safety factor of the current period; and are the weight coefficients corresponding to the peak time period and the trough time period respectively.

[0066] Through the above technical solution, this embodiment provides a method for calculating the network traffic safety factor. First, the amount of data transmitted in the peak time period and the valley time period in the current cycle, as well as the maximum amount of data transmitted and the minimum amount of data transmitted in the peak time period and the valley time period in all cycles are obtained. Then, the amount of data transmitted in the peak time period and the valley time period of all cycles is combined, and the formula is used to calculate the network traffic safety factor. Calculate the network traffic safety factor of the current cycle, where: and It reflects the fluctuation of the amount of data transmitted during peak time and trough time respectively.

[0067] As a further description of the solution of the present invention, the working process of step S4 includes:

[0068] The historical data after feature selection is divided into a training set and a validation set in a ratio of 80:20. The training set is used for model training, and the validation set is used for model parameter tuning, so as to ultimately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of the data assets.

[0069] As a further description of the solution of the present invention, the working process of step S5 includes:

[0070] The user behavior pattern security factor, data access security factor and network traffic security factor of the current period are compared with the corresponding thresholds respectively. If any of the security factors is greater than or equal to the corresponding threshold, it means that there are security risks in the data assets in the current period. If none of the security factors is greater than or equal to the corresponding threshold, the probability of security risks in the data assets in the current period is further judged.

[0071] As a further description of the solution of the present invention, the specific process of further determining the probability that the data assets in the current period have security risks includes:

[0072] The current cycle data asset safety volatility coefficient is calculated using the following formula:

[0073] ;

[0074] In the formula, , and are the thresholds corresponding to the user behavior pattern security factor, data access security factor, and network traffic security factor, respectively. , and are the weight coefficients corresponding to the user behavior pattern security factor, data access security factor and network traffic security factor, respectively. It is the safety volatility coefficient of data assets in the current cycle;

[0075] Similarly, calculate the security volatility coefficient of data assets in the previous cycle , and then by the formula Calculate the probability that data assets in the current cycle have security risks.

[0076] Through the above technical solution, this implementation provides a data asset security assessment method, which compares the user behavior pattern security factor, data access security factor and network traffic security factor of the current period with the corresponding thresholds. If any of the security factors is greater than or equal to the corresponding threshold, it means that there are security risks in the data assets in the current period. If no security factor is greater than or equal to the corresponding threshold, the data asset security volatility coefficient of the current period and the data asset security volatility coefficient of the previous period are calculated respectively, and then the formula is used to evaluate the data asset security volatility coefficient. Calculate the probability that data assets in the current cycle have security risks.

[0077] A data asset security dynamic monitoring system based on machine learning, the system comprising:

[0078] Data collection module, used to collect dynamic parameter information of data assets in real time;

[0079] The data processing module is used to ensure the quality and consistency of data through data cleaning, denoising, format conversion and standardization operations for subsequent analysis and processing;

[0080] A feature extraction module is used to extract features related to data asset security from preprocessed data;

[0081] The security assessment module is used to assess the security status of data assets in the current cycle based on data asset security-related features.

[0082] The above contents are merely examples and explanations of the concept of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A data asset security dynamic monitoring method based on machine learning, characterized in that: The method comprises the following steps: Step S1: Deploy collection points in the generation, storage, transmission and use of data assets, collect relevant information and perform preprocessing; Step S2, extracting user behavior pattern characteristics, data access characteristics and network traffic characteristics of data assets from the preprocessed data; Step S3: Analyze the newly collected data after preprocessing and feature extraction to obtain the user behavior pattern security factor, data access security factor and network traffic security factor of the data asset; Step S4: construct an asset security monitoring model based on machine learning, train the model through historical data, and optimize the performance of the model so that it can accurately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of data assets; Step S5: Compare the user behavior pattern security factor, data access security factor, and network traffic security factor of the data asset with the corresponding standard values ​​to determine whether the data asset is in a safe state; The user behavior pattern characteristics include login duration and operation frequency; The data access characteristics include access source IP, access duration and access frequency; The network traffic characteristics include traffic size; The specific working process of step S3 includes: Collect data access parameters at regular intervals, obtain the number of access source IPs in the current cycle, obtain the access duration of each access source IP in the current cycle, obtain the access frequency of each access source IP in the current cycle, and then build a mathematical calculation model for the data access security factor. The expression is: ； Where n is the total number of cycles, is the access duration of the jth IP in the current cycle, is the access duration of the jth IP in the i-th period, is the maximum access duration of the jth IP in all cycles, is the minimum access duration of the jth IP in all cycles; is the access frequency of the jth IP in the current period, is the access frequency of the jth IP in the i-th period, is the maximum access frequency of the jth IP in all cycles, is the minimum access frequency of the jth IP in all cycles; where i belongs to n, is the data access security factor of the jth IP in the current period; and are the weight coefficients corresponding to the access duration and access frequency respectively; m is the number of access source IPs in the current cycle, is the data access safety factor for the current cycle; The specific working process of step S3 also includes: Collect network traffic parameters at regular intervals to obtain the amount of data transmitted during the peak period of the current cycle, and the amount of data transmitted during the valley period of the current cycle. Then, construct a mathematical calculation model for the network traffic safety factor. The expression is: ； Where n is the total number of cycles, is the amount of data transmitted during the peak period of the current cycle, is the amount of data transmitted during the peak time period in the i-th cycle, is the maximum amount of data transmitted during the peak time period in all cycles, The minimum amount of data transmitted during the peak time period in all cycles; is the amount of data transmitted during the low period of the current cycle. is the amount of data transmitted during the low period of the i-th cycle, is the maximum amount of data transmitted during the low period of all cycles, is the minimum amount of data transmitted during the low period of all cycles; where i belongs to n, is the network traffic safety factor of the current period; and are the weight coefficients corresponding to the peak time period and the trough time period respectively.

2. According to the method of dynamic data asset security monitoring based on machine learning in claim 1, it is characterized in that: The specific working process of step S3 also includes: Collect user behavior parameters at regular intervals to obtain the user's login duration and operation frequency in the current cycle, and then build a mathematical calculation model for the safety factor of the user behavior pattern. The expression is: ； Where n is the total number of cycles, The user's login time in the current cycle. is the user's login duration in the i-th cycle, is the maximum login duration of users in all cycles. The minimum login duration of the user in all cycles; The user login frequency in the current period. is the user login frequency in the i-th period, is the maximum login frequency of users in all cycles, is the minimum login frequency of users in all cycles; where i belongs to n, is the safety factor of the user behavior pattern in the current cycle; and They are the weight coefficients corresponding to login duration and login frequency respectively.

3. According to claim 2, a data asset security dynamic monitoring method based on machine learning is characterized in that: The working process of step S4 includes: The historical data after feature selection is divided into a training set and a validation set in a ratio of 80:

20. The training set is used for model training, and the validation set is used for model parameter tuning, so as to ultimately determine the thresholds of the user behavior pattern security factor, data access security factor, and network traffic security factor of the data assets.

4. According to the method of dynamic data asset security monitoring based on machine learning in claim 3, it is characterized in that: The working process of step S5 includes: The user behavior pattern security factor, data access security factor and network traffic security factor of the current period are compared with the corresponding thresholds respectively. If any of the security factors is greater than or equal to the corresponding threshold, it means that there are security risks in the data assets in the current period. If none of the security factors is greater than or equal to the corresponding threshold, the probability of security risks in the data assets in the current period is further judged.

5. According to claim 4, a data asset security dynamic monitoring method based on machine learning is characterized in that: The specific process of further determining the probability of security risks in the current cycle data assets includes: The current cycle data asset safety volatility coefficient is calculated using the following formula: ； In the formula, , and are the thresholds corresponding to the user behavior pattern security factor, data access security factor, and network traffic security factor, respectively. , and are the weight coefficients corresponding to the user behavior pattern security factor, data access security factor and network traffic security factor, respectively. It is the safety volatility coefficient of data assets in the current cycle; Similarly, calculate the security volatility coefficient of data assets in the previous cycle , and then by the formula Calculate the probability that data assets in the current cycle have security risks.

Images