Data security management method and system applied to big data management platform

Abstract

The present invention discloses a data security management method and system applied to a big data management platform, which relates to the field of data security management, including: obtaining a real-time dynamic data set of big data, dividing the data set and adding it to an idle data processing queue; establishing a data threat identification model, detecting abnormal data risk threats of the platform in real time, and taking automated response measures based on the threat type; using differential privacy technology to query the data that has been stored in the database, and adjusting the amount of noise during the query in real time according to the data risk threat situation; performing sensitive data screening and cache backup on dynamic data based on the abnormal data risk threat situation; detecting the current processing speed and the accumulation of tasks to be processed, and automatically adjusting resource allocation in combination with the abnormal data risk threat.

Description

Technical Field

[0001] The present invention relates to the field of data security management, and in particular to a data security management method and system applied to a big data management platform. Background Art

[0002] With the rapid development of big data technology, data security issues are becoming more and more important. Traditional data security management methods are difficult to cope with complex security threats in big data environments, such as data leakage, data tampering, and risky access. Existing technologies mainly rely on encryption and access control, but the effectiveness and adaptability of these methods are challenged in big data environments. In addition, traditional data security management methods usually only target or respond to data security issues in a certain link of the data processing process, lacking data security management for the entire data processing process.

[0003] Therefore, how to implement a data security management method that comprehensively considers the entire data processing life cycle, including data processing distribution, data threat identification, database security query, automatic backup of sensitive data, and adaptive adjustment of resource allocation, is an urgent problem that needs to be solved. Summary of the invention

[0004] In order to solve the above technical problems, a data security management method applied to a big data management platform is provided. This technical solution solves the problems raised in the above background technology.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] Data security management methods applied to big data management platforms include:

[0007] Obtain a real-time dynamic data set of big data, split the data set and add it to the idle data processing queue;

[0008] Establish a data threat identification model to detect abnormal data risk threats on the platform in real time and take automated response measures based on the threat type;

[0009] Differential privacy technology is used to query stored data, and the amount of noise during query is adjusted in real time according to the data risk threat situation;

[0010] Perform sensitive data screening and cache backup of dynamic data based on abnormal data risk threats;

[0011] Detect the current processing speed and the backlog of pending tasks, and automatically adjust resource allocation based on abnormal data risk threats.

[0012] Preferably, the step of obtaining a big data real-time dynamic data set and dividing the data set and adding it to an idle data processing queue specifically includes:

[0013] Get the dynamic data set within the set time window, set the data processing segmentation window size based on the available resources remaining on the server, and divide the dynamic data set according to the segmentation window size;

[0014] Obtain the load conditions of all data processing threads and the number of tasks accumulated in the corresponding data processing task queues, calculate the idle evaluation value of the data processing thread by combining the ratio of the number of tasks accumulated for each processing thread to the thread processing rate and the set initial load weight of each thread;

[0015] After the idle evaluation values ​​of all processing threads are sorted in reverse order, the segmented dynamic data are selected and added to the task waiting queue of the data processing thread in sequence according to the sorting results.

[0016] Preferably, the establishment of a data threat identification model to detect abnormal data risk threats of the platform in real time and taking automated response measures based on threat types specifically include:

[0017] Obtain historical processing data from the system platform as training data, classify the training data based on the set data classification, and batch-label the data after classification;

[0018] Preprocess and extract features of batch labeled data, establish data classification feature vectors, select convolutional neural networks, train data classification and recognition models based on feature-extracted training data, and deploy the trained models into the system;

[0019] Set up data classification threat identification division of labor and establish corresponding threat identification models for each set data classification, specifically:

[0020] Based on the historical risk screening results of the current data classification, set the risk threat classification corresponding to the data classification;

[0021] Retrieve the data specifically involved in the risk screening results in the training data corresponding to the current data classification, mark it as risk source data, and attribute and label all risk source data based on the set threat classification;

[0022] Preprocess and extract features of risk source data, establish risk data threat classification feature vectors, select convolutional neural networks, and train a dedicated risk threat identification model for current data classification based on the risk source data after feature extraction;

[0023] Combine and deploy the dedicated risk threat identification models corresponding to all data classifications into the system;

[0024] Set corresponding response strategies based on historical risk threat handling methods and establish risk threat-response strategy associations;

[0025] Get the data processing thread, identify the data type of the real-time data being processed in the data processing thread through the data classification recognition model, select the corresponding dedicated risk threat identification model for risk threat identification based on the data type identification result, and perform automated response strategy selection and data risk threat processing based on the risk threat identification result and the risk threat-response strategy association.

[0026] Preferably, the query of the stored data adopts differential privacy technology, and according to the data risk threat situation, the noise amount during the query is adjusted in real time, specifically including:

[0027] Get all the table structures of the database, select the privacy statistical fields, and use the Sigmoid function to adjust the risk of the set privacy budget in real time based on the recognition hit rate of the risk threat recognition model in the current set segmentation window. The specific expression is:

[0028] ;

[0029] In the formula, , They are the set privacy budget and the adjusted privacy budget, Identify hit rates for risk threat identification models;

[0030] Set the statistical field update monitoring window time, obtain the maximum batch size of the most recent time window for inserting the privacy statistical field, and combine it with the adjusted privacy budget , calculate the noise scale parameter;

[0031] Select a random value that obeys the Laplace distribution, and generate Laplace noise according to the noise scale and the selected random value. The specific expression is:

[0032] ;

[0033] In the formula, is the generated Laplace noise, is the sign of the random value, is a random value, Insert batch size for setting privacy statistics field, is the noise scale parameter;

[0034] The generated noise is applied to the query process of the privacy statistical field to increase the privacy of statistical data queries.

[0035] Preferably, the sensitive data screening and cache backup of dynamic data based on abnormal data risk threat conditions specifically includes:

[0036] Obtain the data type identification result of the real-time data being processed by the data processing thread, set the sensitive data classification and sensitive operation keywords, retrieve the data matching the sensitive operation keywords in the sensitive data classification, mark the entire data corresponding to the keyword as sensitive data, and calculate the proportion of sensitive data in the total processed data;

[0037] Obtain the recognition hit rate of the risk threat recognition model within the currently set segment window, perform weighted summation on the sensitive data proportion and the recognition hit rate, and calculate the sensitive data risk assessment value;

[0038] Set a sensitive data risk assessment value threshold. When the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, store all marked sensitive data in the cache backup;

[0039] If the processing thread does not experience an abnormal interruption during the current task processing, the cache backup will be cleared after the task processing is completed. If an abnormal interruption occurs, the breakpoint will be recorded and the cache backup data will be restored and reprocessed.

[0040] Preferably, the detecting of the current processing speed and the accumulation of tasks to be processed and automatically adjusting resource allocation in combination with the risk threat of abnormal data specifically includes:

[0041] Detect the processing speed of all current data processing threads and the backlog of pending tasks, and recalculate the idle evaluation values ​​of all current data processing threads;

[0042] Obtain the sensitive data risk assessment values ​​of all current data processing threads, use the ratio of the idle assessment value to the sensitive data risk assessment value as the resource allocation tendency coefficient, calculate the average value of the resource allocation tendency coefficients of all data processing threads, and mark it as the thread processing resource allocation reference line;

[0043] Based on the deviation of the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line, the computing resources and storage resources of each processing thread are adjusted in proportion.

[0044] Furthermore, a data security management system applied to a big data management platform is proposed, which is used to implement the data security management method applied to a big data management platform as described above, including:

[0045] A data acquisition and processing distribution module, which acquires a dynamic data set within a set time window, divides the dynamic data set into segmented window sizes based on the remaining available resources of the server, calculates the idle evaluation value of the data processing thread, and sequentially adds the dynamic data set to the task waiting queue of the data processing thread according to the calculation results;

[0046] A data type and risk threat identification module, which uses the historical processing data of the system platform as training data, trains a data classification identification model, and based on the historical risk screening processing results of data classification, trains a dedicated risk threat identification model for data classification, and simultaneously establishes a risk threat-response strategy association relationship, identifies the type and risk threat of the currently processed data, and performs automated response strategy selection and data risk threat processing;

[0047] A database query security module, which performs real-time risk adjustment on a set privacy budget according to the recognition hit rate of the risk threat recognition model in the currently set segment window, calculates a noise scale parameter based on the insertion batch size of the privacy statistics field and the adjusted privacy budget, and then selects a random value, generates Laplace noise according to the noise scale and the selected random value, and applies the generated noise to the query process of the privacy statistics field;

[0048] A sensitive data automatic backup module, which calculates the sensitive data risk assessment value according to the weighted sum of the proportion of sensitive data to the total processed data and the recognition hit rate of the risk threat recognition model in the currently set segment window, and when the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, stores all marked sensitive data in the cache backup;

[0049] A system resource allocation adaptive adjustment module, which uses the average value of the ratio of the idle assessment value to the sensitive data risk assessment value of all data processing threads as the thread processing resource allocation reference line, and proportionally adjusts the computing resources and storage of each processing thread based on the deviation between the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] Acquire real-time dynamic data sets of big data, split the data sets and add them to the idle data processing queue; establish a data threat identification model to detect abnormal data risk threats on the platform in real time, and take automated response measures based on the threat type; use differential privacy technology to query the stored data, and adjust the amount of noise during the query in real time according to the data risk threat situation; perform sensitive data screening and cache backup of dynamic data based on the abnormal data risk threat situation; detect the current processing speed and the backlog of pending tasks, and automatically adjust resource allocation based on abnormal data risk threats.

[0052] A data security management method has been implemented that comprehensively considers the entire data processing life cycle, including data processing distribution, data threat identification, database security query, automatic backup of sensitive data, and adaptive adjustment of resource allocation, to meet the needs of efficient protection and management of massive data and ensure the security and privacy of data during transmission, storage, and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the data security management method applied to a big data management platform of the present invention;

[0054] Figure 2 The flowchart of the present invention for obtaining a large data set in real time and in a dynamic manner, and then dividing the data set and adding it to an idle data processing queue;

[0055] Figure 3 A flowchart for establishing a data threat identification model for the present invention, detecting abnormal data risk threats on the platform in real time, and taking automated response measures based on threat types;

[0056] Figure 4 The invention adopts differential privacy technology to query the stored data and adjusts the noise amount flow chart in real time according to the data risk threat situation;

[0057] Figure 5 This is a flow chart of sensitive data screening and cache backup of dynamic data based on abnormal data risk threat situations of the present invention;

[0058] Figure 6 A flowchart of the present invention for detecting the current processing speed and the backlog of pending tasks, and automatically adjusting resource allocation in combination with abnormal data risk threats;

[0059] Figure 7 It is a schematic diagram of the structure of the data security management system applied to the big data management platform of the present invention. DETAILED DESCRIPTION

[0060] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0061] Reference Figure 1 As shown, the data security management method applied to the big data management platform includes:

[0062] Obtain a real-time dynamic data set of big data, split the data set and add it to the idle data processing queue;

[0063] Establish a data threat identification model to detect abnormal data risk threats on the platform in real time and take automated response measures based on the threat type;

[0064] Differential privacy technology is used to query stored data, and the amount of noise during query is adjusted in real time according to the data risk threat situation;

[0065] Perform sensitive data screening and cache backup of dynamic data based on abnormal data risk threats;

[0066] Detect the current processing speed and the backlog of pending tasks, and automatically adjust resource allocation based on abnormal data risk threats.

[0067] Reference Figure 2 As shown, a big data real-time dynamic data set is obtained, and the data set is segmented and added to an idle data processing queue.

[0068] Get the dynamic data set within the set time window, set the data processing segmentation window size based on the available resources remaining on the server, and divide the dynamic data set according to the segmentation window size;

[0069] During the segmentation process, it is necessary to ensure that each segmented data segment must be smaller than or equal to the set segmentation window size to ensure the integrity of the data after segmentation.

[0070] Obtain the load conditions of all data processing threads and the number of tasks accumulated in the corresponding data processing task queues, take the ratio of the number of tasks accumulated to the thread processing rate of each processing thread, combine it with the set initial load weight of each thread, and use the product of the ratio of the number of tasks accumulated to the thread processing rate and the load weight as the idle evaluation value of the data processing thread;

[0071] After the idle evaluation values ​​of all processing threads are sorted in reverse order, the segmented dynamic data are selected and added to the task waiting queue of the data processing thread in sequence according to the sorting results.

[0072] The purpose of this process is to distribute sequential processing tasks according to the remaining thread processing capacity, which helps to avoid the situation where some threads are overloaded or idle, and improves the system resource utilization and the load balancing level of the data processing process.

[0073] Reference Figure 3 As shown, a data threat identification model is established to detect abnormal data risk threats on the platform in real time and take automated response measures based on the threat type.

[0074] Obtain historical processing data from the system platform as training data, classify the training data based on the set data classification, and batch-label the data after classification;

[0075] Preprocess and extract features of batch labeled data, establish data classification feature vectors, select convolutional neural networks, train data classification and recognition models based on feature-extracted training data, and deploy the trained models into the system;

[0076] Among them, the establishment of data classification feature vectors should include multi-dimensional feature extraction, including data generation frequency domain features, data length features, and data regularity features;

[0077] Set up data classification threat identification division of labor and establish corresponding threat identification models for each set data classification, specifically:

[0078] Based on the historical risk screening results of the current data classification, set the risk threat classification corresponding to the data classification;

[0079] Retrieve the data specifically involved in the risk screening results in the training data corresponding to the current data classification, mark it as risk source data, and attribute and label all risk source data based on the set threat classification;

[0080] Preprocess and extract features of risk source data, establish risk data threat classification feature vectors, select convolutional neural networks, and train a dedicated risk threat identification model for current data classification based on the risk source data after feature extraction;

[0081] Among them, the establishment of risk data threat classification feature vector should include multi-dimensional feature extraction, including data generation frequency domain features, risk threat data keyword similarity features, data length features, and data regularity features;

[0082] Combine and deploy the dedicated risk threat identification models corresponding to all data classifications into the system;

[0083] Set corresponding response strategies based on historical risk threat handling methods and establish risk threat-response strategy associations;

[0084] Get the data processing thread, identify the data type of the real-time data being processed in the data processing thread through the data classification recognition model, select the corresponding dedicated risk threat recognition model for risk threat recognition based on the data type recognition result, and perform automated response strategy selection and data risk threat processing based on the risk threat recognition result and the risk threat-response strategy association, thus realizing the automated process of data classification recognition-risk threat recognition-response processing.

[0085] Reference Figure 4 As shown in the figure, differential privacy technology is used to query the stored data. According to the data risk threat situation, the noise amount during the query is adjusted in real time. Specifically, it includes:

[0086] Get all the table structures of the database, select the privacy statistical fields, and use the Sigmoid function to adjust the risk of the set privacy budget in real time based on the recognition hit rate of the risk threat recognition model in the current set segmentation window. The specific expression is:

[0087] ;

[0088] In the formula, , They are the set privacy budget and the adjusted privacy budget, Identify hit rates for risk threat identification models;

[0089] When the risk threat identification hit rate is high, it indicates that the system security risk is high at this time. Reducing the privacy budget can increase the noise scale during data query and make statistical query more secure.

[0090] Set the statistical field update monitoring window time, obtain the maximum batch size of the most recent time window for inserting the privacy statistical field, and combine it with the adjusted privacy budget , calculate the noise scale parameter;

[0091] Select a random value that obeys the Laplace distribution, and generate Laplace noise according to the noise scale and the selected random value. The specific expression is:

[0092] ;

[0093] In the formula, is the generated Laplace noise, is the sign of the random value, is a random value, Insert batch size for setting privacy statistics field, is the noise scale parameter;

[0094] The generated noise is applied to the query process of the privacy statistical field to increase the privacy of the statistical data query. Specifically, the true value of the statistical query is added to the calculated noise value to obtain an approximate statistical value.

[0095] Reference Figure 5 As shown, sensitive data is screened and cached for backup on dynamic data based on abnormal data risk threats.

[0096] Obtain the data type identification result of the real-time data being processed by the data processing thread, set the sensitive data classification and sensitive operation keywords, retrieve the data matching the sensitive operation keywords in the sensitive data classification, mark the entire data corresponding to the keyword as sensitive data, and calculate the proportion of sensitive data in the total processed data;

[0097] Obtain the recognition hit rate of the risk threat recognition model within the currently set segment window, perform weighted summation on the sensitive data proportion and the recognition hit rate, and calculate the sensitive data risk assessment value;

[0098] Set a sensitive data risk assessment value threshold. When the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, store all marked sensitive data in the cache backup;

[0099] Among them, the sensitive data risk assessment threshold is set by personnel in this field based on the risk threat situation of real-time data processing and specific security requirements, and will not be repeated here.

[0100] When an uncaught exception is found during the thread processing data, the thread will terminate. This is one of the most common reasons for thread interruption. When the data format that the system can process is leaked, outsiders can attack the system by inputting a large amount of non-compliant data formats and risky data, which may cause the data processing thread to be frequently interrupted. In the process, some important sensitive data operations will also be lost.

[0101] If the processing thread does not experience an abnormal interruption during the current task processing, the cache backup will be cleared after the task processing is completed. If an abnormal interruption occurs, the breakpoint will be recorded and the cache backup data will be restored and reprocessed.

[0102] Reference Figure 6 As shown, the current processing speed and the backlog of pending tasks are detected, and resource allocation is automatically adjusted based on abnormal data risk threats. Specifically, the following are included:

[0103] Detect the processing speed of all current data processing threads and the backlog of pending tasks, and recalculate the idle evaluation values ​​of all current data processing threads;

[0104] Obtain the sensitive data risk assessment values ​​of all current data processing threads, use the ratio of the idle assessment value to the sensitive data risk assessment value as the resource allocation tendency coefficient, calculate the average value of the resource allocation tendency coefficients of all data processing threads, and mark it as the thread processing resource allocation reference line;

[0105] When the idle evaluation value of the thread increases and the risk evaluation value of sensitive data decreases, it indicates that the remaining resources of the current thread are sufficient and no more resources need to be allocated for risk data processing. At this time, some idle resources of the current thread can be recycled. Conversely, when the idle evaluation value of the thread decreases and the risk evaluation value of sensitive data increases, it indicates that the remaining resources of the current thread are tight and more system resources are needed to ensure the efficiency of risk data processing. At this time, more resources need to be allocated to the current processing thread.

[0106] Based on the deviation between the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line, the computing resources and storage resources of each processing thread are adjusted proportionally, thus achieving a dynamic allocation balance of data processing resources.

[0107] For further information, see Figure 7 As shown, based on the same inventive concept as the data security management method applied to the big data management platform, the present solution also proposes a data security management system applied to the big data management platform, including:

[0108] A data acquisition and processing distribution module, which acquires a dynamic data set within a set time window, divides the dynamic data set into segmented window sizes based on the remaining available resources of the server, calculates the idle evaluation value of the data processing thread, and sequentially adds the dynamic data set to the task waiting queue of the data processing thread according to the calculation results;

[0109] A data type and risk threat identification module, which uses the historical processing data of the system platform as training data, trains a data classification identification model, and based on the historical risk screening processing results of data classification, trains a dedicated risk threat identification model for data classification, and simultaneously establishes a risk threat-response strategy association relationship, identifies the type and risk threat of the currently processed data, and performs automated response strategy selection and data risk threat processing;

[0110] A database query security module, which performs real-time risk adjustment on a set privacy budget according to the recognition hit rate of the risk threat recognition model in the currently set segment window, calculates a noise scale parameter based on the insertion batch size of the privacy statistics field and the adjusted privacy budget, and then selects a random value, generates Laplace noise according to the noise scale and the selected random value, and applies the generated noise to the query process of the privacy statistics field;

[0111] A sensitive data automatic backup module, which calculates the sensitive data risk assessment value according to the weighted sum of the proportion of sensitive data to the total processed data and the recognition hit rate of the risk threat recognition model in the currently set segment window, and when the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, stores all marked sensitive data in the cache backup;

[0112] A system resource allocation adaptive adjustment module, which uses the average value of the ratio of the idle assessment value to the sensitive data risk assessment value of all data processing threads as the thread processing resource allocation reference line, and proportionally adjusts the computing resources and storage of each processing thread based on the deviation between the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line.

[0113] Furthermore, the present solution also proposes a data security management method storage medium applied to a big data management platform, on which a computer-readable program is stored. When the computer-readable program is called, the above-mentioned data security management method applied to a big data management platform is executed.

[0114] It is understandable that the storage medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid state drive (SSD).

[0115] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A data security management method applied to a big data management platform, characterized in that: include: Obtain a real-time dynamic data set of big data, split the data set and add it to the idle data processing queue; Establish a data threat identification model to detect abnormal data risk threats on the platform in real time and take automated response measures based on the threat type; Differential privacy technology is used to query stored data, and the amount of noise during query is adjusted in real time according to the data risk threat situation; Perform sensitive data screening and cache backup of dynamic data based on abnormal data risk threats; Detect the current processing speed and the backlog of pending tasks, and automatically adjust resource allocation based on abnormal data risk threats; The establishment of a data threat identification model to detect abnormal data risk threats on the platform in real time and taking automated response measures based on threat types specifically include: Obtain historical processing data from the system platform as training data, classify the training data based on the set data classification, and batch-label the data after classification; Preprocess and extract features of batch labeled data, establish data classification feature vectors, select convolutional neural networks, train data classification and recognition models based on feature-extracted training data, and deploy the trained models into the system; Set up data classification threat identification division of labor and establish corresponding threat identification models for each set data classification, specifically: Based on the historical risk screening results of the current data classification, set the risk threat classification corresponding to the data classification; Retrieve the data specifically involved in the risk screening results in the training data corresponding to the current data classification, mark it as risk source data, and attribute and label all risk source data based on the set threat classification; Preprocess and extract features of risk source data, establish risk data threat classification feature vectors, select convolutional neural networks, and train a dedicated risk threat identification model for current data classification based on the risk source data after feature extraction; Combine and deploy the dedicated risk threat identification models corresponding to all data classifications into the system; Set corresponding response strategies based on historical risk threat handling methods and establish risk threat-response strategy associations; Obtain the data processing thread, identify the data type of the real-time data being processed in the data processing thread through the data classification recognition model, select the corresponding dedicated risk threat recognition model for risk threat recognition based on the data type recognition result, and perform automated response strategy selection and data risk threat processing based on the risk threat recognition result and the risk threat-response strategy association relationship; The differential privacy technology is used to query the stored data, and the noise amount during the query is adjusted in real time according to the data risk threat situation, specifically including: Get all the table structures of the database, select the privacy statistical fields, and use the Sigmoid function to adjust the risk of the set privacy budget in real time based on the recognition hit rate of the risk threat recognition model in the current set segmentation window. The specific expression is: ； In the formula, , They are the set privacy budget and the adjusted privacy budget, Identify hit rates for risk threat identification models; Set the statistical field update monitoring window time, obtain the maximum batch size of the most recent time window for inserting the privacy statistical field, and combine it with the adjusted privacy budget , calculate the noise scale parameter; Select a random value that obeys the Laplace distribution, and generate Laplace noise according to the noise scale and the selected random value. The specific expression is: ； In the formula, is the generated Laplace noise, is the sign of the random value, is a random value, Insert batch size for setting privacy statistics field, is the noise scale parameter; The generated noise is applied to the query process of the privacy statistical field to increase the privacy of statistical data queries.

2. The data security management method applied to a big data management platform according to claim 1, characterized in that: The obtaining of a big data real-time dynamic data set and segmenting the data set and adding it to an idle data processing queue specifically includes: Get the dynamic data set within the set time window, set the data processing segmentation window size based on the available resources remaining on the server, and divide the dynamic data set according to the segmentation window size; Obtain the load conditions of all data processing threads and the number of tasks accumulated in the corresponding data processing task queues, calculate the idle evaluation value of the data processing thread by combining the ratio of the number of tasks accumulated for each processing thread to the thread processing rate and the set initial load weight of each thread; After the idle evaluation values ​​of all processing threads are sorted in reverse order, the segmented dynamic data are selected and added to the task waiting queue of the data processing thread in sequence according to the sorting results.

3. The data security management method applied to a big data management platform according to claim 2 is characterized in that: The sensitive data screening and cache backup of dynamic data based on abnormal data risk threat conditions specifically include: Obtain the data type identification result of the real-time data being processed by the data processing thread, set the sensitive data classification and sensitive operation keywords, retrieve the data matching the sensitive operation keywords in the sensitive data classification, mark the entire data corresponding to the keyword as sensitive data, and calculate the proportion of sensitive data in the total processed data; Obtain the recognition hit rate of the risk threat recognition model within the currently set segment window, perform weighted summation on the sensitive data proportion and the recognition hit rate, and calculate the sensitive data risk assessment value; Set a sensitive data risk assessment value threshold. When the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, store all marked sensitive data in the cache backup; If the processing thread does not experience an abnormal interruption during the current task processing, the cache backup will be cleared after the task processing is completed. If an abnormal interruption occurs, the breakpoint will be recorded and the cache backup data will be restored and reprocessed.

4. The data security management method applied to a big data management platform according to claim 3 is characterized in that: The detection of the current processing speed and the accumulation of pending tasks and the automatic adjustment of resource allocation in combination with the abnormal data risk threat specifically include: Detect the processing speed of all current data processing threads and the backlog of pending tasks, and recalculate the idle evaluation values ​​of all current data processing threads; Obtain the sensitive data risk assessment values ​​of all current data processing threads, use the ratio of the idle assessment value to the sensitive data risk assessment value as the resource allocation tendency coefficient, calculate the average value of the resource allocation tendency coefficients of all data processing threads, and mark it as the thread processing resource allocation reference line; Based on the deviation of the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line, the computing resources and storage resources of each processing thread are adjusted in proportion.

5. A data security management system applied to a big data management platform, used to implement the data security management method applied to a big data management platform as claimed in any one of claims 1 to 4, characterized in that: include: A data acquisition and processing distribution module, which acquires a dynamic data set within a set time window, divides the dynamic data set into segmented window sizes based on the remaining available resources of the server, calculates the idle evaluation value of the data processing thread, and sequentially adds the dynamic data set to the task waiting queue of the data processing thread according to the calculation results; A data type and risk threat identification module, which uses the historical processing data of the system platform as training data, trains a data classification identification model, and based on the historical risk screening processing results of data classification, trains a dedicated risk threat identification model for data classification, and simultaneously establishes a risk threat-response strategy association relationship, identifies the type and risk threat of the currently processed data, and performs automated response strategy selection and data risk threat processing; A database query security module, which performs real-time risk adjustment on a set privacy budget according to the recognition hit rate of the risk threat recognition model in the currently set segment window, calculates a noise scale parameter based on the insertion batch size of the privacy statistics field and the adjusted privacy budget, and then selects a random value, generates Laplace noise according to the noise scale and the selected random value, and applies the generated noise to the query process of the privacy statistics field; A sensitive data automatic backup module, which calculates the sensitive data risk assessment value according to the weighted sum of the proportion of sensitive data to the total processed data and the recognition hit rate of the risk threat recognition model in the currently set segment window, and when the risk assessment value is greater than or equal to the sensitive data risk assessment value threshold, stores all marked sensitive data in the cache backup; A system resource allocation adaptive adjustment module, which uses the average value of the ratio of the idle assessment value to the sensitive data risk assessment value of all data processing threads as the thread processing resource allocation reference line, and proportionally adjusts the computing resources and storage of each processing thread based on the deviation between the resource allocation tendency coefficient of each processing thread and the thread processing resource allocation reference line.

Images