A big data privacy protection intelligent collaboration system
By combining multi-dimensional sensitivity assessment with multi-objective optimization, the problem of insufficient dynamic scheduling capability of privacy budget in existing technologies is solved. This enables dynamic intelligent scheduling and real-time perception of the entire privacy budget process, improving the accuracy of privacy risk assessment and adaptive optimization of budget resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU VOCATIONAL COLLEGE OF BUSINESS
- Filing Date
- 2026-06-09
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies lack the ability to dynamically and intelligently schedule privacy budgets, and auditing and risk assessment are disconnected, making it impossible to achieve real-time perception of privacy risks throughout the entire process and adaptive optimization of budget resources.
By employing a method that combines multi-dimensional sensitivity assessment with multi-objective optimization, and through privacy budget management, monitoring, auditing, and risk assessment modules, dynamic and intelligent scheduling of the privacy budget is achieved.
It enables differentiated and refined initial privacy budget configuration, dynamic monitoring of budget usage, multi-dimensional real-time quantitative assessment and precise early warning, ensuring that the independence of the differential privacy combination theorem is not compromised.
Smart Images
Figure CN122365583A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data security and privacy technology, specifically to a big data privacy protection intelligent collaboration system. Background Technology
[0002] With the rapid development of the digital economy, the demand for cross-institutional and cross-domain big data collaboration is growing, and privacy computing technology has become a key support for ensuring the secure flow of data elements. In multi-party data collaboration scenarios, how to realize data value mining while protecting personal privacy and business secrets has become a core issue of widespread concern in the industry. Currently, mainstream solutions primarily employ static privacy budget allocation, periodic manual auditing, and rule-based risk assessment mechanisms to achieve privacy protection through fixed thresholds and simple log analysis. Specifically, existing systems typically allocate privacy budget quotas once before task initiation, lacking dynamic monitoring and real-time adjustment capabilities during execution; auditing is mostly focused on post-event verification, failing to cover the entire data lifecycle; and risk assessment relies on single-dimensional rule matching, unable to effectively integrate multi-source heterogeneous features. However, the above solutions have significant shortcomings: static budget allocation cannot adapt to the dynamic changes in actual task consumption and system load, which can easily lead to premature budget depletion or resource waste; auditing and risk assessment are independent of each other, with risk discovery lagging behind budget consumption, making it difficult to form an effective closed loop; in addition, the risk threshold is fixed and cannot be adaptively adjusted according to the actual operating situation, resulting in a high false alarm and false negative rate; in summary, existing technologies lack the ability to dynamically and intelligently schedule privacy budgets, and auditing, assessment and budget management are isolated from each other, making it impossible to achieve real-time perception of privacy risks throughout the entire process and adaptive optimization of budget resources.
[0003] Therefore, the present invention provides a big data privacy protection intelligent collaboration system. Summary of the Invention
[0004] To address the aforementioned technical issues, this paper provides a big data privacy protection intelligent collaboration system. This technical solution solves the problems of lacking dynamic intelligent scheduling capabilities for privacy budgets, the disconnect between auditing, evaluation, and budget management, and the inability to achieve real-time perception of privacy risks throughout the entire process and adaptive optimization of budget resources.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A big data privacy-preserving intelligent collaboration system includes: The privacy budget management module has budget redistribution rules. It obtains the data type, source, de-identification degree and content sensitivity of each collaborative task, calculates the data sensitivity level, determines the task priority weight based on business value, time urgency and trust level of participants, and allocates an initial privacy budget quota to each collaborative task in combination with the available resources of the privacy budget pool. The available resources of the privacy budget pool are the current remaining budget amount of the privacy budget pool. The privacy budget monitoring module collects the privacy budget consumption rate and usage rate of each collaborative task, and generates system access records and data processing behavior data. The end-to-end privacy audit module performs compliance checks and privacy impact assessments on the data collection, transmission, processing, and output stages of the system, generating structured audit logs. The privacy risk assessment module extracts privacy risk features from structured audit logs, system access records, and data processing behavior data. It uses a structured feature weighted scoring method to calculate the privacy risk score for each data processing stage. The privacy risk score is compared with a risk threshold to generate risk warning information. The risk threshold is dynamically generated by an adaptive threshold adjustment mechanism. The privacy budget scheduling module receives risk warning information, generates a privacy budget adjustment strategy based on the initial privacy budget quota and privacy budget consumption rate, and issues budget adjustment instructions to each collaborative task. When a collaborative task is completed or terminated, the unconsumed privacy budget quota is returned to the privacy budget pool or allocated to subsequent tasks according to the budget redistribution rules. The unconsumed privacy budget quota is the portion of the privacy budget that has been allocated but not actually consumed by the collaborative task, and its redistribution does not involve the recovery or transfer of consumed privacy budget.
[0006] Preferably, in the privacy budget management module, calculating the data sensitivity level includes: Construct a multi-dimensional sensitivity assessment index system, which includes data type index, data source index, de-identification degree index, and content sensitivity index; The degree of de-identification is assessed by statistically analyzing the number and distribution of direct identifiers, quasi-identifiers, and sensitive identifiers. The content sensitivity index uses keyword matching and regular expressions to identify sensitive content fragments containing information related to personal privacy, trade secrets, and national security. For the four types of indicators, the data sensitivity level is calculated by weighting and summing according to the preset weight coefficients, and the sum of each weight coefficient is 1.
[0007] Preferably, in the privacy budget management module, task priority weights are determined based on business value, time urgency, and trust levels of participants. An initial privacy budget quota is then allocated to each collaborative task based on the available resources of the privacy budget pool, including: The task priority weight is generated by weighting and summing the weight coefficients of each dimension based on business value, time urgency, and trust level of the participants. The sum of the weight coefficients is 1. Using task priority weights as input constraints, the current remaining budget amount of the privacy budget pool as the upper limit of available resources, the optimization objective of ensuring sufficient budget for high-priority tasks, and the constraint that the overall privacy risk of the system does not exceed a given security threshold, the initial privacy budget quota for each collaborative task is solved through constraint optimization.
[0008] Preferably, the privacy budget monitoring module includes: Based on the privacy protection mechanisms and corresponding privacy budget parameters used in the data processing operations of each collaborative task, the privacy budget consumption is accumulated according to the differential privacy combination theorem, and the consumption rate per unit time is calculated. The budget utilization rate for each task is calculated, whereby the budget utilization rate is the ratio of cumulative consumption to the initial privacy budget quota. The privacy budget consumption rate and budget usage rate are packaged into system access records and data processing behavior data.
[0009] Preferably, the end-to-end privacy audit module includes: The data collection process includes authorization verification, data source legality verification, collection scope limitation verification, and collection frequency monitoring. The data transmission process includes encryption verification of the transmission channel, data integrity verification, transmission log recording, and channel quality assessment. Conduct privacy impact assessments on data processing, verify the privacy compliance of processing algorithms, detect the risk of privacy leaks in processing results, and trace the processing chain. The data output process includes verifying the effectiveness of data anonymization, verifying output authorization, detecting residual sensitive information, and assessing the impact of output. The structured audit log is generated by summarizing the audit results from each stage.
[0010] Preferably, in the privacy risk assessment module, the structured feature weighted scoring method calculates the privacy risk score for each data processing stage, including: The following risk characteristic dimensions were extracted from structured audit logs, system access records, and data processing behavior data: Compliance anomaly level is quantified based on the number and severity of non-compliance items during the audit process; Access anomaly is quantified based on the standard deviation of access frequency from the historical mean. Operational sensitivity is quantified based on the proportion of operations involving sensitive data. The degree of associated risk is quantified based on the proportion of high-risk nodes in the participants' historical violation records and data flow. After normalizing each risk characteristic dimension, the privacy risk score is calculated by weighting and summing according to preset weight coefficients, with the sum of each weight coefficient being 1.
[0011] Preferably, the privacy risk assessment module further includes context-aware analysis, specifically: Identify the task type, data asset value level, and participant trust level of the current task, and generate a context correction factor. The context correction factor is the product of the task type correction coefficient, the data asset value correction coefficient, and the participant trust correction coefficient, with each correction coefficient ranging from 0.5 to 2.0. The privacy risk score is multiplicatively corrected using the context correction factor, and a risk warning is generated when the corrected privacy risk score exceeds the risk threshold.
[0012] Preferably, the privacy risk assessment module further includes an adaptive threshold adjustment mechanism, specifically: The underreporting rate and false alarm rate for the most recent W audit periods are calculated using a sliding window. The underreporting rate is the proportion of audit records whose actual risk score exceeds the risk threshold but no warning is triggered out of all audit records that exceed the risk threshold. The false alarm rate is the proportion of audit records whose actual risk score does not exceed the risk threshold but a warning is triggered out of all audit records that do not exceed the risk threshold. When the false negative rate exceeds the target false negative rate, the risk threshold is lowered by adjusting the step size to improve the recall rate; when the false positive rate exceeds the target false positive rate, the risk threshold is raised by adjusting the step size to reduce false positives; where W, adjustment step size, target false negative rate, and target false positive rate are set by the system administrator. The threshold update cycle is performed once every N audit cycles, and after the update, the new risk threshold is synchronized to the privacy budget management module.
[0013] Preferably, in the privacy budget scheduling module, the budget adjustment strategy is generated based on budget adjustment rules and constraint optimization algorithms, specifically as follows: The budget adjustment rules are as follows: when the budget utilization rate of a task exceeds the upper limit of the utilization rate, the budget of the task is reduced by a factor proportional to the amount of excess; when the budget consumption rate of a task exceeds the upper limit of the rate, the budget of the task is reduced by a factor proportional to the proportion of excess; when a risk warning information for a task is received, the budget of the task is reduced to the minimum guaranteed amount. The budget released by the reduction and the remaining budget in the privacy budget pool are allocated to tasks that have not triggered the reduction in proportion to their task priority weights, provided that the total budget constraint is met. The upper limit of utilization, the upper limit of rate, the reduction ratio coefficient, and the minimum guaranteed amount are configured by the system administrator.
[0014] Preferably, the budget reallocation rules in the privacy budget scheduling module include: The redistribution only applies to the privacy budget quota that has been allocated but not consumed by the task, i.e., the difference between the initial privacy budget quota and the consumed privacy budget. The consumed privacy budget does not participate in any redistribution operation to ensure that the independence of the differential privacy combination theorem is not violated. The direct redistribution strategy is applicable to consecutive tasks within the same user or project group, and directly allocates unused quotas to subsequent related tasks according to a given ratio. The proportional reclamation strategy, applicable to tasks between different users or different project groups, returns unused quotas to the privacy budget pool at a given ratio. The credit conversion strategy converts unused quota into the task initiator's privacy credit value according to a given conversion coefficient. The privacy credit value is used to increase the priority weight of subsequent tasks. Before redistribution, the privacy risk score at the time of task termination is locked. If the score exceeds the risk threshold, all unused quotas for the task are returned to the privacy budget pool and do not participate in any redistribution or conversion; the redistribution process is fully recorded.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a quota allocation method that combines multi-dimensional sensitivity assessment with multi-objective optimization, achieving differentiated and refined initial privacy budget configuration with a unique optimal solution. By collecting privacy budget consumption rates and usage rates in real time, it enables dynamic monitoring of budget usage. By combining full-process privacy auditing with structured feature-weighted scoring, it achieves multi-dimensional real-time quantitative assessment and accurate early warning of privacy risks, with low computational complexity and strong interpretability. Budget reallocation is strictly limited to the reallocation of unconsumed quotas; consumed privacy budgets do not participate in any reallocation operations, ensuring that the independence of the differential privacy combination theorem is not violated. Attached Figure Description
[0016] Figure 1 This is a system framework diagram of the present invention. Detailed Implementation
[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.
[0018] Reference Figure 1 As shown, a big data privacy-preserving intelligent collaboration system includes: The privacy budget management module maintains a privacy budget pool, implements budget reallocation rules, obtains the data type, source, de-identification level, and content sensitivity of each collaborative task, calculates the data sensitivity level, determines task priority weights based on business value, time urgency, and participant trust levels, and allocates initial privacy budget quotas to each collaborative task in conjunction with the available resources in the privacy budget pool. In the privacy budget management module, the available resources in the privacy budget pool refer to the current remaining budget amount. The calculation of data sensitivity levels includes: Construct a multi-dimensional sensitivity assessment index system, which includes data type index, data source index, de-identification degree index, and content sensitivity index; The degree of de-identification is assessed by statistically analyzing the number and distribution of direct identifiers, quasi-identifiers, and sensitive identifiers. Direct identifiers are data items that can uniquely identify a specific natural person on their own, such as ID card numbers, mobile phone numbers, biometric features, and precise geographic coordinates. Quasi-identifiers are data items that cannot uniquely identify a specific natural person on their own, but may identify a specific individual when combined with other information, such as gender, date of birth, age range, postal code, and occupation category. Sensitive identifiers are data items that, if leaked or illegally used, could easily infringe upon the personal dignity or endanger the personal or property safety of a natural person, such as ethnicity, religious beliefs, medical and health information, financial account information, and whereabouts. The system uses regular expression matching, field name semantic recognition, and comparison with a preset identifier library to count the number of fields, record coverage ratio, and distribution density of the above three types of identifiers in the dataset, and generates a de-identification degree score. .
[0019] The content sensitivity index uses keyword matching and regular expressions to identify sensitive content fragments containing information related to personal privacy, trade secrets, and national security. The content fragments of the data to be evaluated are matched against a pre-set sensitive keyword database and regular expression rule database. The number and coverage ratio of matched keywords and rules are counted to generate a content sensitivity score. .
[0020] For the four types of indicators, the data sensitivity level is calculated by weighting and summing according to preset weight coefficients, with the sum of each weight coefficient being 1; data type scoring. The score is determined based on the data category; for example, genetic data receives the highest score, followed by medical data, and general commercial data receives the lowest score. The score is also based on the data source. Determined based on the data provider's security level; de-identification level score. The score is determined based on statistical results from three types of identifiers; the more identifiers, the higher the score. (Content sensitivity score) The weighted sum is determined based on the number and coverage ratio of the matched keywords and rules. The sum of all weight coefficients in the weighted summation is 1, and this is configured by the system administrator according to the business scenario. The scoring range is on a 100-point scale; a higher score indicates higher data sensitivity.
[0021] In the privacy budget management module, task priority weights are determined based on business value, time urgency, and trust levels of participants. Initial privacy budget quotas are then allocated to each collaborative task based on the available resources of the privacy budget pool, including: The task priority weight is generated by weighting and summing the weights of business value, time urgency, and participant trust level as evaluation dimensions. Business value is quantified based on the expected revenue or data asset value of the task; time urgency is calculated as the reciprocal of the difference between the task deadline and the current time; participant trust level is comprehensively assessed based on the number of privacy violations and the fulfillment rate in historical collaboration records. The sum of all weight coefficients is 1, and this is configured by the system administrator.
[0022] The initial privacy budget quota is allocated using a constraint optimization algorithm. Task priority weights are used as input constraints, the current remaining budget in the privacy budget pool is used as the upper limit of available resources, the optimization objective is to ensure high-priority tasks receive sufficient budget, and the constraint is that the overall system privacy risk does not exceed a given security threshold. The initial privacy budget quota for each collaborative task is solved through constraint optimization. The given security threshold is a system-level privacy security constraint threshold, set by the privacy budget management module during system initialization or configuration updates, used to constrain the overall risk upper limit of the quota allocation scheme. This given threshold differs from the risk threshold used by the privacy risk assessment module for real-time comparison: the given threshold acts as a static optimization constraint in the quota allocation phase, while the risk threshold acts as a dynamic risk assessment in the task execution phase; their functional domains are independent, and their values may differ.
[0023] In practice, the above-mentioned constrained optimization problem can be solved using linear programming, and has a unique optimal solution.
[0024] The privacy budget monitoring module collects the privacy budget consumption rate and usage rate of each collaborative task, and generates system access records and data processing behavior data. The privacy budget monitoring module includes: Based on the privacy protection mechanisms and corresponding privacy budget parameters employed in the data processing operations of each collaborative task, the privacy budget consumption is accumulated according to the differential privacy combination theorem. For example, if a task performs 100 queries under a differential privacy mechanism with a privacy budget parameter ε=0.1, then according to the Gaussian differential privacy combination theorem, the accumulated consumption is approximately 1.41. The consumption rate per unit time is the accumulated consumption divided by the execution time.
[0025] The budget utilization rate for each task is calculated, whereby the budget utilization rate is the ratio of cumulative consumption to the initial privacy budget quota. rate at which privacy budget is consumed and budget utilization rate The system access logs and data processing behavior data are packaged and generated. System access logs include user ID, access time, access operation, accessed resources, and access results; data processing behavior data includes data ID, processing time, processing type, processing parameters, and processing output. Both types of data are output in a structured log format for subsequent auditing and risk assessment.
[0026] Regarding the boundaries of responsibility between system access logs and audit logs: The system access logs generated by the privacy budget monitoring module focus on recording access behaviors directly related to privacy budget consumption. Their core purpose is to provide raw behavioral data for tracing privacy budget consumption and calculating its rate. The structured audit logs generated by the end-to-end privacy audit module focus on recording compliance checks and privacy impact assessment results. Their core purpose is to provide audit evidence for risk tracing and compliance verification. The two modules differ in their recording dimensions and generating entities. The monitoring module's system access logs serve as behavioral-level raw input, while the audit module's audit logs serve as compliance-level assessment output, jointly supporting the multi-source feature extraction of the privacy risk assessment module.
[0027] The end-to-end privacy audit module performs compliance checks and privacy impact assessments on the data collection, transmission, processing, and output stages of the system, generating structured audit logs. The end-to-end privacy audit module includes: The data collection process includes authorization verification, data source legality verification, collection scope limitation verification, and collection frequency monitoring. Auditing of the data collection process includes: verification of collection authorization, checking whether the collection behavior has obtained valid authorization from the data subject or data provider; verification of data source legality, checking whether the data source complies with laws, regulations and contractual agreements; verification of collection scope limitation, comparing the consistency between the actual collected fields and the authorized scope; and monitoring of collection frequency, counting the number of collections per unit time and identifying abnormally high-frequency collection behaviors.
[0028] The data transmission process includes encryption verification of the transmission channel, data integrity verification, transmission log recording, and channel quality assessment. Auditing of data transmission includes: encryption verification of the transmission channel, checking whether the transmission protocol enables encryption mechanisms such as TLS or SM4; data integrity verification, confirming that the data has not been tampered with before and after transmission by comparing hash values; transmission log recording, recording the start and end time of transmission, source address, destination address and the amount of data transmitted; and channel quality assessment, monitoring transmission delay, packet loss rate and number of retransmissions.
[0029] Conduct privacy impact assessments on data processing, verify the privacy compliance of processing algorithms, detect the risk of privacy leaks in processing results, and trace the processing chain. Audits of the data processing stage include: privacy impact assessment, analyzing the potential impact of processing operations on the privacy rights of data subjects; privacy compliance verification of processing algorithms, checking whether the processing algorithms meet privacy protection standards such as differential privacy, k-anonymity, or l-diversity; privacy leakage risk detection of processing results, assessing the leakage risk of output results through member inference attack simulation or attribute inference attack simulation; and processing chain tracing, recording the entire chain of data processing operation logs, supporting forward tracing and reverse backtracking.
[0030] The data output process includes verifying the effectiveness of data anonymization, verifying output authorization, detecting residual sensitive information, and assessing the impact of output. Auditing of the data output process includes: verifying the effectiveness of data anonymization, checking whether the output data has been de-identified or pseudonymous; verifying output authorization, confirming that the output behavior has been authorized by the data subject or administrator; detecting residual sensitive information, using keyword matching and regular expressions to scan for residual sensitive information in the output content; and assessing the impact of output, evaluating the degree of privacy damage that may be caused if the output data is obtained by a third party.
[0031] The structured audit log is generated by summarizing the audit results from each stage.
[0032] The privacy risk assessment module extracts privacy risk features from structured audit logs, system access records, and data processing behavior data. It uses a structured feature weighted scoring method to calculate the privacy risk score for each data processing stage. The privacy risk score is compared with a risk threshold to generate risk warning information. The risk threshold is dynamically generated by an adaptive threshold adjustment mechanism. The risk threshold serves as a real-time risk assessment benchmark, uniformly maintained by the privacy budget management module, and supports dynamic updates via an adaptive threshold adjustment mechanism. The risk threshold is used to determine whether a single task or data processing stage triggers a high-risk state: when the privacy risk score exceeds the risk threshold, a risk warning is generated and fed back to the privacy budget scheduling module; when it does not exceed the threshold, the current stage is considered to have controllable risk. This risk threshold differs from the given threshold during quota allocation: the risk threshold applies to dynamic risk assessment during runtime and can be adaptively adjusted based on the false negative and false positive rates; the given threshold applies to static optimization constraints during allocation and is typically set by the system administrator based on the business security level.
[0033] The privacy risk assessment module uses a structured feature-weighted scoring method to calculate the privacy risk score, including: The following four risk characteristic dimensions were extracted from structured audit logs, system access records, and data processing behavior data: Compliance anomaly The number and severity of non-compliance items in the audit process are quantified. During implementation, the inspection result of each audit process is assigned a value (0 for pass, 1 for warning, and 2 for failure), and then the weighted sum is normalized to [0,1].
[0034] Access anomaly The quantification is based on the standard deviation of the access frequency from the historical mean. During implementation, the access frequency of each user in the current audit period is statistically analyzed, the deviation from the historical mean is calculated, the maximum deviation is taken and divided by the historical standard deviation, and then truncated to [0,1].
[0035] Operational sensitivity Quantification based on the proportion of operations involving sensitive data. In implementation, the proportion of data processing operations involving sensitive identifiers out of the total number of operations is calculated.
[0036] Associated risk level Quantification is based on the proportion of high-risk nodes in the participants' historical violation records and data flow. During implementation, the participants' historical violation counts and the proportion of current data flow passing through high-risk nodes are extracted from the relationship graph, weighted, summed, and normalized to [0,1].
[0037] After normalizing each risk feature dimension to the range [0,1], the privacy risk score is calculated by weighting and summing according to preset weight coefficients. The formula is as follows:
[0038] in Each weighting coefficient is configured by the system administrator according to risk preference.
[0039] The privacy risk assessment module also includes context correction, specifically: Identify the task type, data asset value level, and participant trust level of the current task, and generate a context correction factor using the following formula:
[0040] in Adjust the coefficient according to the task type, for example, take 1.0 for statistical analysis, 1.2 for model training, and 1.5 for data export; This is a data asset value adjustment coefficient; the higher the value, the larger the value. This is a trust adjustment factor for the participants; the higher the trust level, the smaller the value. The value range for each adjustment factor is (0.5, 2.0), and it is configured by the system administrator.
[0041] The revised privacy risk score is:
[0042] when Risk warning information is generated when the risk threshold is exceeded.
[0043] The privacy risk assessment module also includes an adaptive threshold adjustment mechanism, specifically: The false positive rate (FNR) and missed detection rate (FPR) for the most recent W audit periods are calculated using a sliding window. The missed detection rate is the proportion of audit records where the actual risk score exceeds the risk threshold but no alert is triggered, out of all audit records exceeding the risk threshold. The false positive rate is the proportion of audit records where the actual risk score does not exceed the risk threshold but an alert is triggered, out of all audit records not exceeding the risk threshold. The missed detection rate is calculated based on manual review of triggered risk alerts; if the review confirms that an alert should have been triggered but the system did not, it is counted as a missed detection. The false positive rate is calculated based on manual review of triggered risk alerts; if the review confirms that no alert is needed, it is counted as a false positive.
[0044] The threshold update rule is: when the FNR exceeds the target false negative rate... hour,
[0045] When FPR does not exceed the target false alarm rate hour,
[0046] Where θ is the risk threshold. For the updated risk threshold, The current risk threshold before the update is given, and η is the adjustment step size, ranging from (0,1). and Set by the system administrator. The threshold is updated every N audit cycles, and the new risk threshold is synchronized to the privacy budget management module after each update.
[0047] For example, let W=100, η=0.5, =0.05, =0.10, N=50. If FNR=0.08 in the most recent 100 audit periods > =0.05, then Lowering the threshold can improve recall.
[0048] The privacy budget scheduling module receives risk warning information, generates a privacy budget adjustment strategy based on the initial privacy budget quota and privacy budget consumption rate, and issues budget adjustment instructions to each collaborative task. When a collaborative task is completed or terminated, the unconsumed privacy budget quota is returned to the privacy budget pool or allocated to subsequent tasks according to the budget redistribution rules. The unconsumed privacy budget quota is the portion of the privacy budget that has been allocated but not actually consumed by the collaborative task, and its redistribution does not involve the recovery or transfer of consumed privacy budget.
[0049] In the privacy budget scheduling module, the budget adjustment strategy is generated based on budget adjustment rules and constraint optimization algorithms, specifically as follows: Budget adjustment rules include: When the budget utilization rate of task j Exceeding the upper limit of usage At that time, the budget was reduced, and the amount was reduced:
[0050] in This is the utilization rate adjustment factor, with a value range of (0,1). When the budget consumption rate of task j Exceeding the upper limit of the rate At that time, the budget was reduced, and the amount was reduced:
[0051] in This is the rate adjustment coefficient, with a value range of (0,1); When a risk warning is received for task j, the budget for task j is reduced to the minimum guaranteed amount. Reduce the released budget and the remaining budget in the privacy budget pool, provided that constraints are met. Below, based on task priority weights The proportion is allocated to tasks that have not triggered reduction; that is, for task k that has not triggered reduction, the increment is:
[0052] in The sum of all budgets released by reductions and the remaining available budget in the budget pool.
[0053] The above , , , Configured by the system administrator. This is the minimum budget guarantee, i.e., the minimum privacy budget amount that a task retains even after triggering a risk warning. It is configured by the system administrator. For example, setting... =0.8, =1.5ε / hour, =0.3, =0.5, =0.5ε.
[0054] The budget reallocation rules in the privacy budget scheduling module include: The redistribution applies only to privacy budget quotas that have been allocated but not yet consumed for tasks:
[0055] in As the initial privacy budget allocation for this task, Privacy budget already consumed for this task. Privacy budget already consumed. It does not participate in any redistribution operations to ensure that the independence of the differential privacy combination theorem is not violated. The differential privacy combination theorem specifies the cumulative privacy loss from multiple queries. Each query consumes It is not reusable or recyclable; the redistribution operation in this invention only applies to quotas that have not yet been consumed by any query. Allocating them to other tasks is equivalent to adding an independent quota for that task, which does not violate the combinatorial theorem.
[0056] The direct redistribution strategy applies to consecutive tasks within the same user or project group. Unused quotas are directly allocated to subsequent related tasks at a given ratio. When a task is completed, the system checks whether the task initiator and the initiator of subsequent related tasks belong to the same user or project group. If the conditions are met, the unused quotas are directly allocated to the initial quota of subsequent related tasks at a given ratio.
[0057] The proportional reclamation strategy is applicable to tasks between different users or different project groups. Unused quotas are returned to the privacy budget pool at a given ratio, and then allocated to subsequent tasks as needed by the privacy budget management module.
[0058] The credit conversion strategy transforms unused quota into the task initiator's privacy credit value using a given conversion coefficient. This privacy credit value is accumulated and used to increase the priority weight of subsequent tasks. Task initiators with higher credit values receive a higher priority weight during quota allocation.
[0059] Before performing the redistribution, lock the privacy risk score at the time the task terminates. ,like If the risk threshold is exceeded, all unused quota for that task will be returned to the privacy budget pool and will not participate in any redistribution or conversion. The entire redistribution process will be recorded, including the source task identifier, the amount of unused quota, the redistribution strategy type, the target task identifier, and the execution timestamp.
[0060] Taking a cross-institutional medical data collaborative analysis scenario as an example, the overall system workflow is explained as follows: Step S1: Collaborative Task A (initiated by a hospital, aiming to train a diabetes prediction model) enters the system. The privacy budget management module obtains the data type (electronic medical record), source (tertiary hospital), degree of de-identification (direct identifiers have been removed, quasi-identifiers are retained), and content sensitivity (including disease diagnosis and medication records) of Task A, and calculates the scores for each dimension as follows. =85、 =70、 =60、 =90, weighted sum yields a sensitivity level of 78.5. Based on a three-dimensional assessment of business value, time urgency, and participant trust levels, the weighted sum yields a task priority weight of 0.77. Combining the currently available resources in the privacy budget pool, constraint optimization yields an initial quota ε of 15 for task A.
[0061] Step S2: During the execution of Task A, the privacy budget monitoring module collects the privacy budget consumption rate and usage rate. After the 100th differential privacy query (ε=0.1), Task A has accumulated a consumption of approximately 1.41 according to the differential privacy combination theorem, with a consumption rate of 0.705ε / h and a usage rate of 9.4%. The monitoring module packages the above data to generate system access records and data processing behavior data.
[0062] Step S3: The full-process privacy audit module audits the data collection (authorization valid), transmission (TLS encryption enabled), processing (differential privacy mechanism compliant), and output (de-identification effect verified) stages of Task A, and generates structured audit logs.
[0063] Step S4: The privacy risk assessment module extracts four-dimensional risk features from audit logs, system access records, and data processing behavior data: compliance anomaly score 0.1 (only 1 warning), access anomaly score 0.2 (normal access frequency), operation sensitivity score 0.6 (involves sensitive diagnostic data), and association risk score 0.3 (no violation records by participating parties). The weighted sum yields a risk score of 0.33. The context correction factor is 1.248 (task type 1.2 × data value 1.3 × participating party trust 0.8), resulting in a corrected risk score of 0.41. The current risk threshold is 0.50, which is not exceeded, and no risk warning is triggered.
[0064] Step S5: When Task A executes its 180th query, its consumption rate suddenly increases to 1.2ε / h, exceeding the upper limit of 1.0ε / h. The scheduling module triggers the rate adjustment rule, reducing Task A's budget by 1.5ε proportionally to the excess. The released budget and the remaining budget in the budget pool are allocated to other tasks according to their priority weights.
[0065] Step S6: Task A completes successfully, consuming 2.5ε, with 12.5ε remaining unused. The risk score of Task A at the time of termination is locked at 0.35, which does not exceed the risk threshold of 0.50. The system checks that Task A and its subsequent associated Task C belong to the same project group, and applies a direct redistribution strategy, directly allocating 80% (i.e., 10.0ε) of the unused quota to Task C's initial quota. The redistribution process is fully recorded. If the final risk score of Task A exceeds the risk threshold, all unused quota is returned to the privacy budget pool and does not participate in any redistribution.
[0066] Through the collaboration of the above modules, the system achieves dynamic intelligent scheduling of privacy budgets and closed-loop risk management.
[0067] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.
Claims
1. A big data privacy-protecting intelligent collaboration system, characterized in that, include: The privacy budget management module maintains the privacy budget pool, budget redistribution rules, obtains the data type, source, de-identification degree and content sensitivity of each collaborative task, calculates the data sensitivity level, determines the task priority weight based on business value, time urgency and the trust level of the participants, and allocates an initial privacy budget quota to each collaborative task in combination with the available resources of the privacy budget pool. The available resources of the privacy budget pool are the current remaining budget amount of the privacy budget pool. The privacy budget monitoring module collects the privacy budget consumption rate and usage rate of each collaborative task, and generates system access records and data processing behavior data. The end-to-end privacy audit module performs compliance checks and privacy impact assessments on the data collection, transmission, processing, and output stages of the system, generating structured audit logs. The privacy risk assessment module extracts privacy risk features from structured audit logs, system access records, and data processing behavior data. It uses a structured feature weighted scoring method to calculate the privacy risk score for each data processing stage. The privacy risk score is compared with a risk threshold to generate risk warning information. The risk threshold is dynamically generated by an adaptive threshold adjustment mechanism. The privacy budget scheduling module receives risk warning information, generates a privacy budget adjustment strategy based on the initial privacy budget quota and privacy budget consumption rate, and issues budget adjustment instructions to each collaborative task. When a collaborative task is completed or terminated, the unconsumed privacy budget quota is returned to the privacy budget pool or allocated to subsequent tasks according to the budget redistribution rules. The unconsumed privacy budget quota is the portion of the privacy budget that has been allocated but not actually consumed by the collaborative task, and its redistribution does not involve the recovery or transfer of consumed privacy budget.
2. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, The privacy budget management module calculates data sensitivity levels, including: Construct a multi-dimensional sensitivity assessment index system, which includes data type index, data source index, de-identification degree index, and content sensitivity index; The degree of de-identification is assessed by statistically analyzing the number and distribution of direct identifiers, quasi-identifiers, and sensitive identifiers. The content sensitivity index uses keyword matching and regular expressions to identify sensitive content fragments containing information related to personal privacy, trade secrets, and national security. For the four types of indicators, the data sensitivity level is calculated by weighting and summing according to the preset weight coefficients, and the sum of each weight coefficient is 1.
3. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, In the privacy budget management module, task priority weights are determined based on business value, time urgency, and trust levels of participants. Initial privacy budget quotas are then allocated to each collaborative task based on available resources in the privacy budget pool, including: The task priority weight is generated by weighting and summing the weight coefficients of each dimension based on business value, time urgency, and trust level of the participants. The sum of the weight coefficients is 1. Using task priority weights as input constraints, the current remaining budget amount of the privacy budget pool as the upper limit of available resources, the optimization objective of ensuring sufficient budget for high-priority tasks, and the constraint that the overall privacy risk of the system does not exceed a given security threshold, the initial privacy budget quota for each collaborative task is solved through constraint optimization.
4. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, The privacy budget monitoring module includes: Based on the privacy protection mechanisms and corresponding privacy budget parameters used in the data processing operations of each collaborative task, the privacy budget consumption is accumulated according to the differential privacy combination theorem, and the consumption rate per unit time is calculated. The budget utilization rate for each task is calculated, whereby the budget utilization rate is the ratio of cumulative consumption to the initial privacy budget quota. The privacy budget consumption rate and budget usage rate are packaged into system access records and data processing behavior data.
5. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, The end-to-end privacy audit module includes: The data collection process includes authorization verification, data source legality verification, collection scope limitation verification, and collection frequency monitoring. The data transmission process includes encryption verification of the transmission channel, data integrity verification, transmission log recording, and channel quality assessment. Conduct privacy impact assessments on data processing, verify the privacy compliance of processing algorithms, detect the risk of privacy leaks in processing results, and trace the processing chain. The data output process includes verifying the effectiveness of data anonymization, verifying output authorization, detecting residual sensitive information, and assessing the impact of output. The structured audit log is generated by summarizing the audit results from each stage.
6. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, In the privacy risk assessment module, the structured feature weighted scoring method calculates the privacy risk score for each data processing stage, including: The following risk characteristic dimensions were extracted from structured audit logs, system access records, and data processing behavior data: Compliance anomaly level is quantified based on the number and severity of non-compliance items during the audit process; Access anomaly is quantified based on the standard deviation of access frequency from the historical mean. Operational sensitivity is quantified based on the proportion of operations involving sensitive data. The degree of associated risk is quantified based on the proportion of high-risk nodes in the participants' historical violation records and data flow. After normalizing each risk characteristic dimension, the privacy risk score is calculated by weighting and summing according to preset weight coefficients, with the sum of each weight coefficient being 1.
7. The big data privacy protection intelligent collaboration system according to claim 6, characterized in that, The privacy risk assessment module also includes context-aware analysis, specifically: Identify the task type, data asset value level, and participant trust level of the current task, and generate a context correction factor. The context correction factor is the product of the task type correction coefficient, the data asset value correction coefficient, and the participant trust correction coefficient, with each correction coefficient ranging from 0.5 to 2.
0. The privacy risk score is multiplicatively corrected using the context correction factor, and a risk warning is generated when the corrected privacy risk score exceeds the risk threshold.
8. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, The privacy risk assessment module also includes an adaptive threshold adjustment mechanism, specifically: The underreporting rate and false alarm rate for the most recent W audit periods are calculated using a sliding window. The underreporting rate is the proportion of audit records whose actual risk score exceeds the risk threshold but no warning is triggered out of all audit records that exceed the risk threshold. The false alarm rate is the proportion of audit records whose actual risk score does not exceed the risk threshold but a warning is triggered out of all audit records that do not exceed the risk threshold. When the false negative rate exceeds the target false negative rate, the risk threshold is lowered by adjusting the step size to improve the recall rate; when the false positive rate exceeds the target false positive rate, the risk threshold is raised by adjusting the step size to reduce false positives; where W, adjustment step size, target false negative rate, and target false positive rate are set by the system administrator. The threshold update cycle is performed once every N audit cycles, and after the update, the new risk threshold is synchronized to the privacy budget management module.
9. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, In the privacy budget scheduling module, the budget adjustment strategy is generated based on budget adjustment rules and constraint optimization algorithms, specifically as follows: The budget adjustment rules are as follows: when the budget utilization rate of a task exceeds the upper limit of the utilization rate, the budget of the task is reduced by a factor proportional to the amount of excess; when the budget consumption rate of a task exceeds the upper limit of the rate, the budget of the task is reduced by a factor proportional to the proportion of excess; when a risk warning information for a task is received, the budget of the task is reduced to the minimum guaranteed amount. The budget released by the reduction and the remaining budget in the privacy budget pool are allocated to tasks that have not triggered the reduction in proportion to their task priority weights, provided that the total budget constraint is met. The upper limit of utilization, the upper limit of rate, the reduction ratio coefficient, and the minimum guaranteed amount are configured by the system administrator.
10. The big data privacy protection intelligent collaboration system according to claim 1, characterized in that, The budget reallocation rules in the privacy budget scheduling module include: The redistribution only applies to the privacy budget quota that has been allocated but not consumed by the task, i.e., the difference between the initial privacy budget quota and the consumed privacy budget. The consumed privacy budget does not participate in any redistribution operation to ensure that the independence of the differential privacy combination theorem is not violated. The direct redistribution strategy is applicable to consecutive tasks within the same user or project group, and directly allocates unused quotas to subsequent related tasks according to a given ratio. The proportional reclamation strategy, applicable to tasks between different users or different project groups, returns unused quotas to the privacy budget pool at a given ratio. The credit conversion strategy converts unused quota into the task initiator's privacy credit value according to a given conversion coefficient. The privacy credit value is used to increase the priority weight of subsequent tasks. Before redistribution, the privacy risk score at the time of task termination is locked. If the score exceeds the risk threshold, all unused quotas for the task are returned to the privacy budget pool and do not participate in any redistribution or conversion; the redistribution process is fully recorded.