Differential privacy parameter protection method in AI model iteration

Abstract

This application relates to the field of data processing technology and provides a differential privacy parameter protection method in AI model iteration. By organically combining the four steps of parameter preprocessing before AI model iteration, privacy monitoring and compensation during iteration, and vulnerability detection and parameter optimization after iteration, a privacy protection system covering the entire iteration cycle of the AI model is constructed; each step is synergistically related and can adaptively adjust privacy protection and parameter settings in real time according to multiple factors such as AI model training progress, parameter sensitivity changes, privacy leakage conditions, and attack test results, so as to better adapt to the complex and changeable AI model training environment; from discovering vulnerabilities through attack testing, to locating vulnerability noise parameters, and then to optimizing the noise parameter set according to parameter contribution rate and vulnerability conditions, a complete vulnerability prevention and repair closed loop is formed, which helps to timely discover and solve potential privacy and security issues in AI models and continuously improve the security and reliability of AI models.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a differential privacy parameter protection method in AI model iteration. Background Art

[0002] With the rapid development of artificial intelligence technology, AI models have been widely used in many fields such as healthcare, finance, and intelligent driving. AI models improve their performance by learning and training on large amounts of data. However, during the training and iteration process of AI models, parameter privacy issues have gradually become a key factor restricting their safe application. Currently, differential privacy technology, as a strict privacy protection theory, has been widely used in parameter protection of AI models.

[0003] Existing technologies typically employ a static allocation of privacy budget and noise. This approach is simple and straightforward, but it fails to consider the dynamic changes in parameter sensitivity during AI model training and the varying privacy protection requirements at different training stages, resulting in poor privacy protection. Highly sensitive parameters, which are more susceptible to privacy leaks, cannot be effectively protected due to insufficient allocated privacy budget and noise. On the other hand, excessive privacy protection for less sensitive parameters can introduce excessive noise, impacting AI model training and performance. Most approaches fail to address how to properly allocate privacy budget and noise during AI model iteration, as well as accurately monitor and optimize vulnerabilities. Summary of the Invention

[0004] To address the shortcomings of the existing technology, this application provides a differential privacy parameter protection method in AI model iteration. The method includes: before each AI model iteration, using a clustering algorithm to identify the gradient distribution and iteration amplitude of the parameters to dynamically divide the sensitivity levels of the parameters, and dynamically dividing the training phases according to the training progress of the AI model. The training phases and parameter sensitivity levels are analyzed to comprehensively allocate the privacy budget and noise amount for each parameter;

[0005] After each AI model iteration, the cumulative privacy leakage of each parameter in the historical AI model iteration is calculated based on the privacy budget of each parameter. The cumulative privacy leakage of each parameter is determined to trigger compensation for the noise amount of each parameter in the next round of AI model iteration. The compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase, and the compensated noise parameter set is output;

[0006] Output the AI model based on the loss changes of the AI model iteration, generate an adversarial sample set based on the noise parameter set, perform a simulated attack test based on the adversarial sample set through member inference attack, and determine the attack success rate in the simulated attack test. Analyze the attack success rate to trigger vulnerability detection and locate the vulnerability noise parameters at the same time;

[0007] The SHAP value is used to analyze the contribution rate of each parameter to the AI model. The noise amount of each parameter is comprehensively updated based on the contribution rate and vulnerability noise parameter to optimize the noise parameter set.

[0008] As an optional implementation, the logic for allocating the privacy budget and noise amount for each parameter includes:

[0009] Based on the index mechanism of differential privacy, the importance of each parameter to the AI model during the training phase is analyzed, and the utility measurement value of each parameter is determined comprehensively based on the importance of each parameter to the AI model and the sensitivity value of each parameter;

[0010] Based on the utility metric of each parameter, the probability of obtaining the privacy budget for each parameter through the exponential mechanism of differential privacy is calculated;

[0011] A privacy budget is allocated to each parameter according to the probability of each parameter obtaining a privacy budget, and a noise amount is allocated to each parameter based on the sensitivity level of the parameter and the privacy budget allocated to each parameter.

[0012] As an optional implementation, the sub-logic of dividing the sensitivity level of the parameter includes:

[0013] Before each AI model iteration, the gradient distribution of the parameters is clustered using the DBSCAN density clustering algorithm to divide it into multiple clusters. The neighborhood radius of the DBSCAN density clustering algorithm is dynamically adjusted according to the compactness of the gradient distribution of the parameters.

[0014] Calculate the average iteration amplitude of the parameters within each cluster, and determine the degree of fluctuation of the iteration amplitude in combination with the average iteration amplitude of the parameters within each cluster;

[0015] The tightness of the parameter gradient distribution, the average iteration amplitude and the fluctuation degree of the iteration amplitude are weighted and summed to comprehensively determine the sensitivity value of each parameter. The sensitivity level of the parameter is dynamically divided according to the sensitivity value of each parameter.

[0016] As an optional implementation, the division sub-logic of the training phase includes:

[0017] Build a training progress indicator system, including the number of AI model iterations, the completion rate of training data, and the change in loss between AI model iterations;

[0018] The training progress is evaluated by fuzzy comprehensive evaluation method, and the weight vector of each indicator system is determined based on the hierarchical analysis method;

[0019] Fuzzy judgment rules are set for each indicator system respectively to obtain the evaluation results of each indicator system, and the evaluation results of each indicator system and the weight vector are weighted and summed to obtain a comprehensive evaluation value. The training stage is divided according to the comprehensive evaluation value.

[0020] As an optional implementation manner, the output logic of the noise parameter set includes:

[0021] Set a privacy leakage threshold for each parameter based on the privacy protection requirements of the training phase and the AI model. Compare the cumulative privacy leakage of each parameter with the privacy leakage threshold of each parameter to determine whether to trigger compensation for the noise amount of each parameter in the next round of AI model iteration;

[0022] When it is necessary to trigger compensation for the noise amount of each parameter in the next round of AI model iteration, the compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase;

[0023] Locate the parameters that need to compensate for noise in the next round of AI model iteration, determine the compensated noise amount based on the noise compensation strength, summarize the noise amounts after all parameters are compensated, and output the compensated noise parameter set.

[0024] As an optional implementation manner, the calculation sub-logic of the cumulative privacy leakage amount of each parameter includes:

[0025] After each AI model iteration, a privacy leakage risk assessment matrix is constructed based on the degree of change in the gradient distribution of parameters, the degree of correlation between parameters, and the training stage of the current AI model iteration;

[0026] Calculate the privacy leakage of each parameter after each AI model iteration based on the privacy budget of each parameter and the privacy leakage risk assessment matrix;

[0027] The iteration weight factor after each AI model iteration is determined according to the inverse of the number of iterations of the AI model. The privacy leakage amount of each parameter after each AI model iteration is multiplied by the iteration weight factor, and the sum is accumulated to obtain the cumulative privacy leakage amount of each parameter.

[0028] As an optional implementation manner, the vulnerability noise parameter location logic includes:

[0029] Set an attack success rate threshold based on the privacy protection requirements of the AI model and historical attack test data. When the attack success rate exceeds the attack success rate threshold, vulnerability detection is triggered.

[0030] During the simulated attack test, the value changes of each parameter under different adversarial sample sets and the attack results are recorded. Statistical analysis is performed to calculate the correlation coefficient between the value changes of each parameter under different adversarial sample sets and the attack success rate. Candidate vulnerability noise parameters are screened based on the correlation coefficient.

[0031] During the AI model training phase, the normal value of each parameter is collected to determine the normal behavior of the parameter. During the simulated attack test, the degree of deviation of each parameter from the normal behavior is determined to determine the abnormal parameter.

[0032] A parameter dependency graph is constructed through a graph neural network. The nodes of the parameter dependency graph represent parameters, the edges of the parameter dependency graph represent the dependency relationship between parameters, and the weights of the edges of the parameter dependency graph represent the dependency strength between parameters.

[0033] Traverse the abnormal parameters of the parameter dependency graph, trace the vulnerability propagation path, and locate the vulnerability noise parameters.

[0034] As an optional implementation manner, the sub-logic for determining the attack success rate includes:

[0035] Configure a loss change threshold. After each AI model iteration, calculate the difference between the loss value of the current AI model iteration and the loss value of the previous AI model iteration. If the loss difference is less than the loss change threshold for multiple consecutive times, the current AI model is considered to have converged and is selected as the output AI model.

[0036] The noise parameter set is stratified according to the training stage and the parameter sensitivity level, and the adversarial sample set is generated by combining the stratification results of the noise parameter set;

[0037] Conduct simulated attack tests based on adversarial sample sets through member inference attacks, and dynamically adjust the generation method and attack parameters of adversarial sample sets based on the changes in AI model iterations during the simulated attack tests.

[0038] According to the results of the simulated attack test, the ratio of the number of samples with successful attacks to the total number of samples in the adversarial sample set is calculated to determine the attack success rate in the simulated attack test.

[0039] As an optional implementation, the optimization logic of the noise parameter set includes:

[0040] Update the usage contribution rate of each parameter according to the vulnerability noise parameter to obtain the update factor of each parameter;

[0041] Determining the adjustment direction of the noise amount of each parameter according to the update factor of each parameter;

[0042] Determine the adjustment range of the noise amount of each parameter based on the training stage and the cumulative privacy leakage of each parameter;

[0043] The noise amount of each parameter is updated in combination with the adjustment direction and adjustment amplitude of the noise amount of each parameter to optimize the noise parameter set.

[0044] As an optional implementation, the analysis sub-logic of the usage contribution rate includes:

[0045] Collect the adversarial sample sets of the AI model during the simulated attack test and the values of each parameter under different adversarial sample sets, and integrate them into sample data;

[0046] The sample data is stratified according to the training stage and the sensitivity level of the parameters, and a fixed number of sample data are randomly selected from each layer;

[0047] Based on the combination of parallel computing and approximate computing, the SHAP algorithm is used to calculate the SHAP value of each parameter on the sample data;

[0048] Analyze the SHAP value of each parameter on the sample data to obtain the absolute value and variance of the SHAP value. Analyze the absolute value and variance of the SHAP value to obtain the contribution rate of each parameter to the use of the AI model.

[0049] Compared with the existing technology, the beneficial effects of this application are: by organically combining the four steps of parameter preprocessing before AI model iteration, privacy monitoring and compensation during iteration, and vulnerability detection and parameter optimization after iteration, a privacy protection system covering the entire iteration cycle of the AI model is constructed, avoiding privacy protection gaps or loopholes, and ensuring that parameter privacy can be effectively protected at all stages of AI model training and application; the steps are interrelated and dynamically coordinated, and can adaptively adjust privacy protection strategies and parameter settings in real time according to multiple factors such as AI model training progress, parameter sensitivity changes, privacy leakage conditions, and attack test results. This dynamism enables the protection method to better adapt to the complex and changeable AI model training environment, and improve the pertinence and effectiveness of privacy protection; from discovering vulnerabilities through attack testing, to locating vulnerability noise parameters, and then to optimizing the noise parameter set based on parameter contribution rate and vulnerability conditions, a complete vulnerability prevention and repair closed loop is formed, which helps to timely discover and solve potential privacy and security issues in AI models, and continuously improve the security and reliability of AI models.

[0050] By identifying the gradient distribution and iteration amplitude of parameters through clustering algorithms and dynamically dividing the sensitivity levels of parameters, the parameters can be accurately classified according to their actual behavioral characteristics during the training process, making subsequent privacy protection measures more targeted. The privacy budget and noise amount are comprehensively allocated based on the training stage and the parameter sensitivity level, realizing the rational allocation of privacy protection resources. For parameters with high sensitivity and parameters in the critical training stage, more privacy budget and noise amount are allocated, thereby improving the utilization efficiency of privacy protection resources.

[0051] The cumulative privacy leakage of parameters is calculated after each AI model iteration, which can monitor the privacy status of parameters in real time, promptly discover potential privacy leakage risks, and provide timely data support for taking corresponding protection measures; the noise compensation is triggered according to the cumulative privacy leakage, and the compensation intensity is dynamically adjusted according to the training stage, so that the AI model can respond quickly when facing privacy leakage, enhance privacy protection by adjusting the noise amount, and avoid unnecessary impact on AI model performance caused by excessive compensation.

[0052] Based on the noise parameter set, an adversarial sample set is generated for simulated attack testing. By analyzing the attack success rate to trigger vulnerability detection, privacy vulnerabilities in the AI model can be effectively discovered, providing a clear target for subsequent repair work; using a variety of technical means to locate the vulnerability noise parameters helps to gain an in-depth understanding of the root cause of the vulnerability, providing an accurate parameter basis for targeted vulnerability repair and optimization of the AI model, and improving the security and stability of the AI model.

[0053] By analyzing the contribution rate of each parameter to the use of the AI model through the SHAP value, the importance of the parameter to the AI model is scientifically evaluated from a quantitative perspective, providing an objective basis for optimizing the noise parameter set; the noise amount of each parameter is comprehensively updated according to the use contribution rate and vulnerability noise parameters, realizing the optimized configuration of the noise parameter set, enabling the AI model to better exert its performance advantages while ensuring privacy, thereby improving the overall quality of the AI model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be derived from these drawings without inventive work. Among them:

[0055] Figure 1 Flowchart of the differential privacy parameter protection method in AI model iteration provided in an embodiment of the present application;

[0056] Figure 2A sub-logic diagram for dividing the sensitivity levels of parameters in the differential privacy parameter protection method in the AI model iteration provided in an embodiment of the present application;

[0057] Figure 3 A sub-logic diagram for determining the attack success rate of the differential privacy parameter protection method in the AI model iteration provided in an embodiment of the present application;

[0058] Figure 4 This is a sub-logic diagram for analyzing the contribution rate of the differential privacy parameter protection method in the AI model iteration provided in the embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are clearly and completely described below in conjunction with the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0060] like Figure 1 As shown, it is a flowchart of the differential privacy parameter protection method in the AI model iteration provided by an embodiment of the present application. The differential privacy parameter protection method in the AI model iteration includes:

[0061] S1. Before each AI model iteration, a clustering algorithm is used to identify the gradient distribution and iteration amplitude of the parameters to dynamically divide the sensitivity levels of the parameters. The training phases are dynamically divided according to the training progress of the AI model. The training phases and parameter sensitivity levels are analyzed to comprehensively allocate the privacy budget and noise amount for each parameter.

[0062] Furthermore, if Figure 2 As shown, the sub-logic of parameter sensitivity level division includes:

[0063] Before each AI model iteration, the gradient distribution of the parameters is clustered using the DBSCAN density clustering algorithm to divide it into multiple clusters. The neighborhood radius of the DBSCAN density clustering algorithm is dynamically adjusted according to the compactness of the gradient distribution of the parameters.

[0064] Calculate the average iteration amplitude of the parameters within each cluster, and determine the degree of fluctuation of the iteration amplitude in combination with the average iteration amplitude of the parameters within each cluster;

[0065] The tightness of the parameter gradient distribution, the average iteration amplitude and the fluctuation degree of the iteration amplitude are weighted and summed to comprehensively determine the sensitivity value of each parameter. The sensitivity level of the parameter is dynamically divided according to the sensitivity value of each parameter.

[0066] There are many parameters in the AI model, and the behavior of different parameters during the training process varies significantly. In order to achieve more refined privacy protection, the parameters need to be classified so that the privacy budget and noise amount can be allocated in a targeted manner. The gradient distribution of the parameters reflects the changing trend and activity level of the parameters during the training process. By analyzing them through clustering algorithms, parameters with similar gradient distribution characteristics can be grouped into a cluster. The neighborhood radius is dynamically adjusted because the distribution of parameters changes dynamically during the training process. A fixed neighborhood radius cannot adapt to such changes, which will lead to inaccurate clustering results.

[0067] Before each iteration of the AI model, the gradient data of all parameters are obtained and used as the input of the DBSCAN density clustering algorithm. At the beginning of the DBSCAN density clustering algorithm, an unvisited data point (that is, parameter) is randomly selected as the core point, and then all density-reachable points in the neighborhood of the core point are found, and these points are formed into a cluster. During the clustering process, the number of data points in the neighborhood of each data point is calculated in real time to evaluate the density of the area. If the density of data points (parameters) in a certain area is high, it means that the parameter distribution in this area is relatively dense. At this time, the neighborhood radius is appropriately reduced to make the clustering more refined and avoid mistakenly classifying parameters with large differences into one category. Conversely, if the data point density is low, the neighborhood radius is increased to ensure that parameters with similar characteristics can be included in the same cluster.

[0068] The DBSCAN density clustering algorithm, which dynamically adjusts the neighborhood radius, can more accurately identify the gradient distribution characteristics of parameters and group parameters with similar behavior into the same cluster. This refined clustering approach makes subsequent parameter sensitivity assessments more accurate, laying a solid foundation for the proper allocation of privacy budgets and noise levels. Accurate clustering results provide reliable data grouping for calculating the average iteration amplitude of parameters within each cluster. Based on reasonable clustering, the subsequent calculations of the average iteration amplitude and iteration amplitude fluctuation can more accurately reflect the actual changes in the parameters, thereby improving the accuracy of parameter sensitivity calculations.

[0069] The parameter iteration amplitude reflects the magnitude of the parameter change in each iteration. The average iteration amplitude measures the overall level of parameter change within a cluster, while the fluctuation of the iteration amplitude reflects the stability of the parameter change. These two metrics are crucial for assessing parameter sensitivity. Parameters with large fluctuation amplitudes and unstable fluctuations are more likely to leak private information during training and require more attention and protection. For each cluster, all parameters within the cluster are traversed, and their parameter values before the current iteration and after the previous iteration are obtained. The iteration amplitude of each parameter is calculated as the absolute value of the difference between the current parameter value and the previous parameter value. The iteration amplitudes of all parameters are then added and divided by the number of parameters within the cluster to obtain the average iteration amplitude of the parameters within the cluster. To determine the degree of iteration amplitude fluctuation, the square of the difference between the iteration amplitude of each parameter and the average iteration amplitude is calculated. These squared values are added and divided by the number of parameters within the cluster. Finally, the square root of the result is taken to obtain the standard deviation, which is used as a measure of the degree of iteration amplitude fluctuation. A larger standard deviation indicates more severe iteration amplitude fluctuation.

[0070] By calculating the average iteration amplitude and the degree of fluctuation of the iteration amplitude, we can gain an in-depth understanding of the changing characteristics of the parameters within each cluster from a quantitative perspective, which makes the evaluation of parameter sensitivity more comprehensive and objective, avoids the limitation of judging based on only a single indicator, and improves the accuracy of parameter classification. The obtained average iteration amplitude and the degree of fluctuation of the iteration amplitude, together with the tightness of the parameter gradient distribution, serve as an important basis for calculating the parameter sensitivity value. The accuracy of these two indicators can make the sensitivity value more realistically reflect the actual sensitivity of the parameter, thereby providing accurate data support for the subsequent dynamic division of the parameter sensitivity level according to the sensitivity value.

[0071] To provide targeted privacy protection for parameters, parameters need to be graded according to their sensitivity. This allows for prioritizing sensitive parameters when allocating privacy budgets and noise levels. The sensitivity value is calculated using multiple indicators obtained in the previous steps, such as the tightness of the parameter gradient distribution, the average iteration amplitude, and the fluctuation of the iteration amplitude. This allows for a more comprehensive assessment of the parameter sensitivity. A corresponding weight is assigned to each indicator, which is optimized through expert experience or a machine learning algorithm. The tightness of the parameter gradient distribution, the average iteration amplitude, and the fluctuation of the iteration amplitude are weighted and summed according to the set weights to obtain the sensitivity value of each parameter. For example, the tightness of the parameter gradient distribution has a weight of 0.4, the average iteration amplitude has a weight of 0.3, and the fluctuation of the iteration amplitude has a weight of 0.3. Parameters are then divided into three sensitivity levels: high, medium, and low, based on the sensitivity value. The sensitivity thresholds include minimum and maximum sensitivity. A sensitivity value greater than the maximum sensitivity is considered high, a sensitivity value less than the minimum sensitivity is considered low, and a sensitivity value between the two is considered medium.

[0072] By calculating the sensitivity value based on multiple indicators and dividing it into sensitivity levels, the importance of parameters in privacy protection can be more accurately reflected. This grading method enables a more reasonable allocation of privacy protection resources, improving the efficiency and effectiveness of overall privacy protection. The clear parameter sensitivity level division results provide a key parameter classification basis for subsequent analysis of the training stage and the parameter sensitivity level to comprehensively allocate the privacy budget and noise amount of each parameter. Parameters of different sensitivity levels will have different strategies for allocating privacy budget and noise amount. Parameters with high sensitivity will be allocated more privacy budget and noise amount to enhance their privacy protection.

[0073] Furthermore, the sub-logic of the training phase includes:

[0074] Build a training progress indicator system, including the number of AI model iterations, the completion rate of training data, and the change in loss between AI model iterations;

[0075] The training progress is evaluated by fuzzy comprehensive evaluation method, and the weight vector of each indicator system is determined based on the hierarchical analysis method;

[0076] Fuzzy judgment rules are set for each indicator system respectively to obtain the evaluation results of each indicator system, and the evaluation results of each indicator system and the weight vector are weighted and summed to obtain a comprehensive evaluation value. The training stage is divided according to the comprehensive evaluation value.

[0077] The training of AI models is a dynamic process. The training characteristics and requirements of models at different stages are different, and the requirements for parameter privacy protection are also different. In order to more reasonably allocate the privacy budget and noise amount, it is necessary to accurately judge the training stage of the AI model. By constructing a system that includes indicators such as the number of iterations of the AI model, the completion ratio of training data, and the loss change of the AI model iteration, the training progress can be comprehensively evaluated from multiple dimensions. During the AI model training process, the number of iterations of the AI model is recorded in real time. After each training data processing is completed, the completion ratio of the training data is updated. At the same time, after each iteration, the loss value of the current iteration is calculated and compared with the previously recorded loss value to obtain the loss change of the AI model iteration. These three indicators are integrated together to form an indicator system for training progress.

[0078] This indicator system reflects the status of model training from different angles and provides comprehensive data support for accurately judging the training stage. By comprehensively considering these three indicators, we can more accurately grasp the progress of model training and avoid the errors caused by judging the training stage based on a single indicator. The constructed indicator system provides specific evaluation objects and data basis for the subsequent evaluation of training progress through fuzzy comprehensive evaluation method. Based on these indicators, we can further determine the weight of each indicator in evaluating the training progress, and perform fuzzy judgment and comprehensive calculation to accurately divide the training stage.

[0079] The evaluation of training progress involves multiple indicator systems, and the nature and importance of these indicator systems vary, making it difficult to conduct direct and accurate quantitative comparisons. The fuzzy comprehensive evaluation method can handle fuzzy and uncertain information. By introducing weight vectors, it can comprehensively consider the importance of each indicator system and more accurately evaluate training progress. Determining weight vectors based on the hierarchical analysis method can transform complex multi-indicator decision-making problems into a hierarchical structure. The relative importance of each indicator is determined through pairwise comparison, making the determination of weights more scientific and reasonable. First, using the hierarchical analysis method, the training progress evaluation problem is decomposed into a target layer (training progress evaluation), a criterion layer (the number of AI model iterations, the completion rate of training data, and the loss change of AI model iterations), and an indicator layer (specific indicator data). Then, through expert consultation or analysis of historical training data, a pairwise comparison matrix is constructed between the indicator systems in the criterion layer. The weight vector of each indicator system is calculated based on the comparison matrix. For example, the maximum eigenvalue and corresponding eigenvector of the matrix are calculated and normalized to obtain the weight vector of each indicator system. The obtained weight vector is then applied to the fuzzy comprehensive evaluation method to weight the evaluation results of each indicator system.

[0080] The fuzzy comprehensive evaluation method combined with the hierarchical analysis method is used to determine the weight vector, which can fully consider the fuzziness and importance differences of each indicator in the training progress evaluation, so that the evaluation results can more objectively and accurately reflect the actual training stage of the model. This method avoids subjective arbitrariness and improves the scientificity and reliability of the division of training stages. The accurately determined weight vector and evaluation method provide key parameters and methodological basis for the subsequent setting of fuzzy judgment rules and comprehensive calculations. The weighted summation of the evaluation results of each indicator based on the weight vector can obtain a more accurate comprehensive evaluation value, thereby realizing the precise division of training stages.

[0081] In order to convert the data in the training progress indicator system into specific training stage judgments, it is necessary to establish corresponding judgment rules. Since the change in training progress is a continuous process and it is difficult to divide it with precise boundaries, fuzzy judgment rules can better adapt to this fuzziness and more accurately reflect the actual situation of the training stage. Fuzzy judgment rules are set for the three indicator systems of the number of iterations of the AI model, the completion ratio of training data and the loss change of the AI model iteration. For example, for the number of iterations of the AI model, when the number of iterations is small, the membership of the "initial stage" is set to be higher. As the number of iterations increases, the membership of the "initial stage" gradually decreases, and the membership of the "intermediate stage" and "late stage" gradually increases. For the completion ratio of training data, the membership of the "initial stage" gradually decreases, and the membership of the "middle stage" and "late stage" gradually increases. When the completion ratio is low, the membership of the "initial stage" is high, and when it is close to completion, the membership of the "later stage" is high; for the loss change of the AI model iteration, when the loss change is large, the membership of the "initial stage" is high, among which the large loss change can be obtained by comparing the loss change with the loss change threshold, and when the loss change gradually stabilizes, the membership of the "later stage" is high; then, according to the evaluation results of each indicator and the set fuzzy judgment rules, the membership of each indicator in different training stages is obtained; finally, the membership of each indicator in different training stages is weightedly summed with the corresponding weight vector to obtain a comprehensive evaluation value. According to the size of the comprehensive evaluation value, the training process is divided into different stages such as the initial exploration stage, the stable training stage and the fine adjustment stage.

[0082] By setting fuzzy judgment rules, we can more flexibly handle the ambiguity and uncertainty in training progress evaluation, making the division of training stages more in line with the actual characteristics of AI model training. This division method can provide an accurate stage basis for the subsequent reasonable allocation of privacy budget and noise amount according to the characteristics of different training stages. The clear training stage division results, combined with the parameter sensitivity level division results obtained previously, provide two key reference dimensions for the comprehensive allocation of privacy budget and noise amount for each parameter. In different training stages, for parameters of different sensitivity levels, different privacy budget and noise amount allocation logic can be used to achieve more effective privacy protection and model performance balance.

[0083] Specifically, the privacy budget and noise allocation logic for each parameter includes:

[0084] Based on the index mechanism of differential privacy, the importance of each parameter to the AI model during the training phase is analyzed, and the utility measurement value of each parameter is determined comprehensively based on the importance of each parameter to the AI model and the sensitivity value of each parameter;

[0085] Based on the utility metric of each parameter, the probability of obtaining the privacy budget for each parameter through the exponential mechanism of differential privacy is calculated;

[0086] A privacy budget is allocated to each parameter according to the probability of each parameter obtaining a privacy budget, and a noise amount is allocated to each parameter based on the sensitivity level of the parameter and the privacy budget allocated to each parameter.

[0087] In differential privacy protection, in order to reasonably allocate the privacy budget, it is necessary to evaluate the importance and sensitivity of each parameter to the AI model, so as to determine the priority of each parameter in privacy protection. By comprehensively considering the importance and sensitivity value of each parameter to the AI model and calculating the utility measurement value, a quantitative basis can be provided for the allocation of the privacy budget, so that the privacy budget can be more reasonably allocated to each parameter. Based on the index mechanism of differential privacy, the importance of each parameter to the AI model is first determined. The importance is determined by analyzing factors such as the position of the parameter in the AI model structure and the degree of influence on the model output results. For example, for the key layer parameters in the neural network, their importance is relatively high, and for some auxiliary parameters, their importance is relatively low. Then, combined with the previous steps, we can get The sensitivity value of each parameter is calculated, and the importance and sensitivity values are comprehensively calculated to obtain the utility measurement value of each parameter. A weighted summation method is adopted. For example, the importance weight is 0.6 and the sensitivity weight is 0.4. The utility measurement value is obtained by weighting the two. By calculating the utility measurement value, the importance and sensitivity of the parameters can be quantitatively integrated, providing a unified evaluation standard for the allocation of the privacy budget. This quantitative evaluation method makes the allocation of the privacy budget more scientific and reasonable, avoids blind allocation, and improves the efficiency of the use of the privacy budget. The utility measurement value of each parameter is the basis for the subsequent calculation of the parameter to obtain the privacy budget probability. Calculating the probability based on the utility measurement value can ensure that the privacy budget is allocated according to the actual importance and sensitivity of the parameter, thereby achieving more effective privacy protection.

[0088] Under the exponential mechanism of differential privacy, in order to achieve privacy protection of parameters, it is necessary to determine the probability of each parameter obtaining a privacy budget based on the utility measurement value of the parameter. Through the probability distribution method, the impact on the performance of the AI model can be minimized while ensuring a certain privacy protection effect, so that privacy protection and model performance are balanced; according to the utility measurement value of each parameter, the exponential mechanism formula of differential privacy is used to calculate the probability of each parameter obtaining a privacy budget. The basic principle of the exponential mechanism formula is that the higher the utility measurement value of the parameter, the greater the probability of obtaining a privacy budget. During the specific calculation, the utility measurement value of each parameter is exponentially calculated, and then the exponential calculation results of all parameters are normalized so that all parameters obtain a privacy budget. The sum of the probabilities is 1, so we get the probability of each parameter obtaining the privacy budget under the current circumstances; by calculating the probability based on the utility metric, we achieve fair and reasonable distribution of the privacy budget. Parameters with high utility metric values obtain more privacy budget and can better protect their privacy. Parameters with low utility metric values obtain less privacy budget, which reduces the impact on model performance to a certain extent and helps to balance privacy protection and model performance. The probability of each parameter obtaining the privacy budget is the direct basis for subsequently allocating the privacy budget to each parameter according to probability. After allocating the privacy budget according to probability, combined with the parameter sensitivity level, we can further determine the amount of noise that should be allocated to each parameter, completing the entire privacy budget and noise allocation process.

[0089] After determining the probability of each parameter obtaining the privacy budget, the privacy budget needs to be allocated to each parameter according to the probability to achieve privacy protection of the parameter. At the same time, according to the sensitivity level of the parameter and the allocated privacy budget, a corresponding amount of noise is allocated to each parameter, which can further enhance the privacy protection effect and balance privacy protection and model performance to a certain extent. According to the probability of each parameter obtaining the privacy budget calculated above, the total privacy budget is allocated to each parameter. If the total privacy budget is B and the probability of a parameter obtaining the privacy budget is P, then the privacy budget allocated to the parameter is B×P. Then, according to the sensitivity level of the parameter, the amount of noise is allocated to the parameters allocated to the privacy budget. For parameters with high sensitivity levels, since they are more likely to leak privacy, more noise will be allocated when allocated to the same privacy budget. For parameters with low sensitivity levels, relatively less noise will be allocated. At the same time, considering the amount of privacy budget allocated, parameters with large privacy budgets will also have an appropriate increase in noise to better protect their privacy.

[0090] Personalized privacy protection of parameters is achieved by allocating privacy budgets according to probability and allocating noise amounts based on parameter sensitivity levels and privacy budgets. This allocation method can reasonably allocate privacy protection resources based on the actual conditions of the parameters. While effectively protecting parameter privacy, it minimizes the impact on model performance, thereby improving the overall privacy protection effect and the usability of the AI model. After completing the allocation of privacy budgets and noise amounts for each parameter, it lays the foundation for further adjusting the noise amount according to privacy leakage and attack test results during the subsequent model iteration process. Reasonable initial allocation can enable the model to obtain better privacy protection in the early stages of training, and make subsequent adjustments more targeted and effective.

[0091] S2. After each AI model iteration, the cumulative privacy leakage of each parameter in the historical AI model iteration is calculated based on the privacy budget of each parameter, and the cumulative privacy leakage of each parameter is determined to trigger compensation for the noise amount of each parameter in the next round of AI model iteration. The compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase, and the compensated noise parameter set is output.

[0092] Furthermore, the calculation sub-logic of the cumulative privacy leakage of each parameter includes:

[0093] After each AI model iteration, a privacy leakage risk assessment matrix is constructed based on the degree of change in the gradient distribution of parameters, the degree of correlation between parameters, and the training stage of the current AI model iteration;

[0094] Calculate the privacy leakage of each parameter after each AI model iteration based on the privacy budget of each parameter and the privacy leakage risk assessment matrix;

[0095] The iteration weight factor after each AI model iteration is determined according to the inverse of the number of iterations of the AI model. The privacy leakage amount of each parameter after each AI model iteration is multiplied by the iteration weight factor, and the sum is accumulated to obtain the cumulative privacy leakage amount of each parameter.

[0096] During the iteration process of an AI model, the privacy leakage risk of different parameters is affected by multiple factors, and a single metric cannot fully assess it. To accurately quantify the privacy leakage risk of each parameter in each iteration, it is necessary to comprehensively consider the parameter's own variation characteristics, its correlation with other parameters, and the training stage. This is done by constructing a privacy leakage risk assessment matrix, providing a multi-dimensional assessment basis for subsequent calculations of privacy leakage. After each AI model iteration, the degree of change in the parameter gradient distribution is first analyzed. By calculating the difference in the parameter gradient distribution between the current iteration and the previous iteration, the parameter's activity level is determined. The greater the gradient change, the more active the parameter, and the higher the privacy leakage risk. Secondly, the degree of correlation between parameters is considered. Using methods such as correlation analysis, the correlation coefficient between the parameter and other parameters is calculated. If one of the closely related parameters is leaked, the privacy leakage risk of the other related parameters increases. Finally, considering the training stage of the current AI model iteration, the model is less sensitive to parameter changes in the early stages of training, and the privacy leakage risk is relatively small. In the later stages of training, when the AI model is nearing convergence, even small changes in the parameters will affect model performance, and the privacy leakage risk is higher. Using these three factors as the dimensions of the matrix, each parameter is evaluated and scored to construct a privacy leakage risk assessment matrix.

[0097] This risk assessment matrix comprehensively evaluates the privacy leakage risk of parameters from multiple perspectives, avoiding the one-sidedness of judgment based on a single factor. It converts complex risk factors into computable matrix data in a quantitative manner, making the assessment of the privacy leakage risk of each parameter more accurate and objective, providing a reliable basis for the subsequent precise calculation of the amount of privacy leakage. The constructed privacy leakage risk assessment matrix provides specific risk assessment data for calculating the amount of privacy leakage based on the privacy budget of each parameter. Each element in the matrix corresponds to the risk value of a parameter under a specific influencing factor. These values will directly participate in the calculation of the amount of privacy leakage and determine the degree of privacy leakage of each parameter in this iteration.

[0098] Given the privacy budget and privacy leakage risk assessment matrix of each parameter, in order to timely understand the privacy leakage of the parameters in each iteration, it is necessary to calculate the privacy leakage of each parameter after this iteration, so as to cumulatively track the privacy leakage of the parameters and provide data support for the subsequent adjustment of the privacy protection strategy; according to the corresponding values in the privacy budget and privacy leakage risk assessment matrix of each parameter, the privacy leakage amount is calculated using a proportional calculation method. If the privacy budget of a parameter is a fixed value, the risk assessment value of privacy leakage in this iteration is the value of the corresponding element in the matrix. The risk assessment value is compared with the privacy budget, and the privacy leakage amount of the parameter in this iteration is calculated according to a certain ratio. The specific calculation method is that the privacy leakage amount is equal to the privacy budget multiplied by the proportion of the risk assessment value in the sum of the risk assessment values of all parameters, so as to determine the degree of privacy leakage of each parameter in this iteration.

[0099] By calculating the amount of privacy leakage after each iteration, the privacy protection status of each parameter can be monitored in real time, and parameters with higher privacy leakage risks can be discovered in a timely manner. This quantitative calculation method makes the privacy leakage situation clearer and more intuitive, facilitates subsequent targeted protection and adjustment of parameters, and improves the timeliness and effectiveness of privacy protection. The privacy leakage amount obtained each time is the basic data for calculating the cumulative privacy leakage amount. This data will be combined with the privacy leakage amount of the previous iteration and accumulated by introducing an iterative weight factor to obtain the cumulative privacy leakage amount of each parameter, providing a key basis for determining whether noise compensation is needed.

[0100] Only knowing the amount of privacy leakage in each iteration cannot fully grasp the privacy leakage of the parameters in the entire training process. It is necessary to accumulate the privacy leakage in each iteration to form the cumulative privacy leakage in order to comprehensively evaluate the privacy risk of the parameters and provide an accurate judgment standard for the subsequent triggering of the noise compensation mechanism; the iteration weight factor after each AI model iteration is determined according to the inverse of the number of iterations of the AI model. As the number of iterations increases, the impact of early iterations on the current privacy leakage is relatively small, so the iteration weight factor decreases with the increase in the number of iterations. The privacy leakage of each parameter after each AI model iteration is multiplied by the corresponding iteration weight factor, and then accumulated to obtain the cumulative privacy leakage of each parameter. For example, the privacy leakage of the first iteration is multiplied by 1, the privacy leakage of the second iteration is multiplied by 1 / 2, the privacy leakage of the third iteration is multiplied by 1 / 3, and so on. The results of all iterations are added together to obtain the cumulative privacy leakage of each parameter in the current training stage.

[0101] The cumulative privacy leakage comprehensively considers the privacy leakage of parameters during the entire training process, and can more comprehensively reflect the privacy risk level of the parameters. By introducing the iteration weight factor, the impact of different iteration stages on the cumulative results is reasonably distributed, making the cumulative calculation results more in line with the actual situation, and providing a more reliable basis for subsequent decision-making. The calculated cumulative privacy leakage of each parameter is the key basis for determining whether to trigger compensation for the noise amount of each parameter in the next round of AI model iteration. The cumulative privacy leakage is compared with the pre-set privacy leakage threshold to determine whether the parameter noise amount needs to be compensated and how to adjust the compensation intensity according to the training stage.

[0102] Specifically, the output logic of the noise parameter set includes:

[0103] Set a privacy leakage threshold for each parameter based on the privacy protection requirements of the training phase and the AI model. Compare the cumulative privacy leakage of each parameter with the privacy leakage threshold of each parameter to determine whether to trigger compensation for the noise amount of each parameter in the next round of AI model iteration;

[0104] When it is necessary to trigger compensation for the noise amount of each parameter in the next round of AI model iteration, the compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase;

[0105] Locate the parameters that need to compensate for noise in the next round of AI model iteration, determine the compensated noise amount based on the noise compensation strength, summarize the noise amounts after all parameters are compensated, and output the compensated noise parameter set.

[0106] In order to clarify under what circumstances the parameter noise needs to be compensated, it is necessary to set a privacy leakage threshold for each parameter according to the training stage and the privacy protection requirements of the AI model. By comparing the cumulative privacy leakage of each parameter with the threshold, it is possible to quickly determine whether the compensation mechanism needs to be triggered to ensure timely protection of parameters with higher privacy leakage risks; according to different training stages and combined with the privacy protection requirements of the AI model, a corresponding privacy leakage threshold is set for each parameter. In the early stages of training, the model has a relatively high tolerance for privacy leakage, so a higher threshold is set; in the later stages of training, in order to ensure the security of the model, the threshold should be set lower, and the cumulative privacy leakage of each parameter is compared with the corresponding privacy leakage threshold. If the cumulative privacy leakage is greater than the privacy leakage threshold, it is determined that compensation for the noise of the parameter in the next round of AI model iteration needs to be triggered; otherwise, no compensation is performed.

[0107] By setting a privacy leakage threshold and performing comparative judgment, precise control of noise compensation can be achieved, unnecessary noise compensation can be avoided, and the impact on AI model performance can be reduced. At the same time, timely measures can be taken when the risk of parameter privacy leakage is high, effectively balancing privacy protection and AI model performance. The judgment result determines whether noise compensation is required for the parameters, as well as the subsequent adjustment of the noise compensation intensity. If compensation needs to be triggered, the compensation intensity will be dynamically adjusted according to the division results of the training phase to more reasonably protect parameter privacy; if compensation is not required, the current noise parameter set will continue to be used for the next round of iteration.

[0108] The AI model has different sensitivities to parameters at different training stages. In order to effectively protect privacy while minimizing the impact on the performance of the AI model, it is necessary to dynamically adjust the noise compensation intensity based on the division results of the training stage, so that at different stages of training, noise compensation resources can be reasonably allocated according to actual needs to achieve the best balance between privacy protection and AI model performance; in the early stages of training, since the AI model is still in the stage of learning data features, its tolerance for noise is relatively high. At this time, the noise compensation intensity is appropriately reduced to reduce the interference of noise on the AI model learning process, allowing the AI model to converge faster. In the later stages of training, the AI model is close to convergence and is more sensitive to parameter changes. In order to ensure parameter privacy and security, the noise compensation intensity needs to be increased to improve the level of privacy protection. The specific adjustment method is to set different compensation intensity coefficients according to the training stage. For example, the compensation intensity coefficient at the beginning of training is 0.5, and the coefficient at the end of training is 1.5. The originally calculated noise compensation amount is multiplied by the corresponding compensation intensity coefficient to obtain the adjusted compensation intensity.

[0109] Dynamically adjusting the noise compensation intensity can flexibly adjust the privacy protection strategy according to the actual situation of model training, promote the rapid learning of the AI model in the early stage of training, and ensure the privacy security of the AI model in the later stage of training. It effectively avoids the problems of over-protection or under-protection caused by fixed compensation intensity, and improves the adaptability and effectiveness of the privacy protection scheme. The adjusted noise compensation intensity provides a key parameter for determining the compensated noise amount. Combined with the parameters that need to compensate for the noise amount, the specific compensated noise amount is calculated according to the compensation intensity, and then the compensated noise amounts of all parameters are summarized to form a new noise parameter set.

[0110] After determining the parameters that need to be compensated for noise and the corresponding compensation strength, in order to update the noise parameter set so that it can better protect parameter privacy, it is necessary to calculate the noise amount of each parameter after compensation, and summarize the compensated noise amounts of all parameters, and output a new noise parameter set for the next round of AI model iteration; locate the parameters that need to be compensated for noise in the next round of AI model iteration, and calculate the compensated noise amount based on the original noise amount of each parameter and the determined compensation strength. If the original noise amount of a parameter is a fixed value and the compensation strength coefficient is 1.2, the compensated noise amount is equal to the original noise amount multiplied by 1.2. The compensated noise amounts of all parameters that need to be compensated for noise are summarized and merged with the original noise amounts of the parameters that do not need to be compensated to form a new noise parameter set. Finally, this new noise parameter set is output as the basis for the noise parameters used in the next round of AI model iteration.

[0111] The output compensated noise parameter set can provide more appropriate noise protection for the parameters based on the privacy leakage of the parameters and the needs of the training stage. Through precise adjustment of the noise amount, the privacy protection ability of the parameters is effectively enhanced, while reducing the adverse impact on model performance, ensuring the security and effectiveness of the AI model during the iteration process. The new noise parameter set will serve as the basis for the next round of AI model iteration, affecting the privacy protection and model performance in the subsequent model training process. It also provides a new parameter basis for generating adversarial sample sets and conducting simulated attack tests, thereby continuing to evaluate and optimize the security of the AI model.

[0112] S3. Output the AI model according to the loss change of the AI model iteration, generate an adversarial sample set based on the noise parameter set, perform a simulated attack test based on the adversarial sample set through member inference attack, and determine the attack success rate in the simulated attack test. Analyze the attack success rate to trigger vulnerability detection and locate the vulnerability noise parameters at the same time.

[0113] Furthermore, if Figure 3 As shown, the sub-logic for determining the attack success rate includes:

[0114] Configure a loss change threshold. After each AI model iteration, calculate the difference between the loss value of the current AI model iteration and the loss value of the previous AI model iteration. If the loss difference is less than the loss change threshold for multiple consecutive times, the current AI model is considered to have converged and is selected as the output AI model.

[0115] The noise parameter set is stratified according to the training stage and the parameter sensitivity level, and the adversarial sample set is generated by combining the stratification results of the noise parameter set;

[0116] Conduct simulated attack tests based on adversarial sample sets through member inference attacks, and dynamically adjust the generation method and attack parameters of adversarial sample sets based on the changes in AI model iterations during the simulated attack tests.

[0117] According to the results of the simulated attack test, the ratio of the number of samples with successful attacks to the total number of samples in the adversarial sample set is calculated to determine the attack success rate in the simulated attack test.

[0118] Before conducting an attack test on an AI model, it is necessary to ensure that the AI model is in a stable state, otherwise the test results may be inaccurate and meaningless. During the training process of the AI model, the loss value will change with the iteration. When the loss value changes tend to be stable, it indicates that the AI model has basically converged. At this time, conducting an attack test can more realistically reflect the security of the model; configure a loss change threshold, which can be determined through experience or preliminary experiments based on the AI model type and training data characteristics. After each AI model iteration, calculate the difference between the loss value of the current iteration and the loss value of the previous iteration, and continuously record the loss differences of multiple iterations (for example, 5 times). When these differences are continuously less than the preset threshold, the loss value is increased. When the loss change threshold is determined, the current AI model is deemed to have converged, and the AI model is determined as the output AI model for subsequent operations; if the conditions are not met, the AI model iteration is continued and the judgment is repeated; in this way, AI models in a stable state can be effectively screened out, invalid attack tests on AI models in an unstable training stage can be avoided, the reliability and effectiveness of the attack test data can be ensured, and the foundation for subsequent accurate evaluation of model security is laid. The converged AI model provides a stable target model for the subsequent generation of adversarial sample sets. Only by generating adversarial samples based on the converged AI model and conducting attack tests can valuable attack success rate data be obtained for analyzing model vulnerabilities.

[0119] AI models at different training stages and parameter sensitivity levels react differently to attacks. In order to test the security of the model more comprehensively and accurately, it is necessary to stratify the noise parameter sets according to these factors, generate adversarial sample sets in a targeted manner, and improve the effectiveness and accuracy of the attack test; according to the aforementioned training stages and parameter sensitivity levels, the noise parameter sets are stratified, for example, the training stages are divided into early, middle and late stages, and the parameter sensitivity levels are divided into high, medium and low. In this way, multiple different parameter set categories are obtained. For each category, different strategies are used to generate adversarial sample sets. For parameter sets with high sensitivity levels and in the late stages of training, when generating adversarial samples, the perturbation amplitude and diversity of sample features are increased to more rigorously test the model's defense capabilities on key parameters; for parameter sets with low sensitivity levels and in the early stages of training, the perturbation amplitude is appropriately reduced to avoid excessive interference with the model training process.

[0120] The hierarchical generation of adversarial sample sets can fully consider the characteristics of AI models at different stages and parameter sensitivities, making attack samples more targeted and more effective in detecting potential vulnerabilities in models under different circumstances. This improves the comprehensiveness and accuracy of attack tests in assessing the security of AI models. The generated adversarial sample sets are the basic data for conducting member inference attack simulation tests. High-quality and targeted adversarial sample sets can make simulated attacks closer to actual attack scenarios, thereby obtaining a more accurate attack success rate and providing a reliable basis for subsequent analysis of model vulnerabilities.

[0121] During the simulated attack test, the AI model will react differently to different attack methods. In order to improve the success rate and accuracy of the attack test, it is necessary to dynamically adjust the generation method and attack parameters of the adversarial sample set based on the real-time feedback of the AI model during the attack process to better break through the model defense; when conducting simulated attack tests based on the adversarial sample set through member inference attack, monitor the changes in the AI model iteration process in real time, such as changes in the AI model output results and fluctuations in the loss value. If it is found that a certain type of adversarial sample attack has a low success rate, analyze the characteristics of the sample and the response of the AI model, and adjust the generation method of the adversarial sample, such as changing the direction, position or intensity of the feature perturbation; at the same time, adjust the attack parameters, such as the number of iterations and step size of the attack algorithm. If it is found that the adversarial sample generated based on a certain type of feature perturbation is difficult to change the AI model output, try to change the distribution pattern of the perturbation or add feature perturbations in other dimensions.

[0122] Dynamically adjusting the attack strategy makes attack testing more flexible and adaptable, and can timely optimize the attack method according to the actual defense situation of the AI model, which helps to more deeply explore the vulnerabilities of the AI model, improve the attack test's ability to detect the security risks of the AI model, and avoid missed vulnerabilities caused by fixed attack strategies. The attack test results obtained after dynamic adjustment are more real and reliable. The attack success rate obtained based on this statistics can more accurately reflect the security status of the AI model, and provide more accurate data support for subsequent triggering vulnerability detection and locating vulnerability noise parameters based on the attack success rate.

[0123] The attack success rate is a key indicator for measuring the security of AI models in simulated attack tests. By calculating the attack success rate, the AI model's ability to resist attacks can be intuitively evaluated, providing a quantitative basis for subsequent judgment of whether the AI model has vulnerabilities and the location of vulnerabilities. After the simulated attack test is completed, the number of successfully attacked samples is counted, that is, the number of samples for which the AI model's prediction results are inconsistent with the true labels and meet the attack target. At the same time, the total number of samples in the adversarial sample set is counted, and the ratio of the number of successfully attacked samples to the total number of samples is calculated. This ratio is determined as the attack success rate in the simulated attack test. The attack success rate reflects the security of the AI model under simulated attacks in a concise and clear quantitative manner, making it convenient for researchers and developers to quickly understand the security performance of the AI model, compare it with the preset security standards, and determine whether the AI model needs further optimization and improvement. The calculated attack success rate is an important basis for triggering vulnerability detection. When the attack success rate is greater than the preset attack success rate threshold, the vulnerability detection process will be triggered, and then the vulnerability noise parameters will be located to enable targeted security reinforcement of the AI model.

[0124] Specifically, the logic for locating vulnerability noise parameters includes:

[0125] Set an attack success rate threshold based on the privacy protection requirements of the AI model and historical attack test data. When the attack success rate exceeds the attack success rate threshold, vulnerability detection is triggered.

[0126] During the simulated attack test, the value changes of each parameter under different adversarial sample sets and the attack results are recorded. Statistical analysis is performed to calculate the correlation coefficient between the value changes of each parameter under different adversarial sample sets and the attack success rate. Candidate vulnerability noise parameters are screened based on the correlation coefficient.

[0127] During the AI model training phase, the normal value of each parameter is collected to determine the normal behavior of the parameter. During the simulated attack test, the degree of deviation of each parameter from the normal behavior is determined to determine the abnormal parameter.

[0128] A parameter dependency graph is constructed through a graph neural network. The nodes of the parameter dependency graph represent parameters, the edges of the parameter dependency graph represent the dependency relationship between parameters, and the weights of the edges of the parameter dependency graph represent the dependency strength between parameters.

[0129] Traverse the abnormal parameters of the parameter dependency graph, trace the vulnerability propagation path, and locate the vulnerability noise parameters.

[0130] To clarify the circumstances under which the vulnerability detection process is initiated, an attack success rate threshold needs to be set based on the privacy protection requirements of the AI model and historical attack test data. When the actual attack success rate exceeds this threshold, it indicates that the AI model has a security vulnerability and requires further in-depth detection and location. By analyzing a large amount of historical attack test data and combining it with the specific application scenarios and privacy protection requirements of the AI model, an appropriate attack success rate threshold is determined. After obtaining the attack success rate in the simulated attack test, it is compared with the set attack success rate threshold. If the attack success rate is greater than the threshold, the vulnerability detection process is immediately triggered. If it is less than or equal to the threshold, the current security of the AI model is considered to be within an acceptable range and vulnerability detection is not performed temporarily. Setting an attack success rate threshold provides a clear starting condition for vulnerability detection, avoiding unnecessary detection operations and improving detection efficiency. It also ensures that detection is initiated promptly when the security risk of the AI model is high, protecting the privacy, security and reliability of the AI model. After the vulnerability detection is triggered, a series of subsequent operations will be initiated to locate the vulnerability noise parameters. Based on the detection process triggered by the attack success rate, subsequent parameter analysis and location will be more targeted and focus on the vulnerable areas.

[0131] During the simulated attack test, changes in each parameter will affect the attack results. By calculating the correlation coefficient between parameter value changes and the attack success rate, we can screen out parameters closely related to attack success, narrow the scope of vulnerability noise parameters to be screened, and improve positioning efficiency. During the simulated attack test, the value changes of each parameter under different adversarial sample sets and the corresponding attack results (success or failure) are recorded in detail. Statistical methods, such as the Pearson correlation coefficient calculation method, are used to perform a correlation analysis between the value changes of each parameter under different adversarial sample sets and the attack success rate. The correlation coefficient is calculated, and a correlation coefficient threshold is set. Parameters with correlation coefficients greater than the threshold are screened as candidate vulnerability noise parameters. Screening candidate vulnerability noise parameters can quickly locate the vulnerability-related parameters from a large number of parameters, reducing the number of parameters for subsequent analysis, making positioning more targeted and efficient, avoiding indiscriminate analysis of all parameters, saving time and computing resources, and the screened candidate vulnerability noise parameters serve as the basis for further identifying abnormal parameters and locating vulnerabilities. Based on these candidate parameters, we will conduct a more in-depth analysis in combination with the normal behavior of the parameters to gradually narrow the scope of the vulnerability.

[0132] In order to accurately identify vulnerability noise parameters, it is necessary to first determine the behavior pattern of the parameters under normal circumstances, and then find out the abnormal parameters that deviate from the normal behavior by comparing the behavior of the parameters in the simulated attack test. These abnormal parameters are the key factors that cause the vulnerability; in the training stage of the AI model, collect the value data of each parameter in the normal training process, and determine the normal value range and distribution characteristics of the parameters through statistical analysis and other methods to define the normal behavior of the parameters. In the process of simulated attack testing, monitor the value of each parameter in real time, compare it with the predetermined normal behavior, and calculate the degree of deviation of the parameter value from the normal value range. For example, by calculating indicators such as deviation distance and deviation probability, a deviation threshold is set. When the deviation of a parameter is greater than the deviation threshold, the parameter is determined to be an abnormal parameter. Determining abnormal parameters can further screen out parameters with truly abnormal behaviors from candidate vulnerability noise parameters, and more accurately locate the parameters that cause vulnerabilities. By comparing with normal behaviors, it can effectively eliminate the interference of normal parameter fluctuations on vulnerability positioning, thereby improving positioning accuracy. The determined abnormal parameters will serve as the starting point for tracing the vulnerability propagation path in the parameter dependency graph. Based on these abnormal parameters, the parameter dependency graph is used to analyze the relationship between parameters, and the vulnerability noise parameters are gradually located.

[0133] Parameters in AI models do not exist independently; complex dependencies exist between them. Constructing a parameter dependency graph can intuitively display the interactions between parameters, helping to analyze the propagation paths of vulnerabilities between parameters, thereby more accurately locating vulnerability noise parameters. Using graph neural network technology, each parameter in the AI model is treated as a node in the graph. By analyzing the transmission and influence relationships of parameters during the AI model calculation process, the edges between nodes are determined, and the edges represent the dependencies between parameters. Machine learning algorithms, such as gradient-based optimization algorithms, are used to learn the strength of the dependencies between parameters and represent them as edge weights. If a change in parameter A has a greater impact on parameter C, the weight of the edge between node a and node c is higher. The parameter dependency graph clearly presents the complex relationships between parameters and provides an intuitive visualization tool for analyzing vulnerability propagation. It helps researchers grasp the interactions between parameters from a holistic perspective, more systematically analyze the causes and propagation paths of vulnerabilities, and improve the accuracy and comprehensiveness of vulnerability location. The constructed parameter dependency graph is the basic framework for traversing abnormal parameters and tracking vulnerability propagation paths. Based on the graph structure, it is possible to gradually find vulnerability noise parameters by following the dependencies between parameters and starting from the abnormal parameters.

[0134] After identifying the abnormal parameters and constructing the parameter dependency graph, it is necessary to traverse the abnormal parameters and trace the vulnerability propagation path along the parameter dependencies, ultimately locating the noise parameters that cause the model vulnerability for targeted repair and optimization. Starting from the identified abnormal parameter node, traverse the parameter dependency graph. Based on the edge weights and dependencies, prioritize searching along edges with higher weights, as these edges represent closer parameter dependencies and are more likely to propagate the vulnerability. During the traversal process, record the parameter nodes and propagation paths passed through. When a series of parameter nodes are found whose changes can reasonably explain the successful attack, these parameters are considered to be vulnerable noise parameters. Traversing the parameter dependency graph to trace the vulnerability propagation path allows for in-depth analysis of the root cause of the vulnerability and accurate location of the noise parameters that cause model security issues. This provides a clear target for subsequent repair and optimization of the AI model, helping to improve the model's security and reliability. The located vulnerable noise parameters will serve as an important basis for optimizing the noise parameter set. Based on these vulnerable noise parameters and each parameter's contribution to the AI model's usage, the noise amount of each parameter is comprehensively updated to further optimize the privacy protection and security performance of the AI model.

[0135] S4. Analyze the contribution rate of each parameter to the AI model through the SHAP value, and comprehensively update the noise amount of each parameter based on the contribution rate and the vulnerability noise parameter to optimize the noise parameter set.

[0136] Furthermore, if Figure 4 As shown, the analysis sub-logic using contribution rate includes:

[0137] Collect the adversarial sample sets of the AI model during the simulated attack test and the values of each parameter under different adversarial sample sets, and integrate them into sample data;

[0138] The sample data is stratified according to the training stage and the sensitivity level of the parameters, and a fixed number of sample data are randomly selected from each layer;

[0139] Based on the combination of parallel computing and approximate computing, the SHAP algorithm is used to calculate the SHAP value of each parameter on the sample data;

[0140] Analyze the SHAP value of each parameter on the sample data to obtain the absolute value and variance of the SHAP value. Analyze the absolute value and variance of the SHAP value to obtain the contribution rate of each parameter to the use of the AI model.

[0141] To analyze the contribution rate of each parameter to the use of the AI model, we first need to collect relevant data, the adversarial sample set of the AI model during the simulated attack test and the value of each parameter under different adversarial sample sets, which contain the behavioral information of the parameters in the actual attack scenario. Integrating this data can provide a comprehensive basis for subsequent analysis, making the evaluation of the parameter contribution rate more practical. After completing the simulated attack test, collect the adversarial sample set generated in the process, record the specific feature data of each adversarial sample, and obtain the value of each parameter under different adversarial sample sets. For example, for the neural network model, record the weight value of each layer parameter when inputting different adversarial samples, etc., and put these data into the database. The integrated sample data are integrated to form a sample data set containing adversarial sample features and parameter values for subsequent stratification and analysis; the integrated sample data associates the parameter values with the adversarial samples in the actual attack scenario, providing a complete data basis for analyzing the role of parameters when the model faces attacks, avoiding the one-sidedness of analysis caused by data dispersion, and making the subsequent calculation of usage contribution rate more accurate and reliable. The integrated sample data is the direct input for sample stratification. The stratification operation based on this data can reasonably classify the data according to the different training stages and parameter sensitivity levels, providing a more targeted data subset for the subsequent calculation of the SHAP value of each parameter.

[0142] At different training stages and parameter sensitivity levels, the parameters affect the AI model in different ways and degrees. By stratifying the sample data, data with similar characteristics can be divided into the same layer, so that when calculating the contribution rate of parameter usage, the role of parameters in different situations can be more accurately considered, thereby improving the accuracy and effectiveness of the analysis. According to the aforementioned training stages and parameter sensitivity levels, the integrated sample data is stratified. For example, the sample data is first divided into three categories according to the training stage, and then further subdivided according to the parameter sensitivity level in each category. A fixed number of sample data are randomly sampled from each layer to ensure that the data in each layer is representative to a certain extent, while avoiding The large amount of data leads to high computational complexity; the stratification of sample data enables subsequent analysis to more carefully consider the impact of different training stages and parameter sensitivity on the parameter contribution rate. Through stratified sampling, the representativeness of the data is guaranteed, the amount of calculation is reduced, and the analysis efficiency is improved. At the same time, the stratified data is more convenient for targeted analysis and comparison, which helps to discover the differences in the importance of parameters in different situations. The stratified sample data provides a suitable data subset for calculating the SHAP value of each parameter. Calculating the SHAP value based on these stratified data can more accurately reflect the contribution of the parameters to the AI model under different conditions, laying the foundation for the final determination of the contribution rate.

[0143] SHAP values are an important tool for interpreting model outputs. By calculating the SHAP value of each parameter, the parameter's influence on the model output can be quantified, thereby determining the parameter's contribution to the AI model's usage. This approach, based on a combination of parallel and approximate computing, improves computational efficiency while maintaining accuracy, adapting to the needs of large-scale data and complex models. Using parallel computing technology, stratified sample data is distributed across multiple compute nodes for simultaneous processing. For each compute node, approximate computing methods are used to accelerate the SHAP value calculation process. For example, a sampling-based approximate algorithm extracts a certain number of samples from the sample data for SHAP value calculation to reduce the computational effort. The SHAP algorithm calculates the SHAP value of each parameter on the sample data, yielding a score for each parameter's impact on the AI model's output. This combination of parallel and approximate computing significantly improves the efficiency of SHAP value calculation, enabling rapid results when processing large-scale sample data. Approximate computing also ensures accuracy to a certain extent, accurately reflecting the parameter's impact on the AI model. This provides an efficient and reliable computational foundation for subsequent analysis of usage contribution rates. The calculated SHAP value of each parameter is the core data for analyzing usage contribution rates. By further analyzing the absolute value and variance of these values, we can fully understand the contribution and stability of the parameters to the AI model, thereby determining the contribution rate of each parameter and providing an important basis for optimizing the noise parameter set.

[0144] Simply obtaining the SHAP value of each parameter is not sufficient to determine its contribution rate to the use of the AI model. It is necessary to further analyze the absolute value and variance of the SHAP value. The absolute value reflects the degree of influence of the parameter on the output of the AI model, while the variance reflects the stability of the parameter influence. Taking these two aspects into consideration, we can more comprehensively and accurately evaluate the importance of the parameter to the AI model, thereby determining the contribution rate. Analyze the SHAP value of each parameter on the sample data and calculate its absolute value. The larger the absolute value, the greater the influence of the parameter on the output of the AI model. At the same time, calculate the variance of the SHAP value. The smaller the variance, the more stable the influence of the parameter on the output of the AI model on different samples. Taking into account the absolute value and variance of the SHAP value, set a weight for each parameter. The distribution of weights is adjusted according to actual needs and experience. The absolute value and variance are comprehensively calculated according to the set weight to obtain the contribution rate of each parameter to the use of the AI model.

[0145] By comprehensively analyzing the absolute value and variance of the SHAP value to determine the usage contribution rate, it can more comprehensively reflect the actual role of the parameters in the AI model. It not only considers the degree of influence of the parameters on the output of the AI model, but also considers the stability of its influence, making the assessment of parameter importance more accurate and reasonable. This provides a scientific basis for the subsequent optimization of the noise parameter set according to the usage contribution rate. The determined usage contribution rate of each parameter is one of the key factors for optimizing the noise parameter set. In subsequent steps, the vulnerability noise parameter will be combined, and the noise amount of each parameter will be updated and adjusted according to the usage contribution rate to achieve the optimization of the noise parameter set and improve the performance and privacy protection capabilities of the AI model.

[0146] Specifically, the optimization logic of the noise parameter set includes:

[0147] Update the usage contribution rate of each parameter according to the vulnerability noise parameter to obtain the update factor of each parameter;

[0148] Determining the adjustment direction of the noise amount of each parameter according to the update factor of each parameter;

[0149] Determine the adjustment range of the noise amount of each parameter based on the training stage and the cumulative privacy leakage of each parameter;

[0150] The noise amount of each parameter is updated in combination with the adjustment direction and adjustment amplitude of the noise amount of each parameter to optimize the noise parameter set.

[0151] The existence of vulnerability noise parameters will affect the actual contribution of the parameters to the model. Therefore, it is necessary to update the usage contribution rate of each parameter based on the vulnerability noise parameters to obtain an update factor that more accurately reflects the effect of the parameter on the model when there is a vulnerability. This will help to adjust the noise amount more reasonably in the future and balance the AI model performance and privacy protection; according to the vulnerability noise parameters located above, the usage contribution rate of each parameter is adjusted. For vulnerability noise parameters, their usage contribution rate is reduced, such as multiplying their usage contribution rate by a coefficient less than 1 to reflect their negative impact on the AI model. For non-vulnerability noise parameters, if they are correlated with vulnerability noise parameters, they are also adjusted accordingly. The contribution rate is used to finally obtain the update factor of each parameter, which comprehensively considers the impact of vulnerability factors on the contribution of parameters; updating the contribution rate to obtain the update factor makes the evaluation of parameter contribution more in line with the actual situation of the AI model in the presence of vulnerabilities, and can more accurately identify parameters that have a significant impact on the performance and security of the AI model, providing a more reliable basis for the subsequent reasonable adjustment of the noise amount, which helps to improve the stability and security of the AI model. The update factor of each parameter is the basis for determining the direction of noise adjustment. According to the size and positive and negative of the update factor, it can be judged whether the noise amount of each parameter should be increased or decreased, thereby providing directional guidance for optimizing the noise parameter set.

[0152] In order to optimize the noise parameter set, it is necessary to determine the adjustment direction of the noise amount based on the update factor of each parameter. A reasonable adjustment direction can minimize the impact on the performance of the AI model while protecting the privacy of the parameters, thereby achieving a balance between privacy protection and AI model performance. Analyze the update factor of each parameter. If the update factor is greater than a set threshold, such as 0, it means that the parameter contributes greatly to the AI model and is relatively stable. While ensuring privacy, its noise amount should be appropriately reduced to improve the performance of the AI model. If the update factor is less than the threshold, it means that the contribution of the parameter is small or there is a potential risk. Its noise amount needs to be increased to enhance privacy protection. The adjustment direction of the noise amount of each parameter is determined based on the specific value and positive or negative situation of the update factor.

[0153] Determining the direction of noise adjustment can adjust the noise amount in a targeted manner according to the actual conditions of the parameters, avoiding the adverse effects of blindly increasing or decreasing the noise amount on the performance and privacy protection of the AI model. It helps to achieve a reasonable balance between privacy protection and performance between different parameters, and improve the overall performance of the AI model. The determined direction of noise adjustment provides directional guidance for the subsequent determination of the noise adjustment range. On the basis of the known adjustment direction, combined with factors such as the training stage and the accumulated privacy leakage, the adjustment range can be further determined, which can more accurately optimize the noise parameter set.

[0154] After determining the adjustment direction of the noise amount, the specific adjustment range needs to be determined based on the training stage and the cumulative privacy leakage of each parameter. The model has different sensitivities to noise at different training stages, and the cumulative privacy leakage also reflects the degree of privacy risk of the parameter. Comprehensive consideration of these factors can more reasonably adjust the noise amount, which protects privacy without affecting the performance of the AI model. According to the training stages divided above, in the early stage of training, the AI model has a relatively high tolerance for noise. At this time, the adjustment range should be appropriately increased so that the noise amount of the parameter can be adjusted more quickly to adapt to the training needs of the AI model. In the later stage of training, the AI model is close to convergence and is more sensitive to noise. The adjustment range should be reduced to avoid excessive adjustment that affects the stability of the AI model. At the same time, refer to the cumulative privacy leakage of each parameter calculated above. For parameters with higher cumulative privacy leakage, the adjustment range of the noise amount should be increased to strengthen privacy protection; for parameters with lower cumulative privacy leakage, the adjustment range should be appropriately reduced. Taking into account factors such as the training stage and cumulative privacy leakage, the appropriate noise adjustment range is determined for each parameter.

[0155] Determining the noise adjustment amplitude based on the training stage and the cumulative privacy leakage can make the noise adjustment more in line with the actual needs of the AI model and the privacy status of the parameters, and achieve reasonable noise adjustment at different training stages, which can not only ensure the learning efficiency of the AI model in the early stage of training, but also ensure the stability and privacy security of the AI model in the later stage. At the same time, it strengthens protection for parameters with high privacy risks and improves the overall privacy protection level of the AI model; the noise adjustment amplitude of each parameter is combined with the previously determined adjustment direction to provide a specific numerical basis for updating the noise amount of each parameter. Updating the noise amount according to the adjustment direction and amplitude can optimize the noise parameter set, so that the AI model can achieve a better balance between performance and privacy protection.

[0156] The adjustment direction and amplitude of the noise amount of each parameter are determined through the previous steps. Finally, it is necessary to combine this information to update the noise amount of each parameter, thereby optimizing the noise parameter set and making the AI model have better performance in facing attacks and protecting privacy. According to the adjustment direction and amplitude of the noise amount of each parameter, the current noise parameter set is updated. For parameters that need to increase the noise amount, their noise amount is increased according to the determined adjustment amplitude. For parameters that need to reduce the noise amount, their noise amount is reduced accordingly. After the noise amount of all parameters is updated, a new noise parameter set is formed. This set reflects the noise configuration optimized according to factors such as the parameter usage contribution rate, vulnerability status, training stage, and cumulative privacy leakage. The updated noise parameter set can better adapt to the performance and privacy protection requirements of the AI model. By reasonably adjusting the noise amount of each parameter, while protecting the privacy of the parameters, the impact of noise on the performance of the AI model is minimized, thereby improving the security and reliability of the AI model. The new noise parameter set provides a more optimized privacy protection foundation for subsequent iterations of the AI model. The optimized noise parameter set will serve as one of the inputs for the next AI model iteration. In subsequent iterations, the AI model will be trained and learned based on this noise parameter set, which will help to further improve the performance and privacy protection capabilities of the AI model. At the same time, the new noise parameter set will also affect the results of subsequent attack testing and vulnerability detection operations on the AI model.

[0157] The logic reflected by the above four steps includes: when the AI model has not converged, a single iteration is executed from step S1 to step S2; when the AI model converges, step S3 is executed; when a vulnerability is detected, step S4 is executed and the training is iterated again back to step S1; if no vulnerability is detected, the final AI model is output. The AI model will go through multiple rounds of iterations, for example, after 10 rounds of iterations, each round number label is given, then the 4th iteration is the next round of iteration after the 3rd iteration.

Claims

1. The differential privacy parameter protection method in AI model iteration is characterized by: include: Before each AI model iteration, a clustering algorithm is used to identify the gradient distribution and iteration amplitude of the parameters to dynamically classify the sensitivity levels of the parameters. The training phases are dynamically divided according to the training progress of the AI model. The training phases and parameter sensitivity levels are analyzed to comprehensively allocate the privacy budget and noise amount for each parameter. After each AI model iteration, the cumulative privacy leakage of each parameter in the historical AI model iteration is calculated based on the privacy budget of each parameter. The cumulative privacy leakage of each parameter is determined to trigger compensation for the noise amount of each parameter in the next round of AI model iteration. The compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase, and the compensated noise parameter set is output; Output the AI model based on the loss changes of the AI model iteration, generate an adversarial sample set based on the noise parameter set, perform a simulated attack test based on the adversarial sample set through member inference attack, and determine the attack success rate in the simulated attack test. Analyze the attack success rate to trigger vulnerability detection and locate the vulnerability noise parameters at the same time; The SHAP value is used to analyze the contribution rate of each parameter to the AI model. The noise amount of each parameter is comprehensively updated based on the contribution rate and vulnerability noise parameter to optimize the noise parameter set.

2. The differential privacy parameter protection method in AI model iteration according to claim 1, characterized in that: The privacy budget and noise allocation logic for each parameter includes: Based on the index mechanism of differential privacy, the importance of each parameter to the AI model during the training phase is analyzed, and the utility measurement value of each parameter is determined comprehensively based on the importance of each parameter to the AI model and the sensitivity value of each parameter; Based on the utility metric of each parameter, the probability of obtaining the privacy budget for each parameter through the exponential mechanism of differential privacy is calculated; A privacy budget is allocated to each parameter according to the probability of each parameter obtaining a privacy budget, and a noise amount is allocated to each parameter based on the sensitivity level of the parameter and the privacy budget allocated to each parameter.

3. The differential privacy parameter protection method in AI model iteration according to claim 2, characterized in that: The sub-logic of dividing the sensitivity level of the parameter includes: Before each AI model iteration, the gradient distribution of the parameters is clustered using the DBSCAN density clustering algorithm to divide it into multiple clusters. The neighborhood radius of the DBSCAN density clustering algorithm is dynamically adjusted according to the compactness of the gradient distribution of the parameters. Calculate the average iteration amplitude of the parameters within each cluster, and determine the degree of fluctuation of the iteration amplitude in combination with the average iteration amplitude of the parameters within each cluster; The tightness of the parameter gradient distribution, the average iteration amplitude and the fluctuation degree of the iteration amplitude are weighted and summed to comprehensively determine the sensitivity value of each parameter. The sensitivity level of the parameter is dynamically divided according to the sensitivity value of each parameter.

4. The differential privacy parameter protection method in AI model iteration according to claim 3, characterized in that: The division sub-logic of the training phase includes: Build a training progress indicator system, including the number of AI model iterations, the completion rate of training data, and the change in loss between AI model iterations; The training progress is evaluated by fuzzy comprehensive evaluation method, and the weight vector of each indicator system is determined based on the hierarchical analysis method; Fuzzy judgment rules are set for each indicator system respectively to obtain the evaluation results of each indicator system, and the evaluation results of each indicator system and the weight vector are weighted and summed to obtain a comprehensive evaluation value. The training stage is divided according to the comprehensive evaluation value.

5. The differential privacy parameter protection method in AI model iteration according to claim 4, characterized in that: The output logic of the noise parameter set includes: Set a privacy leakage threshold for each parameter based on the privacy protection requirements of the training phase and the AI model. Compare the cumulative privacy leakage of each parameter with the privacy leakage threshold of each parameter to determine whether to trigger compensation for the noise amount of each parameter in the next round of AI model iteration; When it is necessary to trigger compensation for the noise amount of each parameter in the next round of AI model iteration, the compensation intensity of the noise amount is dynamically adjusted according to the division results of the training phase; Locate the parameters that need to compensate for noise in the next round of AI model iteration, determine the compensated noise amount based on the noise compensation strength, summarize the noise amounts after all parameters are compensated, and output the compensated noise parameter set.

6. The differential privacy parameter protection method in AI model iteration according to claim 5, characterized in that: The calculation sub-logic of the cumulative privacy leakage of each parameter includes: After each AI model iteration, a privacy leakage risk assessment matrix is constructed based on the degree of change in the gradient distribution of parameters, the degree of correlation between parameters, and the training stage of the current AI model iteration; Calculate the privacy leakage of each parameter after each AI model iteration based on the privacy budget of each parameter and the privacy leakage risk assessment matrix; The iteration weight factor after each AI model iteration is determined according to the inverse of the number of iterations of the AI model. The privacy leakage amount of each parameter after each AI model iteration is multiplied by the iteration weight factor, and the sum is accumulated to obtain the cumulative privacy leakage amount of each parameter.

7. The differential privacy parameter protection method in AI model iteration according to claim 6, characterized in that: The positioning logic of the vulnerability noise parameters includes: Set an attack success rate threshold based on the privacy protection requirements of the AI model and historical attack test data. When the attack success rate exceeds the attack success rate threshold, vulnerability detection is triggered. During the simulated attack test, the value changes of each parameter under different adversarial sample sets and the attack results are recorded. Statistical analysis is performed to calculate the correlation coefficient between the value changes of each parameter under different adversarial sample sets and the attack success rate. Candidate vulnerability noise parameters are screened based on the correlation coefficient. During the AI model training phase, the normal value of each parameter is collected to determine the normal behavior of the parameter. During the simulated attack test, the degree of deviation of each parameter from the normal behavior is determined to determine the abnormal parameter. A parameter dependency graph is constructed through a graph neural network. The nodes of the parameter dependency graph represent parameters, the edges of the parameter dependency graph represent the dependency relationship between parameters, and the weights of the edges of the parameter dependency graph represent the dependency strength between parameters. Traverse the abnormal parameters of the parameter dependency graph, trace the vulnerability propagation path, and locate the vulnerability noise parameters.

8. The differential privacy parameter protection method in AI model iteration according to claim 7, characterized in that: The sub-logic for determining the attack success rate includes: Configure a loss change threshold. After each AI model iteration, calculate the difference between the loss value of the current AI model iteration and the loss value of the previous AI model iteration. If the loss difference is less than the loss change threshold for multiple consecutive times, the current AI model is considered to have converged and is selected as the output AI model. The noise parameter set is stratified according to the training stage and the parameter sensitivity level, and the adversarial sample set is generated by combining the stratification results of the noise parameter set; Conduct simulated attack tests based on adversarial sample sets through member inference attacks, and dynamically adjust the generation method and attack parameters of adversarial sample sets based on the changes in AI model iterations during the simulated attack tests. According to the results of the simulated attack test, the ratio of the number of samples with successful attacks to the total number of samples in the adversarial sample set is calculated to determine the attack success rate in the simulated attack test.

9. The differential privacy parameter protection method in AI model iteration according to claim 8, characterized in that: The optimization logic of the noise parameter set includes: Update the usage contribution rate of each parameter according to the vulnerability noise parameter to obtain the update factor of each parameter; Determining the adjustment direction of the noise amount of each parameter according to the update factor of each parameter; Determine the adjustment range of the noise amount of each parameter based on the training stage and the cumulative privacy leakage of each parameter; The noise amount of each parameter is updated in combination with the adjustment direction and adjustment amplitude of the noise amount of each parameter to optimize the noise parameter set.

10. The differential privacy parameter protection method in AI model iteration according to claim 9, characterized in that: The analysis sub-logic of the usage contribution rate includes: Collect the adversarial sample sets of the AI model during the simulated attack test and the values of each parameter under different adversarial sample sets, and integrate them into sample data; The sample data is stratified according to the training stage and the sensitivity level of the parameters, and a fixed number of sample data are randomly selected from each layer; Based on the combination of parallel computing and approximate computing, the SHAP algorithm is used to calculate the SHAP value of each parameter on the sample data; Analyze the SHAP value of each parameter on the sample data to obtain the absolute value and variance of the SHAP value. Analyze the absolute value and variance of the SHAP value to obtain the contribution rate of each parameter to the use of the AI model.

Images