Classification model stealing defense method, system and equipment for discretization disturbance sensitive data prediction and medium

Abstract

The invention discloses a classification model stealing defense method, system and device for discretization disturbance sensitive data prediction and a medium, and belongs to the technical field of data privacy security, and the method comprises the steps: obtaining a prediction probability vector of an input sample, and calculating a sample abnormity degree index according to the prediction probability vector; based on the sample anomaly degree index, generating a random disturbance vector satisfying a constraint condition, the maximum probability position of the random disturbance vector being different from the maximum probability position of the prediction probability vector; determining a combination weight according to a sample anomaly degree index, and combining the prediction probability vector and the random disturbance vector based on the combination weight to generate a disturbed prediction vector; discretization processing is carried out on the perturbed prediction vector, and a continuous probability value is converted into discrete grade representation. According to the method, the disturbance intensity is adaptively adjusted by calculating the malicious degree of the sample, the decision boundary speculation of an attacker is destroyed in cooperation with a discretization post-processing method, and the accuracy of the obtained substitution model is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data privacy security, in particular to a discrete perturbation sensitive data prediction classification model theft defense method, system, device and medium. BACKGROUND

[0002] In the sensitive data transaction task scenario, the transaction parties exchange information under the premise of ensuring data security and compliance, and the transaction data often involves high-value business secrets and personal privacy, so the security protection measures are particularly critical. Model theft defense technology has also attracted widespread attention in this context, and its core goal is to prevent attackers from stealing the internal information of the target model or replicating its functions through limited queries, thereby avoiding illegal profits and protecting the security of sensitive data in the transaction process. Specifically, considering that attackers often rely on key information such as predicted labels and confidence to implement model theft, defense strategies not only need to reduce the attack success rate by reducing or obscuring the information provided by the API, but also should combine multi-level security mechanisms such as differential privacy and encrypted transmission to build a comprehensive and hierarchical defense system. At the same time, in the sensitive data transaction task, it is also an important consideration for defense strategy design to ensure the high classification accuracy and response speed of the model for legitimate users, as this directly affects the efficiency of data transactions and the trust between the parties. As can be seen, in the sensitive data transaction task scenario, it is of great significance to build a systematic model theft defense mechanism to protect data assets, maintain business interests and promote safe transactions.

[0003] 1. Model theft defense method based on precision preservation

[0004] The core of the defense method based on precision preservation is to interfere with the confidence distribution of the model output or directly delete the confidence information, thereby weakening the ability of the attacker to reconstruct the model through the output information under the premise of ensuring the core function of the model (such as the correctness of the top-1 label). This method mainly adds random noise to the output probability through differential privacy technology, so that the attacker cannot accurately distinguish between the real prediction distribution of the model and the distribution after noise interference. The defense method based on precision preservation can effectively reduce the amount of information obtained by the attacker from the target model, but this method is ineffective for attacks that do not require model prediction confidence to generate a substitute model.

[0005] 2. Model theft defense method based on precision limitation

[0006] The model stealing defense method based on precision limitation destroys the ability of the attacker to build a substitute model by actively destroying the availability of the target model output. The core of this method is to sacrifice the classification accuracy of the target model itself, and to inject disturbance into the prediction result to make the attacker unable to learn effective knowledge from the polluted output. The disturbance method can be divided into the following three dimensions: 1) make the angle deviation between the gradient of the posterior probability of the disturbed sample and the original model gradient maximum; 2) distinguish the input samples by the sample anomaly detection method, and inject error information into the input sample prediction result by training an error function; 3) apply higher intensity label offset to the samples in the specific semantic region (such as the eye region in face recognition). The defense method based on precision limitation can effectively reduce the accuracy of the substitute model generated by the attacker, but at the cost of sacrificing the accuracy of the target model, which will affect the user experience of the target model.

[0007] In summary, the existing model stealing defense method adds disturbance to the original prediction result of the target model output, affects the confidence or label information obtained by the attacker, and then affects the accuracy of the substitute model generated by the attacker. However, there are the following problems: 1) only the size of the element value in the prediction probability vector of the target model output is modified, and the prediction label is not changed, resulting in little difference between the prediction probability vector after disturbance and before disturbance, and the influence degree on the accuracy of the substitute model generated by the attacker is small, and the defense strength is insufficient; 2) increasing the defense strength will reduce the accuracy of the target model, affecting the use of normal users. SUMMARY

[0008] In view of the above problems, the present application is proposed.

[0009] Therefore, the technical problem solved by the present application is: how to solve the problem that the defense effect of the disturbance method which only fine-tunes the prediction probability value without changing the label is limited, and increasing the disturbance strength will significantly reduce the practical value of the model, and propose a classification model stealing defense method with random disturbance.

[0010] To solve the above technical problems, the present application provides the following technical solutions: a discrete disturbance sensitive data prediction classification model stealing defense method, comprising: obtaining a prediction probability vector of an input sample, calculating a sample anomaly degree index according to the prediction probability vector;

[0011] Based on the sample anomaly degree index, a random disturbance vector satisfying a constraint condition is generated, wherein the maximum probability position of the random disturbance vector is different from the maximum probability position of the prediction probability vector;

[0012] According to the sample anomaly degree index, a combination weight is determined, and the prediction probability vector and the random disturbance vector are combined based on the combination weight to generate a disturbed prediction vector;

[0013] Discretize the disturbed prediction vector to convert continuous probability values into discrete level representation.

[0014] As a preferred scheme of the discrete perturbation sensitive data prediction classification model stealing defense method of the present application, wherein: the calculation of the sample anomaly degree index according to the prediction probability vector comprises:

[0015] Extracting the maximum probability value from the prediction probability vector;

[0016] Comparing the maximum probability value with the anomaly detection threshold;

[0017] Based on the comparison result of the maximum probability value and the anomaly detection threshold, the sample anomaly degree index is calculated by a mapping function.

[0018] The beneficial effects of the preferred technical scheme are: when it is necessary to evaluate the anomaly degree of the input sample, that is, by extracting the maximum probability value in the prediction probability vector and comparing it with the anomaly detection threshold, and then calculating the sample anomaly degree index based on the comparison result through the mapping function. Therefore, this technical scheme can convert the continuous probability distribution characteristics into a quantifiable anomaly degree evaluation index, and does not need to rely on complex statistical models or historical data accumulation. When facing unknown distribution query samples, the anomaly degree can also be quickly judged through the key feature of the maximum probability value, thereby providing a reliable basis for subsequent differentiated processing. At the same time, the introduction of the mapping function makes the anomaly degree index have good numerical stability, that is, even in the case of large probability value fluctuations, the continuity and consistency of the evaluation results can be maintained.

[0019] As a preferred scheme of the discrete perturbation sensitive data prediction classification model stealing defense method of the present application, wherein: according to the comparison result, the input sample is identified as different sample categories;

[0020] Based on the sample category, the corresponding sample anomaly degree index is generated through the mapping function.

[0021] As a preferred scheme of the discrete perturbation sensitive data prediction classification model stealing defense method of the present application, wherein: the generation of the random perturbation vector satisfying the constraint condition comprises:

[0022] Identifying the category identifier corresponding to the maximum probability position in the prediction probability vector;

[0023] Generating a perturbation probability distribution corresponding to the prediction probability vector;

[0024] Setting a peak value in the perturbation probability distribution, and assigning the peak value to a position different from the category identifier;

[0025] processing the perturbation probability distribution to obtain the random perturbation vector.

[0026] The beneficial effects of the preferred technical solution are: when it is necessary to construct a perturbation vector, that is, by identifying the original predicted class label, generating a corresponding perturbation probability distribution, and assigning a peak value to a position different from the original class label. Therefore, this design ensures that the perturbation vector and the original prediction are fundamentally different in the class level, thereby fundamentally destroying the attacker's ability to speculate on the decision boundary. And this scheme sets a peak value in the perturbation probability distribution, not only ensures the rationality and credibility of the perturbation vector, but also makes the output after perturbation still conform to the basic characteristics of the probability distribution. When the attacker tries to approximate the real model through multiple queries, because each generated perturbation vector has randomness but also meets the preset constraints, it can maintain the defense effect while avoiding forming a fixed pattern that can be identified.

[0027] As a preferred scheme of the discrete perturbation sensitive data prediction classification model theft defense method of the application, wherein: the value of the peak value is not less than the peak value threshold;

[0028] If the generated peak position is the same as the position corresponding to the class label, the peak position is regenerated until they are different;

[0029] The other elements in the perturbation probability distribution except the peak value are randomly assigned the remaining probability values.

[0030] As a preferred scheme of the discrete perturbation sensitive data prediction classification model theft defense method of the application, wherein: the combination of the prediction probability vector and the random perturbation vector based on the combination weight includes:

[0031] The perturbed prediction vector is calculated in a linear weighting manner;

[0032] When the sample anomaly degree index is less than the threshold value, the perturbed prediction vector is more biased towards the random perturbation vector;

[0033] When the sample anomaly degree index is greater than or equal to the threshold value, the perturbed prediction vector is more biased towards the prediction probability vector.

[0034] The beneficial effects of the preferred technical solution are: when the final defense output needs to be generated, that is, the combination weight is determined according to the sample abnormality degree index, the original prediction vector and the disturbance vector are fused in a linear weighting manner. Therefore, the adaptive combination mechanism realizes the dynamic balance between defense strength and model availability. When the sample abnormality degree index is less than the threshold value, it is identified as a potential attack sample, at this time the disturbance vector occupies the dominant position, and then strong interference is formed on the attacker; when the sample abnormality degree index is greater than or equal to the threshold value, it is identified as a normal user query, at this time the original prediction vector remains dominant, thereby maintaining the service quality of the model to the legitimate user. And the calculation complexity of the linear weighting method is low, even in a high-concurrency query scenario, the response efficiency of the system can be maintained, and at the same time, the numerical instability problem caused by complex nonlinear transformation can be avoided.

[0035] As a preferred scheme of the discrete perturbation sensitive data prediction classification model stealing defense method, wherein: the discrete processing of the perturbed prediction vector comprises:

[0036] The class confidence whose continuous probability value is greater than the first threshold value is converted into a first level;

[0037] The class confidence whose continuous probability value is between the second threshold value and the first threshold value is converted into a second level;

[0038] The class confidence whose continuous probability value is less than the second threshold value is converted into a third level;

[0039] Wherein, the first level, the second level and the third level correspond to high, medium and low discrete level representation respectively.

[0040] The application provides a discrete perturbation sensitive data prediction classification model stealing defense system.

[0041] To solve the above technical problems, the application further provides the following technical scheme: a discrete perturbation sensitive data prediction classification model stealing defense system, comprising: an anomaly detection module, configured to obtain a prediction probability vector of an input sample, and calculate a sample abnormality degree index according to the prediction probability vector;

[0042] A disturbance generation module is configured to generate a random disturbance vector that satisfies a constraint condition based on the sample abnormality degree index, wherein the maximum probability position of the random disturbance vector is different from the maximum probability position of the prediction probability vector;

[0043] An adaptive combination module is configured to determine a combination weight according to the sample abnormality degree index, combine the prediction probability vector and the random disturbance vector based on the combination weight, and generate a perturbed prediction vector;

[0044] A discretization processing module is configured to perform discretization processing on the perturbed prediction vector, and convert continuous probability values into discrete level representations.

[0045] The application provides a computer device, including a memory and a processor, and the memory stores a computer program.

[0046] The application provides a computer readable storage medium, which stores a computer program.

[0047] The application has the beneficial effects that, compared with the prior art, the application uses a distribution-out detection method to determine the malicious degree of an input sample, combines a random perturbation addition strategy, adds a preset perturbation to a prediction result of the input sample, and performs discretization processing on the prediction result, thereby reducing the possibility that an attacker infers a model decision boundary through continuous confidence values, ensuring that the defense effect is improved, and making the degree of accuracy reduction of a target model controlled within an acceptable range. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0049] Figure 1 The principle diagram of the classification model stealing defense method for discretization perturbation sensitive data prediction provided by an embodiment of the application. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings in the specification. Obviously, the described embodiments are only some of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the protection scope of the application.

[0051] Embodiment 1, refer to Figure 1 For the first embodiment of the application, the embodiment provides a classification model stealing defense method for discretization perturbation sensitive data prediction, including:

[0052] S100: Obtain a prediction probability vector of an input sample, and calculate a sample anomaly degree index according to the prediction probability vector;

[0053] S200: Based on the sample anomaly degree index, a random perturbation vector satisfying a constraint condition is generated, wherein the maximum probability position of the random perturbation vector is different from the maximum probability position of the prediction probability vector;

[0054] S300: Determine a combination weight according to the sample anomaly degree index, and combine the prediction probability vector and the random perturbation vector based on the combination weight to generate a perturbed prediction vector;

[0055] S400: Discretize the perturbed prediction vector to convert continuous probability values into discrete level representation.

[0056] It should be noted that in the sensitive data transaction scenario, the classification model provides services to the outside through the API interface, and the prediction probability vector returned by each query contains n-dimensional continuous values (n is the number of categories), and the value range of each dimension is 0 to 1. Attackers can obtain a large number of input-output pairs by constructing boundary samples and querying multiple times, and then train a substitute model to replicate the function of the target model. The existing defense method mainly adds noise to the original prediction probability or deletes part of the confidence information, but there are two technical defects: first, only the probability value is modified while the prediction category remains unchanged, because the position of the maximum value in the n-dimensional vector is not changed, the attacker can still eliminate the noise effect by multiple queries; second, the continuous probability value exposes the decision boundary information of the model, because the probability difference between adjacent categories reflects the distance of the sample to the decision boundary.

[0057] Therefore, in view of the above-mentioned defense deficiency problem, through the steps of S100 to S400, first, the sample anomaly degree index is calculated in S100 to distinguish between normal queries and attack queries; then a completely different category perturbation vector is generated in S200, because the maximum probability position of the perturbation vector is different from the original prediction, which fundamentally changes the returned prediction category; then the combination is performed according to the anomaly degree in S300 to realize the differentiated processing of different queries; finally, the continuous probability value is discretized into a level representation in S400, which eliminates the accurate numerical information. Through the synergistic effect of the above four steps, the attacker's ability to speculate on the decision boundary is destroyed, and the service quality of normal users is guaranteed.

[0058] Embodiment 2, refer to Figure 1 For the second embodiment of the present application, based on the above embodiment, a classification model theft defense method for discretizing perturbation sensitive data prediction is provided.

[0059] In the embodiments of the present application, the step S100 of calculating the sample anomaly degree index according to the prediction probability vector comprises steps A1 to A3:

[0060] A1: extracting the maximum probability value from the prediction probability vector;

[0061] A2: comparing the maximum probability value with an anomaly detection threshold;

[0062] A3: based on the comparison result of the maximum probability value and the anomaly detection threshold, calculating the sample anomaly degree index through a mapping function.

[0063] It should be noted that the process of extracting the maximum probability value from the prediction probability vector in step A1 is to traverse the n-dimensional prediction probability vector and find the element with the largest value. Because the prediction result of the classification model for each input sample is represented as a probability distribution of n categories, where n is the total number of categories of the model, the maximum probability value reflects the confidence degree of the model in classifying the sample. When the maximum probability value is close to 1, it indicates that the model is very certain about the classification result; when the maximum probability value is close to 1 / n, it indicates that the model's judgment of each category is relatively ambiguous.

[0064] The comparison process in step A2 is essentially to determine whether the input sample conforms to the probability distribution characteristics of normal queries. The query samples of normal users usually have clear category attribution, so the prediction probability vector will show a clear peak distribution, i.e., the probability value of a certain category is significantly higher than that of other categories. Attackers often construct ambiguous samples at the category boundary in order to probe the decision boundary of the model, and such samples have a relatively flat distribution of prediction probability vectors.

[0065] In step A3, the input sample is identified as different sample categories according to the comparison result, and the corresponding sample anomaly degree index is generated through a mapping function based on the sample categories. The mapping function converts the discrete comparison result into a continuous anomaly degree value, providing a quantitative basis for subsequent adaptive processing.

[0066] For example, in the medical image classification scenario, a normal chest X-ray query will produce a clear prediction result, such as a probability of 0.92 for the normal category and probabilities less than 0.05 for the other 11 pathological categories. In this case, the maximum probability value 0.92 is greater than the threshold (e.g., 0.7), and the sample is identified as a normal query. The boundary sample constructed by the attacker may have a probability of 0.35 for the normal category and a probability of 0.32 for the pneumonia category, with probabilities between 0.01 and 0.05 for the remaining categories. The maximum probability value 0.35 is less than the threshold, and the sample is identified as an abnormal query.

[0067] The mapping function can take the form of a sigmoid function, which maps the difference between the maximum probability value and the threshold value to the range of 0 to 1. When the maximum probability value is much greater than the threshold value, the abnormality degree index is close to 1, indicating a high confidence normal sample; when the maximum probability value is much less than the threshold value, the abnormality degree index is close to 0, indicating a possible attack sample. The smoothing property of the sigmoid function avoids the mutation problem that may be caused by a hard threshold.

[0068] In an optional embodiment, the calculation of the sample abnormality degree index in step S100 can also be based on the entropy value of the prediction probability vector. The information entropy of the prediction probability vector is calculated, and the greater the entropy value, the more uniform the probability distribution, and the more likely the sample is an attack sample. By setting an entropy value threshold, when the calculated entropy value exceeds the threshold, the sample is considered abnormal. The calculation method of information entropy is to multiply the probability value of each class by the negative of its logarithm value, and then sum. Because the information entropy can comprehensively reflect the uncertainty of the probability distribution, it is more comprehensive than considering only the maximum probability value.

[0069] In another optional embodiment, the calculation of the sample abnormality degree index in step S100 can also be based on the variance of the prediction probability vector. The variance of the n probability values is calculated, and the smaller the variance, the closer the probabilities of each class, and the more likely the sample is on the decision boundary. By pre-setting a variance threshold, when the actual variance is less than the threshold, the sample is determined to be abnormal. The calculation of variance first calculates the probability mean value, and then calculates the sum of the squares of the difference between each probability value and the mean value divided by the number of classes. Because the variance can measure the dispersion degree of the probability distribution, it has good effect on identifying flat-distributed attack samples.

[0070] In the embodiments of the present application, the random perturbation vector satisfying the constraint condition is generated in step S200, including the following steps B1-B4:

[0071] B1: identifying the class label corresponding to the maximum probability position in the prediction probability vector;

[0072] B2: generating a perturbation probability distribution corresponding to the prediction probability vector;

[0073] B3: setting a peak value in the perturbation probability distribution, and assigning the peak value to a position different from the class label;

[0074] B4: processing the perturbation probability distribution to obtain a random perturbation vector.

[0075] It should be noted that the process of identifying the class label in step B1 is to record the class index with the highest confidence in the original prediction. Because the subsequently generated perturbation vector needs to form a difference in class with the original prediction, the dominant class of the original prediction must be determined first. The class label is usually represented as an integer between 0 and n-1, where n is the total number of classes.

[0076] In the step B2, a vector with the same dimension as the original prediction vector is initialized, and each element can be set to a random small value. Since the final perturbation vector needs to satisfy the basic properties of the probability distribution, i.e., each element is non-negative and the sum is 1, subsequent normalization processing needs to be considered during the generation process.

[0077] In the step B3, the value of the peak is not lower than the peak threshold value; if the generated peak position is the same as the position corresponding to the class identifier, the peak position is regenerated until they are different. Peak setting is a key step in generating a perturbation vector, because only a probability distribution with a clear peak value can simulate the real prediction result, thereby effectively misleading the attacker. The setting of the peak threshold value needs to balance the perturbation effect and credibility, and a too low peak value will make the perturbation vector unnatural, and a too high peak value may reduce the randomness of the perturbation.

[0078] In the step B4, the remaining probability values are randomly allocated to the other elements in the perturbation probability distribution except the peak value. In the specific implementation, the remaining probability values (i.e., 1 minus the peak value) are first calculated, and then these remaining probabilities are randomly allocated to the other n-1 positions. The allocation method can use uniform distribution plus random perturbation, or use exponential distribution or other distribution forms to ensure that the generated perturbation vector has sufficient randomness.

[0079] For example, in a document classification task, the original prediction vector has 16 categories, and the maximum probability value 0.85 corresponds to the 3rd category (financial report category), and the class identifier is 3 at this time. When generating the perturbation vector, the 8th category (technical document category) is randomly selected as the peak position, the peak value is set to 0.82, and the remaining 0.18 is randomly allocated to the other 15 categories. Because the random position and random allocation scheme are regenerated each time the query is performed, even if the same sample is queried multiple times, the attacker will get different perturbation results, so that the attacker cannot eliminate the perturbation effect by statistical averaging.

[0080] In an optional implementation, the random perturbation vector generated in the step S200 can also use a multi-peak distribution strategy. In addition to setting a main peak value, one or more secondary peak values can also be set, so that the perturbation vector presents a more complex distribution form. The main peak value still needs to avoid the maximum probability position of the original prediction, and the secondary peak value can be randomly distributed in other positions. The multi-peak distribution can increase the complexity of the perturbation vector, making it more difficult for the attacker to analyze the real prediction pattern.

[0081] In another alternative implementation, the random perturbation vector generated in step S200 can also be based on a predefined perturbation template library. A plurality of probability distribution templates satisfying the constraint condition are constructed in advance, each template having different distribution characteristics. When a perturbation vector needs to be generated, a template is randomly selected, and then circularly shifted according to the original predicted class label, to ensure that the peak position of the template is different from the original prediction. The template library method can guarantee the quality of the generated perturbation vector, while reducing the real-time computing overhead.

[0082] In the embodiments of the present application, the combination of the predicted probability vector and the random perturbation vector based on the combination weight in step S300 includes the following steps C1-C3:

[0083] C1: the perturbed predicted vector is calculated using linear weighting;

[0084] C2: when the sample abnormality degree index is less than the threshold value, the perturbed predicted vector is more biased towards the random perturbation vector;

[0085] C3: when the sample abnormality degree index is greater than or equal to the threshold value, the perturbed predicted vector is more biased towards the predicted probability vector.

[0086] It should be noted that linear weighting combination is a simple and effective vector fusion method. The combination process realizes the adaptive fusion of the original predicted vector and the perturbation vector by using the abnormality degree index as the weight. When the abnormality degree index is α, the calculation method of the perturbed predicted vector is: α times the original predicted vector plus (1-α) times the perturbation vector. Because the value of α is in the range of 0 to 1, the combined vector still satisfies the probability distribution property.

[0087] The threshold value is usually set to 0.5, so that when the abnormality degree index is less than 0.5, the combination weight (1-α) is greater than 0.5, and the perturbation vector dominates in the final result; otherwise, the original predicted vector dominates. The selection of the threshold value needs to balance between defense effect and service quality.

[0088] For example, for an object recognition task, when an attack sample is detected (the abnormality degree index is 0.2), the combination calculation is: 0.2 times the original predicted vector plus 0.8 times the perturbation vector, and the final output mainly reflects the characteristics of the perturbation vector, effectively misleading the attacker. For a normal query (the abnormality degree index is 0.9), the combination calculation is: 0.9 times the original predicted vector plus 0.1 times the perturbation vector, and the final output basically maintains the accuracy of the original prediction, only introducing slight perturbation.

[0089] In an alternative embodiment, the weighted combination in step S300 can also adopt a nonlinear combination manner. For example, the abnormality degree indicator is transformed using a quadratic function or an exponential function to obtain a nonlinear combination weight. The nonlinear transformation can achieve different combination strategies in different intervals of the abnormality degree indicator, such as adopting a more aggressive weight allocation in an extreme case (very normal or very abnormal).

[0090] In another alternative embodiment, the weighted combination in step S300 can also introduce a dynamic adjustment mechanism. According to historical query records and defense effect feedback, the threshold value or the weight calculation manner is adjusted. For example, if it is detected that the number of abnormal queries increases in a period of time, the threshold value can be temporarily reduced to enhance the defense strength; when the system runs stably, the threshold value is appropriately increased to optimize the normal user experience.

[0091] In the embodiment of the present application, the discrete processing of the perturbed prediction vector in step S400 includes the following steps D1-D3:

[0092] D1: converting the category confidence whose continuous probability value is greater than a first threshold value into a first level;

[0093] D2: converting the category confidence whose continuous probability value is between a second threshold value and the first threshold value into a second level;

[0094] D3: converting the category confidence whose continuous probability value is less than the second threshold value into a third level;

[0095] Among them, the first level, the second level and the third level respectively correspond to high, medium and low discrete level representations.

[0096] It should be noted that the discrete processing is the last line of defense of the defense mechanism. By mapping the continuous probability value to a limited discrete level, the accurate numerical information is completely eliminated. Because the attacker cannot infer the original continuous probability value from the discrete level, it is difficult to accurately estimate the decision boundary of the model even if a large number of query results are obtained.

[0097] The setting of the first threshold value and the second threshold value needs to consider the needs of the actual application scene. In the scene that needs to retain more information, more discrete levels and threshold values can be set; in the scene that requires higher security, only two levels (such as high and low) can be used. The specific value of the threshold value is usually determined through experiments to ensure that the discrete result can provide necessary classification information and effectively prevent model stealing.

[0098] For example, in the sensitive data classification scenario, the original continuous probability values such as 0.92, 0.68, 0.45, 0.23, etc. are uniformly represented as high, medium, medium, and low after discretization processing. Attackers can only know the confidence level of a certain category in a certain interval, and cannot obtain the accurate probability value. Even if the same discrete level is obtained by multiple queries for the same category, the specific probability range of the category cannot be inferred.

[0099] In an optional implementation, the discretization processing in step S400 can also use fuzzy representation. In addition to using a certain level label, a fuzzy set theory can be used to assign multiple levels of membership to each probability value. For example, the probability value 0.48 can belong to both the medium level (membership 0.7) and the low level (membership 0.3). Fuzzy representation can provide more semantic information while protecting accurate information.

[0100] In another optional implementation, the discretization processing in step S400 can also combine a randomization strategy. The probability values near the level boundary are classified into adjacent levels with a certain probability. For example, the probability value 0.51 (slightly higher than the threshold value of 0.5) has an 80% probability of being classified as a high level and a 20% probability of being classified as a medium level. The randomization strategy further increases the difficulty for attackers to guess the original probability value.

[0101] In summary, the present application realizes effective protection of the classification model through four key steps: first, identifying potential attack queries through anomaly detection, then generating a random perturbation vector with category difference, then adaptively combining the original prediction and the perturbation vector according to the anomaly degree, and finally eliminating the accurate numerical information through discretization processing. The entire defense process not only ensures effective interference to attackers, but also maintains the service quality of normal users, providing a feasible technical solution for the secure deployment of sensitive data classification models. Embodiment 3 is a third embodiment of the present application, which provides a classification model theft defense system for discretization perturbation sensitive data prediction, comprising:

[0102] An anomaly detection module for obtaining a prediction probability vector of an input sample, and calculating a sample anomaly degree index according to the prediction probability vector;

[0103] A perturbation generation module for generating a random perturbation vector that satisfies a constraint condition based on the sample anomaly degree index, wherein the maximum probability position of the random perturbation vector is different from the maximum probability position of the prediction probability vector;

[0104] An adaptive combination module for determining a combination weight according to the sample anomaly degree index, and combining the prediction probability vector and the random perturbation vector based on the combination weight to generate a perturbed prediction vector;

[0105] The discretization processing module is configured to perform discretization processing on the perturbed prediction vector, and convert continuous probability values into discrete level representations.

[0106] Embodiment 4 is the fourth embodiment of the present application, which is different from the first three embodiments in that: if the function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0107] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logical functions, and can be specifically embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system that includes a processor, or other system that can fetch the instructions from an instruction execution system, apparatus, or device and execute the instructions. For the purposes of this specification, a computer-readable medium can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device, or in conjunction with these instruction execution systems, apparatus, or devices.

[0108] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic editing, interpretation, or processing, if necessary, in other suitable ways, and then stored in a computer memory.

[0109] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0110] Example 5, refer to Figure 1 For the fifth embodiment of the present application, a discrete perturbation sensitive data prediction classification model theft defense method is provided. In order to verify the beneficial effects of the present application, scientific demonstration is carried out through experiments.

[0111] The experimental data comes from classical image classification data sets, including medical image data set MedMNIST, general object image recognition data set CIFAR10 and document scanning image data set RVL-CDIP. The experimental data is shown in Table 1.

[0112] Table 1 Model theft defense experimental data (column)

[0113]

[0114] The software and hardware environment relied on by the experiment is shown in Table 2 and Table 3.

[0115] Table 2 List of software used in the experiment

[0116]

[0117] Table 3 List of hardware used in the experiment

[0118]

[0119]

[0120] The evaluation index and calculation method used in the experiment are as follows.

[0121] (1) Target model accuracy (F_acc). Calculate the accuracy F_acc of the target model under the conditions of with and without defense, respectively, i.e.:

[0122] F_acc = iTPi total

[0123] Where TPi represents the number of samples of a certain class correctly classified as that class, iTPi represents the total number of correctly classified samples, and total represents the total number of samples.

[0124] (2) Substitute model accuracy (F'_acc). The accuracy of the substitute model generated after the target model is attacked by different model stealing attack methods is calculated respectively, and the calculation formula is the same as the accuracy of the target model.

[0125] (3) Accuracy drop degree (decacc). The drop degree decacc of the values of the target model accuracy Facc and the substitute model accuracy F'_acc in the defense state compared with the non-defense state is calculated, that is:

[0126] dec_acc = accdef - accund

[0127] Wherein, accdef represents the accuracy of the model M in the defense state, and accund represents the accuracy of the model M in the non-defense state.

[0128] The experimental process and parameter settings are as follows:

[0129] (1) The target model is trained using the MedMNIST dataset, and the experimental parameter settings are shown in Table 4; the target model test set is used as the benign user query sequence Q, and the accuracy of the target model in the non-defense state is tested;

[0130] Table 4 Training parameters and values of the target model

[0131]

[0132] (2) The target model is subjected to NPD attack and AdvPD attack respectively, and the accuracy of the generated substitute model is tested using the same benign user query sequence Q as in (1); the discrete confidence is converted into a random value satisfying the normal distribution during the attack process;

[0133] (3) The defense method described in the application is used to modify the internal structure of the target model to obtain a defense model, and the defense model is trained. The model structure and training parameters of the defense model are the same as those of the corresponding target model; the accuracy of the target model in the defense state is tested using the same benign user query sequence Q as in (1), and the drop degree compared with the accuracy of the target model in the non-defense state is tested;

[0134] (4) The defense target model is subjected to NPD attack and AdvPD attack respectively, and the attack process parameter settings are shown in Table 5; the accuracy of the generated substitute model is tested using the same benign user query sequence Q as in (1), and the drop degree compared with the accuracy of the substitute model in the non-defense state is tested;

[0135] (5) Train the target model using CIFAR10 and RVL-CDIP data sets respectively, and the parameter settings are shown in Table 4; perform steps (1) to (4) above, and test the accuracy and accuracy reduction of the target model and the substitute model under different conditions.

[0136] Table 5 Model stealing attack process parameters and values

[0137]

[0138] Exemplarily, the specific process of the present application is as follows:

[0139] Step 1, calculate the malicious degree of the input sample by using the out-of-distribution detection method.

[0140] Step 1.1, let the input sample be x, and the prediction probability vector of the target model for the input sample be Y = [y1, y2…, yn], where the maximum value is ymax.

[0141] Step 1.2, the attack sample recognition threshold is δ, then when ymax is greater than δ, x is a normal sample, otherwise it is an attack sample. The malicious degree value α of the input sample is generated by a sigmoid function S(z):

[0142] α = S(ymax-δ)

[0143] Sz = 11 + e-z

[0144] Step 2, generate a random disturbance vector combining the maximum probability position of the original prediction vector and the preset peak value.

[0145] Step 2.1, let the input sample be x, and the prediction probability vector of the target model F for the input sample be Y = (y1, y2,…, yn), let the maximum value ymax correspond to the class c, and record the index of the class c as the maximum probability bit M.

[0146] Step 2.2, randomly generate an n-dimensional disturbance probability vector Y' = (y'1, y'2,…, y'n), let the maximum value be y'max, and at the same time, in order to make the disturbance vector mislead the attacker to generate a substitute model with a higher probability, it is required that there is a peak value in Y', so that y'max is not less than 0.8.

[0147] Step 2.3, randomly generate an integer m between 1 and n, if m is not equal to M, then:

[0148] M = m

[0149] ym = y'max

[0150] If m is equal to M, then m is regenerated until the two are not equal. The remaining n-1 probabilities are randomly assigned to the index positions of the n-dimensional perturbation probability vector (y'1, y'2,..., y'n) except the index position of y'max, thereby obtaining the prediction probability vector Y' after random perturbation.

[0151] Step 3, based on the sample malicious value, the perturbation strength is calculated, the original prediction vector and the random perturbation vector are combined to generate the final perturbation vector as the model output to prevent the attacker from stealing.

[0152] Step 3.1, the final perturbation vector Y is generated by combining a sigmoid function S(z):

[0153] Y = aY + 1-aY'

[0154] Where Y represents the original target model prediction probability vector, Y' represents the prediction probability vector after random perturbation, and a is the output of the sample malicious degree calculation module. Thus, when the input sample is identified as an attack sample, a [0, 0.5], at this time Y is more biased towards the prediction result Y' after random perturbation; when the input sample is identified as a normal sample, a [0.5, 1], at this time Y is more biased towards the prediction result Y of the original target model.

[0155] Step 4, the continuity of the model output numerical vector is discretized.

[0156] Step 4.1, the class confidence greater than 0.5 in the prediction probability vector is output as high, the class confidence between 0.3 and 0.5 is output as medium, and the class confidence less than 0.3 is output as low.

[0157] In summary, the model theft defense method proposed by the present application can make the accuracy of the substitute model generated by the attacker decrease more greatly in a specific attack scenario. The present application achieves the best results on two models (LeNet and ResNet18), among which the accuracy of the substitute model decreases the most on the MedMNIST dataset, reaching 65.21%. At the same time, the influence on the accuracy of the target model is small, and the decrease is within an acceptable range. In particular, on the nnRVL-CDIP dataset, the decrease in the accuracy of the target model is optimal, as low as 0.2%.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A method for defense against stealing of a classification model for discretized perturbed sensitive data prediction, the method comprising: comprising, ​ obtaining a prediction probability vector of an input sample, and calculating a sample anomaly degree index according to the prediction probability vector; generating a random perturbation vector satisfying a constraint condition based on the sample anomaly degree index, wherein a maximum probability position of the random perturbation vector is different from a maximum probability position of the prediction probability vector; determining a combination weight according to the sample anomaly degree index, and combining the prediction probability vector and the random perturbation vector based on the combination weight to generate a perturbed prediction vector; performing discretization processing on the perturbed prediction vector to convert continuous probability values into discrete level representations.

2. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 1, wherein: The calculation of the sample anomaly degree index according to the prediction probability vector comprises: extracting a maximum probability value from the prediction probability vector; comparing the maximum probability value with an anomaly detection threshold value; calculating the sample anomaly degree index through a mapping function based on the comparison result of the maximum probability value and the anomaly detection threshold value.

3. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 2, wherein: According to the comparison result, the input sample is identified as different sample categories; based on the sample category, the mapping function is used to generate the corresponding sample anomaly degree index.

4. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 3, wherein: The generation of the random perturbation vector satisfying the constraint condition comprises: identifying a category identifier corresponding to the maximum probability position in the prediction probability vector; generating a perturbation probability distribution corresponding to the prediction probability vector; setting a peak value in the perturbation probability distribution, and assigning the peak value to a position different from the category identifier; processing the perturbation probability distribution to obtain the random perturbation vector.

5. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 4, wherein: The numerical value of the peak value is not less than a peak threshold value; if the generated peak position is the same as the position corresponding to the category identifier, the peak position is regenerated until they are different; the remaining probability values are randomly assigned to the elements other than the peak value in the perturbation probability distribution.

6. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 5, wherein: The combination of the prediction probability vector and the random perturbation vector based on the combination weight comprises: calculating the perturbed prediction vector in a linear weighting manner; when the sample anomaly degree index is less than a boundary value, the perturbed prediction vector is more biased towards the random perturbation vector; when the sample anomaly degree index is greater than or equal to the boundary value, the perturbed prediction vector is more biased towards the prediction probability vector.

7. The discretized perturbed sensitive data prediction classification model exfiltration defense method of claim 6, wherein: The discretization processing of the perturbed prediction vector comprises: converting a category confidence value greater than a first threshold value into a first level; converting a category confidence value between a second threshold value and the first threshold value into a second level; converting a category confidence value less than the second threshold value into a third level; wherein the first level, the second level and the third level correspond to high, medium and low discrete level representations, respectively.

8. A system for defense against stealing of a classification model for prediction of discretized perturbation-sensitive data, applying the method for defense against stealing of a classification model for prediction of discretized perturbation-sensitive data according to any one of claims 1 to 7, characterized in that comprising: an anomaly detection module configured to obtain a prediction probability vector of an input sample, and calculate a sample anomaly degree index according to the prediction probability vector; The disturbance generation module is configured to generate a random disturbance vector satisfying a constraint condition based on the sample anomaly degree index, wherein a maximum probability position of the random disturbance vector is different from a maximum probability position of the predicted probability vector; The adaptive combination module is configured to determine a combination weight according to the sample anomaly degree index, combine the predicted probability vector and the random disturbance vector based on the combination weight, and generate a disturbed predicted vector. The discretization processing module is configured to perform discretization processing on the disturbed predicted vector, and convert continuous probability values into discrete level representations. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the discrete perturbation sensitive data prediction classification model stealing defense method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the discrete perturbation sensitive data prediction classification model stealing defense method in any one of claims 1 to 7.

Images