Method and device for generating individual prejudice sample in deep neural network

Abstract

The invention relates to the field of deep learning, in particular to a method and a device for generating individual prejudice samples in a deep neural network. Comprising the following steps: selecting clustering centers in different areas of a data set, clustering original data in the data set according to the clustering centers, and taking clustered samples as clustering results; according to global protection attributes of the samples, judging whether the samples in the clustering result are prejudice samples or not, performing global disturbance on the samples which are judged to be negative, and taking the samples which are judged to be positive and the samples subjected to global disturbance as global prejudice samples; and selecting local protection attributes under gradient guidance, performing local disturbance on the local protection attributes in the global prejudice samples to obtain local prejudice samples, and taking the global prejudice samples and the local prejudice samples as generated individual prejudice samples. According to the method, the problem of overfitting possibly caused by accelerating the iteration process by introducing momentum optimization is solved, the complexity of the generation process is controlled, and the generalization of the method is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and in particular to a method and device for generating individual bias samples in a deep neural network. Background Art

[0002] Deep Neural Network (DNN) models are gradually being widely used, including fraud detection, facial recognition, autonomous driving, and medical diagnosis. Although DNN models have shown amazing performance in many applications, their reliability is still a concern. For applications with social impact, a detectable property of DNN models is fairness (i.e., non-discrimination). Since social bias often exists in training data, the resulting DNN model may inadvertently introduce bias. Therefore, it is necessary to train the DNN model by producing individual bias samples so that the DNN model can complete bias detection.

[0003] In the existing bias sample generation methods, in order to detect the diversity of individual bias samples globally as much as possible, it is necessary to cluster the samples in the original data set, usually using the K-means algorithm. The advantages of this algorithm are its simplicity, ease of implementation and high efficiency, but in practical applications, it is very sensitive to the selection of the initial cluster center, and is sensitive to outliers and noise, making it difficult to find clusters of very different sizes and perform incremental calculations. In other bias sample generation methods, gradient guidance is introduced in the detection process, and the momentum optimization algorithm is used based on gradient guidance. The effectiveness and computational efficiency of generating bias samples have been greatly improved, but there will be problems with model overfitting.

[0004] In view of this, how to overcome the defects of the existing technology and solve the problem of poor quality of individual bias samples produced by the existing bias generation method is a problem to be solved in this technical field. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention solves the problem of poor quality of individual bias samples produced by the existing bias generation method.

[0006] The embodiment of the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for generating individual biased samples in a deep neural network, specifically: selecting cluster centers in different areas of a data set, clustering the original data in the data set according to the cluster centers, and using the clustered samples as clustering results; judging whether the samples in the clustering results are biased samples according to the global protection attributes of the samples, globally perturbing the samples with a negative judgment result, and using the samples with a positive judgment result and the globally perturbed samples as global biased samples; selecting local protection attributes under the guidance of gradients, locally perturbing the local protection attributes in the global biased samples to obtain local biased samples, and using the global biased samples and the local biased samples as generated individual biased samples.

[0008] Preferably, the selecting cluster centers in different regions of the data set specifically includes: selecting a specified number of candidate cluster centers from the samples according to the distances between different samples in the data set; assigning the samples to the candidate cluster centers that are closest according to the distances between the remaining samples and the candidate cluster centers, each candidate cluster center and its assigned samples forming an area; in each area, updating the candidate cluster centers of the area to the samples of all sample centers according to the distances of all samples; iteratively updating the updated candidate cluster centers until the candidate cluster centers in each area no longer change, and using the candidate cluster centers that no longer change as the cluster centers of each area.

[0009] Preferably, the method of selecting a specified number of candidate cluster centers from the samples based on the distances between different samples in the data set specifically includes: randomly selecting a sample as the initial centroid, and calculating the square of the distance from the initial centroid to each sample; calculating the candidate probability of each sample based on the square of the distance from the initial centroid to each sample and the total number of samples, and taking the sample with the highest probability as the next centroid; iteratively calculating the next centroid until the number of obtained centroids reaches the specified number, and taking the obtained centroids as candidate cluster centers.

[0010] Preferably, the method of determining whether a sample in the clustering result is a biased sample based on the global protection attribute of the sample specifically includes: taking each sample in the clustering result as a first sample in turn, and for each first sample, searching whether a second sample exists; when the second sample differs from the first sample only in the global protection attribute, taking the first sample and each second sample as a pair of biased samples.

[0011] Preferably, the globally perturbed samples with a negative judgment result specifically include: obtaining the local gradient of the sample according to the gradient direction of the sample, regularizing the local gradient to obtain a gradient guidance result, wherein each dimension in the gradient guidance result represents the global perturbation direction of the corresponding attribute; iteratively globally perturbing the corresponding attributes in the sample according to the gradient guidance result until a biased sample is obtained or the maximum number of iterations is reached, and using the globally perturbed sample as a global biased sample.

[0012] Preferably, the regularizing the local gradient to obtain the gradient guidance result specifically includes: regularizing the local gradient using the following formula:

[0013]

[0014] Among them, L is the result of the loss function Loss, E in is the sample error without regularization term, λ is the regularization parameter, is the square of the momentum vector acceleration weight, and j is the number of dimensions of the momentum vector.

[0015] Preferably, the global perturbation direction includes descending, unchanged and ascending, and the iterative global perturbation of the corresponding attributes in the sample according to the gradient guidance result, for each non-global protection attribute, specifically includes: when the global perturbation direction is descending, reducing the value of the non-global protection attribute by 1; when the global perturbation direction is unchanged, the value of the non-global protection attribute remains unchanged; when the global perturbation direction is ascending, increasing the value of the non-global protection attribute by 1.

[0016] Preferably, the selecting of the local protection attribute under the guidance of the gradient specifically includes: obtaining the gradient of each non-global protection attribute in the global bias sample, normalizing the inverse of the sum of the absolute values ​​of the gradients, and taking the non-global protection attribute with the largest normalized result as the local protection attribute.

[0017] In a second aspect, the present invention provides a device for generating individual bias samples in a deep neural network, specifically: comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for generating individual bias samples in the deep neural network in the first aspect.

[0018] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method described in the first aspect.

[0019] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0020] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0021] Compared with the prior art, clustering centers are selected in multiple regions for clustering, so that the global bias samples found in the global generation are more diverse, and the accuracy of the fairness detection of the DNN model is improved; at the same time, in the preferred scheme of the present invention, the square of the momentum vector acceleration weight is used for regularization, which solves the overfitting problem that may be caused by introducing momentum optimization to speed up the iteration process, controls the complexity of the generation process, and improves the generalization of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flow chart of a method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0024] Figure 2 A flow chart of another method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0025] Figure 3 A flow chart of another method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0026] Figure 4 A flow chart of another method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0027] Figure 5 A flow chart of another method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the implementation process of the generation method provided by an embodiment of the present invention in a certain scenario;

[0029] Figure 7 A schematic diagram of the structure of a device for generating individual bias samples in a deep neural network provided by an embodiment of the present invention;

[0030] The reference numerals are as follows:

[0031] 11: processor; 12: memory. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0034] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0035] Embodiment 1:

[0036] In order to solve the defects of existing individual bias generation methods, it is necessary to improve the quality of clustering, reduce the deviation of clustering results, and at the same time control the complexity of the DNN model and improve the generalization of the DNN model.

[0037] like Figure 1 As shown, the specific steps of the method for generating individual bias samples in a deep neural network provided by an embodiment of the present invention are as follows:

[0038] Step 101: Select cluster centers in different areas of the data set, cluster the original data in the data set according to the cluster centers, and use the clustered samples as clustering results.

[0039] The generation method provided in this embodiment includes two parts: a global generation part and a local generation part. In the global generation process, the collected raw data is first preprocessed to form a data set, the samples in the data set are clustered, and seed samples are selected from each cluster in a cyclic manner to perform a global generation algorithm. Furthermore, in the global generation part, the quality of clustering is improved by optimizing the selection of the initial cluster center in the clustering operation, so that the cluster centers selected by the clustering are as dispersed as possible, so as to reduce the deviation of the clustering results and improve the detection accuracy of the DNN model.

[0040] Step 102: According to the global protection attribute of the sample, determine whether the sample in the clustering result is a biased sample, perform global perturbation on the sample with a negative determination result, and use the sample with a positive determination result and the globally perturbed sample as the global biased sample.

[0041] When perturbing, the values ​​of certain attributes should not change under any circumstances, so these attributes cannot be perturbed, such as attributes that serve as unique identifiers, which are global protection attributes, and the remaining changeable attributes are non-global protection attributes. In some scenarios, some non-global protection attributes should not change or do not need to be perturbed, such as attributes that currently require bias detection, which are local protection attributes, and the remaining attributes are non-local protection attributes. In this embodiment, global protection attributes and local protection attributes are collectively referred to as protection attributes, and non-global protection attributes and non-local protection attributes are collectively referred to as non-protection attributes.

[0042] In the global generation part, as many different types of global bias samples as possible are generated. Each sample is obtained from the clustering results in a cyclic manner to determine whether the sample is a bias sample. If the judgment is yes, the sample is a bias sample. If the judgment is no, the sample is iteratively globally perturbed under the guidance of the predicted category gradient until a single bias sample is generated or the maximum number of iterations is reached, and then it is used as a bias sample.

[0043] Step 103: Select local protection attributes under the guidance of gradient, locally perturb the local protection attributes in the global bias sample to obtain the local bias sample, and use the global bias sample and the local bias sample as the generated individual bias sample.

[0044] In the local generation process, a global bias sample is selected and locally perturbed to minimize the change in the DNN model output to maintain the DNN model's prediction of the sample. At this point, the perturbed global bias sample is still a bias sample, and only differs in some non-protective attributes. Therefore, a new individual bias sample can be obtained based on the original sample.

[0045] After steps 101 to 103 provided in this embodiment, a high-quality individual bias sample can be obtained.

[0046] In a certain scenario of this embodiment, three data sets are used: data set 1, data set 2, and data set 3, all of which are data sets commonly used in fairness detection; attribute 1, attribute 2, and attribute 3 are selected as protection attributes. In this embodiment, adversarial samples are generated on these three protection attributes respectively to detect whether the DNN model is biased on these three protection attributes.

[0047] The existing K-means algorithm is very sensitive to the selection of the initial cluster center and is prone to fall into the local optimal solution, resulting in poor clustering effect. In this embodiment, the initial cluster center is selected in a more reasonable way, so that the algorithm is more likely to find the global optimal solution instead of falling into the local optimum. The quality of clustering is improved by optimizing the selection of the initial cluster center, making the selected centroid as dispersed as possible to reduce the deviation of the clustering result, thereby improving the stability and quality of clustering.

[0048] like Figure 2 As shown, cluster centers can be selected in different regions of the dataset in the following way.

[0049] Step 201: According to the distances between different samples in the data set, a specified number of candidate cluster centers are selected from the samples.

[0050] In the data set, each sample corresponds to a data point. According to the attributes of the sample, the position of each data point and the distance between different data points can be obtained.

[0051] First, select a data point i as the initial centroid. In general scenarios, the initial centroid can usually be randomly selected. Starting from the initial centroid, the candidate probability of each sample is calculated based on the distance between the remaining data points i and the initial centroid, and the sample with the highest candidate probability is used as the next centroid.

[0052] Repeat the above process until the specified number of c_num centroids is obtained, and all the obtained centroids are used as candidate cluster centers. In actual implementation, the number of centroids is consistent with the number of cluster centers when the DNN model performs bias detection. In general scenarios, the value of c_num is set to 4.

[0053] Step 202: According to the distances between the remaining samples and the candidate cluster centers, the samples are assigned to the candidate cluster centers with the closest distances. Each candidate cluster center and its assigned samples form a region.

[0054] When clustering, each cluster center needs to be used as the regional center, and the samples closest to the cluster center need to be divided into the region of each cluster center. After obtaining the candidate cluster center, the distance between the remaining samples and each candidate cluster center needs to be calculated, and each sample needs to be assigned to the candidate cluster center closest to it to complete the regional division.

[0055] Step 203: In each region, based on the distances of all samples, the candidate cluster center of the region is updated to the sample of all sample centers.

[0056] After completing the region division in step 202, each candidate cluster center is not necessarily the actual center of the corresponding region. Therefore, it is necessary to calculate the actual cluster center of each region based on the distance between samples. In actual implementation, the mean distance of all samples assigned to each region can be used as the updated cluster center of the region.

[0057] Step 204: iteratively update the updated candidate cluster centers until the candidate cluster centers in each region no longer change, and use the candidate cluster centers that no longer change as the cluster centers of each region.

[0058] According to the updated cluster center point, the region division in step 202 and the cluster center division in step 203 are performed again until the cluster center no longer changes, which indicates that the best region division has been obtained and the best cluster center in each region has been obtained. According to the test results in the actual scene, the clustering method provided in this embodiment can increase the number of biased samples by about 20%.

[0059] After steps 201 to 204 provided in this embodiment, dispersed cluster centers of multiple regions can be obtained, thereby reducing the deviation of clustering results.

[0060] like Figure 3 As shown, in step 201, a specified number of candidate cluster centers may be selected from the samples according to the distances between different samples in the data set in the following manner.

[0061] Step 301: Randomly select a sample as the initial centroid and calculate the square of the distance from the initial centroid to each sample.

[0062] After obtaining the initial centroid, use the initial centroid as the current centroid c and calculate the square of the distance d(i,c) from the current centroid c to the data point i.

[0063] Step 302: Calculate the candidate probability of each sample based on the square of the distance from the initial centroid to each sample and the total number of samples, and use the sample with the highest probability as the next centroid.

[0064] For the current centroid c, calculate the candidate probability of each data point i as a candidate cluster center. The candidate probability is inversely proportional to the square of the distance between the current centroid c and the data point i. The candidate probability can be calculated according to the following formula:

[0065]

[0066] Among them, P(i) is the candidate distance of data point i, d(i,c) is the square of the distance between the current centroid c and data point i, sum_j() represents the sum of the reciprocals of the distances between all data points i and the current centroid c, and j represents the index of data point i.

[0067] After obtaining the candidate probabilities of all data points, the data point with the highest candidate probability is taken as the next centroid.

[0068] Step 303: Iteratively calculate the next centroid until the number of obtained centroids reaches the specified number, and use the obtained centroids as candidate cluster centers.

[0069] The next centroid obtained in step 302 is used as the current centroid, and steps 301 and 302 are repeated to obtain a new centroid each time until the number of centroids reaches the specified number c_num.

[0070] After steps 301 to 303 provided in this embodiment, all candidate cluster centers can be obtained.

[0071] After clustering each region according to the above cluster centers, biased samples can be obtained in the clustering results. Specifically: each sample in the clustering results is taken as the first sample in turn, and for each first sample, whether there is a second sample is found; when the second sample and the first sample differ only in the global protection attribute, the first sample and each second sample are taken as a pair of biased samples.

[0072] In a real scenario, the output results of a DNN model D trained on a dataset X may contain discrimination.

[0073] The attribute set A of the data set X = {A1, A2, A3, A4, ..., An}. Assume that each attribute A i There is a valuation domain I i , then the input domain for clustering is I = I1 × I2 × … × I n , the input domain represents all possible combinations of attribute A valuation. In the following, P is used to represent the protected attribute set, and NP is used to represent the non-protected attribute set, where For example: when the protection attributes are A2 and A3, P = {A2, A3}, NP = {A1, A4, ..., An}.

[0074] Each sample x in the dataset X = {x1, x2, ..., x n}, where x i It is attribute A i The value of x i The value of satisfies the valuation domain I i . If there is another sample x′∈I that satisfies the following conditions, then x is an individual bias sample used when training the DNN model D. The conditions that the sample x′ must satisfy include:

[0075] (a) This condition means: a protection attribute p is selected from the protection attribute set P, and the sample x and the sample x' are different in the protection attribute p.

[0076] (b) This condition means that there is a non-protected attribute q in the non-protected attribute set NP, and sample x and sample x' are the same in any non-protected attribute q.

[0077] (c) D(x)≠D(x′), which means that the model outputs different results for sample x and sample x′.

[0078] For any set of samples x and samples x′ that meet the conditions, (x, x′) is called an individual bias sample pair.

[0079] In a specific implementation, a loop method can be used to sequentially obtain each sample in the clustering result, and the obtained sample is called seed x. If seed x meets the judgment conditions of the above-mentioned individual biased samples, it is added to the biased sample set as a global biased sample. In a specific implementation, for each seed x, all samples that differ from seed x only in protection attributes are added to the set similar_x, and the set size is the number of all possible combinations of protection attributes in I except seed x. Then, by searching similarr_x, it is determined whether seed x is a biased sample. If a sample has a prediction result different from seed x, then according to the conditions of the above-mentioned individual biased samples, seed x is a separate biased sample, and it is added to the biased sample set g_id. Then, continue to determine whether the next seed x is a biased sample until the determination of each sample in the clustering result is completed.

[0080] In a specific scenario, in data set 1, the attribute set A = {A1, A2, A3, A4, A5}, there is a seed x = [60, 2, 1, 1, 1], and the sample x′ = [60, 2, 1, 1, 0] is found. Sample x and sample x′ differ only in A5, which is the protection attribute, and are the same in other attributes. However, the outputs of the two samples through the DNN model D are not the same. Then these two samples are biased sample pairs, and samples x = [60, 2, 1, 1, 1] and x′ = [60, 2, 1, 1, 0] will both be added to the biased sample set g_id.

[0081] On the other hand, if the seed x does not meet the definition of a biased sample, a gradient-based search is further performed in the clustering results, thereby iteratively perturbing the seed x globally under the guidance of the predicted category gradient until a single instance that can be used for bias discrimination is generated, or the maximum number of iterations max_iter is reached. In general scenarios, the maximum number of iterations is set to 10. In specific implementations, the set similar_x can be traversed, and a sample x′ can be selected from it. The result of the perturbation of sample x is F(x), and the result of the perturbation of sample x′ is F(x′). Through global perturbation, ||F(x)-F(x′)||2 is maximized, that is, the 2-norm of the two samples is maximized. The segmentation result is intuitively represented as the distance between the two samples is as far as possible, making x and x' easier to be segmented by the decision boundary.

[0082] The segmentation results are as follows Figure 4 In the figure, (x, x′) is the biased sample pair before perturbation. After global perturbation of x and x′, a new biased sample pair (x a , x a ′), that is, the biased sample pair found after the global perturbation is successful. In the specific implementation, when no perturbation is performed, the DNN model outputs the same results for sample x and sample x′; after perturbing the attributes of sample x and sample x′ except for the protection attribute, if the DNN model outputs the same results for the perturbed sample x a and sample x a ′The output results are inconsistent, then the sample x a and sample x a ' is a pair of new biased samples, sample x a and sample x a ′ are all new bias samples.

[0083] Further, in this embodiment, the gradient information in each time step is accumulated by momentum optimization to simulate the inertial effect of the object in motion. In actual calculations, momentum can be represented by a momentum vector, which can memorize the gradient in the previous time step and gradually accumulate the previous gradient information at different time steps. When the gradient direction of the current time step is consistent with the gradient direction of the previous time step, the momentum vector will accelerate the weight update; when the gradient direction of the current time step is inconsistent with the gradient direction of the previous time step, the momentum vector will slow down the weight update. This helps to stabilize the update direction, get rid of the dilemma of local minimum or maximum value, smooth the update process to a certain extent, reduce shock, and make the weight update more stable, and can promote the iterative search process and improve the success rate of discovering individual differences in the global generation process.

[0084] like Figure 5As shown, the seed x that does not meet the definition of biased samples can be globally perturbed in the following way.

[0085] Step 401: Obtain the local gradient of the sample according to the gradient direction of the sample, and regularize the local gradient to obtain a gradient guidance result.

[0086] For each bias sample pair (x, x'), calculate the gradient of the momentum vector input in each time step, and convert the momentum vector in the previous time step into a local gradient by multiplying it by the attenuation coefficient η. In the specific implementation, the value of the attenuation coefficient η can be adjusted to ensure that the best bias sample generation performance can be achieved. Then normalize the local gradient to obtain the gradient guidance result.

[0087] Step 402: Perform iterative global perturbations on the corresponding attributes in the sample according to the gradient guidance result until a biased sample is obtained or the maximum number of iterations is reached, and the globally perturbed sample is used as a global biased sample.

[0088] Each dimension in the gradient guidance result represents the global perturbation direction of the corresponding attribute. The global perturbation direction represented by each dimension in the gradient guidance result adjusts the value of each dimension in the attribute of the sample to obtain the perturbed sample. In the specific implementation, when the global perturbation direction is descending, the value of the non-global protection attribute is reduced by 1; when the global perturbation direction is unchanged, the value of the non-global protection attribute remains unchanged; when the global perturbation direction is ascending, the value of the non-global protection attribute is increased by 1.

[0089] After step 401 and step 402 provided in this embodiment, the global perturbation can be completed under the guidance of the gradient.

[0090] In the regularization process of step 401, in order to prevent possible overfitting, the square sum of the weight parameters can be added to the original loss function to speed up the global perturbation in the momentum optimization step in the global generation algorithm. The regularization formula is as follows:

[0091]

[0092] Among them, L is the result of the loss function Loss, E in is the sample error without regularization term, λ is the regularization parameter, is the square of the momentum vector acceleration weight, and j is the number of dimensions of the momentum vector.

[0093] The above regularization method will make the weight value smaller, but it will not directly lead to weight sparseness, so it does not have the effect of feature selection, and can effectively control the complexity of the DNN model and improve the generalization of the DNN model. After using the above regularization method, the efficiency of generating global biased samples will be improved, while avoiding the problem of skewed cluster centers that may arise in the multi-region clustering process, and improving the stability of the algorithm and clustering quality. According to the test results in actual scenarios, the regularization method provided in this embodiment can improve the efficiency of the algorithm, and the speed of generating biased samples can be improved by about 10%.

[0094] In a specific scenario, in data set 3, the attribute set A = {A1, A2, A3, A4, A5}, there are samples x = [1, 0, 0, 40, 100], and samples x′ = [0, 0, 0, 40, 100]. These two samples are different in attribute A1 and protection attribute A5, but are the same in other attributes. At the same time, they are also the same class in the output results of DNN model D. Therefore, this pair of samples can be used to perturb the same non-protection attribute to generate a pair of new biased samples, and then perform global perturbations on the next two attributes, while ensuring that the global perturbations are within a certain range and will not change the original outputs of the two samples at the same time. In this scenario, the attenuation coefficient η is set to 0.5, and the gradient guidance result is [1, 0, -1, 1, -1]; where 1 indicates that the global perturbation direction is rising, 0 indicates that the global perturbation direction is unchanged, and -1 indicates that the global perturbation direction is falling. Since attribute A3 is already 0 and cannot be reduced, attribute A3 is still set to 0. On the other hand, in this embodiment, the maximum value of attribute A4 is 40. The 100 in the sample is judged to be an error value, and it is normalized to 40, and 1 is subtracted according to the gradient guidance. After the perturbation is completed, a new global bias sample pair is obtained: sample x = [1, 0, 0, 41, 39], sample x′ = [0, 0, 0, 41, 39].

[0095] After completing the global perturbation, for each seed x, a similar sample x′ can always be found in the bias sample set g_id, so that D(x)≠D(x′), that is, the output of the DNN model to be tested for sample x and sample x′ is different. At this time, the non-protective attributes in the sample can be further selected according to the gradient for local perturbation to continuously generate more individual bias samples. In this embodiment, the gradient output by the DNN model is used to guide attribute selection and perturbation. Specifically: obtain the gradient of each non-global protection attribute in the global bias sample, normalize the inverse of the sum of the absolute values ​​of the gradient, and use the non-global protection attribute with the largest normalization result as the local protection attribute.

[0096] In a certain scenario, on the global bias sample x = [1, 0, 0, 41, 39], the non-protected attributes are first used as contributing attributes to form a probability distribution, and the sum of the absolute values ​​of the gradients is calculated, and the sum of the absolute values ​​of the gradients is used as an indicator of the impact of each non-protected attribute on the prediction of the DNN model. In actual implementation, the attributes with less impact on the prediction of the DNN model tend to be selected. Therefore, the inverse of the sum of the absolute values ​​of the corresponding gradients can be normalized. After calculation, the probability distribution of attribute selection in the global bias sample x is [0, 0.002, 0.027, 0.612, 0.131], where each item represents the probability of selecting the corresponding attribute when perturbing. Due to the inverse normalization, when the attribute represented by the largest item in the probability distribution is selected for perturbation, the impact on the result of the DNN model prediction is the smallest. After selecting the attribute according to the probability distribution, the perturbation direction is determined according to the uniform random distribution in the two perturbation directions of the ascending and descending directions. If the perturbed sample x violates individual fairness, the sample x is used as the local biased sample, and the process is repeated to continuously generate biased samples; otherwise, the sample is reset to the original value, the attribute is reselected or the perturbation direction is re-determined to perturb again, or other samples are selected for perturbation.

[0097] In a certain scenario, the overall implementation process of the method provided in this embodiment is as follows: Figure 6 As shown, the sample set of the DNN model is used as a data set for generating individual bias samples, and individual bias samples are obtained through global generation and non-generation for use in individual bias detection of the DNN model.

[0098] The present embodiment provides a method for generating individual bias samples in a deep neural network. Under the premise of ensuring the accuracy and efficiency of generating individual bias samples, multiple cluster centers are used for regional clustering to improve the accuracy of clustering, optimize the selection of initial centroids, reduce the deviation of clustering results, and improve the diversity of generated individual bias samples; in the process of globally generating bias samples, a momentum optimization algorithm is used to accelerate the convergence speed of the neural network and improve the training efficiency; in order to prevent the overfitting problem, the square sum of the weight parameters is added to the original loss function during regularization to control the complexity of the DNN model and improve the generalization of the DNN model. Therefore, through the method provided in this embodiment, the quality of the generated biased samples can be optimized, and the accuracy and efficiency of the fairness detection of the DNN model can be improved.

[0099] Embodiment 2:

[0100] Based on the method for generating individual bias samples in a deep neural network provided in the above embodiment 1, the present invention also provides a device for generating individual bias samples in a deep neural network that can be used to implement the above method, such as Figure 7, which is a schematic diagram of the device architecture of an embodiment of the present invention. The device for generating individual bias samples in a deep neural network of this embodiment includes one or more processors 11 and a memory 12. Figure 7 A processor 11 is taken as an example.

[0101] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0102] The memory 12 is a non-volatile computer-readable storage medium as a method for generating individual bias samples in a deep neural network, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the method for generating individual bias samples in a deep neural network in Example 1. The processor 11 executes various functional applications and data processing of the device for generating individual bias samples in a deep neural network by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, the method for generating individual bias samples in a deep neural network in Example 1 is implemented.

[0103] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The program instructions / modules are stored in the memory 12, and when executed by one or more processors 11, the method for generating individual bias samples in the deep neural network in the above embodiment 1 is executed, for example, the above described Figure 1 Zhihe Figure 5 The steps shown.

[0105] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 7 A processor 11 in the embodiment may enable the one or more processors to execute the method for generating individual bias samples in a deep neural network in embodiment 1, for example, executing the above-described Figures 1 to 5 The steps shown can also be implemented Figure 6 The various modules and units described.

[0106] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0107] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating individual bias samples in a deep neural network, characterized in that: include: Select cluster centers in different areas of the data set, cluster the original data in the data set according to the cluster centers, and use the clustered samples as clustering results; According to the global protection attribute of the sample, determine whether the sample in the clustering result is a biased sample, perform global perturbation on the sample with a negative judgment result, and use the sample with a positive judgment result and the sample after global perturbation as the global biased sample; Under the guidance of gradient, local protection attributes are selected, and local protection attributes in global bias samples are locally perturbed to obtain local bias samples, and global bias samples and local bias samples are used as generated individual bias samples.

2. The method for generating individual bias samples in a deep neural network according to claim 1, characterized in that: The selecting of cluster centers in different regions of the data set specifically includes: According to the distance between different samples in the data set, a specified number of candidate cluster centers are selected from the samples; According to the distance between the remaining samples and the candidate cluster center, the samples are assigned to the candidate cluster center with the nearest distance. Each candidate cluster center and its assigned samples form a region. In each region, based on the distances of all samples, the candidate cluster center of the region is updated to the sample of all sample centers; The updated candidate cluster centers are iteratively updated until the candidate cluster centers in each region no longer change, and the candidate cluster centers that no longer change are used as the cluster centers of each region.

3. The method for generating individual bias samples in a deep neural network according to claim 2, characterized in that: The step of selecting a specified number of candidate cluster centers from the samples according to the distances between different samples in the data set specifically includes: Randomly select a sample as the initial centroid and calculate the square of the distance from the initial centroid to each sample; According to the square of the distance from the initial centroid to each sample and the total number of samples, the candidate probability of each sample is calculated, and the sample with the highest probability is taken as the next centroid; Iteratively calculate the next centroid until the number of centroids obtained reaches the specified number, and use the obtained centroids as candidate cluster centers.

4. The method for generating individual bias samples in a deep neural network according to claim 1, characterized in that: The step of determining whether a sample in the clustering result is a biased sample according to the global protection attribute of the sample specifically includes: Take each sample in the clustering result as the first sample in turn, and for each first sample, find out whether there is a second sample; When the second sample differs from the first sample only in the global protection attribute, the first sample and each second sample are regarded as a pair of bias samples.

5. The method for generating individual bias samples in a deep neural network according to claim 1, characterized in that: The globally perturbing the samples with a negative judgment result specifically includes: Obtain the local gradient of the sample according to the gradient direction of the sample, and regularize the local gradient to obtain the gradient guidance result, where each dimension in the gradient guidance result represents the global perturbation direction of the corresponding attribute; According to the gradient guidance results, the corresponding attributes in the sample are iteratively globally perturbed until a biased sample is obtained or the maximum number of iterations is reached, and the globally perturbed sample is used as the global biased sample.

6. The method for generating individual bias samples in a deep neural network according to claim 5, characterized in that: The regularization of the local gradient to obtain the gradient guidance result specifically includes: The local gradient is regularized using the following formula: Among them, L is the result of the loss function Loss, E in is the sample error without regularization term, λ is the regularization parameter, is the square of the momentum vector acceleration weight, and j is the number of dimensions of the momentum vector.

7. The method for generating individual bias samples in a deep neural network according to claim 6, characterized in that: The global perturbation direction includes descending, unchanged and ascending. The iterative global perturbation of the corresponding attributes in the sample according to the gradient guidance result specifically includes: When the global disturbance direction is downward, the value of the non-global protection attribute is reduced by 1; When the global perturbation direction is unchanged, the value of the non-global protection attribute remains unchanged; When the global disturbance direction is upward, increase the value of the non-global protection attribute by 1.

8. The method for generating individual bias samples in a deep neural network according to claim 1, characterized in that: The selecting of local protection attributes under the guidance of gradient specifically includes: Get the gradient of each non-global protection attribute in the global bias sample, normalize the inverse of the sum of the absolute values ​​of the gradients, and take the non-global protection attribute with the largest normalized result as the local protection attribute.

9. A device for generating individual bias samples in a deep neural network, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for generating individual bias samples in a deep neural network according to any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for generating individual bias samples in a deep neural network as described in any one of claims 1-8.

Images