Fairness optimization for deep neural network pruning technology and image classification methods

Abstract

The present invention discloses a fairness optimization and image classification method for deep neural network pruning technology, which relates to the field of artificial intelligence security technology. The fairness optimization method includes: obtaining a training data set; training an initial neural network model through the training data set to obtain a neural network model; pruning the neural network model; obtaining geometric metrics of samples of different sensitive attribute subgroups through the training data set and the pruned neural network model, averaging the geometric metrics of samples of different sensitive attribute subgroups, calculating extreme fairness constraints, calculating overall losses using the extreme fairness constraints and classification loss terms, fine-tuning the pruned neural network model through the overall losses, and obtaining a target pruned model. The present invention can balance inter-group performance at the feature level to address the shortcomings of existing research in that it cannot be directly deployed in resource-limited terminal devices and cannot be quickly integrated with existing pruning technologies.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and in particular to fairness optimization and image classification methods for deep neural network pruning technology. Background Art

[0002] Deep neural networks have demonstrated remarkable performance in a variety of deep learning tasks, including image classification, face recognition, object detection, and semantic segmentation. However, as the complexity of tasks increases, model architectures become increasingly large, posing significant challenges to model deployment on terminal devices. To address this issue, model compression techniques are widely used to reduce the number of model parameters and computational costs, maintaining model efficiency while ensuring prediction accuracy. As one of the primary means of model compression, pruning technology aims to optimize model structure, reduce resource consumption, and minimize the impact on model performance by identifying and removing "redundant" or "unnecessary" weights in the model.

[0003] However, the application of pruning technology may not only introduce new biases, but also exacerbate existing biases, which is manifested in the widening performance differences between different subgroups of the pruned model. For example, in the face classification task, the pruned model may enhance the recognition accuracy of a specific ethnic group (such as people with lighter skin), while reducing the recognition accuracy of another ethnic group (such as people with darker skin). This phenomenon becomes more obvious as the degree of pruning deepens. Similarly, unfair artificial intelligence systems may also lead to serious consequences in practical applications. For example, unfairness in the classification of chest X-ray images may lead to medical diagnosis errors; and unfairness in face recognition may exacerbate discrimination against specific social groups. If the original model already has bias, the pruning process may further amplify these unfair phenomena.

[0004] While evidence suggests that pruning can amplify model bias, our understanding of how pruning specifically impacts model fairness remains limited. Existing research has attempted to explain this phenomenon by analyzing differences in gradient norms and Hessian matrices between groups. However, these factors are computationally expensive and difficult to implement for models with large parameter counts, failing to provide practical guidance for mitigating bias caused by pruning. Furthermore, existing fair pruning schemes often require complex calculations and sorting of all parameters, or the introduction of additional network modules, which not only increases computational complexity and storage requirements but also contradicts the goal of pruning to simplify the model. Furthermore, existing research primarily focuses on designing specific fair pruning frameworks, which cannot be flexibly integrated into different pruning techniques.

[0005] Therefore, it is particularly important to explore a fairness optimization and image classification method for deep neural network pruning technology that can be directly deployed in resource-limited terminal devices and quickly integrated with existing pruning technologies. Summary of the Invention

[0006] In order to solve the technical problems existing in the above-mentioned prior art, the present invention proposes a fairness optimization and image classification method for deep neural network pruning technology, aiming to solve the shortcomings of existing research that cannot be directly deployed in terminal devices with limited resources and cannot be quickly integrated with existing pruning technology.

[0007] On the one hand, to achieve the above-mentioned objectives, the present invention provides a fairness optimization method for deep neural network pruning technology, comprising:

[0008] Obtaining a training data set, wherein the training data set is image data including target labels and sensitive attributes;

[0009] Training an initial neural network model using the training data set to obtain a neural network model;

[0010] Pruning the neural network model;

[0011] Through the training data set and the pruned neural network model, the geometric metrics of the samples of different sensitive attribute subgroups are obtained, and the geometric metrics of the samples of different sensitive attribute subgroups are averaged, and the extreme fairness constraint item is calculated. The overall loss function is calculated using the extreme fairness constraint item and the classification loss item, and the pruned neural network model is fine-tuned and trained using the overall loss function to obtain the target neural network pruning model.

[0012] Preferably, the neural network model is used to extract sample feature vectors of the training data set, and the neural network model includes a feature extractor and a classifier.

[0013] Preferably, pruning the neural network model includes:

[0014] Set the pruning rate;

[0015] Calculate the L1 norm of all parameters of the neural network model and sort them;

[0016] The sorted parameters are proportionally set to zero according to the pruning rate to prune the neural network model.

[0017] Preferably, during the process of pruning the neural network model, the last classification layer in the neural network model is not pruned.

[0018] Preferably, calculating the extreme fairness constraint item includes:

[0019] The training image dataset is divided into several batches. In the training of each batch, a feature vector is obtained after each sample is input into a feature extractor, and a geometric metric corresponding to each sample is calculated;

[0020] The average length and average angle of the K sensitive attribute subgroup samples in the current batch are respectively counted, and the extreme fairness constraint item is calculated according to the average length and average angle.

[0021] Preferably, the geometric measurement corresponding to the sample includes the length of the sample and the angle of the sample, the length of the sample is calculated using the L1 norm of the feature vector, and the angle of the sample is calculated using the classification weight corresponding to the feature vector and the target label of the feature vector.

[0022] Preferably, calculating the extreme fairness constraint item by using the average length and the average angle includes:

[0023] The difference between the maximum and minimum geometric metrics among the K sensitive attribute subgroups is measured to calculate the extreme fairness constraint, specifically:

[0024]

[0025]

[0026] L fair =λ length L length +λ angle L angle ;

[0027] Where, L length is the range length constraint item, is the maximum average length of the K sensitive attribute subgroups, is the minimum average length of the K sensitive attribute subgroups, L angle is the range angle constraint term, is the maximum average angle among the K sensitive attribute subgroups, is the minimum average angle among K sensitive attribute subgroups, L fair is the extreme fairness constraint term, λ length is the weight coefficient of the range length constraint term, λ angle is the weight coefficient of the range angle constraint term.

[0028] Preferably, the total loss function of the target neural network pruning model includes a classification loss term and a fairness loss term, wherein the classification loss term uses a cross entropy loss function term to calculate the classification loss of the sample, and the fairness loss term uses a geometric metric range that constrains different sensitive subgroups to calculate the fairness loss of the sample;

[0029] Among them, the total loss function is:

[0030] L=L ce +Lfair ;

[0031] Where L is the total loss function, L ce is the classification loss term, L fair It is the fair loss item.

[0032] Preferably, in the fine-tuning training stage, the improved total loss function is used for training, and the Adam algorithm is used to perform gradient updates on the remaining parameters of the model.

[0033] On the other hand, to achieve the above object, the present invention also provides an image classification method, comprising:

[0034] Get the image to be classified;

[0035] The image to be classified is input into the neural network pruning model constructed by the method to obtain a classification result.

[0036] Compared with the prior art, the present invention has the following advantages and technical effects:

[0037] (1) The present invention can be directly deployed on the model of the terminal device to improve fairness. Since the geometric metric used in the present invention is easy to calculate, it only requires the intermediate variables of conventional training, without additional calculations and without the need for additional network modules. This makes it more suitable for the actual deployment scenarios of deep neural networks. The method of the present invention takes into account the relationship between the effect of pruning and practical applications, and has high operability and practicality.

[0038] (2) The present invention can be quickly integrated into the training phase of the existing pruning algorithm, thereby directly improving the fairness effect of the pruning algorithm that only considers computational efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0040] Figure 1 This is a flow chart of a fairness optimization method for deep neural network pruning technology according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram illustrating the practical significance of geometric metrics in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] This embodiment discloses a fairness optimization method for deep neural network pruning technology, aiming to address the potential bias introduced by existing pruning technologies in applications and the shortcomings of existing research in being unable to be directly deployed in resource-limited terminal devices and quickly integrated with existing pruning technologies, thereby achieving the purpose of optimizing the fairness of the pruning model.

[0045] Fairness optimization methods for deep neural network pruning techniques, such as Figure 1 ,include:

[0046] Obtain a training dataset, where the training dataset is image data containing target labels and sensitive attributes;

[0047] Train the initial neural network model using the training data set to obtain the neural network model;

[0048] Pruning neural network models;

[0049] Through the training data set and the pruned neural network model, the geometric measurements of samples of different sensitive attribute subgroups are obtained, and the geometric measurements of samples of different sensitive attribute subgroups are averaged, and the extreme fairness constraint item is calculated. The overall loss function is calculated using the extreme fairness constraint item and the classification loss item. The pruned neural network model is fine-tuned and trained using the overall loss function to obtain the target neural network pruning model.

[0050] Specifically, first, a training image dataset including target labels and sensitive attributes is obtained; secondly, an initial neural network model is trained to extract sample feature vectors; then, the trained neural network model is pruned according to the set pruning rate; the average geometric metric of each sensitive attribute subgroup in each batch training of the pruned model is calculated, and an extreme fairness constraint term is designed; finally, the model is iteratively trained using the classification loss term and the fairness loss term, and the model gradient is updated to finally obtain a pruned model that is both fair and accurate.

[0051] Furthermore, a training image dataset is obtained. In this embodiment, the dataset includes multiple samples, each of which includes training image data, a corresponding target label, and sensitive attributes (such as gender, race, etc.).

[0052] Specifically, a high-accuracy neural network model is trained on the acquired data set. The neural network model includes a feature extractor and a classifier, which can effectively extract the sample feature vector X and achieve accurate classification, thereby more accurately calculating the geometric measurement of the sample.

[0053] Furthermore, pruning the neural network model includes:

[0054] Set the pruning rate p%. The larger the p%, the more weights are set to zero or directly removed, and the smaller the capacity of the neural network is and the more efficient it is.

[0055] Calculate the L1 norm of all parameters of the neural network model and sort them;

[0056] The sorted parameters are proportionally set to zero according to the set pruning rate to prune the neural network model.

[0057] Specifically, the trained neural network model is pruned according to a set pruning rate. The pruning method used in this embodiment can be replaced with different importance calculation methods, such as gradient importance and activation value importance, and can be replaced with different pruning granularity, such as filter pruning and channel pruning. This embodiment is applied during the training process included in its pruning process to alleviate the unfairness caused by pruning. Since the classification weight W is required for subsequent calculations, the final classification layer is not pruned by default.

[0058] Furthermore, the calculation of the extreme fairness constraint item includes:

[0059] The training image dataset (i.e., training set) is divided into several batches. In the training of each batch, the feature vector obtained after each sample is input into the feature extractor is obtained, and the geometric metric corresponding to each sample is calculated;

[0060] The average length and average angle of the K sensitive attribute subgroup samples in the current batch are respectively counted, and the extreme fairness constraint item is calculated based on the average length and average angle.

[0061] Specifically, the geometric metrics corresponding to the sample include the length of the sample and the angle of the sample:

[0062] Sample length: Use the L1 norm of the feature vector X to calculate the length ‖X‖;

[0063] The angle of the sample: The angle α is calculated by the classification weight W corresponding to the feature vector X and its target label:

[0064] Where W T is the transpose of the classification weight W, and ||W|| is the L1 norm of the classification weight W.

[0065] In this embodiment, the angle is measured using the cosine similarity between the feature vector X and the classification weight W.

[0066] For samples of K sensitive attribute subgroups (such as different races, different genders) in the current batch, the average length and average angle of the sensitive subgroups are calculated. For example, if there are n samples in the current batch, k The average length of the kth subgroup of samples is: The average angle is: Where, is the length of the i-th sample in the k-th subgroup, is the angle of the i-th sample in the k-th subgroup.

[0067] like Figure 2 , average length Represents the classification confidence of samples in the sensitive attribute subgroup. When the length is larger, the sample is farther away from the classification center, and thus it is easier to be correctly classified. The average angle Represents the intra-class compactness of samples in sensitive subgroup k. The smaller the value, the closer the sample feature is to the correct category weight.

[0068] Furthermore, since the length ||X|| and angle α corresponding to the sample are related to the loss calculated for each sample:

[0069]

[0070] Where n is the total number of samples, |Y| is the total number of categories, is the jth incorrect category y of the kth sample jk Corresponding classification weight W yjk The L1 norm of is the kth sample feature vector X k and the jth incorrect category y jk The corresponding classification weight W yjk The angle between is the kth sample feature vector X k With the correct category y k The corresponding classification weight W k The angle between The correct category y for the kth sample k Corresponding classification weight W k The L1 norm of .

[0071] The size of the loss is related to the accuracy of sample classification, so it can be used as a variable to control the differences between different sensitive subgroups to design a fairness constraint;

[0072] The fairness of the model is achieved by measuring the difference between the maximum and minimum geometric metrics among K sensitive attribute subgroups and calculating the extreme fairness constraint term. The formula is as follows:

[0073]

[0074]

[0075] L fair =λ length L length +λ angle L angle ;

[0076] Where, L length is the range length constraint item, is the maximum average length of the K sensitive attribute subgroups, is the minimum average length of the K sensitive attribute subgroups, L angle is the range angle constraint term, is the maximum average angle among the K sensitive attribute subgroups, is the minimum average angle among K sensitive attribute subgroups, L fair is the extreme fairness constraint term, λ length is the weight coefficient of the range length constraint term, λ angle is the weight coefficient of the range angle constraint term.

[0077] In this embodiment, λ length =0.01,λ angle =0.01.

[0078] Furthermore, the total loss function of the target neural network pruning model includes a classification loss term and a fairness loss term, wherein the classification loss term uses a cross entropy loss function term to calculate the classification loss of the sample, and the fairness loss term uses a geometric metric range that constrains different sensitive subgroups to calculate the fairness loss of the sample;

[0079] Among them, the total loss function is:

[0080] L=L ce +L fair ;

[0081] Where L is the total loss function, L ce is the classification loss term, L fair is the fair loss item;

[0082]

[0083] Where, f θ (x k ) is the predicted output corresponding to the k-th sample input of the neural network f with model parameters θ, y k is the correct category corresponding to the kth sample, x kis the kth sample, P(f θ (x k )=y k |x k ) is the probability that the neural network f with model parameters θ predicts the kth input sample as the correct category.

[0084] Furthermore, in the fine-tuning training phase after pruning, an improved total loss function is used for training, and the Adam algorithm is used to perform gradient updates on the remaining model parameters, so that the model can constrain the geometric metrics between different sensitive groups, thereby constraining the performance differences between each subgroup, and ultimately improving the fairness of the pruned model while ensuring accuracy.

[0085] This embodiment can be quickly integrated into the training phase of an existing pruning algorithm, thereby directly improving the fairness of a pruning algorithm that only considers computational efficiency and accuracy. As shown in Table 1, by applying this embodiment to the training phase of different pruning algorithms, the fairness effect of ensuring accuracy and improving accuracy is achieved.

[0086] This embodiment can achieve good results on different tasks. As shown in Table 2, it can maintain classification accuracy and improve fairness at high pruning rates in different models and datasets.

[0087] The fairness metric in Tables 1 and 2 uses the standard deviation, that is, the standard deviation of the accuracy of each sensitive attribute subgroup as a measure of the difference between statistical subgroups. The smaller the value, the more fair it is.

[0088] Table 1

[0089] Pruning method Classification accuracy Standard deviation SNIP 79.881 15.893 SNIP+ours 78.110 15.045 Lottery 77.565 14.056 Lottery+ours 77.429 13.404 Grasp 77.020 15.484 Grasp+ours 77.202 13.056 FPGM (pruning 70%) 87.420 8.083 FPGM (pruning 70%) + ours 88.101 7.962 FPGM (pruning 90%) 82.732 9.672 FPGM (pruning 90%) + ours 81.880 7.881

[0090] Table 2

[0091]

[0092]

[0093] The optimization method of this embodiment is suitable for deployment on terminal devices and can be quickly integrated into existing pruning technologies, effectively addressing fairness risks that may arise when applying pruning techniques. This approach can reduce or eliminate potential bias introduced by pruning while ensuring model efficiency and lightweightness, thereby improving the overall fairness and reliability of the model.

[0094] This embodiment also provides an image classification method, including:

[0095] Get the image to be classified;

[0096] The image to be classified is input into the target neural network pruning model constructed by the above method to obtain the classification result.

[0097] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A fairness optimization method for deep neural network pruning technology, characterized by: include: Obtaining a training data set, wherein the training data set is image data including target labels and sensitive attributes; Training an initial neural network model using the training data set to obtain a neural network model; Pruning the neural network model; Using the training data set and the pruned neural network model, geometric metrics of samples of different sensitive attribute subgroups are obtained, and the geometric metrics of the samples of different sensitive attribute subgroups are averaged to calculate a range fairness constraint term. The overall loss function is calculated using the range fairness constraint term and the classification loss term. The pruned neural network model is fine-tuned using the overall loss function to obtain a target neural network pruned model. Calculating the extreme fairness constraint item includes: The training image dataset is divided into several batches. In the training of each batch, a feature vector is obtained after each sample is input into a feature extractor, and a geometric metric corresponding to each sample is calculated; Count the current batch separately The average length and average angle of the sensitive attribute subgroup samples, and the extreme fairness constraint item is calculated by the average length and average angle; The extreme fairness constraint item is calculated using the average length and the average angle, including: The difference between the maximum and minimum geometric metrics among the K sensitive attribute subgroups is measured to calculate the extreme fairness constraint, specifically: ； ； ； Where, is the range length constraint item, is the maximum average length of the K sensitive attribute subgroups, is the minimum average length of the K sensitive attribute subgroups, is the range angle constraint term, is the maximum average angle among the K sensitive attribute subgroups, is the minimum average angle among the K sensitive attribute subgroups, is the extreme fairness constraint, is the weight coefficient of the range length constraint term, is the weight coefficient of the range angle constraint term.

2. The fairness optimization method for deep neural network pruning technology according to claim 1 is characterized in that: The neural network model is used to extract sample feature vectors of the training data set, and the neural network model includes a feature extractor and a classifier.

3. The fairness optimization method for deep neural network pruning technology according to claim 1 or 2, characterized in that: Pruning the neural network model includes: Set the pruning rate; Calculate the L1 norm of all parameters of the neural network model and sort them; The sorted parameters are proportionally set to zero according to the pruning rate to prune the neural network model.

4. The fairness optimization method for deep neural network pruning technology according to claim 3 is characterized in that: During the process of pruning the neural network model, the last classification layer in the neural network model is not pruned.

5. The fairness optimization method for deep neural network pruning technology according to claim 1 is characterized in that: The geometric metric corresponding to the sample includes the length of the sample and the angle of the sample. The length of the sample is calculated using the L1 norm of the feature vector, and the angle of the sample is calculated using the classification weight corresponding to the feature vector and the target label of the feature vector.

6. The fairness optimization method for deep neural network pruning technology according to claim 1, characterized in that: The total loss function of the target neural network pruning model includes a classification loss term and a fairness loss term, wherein the classification loss term uses a cross entropy loss function term to calculate the classification loss of the sample, and the fairness loss term uses a geometric metric range that constrains different sensitive subgroups to calculate the fairness loss of the sample; Among them, the total loss function is: ； Where, is the total loss function, is the classification loss term, It is the fair loss item.

7. The fairness optimization method for deep neural network pruning technology according to claim 1, characterized in that: In the fine-tuning training stage, the improved total loss function is used for training, and the The algorithm performs gradient updates on the remaining parameters of the model.

8. An image classification method, characterized in that: include: Get the image to be classified; The image to be classified is input into the target neural network pruning model constructed by the method according to any one of claims 1 to 7 to obtain a classification result.

Images