Multi-dimensional fairness automatic analysis method and system for large image model
By generating image samples with attribute changes and calculating multidimensional indicators, the standardization and automation issues in the fairness assessment of large image models are solved, realizing multidimensional quantitative and visual fairness analysis, and improving the automation level and interpretability of the assessment results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-07
- Publication Date
- 2026-03-24
AI Technical Summary
Existing large image models lack standardized sensitive attribute intervention mechanisms in fairness assessment. The fairness assessment index system is singular and cannot fully reflect the multidimensional performance of the model output. Model bias is difficult to explain, and there is a lack of visualization and traceability of result bias. The fairness assessment process relies heavily on manual grouping and subjective judgment, has a low degree of automation, and is difficult to conduct black-box model assessment.
By extracting and classifying the sensitive attribute labels of the input images, image samples with attribute changes are generated. The output change index and fairness index are calculated. The multidimensional index system and GradCAM difference map are used to locate model bias, and an automated and interpretable fairness analysis system is constructed.
It achieves structured and automated image fairness assessment, reduces the cost of manual intervention, provides multi-dimensional quantitative assessment, supports black-box model assessment, locates specific semantic regions of model bias, and improves assessment efficiency and repeatability.
Smart Images
Figure CN121725318A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of artificial intelligence explainability and model ethics, and particularly relates to a multi-dimensional fairness automatic analysis method and system for an image large model. BACKGROUND
[0002] With the rapid development of artificial intelligence technology, image large models (VFMs) based on deep neural networks have been widely applied in medical diagnosis, public security, intelligent recommendation, social media, judicial assistance and other key scenarios. The new generation of visual models represented by CLIP (Contrastive Language-Image Pretraining), DINOv2 and BLIP2 not only have powerful multi-modal expression ability, but also have surpassed traditional methods in image classification, target detection, image-text matching and image generation.
[0003] Such models generally use large-scale Internet data for pre-training, have hundreds of millions or even billions of image-text pairs, and have the ability to automatically learn general semantic structures from raw data. However, the "scale" of large models also hides serious bias and fairness problems. On the one hand, social biases such as gender discrimination, racial stereotypes and skin color bias hidden in the training data will be passed to the model parameters in an implicit form; on the other hand, the actual performance of large models after deployment has the risk of performance degradation and misjudgment for certain groups, and may even cause social ethics controversy and legal liability problems.
[0004] Existing research has shown that in tasks such as gender recognition, face recognition and recruitment screening, the recognition accuracy of models for women, people of color and older people is significantly lower than that for mainstream groups. For example, the famous "Gender Shades" study pointed out that the recognition accuracy of commercial face recognition systems of IBM and Microsoft for white male is as high as 99% or more, while the recognition rate for African American women is less than 65%. Similarly, in image-text matching systems (such as CLIP), there is a phenomenon that black women are more frequently matched with negative semantic words, showing systematic semantic unfairness.
[0005] In this context, a number of international research institutions and technical organizations have successively issued AI model auditing and fairness evaluation guidelines (such as IEEE P7003, EU AI Act, etc.), requiring pre-deployment compliance evaluation of large models facing the public. Meta AI, OpenAI, Google DeepMind, etc. also launched internal fairness test frameworks (such as FACET, FairFace, Project Respect), trying to quantify the performance difference of the model under different sensitive attributes to identify potential bias risks.
[0006] However, the current tools and methods for fairness evaluation of visual large models still have the following outstanding problems: There is a lack of standardized sensitive attribute intervention mechanism, it is difficult to build a controlled sample control group; the fairness evaluation index system is single, it is difficult to fully reflect the multi-dimensional performance of the model output; the model bias is difficult to explain, there is a lack of visualization and traceability of the result bias; the fairness evaluation process is highly dependent on manual grouping and subjective judgment, and the degree of automation is low; the black box model evaluation is difficult to carry out, and there is a lack of adaptive evaluation strategy. SUMMARY
[0007] The technical problem to be solved by the present application is to provide a multi-dimensional fairness automatic analysis method and system for image large models to provide a structured and automated image fairness evaluation process and reduce the cost of manual intervention.
[0008] To solve the above technical problems, the technical scheme adopted by the present application is: A multi-dimensional fairness automatic analysis method for image large models, comprising the steps of: S1, extracting the sensitive attribute label of the input image sample, and classifying the input image sample according to the sensitive attribute; S2, performing attribute change processing on the input image sample based on the sensitive attribute type of the input image sample to obtain an attribute changed image sample, and taking the input image sample and the attribute changed image sample as an image pair; S3, extracting the output representation of the image pair after passing through the image large model, and calculating the output change index of the input image sample before and after the attribute change processing according to the output representation; S4, judging whether the output change index of the input image sample before and after the attribute change processing is higher than a first preset threshold value, if higher, further calculating the fairness index quantitative score of the input image sample and the attribute changed image sample, and when the fairness index quantitative score is higher than a second preset threshold value, comparing the difference map obtained by comparing the image activation maps of the input image sample and the attribute changed image sample, and locating the specific semantic area where the model bias occurs according to the difference map.
[0009] Further, the sensitive attribute label comprises one or more of gender, race, and age, expressed as:
[0010] gender ∈ {male, female} race ∈ {white, black, Asian, Indian, Hispanic, Middle Eastern, other} age ∈ {0-20, 21-40, 41-60, >60} In the above formula, gender is gender, race is race, and age is age.
[0011] Further, in step S2, when the attribute changing processing is performed on the input image sample based on the type of sensitive attribute of the input image sample, if the type of sensitive attribute of the input image sample is gender, the steps of the attribute changing processing are as follows: generate the input image sample the encoding after gender inversion , expressed as:
[0012] In the above formula, is the latent encoding vector of the input image sample, is the gender direction vector, is the intensity control coefficient; According to the encoding generate the attribute changed image sample , expressed as:
[0013] In the above formula, denotes the decoder of the pre-trained image generation model, used to reconstruct the encoding vector in the latent space into an image sample; If the type of sensitive attribute of the input image sample is race, the steps of the attribute changing processing are as follows: generate the latent encoding of the input image sample after race conversion , expressed as:
[0014] In the above formula, is the race direction vector; According to the encoding generate the attribute changed image sample , expressed as: ; If the sensitive attribute type of the input image sample is age, the attribute changing process is as follows: The latent encoding of the input image sample after race conversion is generated , the expression is:
[0015] In the above formula, is the age direction vector; According to the encoding The attribute changed image sample is generated , the expression is: .
[0016] Further, the output change indicators include cosine similarity change and category prediction confidence change, wherein the expression for calculating the cosine similarity change of the input image sample and the attribute changed image sample is:
[0017] In the above formula, and are the deep feature representations of the input image sample and the attribute changed image sample extracted after passing through the image large model to be analyzed, respectively; The expression for category prediction confidence change is:
[0018] In the above formula, represents the confidence probability that the input image sample is predicted by the image large model to be its real category, represents the confidence probability that the attribute changed image sample is predicted by the same model to be the same real category.
[0019] Further, the fairness indicator quantification score includes statistical performance bias, representation bias index, and embedding drift indicator, wherein the expression for statistical performance bias is:
[0020] In the above formula, represents that the model makes a positive example prediction, respectively represent two different sensitive attribute values; The expression for bias index is:
[0021] In the above formula, are the embedding means of the i, j attribute groups, respectively, and K is the total number of sensitive attribute groups; The expression for embedding drift indicator is:
[0022] In the above formula, and are output representations of input image samples and attribute-altered image samples, respectively, represents a distance measurement function for measuring the difference between two output representations.
[0023] Further, the embedding mean refers to an average feature vector obtained by performing arithmetic averaging on output feature vectors obtained by inputting all images of the same sensitive attribute group into the model, and the specific calculation steps are as follows: Group all image samples according to sensitive attributes, input each sample in the group into the image model to be analyzed, and extract the corresponding output representation; Arithmetic average all output representations to calculate the embedding mean vector of the group The calculation formula is:
[0024] In the above formula, is a vector with the same dimension as the feature embedding dimension of the model output, is the number of samples in the i-th attribute group, is the output feature vector of the k-th sample, and Σ represents the sum of the feature vectors of all samples.
[0025] Further, in step S4, the difference map is obtained by comparing the image activation maps of the input image sample and the attribute-altered image sample, specifically including: For input image sample and attribute-altered image sample , extract the corresponding gradient weight α and feature map F at the last convolution layer; According to the gradient weight α and the feature map F, calculate the image activation map of the input image sample and the attribute-altered image sample , the expression is:
[0026] In the above formula, is the gradient weight of the k-th feature map to class c, is the feature map of the k-th channel of the last convolution layer; Convert the image activation maps of the input image sample and the attribute-altered image sample into heat maps and Calculate the difference between the two heat maps to obtain the difference map as the attention region drift measure, and the calculation formula is:
[0027] In the above formula, This indicates that the heatmap has been normalized. This is an absolute value operation on a pixel-by-pixel basis.
[0028] Further, in step S4, locating the specific semantic region generated by the model deviation based on the difference map specifically includes: Thresholding is applied to the region of interest drift measure in the difference map to extract regions of significant change. The expression is as follows:
[0029] In the above formula, The preset significance threshold, For indicator functions, For the generated binarization mask; Binarization mask Connectivity component analysis is performed to identify spatially continuous and independent connected regions, where each connected region represents a potential semantic block whose model attention has changed significantly due to changes in sensitive attributes; Each identified connected region is mapped back to the corresponding pixel region of the original input image sample, and the average drift metric within each connected region is calculated and used as a quantitative indicator of the degree of deviation in that region. Finally, the output contains all connected components. This provides a list of corresponding bias intensities, thereby enabling the localization and attribution of specific semantic regions of model bias.
[0030] A computer-readable storage medium storing a computer program / instructions programmed or configured to execute, via a processor, steps of a method for automated multidimensional fairness analysis of large image models.
[0031] A computer program product includes a computer program / instructions programmed or configured to execute steps of a multidimensional fairness automated analysis method for large image models via a processor.
[0032] Compared with the prior art, the advantages of the present invention are as follows: This invention enables fine-grained grouping management of model inputs by extracting and classifying sensitive attributes of input images, providing fundamental data support for subsequent identification of potential model biases. Image attribute change processing based on sensitive attributes generates comparative samples without altering the semantics of the image subject, effectively simulating the impact of attribute changes on model output and providing controllable samples for subsequent bias detection and fairness analysis. By comparing the output representations of image pairs and calculating change indices, the sensitivity of the model to attribute changes can be quantitatively measured, providing a quantitative basis for bias intensity and improving the objectivity and repeatability of bias detection. When the output change index exceeds a threshold, fairness quantification scoring and activation map difference analysis are introduced to effectively identify the source of bias. Visualizing the difference map allows for locating the model's bias response to specific semantic regions, making the bias not only detectable and quantifiable but also interpretable, providing a clear direction for model optimization and fairness improvement. Attached Figure Description
[0033] Figure 1 This is a flowchart of the automatic multidimensional fairness analysis method for large image models according to an embodiment of the present invention. Detailed Implementation
[0034] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0035] The following explains some of the abbreviations used in this invention: CLIP: Contrastive Language–Image Pretraining, a pre-trained model for image-text comparison. SPD: Statistical Parity Difference; RBI: Representation Bias Index, indicating a deviation index; GradCAM: Gradient-weighted Class Activation Mapping, a visual interpretation method used to extract regions of interest from a model; FairFace: A publicly available dataset of face images labeled with various sensitive attributes; StyleGAN: An image generation model based on generative adversarial networks.
[0036] Currently, the tools and methods used for fairness assessment of large visual models have the following prominent problems: 1. The lack of standardized intervention mechanisms for sensitive attributes makes it difficult to construct controlled sample control groups. Current fairness analyses of image models largely rely on comparisons of data from different attribute groups within existing public datasets (such as CelebA, UTKFace, and FairFace). However, these methods depend on "naturally distributed" group samples, which exhibit significant differences across non-sensitive attribute dimensions such as age, gender, background, emotion, and lighting, making it difficult to exclude the interference of these non-sensitive variables. Therefore, the analysis results cannot accurately pinpoint whether the model is truly sensitive to a particular attribute or whether the prediction fluctuations are due to other confounding factors.
[0037] For example, when comparing the model output of "Asian young women" with "Caucasian elderly men," it is impossible to determine whether the bias comes from "gender" or "age + race + background." A truly reliable method for assessing fairness should construct "attribute control image pairs" (i.e., changing only one sensitive attribute while keeping all other conditions completely unchanged), thereby explaining the model's behavior in the form of an "intervention experiment."
[0038] Currently, only a few studies use generative models (such as StyleGAN) or semantic transfer methods to synthesize controlled image pairs. These methods have not yet been standardized or engineered, resulting in inconsistent evaluation methods and unreproducible research results.
[0039] 2. The fairness assessment indicator system is too simplistic and fails to comprehensively reflect the multidimensional performance of the model output. Most existing assessment methods simplify "fairness" to whether "accuracy is equal across different groups" or whether there are "differences in false positive rates." These methods suffer from two core problems: Question 1: The output of a model is not necessarily a classification result. For example, basic models such as CLIP, DINO, and SAM output image embedding vectors or feature maps, which cannot be evaluated using accuracy or F1-score.
[0040] Question 2: Even classification models may produce subtle differences in the confidence distribution, embedding spatial location, and image-text alignment quality of certain groups. These "substructural biases" are not reflected in the final predicted category and cannot be captured by traditional metrics.
[0041] Furthermore, the focus of fairness varies across different tasks: face recognition focuses on false acceptance rate, image and text retrieval focuses on alignment representation distance, and recommendation systems focus on click prediction bias. Therefore, fairness assessment urgently needs to develop a generalizable, interpretable, and composable indicator system, such as statistical bias (SPD), representation bias (RBI), embedding drift, and visual region variation (GradCAM difference).
[0042] While some academics have attempted to propose these metrics, no consistent standards have been established, and industry practice lacks engineering-ready implementation frameworks and computational tools.
[0043] 3. Model bias is difficult to interpret, and there is a lack of visual traceability to identify the source of biases in the results. After identifying biases in the model, decision-makers need to answer two key questions: "Why did the bias occur?" and "In which part of the model did the bias occur?" However, most current analytical methods only stay at the "outcome level," that is, whether there are significant performance differences across different groups, but fail to delve into the internal mechanisms that cause the bias.
[0044] Taking facial recognition models as an example, if the accuracy rate is significantly lower for Black women than for other groups, developers need to figure out whether the bias is due to failed feature extraction, misfocusing of attention regions, missing training set samples, or abnormal responses from other intermediate layers. However, existing analysis tools are almost incapable of answering these questions.
[0045] Traditional visualization methods such as Grad-CAM, SHAP, and LIME, while widely used in interpretability analysis, have not been systematically applied to fairness assessment, and there is a lack of mature frameworks that combine "attribution change" and "attribute intervention." This makes it impossible to pinpoint remedial paths even when model biases are detected, and model improvement lacks effective guidance.
[0046] 4. The fairness assessment process relies heavily on manual grouping and subjective judgment, with low automation. Many evaluation efforts still rely on manually constructing test sets, manually labeling sensitive attributes, and manually grouping and comparing data, lacking a repeatable and scalable evaluation process. This is especially true in industrial-grade models (such as CLIP), where the number of input samples is enormous and the output types are diverse (image embeddings, contrast scores, open-ended output text), making manual intervention insufficient for the evaluation needs.
[0047] Furthermore, manually segmenting groups suffers from issues such as inconsistent classification standards, imbalanced group samples, and incorrect labeling. Different evaluators use different standards to segment data, resulting in vastly different evaluation results, making it difficult to use as a basis for pre-deployment compliance analysis.
[0048] In actual deployment processes, enterprises want to incorporate fairness testing into the CI / CD (Continuous Integration / Continuous Deployment) process, automatically performing sensitivity regression tests on new model versions. However, current toolchains offer extremely weak support for fairness analysis, lacking programmable and parameterized evaluation modules.
[0049] 5. Black-box model evaluation is difficult to conduct and lacks adaptable evaluation strategies. In many real-world scenarios, models provide services as black boxes, such as CLIP models encapsulated as APIs, ONNX models on mobile devices, and SaaS services provided by enterprises. These models often do not expose intermediate layer representations, gradient information, attention maps, or other structures, only providing the ability to map inputs to outputs.
[0050] Existing fairness assessment methods rely heavily on intermediate layer representations for analysis. For example, methods such as using embedding space for bias measurement and attention heatmap-based attribution cannot be used on black-box models.
[0051] Therefore, the fairness issue in current visual AI systems is not only a technical challenge, but also a real need in terms of law, ethics, and product safety. Developing an automated, explainable fairness analysis method with causal intervention capabilities has become a key prerequisite for the reliable deployment of large-scale models.
[0052] To address the potential bias issues that may arise when large image models process sensitive attribute data, this invention proposes a multi-dimensional, interpretable, and quantifiable indicator system for automated multi-dimensional fairness analysis of large image models. This method encompasses the entire process from input intervention and output evaluation to visual interpretation, and is applicable to scenarios such as image recognition, image-text understanding, and search recommendation deployed to the public. It is particularly suitable for fundamental visual models such as CLIP, DINOv2, BLIP2, and SAM. This method can help model developers, product deployers, and regulatory agencies identify and correct potential biases, providing tools and methodological support for building compliant, reliable, and fair visual AI systems. Each step of this method can be deployed independently and embedded into a model evaluation system for modular evaluation.
[0053] like Figure 1 As shown, the automatic multidimensional fairness analysis method for large image models in this embodiment includes the following steps: S1, Sensitive Attribute Identification and Sample Preprocessing: Extract sensitive attribute labels from input image samples and classify the input image samples according to their sensitive attributes; S2, Attribute-Controlled Image Pair Generation (Intervention Design): Based on the sensitive attribute type of the input image sample, the input image sample is subjected to attribute modification processing to obtain attribute-modified image samples. The input image sample and the attribute-modified image sample are used as image pairs. S3, Model Output Consistency Analysis: Extract the output representation of the image pair after passing through the large image model, and calculate the output change index before and after the attribute change processing of the input image sample based on the output representation; S4, Multi-index fairness quantification system, model bias attribution and interpretability analysis: Determine whether the output change index of the input image sample before and after attribute change processing is higher than the first preset threshold. If it is higher, further calculate the fairness index quantification score of the input image sample and the attribute-changed image sample. When the fairness index quantification score is higher than the second preset threshold, compare the image activation maps of the input image sample and the attribute-changed image sample to obtain a difference map. Based on the difference map, locate the specific semantic region where the model bias is generated.
[0054] In this embodiment, the sensitive attribute tag Including one or more of gender, race, and age, the expression is:
[0055] gender ∈ {male, female} race ∈ {white, black, Asian, Indian, Latino / Hispanic, Middle Eastern, other} age ∈ {0–20, 21–40, 41–60,>60} In the above formula, gender is the sex, race is the race, and age is the age.
[0056] Specifically, the received raw image dataset is denoted as: ; For each image Sensitive attribute labels are automatically extracted using a pre-trained face attribute recognition network (such as FairFace). A multidimensional attribute matrix A is constructed as the basis for image grouping.
[0057] For example, if an image is identified as {female, Asian, 25}, it can be grouped into the "Asian young women" group, providing a basis for generating a control group later.
[0058] The core challenge of fairness analysis lies in generating image pairs from the input that change only sensitive attributes while maintaining other semantic consistency. Traditional methods compare images from different sources, leading to inaccurate bias analysis due to excessive interfering variables. Therefore, this embodiment employs attribute-controlled image generation technology, utilizing StyleGAN or InterFaceGAN-like models to control variations along directions such as "gender" and "skin color" in the latent space.
[0059] In this embodiment, in step S2, when performing attribute modification processing on the input image sample based on the sensitive attribute type of the input image sample, if the sensitive attribute type of the input image sample is gender, the attribute modification processing steps are as follows: Generate input image samples Gender-reversed encoding The expression is:
[0060] In the above formula, The latent encoding vector of the input image sample. The direction vector is the gender. This is the strength control coefficient; According to the code Generate image samples with changed attributes The expression is:
[0061] In the above formula, This represents the decoder of a pre-trained image generation model, used to reconstruct image samples from encoded vectors in the latent space; Similarly, when the sensitive attribute type of the input image sample is race or age, the attribute change processing can be analogous to the gender processing logic. By shifting along a specific attribute direction in its corresponding latent coding space, the image sample with the changed attribute is generated. If the sensitive attribute type of the input image sample is race, the attribute change processing steps are as follows: Generate latent encoding after race conversion of input image samples The expression is:
[0062] In the above formula, The race direction vector; According to the code Generate image samples with changed attributes The expression is: ; If the sensitive attribute type of the input image sample is age, the attribute change processing steps are as follows: Generate latent encoding after race conversion of input image samples The expression is:
[0063] In the above formula, The age direction vector; According to the code Generate image samples with changed attributes The expression is: .
[0064] It is understandable that by generating a modified version of each sample with different attributes, a precise comparison image pair can be achieved, where only sensitive attributes differ. This can eliminate interfering factors and improve the credibility of fairness indicators.
[0065] In this embodiment, after generating the image pairs, the comparison images are input into the large model to be analyzed (such as CLIP or BLIP), and its output representation is extracted. For each image pair: , For the backbone network output of the model, such as the last layer's CLS token, image-text alignment embedding, or softmax probability, calculate the following two output change metrics: The cosine similarity change and category prediction confidence change (if classification is involved) are calculated using the following expression:
[0066] In the above formula, and These are the deep feature representations extracted from the input image samples and attribute-modified image samples after passing through a large image model (such as a classification, recognition, or generation model). They are typically the output vectors of the last hidden layer or feature layer of the model; cosine similarity variation. The result is directly used as the quantitative result of the embedding drift index to evaluate the sensitivity of the model's internal representation to changes in sensitive attributes; when the value exceeds a preset threshold, subsequent image activation map comparison analysis is triggered. The expression for the change in confidence level for category prediction is:
[0067] In the above formula, This represents the confidence probability that an input image sample is predicted to be its true class by a large image model. This represents the confidence probability that an image sample with changed attributes is predicted as the same true class by the same model. It quantifies the model's classification certainty for the same semantic content, and the changes that occur when only sensitive attributes are changed. The larger the value, the more significantly the model's prediction results are affected by sensitive attributes, revealing fairness bias at the decision-making level.
[0068] It is understandable that by statistically analyzing the average differences of such indicators among different groups, the sensitivity of a model to specific attributes can be determined.
[0069] In this embodiment, to comprehensively measure the imbalance of model output across the sensitive attribute dimensions, a fairness index is designed. The fairness index quantifies scores by including Statistical Parity Difference (SPD), Representation Bias Index (RBI), and Embedding Drift (ED). The expression for Statistical Parity Difference is as follows:
[0070] In the above formula, This indicates that the model made a positive prediction (such as successful identification or being classified into a specific category). These represent two different sensitive attribute values (for example, a=0 represents "male" and a=1 represents "female"). Statistical performance bias is used to measure the difference in the probability of a model outputting a positive class across groups, and is applicable to situations where the distribution of positive examples is uneven in classification tasks; The expression for the deviation index is:
[0071] In the above formula, are the embedding mean values of the i-th and j-th attribute groups, respectively, and K is the total number of sensitive attribute groups in them; The deviation index represents the center offset of different attribute groups in the representation space; The expression for the embedded drift metric is:
[0072] In the above formula, and The output representations are the input image sample and the attribute-changed image sample, respectively. This represents a distance metric function used to measure the difference between two output representations; Embedding drift is used to evaluate the representational perturbation of a model after attribute changes. The larger the value, the more sensitive the model is to the sensitive attributes, indicating potential bias.
[0073] In this embodiment, the embedding mean refers to the average feature vector obtained by arithmetically averaging the output feature vectors of all images from the same sensitive attribute group (e.g., the "male" group) input into the model. It represents the central position of the group in the model feature space and serves as a benchmark for measuring the overall differences in feature distribution among different groups. The specific calculation steps are as follows: Grouping and Extraction: All image samples are grouped according to sensitive attributes (such as gender, race, etc.). Each sample within a group is input into the large image model to be analyzed, and the corresponding output representation is extracted. For example, suppose we want to calculate the embedding mean of the i-th attribute group (e.g., the "male" group), which has a total of [number missing] samples. If there are 10 samples, then each sample in the group will be... Input a large model of the image to be analyzed, and extract its output representation (i.e., feature embedding vectors) to obtain a set of vectors: ; Calculate the mean: Perform an arithmetic mean over all output representations to calculate the embedding mean vector for that group. The calculation formula is:
[0074] In the above formula, It is a vector with the same dimension as the feature embedding dimension of the model output. It is the number of samples in the i-th attribute group. Σ is the output feature vector of the k-th sample, and Σ represents the summation of the feature vectors of all samples.
[0075] It is understandable that the three fairness indicators, from the three dimensions of model decision-making, group representation, and individual semantics, together constitute a multi-dimensional fairness quantification system, used to comprehensively evaluate the imbalance of the model on sensitive attributes: Statistical performance bias is used to measure the difference in model decision results between different sensitive attribute groups; The deviation index is used to measure the degree of separation between the group feature centers of different sensitive attribute groups in the model representation space; Embedding drift metrics are used to measure the semantic changes in the model representation space of individual samples caused by changes in sensitive attributes.
[0076] The fairness index quantification system, together with the output change index in step S3, constitutes a complete evaluation chain: the embedding drift index and cosine similarity change work together to quantify the changes in the representation layer from the perspectives of distance measurement and similarity, respectively; the statistical performance bias and category prediction confidence change work together to quantify the changes in the decision layer from the perspectives of group statistics and individual differences, respectively. In subsequent steps, the evaluation results of the fairness index quantification score serve as a direct basis for judging whether the model has potential fairness risks, and provide quantitative guidance for locating specific semantic regions of model bias.
[0077] In this embodiment, step S4, comparing the image activation maps of the input image sample and the attribute-changed image sample to obtain a difference map, specifically includes: For input image samples Image samples with attribute changes In the last convolutional layer, the corresponding gradient weights α and feature maps F are extracted; Calculate the input image samples based on the gradient weight α and the feature map F. Image samples with attribute changes The image activation map is expressed as:
[0078] In the above formula, The gradient weights of the k-th feature map with respect to class c. This is the feature map of the k-th channel of the last convolutional layer; Input image sample Image samples with attribute changes Image activation map converted into heat map and Calculate the difference between the two heatmaps. The difference map is obtained as a measure of the drift of the region of interest, and the calculation formula is as follows:
[0079] In the above formula, This indicates that the heatmap has been normalized (e.g., scaled to the [0, 1] range). This is a pixel-by-pixel absolute value operation. The difference map highlights semantic locations where changes in sensitive attributes cause shifts in the model's region of interest; these locations represent the specific areas where model bias may occur.
[0080] In this embodiment, step S4, locating the specific semantic region generated by the model deviation based on the difference map, specifically includes: Thresholding is applied to the region of interest drift measure in the difference map to extract regions of significant change. The expression is as follows:
[0081] In the above formula, The preset significance threshold (which can be determined based on...) The statistical distribution is determined by taking the mean plus a certain number of standard deviations. For indicator functions, For the generated binarization mask; Binarization mask Connectivity component analysis is performed to identify spatially continuous and independent connected regions. Each connected region This represents a semantic block where the model's attention has changed significantly due to a potential change in sensitive attributes; Each identified connected region Map back to the corresponding pixel regions of the original input image samples, and compute each connected region. Average drift metric within And it serves as a quantitative indicator of the degree of deviation in that region; Finally, the output contains all connected components. and the corresponding deviation strength This list allows for the location and attribution of specific semantic regions of model bias.
[0082] It is understandable that, in order to locate the specific semantic regions caused by model bias, the Grad-CAM (Gradient-weighted Class Activation Mapping) mechanism is introduced to compare the image activation maps before and after changes in sensitive attributes. If a change in a certain attribute causes a significant attention shift (such as a change in skin color causing attention to focus on the facial edges), it can be considered that the model's focus is greatly affected by the attribute, which is extremely important for developers to analyze the model's decision-making logic.
[0083] Furthermore, the above methods can be integrated into an end-to-end fairness analysis system, mainly including: a data input and attribute annotation module, an attribute control image generation module, a model output extraction module, a multi-dimensional index calculation module, and a bias visualization and report generation module. All modules have interface-based and automated operation capabilities, facilitating deployment into the testing process before AI models go live.
[0084] If it is not possible to generate control image pairs, publicly available balanced datasets (such as FairFace) can be used for grouped comparisons; if the target model is a black-box model, only the output class probability can be used for analysis; the visual attribution module can also use LIME or Integrated Gradients instead of GradCAM.
[0085] Compared with the prior art, the present invention has the following beneficial effects: Provides a structured and automated evaluation process to reduce the cost of manual intervention; A multi-indicator system is introduced to support fairness analysis of multi-task models; It supports attribute-level controllable analysis and has causal comparison capabilities; It has good scalability and is suitable for tasks such as face recognition, image and text retrieval, and image question answering; It can provide fairness audit support for the deployment of large models and meet AI compliance requirements.
[0086] To address the difficulty in constructing control samples with changes in sensitive attributes, this invention proposes a style transfer-based sensitive attribute control image generation method to construct sensitive attribute intervention sample pairs with controllable input and semantic consistency. To address the difficulty in quantifying bias in model output, a multi-index fairness measurement system for cross-group output differences is designed, providing multi-dimensional quantification methods such as statistical bias, representation bias, and semantic drift. To address the inability to pinpoint the causes of bias, GradCAM visual attribution is introduced to perform difference analysis on the model's areas of interest, supporting bias interpretation and attribution localization, which aids in model optimization. Finally, to address the lack of a unified evaluation framework, a modular fairness evaluation system is constructed, integrating data preprocessing, model evaluation, and result visualization, providing a one-stop solution to improve evaluation efficiency and repeatability.
[0087] The present invention further provides a computer-readable storage medium storing a computer program / instructions programmed or configured to execute, via a processor, the steps of a multidimensional fairness automatic analysis method for large image models.
[0088] The present invention further provides a computer program product, including a computer program / instructions programmed or configured to execute, via a processor, steps of a method for automated multidimensional fairness analysis of large image models.
[0089] The system and medium of the present invention, corresponding to the methods described above, also have the advantages described above.
[0090] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.
[0091] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. An automatic multidimensional fairness analysis method for large image models, characterized in that, Including the following steps: S1, extract the sensitive attribute labels of the input image samples and classify the input image samples according to the sensitive attributes; S2, perform attribute modification processing on the input image sample based on the sensitive attribute type of the input image sample to obtain the attribute-modified image sample, and use the input image sample and the attribute-modified image sample as an image pair; S3, extract the output representation of the image pair after processing the large image model, and calculate the output change index of the input image sample before and after attribute change processing based on the output representation; S4. Determine whether the output change index of the input image sample before and after the attribute change processing is higher than the first preset threshold. If it is higher, further calculate the fairness index quantification score of the input image sample and the attribute-changed image sample. When the fairness index quantification score is higher than the second preset threshold, compare the image activation maps of the input image sample and the attribute-changed image sample to obtain a difference map. Based on the difference map, locate the specific semantic region generated by the model deviation.
2. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, The sensitive attribute label Including one or more of gender, race, and age, the expression is: gender ∈ {male, female} race ∈ {white, black, Asian, Indian, Latino / Hispanic, Middle Eastern, other} age ∈ {0–20, 21–40, 41–60, >60} In the above formula, gender is the sex, race is the race, and age is the age.
3. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, In step S2, when performing attribute modification processing on the input image sample based on the sensitive attribute type of the input image sample, if the sensitive attribute type of the input image sample is gender, the attribute modification processing steps are as follows: Generate input image samples Gender-reversed encoding The expression is: In the above formula, The latent encoding vector of the input image sample. The direction vector is the gender. This is the strength control coefficient; According to the code Generate image samples with changed attributes The expression is: In the above formula, This represents the decoder of a pre-trained image generation model, used to reconstruct image samples from encoded vectors in the latent space; If the sensitive attribute type of the input image sample is race, the attribute change processing steps are as follows: Generate latent encoding after race conversion of input image samples The expression is: In the above formula, The race direction vector; According to the code Generate image samples with changed attributes The expression is: ； If the sensitive attribute type of the input image sample is age, the attribute change processing steps are as follows: Generate latent encoding after race conversion of input image samples The expression is: In the above formula, The age direction vector; According to the code Generate image samples with changed attributes The expression is: 。 4. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, The output change metrics include cosine similarity change and category prediction confidence change, wherein the expression for calculating the cosine similarity change between the input image sample and the attribute-changed image sample is: In the above formula, and These are the deep feature representations extracted from the input image samples and attribute-changed image samples after passing through the large model of the image to be analyzed; The expression for the change in confidence level for category prediction is: In the above formula, This represents the confidence probability that an input image sample is predicted to be its true class by a large image model. This represents the confidence probability that an image sample with changed attributes is predicted as the same true class by the same model.
5. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, The fairness index quantification score includes statistical performance bias, representation bias index, and embedding drift index, wherein the expression for statistical performance bias is: In the above formula, This indicates that the model makes a positive prediction. These represent two different sensitive attribute values; The expression for the deviation index is: In the above formula, are the embedding mean values of the i-th and j-th attribute groups, respectively, and K is the total number of sensitive attribute groups in them; The expression for the embedded drift metric is: In the above formula, and The output representations are the input image sample and the attribute-changed image sample, respectively. This represents a distance metric function used to measure the difference between two output representations.
6. The automatic multidimensional fairness analysis method for large image models according to claim 5, characterized in that, The embedding mean refers to the average feature vector obtained by arithmetically averaging the output feature vectors of all images from the same sensitive attribute group input into the model. The specific calculation steps are as follows: All image samples are grouped according to their sensitive attributes. Each sample in the group is input into the large image model to be analyzed, and the corresponding output representation is extracted. The arithmetic mean of all output representations is calculated to obtain the embedding mean vector for that group. The calculation formula is: In the above formula, It is a vector with the same dimension as the feature embedding dimension of the model output. It is the number of samples in the i-th attribute group. Σ is the output feature vector of the k-th sample, and Σ represents the summation of the feature vectors of all samples.
7. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, In step S4, the image activation maps of the input image sample and the attribute-changed image sample are compared to obtain a difference map, specifically including: For input image samples Image samples with attribute changes In the last convolutional layer, the corresponding gradient weights α and feature maps F are extracted; Calculate the input image samples based on the gradient weight α and the feature map F. Image samples with attribute changes The image activation map is expressed as: In the above formula, The gradient weights of the k-th feature map with respect to class c. This is the feature map of the k-th channel of the last convolutional layer; Input image sample Image samples with attribute changes Image activation map converted into heat map and The difference between the two heatmaps is calculated to obtain a difference map, which is used as a measure of the drift of the region of interest. The calculation formula is as follows: In the above formula, This indicates that the heatmap has been normalized. This is an absolute value operation on a pixel-by-pixel basis.
8. The automatic multidimensional fairness analysis method for large image models according to claim 1, characterized in that, In step S4, the specific semantic region generated by the model deviation is located based on the difference map, which specifically includes: Thresholding is applied to the region of interest drift measure in the difference map to extract regions of significant change. The expression is as follows: In the above formula, The preset significance threshold, For indicator functions, For the generated binarization mask; Binarization mask Connectivity component analysis is performed to identify spatially continuous and independent connected regions, where each connected region represents a potential semantic block whose model attention has changed significantly due to changes in sensitive attributes; Each identified connected region is mapped back to the corresponding pixel region of the original input image sample, and the average drift metric within each connected region is calculated and used as a quantitative indicator of the degree of deviation in that region. Finally, the output contains all connected components. This provides a list of corresponding bias intensities, thereby enabling the localization and attribution of specific semantic regions of model bias.
9. A computer-readable storage medium storing a computer program / instructions, characterized in that, The computer program / instructions are programmed or configured to execute, via a processor, the steps of the automated multidimensional fairness analysis method for large image models according to any one of claims 1 to 8.
10. A computer program product, comprising a computer program / instructions, characterized in that, The computer program / instructions are programmed or configured to execute, via a processor, the steps of the automated multidimensional fairness analysis method for large image models as described in any one of claims 1 to 8.