Depth image classification model evaluation method and system based on background pseudo-correlation measurement

Abstract

The invention discloses a depth image classification model evaluation method and system based on background pseudo-correlation measurement, and the method comprises the steps: obtaining a foreground target mask through a pre-training semantic segmentation model, and separating a foreground image; and generating various background images by using a text-to-image generation model in combination with random noise and semantic guidance weight. Then, background controllability constraint is adopted to adjust background change, meanwhile, the foreground is kept unchanged, and foreground and background images are fused to construct a composite image set; and then, inputting the synthesized images into a to-be-evaluated model, calculating category prediction probability difference, semantic representation of a feature extraction layer and an uncertainty metric value, and finally obtaining a correlation analysis error value to evaluate the dependency degree of the deep learning classification model on the pseudo-correlation features. Through combination of semantic segmentation, text-to-image generation and background controllability constraint technologies, quantification of the background pseudo-correlation dependency degree of the deep learning classification model is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for evaluating deep image classification models based on background pseudo-correlation metrics. Background Technology

[0002] With the rapid development of deep learning technology, image classification models based on architectures such as Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs) have made significant progress in fields such as object recognition, scene analysis, medical image diagnosis, and remote sensing recognition. These models typically perform end-to-end training on large-scale image data to automatically extract features from images and make classification predictions. However, the feature learning process of deep learning classification models essentially relies on statistical correlations in the data, rather than true causal relationships.

[0003] During training, models often simultaneously capture the co-occurrence relationship between foreground targets and the background environment. When a certain category of target in the training data exhibits statistical commonality with a specific background, the model is prone to incorrectly treating background features as the basis for discrimination, thus forming a so-called spurious correlation. For example, in waterbird / landbird classification, since waterbird samples mostly co-occur with the water surface, the classifier often mistakenly uses the water surface background as the basis for discrimination; in target recognition, tank images often appear in grassland or sandy scenes, and the model may learn a non-causal association of "grassland → tank"; in medical image diagnosis, differences in equipment, lighting, or texture backgrounds may also be mistakenly identified by the model as disease features; and placing objects in inconsistent environmental backgrounds (such as "a cow placed in the ocean") can lead to serious misclassification by the model.

[0004] This spurious correlation problem may not be obvious under the independent and identically distributed (IID) assumption, but under conditions of background change, domain shift, or open environment, the model's predictive stability and generalization ability will significantly decrease. Specifically, this manifests as: I. The model's outputs are inconsistent for the same foreground target under different backgrounds; II. The model exhibits high sensitivity to slight perturbations of irrelevant background features; III. The model cannot distinguish between causal features (foreground) and non-causal features (background).

[0005] Therefore, effectively assessing the dependence of deep learning classification models on the background and characterizing their spurious correlation risk has become an important direction for research on model interpretability and robustness. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a method and system for evaluating deep image classification models based on background pseudo-correlation metrics. By combining semantic segmentation, controllable text-to-image generation and background replacement, and a metric mechanism based on prediction probability and feature consistency, a synthetic image set with a constant foreground and varying background is constructed. This quantifies the dependence of deep image classification models on background pseudo-correlation, enabling accurate evaluation of their correlation analysis errors.

[0007] On the one hand, evaluation methods for deep image classification models based on background pseudo-correlation metrics include:

[0008] S1. Acquire the input image and use a pre-trained semantic segmentation model to segment the foreground target in the input image, generate a foreground mask, and separate the foreground image from the input image based on the foreground mask;

[0009] S2 utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby achieving adjustable preset of the background variation range and obtaining background images of various styles.

[0010] S3 involves fusing the foreground and background images to construct a set of synthetic images with a constant foreground and varying background. During the image fusion process, a background controllability constraint based on generation consistency is introduced. By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process; the changes in the synthetic samples mainly come from background factors, so as to suppress the occurrence of non-causal offsets in the synthetic image;

[0011] S4. For a dataset of synthetic images with the same foreground, input the set of synthetic images one by one into the deep learning image classification model to be evaluated, calculate the probability distribution entropy of each synthetic image and obtain the uncertainty metric value; based on the uncertainty metric value, obtain the predicted probability of the correct label corresponding to each synthetic image with the same foreground and the semantic representation of the feature extraction layer of each synthetic image.

[0012] S5, calculate the relationship between the predicted probability of the correct label for each synthesized image and the baseline predicted probability. The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. Based on the semantic representation of the feature extraction layer of each synthetic image, the weighted combination of the cosine similarity term and the Euclidean distance term is calculated to obtain the counterfactual consistency term, and finally the correlation analysis error value is obtained.

[0013] Furthermore, in S1, the semantic segmentation model is based on DeepLabV3 or its variants, and is fine-tuned under weak supervision for segmentation tasks of specific target categories by combining manually labeled samples with pseudo-label generation strategies. The generated foreground mask is obtained through the semantic segmentation model fine-tuned on specific target categories.

[0014] Furthermore, in S2, the text-to-image generation model is the StableDiffusion model. In the generation stage of the StableDiffusion model, a controllable perturbation parameter is introduced. By setting the intensity of random noise and semantic guidance weights, the adjustability of the background change amplitude is achieved, so as to obtain the model's sensitivity to background changes of different intensities.

[0015] Furthermore, in S3, background controllability constraints based on generation consistency are implemented. The calculation formula is as follows:

[0016] ;

[0017] in, Indicates the input image; Represents the foreground image; This represents the generated background image; This indicates an operation that blends the foreground and background. This represents the intermediate feature extraction function of the pre-trained model, used to measure semantic feature differences; Structural similarity metrics are used to measure the degree of consistency between the fused image and the original image in terms of structure and brightness distribution. , This is the balance coefficient.

[0018] Furthermore, in S2, each foreground image in the synthetic image set is fused with at least 16 different generated background images to ensure the diversity and statistical significance of background variations.

[0019] Furthermore, in S5, the formula for calculating the uncertainty metric is as follows:

[0020] ;

[0021] in, Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability.

[0022] Furthermore, in S5, the formula for calculating the counterfactual consistency term is as follows:

[0023] ;

[0024]

[0025] in, This represents an average of the features of the same foreground against different backgrounds; This represents the total number of composite images of foreground i; Indicates the first The prospect in the Feature representation in this context; This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 The average characteristics of a foreground under all background conditions.

[0026] Furthermore, the formula for calculating the correlation analysis error metric BSCM is as follows:

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] ;

[0033] in, For the first The baseline prediction probability of a given prospect; For the first The prospect in the The predicted probability of the correct label corresponding to the synthesized image under such background; Represents the foreground image The total number of composite images; Indicates the foreground The degree of dependence on background during classification; Represents the foreground obtained after segmentation The total number; This represents a counterfactual consistency term, constraining the model to produce consistent feature representations under different contextual conditions; This represents an average of the features of the same foreground against different backgrounds; Indicates the first The prospect in the Feature representation in this context; This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 A prospect in Average characteristics under different background conditions; The foreground i is represented by an average uncertainty measure based on the synthetic image set; Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability; , , For weight distribution; This represents vector multiplication.

[0034] On the other hand, evaluation systems for deep image classification models based on background pseudo-correlation metrics include:

[0035] The separation module acquires the input image and uses a pre-trained semantic segmentation model to segment the foreground target in the input image, generates a foreground mask, and separates the foreground image from the input image based on the foreground mask;

[0036] The background generation module utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby enabling adjustable presets of the background variation range and obtaining background images of various styles.

[0037] The fusion module fuses the foreground and background images to construct a set of synthetic images with a constant foreground and varying background. During the image fusion process, a background controllability constraint based on generation consistency is introduced. By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process; the changes in the synthetic samples mainly come from background factors, so as to suppress the occurrence of non-causal offsets in the synthetic image;

[0038] The uncertainty measurement module, for a dataset of synthetic images with the same foreground, inputs the set of synthetic images one by one into the deep learning image classification model to be evaluated, calculates the probability distribution entropy of each synthetic image and obtains the uncertainty measurement value accordingly; based on the uncertainty measurement value, it obtains the predicted probability of the correct label corresponding to each synthetic image with the same foreground and the semantic representation of the feature extraction layer of each synthetic image.

[0039] The error calculation module calculates the difference between the predicted probability of the correct label for each synthesized image and the baseline predicted probability. The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. Based on the semantic representation of the feature extraction layer of each synthetic image, the weighted combination of the cosine similarity term and the Euclidean distance term is calculated to obtain the counterfactual consistency term, and finally the correlation analysis error value is obtained.

[0040] The present invention adopts the above technical solution and has the following beneficial effects:

[0041] (1) By constructing a set of synthetic images with a constant foreground and diverse backgrounds, this invention can accurately quantify the sensitivity of image classification models to background changes and effectively reveal whether the model relies on non-causal background pseudo-correlation for prediction.

[0042] (2) By introducing background controllability constraints based on generative consistency and counterfactual consistency terms, this invention suppresses non-causal bias and enables the model to learn more stable and causal semantic features under different background conditions.

[0043] (3) This invention constructs a comprehensive correlation analysis error metric (BSCM) by integrating prediction probability differences, feature similarity and prediction uncertainty, providing a clear and quantifiable basis for model diagnosis, optimization and reliable deployment. Attached Figure Description

[0044] Figure 1 This is a flowchart of the deep image classification model evaluation method based on background pseudo-correlation metric, as described in an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of an experiment using the background pseudo-correlation measurement method in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram illustrating the synthesis of the same foreground image against different backgrounds according to an embodiment of the present invention;

[0047] Figure 4 This is a heat map of various images during classification in an embodiment of the present invention;

[0048] Figure 5 This is a diagram of the evaluation system for a deep image classification model based on background pseudo-correlation metric, as described in an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0050] like Figure 1As shown, the present invention provides a method for evaluating deep image classification models based on background pseudo-correlation metrics, comprising:

[0051] S1: Acquire the input image and use a pre-trained semantic segmentation model to segment the foreground target in the input image, generate a foreground mask, and separate the foreground image from the input image based on the foreground mask.

[0052] Specifically, in S1, the semantic segmentation model is based on DeepLabV3 or its variants, and is fine-tuned under weak supervision for segmentation tasks of specific target categories by combining manually labeled samples with pseudo-label generation strategies. The generated foreground mask is obtained by the semantic segmentation model fine-tuned on specific target categories.

[0053] S2 utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby enabling adjustable presets of background variation range and obtaining background images of various styles.

[0054] Specifically, the text-to-image generation model is the StableDiffusion model. In the generation stage of the StableDiffusion model, controllable perturbation parameters are introduced. By setting the intensity of random noise and semantic guidance weights, the adjustability of the background change range is achieved, so as to obtain the model's sensitivity to background changes of different intensities.

[0055] S3 involves fusing the same foreground and background images to construct a set of synthetic images with a constant foreground and varying background, introducing a background controllability constraint based on generation consistency. By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process. The changes in the synthesized samples mainly come from background factors, thus suppressing the occurrence of non-causal shifts in the synthesized images.

[0056] For each set of synthetic images of the foreground, there is at least a synthetic image containing the foreground image and 16 different generated background images to ensure the diversity and statistical significance of background variation.

[0057] Specifically, the purpose of feature similarity constraints and structural similarity constraints is to ensure that the features of the synthesized image remain basically unchanged compared with the original image in the feature semantic space, thereby reducing false differences; the structural similarity index is used to measure the degree of consistency between the original foreground of the synthesized image and the structure and brightness distribution.

[0058] Specifically, in S3, the process of inputting the synthetic image set into the deep learning image classification model to be evaluated also includes: calculating the push-pull heat map H for classification prediction of the input image and each synthetic image; the push-pull heat map is constructed by combining the forward sensitivity analysis and backward gradient propagation information of the deep learning image classification model to be evaluated, so as to simultaneously capture the positive response strength and backward gradient contribution of the model to the input features, thereby obtaining a more accurate distribution of the region of interest. The calculation formula of the push-pull heat map H is as follows:

[0059] ;

[0060] in, Indicates the contribution of the backward gradient; Indicates forward sensitivity response; is the weighting fusion factor; H is the final generated heat map.

[0061] Specifically, based on the background controllability constraints of generation consistency The calculation formula is as follows:

[0062] ;

[0063] in, Indicates the input image; Represents the foreground image; This represents the generated background image; This indicates an operation that blends the foreground and background. This represents the intermediate feature extraction function of the pre-trained model, used to measure semantic feature differences; Structural similarity metrics are used to measure the degree of consistency between the fused image and the original image in terms of structure and brightness distribution. , This is the balance coefficient.

[0064] The formula for calculating the background interference metric is as follows:

[0065] ;

[0066] in, Indicates background interference measurement; Represents a classification model; This represents the foreground segmentation and background compositing operations; This indicates linear scaling of the entire image pixels; Indicates the input image; This represents the background image.

[0067] S4. For a dataset of synthetic images with the same foreground, input the synthetic images one by one into the deep learning image classification model to be evaluated, calculate the probability distribution entropy of each synthetic image to obtain the uncertainty measure; at the same time, obtain the predicted probability of the correct label (classification label of the original image) corresponding to each synthetic image with the same foreground, and the semantic representation of the feature extraction layer of each synthetic image.

[0068] S5, calculate the predicted probability of the correct label (classification label of the original image) for each synthesized image and the baseline predicted probability (classification probability of the synthesized image with foreground and neutral background (e.g., uniform noise)). The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. The cosine similarity term and the Euclidean distance term are calculated based on the semantic representation of the feature extraction layer of each synthetic image to obtain the counterfactual consistency term. Finally, the correlation analysis error value is obtained.

[0069] The formula for calculating the uncertainty metric is as follows:

[0070] ;

[0071] in, Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability.

[0072] The formula for calculating the counterfactual consistency term is as follows:

[0073] ;

[0074]

[0075] in, This represents an average of the features of the same foreground against different backgrounds; This represents the total number of composite images of foreground i; Indicates the first The prospect in the Feature representation in this context (intermediate layer feature representation of the model, usually the output of the penultimate layer); This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 The average characteristics of a foreground under all background conditions.

[0076] The formula for calculating the correlation analysis error value (BSCM) is as follows:

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] in, For the first The baseline prediction probability of a given prospect; For the first The prospect in the The predicted probability of the correct label corresponding to the synthesized image under such background; Represents the foreground image The total number of composite images; Indicates the foreground The degree of dependence on background during classification Represents the foreground obtained after segmentation The total number; This represents a counterfactual consistency term, constraining the model to produce consistent feature representations under different contextual conditions; This represents an average of the features of the same foreground against different backgrounds; Indicates the first The prospect in the Feature representation in this context; This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 A prospect in Average characteristics under different background conditions; The foreground i is represented by an average uncertainty measure based on the synthetic image set; Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability; , , For weight distribution; This represents vector multiplication.

[0084] Specifically, in the background compositing process of this example, diverse backgrounds are constructed to replace the original scene. For example, diverse background generation utilizes the Stable Diffusion text-to-image generation model to automatically generate diverse background images based on prompts (such as "pine forest at sunrise" or "snowy valley"). Another example is image compositing: the segmented foreground is composited with the generated diverse background images; a foreground object must be stitched with at least 16 different backgrounds to obtain a diverse background. (j=1, 2, ..., 16), construct a complete set of composite images of foreground and background. The above content calculates the pseudo-correlation dependency of the classification model on the background during classification. Building upon this, a more intuitive visualization is proposed—attention visualization: comparing the original image and the synthesized image, such as... Figure 3 As shown, push-pull heatmaps predicting model heats are calculated. These heatmaps are constructed by combining forward sensitivity analysis and backward gradient propagation information of the deep learning image classification model being evaluated. This allows for the simultaneous capture of the model's positive response strength and backward gradient contribution to input features, resulting in a more accurate distribution of the region of interest. This can be used to visualize the importance of the model to regions in an image. By overlaying the heatmap onto the input image, we can visually compare the changes in the model's attention region between the original and synthetic backgrounds (e.g., ...). Figure 3 This allows us to verify whether the drift in classification decisions is accompanied by attention shifts, thus providing an intuitive analysis of whether the model pays significant attention to spurious background features during classification. For example... Figure 4 As shown, Figure 3 A diagram illustrating the popularity of each image during classification.

[0085] In summary, the main innovations of this invention include: a generative background replacement evaluation pipeline: accurately extracting tank foregrounds using a finely tuned DeepLabV3 and employing Stable Diffusion to synthesize diverse battlefield backgrounds, constructing rich test samples; and a correlation analysis error metric: defining a metric based on... Norm-based BSCM measures the strength of a model's dependence on the background by the change in the tank classification probability vector under different backgrounds; Cross-model and training strategy comparison: On the tank dataset, the BSCM values ​​of classification models such as VGG16, ResNet, and EfficientNet are compared; Reproducible end-to-end process: A complete experimental scheme is provided from foreground segmentation, background generation, image synthesis to classification evaluation, realizing closed-loop verification of training environment randomization and metric evaluation.

[0086] Compared with the prior art, the present invention is based on The background pseudo-correlation measure of norm provides a stable, continuous, and differentiable evaluation metric that can systematically analyze the model's sensitivity to background changes. This metric is not only applicable to the model evaluation stage but can also serve as an important basis for training optimization. For example, it can be used to introduce background invariance regularization, background smoothness constraints, or design targeted data augmentation strategies to guide the model to focus more on foreground semantic features rather than volatile background patterns. Secondly, at the theoretical level, this invention ensures the robustness of the metric under complex real-world conditions through Lipschitz continuity and BatchNorm invariance analysis: on the one hand, foreground segmentation and background synthesis are modeled as functions g, and the classification network is modeled as a function f, proving the composite mapping... It exhibits Lipschitz stability against background perturbations, meaning that bounded background perturbations inevitably lead to an upper bound on the change in the output probability distribution. On the other hand, through spectral normalization and normalization layer verification, it is demonstrated that mechanisms such as BatchNorm can effectively maintain the invariance of the segmentation, synthesis, and classification processes to scale changes when the input image is scaled as a whole (such as changes in brightness or sensor gain). This ensures that the background interference intensity index remains stable and consistent under different resolutions, lighting conditions, and device conditions, supporting repeatable system evaluation across scenes and platforms.

[0087] Specifically, regarding the stability proof, let the original input image be 𝑋, and the foreground segmentation and background compositing operations are as follows:

[0088]

[0089] Here, Mask is the foreground mask extracted by DeepLabV3, and Blend represents concatenating or blurring the foreground with the background B. The classification model is... The output is a probability vector. .

[0090] According to Lipschitz's definition of continuity, for any two backgrounds If a constant exists Make

[0091]

[0092] Then it is called Regarding input It is Kf-Lipschitz continuous. Existing work has shown that common convolutional neural networks (such as VGG, ResNet, and EfficientNet) can obtain a computable Lipschitz upper bound when the spectral norm of the weights in each layer is constrained.

[0093] And due to foreground splicing operation Essentially a linear superposition or Gaussian blur, it also satisfies the Lipschitz property for differences in background pixels, meaning there exists a constant. , making

[0094] ;

[0095] Therefore, composite mapping Right now It also satisfies Lipschitz continuity:

[0096]

[0097] make The maximum perturbation amplitude of the background pixels is represented by the output probability distribution. difference:

[0098] ;

[0099] This indicates that when the background change is bounded, the indicator... It is also bounded, thus proving the stability of the method under background perturbations.

[0100] Regarding the proof of scale invariance, assume that the entire image pixels are linearly scaled: Similarly, background Considering the segmentation and synthesis stages: DeepLabV3 has segmentation invariance. DeepLabV3's convolutional layers are followed by BatchNorm and an activation function. BatchNorm has a scale normalization effect on the input features, and its output...

[0101] ;

[0102] It is unaffected by the overall linear scaling of z. Therefore, the segmentation mask and Maintain consistency.

[0103] Regarding background composition consistency: After scaling the composed background, splicing or blurring with the foreground only changes the global brightness, without affecting the geometry of the foreground mask or the composition splicing method. This can be considered...

[0104]

[0105] Regarding scale invariance of classification networks: pre-trained classification models also include normalization layers such as BatchNorm, and the input features... After BatchNorm, its normalization features are not affected. Impact, and subsequent activation, full connectivity and The calculation of the probability distribution is based on the normalization feature, ensuring...

[0106]

[0107] That is, the probability distribution of the classification output remains strictly unchanged under the overall pixel scaling.

[0108] In summary, background interference measurement

[0109]

[0110] In any positive scaling transformation By maintaining consistency, we have proven the scale invariance of the method.

[0111] Through the above theoretical analysis, not only is a proof of the bounded stability of the background interference index provided from the perspective of Lipschitz continuity, but also, by utilizing the normalization mechanism of BatchNorm and the invariance of function composition, it is proven that the index remains invariant under arbitrary input scale transformations. This further ensures the theoretical reliability and practical robustness of the background interference assessment method, laying the foundation for its subsequent extension to more complex tasks.

[0112] like Figure 5 As shown, this embodiment also discloses a method for evaluating deep image classification models based on background pseudo-correlation metrics, including:

[0113] Separation module 51 acquires the input image and uses a pre-trained semantic segmentation model to segment the foreground target in the input image, generates a foreground mask, and separates the foreground image from the input image based on the foreground mask;

[0114] Background generation module 52 utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby achieving adjustable preset of the background variation range and obtaining background images of various styles.

[0115] The fusion module 53 fuses the foreground and background images to construct a set of synthetic images with a constant foreground and varying background. During the image fusion process, a background controllability constraint based on generation consistency is introduced. By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process; the changes in the synthetic samples mainly come from background factors, so as to suppress the occurrence of non-causal offsets in the synthetic image;

[0116] Uncertainty measurement module 54, for a dataset of synthetic images with the same foreground, inputs the set of synthetic images one by one into the deep learning image classification model to be evaluated, calculates the probability distribution entropy of each synthetic image and obtains the uncertainty measurement value accordingly; based on the uncertainty measurement value, it obtains the predicted probability of the correct label corresponding to each synthetic image with the same foreground and the semantic representation of the feature extraction layer of each synthetic image.

[0117] Error calculation module 55 calculates the difference between the predicted probability of the correct label for each synthesized image and the baseline predicted probability. The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. Based on the semantic representation of the feature extraction layer of each synthetic image, the weighted combination of the cosine similarity term and the Euclidean distance term is calculated to obtain the counterfactual consistency term, and finally the correlation analysis error value is obtained.

[0118] The specific implementation of the deep image classification model evaluation system based on background pseudo-correlation metric is the same as that of the deep image classification model evaluation method based on background pseudo-correlation metric, and will not be described again in this embodiment.

[0119] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for evaluating deep image classification models based on background pseudo-correlation metrics, characterized in that, Includes the following steps: S1. Acquire the input image and use a pre-trained semantic segmentation model to segment the foreground target in the input image, generate a foreground mask, and separate the foreground image from the input image based on the foreground mask; S2 utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby achieving adjustable preset of the background variation range and obtaining background images of various styles. S3, performs image fusion on the foreground and background images to construct a set of synthetic images with a constant foreground and a changing background; Introducing background controllability constraints based on generative consistency during image fusion process By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process; the changes in the synthetic samples mainly come from background factors, so as to suppress the occurrence of non-causal offsets in the synthetic image; S4. For a dataset of synthetic images with the same foreground, input the set of synthetic images one by one into the deep learning image classification model to be evaluated, calculate the probability distribution entropy of each synthetic image, and obtain the uncertainty measure value accordingly. Based on the uncertainty metric, the predicted probability of the correct label for each synthetic image with the same foreground and the semantic representation of the feature extraction layer of each synthetic image are obtained. S5, calculate the relationship between the predicted probability of the correct label for each synthesized image and the baseline predicted probability. The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. Based on the semantic representation of the feature extraction layer of each synthetic image, the weighted combination of the cosine similarity term and the Euclidean distance term is calculated to obtain the counterfactual consistency term, and finally the correlation analysis error value is obtained.

2. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S1, the semantic segmentation model is based on DeepLabV3 or its variants, and is fine-tuned under weak supervision for segmentation tasks of specific target categories by combining manually labeled samples with pseudo-label generation strategies. The generated foreground mask is obtained by the semantic segmentation model fine-tuned on specific target categories.

3. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S2, the text-to-image generation model is the StableDiffusion model. In the generation stage of the StableDiffusion model, a controllable perturbation parameter is introduced. By setting the intensity of random noise and semantic guidance weights, the adjustability of the background change range is achieved, so as to obtain the model's sensitivity to background changes of different intensities.

4. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S3, background controllability constraints based on generative consistency are used. The calculation formula is as follows: ； in, Indicates the input image; Represents the foreground image; This represents the generated background image; This indicates an operation that blends the foreground and background. This represents the intermediate feature extraction function of the pre-trained model, used to measure semantic feature differences; Structural similarity metrics are used to measure the degree of consistency between the fused image and the original image in terms of structure and brightness distribution. , This is the balance coefficient.

5. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S2, each foreground image in the synthetic image set is fused with at least 16 different generated background images to ensure the diversity and statistical significance of background variations.

6. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S5, the formula for calculating the uncertainty measure is as follows: ； in, Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability.

7. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, characterized in that, In S5, the formula for calculating the counterfactual consistency term is as follows: ； in, This represents an average of the features of the same foreground against different backgrounds; This represents the total number of composite images of foreground i; Indicates the first The prospect in the Feature representation in this context; This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 The average characteristics of a foreground under all background conditions.

8. The method for evaluating a deep image classification model based on background pseudo-correlation metric according to claim 1, wherein the calculation formula for the correlation analysis error metric BSCM is as follows: ； ； ； ； ; ； in, For the first The baseline prediction probability of a given prospect; For the first The prospect in the The predicted probability of the correct label corresponding to the synthesized image under such background; Represents the foreground image The total number of composite images; Indicates the foreground The degree of dependence on background during classification; Represents the foreground obtained after segmentation The total number; This represents a counterfactual consistency term, constraining the model to produce consistent feature representations under different contextual conditions; This represents an average of the features of the same foreground against different backgrounds; Indicates the first The prospect in the Feature representation in this context; This is the balance coefficient between cosine similarity and Euclidean distance; Table 1 A prospect in Average characteristics under different background conditions; The foreground i is represented by an average uncertainty measure based on the synthetic image set; Indicates the first The prospect in the The uncertainty value when performing classification prediction under certain backgrounds; When performing uncertainty measurement, the selection method is as follows: Before classification Predicted probability; express When classifying, the tags are The classification probability; , , For weight distribution; This represents vector multiplication.

9. A deep image classification model evaluation system based on background pseudo-correlation metric, characterized in that, include: The separation module acquires the input image and uses a pre-trained semantic segmentation model to segment the foreground target in the input image, generates a foreground mask, and separates the foreground image from the input image based on the foreground mask; The background generation module utilizes a text-to-image generation model to introduce random noise intensity and semantic guidance weights during the generation stage, thereby enabling adjustable presets of the background variation range and obtaining background images of various styles. The fusion module merges the foreground and background images to construct a set of synthetic images with a constant foreground and a changing background. Introducing background controllability constraints based on generative consistency during image fusion process By using feature similarity constraints and structural similarity constraints, semantic feature differences and structural distortion are limited, ensuring that the foreground semantics remain stable during the background replacement process; the changes in the synthetic samples mainly come from background factors, so as to suppress the occurrence of non-causal offsets in the synthetic image; The uncertainty measurement module, for a dataset of synthetic images with the same foreground, inputs the set of synthetic images one by one into the deep learning image classification model to be evaluated, calculates the probability distribution entropy of each synthetic image, and obtains the uncertainty measurement value accordingly. Based on the uncertainty metric, the predicted probability of the correct label for each synthetic image with the same foreground and the semantic representation of the feature extraction layer of each synthetic image are obtained. The error calculation module calculates the difference between the predicted probability of the correct label for each synthesized image and the baseline predicted probability. The norm difference value is calculated, and the average of all norm difference values ​​is used to obtain the background pseudo-correlation measure. Based on the semantic representation of the feature extraction layer of each synthetic image, the weighted combination of the cosine similarity term and the Euclidean distance term is calculated to obtain the counterfactual consistency term, and finally the correlation analysis error value is obtained.

Images