Image classification robustness test enhancement method and system based on artificial intelligence model

Abstract

The invention discloses an artificial intelligence model-based image classification robustness test enhancement method and system, and the method comprises the steps: carrying out the deep semantic analysis of the image content based on a multi-modal artificial intelligence model, automatically recognizing and extracting the class related features and non-class related features in an image, and generating a structured feature analysis report; the method comprises the following steps: generating a Keep Strategy and a Replace Strategy which are complementary to each other, and generating a Keep Strategy and a Replace Strategy which are complementary to each other; performing image editing operation based on the generated strategy to generate a corresponding image sample; performing automatic quality verification on the edited image sample, filtering out low-quality samples which do not conform to expectation, and generating a test sample; and comprehensively evaluating the target classification model based on the generated test samples, calculating performance indexes of the model on different types of test samples, generating a detailed robustness analysis report, and identifying weak links and improvement directions of the model. According to the scheme, a complete'generation-test-evaluation 'closed-loop system can be established, a standardized and quantifiable robustness evaluation index system is formed, and the model robustness can be scientifically and quantitatively evaluated.

Description

Technical Field

[0001] This invention relates to software testing technology, specifically to testing and enhancement technology for image classification models. Background Technology

[0002] Currently, image classification model testing and enhancement techniques are mainly divided into three categories: traditional data augmentation, deep learning-based augmentation, and adversarial example generation. Each category proposes different improvement ideas for the model's generalization ability and robustness.

[0003] First, traditional data augmentation methods aim to increase the diversity of training data by performing a series of predefined transformations on the original image. These methods typically include geometric transformations (such as random rotation, flipping, scaling, and cropping) to simulate different shooting angles and distances; color space adjustments (such as changes in brightness, contrast, saturation, and hue) to adapt to diverse lighting and camera conditions; and the addition of noise (such as Gaussian noise, salt-and-pepper noise, and Poisson noise) to simulate interference during image acquisition. While these operations are simple and efficient, they are mostly rule-based and lack a deep understanding of the image content.

[0004] With the development of deep learning, more complex and intelligent enhancement methods have emerged. These methods leverage the capabilities of neural networks to perform more targeted transformations on images. For example, Random Erasing randomly occludes regions in an image, prompting the model to learn more local features; the CutMix method replaces parts of one image with content from another image and mixes the labels accordingly, thereby improving the model's localization and generalization abilities; methods such as SamplePairing and Mixup generate new training samples by weighted mixing of pixels and labels from different categories of images, effectively alleviating the overfitting problem of the model.

[0005] Furthermore, adversarial example generation techniques are also widely used to test model robustness. These methods examine a model's performance under extreme conditions by adding carefully designed micro-perturbations to the original image. Typical examples include FGSM (FastGradient Sign Method) and PGD (Projected Gradient Descent), which generate adversarial examples through single-step or multi-step gradient perturbations, respectively; C&W attacks employ optimization methods to generate visually imperceptible but misleading examples. These techniques provide important tools for evaluating model security and robustness.

[0006] However, existing image classification model testing and enhancement techniques have revealed several shortcomings in practical applications. First, current methods generally lack a systematic overfitting testing framework. While these methods aim to improve the overall performance of the model, they often fail to effectively distinguish whether the model is overly reliant on environmental factors such as background, lighting, and viewpoint, which are unrelated to the category. They also struggle to determine whether the model has merely memorized the surface appearance of the category without truly understanding its core semantic features. Furthermore, existing technologies lack a standardized and quantifiable robustness evaluation system, resulting in a lack of scientific basis for assessing model robustness.

[0007] Secondly, most existing data augmentation strategies employ pre-defined random transformation rules, lacking intelligence and specificity. These transformations are often unrelated to the image content, easily damaging key category features and even generating unreasonable images. Because they cannot be tailored to the characteristics of different categories, the augmentation effect is greatly limited. Furthermore, existing methods have limited ability to understand the semantic content of images, making it difficult to achieve more advanced semantic-level editing operations.

[0008] In terms of sample validation, the quality control mechanisms of existing methods are also imperfect; the effectiveness of augmented samples usually relies on manual evaluation, which is inefficient and highly subjective. There is a lack of automated quality evaluation standards, making it difficult to guarantee the validity and rationality of the generated samples, and it is also impossible to automatically screen out unqualified augmented samples.

[0009] Finally, existing testing methods have limited coverage and lack a comprehensive evaluation system. The testing scenarios are too narrow to cover the various complex challenges models may encounter in real-world applications. Furthermore, the lack of unified robustness evaluation metrics and testing protocols makes it impossible to scientifically quantify the model's performance under different types of challenges. These shortcomings severely restrict the reliability and widespread application of image classification models in real-world scenarios. Summary of the Invention

[0010] To address the problems existing in the prior art, the purpose of this invention is to provide an enhanced robustness testing scheme for image classification based on an artificial intelligence model. This scheme can establish a complete "generation-testing-evaluation" closed-loop system, form a standardized and quantifiable robustness evaluation index system, and can scientifically quantify the robustness of the model.

[0011] To achieve the above objectives, this invention provides an image classification robustness testing enhancement method based on an artificial intelligence model, the method comprising: (1) Based on the multimodal artificial intelligence model, perform deep semantic analysis on the image content, automatically identify and extract category-related features and non-category-related features in the image, and sort and weight the features according to their importance to the classification task to generate a structured feature analysis report; (2) Based on the intelligent analysis results, a complementary feature preservation strategy (Keep Strategy) and a feature replacement strategy (Replace Strategy) are generated. The feature preservation strategy (Keep Strategy) is set to focus on keeping the core category features in the image unchanged, while making significant modifications to the non-core features. The feature replacement strategy (Replace Strategy) is set to remove or replace the core category features in the image, while keeping the non-core features relatively stable. (3) Based on the feature preservation strategy Keep Strategy and feature replacement strategy ReplaceStrategy generated in step (2), perform image editing operations to generate corresponding image samples; (4) Perform automated quality verification on the image samples edited in step (3), use an artificial intelligence model to verify the classification results of the edited image samples, so that the samples generated based on the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are incorrectly classified, filtering out low-quality samples that do not meet expectations, and generating test samples. (5) Based on the test samples generated in step (4), the target classification model is comprehensively evaluated, the performance indicators of the model on different types of test samples are calculated, and a detailed robustness analysis report is generated to identify the weak links and improvement directions of the model.

[0012] Furthermore, step (1) of the method performs deep semantic analysis on image content based on the Visual Language Model (VLM), and automatically evaluates the relevance of each feature to the target category through multi-level feature extraction, including pixel-level, object-level and scene-level feature analysis.

[0013] Furthermore, the feature preservation strategy generated in step (2) of the method, based on the list of non-core features in the structured feature analysis report generated in step (1), sends editing instructions to the VLM to generate prompts, so that the VLM outputs editing instructions in JSON format, including: editing type, source region description, target effect description, and execution order structured information.

[0014] Furthermore, the feature replacement strategy generated in step (2) of the method is based on the core features in the core feature list of the structured feature analysis report generated in step (1), constructs a knowledge base of obfuscated objects, and automatically selects the best obfuscated replacement by integrating the reasoning ability of VLM.

[0015] Furthermore, in step (4) of the method, when performing automated quality verification, multi-dimensional quality assessment is performed using the same VLM and classification result verification is performed using the same VLM.

[0016] Furthermore, when performing robustness evaluation in step (5) of the method, the target classification model is diagnosed for overfitting type using test samples generated by the feature preservation strategy KeepStrategy and the feature replacement strategy Replace Strategy; based on the diagnosis results, VLM is used to subdivide the overfitting problem into specific types such as environment-dependent overfitting, morphological memory overfitting, and feature confusion overfitting.

[0017] To achieve the above objectives, the present invention provides an image classification robustness testing enhancement system based on an artificial intelligence model, the system comprising: The intelligent analysis module is configured to perform deep semantic analysis on image content based on a multimodal artificial intelligence model, automatically identify and extract category-related and non-category-related features in the image, and sort and weight the features according to their importance to the classification task to generate a structured feature analysis report. The strategy generation module is configured to interact with the intelligent analysis module and generate complementary feature preservation strategies (Keep Strategy and Replace Strategy) based on the intelligent analysis results of the intelligent analysis module. The feature preservation strategy (Keep Strategy) is configured to focus on keeping the core category features in the image unchanged while significantly modifying non-core features. The feature replacement strategy (Replace Strategy) is configured to remove or replace the core category features in the image while keeping the non-core features relatively stable. The image editing module is configured to interact with the strategy generation module and perform image editing operations based on the feature preservation strategy (Keep Strategy) and feature replacement strategy (Replace Strategy) generated by the strategy generation module to generate corresponding image samples. The quality verification module is configured to interact with the image editing module and can automatically verify the quality of the image samples edited by the image editing module. It uses an artificial intelligence model to verify the classification results of the edited image samples, so that the samples generated based on the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are misclassified, filtering out low-quality samples that do not meet expectations, and generating test samples. The robustness evaluation module is configured to interact with the quality verification module. It can comprehensively evaluate the target classification model based on the test samples generated by the quality verification module, calculate the model's performance indicators on different types of test samples, and generate a detailed robustness analysis report to identify the model's weaknesses and improvement directions.

[0018] Furthermore, the intelligent analysis module performs deep semantic analysis on image content based on the Visual Language Model (VLM). Through multi-level feature extraction, including pixel-level, object-level, and scene-level feature analysis, it automatically evaluates the relevance of each feature to the target category.

[0019] Furthermore, the feature retention strategy generated by the strategy generation module sends editing instructions to the VLM based on the list of non-core features in the structured feature analysis report, generating prompts so that the VLM outputs editing instructions in JSON format, including: editing type, source region description, target effect description, and structured information of execution order; the generated feature replacement strategy constructs a knowledge base of obfuscated objects based on the core features in the core feature list in the structured feature analysis report, and automatically selects the best obfuscated replacement by integrating the reasoning ability of the VLM.

[0020] Furthermore, the quality verification module performs multi-dimensional quality assessment using the same VLM in the intelligent analysis module and verifies the classification results using the same VLM.

[0021] Furthermore, the robustness evaluation module uses test samples generated by the feature preservation strategy (Keep Strategy) and the feature replacement strategy (Replace Strategy) to diagnose the overfitting type of the target classification model. Based on the diagnostic results, the overfitting problem is further subdivided into specific types such as environment-dependent overfitting, morphological memory overfitting, and feature confusion overfitting using VLM.

[0022] The image classification robustness testing enhancement scheme for artificial intelligence models provided by this invention establishes a complete "generation-testing-evaluation" closed-loop system, forming a standardized and quantifiable robustness evaluation index system, which enables scientific quantitative evaluation of model robustness.

[0023] The image classification robustness testing enhancement scheme based on artificial intelligence model provided by this invention can automate the entire process from image input to overfitting diagnostic report output without any human intervention; at the same time, it automatically filters low-quality samples through multiple verification mechanisms to ensure the validity of test data; during implementation, each functional module is designed independently, which is convenient for maintenance and expansion.

[0024] The solution provided by this invention can be extended to a wider range of application scenarios. For example, in the field of computer vision, it can be extended to tasks such as object detection, semantic segmentation, and instance segmentation; in the field of multimedia processing, it can be extended to tasks such as video classification and audio classification; and in the field of cross-modal applications, it can be extended to tasks such as image retrieval and visual question answering.

[0025] The image classification robustness testing enhancement scheme based on artificial intelligence models provided by this invention has the following technical features compared with existing technologies: (1) Dual testing strategy framework: The present invention provides complementary Keep and Replace strategies. Keep strategy: maintain the core features of the category and significantly modify non-core features (background, lighting, viewpoint, style, etc.) to test the model’s over-reliance on environmental factors. Replace strategy: remove or replace the core features of the category and maintain non-core features to test the model’s overfit to feature morphology. These two strategies complement each other and comprehensively evaluate the model’s overfitting problem from the two dimensions of “environmental robustness” and “feature understanding depth” respectively. At the same time, based on the overfitting mechanism of deep learning models, a systematic overfitting test and diagnosis of the model’s overfitting problem is realized.

[0026] (2) Deep application of multimodal large models: In the intelligent image analysis stage, the present invention uses VLM (such as InternVL3) to perform deep semantic analysis on images and identify category-related and non-category-related features; at the same time, it interacts with VLM through natural language prompts to generate structured editing instructions and verification results; and through the collaborative work of VLM in the analysis, generation and verification stages, a complete "understanding → generation → verification" workflow is formed; furthermore, the same VLM is used to automatically verify the editing effect to ensure sample quality and test effectiveness. (3) Based on VLM, the solution of this invention uses VLM to subdivide overfitting problems into specific types such as environment-dependent overfitting, morphological memory overfitting, and feature confusion overfitting. It not only detects overfitting, but also diagnoses the specific causes and types of overfitting. At the same time, it generates targeted model improvement suggestions based on the deep understanding capabilities of VLM, forming a complete technical closed loop from "discovering the problem" to "solving the problem". Attached Figure Description

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0028] Figure 1 This is a schematic diagram illustrating the implementation principle of the enhanced robustness test for image classification based on an artificial intelligence model in this invention. Detailed Implementation

[0029] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0030] To address the problems faced by existing image classification model testing and enhancement techniques, this invention presents an image classification robustness testing and enhancement method based on an artificial intelligence model. This method can establish a complete "generation-testing-evaluation" closed-loop system, thereby enabling scientific quantitative evaluation of model robustness.

[0031] This image classification robustness testing enhancement method based on an artificial intelligence model mainly consists of the following steps in sequence: (1) Based on the multimodal artificial intelligence model, deep semantic analysis is performed on the image content, automatically identifying and extracting category-related and non-category-related features in the image, and sorting and weighting the features according to their importance to the classification task to generate a structured feature analysis report.

[0032] (2) Based on the intelligent analysis results, a complementary feature retention strategy (Keep Strategy) and a feature replacement strategy (Replace Strategy) are generated.

[0033] Among them, the feature preservation strategy Keep Strategy is set to focus on keeping the core category features in the image unchanged, while significantly modifying the non-core features; The feature replacement strategy is set to remove or replace core category features in the image while keeping non-core features relatively stable. The feature replacement strategy, on the other hand, takes the opposite approach, removing or replacing core category features in the image while keeping non-core features relatively stable.

[0034] The two strategies complement each other: they can comprehensively evaluate the model overfitting problem from two dimensions, namely "environmental robustness" and "feature understanding depth", thus enabling a comprehensive test of the model's robustness from different perspectives.

[0035] (3) Based on the feature preservation strategy Keep Strategy and feature replacement strategy generated in step (2), perform image editing operations to generate corresponding image samples.

[0036] This step supports various editing types, including object replacement, background modification, style change, material change, lighting adjustment, and perspective change.

[0037] Specifically, advanced image editing models (such as diffusion editing models, GANs, etc.) can be used to ensure the visual naturalness and physical plausibility of the edited images.

[0038] (4) Perform automated quality verification on the image samples edited in step (3), use artificial intelligence model to verify the classification results of the edited image samples, so that the samples generated based on the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are incorrectly classified, filtering out low-quality samples that do not meet expectations, and generating test samples to ensure the quality and reliability of test data.

[0039] (5) Based on the test samples generated in step (4), the target classification model is comprehensively evaluated, the performance indicators of the model on different types of test samples are calculated, and a detailed robustness analysis report is generated to identify the weak links and improvement directions of the model and provide specific guidance and suggestions for model optimization.

[0040] Regarding the above-mentioned method for enhancing the robustness of image classification based on artificial intelligence models, the implementation scheme of each step in the method is described in detail below.

[0041] In step (1) of this method, an intelligent analysis mechanism of AI model is introduced to use a multimodal artificial intelligence model to perform in-depth understanding and analysis of image content. Specifically, through multi-level feature extraction including pixel-level, object-level and scene-level feature analysis, the correlation between each feature and the target category is automatically evaluated, and precise guidance is provided for subsequent editing strategies.

[0042] Specifically, in step (1), this method sends carefully designed structured prompts to a multimodal large model (such as InternVL3). The prompt format is: "The image is classified as {label}. Please analyze in detail which visual elements in the image reflect the characteristics of {label}. Please sort them by importance...".

[0043] Here, the natural language understanding and generation capabilities of the Visual Language Model (VLM) are utilized to parse the output natural language description into structured data in JSON format, which includes fields such as feature name, importance score, and feature type (core / non-core).

[0044] Building on this, a multi-round dialogue mechanism is further adopted. When the initial analysis results are not detailed enough, follow-up questions are automatically generated for in-depth exploration.

[0045] Furthermore, a targeted feature importance scoring mechanism is established to automatically calculate the weight value of each feature based on the ranking and level of detail of the VLM output.

[0046] Based on this, a structured feature analysis report and multiple editing instructions for two strategies can be generated, providing precise guidance for subsequent editing strategies. The feature analysis report contains specific lists of core features and non-core features. This structured feature analysis report will serve as the basic data for generating test strategies, with the core feature list used to guide feature protection in the Keep strategy and feature replacement in the Replace strategy.

[0047] Step (2) of this method introduces two complementary dual testing strategies (i.e. editing strategies) based on the scheme in step (1): the feature retention strategy (Keep Strategy) and the feature replacement strategy (Replace Strategy).

[0048] Among them, the feature preservation strategy focuses on preserving the core recognition features of the category while significantly modifying non-core features (such as background environment, lighting conditions, visual style, etc.) to test whether the model relies too much on environmental information.

[0049] Specifically, in its implementation, the Keep Strategy sends an editing command to the VLM based on the non-core feature list generated in step (1), prompting: "Based on the {label} feature elements analyzed above, please generate {keep_num} image editing commands, requiring the [core feature list] to remain unchanged, and the [non-core feature list] to be significantly modified..."; This allows VLM to output editing instructions in JSON format, containing structured information such as: editing type (background_change / lighting_change, etc.), source area description, target effect description, and execution order.

[0050] The Replace Strategy, which is the opposite of the Keep Strategy, removes or replaces the core features of a category while keeping non-core features relatively stable, and tests whether the model truly understands the essential features of the category.

[0051] Specifically, in its implementation, the Replace Strategy constructs a knowledge base of obfuscated objects (such as a shape similarity mapping table for "pistol ↔ hair dryer") for core features, and automatically selects the best obfuscated replacement by combining the reasoning ability of VLM.

[0052] Based on this, the present invention further adopts a two-stage replacement strategy: first, VLM is used to generate a replacement scheme, and then the function of image editing models (such as diffusion editing models, GANs, etc.) is used to accurately replace the target region; by preserving the non-core regions of the original image, it is ensured that the replacement process only affects the core features.

[0053] As further explanation, each editing instruction generated in this step based on the feature preservation strategy (Keep Strategy) and the feature replacement strategy (Replace Strategy) includes an expected effect label (Keep Strategy expects the classification result to remain unchanged, while Replace Strategy expects the classification result to change). This label will serve as the verification benchmark for automated quality verification in step (4).

[0054] In the specific implementation of step (4) of this method, automated verification and quality control are accomplished by establishing an automated verification mechanism.

[0055] This automated verification mechanism ensures the quality of generated test samples through multi-dimensional quality assessment, classification result verification, and adaptive quality threshold setting, automatically filtering low-quality samples and retaining high-quality test data.

[0056] The multi-dimensional quality assessment here includes visual quality, semantic consistency, and physical plausibility assessment.

[0057] Meanwhile, single-model and multi-model cross-validation were used to verify the classification results.

[0058] As further explanation, in this method, when assessing visual quality, the same VLM model introduced in step (1) is used to compare the images before and after editing. The quality score is obtained by prompting the words "Image-1 is the original image, Image-2 is the edited image, please evaluate the visual naturalness of Image-2 (1-5 points)". A threshold (e.g., >3 points) is set to filter low-quality samples.

[0059] In this method, the same VLM model introduced in step (1) is used for classification verification during the verification of classification results to ensure the consistency of the verification criteria. For example, for the Keep strategy (i.e., feature preservation strategy) sample, the verification prompt is: "Can Image-2 still be classified as {label}? Return True / False and confidence level"; for the Replace strategy (i.e., feature replacement strategy) sample, the verification prompt is: "Can Image-2 no longer be classified as {label}? Return True / False and confidence level".

[0060] The adaptive threshold mechanism in this method dynamically adjusts the quality threshold based on the historical validation pass rate. When the pass rate is too low, the threshold is automatically lowered, and when it is too high, the threshold is raised.

[0061] As further explanation, in this step, samples that fail the verification based on the automated verification mechanism and their reasons for failure (such as "excessive editing has changed the core features") will be fed back to step (2) to optimize the generation strategy of the next round of editing instructions, forming a closed-loop optimization.

[0062] The image classification robustness testing enhancement scheme based on an artificial intelligence model presented in this example can be configured into a corresponding software program to form an image classification robustness testing enhancement system based on an artificial intelligence model. When this software program runs, it executes the aforementioned image classification robustness testing enhancement method based on an artificial intelligence model and stores it in a suitable storage medium for the processor to retrieve and execute.

[0063] See Figure 1 As shown, the image classification robustness testing enhancement system based on artificial intelligence model formed therefrom is mainly composed of intelligent analysis module 110, strategy generation module 120, image editing module 130, quality verification module 140 and robustness evaluation module 150 working together.

[0064] The intelligent analysis module 110 in this system serves as the core component, performing deep semantic analysis on the input images. This module is configured to utilize multimodal artificial intelligence models (such as vision-language models) to conduct comprehensive deep semantic analysis of the image content, automatically identifying and extracting category-related and non-category-related features from the image, and ranking and weighting the features according to their importance to the classification task.

[0065] The intelligent analysis module 110 also generates a structured feature analysis report and multiple editing instructions for two strategies based on the analysis structure, providing precise guidance for subsequent editing strategies.

[0066] The strategy generation module 120 in this system is configured to interact with the intelligent analysis module 110, and can generate two complementary test strategies based on the intelligent analysis results of the intelligent analysis module 110: the feature preservation strategy Keep Strategy and the feature replacement strategy Replace Strategy. The feature preservation strategy focuses on keeping the core class features in an image unchanged while making significant modifications to non-core features. The feature replacement strategy takes the opposite approach, removing or replacing core category features in the image while keeping non-core features relatively stable.

[0067] These two strategies can comprehensively evaluate the model overfitting problem from two dimensions: "environmental robustness" and "feature understanding depth," thus enabling a comprehensive test of the model's robustness from different perspectives.

[0068] The image editing module 130 in this system is configured to interact with the strategy generation module 120, and can perform image editing operations based on the feature preservation strategy (Keep Strategy) and feature replacement strategy (Replace Strategy) generated by the strategy generation module 120 to generate corresponding image samples.

[0069] This image editing module 130 specifically supports various editing types, including object replacement, background modification, style transformation, material alteration, lighting adjustment, and perspective transformation. In its implementation, advanced image editing models (such as diffusion editing models and GANs) can be used to ensure the visual naturalness and physical plausibility of the edited image.

[0070] The quality verification module 140 in this system is configured to interact with the image editing module 130, enabling automated quality verification of image samples edited by the image editing module 130 to ensure the validity of the generated samples.

[0071] This quality verification module 140 specifically uses an artificial intelligence model to verify the classification results of the edited image, ensuring that the samples generated by the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are misclassified; by filtering out low-quality samples that do not meet expectations, the generated test samples ensure the quality and reliability of the test data.

[0072] The robustness evaluation module 150 in this system is configured to interact with the quality verification module 140, enabling a comprehensive evaluation of the target classification model using the test samples generated by the quality verification module 140.

[0073] This robustness assessment module 150 not only calculates the model's performance metrics on different types of test samples, but also generates a detailed robustness analysis report, identifies the model's weaknesses and areas for improvement, and provides specific guidance and suggestions for model optimization.

[0074] The following details the specific implementation schemes of each functional module in this system.

[0075] The intelligent analysis module 110 in this system utilizes a multimodal artificial intelligence model to perform deep understanding and analysis of image content through an intelligent analysis mechanism that incorporates AI models. This mechanism automatically evaluates the relevance of each feature to the target category through multi-level feature extraction (including pixel-level, object-level, and scene-level feature analysis), providing precise guidance for subsequent editing strategies.

[0076] Specifically, this intelligent analysis module 110 sends carefully designed structured prompts to a multimodal large model (such as InternVL3). The prompt format is: "The image is classified as {label}. Please analyze in detail which visual elements in the image reflect the features of {label}. Please sort by importance...". Utilizing the natural language understanding and generation capabilities of VLM, the output natural language description is parsed into structured data in JSON format by the large language model, containing fields such as feature name, importance score, and feature type (core / non-core).

[0077] Based on this, a multi-round dialogue mechanism is adopted. When the initial analysis results are not detailed enough, follow-up questions are automatically generated for in-depth exploration.

[0078] Furthermore, a targeted feature importance scoring mechanism is established to automatically calculate the weight value of each feature based on the ranking and level of detail of the VLM output.

[0079] Based on this, the intelligent analysis module 110 generates a structured feature analysis report and multiple editing instructions for two strategies, providing precise guidance for subsequent editing strategies. The feature analysis report contains specific lists of core features and non-core features. This structured feature analysis report will serve as the basic data for generating test strategies, with the core feature list used to guide feature protection in the Keep strategy and feature replacement in the Replace strategy.

[0080] In its specific implementation, the strategy generation module 120 in this system focuses on preserving the core recognition features of the category for the generated feature preservation strategy, while significantly modifying non-core features (such as background environment, lighting conditions, visual style, etc.) to test whether the model relies too much on environmental information.

[0081] Furthermore, when implementing the feature preservation strategy generated by the strategy generation module 120, based on the non-core feature list output by the intelligent analysis module 110, it sends editing instructions to the VLM model introduced in the intelligent analysis module 110 to generate prompts: "Based on the {label} feature elements analyzed above, please generate {keep_num} image editing commands, requiring that the [core feature list] remain unchanged, and the [non-core feature list] be significantly modified..." This causes the VLM model to output editing instructions in JSON format, including: editing type (background_change / lighting_change, etc.), source region description, target effect description, execution order, and other structured information.

[0082] The feature replacement strategy generated by the strategy generation module 120 takes the opposite approach, removing or replacing the core features of the category while keeping the non-core features relatively stable, and testing whether the model truly understands the essential features of the category.

[0083] Furthermore, when implementing the feature replacement strategy generated by the strategy generation module 120, a knowledge base of obfuscated objects (such as a shape similarity mapping table of "pistol ↔ hair dryer") is constructed for the core features, and the best obfuscated replacement is automatically selected in combination with the reasoning ability of VLM.

[0084] Meanwhile, a two-stage replacement strategy is adopted: first, the VLM model introduced in the intelligent analysis module 110 is used to generate a replacement scheme, and then the diffusion editing model (i.e., the image editing module 130) is used to accurately replace the target area; in this way, by preserving the non-core areas of the original image, it is ensured that the replacement process only affects the core features.

[0085] Each editing instruction generated by the strategy generation module 120 based on the feature preservation strategy (Keep Strategy) and the feature replacement strategy (Replace Strategy) includes an expected effect label (Keep Strategy expects the classification result to remain unchanged, while Replace Strategy expects the classification result to change). This label is passed to the quality verification module 140 as a verification benchmark.

[0086] The quality verification module 140 in this system completes automated verification and quality control by establishing an automated verification mechanism.

[0087] The automated verification mechanism here ensures the quality of generated test samples through multi-dimensional quality assessment, classification result verification, and adaptive quality threshold setting, automatically filtering low-quality samples and retaining high-quality test data.

[0088] The multi-dimensional quality assessment here includes visual quality, semantic consistency, and physical plausibility assessment.

[0089] Meanwhile, single-model and multi-model cross-validation were used to verify the classification results.

[0090] Furthermore, when implementing visual quality assessment, this quality verification module 140 uses the same VLM model introduced in the intelligent analysis module 110 to compare the images before and after editing. It obtains a quality score by prompting the words "Image-1 is the original image, Image-2 is the edited image, please evaluate the visual naturalness of Image-2 (1-5 points)" and sets a threshold (e.g., >3 points) to filter low-quality samples.

[0091] Furthermore, when verifying the classification results, the quality verification module 140 uses the same VLM model introduced in the intelligent analysis module 110 to perform classification verification, ensuring the consistency of the verification standards. For example, for the Keep strategy sample, the verification prompt is: "Can Image-2 still be classified as {label}? Return True / False and confidence level"; for the Replace strategy sample, the verification prompt is: "Can Image-2 no longer be classified as {label}? Return True / False and confidence level".

[0092] Furthermore, the adaptive threshold mechanism in this quality verification module 140 dynamically adjusts the quality threshold based on the historical verification pass rate. When the pass rate is too low, the threshold is automatically lowered, and when it is too high, the threshold is raised.

[0093] Furthermore, the quality verification module 140 will feed back the samples that fail the verification based on the automated verification mechanism and the reasons for their failure (such as "excessive editing has changed the core features") to the strategy generation module 120 to optimize the generation strategy of the next round of editing instructions, thus forming a closed-loop optimization.

[0094] As can be seen from the above, the core innovation of the image classification robustness testing enhancement system based on the above scheme and artificial intelligence model lies not only in the implementation scheme of each functional module, but also in the organic collaborative system formed between each functional module.

[0095] This system features an innovative VLM-driven end-to-end workflow: the multimodal large model plays a core role in the analysis, strategy generation, and validation stages. Using the same model ensures a high degree of consistency in analysis standards, editing objectives, and validation benchmarks, avoiding the problem of inconsistent objectives caused by model differences in traditional multi-module systems.

[0096] In the specific implementation of this system, the corresponding data flow and information transmission are as follows: First, the intelligent analysis module → strategy generation module delivers a structured feature report (JSON format), which includes a feature list, importance weights, and spatial location information.

[0097] Next, the strategy generation module → verification module transmits editing instructions and expected effect tags.

[0098] Next, the verification module → strategy generation module provides feedback on the verification results and reasons for failure, forming a closed-loop optimization.

[0099] This system innovatively employs a complementary dual-strategy design. The Keep and Replace strategies test the model from two orthogonal dimensions: "environmental robustness" and "feature understanding depth." Both strategies utilize the results of intelligent analysis, but employ opposite editing logics: the Keep strategy protects the core, while the Replace strategy replaces the core. Through differentiated verification standards in the verification module, the Keep strategy expects to remain unchanged, while the Replace strategy expects to change, thus achieving a comprehensive diagnosis of model overfitting issues.

[0100] This system implements quality control throughout the entire process, from multi-round dialogues in the analysis phase to ensure analysis quality, to the injection of constraints in the strategy generation phase to ensure the rationality of editing, and then to multi-dimensional evaluation in the verification phase. The quality control mechanism runs through the entire process, and each link supports the other.

[0101] The following specific application examples further illustrate the application and implementation process of the image classification robustness testing enhancement scheme based on artificial intelligence models provided in this invention.

[0102] When implementing this AI-based image classification robustness testing enhancement scheme, an AI-based image classification robustness testing enhancement software system can be constructed based on the aforementioned scheme. This system can realize a complete end-to-end processing flow, starting from the original image input, through intelligent analysis, strategy generation, image editing, quality verification, and finally outputting a robustness evaluation report. The entire process is fully automated and requires no human intervention.

[0103] Combination Figure 1 As shown, this image classification robustness testing enhancement software system based on an artificial intelligence model automatically completes the process from image input to overfitting diagnostic report output through the following stages: (1) Intelligent image analysis and feature recognition stage.

[0104] The system first receives the original image and its corresponding category label as input. It then uses a multimodal large model (such as InternVL3) to perform deep semantic analysis on the image. The system interacts with the model using natural language, asking: "The image is classified as {label}. Please analyze in detail which visual elements in the image embody the features of {label}. Please rank them by importance and explain why each element embodies the features of {label}." Based on its powerful visual understanding capabilities, the multimodal large model can identify key visual elements in images, including features such as the shape, color, texture, and position of objects, as well as non-core features such as background environment and lighting conditions. The system sorts these features according to their importance to the target category, generating a structured feature analysis report that provides precise guidance for subsequent editing strategies.

[0105] (2) Intelligent generation stage of dual editing strategy.

[0106] Based on the feature analysis results of stage (1), the system generates two complementary editing strategies: the Keep strategy and the Replace strategy.

[0107] For the Keep strategy, the system will ask the multimodal large model: "Based on the {label} feature elements analyzed above, please generate {keep_num} image editing commands. The edited images must still be recognizable as {label} categories. The editing should produce significant visual changes, but retain the core recognition features." The Keep strategy is implemented through seven challenge strategies: extreme environment adaptation (placing objects in unusual but reasonable environments), perspective and scale challenge (using extreme perspective and scale changes), lighting and color extremes (extreme lighting and color challenges), occlusion and partial visibility (skillfully occluding while preserving key features), style and material transformation (changing surface features while maintaining shape and function), scene integration challenge (naturally integrating objects into complex multi-element scenes), and state and form changes (reasonable changes to the state of objects).

[0108] For the Replace strategy, the system will ask: "Based on the {label} feature elements analyzed above, please generate {unkeep_num} image editing commands. The edited image must not be recognizable as the {label} category. The editing should produce as few visual changes as possible, generating high-quality adversarial examples. The obfuscating replacement strategy should be prioritized." The Replace strategy includes three main methods: confusing object replacement (replacing the core element with an object that is similar in shape but different in nature, such as pistol → hair dryer, military uniform / police uniform → ordinary clothes), simulation replacement (replacing real objects with highly similar imitations, models, or pictures), and partial occlusion and suggestion (causing the observer to make a wrong association by occluding key parts).

[0109] (3) Intelligent image editing execution stage.

[0110] In this stage, the system converts the generated editing instructions into structured commands in JSON format, containing detailed information such as the editing type and operation description. These editing instructions are then executed using an advanced image editing model (such as the Step1X diffusion model).

[0111] The editing model can accurately perform various complex image editing operations based on natural language descriptions, including background replacement, object replacement, style transformation, and material change, ensuring the visual naturalness and physical plausibility of the edited image.

[0112] (4) Automated quality verification stage.

[0113] After the editing phase is completed, the system uses a multimodal large model to automatically validate the editing results. For samples generated by the Keep strategy, the system verifies whether they can still be correctly classified into the original category; for samples generated by the Replace strategy, the system verifies whether they have been misclassified (no longer recognized as the original category).

[0114] The verification process is implemented through natural language interaction: "Image-1 is the original image with category label {label}, and Image-2 is the edited image of Image-1. Please determine whether Image-2 can still be classified as {label}. Return True or False." The system only retains the samples that pass the verification and automatically filters out low-quality samples that do not meet the expected results.

[0115] (5) Model overfitting diagnosis stage based on dual strategies.

[0116] Based on the synergistic operation of the three key technical features mentioned above, the system is able to accurately diagnose model overfitting problems at this stage.

[0117] Specifically, the feature classification results identified by the intelligent analysis module, the test samples generated by the dual strategies, and the classification performance data obtained by automated verification together constitute the basis for overfitting diagnosis.

[0118] To further explain, the corresponding diagnostic process is carried out through the following steps: The system uses test samples generated by the Keep and Replace strategies to accurately diagnose overfitting issues in the target classification model. As a core technical aspect of this invention, this operation not only generates test samples but, more importantly, systematically identifies and classifies overfitting problems in the model.

[0119] For samples generated by the Keep strategy, the system accurately identifies the model's over-reliance on environmental factors by comparing the model's classification performance on the original samples and the edited samples. Specifically, if the model's classification accuracy drops significantly on samples where core features are maintained but non-core features such as background, lighting, viewpoint, and style are changed, it indicates that the model has an environmental overfitting problem.

[0120] The system uses a multimodal large model to intelligently analyze test results, not only statistically analyzing accuracy changes, but more importantly, understanding the specific reasons for accuracy declines. For example, when the model performs poorly on samples with varying backgrounds, the system can identify that the model is overly reliant on specific background features (such as indoor environments, specific colors, etc.) rather than truly understanding the core features of the target object.

[0121] For samples generated by the Replace strategy, the system evaluates the model's true understanding of the core features. When the model can still correctly classify samples where core features are replaced but non-core features remain unchanged, it indicates that the model has only memorized the surface morphology of the object, rather than truly understanding its semantic essence.

[0122] The system pays particular attention to test results for confusing substitution samples. For example, if the model still classifies a real handgun as a "weapon" when it is replaced with a hair dryer, it indicates a serious morphological overfitting problem, failing to distinguish the essential functional differences between the objects. This diagnostic capability is something traditional testing methods cannot provide.

[0123] (6) Intelligent overfitting type classification and improvement suggestions generation.

[0124] Based on the test results of the dual strategy, the system uses a multimodal large model for intelligent analysis, and subdivides the overfitting problem of the model into specific types: environment-dependent overfitting (over-reliance on environmental factors such as background and lighting), morphological memory overfitting (only remembering the surface morphology and ignoring the semantic essence), and feature confusion overfitting (unable to distinguish similar but essentially different objects).

[0125] In this way, the system not only identifies problems but also generates targeted improvement suggestions based on the deep understanding capabilities of VLM. For example, for models with environment-dependent overfitting, the system suggests increasing training data with environmental diversity; for models with morphological memory overfitting, the system suggests introducing more semantic-level training samples. This precise improvement suggestion based on specific problem types greatly improves the efficiency and effectiveness of model optimization.

[0126] As can be seen from the above, the present invention can form a verification closed loop and continuous optimization. The entire testing process forms a complete verification closed loop: VLM analysis → dual strategy generation → diffusion model editing → VLM verification → overfitting diagnosis → improvement suggestions. This closed loop can not only test the robustness of existing models, but more importantly, it provides a scientific basis and specific direction for model improvement, realizing a complete process from "identifying problems" to "solving problems".

[0127] This invention innovatively combines a multimodal large model, a dual testing strategy, and diffusion model editing to form a complete overfitting testing system. Although the specific technical implementation may vary, the core design concept and architectural approach have broad applicability and scalability.

[0128] Therefore, based on the innovative design concept and architectural ideas of the present invention, this paper further provides other feasible extensions or alternatives for some specific implementation schemes of the present invention.

[0129] For example, regarding the multimodal large model in this invention, the deep understanding capabilities of the multimodal large model are utilized for intelligent image analysis and editing strategy generation. In practical implementation, besides InternVL3, other multimodal models with similar capabilities can also be employed.

[0130] For example, while the CLIP model is mainly used for image-text matching, it can also achieve in-depth analysis of image content and assessment of feature importance through appropriate cue engineering and fine-tuning; large visual language models such as GPT-4V have stronger reasoning capabilities in image understanding and can be used for more complex semantic analysis and editing instruction generation.

[0131] Therefore, for the solution of this invention, the multimodal large model used needs to be able to work collaboratively in the three stages of analysis, generation and verification in order to support the complete "understanding → generation → verification" workflow.

[0132] Regarding the dual-strategy framework of this invention, the Keep and Replace strategies are the core innovations. In implementing the Keep strategy, in addition to generating editing instructions through natural language prompts, traditional computer vision techniques can be combined. For example, a semantic segmentation model can be used to automatically identify foreground and background regions in an image, and then the background can be replaced programmatically while protecting the foreground objects from modification. Although this method lacks the intelligent understanding capabilities of a Visual Learning Model (VLM), it ensures the accuracy and controllability of the editing process.

[0133] In implementing the Replace strategy, in addition to obfuscated replacement and simulation replacement, more diverse replacement methods can be introduced. For example, 3D modeling technology can be used to generate replacement objects under different viewpoints and lighting conditions, or neural style transfer technology can be used to change the visual style and material of objects while maintaining their basic shape and functional characteristics. Although these replacement methods have different technical paths, they all follow the basic design principle of "replacing core features and maintaining non-core features".

[0134] Regarding the image editing implementation scheme of this invention, although the Step1X diffusion model is used for image editing, any other feasible image generation and editing techniques can also be employed. For example, Generative Adversarial Network (GAN) models, such as StyleGAN and CycleGAN, have unique advantages in style transfer and image transformation, and are particularly suitable for implementing style transformation and material changes in the Keep strategy; Variational Autoencoder (VAE) models perform excellently in controllable image editing and can achieve more precise feature editing.

[0135] Furthermore, while traditional image processing methods lack the intelligence of deep learning, they still hold value in certain specific scenarios. For example, filter-based methods can quickly achieve basic adjustments such as color, brightness, and contrast; morphological manipulation-based methods can precisely control changes in shape and structure. These traditional methods can complement deep learning methods, playing a role in scenarios requiring rapid processing or precise control.

[0136] Regarding the verification mechanism in this invention, the same multimodal large model is used for automated verification of editing effects. While maintaining the core concept of "verification using the same model," verification models with different architectures can be used. For example, multiple classification models with different architectures can be used for integrated verification, and the reliability of verification can be improved through a voting mechanism; or a dedicated binary classification model can be trained specifically to determine whether the edited image meets the expected effect.

[0137] For certain specialized application scenarios, a human-machine collaborative verification mechanism can be introduced. For example, key samples can be manually sampled for verification, or crowdsourcing platforms can be used for large-scale manual verification. While this hybrid verification approach increases labor costs, it provides higher verification accuracy and reliability.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0140] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0148] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] The method, specific system unit, or part thereof of the present invention described above is a pure software architecture. It can be deployed via program code on physical media, such as hard disks, optical discs, or any electronic device (such as smartphones or computer-readable storage media). When a machine loads and executes the program code (e.g., a smartphone loads and executes it), the machine becomes a device for implementing the present invention. The method and device of the present invention can also be transmitted in program code form via transmission media, such as cables, optical fibers, or any other transmission method. When the program code is received, loaded, and executed by a machine (e.g., a smartphone), the machine becomes a device for implementing the present invention.

[0150] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for enhancing the robustness testing of image classification based on an artificial intelligence model, characterized in that, The method includes: (1) Based on the multimodal artificial intelligence model, perform deep semantic analysis on the image content, automatically identify and extract category-related features and non-category-related features in the image, and sort and weight the features according to their importance to the classification task to generate a structured feature analysis report; (2) Based on the intelligent analysis results, a complementary feature preservation strategy (Keep Strategy) and a feature replacement strategy (Replace Strategy) are generated. The feature preservation strategy (Keep Strategy) is set to focus on keeping the core category features in the image unchanged, while making significant modifications to the non-core features. The feature replacement strategy (Replace Strategy) is set to remove or replace the core category features in the image, while keeping the non-core features relatively stable. (3) Based on the feature preservation strategy Keep Strategy and feature replacement strategy ReplaceStrategy generated in step (2), perform image editing operations to generate corresponding image samples; (4) Perform automated quality verification on the image samples edited in step (3), use an artificial intelligence model to verify the classification results of the edited image samples, so that the samples generated based on the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are incorrectly classified, filtering out low-quality samples that do not meet expectations, and generating test samples. (5) Based on the test samples generated in step (4), the target classification model is comprehensively evaluated, the performance indicators of the model on different types of test samples are calculated, and a detailed robustness analysis report is generated to identify the weak links and improvement directions of the model.

2. The image classification robustness test enhancement method according to claim 1, characterized in that, Step (1) of the method involves performing deep semantic analysis on image content based on the Visual Language Model (VLM). Through multi-level feature extraction, including pixel-level, object-level, and scene-level feature analysis, the correlation between each feature and the target category is automatically evaluated.

3. The image classification robustness test enhancement method according to claim 2, characterized in that, The feature preservation strategy (Keep Strategy) generated in step (2) of the method sends an editing instruction to the VLM based on the list of non-core features in the structured feature analysis report generated in step (1), so that the VLM outputs the editing instruction in JSON format, which includes: editing type, source region description, target effect description, and execution order structured information; the feature replacement strategy (Replace Strategy) generated in step (2) constructs a knowledge base of obfuscated objects based on the core features in the core feature list in the structured feature analysis report generated in step (1), and automatically selects the best obfuscated replacement by integrating the reasoning ability of the VLM.

4. The image classification robustness test enhancement method according to claim 2, characterized in that, When performing automated quality verification in step (4) of the method, multi-dimensional quality assessment is performed using the same VLM and classification result verification is performed using the same VLM.

5. The image classification robustness test enhancement method according to claim 2, characterized in that, When performing robustness evaluation in step (5) of the method, the target classification model is diagnosed for overfitting type using test samples generated by the feature preservation strategy Keep Strategy and the feature replacement strategy Replace Strategy; based on the diagnosis results, VLM is used to subdivide the overfitting problem into specific types such as environment-dependent overfitting, morphological memory overfitting, and feature confusion overfitting.

6. A robustness testing enhancement system for image classification based on an artificial intelligence model, characterized in that, The system includes: The intelligent analysis module is configured to perform deep semantic analysis on image content based on a multimodal artificial intelligence model, automatically identify and extract category-related and non-category-related features in the image, and sort and weight the features according to their importance to the classification task to generate a structured feature analysis report. The strategy generation module is configured to interact with the intelligent analysis module and generate complementary feature preservation strategies (Keep Strategy and Replace Strategy) based on the intelligent analysis results of the intelligent analysis module. The feature preservation strategy (Keep Strategy) is configured to focus on keeping the core category features in the image unchanged while significantly modifying non-core features. The feature replacement strategy (Replace Strategy) is configured to remove or replace the core category features in the image while keeping the non-core features relatively stable. The image editing module is configured to interact with the strategy generation module and perform image editing operations based on the feature preservation strategy (Keep Strategy) and feature replacement strategy (Replace Strategy) generated by the strategy generation module to generate corresponding image samples. The quality verification module is configured to interact with the image editing module and can automatically verify the quality of the image samples edited by the image editing module. It uses an artificial intelligence model to verify the classification results of the edited image samples, so that the samples generated based on the feature preservation strategy are still correctly classified, while ensuring that the samples generated by the feature replacement strategy are misclassified, filtering out low-quality samples that do not meet expectations, and generating test samples. The robustness evaluation module is configured to interact with the quality verification module. It can comprehensively evaluate the target classification model based on the test samples generated by the quality verification module, calculate the model's performance indicators on different types of test samples, and generate a detailed robustness analysis report to identify the model's weaknesses and improvement directions.

7. The image classification robustness testing enhancement system according to claim 6, characterized in that, The intelligent analysis module uses the Visual Language Model (VLM) to perform deep semantic analysis on image content. Through multi-level feature extraction, including pixel-level, object-level, and scene-level feature analysis, it automatically evaluates the relevance of each feature to the target category.

8. The image classification robustness testing enhancement system according to claim 7, characterized in that, The strategy generation module generates a feature retention strategy (Keep Strategy) based on the list of non-core features in the structured feature analysis report. It sends editing instructions to the VLM to generate prompts, causing the VLM to output editing instructions in JSON format, including: editing type, source region description, target effect description, and structured information on execution order. The generated feature replacement strategy (Replace Strategy) is based on the core features in the core feature list in the structured feature analysis report. It constructs a knowledge base of obfuscated objects and automatically selects the best obfuscated replacement by integrating the VLM's reasoning ability.

9. The image classification robustness testing enhancement system according to claim 7, characterized in that, The quality verification module performs multi-dimensional quality assessment using the same VLM in the intelligent analysis module and verifies the classification results using the same VLM.

10. The image classification robustness testing enhancement system according to claim 7, characterized in that, The robustness evaluation module uses test samples generated by the feature preservation strategy (Keep Strategy) and the feature replacement strategy (Replace Strategy) to diagnose the overfitting type of the target classification model. Based on the diagnostic results, the VLM is used to further subdivide the overfitting problem into specific types such as environment-dependent overfitting, morphological memory overfitting, and feature confusion overfitting.

Images