An image generation method for neural network model testing

Abstract

The present invention relates to the field of model testing and discloses an image generation method for neural network model testing, comprising obtaining the prediction results and confidence distribution of a target neural network on a standard test image set, identifying the model's weaknesses in texture, structure, and illumination in combination with a sensitivity analysis algorithm, and constructing a model error-prone feature set; based on the error-prone feature set, performing semantically preserved directional perturbations on the original image through a feature parameter injection method; utilizing a perceptual similarity metric and a feature projection algorithm to evaluate the structural consistency between the generated image and the original image; employing an adaptive strategy to optimize generation rules, introducing a model feedback loop mechanism, and dynamically adjusting the perturbation parameter range based on the model's recognition error on the newly generated image; and automatically generating a model performance diagnostic analysis report in combination with the model output change curve and the weakness dimension labeling results. The present invention has the advantage of improving the accuracy of model testing.

Description

Technical Field

[0001] The present invention relates to the field of model testing, and in particular to an image generation method for neural network model testing. Background Art

[0002] With the rapid development of deep learning technology, neural network models have been widely used in computer vision tasks such as image classification, object detection, and image segmentation. Model training and testing often rely on large amounts of well-labeled image data. However, during the model testing phase, a specific set of test images is often required to assess the model's robustness and generalization ability to different input conditions. Existing methods often use manual selection, image augmentation, or GAN generation to construct test sets. However, in practical applications, existing image generation methods face a specific problem: it is difficult to control the generated images for targeted changes to address known weaknesses of the model. For example, when a model is sensitive to edge blur or brightness shifts in certain images, existing image generation tools often cannot precisely control the generation of these features, resulting in insufficient representation of the generated images and test results that fail to truly reflect the model's performance in edge scenarios. Therefore, it is necessary to design an image generation method for neural network model testing that improves the accuracy of model testing. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides an image generation method for neural network model testing, which has the advantage of improving the accuracy of model testing and solves the problems in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose of improving the accuracy of model testing, the present invention provides the following technical solution: an image generation method for neural network model testing, comprising the following steps:

[0005] Obtain the prediction results and confidence distribution of the target neural network on a standard test image set, combine it with a sensitivity analysis algorithm to identify the model's weaknesses in texture, structure, and lighting, and construct a set of model-prone features;

[0006] Based on the error-prone feature set, the original image is perturbed in a semantically preserved directional manner by injecting feature parameters, and a candidate test image set with target feature offset is generated by combining fuzzy control.

[0007] Using perceptual similarity measurement and feature projection algorithm, the relative position of the generated image and the original image in space is analyzed. By calculating the similarity after projection, the structural consistency is judged using Euclidean distance. The difference between the generated image and the original image in the model output is calculated. A response threshold is set, and images with output changes exceeding the response threshold are marked as high-response images. Combined with the structural consistency judgment results, only image samples with consistent structure but significant response are retained as the final valid test samples;

[0008] Adopting an adaptive strategy to optimize the generation rules, introducing a model feedback loop mechanism, and dynamically adjusting the perturbation parameter range according to the recognition error of the model on the newly generated image;

[0009] Combining the model output change curve and weakness dimension labeling results, a model performance diagnosis and analysis report is automatically generated.

[0010] Preferably, the process of constructing the model error-prone feature set is:

[0011] Perform input gradient analysis on the prediction results of the neural network model on a standard test image set, calculate the gradient sensitivity distribution of each input image in different regions, and extract the model's focus areas through visualization technology;

[0012] Divide image features into three main dimensions: texture, structure and illumination;

[0013] Apply controlled perturbations to the test image, record the changes in the model's prediction results, and quantify the model's output fluctuations under different feature perturbations; evaluate the consistency of the model's response in local areas before and after the perturbation, and identify areas with unstable responses; perform cluster analysis on the collected perturbation sensitivity indicators; extract the corresponding texture types, structural patterns, and lighting conditions; identify typical weak areas and feature parameters, and construct a model's error-prone feature set.

[0014] Preferably, the process of generating a candidate test image set with target feature offset in combination with fuzzy control is as follows:

[0015] Construct disturbance control parameters based on the model's error-prone feature set;

[0016] Construct a fuzzy control system with disturbance effect feedback as input and disturbance parameter adjustment as output;

[0017] For each disturbance process, the adjustment amplitude of the disturbance parameters is calculated through the fuzzy inference mechanism according to the model prediction confidence drop, structural similarity, and feature shift degree of the current disturbance image, and the disturbance control parameters are updated;

[0018] Under the guidance of the updated parameters, a perturbation operation is performed on the original image to introduce error-prone features without destroying the main semantic structure, thereby generating image samples with target feature offsets.

[0019] Preferably, the process of evaluating the structural consistency between the generated image and the original image using the perceptual similarity metric and the feature projection algorithm is as follows:

[0020] The original image and the perturbed image are fed into the same pre-trained neural network to extract multi-scale semantic features of the intermediate layer.

[0021] Use perceptual similarity metrics to compare the feature differences between the original image and the perturbed image;

[0022] The feature vectors of the original image and the perturbed image are projected into the same semantic feature space, and their relative positions in the space are analyzed. The structural consistency is judged by calculating the similarity after projection and using the Euclidean distance.

[0023] Preferably, the process of determining whether the structure is consistent is:

[0024] If the Euclidean distance between the original image and the perturbation image in the semantic space is less than or equal to the set distance threshold, the structures are determined to be consistent;

[0025] If the Euclidean distance between the original image and the perturbation image in the semantic space is greater than the set distance threshold, the structures are determined to be inconsistent.

[0026] Preferably, the process of dynamically adjusting the perturbation parameter range according to the recognition error of the model on the newly generated image is:

[0027] Generate a preliminary set of perturbed images based on the error-prone feature set identified in the early stage and the set perturbation range;

[0028] Input the initially generated perturbed image set into the target neural network to obtain the prediction result and corresponding confidence level of each image, and obtain the model output result and confidence level;

[0029] The prediction error is calculated by comparing the model's prediction results with the actual labels.

[0030] Preferably, the process of automatically generating a model performance diagnosis and analysis report is as follows:

[0031] Based on the model output data, draw the model output change curve to show the output change trend on different perturbation images;

[0032] Based on the error-prone feature set and feature parameter perturbations, the potential weak dimensions of the model are marked;

[0033] The model outputs the change curve and the annotation results of the weakness dimension to automatically generate a diagnostic analysis report.

[0034] Preferably, the screening process of high response images is:

[0035] Calculate the difference between the generated image and the original image in the model output results;

[0036] Set a response threshold and mark images whose output changes exceed the threshold as high response images;

[0037] Combined with the structural consistency judgment results, only image samples with consistent structure but significant response are retained as the final valid test samples.

[0038] Compared with the prior art, the present invention provides an image generation method for neural network model testing, which has the following beneficial effects:

[0039] The neural network model test image generation method provided by this invention can accurately identify the model's error-prone areas in feature dimensions such as texture, structure, and illumination. It generates highly targeted test samples through semantically preserved directional perturbation and fuzzy control. It combines perceptual similarity and feature projection algorithms to ensure the structural consistency of the perturbed images. It then selects highly responsive images based on output differences, significantly improving the effectiveness and diversity of test samples. By introducing model feedback and adaptive adjustment mechanisms, it dynamically optimizes perturbation strategies, enables automatic discovery of model weaknesses and performance analysis, and enhances the systematicity, intelligence, and practical value of model testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] Example 1: Please refer to Figure 1 As shown, an image generation method for neural network model testing according to an embodiment of the present invention includes the following steps:

[0043] S1: Obtain the prediction results and confidence distribution of the target neural network on the standard test image set, combine the sensitivity analysis algorithm to identify the model's weaknesses in texture, structure, and lighting, and construct the model's error-prone feature set.

[0044] In S1, the sensitivity analysis algorithm is combined to identify the model's weaknesses in texture, structure, and illumination, and the process of constructing the model's error-prone feature set is as follows:

[0045] Perform input gradient analysis on the prediction results of the neural network model on a standard test image set, calculate the gradient sensitivity distribution of each input image in different regions, and extract the model's focus areas through visualization technology;

[0046] Divide image features into three main dimensions: texture, structure and illumination:

[0047] Texture: Analyzes responses in high-frequency areas, such as dense edges, repetitive patterns, and noise-sensitive areas;

[0048] Structure: Analyze the model's response to fluctuations in image geometry;

[0049] Lighting: Testing the model's stability in areas sensitive to lighting conditions such as brightness changes, shadows, and reflections;

[0050] Apply controlled perturbations to the test image, record the changes in the model's prediction results, and quantify the model's output fluctuations under different feature perturbations; evaluate the consistency of the model's response in local areas before and after the perturbation, and use indicators such as structural similarity, perceptual hashing, and local feature point changes to compare the processing differences between the perturbation image and the original image in the same area to identify areas with unstable responses; perform cluster analysis on the collected perturbation sensitivity indicators to divide image regions or samples with common sensitivity in specific feature dimensions; extract feature parameters such as corresponding texture types, structural patterns, and lighting conditions; identify typical weak areas and their feature parameters, and construct a model's error-prone feature set, which describes the model's sensitivity to specific image feature combinations.

[0051] By obtaining the prediction results and confidence distribution of the target neural network on a standard test image set, combined with a sensitivity analysis algorithm, we perform local perturbation and response evaluation on key perceptual dimensions such as texture, structure, and illumination in the image, identifying the model's performance weaknesses in specific visual features and constructing a feature set containing typical error-prone patterns. This can accurately explore the model's instability in response to complex visual changes, effectively expose potential recognition risks, and provide a basis for the targeted generation of subsequent test images and model performance optimization.

[0052] S2: Based on the error-prone feature set, the original image is subjected to semantically preserved directional perturbations through the feature parameter injection method, and a candidate test image set with target feature offset is generated in combination with fuzzy control.

[0053] The process of generating a candidate test image set with target feature offset in S2 by combining fuzzy control is as follows:

[0054] Based on the model's error-prone feature set, construct perturbation control parameters, such as noise intensity for texture perturbation, rotation angle for structural perturbation, and brightness variation for illumination perturbation. Define an initial perturbation parameter set for each image type and design a corresponding perturbation method to ensure that the perturbation target is clear and the direction is controllable.

[0055] Construct a fuzzy control system that uses disturbance effect feedback as input and disturbance parameter adjustment as output. Develop fuzzy rules based on expert experience or training data. If the model output changes little after the disturbance and the image similarity is high, then appropriately increase the disturbance intensity to achieve dynamic regulation of the disturbance process.

[0056] For each perturbation process, the adjustment amplitude of the perturbation parameters is calculated through the fuzzy inference mechanism based on the input indicators such as the model prediction confidence drop of the current perturbation image, structural similarity, and feature offset degree, and the perturbation control parameters are updated to achieve iterative optimization of the perturbation.

[0057] Under the guidance of the updated parameters, a perturbation operation is performed on the original image to introduce error-prone features without destroying the main semantic structure, thereby generating image samples with target feature offsets.

[0058] By injecting feature parameters based on the model's error-prone feature set, the original image is subjected to semantically preserved, directional perturbations. Without destroying the image's overall semantic structure, these perturbations are targeted to introduce error-prone features in dimensions such as texture, structure, or illumination. A fuzzy control mechanism is then used to dynamically adjust the perturbation intensity and direction, enabling refined adjustment of the perturbation parameters and generating a diverse and controllable set of candidate test images. This image set maintains perceptual consistency while effectively stimulating the model's prediction error on sensitive features, thereby enhancing the coverage and diagnostic capabilities of the test samples and improving the accuracy and efficiency of robustness testing and performance analysis of neural network models.

[0059] S3: Use perceptual similarity measurement and feature projection algorithm to analyze the relative position of the generated image and the original image in space. By calculating the similarity after projection, use Euclidean distance to judge the structural consistency, calculate the difference between the generated image and the original image in the model output results, set the response threshold, and mark the images whose output changes exceed the response threshold as high-response images. Combined with the structural consistency judgment results, only image samples with consistent structure but significant response are retained as the final valid test samples.

[0060] The process of using perceptual similarity measurement and feature projection algorithm to evaluate the structural consistency between the generated image and the original image in S3 is as follows:

[0061] The original image and the perturbed image are fed into the same pre-trained neural network to extract multi-scale semantic features from the middle layer. These features represent the structural information of the image in terms of texture, shape, and semantics, and can be used to measure the consistency of the two images at the perceptual level.

[0062] Use perceptual similarity metrics to compare feature differences between the original image and the perturbed image. This metric can capture structural changes that are sensitive to human vision and assess whether the perturbation has an impact on the model while preserving the image's semantics.

[0063] The feature vectors of the original image and the perturbation image are projected into the same semantic feature space, and their relative positions in the space are analyzed. The structural stability is judged by calculating the similarity after projection and using the Euclidean distance. Let the original image be x and the perturbation image be , input the two into the pre-trained neural network to extract the high-level feature representation vector; the Euclidean distance formula between the two is:

[0064] ;

[0065] Where, It is the semantic feature vector extracted from the original image through the neural network; is the semantic feature vector extracted from the perturbation graph through a neural network; d is the dimension of the feature vector; is the Euclidean distance between the original image and the perturbation image in the semantic space;

[0066] The process of judging whether the structure is consistent is:

[0067] If the Euclidean distance between the original image and the perturbation image in the semantic space is less than or equal to the set distance threshold, the structures are determined to be consistent;

[0068] If the Euclidean distance between the original image and the perturbation image in the semantic space is greater than the set distance threshold, the structures are determined to be inconsistent.

[0069] The technical solution of this embodiment is: to use perceptual similarity measurement and feature projection algorithm to comprehensively evaluate the differences between the generated image and the original image at the structural and semantic levels. In the specific process, the perceptual similarity measurement is first used to measure the fidelity of the image in structural information, and then the high-dimensional image features are mapped to the low-dimensional space through the feature projection algorithm, so as to visualize and analyze its consistency in the feature space. At the same time, the model output difference is introduced as a response indicator, and combined with the above evaluation results, high-response images that have a significant impact on the model are screened out as effective test samples. Through the joint strategy of structural consistency evaluation and model response difference measurement, representative test samples can be effectively screened out, highlighting the sensitivity of the model to changes in key features, thereby improving the coverage and discrimination of the test data set. It not only improves the robustness of image generation and verification, but also provides accurate and efficient technical support for the interpretability analysis of model performance.

[0070] Example 2: Figure 1 As shown, an image generation method for neural network model testing further includes the following steps:

[0071] S4: Adopt an adaptive strategy to optimize the generation rules and introduce a model feedback loop mechanism to dynamically adjust the perturbation parameter range based on the recognition error of the model on the newly generated image.

[0072] The purpose of introducing the model feedback loop mechanism is to dynamically optimize the perturbation generation process. Based on the feedback information of the model's performance on the newly generated images, the parameter range of the perturbation is adjusted in real time, so that the generated perturbation images can more effectively test the weaknesses of the model. By adaptively adjusting the perturbation rules, the quality of the test image set can be improved to ensure that it can cover the potential weak areas of the model.

[0073] The process of dynamically adjusting the perturbation parameter range in S4 according to the recognition error of the model on the newly generated image is:

[0074] Generate a preliminary set of perturbed images based on the error-prone feature set identified in the early stage and the set perturbation range;

[0075] Input the initially generated perturbed image set into the target neural network to obtain the prediction result and corresponding confidence level of each image, and obtain the model output result and confidence level;

[0076] Compare the model's prediction results with the actual labels and calculate the prediction error. The error metric can be the classification error rate or the regression error.

[0077] The prediction error of each perturbed image is used as feedback information to analyze the model's recognition error. If the perturbed image causes the model to make a large prediction error, it means that the perturbation has a significant impact on the model, exposing the model's weaknesses.

[0078] Adjust the perturbation parameter range based on the error information fed back by the model. If the perturbation image changes the model's prediction, the perturbation is effective, and the parameter range for that type of perturbation should be increased. Conversely, if the perturbation fails to affect the model output, the perturbation is ineffective. The adjustment method is as follows: For perturbations with large model errors, increase the perturbation parameter range to make the perturbation more significant; for perturbations with small errors or no change, reduce the perturbation range to reduce unnecessary perturbations.

[0079] By introducing a model feedback loop mechanism, the perturbation generation process dynamically adjusts the perturbation parameter range based on the model's error performance on each round of perturbed images. This adaptive optimization strategy ensures that the generated test image set effectively covers the model's error-prone areas while avoiding unnecessary perturbations, thereby improving the model's robustness testing capabilities. This approach can more accurately reveal model weaknesses in specific feature dimensions, further enhancing the effectiveness and pertinence of model performance diagnostics.

[0080] S5: Combine the model output change curve and the weakness dimension annotation results to automatically generate a model performance diagnosis and analysis report.

[0081] The process of automatically generating the model performance diagnosis and analysis report in S5 is as follows:

[0082] Based on the model output data, draw a model output change curve to show the output change trend on different perturbation images. Specifically, you can compare the impact of each perturbation feature on the model output at different intensities. For example, draw a category change curve, a confidence change curve, or an error change curve. The horizontal axis of the curve represents the perturbation intensity, and the vertical axis represents the output change. The curve can reveal which features have an impact on the model output, as well as the relationship between perturbation intensity and model performance.

[0083] Based on the error-prone feature set and feature parameter perturbations, the potential weak dimensions of the model are marked. Weak dimensions can be features related to image texture, structure, lighting, etc. By analyzing the performance of the model under these features, determine which features are susceptible to perturbations. For example, the model may have large output differences under the perturbation of lighting changes, but react less to texture perturbations. By marking the relationship between each perturbation type and model performance, it is indicated which features are weak dimensions. For example, if the model is unstable to lighting changes, it can be marked as a lighting weakness and relevant analysis can be given;

[0084] The model outputs the change curve and the annotation results of the weakness dimension to automatically generate a diagnostic analysis report. The report content includes the following aspects:

[0085] Model performance under different perturbations: Analyze the output changes of the model on various perturbation images and draw the change curve.

[0086] Key Weakness Feature Analysis: Lists the model's performance weaknesses in different feature dimensions, indicating which feature disturbances have a greater impact on the model.

[0087] Performance evaluation: Based on the trend of model output changes, the model's robustness score under different disturbance intensities is given, as well as possible optimization directions.

[0088] Suggestions and improvement measures: Based on the diagnosis results, make targeted optimization suggestions, such as increasing the training samples of certain features, enhancing the model's robustness to specific disturbances, etc.

[0089] By combining the model output change curve with the weak point dimension annotation results, the automatically generated performance diagnostic analysis report can clearly and detailedly summarize the model's robustness performance, helping developers quickly identify potential model weaknesses and providing data support for subsequent optimization. The automated report generation process improves the efficiency and accuracy of model performance evaluation and provides an important basis for continuous optimization and improvement of the model.

[0090] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0091] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An image generation method for neural network model testing, characterized in that: The following steps are involved: Obtain the prediction results and confidence distribution of the target neural network on a standard test image set, combine it with a sensitivity analysis algorithm to identify the model's weaknesses in texture, structure, and lighting, and construct a set of model-prone features; Based on the error-prone feature set, the original image is perturbed in a semantically preserved directional manner by injecting feature parameters, and a candidate test image set with target feature offset is generated by combining fuzzy control. Using perceptual similarity measurement and feature projection algorithm, the relative position of the generated image and the original image in space is analyzed. By calculating the similarity after projection, the structural consistency is judged using Euclidean distance. The difference between the generated image and the original image in the model output is calculated. A response threshold is set, and images with output changes exceeding the response threshold are marked as high-response images. Combined with the structural consistency judgment results, only image samples with consistent structure but significant response are retained as the final valid test samples; Adopting an adaptive strategy to optimize the generation rules, introducing a model feedback loop mechanism, and dynamically adjusting the perturbation parameter range according to the recognition error of the model on the newly generated image; Combining the model output change curve and weakness dimension labeling results, a model performance diagnosis and analysis report is automatically generated.

2. The image generation method for neural network model testing according to claim 1, characterized in that: The process of building the model's error-prone feature set is: Perform input gradient analysis on the prediction results of the neural network model on a standard test image set, calculate the gradient sensitivity distribution of each input image in different regions, and extract the model's focus area through visualization technology; Divide image features into three main dimensions: texture, structure and illumination; Apply controlled perturbations to the test image, record the changes in the model's prediction results, and quantify the model's output fluctuations under different feature perturbations; evaluate the consistency of the model's response in local areas before and after the perturbation, and identify areas with unstable responses; perform cluster analysis on the collected perturbation sensitivity indicators; extract the corresponding texture types, structural patterns, and lighting conditions; identify typical weak areas and feature parameters, and construct a model's error-prone feature set.

3. The image generation method for neural network model testing according to claim 1, characterized in that: The process of generating a candidate test image set with target feature offset by combining fuzzy control is as follows: Construct disturbance control parameters based on the model's error-prone feature set; Construct a fuzzy control system with disturbance effect feedback as input and disturbance parameter adjustment as output; For each disturbance process, the adjustment amplitude of the disturbance parameters is calculated through the fuzzy inference mechanism according to the model prediction confidence drop, structural similarity, and feature shift degree of the current disturbance image, and the disturbance control parameters are updated; Under the guidance of the updated parameters, a perturbation operation is performed on the original image to introduce error-prone features without destroying the main semantic structure, thereby generating image samples with target feature offsets.

4. The image generation method for neural network model testing according to claim 1, characterized in that: The process of evaluating the structural consistency between the generated image and the original image using perceptual similarity measurement and feature projection algorithm is as follows: The original image and the perturbed image are input into the same pre-trained neural network to extract the multi-scale semantic features of the intermediate layer; Use perceptual similarity metrics to compare the feature differences between the original image and the perturbed image; The feature vectors of the original image and the perturbed image are projected into the same semantic feature space, and their relative positions in the space are analyzed. The structural stability is judged by calculating the similarity after projection using the Euclidean distance.

5. The image generation method for neural network model testing according to claim 1, characterized in that: The process of judging whether the structure is consistent is: If the Euclidean distance between the original image and the perturbation image in the semantic space is less than or equal to the set distance threshold, the structures are determined to be consistent; If the Euclidean distance between the original image and the perturbation image in the semantic space is greater than the set distance threshold, the structures are determined to be inconsistent.

6. The image generation method for neural network model testing according to claim 1, characterized in that: The process of dynamically adjusting the perturbation parameter range according to the recognition error of the model on the newly generated image is as follows: Generate a preliminary set of perturbed images based on the error-prone feature set identified in the early stage and the set perturbation range; Input the initially generated perturbed image set into the target neural network to obtain the prediction result and corresponding confidence level of each image, and obtain the model output result and confidence level; The prediction error is calculated by comparing the model's prediction results with the actual labels.

7. The image generation method for neural network model testing according to claim 1, characterized in that: The process of automatically generating a model performance diagnostic analysis report is as follows: Based on the model output data, draw the model output change curve to show the output change trend on different perturbation images; Based on the error-prone feature set and feature parameter perturbations, the potential weak dimensions of the model are marked; The model outputs the change curve and the annotation results of the weakness dimension to automatically generate a diagnostic analysis report.

8. The image generation method for neural network model testing according to claim 1, characterized in that: The screening process for high response images is: Calculate the difference between the generated image and the original image in the model output results; Set a response threshold and mark images whose output changes exceed the threshold as high response images; Combined with the structural consistency judgment results, only image samples with consistent structure but significant response are retained as the final valid test samples.

Images