Multi-view rectification model interactive analysis method based on integral gradient method

Abstract

The multi-view correction model interactive analysis method based on integral gradient method includes: S1. Extracting features of the input image in the model based on integral gradient method and quantifying the model features of the input image; S2. Optimizing feature points by using the SLIC algorithm to perform superpixel segmentation on the input image and then optimizing feature points; S3. Designing a multi-view visual page to make the ML black-box model interpretable; S4. Optimizing the model based on the multi-view visual page: analyzing the model based on the multi-view visual page, identifying the causes of model classification errors, and optimizing the model. This invention can make neural network models interpretable from a black-box state, enabling the model's input and output results to form a reasonable logical relationship, and making the internal structure of the model more transparent.

Description

Technical Field

[0001] This invention relates to an interpretable model interaction analysis method for machine learning. Background Technology

[0002] Machine learning (ML) is increasingly being used in various social systems, such as the internet and transportation, and these critical systems directly impact people's lives in many ways. Current ML models can achieve high accuracy and perform well on target tasks, sometimes even outperforming humans. This is because ML models can extract the relationship between input features and response output from multiple dimensions of the target task. Their structures are often very complex, and therefore lack logical relationships between input and output, resulting in a lack of interpretability. In other words, the entire process of the model is a "black box," making the results seem unreliable. This is precisely why these complex ML models have not been fully applied in fields such as medicine, defense, law, and finance. In black-box models, the decision-making mechanism of the entire process is opaque, and the intermediate processes are unknown. Interpretable ML models can improve the model's credibility and improve it by understanding the entire decision-making logic, the impact of each feature on the result, and by "treating the model according to its specific needs," rather than relying solely on past experience or even arbitrary methods for parameter tuning as in the past. Summary of the Invention

[0003] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide an interactive optimization method for interpretable models based on SHAP.

[0004] To clarify the logical relationship between the model's input and output for users, making the model interpretable and increasing the credibility of the output results, users can interactively explore and correct the model based on a series of intermediate analysis results. This invention provides a visual analysis method for exploring the model's classification process through multi-view linkage. Visualization components are designed using echarts and d3.js, and the data foundation is calculated using the integral gradient attribution method. First, the feature values ​​of each layer of the VGG11 model are overlaid with the original image (the original image has a certain degree of transparency). Users can roughly understand the distribution of feature values ​​by using different regions generated after segmentation of the original image. Based on their own exploration needs, combined with different trend graphs, users can interactively explore the state of feature value changes within the model, continuously enhancing the magnitude of feature values ​​during the classification process. The model output relationship graph presents the entire process from image input to output classification results, accurately determining the reasons for the model's image recognition errors and continuously providing feedback on the areas to be adjusted (the corresponding feature regions of the image in the model), making the entire classification process interpretable and providing a credible basis for the classification results output. An interactive analysis method is designed to correct the model's erroneous recognition behavior.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A multi-view correction model interactive analysis method based on integral gradient method, the method comprising the following steps:

[0007] S1. Extracting features of the input image in the model based on the integral gradient method: using gradient interpretation to quantify the model features of the input image;

[0008] S11. Use the ImageNet dataset as the background data sample for the VGG model, and use the gradient interpretation algorithm to generate an interpreter to interpret the ML model;

[0009] S12. Input the image into the interpreter for calculation to obtain the feature points of the input image in the model.

[0010] S2. Feature point optimization: The SLIC algorithm is used to perform superpixel segmentation on the input image to optimize the feature points;

[0011] S21. Use the SLIC algorithm to segment the input image into multiple feature regions with similar features.

[0012] S22. Take the set of all feature points in the feature region and form a feature matrix M, which represents the feature points of the region;

[0013] S3. Multi-view visual page design: The design of multi-view visual pages makes the ML black-box model interpretable;

[0014] S31. Display the original image, feature map, and SLIC segmentation map together. You can also select different levels of the model to display features and observe the features generated by the input image in the model and their distribution in the superpixel segmentation map.

[0015] S32. The changes of the feature points of interest in the model's conv and pooling layers are displayed, reflecting the model's ability to recognize the feature points;

[0016] S33. A relationship diagram is used to centrally display the degree of correlation between the model output and feature points, which is used to explain the degree of correlation between the model output and feature points;

[0017] S34. This shows the distribution of the model's output results for the same class of datasets before and after model modification, allowing for real-time monitoring of model performance and providing valuable reference for model optimization.

[0018] S35. Displays a probability histogram of the model's current output, providing direction for model optimization;

[0019] S4. Optimize the model based on multi-view visual pages: Analyze the model based on multi-view visual pages, identify the causes of model classification errors, and optimize the model;

[0020] S41. First, observe the distribution of the model's true label feature points in the input image, and select the feature points that need to be studied.

[0021] S42. The changes in the selected feature points at different levels of the model reflect at which level the model's ability to recognize that feature point was incorrect.

[0022] S43. Enhance the model's ability to identify the corresponding feature point at the corresponding level;

[0023] S44. Optimize the model while ensuring its performance, based on the correlation between the model output and each feature point, the probability distribution of the current model output, and the model's recognition of the current dataset.

[0024] The gradient interpretation method in step S1 refers to the algorithm generated by combining the ideas of integral gradient method, smooth gradient method and SHAP method.

[0025] In step S2, the feature point points to the feature region obtained by the SLIC segmentation algorithm, and the entire region is regarded as a single point.

[0026] This invention employs a neural network attribution method using integral gradient descent to calculate feature point maps of images at various levels within the model. By leveraging image clustering algorithms, an image is segmented into multiple regions based on pixel similarity. Combining these two methods allows for the partitioning of a large number of feature points, enhancing the subsequent analytical capabilities of the entire interpretable model. Based on the feature values ​​corresponding to each feature point, each region is explored. Line graphs are used to illustrate the changes in feature values ​​before and after each convolutional layer in the model, as well as the changes in feature values ​​throughout the entire model process, to analyze the model's classification process. The feature values ​​of each region are adjusted, and the changes in the model's classification process before and after adjustment are monitored in real-time using the Sankey diagram and classification results of the interpretable model. This reflects the changes in feature values ​​at each level of the model, improving the model's ability to identify high-feature regions and enhancing its overall image recognition capabilities, thus making the model's image classification process interpretable.

[0027] The beneficial effects of this invention are: it transforms neural network models from a black-box state into interpretable ones, enabling a reasonable logical relationship between the model's input and output, and making the model's internal structure more transparent. By analyzing the distribution and changing trends of feature points in each level of the model, the invention assesses the model's ability to recognize feature points during classification, allowing users to manually select the model's level and enhancement level to adjust the model, and optimize the model based on evidence. Attached Figure Description

[0028] Figure 1 This is the image feature point display area of ​​the present invention.

[0029] Figure 2a and Figure 2b This is the feature point change trend display area of ​​the present invention, wherein Figure 2a It represents the trend of feature points changing within the Pool layer of the model. Figure 2b It represents the trend of feature points changing in the ReLU layer of the model.

[0030] Figure 3 This refers to the correlation between the model output results and feature points in this invention.

[0031] Figure 4 This is the display area for the model output results of this invention.

[0032] Figure 5 This area showcases the model's ability to recognize datasets.

[0033] Figure 6 This is the model parameter adjustment area of ​​the present invention.

[0034] Figure 7 This is an illustration of the interactive analysis method of the present invention. Detailed implementation method:

[0035] The invention will now be further described with reference to the accompanying drawings.

[0036] Example 1

[0037] Reference Figures 1 to 7 This invention presents a multi-view correction model interactive analysis method based on integral gradient method. The invention uses the React framework to draw front-end visualization components, ECharts+D3 to draw visualization charts, and Python combined with the Flask framework for data processing.

[0038] The present invention provides an interactive analysis method for multi-view correction models based on the integral gradient method, comprising the following steps:

[0039] S1. Extracting Input Image Features from the Model Based on Integral Gradient Method: The gradient interpretation method was proposed in the paper "A Unified Approach to Interpreting Model Predictions" published by Scott M. Lundberg and Su-InLee at NIPS 2017. This invention uses the gradient interpretation method to quantify the model features of the input image;

[0040] S2. Feature Point Optimization: The SLIC algorithm, proposed in PAMI 2012. This invention uses the SLIC algorithm to perform superpixel segmentation on the input image and then optimize feature points;

[0041] S3. Multi-view visual page design: The design of multi-view visual pages makes the ML black-box model interpretable;

[0042] S31. For example Figure 1 As shown, the original image, feature map, and SLIC segmentation map are displayed together. To clearly map the feature points in each region, multiple images can be selected simultaneously. Different levels of the model can also be selected to display features and observe the features generated by the input image in the model and their distribution in the superpixel segmentation map. By stacking the feature point map and the SLIC segmentation map, it can be seen that the information of the feature points (densely distributed in red) needs to be focused on.

[0043] S32. As shown in Figure 2, select Figure 1 The model identifies the feature points in the model and shows how these feature points change in the model's conv and pooling layers, reflecting the model's ability to recognize these feature points.

[0044] S33. For example Figure 3 As shown, the relationship diagram centrally displays the degree of correlation between the model output and feature points, which is used to explain the correlation between the model output and feature points. The model output can be interpreted through the features of the input image in the model, so that the output results are based on evidence.

[0045] S34. For example Figure 4 As shown, the distribution of the model's output results for the same class of datasets before and after model modification is displayed. In the process of optimizing the model, comparing the performance of the model before and after optimization can provide some reference value for this optimization.

[0046] S35. For example Figure 5 As shown, a histogram of the current output of the model is displayed, providing direction in the process of optimizing the model;

[0047] S36. For example Figure 6 As shown, it is possible to... Figures 1-5 Based on the analysis, adjust the model's ability to recognize feature points of the input image at each level;

[0048] S4. Multi-view Visual Page Optimization Model: Based on a multi-view visual page analysis model, Figure 1 2. The analysis needs to focus on the feature points and model hierarchy, and adjustments should be made during the process. Figures 3-5 It provides valuable insights and guidance for adjusting the direction of the model, thereby optimizing it.

[0049] S41. First, observe the distribution of the model's real label feature points in the input image. Red indicates that the feature has a positive impact on the real label, while blue indicates a negative impact. Select the feature points that need to be focused on, and focus on the areas where the red points are more densely distributed.

[0050] S42. Based on the analysis shown in Figure 2, the selected feature point shows an increasing recognition ability in the first half of the model classification process. The influence of this feature point on the model output is constantly increasing. However, after the fourth convolution, it can be clearly seen that the feature value of this feature point drops sharply. Although there are fluctuations in the subsequent recognition process, the feature value has dropped to an extremely low value when the model outputs the result, resulting in a very weak recognition ability of the model for this important feature point. This may be one of the reasons why the model classifies the input image incorrectly.

[0051] S43. Based on the analysis results, adjust the model's ability to identify feature points after the 4th convolution layer. For example, increase the weight region for identifying the feature point by 10 times after the 4th convolution layer to improve the ability to identify feature points in this convolution layer.

[0052] S44. After adjusting the model Figure 3 The results show that the output probability of the true label is improved, the model's correct probability of the true label of the image is increased from 0.119 to 0.79, and the correlation between the output result and the feature point is improved. Figure 4The results show that the model improved the recognition accuracy of the dataset before and after the modification. Figure 5 The results show that the model improves the probability of identifying true labels while significantly reducing the probability of identifying incorrect labels. According to... Figures 3-5 The feedback indicated that the adjustment was reasonable, and the model's performance could be further improved by increasing the recognition capability of this feature point or by adjusting other feature points.

[0053] This invention proposes a multi-view interactive analysis method that uses gradient interpretation to calculate feature points of the input image at each layer of the model and optimizes a large number of feature points. A multi-view visual page is designed to make the model classification process transparent while providing a reasonable explanation of the mapping relationship between the output and input. An interactive analysis method is proposed to dynamically adjust the model based on the shortcomings reflected in the page, thereby improving the model's recognition performance on the dataset and achieving the goal of model optimization.

[0054] Example 2

[0055] This embodiment relates to a method for accurate fish identification based on an interactive analysis method of a multi-view correction model using the integral gradient method of the present invention. The steps include:

[0056] S1. Extracting Input Image Features in the Model Based on Integral Gradient Method: The gradient interpretation method was proposed in the 2017 NIPS paper "A Unified Approach to Interpreting Model Predictions" by Scott M. Lundberg and Su-In Lee. This invention uses the gradient interpretation method to calculate feature points for all fish in the ImageNet dataset. Each fish label can have one or more feature point maps, depending on the model's ability to recognize fish.

[0057] S2. Feature Point Optimization: The SLIC algorithm, proposed in PAMI 2012. This invention uses the SLIC algorithm to perform superpixel segmentation on each type of fish image to optimize feature points.

[0058] S3. Multi-view visual page design: The design of multi-view visual pages makes the ML black-box model interpretable.

[0059] S31. For example Figure 1As shown, the original image, feature map, and SLIC segmentation map are displayed together. To clearly map the feature points in each region, multiple images can be selected simultaneously. Different levels of the model can also be selected to display features, allowing observation of the features generated by the input image within the model and their distribution in the superpixel segmentation map, thus identifying which feature points require attention. By stacking the feature point map and the SLIC segmentation map, it can be seen that the information of the feature points (densely distributed in red) requires special attention.

[0060] S32. As shown in Figure 2, select Figure 1 The model identifies the feature points in the model and shows how these selected feature points change in the model's conv and pooling layers, reflecting the model's ability to recognize these feature points.

[0061] S33. For example Figure 3 As shown, the relationship diagram centrally displays the degree of correlation between the model output and feature points, which is used to explain the correlation between the model output and feature points. The model output can be interpreted through the features of the input image in the model, so that the output results are based on evidence.

[0062] S34. For example Figure 4 As shown, the distribution of the model's output results for the same class of datasets before and after model modification is illustrated. By comparing the performance of the model before and after optimization, this can provide a valuable reference for the optimization process.

[0063] S35. For example Figure 5 As shown, a probability histogram of the model's current output results is displayed, providing direction during the model optimization process.

[0064] S36. For example Figure 6 As shown, it is possible to... Figures 1-5 Based on the analysis, the model's ability to recognize feature points of the input image at each level was adjusted.

[0065] S4. Multi-view Visual Page Optimization Model: Based on a multi-view visual page analysis model, Figure 1 2. The analysis needs to focus on the feature points and model hierarchy, and adjustments should be made during the process. Figures 3-5 Provides valuable reference and direction for adjustment. Figure 7 This describes the entire process of optimizing the model using interactive analysis methods.

[0066] S41. First, observe the distribution of the model's feature points on the input image's true label. Red indicates that the feature has a positive impact on the true label, while blue indicates a negative impact. Select the feature points that need to be focused on, and prioritize the areas where red points are more densely distributed.

[0067] S42. Based on the situation shown in Figure 2, the model's ability to identify feature points changes during the model classification process, making the model's classification process interpretable and providing a basis for subsequent model adjustments.

[0068] S43. According to Figure 1 Based on the results of the analysis, the model's ability to identify feature points at the corresponding level was adjusted.

[0069] S44. Figure 3 As shown, observe whether the influence of the feature point on the model's output true label increases, and use this to determine whether the feature point needs to be adjusted. Figure 4 As shown, the model's performance is judged by demonstrating the recognition results of the dataset for this input image category before and after model modification. Figure 5 As shown, the distribution of the model's current output results can be used to determine whether the direction of adjustment needs to be changed. The model's performance can also be improved by increasing its ability to recognize that feature point or by adjusting other feature points.

[0070] S5. Retain the adjusted model parameters to achieve accurate fish identification: The model's identification performance on different fish datasets is evaluated through iterative optimization to achieve accurate fish identification.

[0071] S51. For each type of fish dataset, the model is analyzed and optimized using an interactive optimization method. The optimized model parameters are retained. When the model is optimized relative to other fish datasets, the adjusted model parameters are retained, and the model is optimized iteratively.

[0072] S52. The model is iteratively optimized using all fish datasets in the ImageNet dataset, ultimately achieving accurate fish identification.

[0073] The accurate fish identification described in this invention refers to all fish categories existing in ImageNet. If you want to add new fish, you can continue to add information from other fish datasets and enable the model to identify the fish. Finally, the model's identification will be made more accurate through interactive analysis methods.

Claims

1. A multi-view correction model interactive analysis method based on integral gradient method, comprising the following steps: S1. Extracting features of the input image in the model based on the integral gradient method: using gradient interpretation to quantify the model features of the input image; S11 uses the ImageNet dataset as background data samples for the VGG model and uses the gradient interpretation algorithm to generate an interpreter to interpret the ML model. S12 feeds the input image into the interpreter for computation, obtaining the feature points of the input image in the ML model; S2. Feature point optimization: The SLIC algorithm is used to perform superpixel segmentation on the input image to optimize feature points; S21. Use the SLIC algorithm to segment the input image into multiple feature regions with similar features; S22. Form a feature matrix by collecting all feature points in the feature region , representing the feature points of the region; S3. Multi-view visual page design: A multi-view visual page was designed to make the ML model interpretable; S31. Display the input image, feature map, and SLIC segmentation map together, and select different levels of the model to display features to observe the features generated by the input image in the model and their distribution in the superpixel segmentation map; S32. The changes of the feature points of interest in the model's conv and pooling layers are displayed, reflecting the model's ability to recognize the feature points; S33. A relationship diagram is used to centrally display the degree of correlation between the model output and feature points, which is used to explain the degree of correlation between the model output and feature points; S34. This shows the distribution of the model's output results for the same category dataset before and after model modification, allowing for real-time monitoring of model performance and providing valuable reference for model optimization. S35. Displays a probability histogram of the model's current output, providing direction for model optimization; S4. Optimize the model based on multi-view visual pages: Analyze the model based on multi-view visual pages, identify the causes of model classification errors, and optimize the model; S41. First, observe the distribution of the model's true label feature points in the input image, and select the feature points that need to be studied. S42. The changes in the selected feature points at different levels of the model reflect at which level the model's ability to recognize that feature point was incorrect. S43. Enhance the model's ability to identify the corresponding feature point at the corresponding level; S44. Optimize the model while ensuring its performance, based on the correlation between the model output and each feature point, the probability distribution of the current model output, and the model's recognition of the current dataset.

2. A multi-view rectification model interactive analysis method based on integral gradient method according to claim 1, characterized in that: The gradient interpretation method in step S1 refers to the algorithm generated by combining the ideas of integral gradient method, smooth gradient method and SHAP method.

3. The multi-view correction model interactive analysis method based on integral gradient method as described in claim 1, characterized in that: In step S2, the feature point points to the feature region obtained by the SLIC segmentation algorithm, and the entire region is regarded as a single point.

Images