Model improvement method and device

Abstract

The invention discloses a model improvement method and device. The method comprises the following steps: acquiring a plurality of first image samples; inputting the second image sample into an image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model; performing gradient analysis on output results of the plurality of intermediate layers to obtain a target gradient thermodynamic diagram; and reducing the weight of a target intermediate layer corresponding to the target gradient thermodynamic diagram to obtain an adjusted image processing model, updating the second image sample into any unused first image sample, and returning to execute the operation of inputting the second image sample into the image processing model based on the adjusted image processing model. Obtaining a plurality of adjusted image processing models until the plurality of first image samples are all used; and determining the image processing model with the highest image processing accuracy in the plurality of adjusted image processing models as a target image processing model. In this way, targeted adjustment can be conducted on the target middle layer needing to be adjusted, and time and labor are saved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model improvement method and device. Background Art

[0002] To improve the inference performance of deep learning models, it's often necessary to adjust the model's network structure to suit specific tasks and achieve higher prediction accuracy, such as in image processing models. To achieve the ideal network structure improvement plan, it's often necessary to perform traversal adjustments to the network structure, then compare the performance metrics of the models obtained with different adjustments to ultimately determine the optimal model adjustment solution.

[0003] However, traversal adjustments to the network structure are time-consuming and labor-intensive. Summary of the Invention

[0004] The embodiments of the present application provide a model improvement method and device, which can determine the target intermediate layer that needs to be adjusted in the image processing model through a gradient heat map, and make targeted adjustments to the target intermediate layer that needs to be adjusted without the need for traversal adjustments to the network structure of the model, saving time and effort.

[0005] In a first aspect, an embodiment of the present application provides a model improvement method, the method comprising:

[0006] Acquire a plurality of first image samples, and output a first processing result after the first image samples are processed by the image processing model, where the first processing result is different from a reference processing result corresponding to the first image sample;

[0007] Inputting the second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, where the plurality of intermediate layers are layers between the input layer and the output layer in the image processing model, and the second image sample is any one of the plurality of first image samples;

[0008] Perform gradient analysis on the output results of multiple intermediate layers respectively to obtain a target gradient heat map, which at least includes areas where the gradient is greater than the gradient threshold;

[0009] reducing the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, updating the second image sample to any unused first image sample, returning to execute the second image sample to be input into the image processing model based on the updated second image sample and the adjusted image processing model, and obtaining output results corresponding to multiple intermediate layers included in the image processing model, until all the first image samples are used, thereby obtaining multiple adjusted image processing models;

[0010] It is determined that the target image processing model is the one with the highest image processing accuracy among the multiple adjusted image processing models.

[0011] In a second aspect, an embodiment of the present application provides a model improvement device, which includes:

[0012] an acquisition module, configured to acquire a plurality of first image samples, and output a first processing result after the first image samples are processed by the image processing model, wherein the first processing result is different from a reference processing result corresponding to the first image sample;

[0013] an input module, configured to input a second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, wherein the plurality of intermediate layers are layers between the input layer and the output layer in the image processing model, and the second image sample is any one of the plurality of first image samples;

[0014] An analysis module is used to perform gradient analysis on the output results of multiple intermediate layers to obtain a target gradient heat map, where the target gradient heat map at least includes areas where the gradient is greater than a gradient threshold;

[0015] a processing module, configured to reduce the weight of a target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, update the second image sample to any unused first image sample, and return to execute, based on the updated second image sample and the adjusted image processing model, input the second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, until all the first image samples are used, thereby obtaining a plurality of adjusted image processing models;

[0016] The first determination module is used to determine that the target image processing model has the highest image processing accuracy among the multiple adjusted image processing models.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions,

[0018] When the processor executes the computer program instructions, the model improvement method as shown in any one of the embodiments of the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the model improvement method shown in any one of the embodiments of the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the model improvement method shown in any one of the embodiments of the first aspect.

[0021] The model improvement method and device of the embodiment of the present application can input the first image sample that is processed incorrectly by the image processing model into the image processing model to obtain the output results corresponding to multiple intermediate layers, and then perform gradient analysis on the output results of the multiple intermediate layers to obtain a target gradient heat map, which at least includes an area where the gradient is greater than the gradient threshold. Then, the weight of the target intermediate layer corresponding to the target gradient heat map is reduced to obtain an adjusted image processing model. After multiple first image samples are used, multiple adjusted image processing models are obtained, and the target image processing model with the highest image processing accuracy is determined. In this way, the target intermediate layer that needs to be adjusted in the image processing model can be determined through the gradient heat map, and the target intermediate layer that needs to be adjusted can be adjusted in a targeted manner without the need for traversal adjustment of the network structure of the model, saving time and effort. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is one of the flow charts of a model improvement method provided by an embodiment of the present application.

[0024] Figure 2 This is the second flow chart of a model improvement method provided by an embodiment of the present application.

[0025] Figure 3 This is a structural diagram of a model improvement device provided by an embodiment of the present application.

[0026] Figure 4 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0028] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0029] As mentioned in the background, existing technologies often require adjustments to the network structure to improve the reasoning performance of computer vision deep learning models, adapting them to specific tasks and achieving higher prediction accuracy. To obtain an ideal network structure improvement solution, extensive ablation experiments are often performed. These experiments involve modifying the connection order of specific layers in the model, increasing or decreasing the influence of certain layers on the results, and then comparing the performance metrics of the models under different network structure adjustments to ultimately determine the optimal model adjustment solution.

[0030] In the field of computer vision, achieving high-performance model structures is often based on extensive experimentation, and model improvements also follow this approach. However, because many models resemble black-box models, their internal processing methods are not user-friendly. Therefore, ablation experiments often struggle to design targeted experiments, and instead involve a traversal of possible model structures. Complex model structures increase the cost of a single ablation experiment and the number of possible adjustments to the model, making the entire process time-consuming and laborious, and potentially leading to extended attempts in incorrect or insignificant directions.

[0031] The model improvement method provided by the embodiment of the present application can input the first image sample that is processed incorrectly by the image processing model into the image processing model to obtain the output results corresponding to the multiple intermediate layers, and then perform gradient analysis on the output results of the multiple intermediate layers to obtain a target gradient heat map, which at least includes an area where the gradient is greater than the gradient threshold, and then reduce the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, until the multiple first image samples are all used, multiple adjusted image processing models are obtained, and then the target image processing model with the highest image processing accuracy is determined. In this way, the target intermediate layer that needs to be adjusted in the image processing model can be determined by the gradient heat map, and the target intermediate layer that needs to be adjusted can be adjusted in a targeted manner without the need for traversal adjustment of the network structure of the model, which saves time and effort, reduces costs, and improves the problem of trying in the wrong or insignificant direction for a long time.

[0032] In order to solve the above problems, an embodiment of the present application provides a model improvement method, which is described in detail below.

[0033] Figure 1 A flow chart of a model improvement method provided by an embodiment of the present application is shown. It should be noted that the model improvement method can be applied to a model improvement device, such as Figure 1 As shown, the model improvement method may include the following steps:

[0034] S110, acquiring a plurality of first image samples;

[0035] S120, inputting the second image sample into the image processing model to obtain output results corresponding to the multiple intermediate layers included in the image processing model;

[0036] S130, performing gradient analysis on the output results of the multiple intermediate layers to obtain a target gradient heat map;

[0037] S140, reducing the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, updating the second image sample to any unused first image sample, and returning to execute, based on the updated second image sample and the adjusted image processing model, inputting the second image sample into the image processing model to obtain output results corresponding to multiple intermediate layers included in the image processing model, until all the first image samples are used, thereby obtaining multiple adjusted image processing models;

[0038] S150: Determine the target image processing model with the highest image processing accuracy among the multiple adjusted image processing models.

[0039] Thus, the first image sample that is processed incorrectly by the image processing model can be input into the image processing model to obtain the output results corresponding to the multiple intermediate layers, and then the output results of the multiple intermediate layers are subjected to gradient analysis to obtain a target gradient heat map, which at least includes an area where the gradient is greater than the gradient threshold. Then, the weight of the target intermediate layer corresponding to the target gradient heat map is reduced to obtain an adjusted image processing model. After the multiple first image samples are used, multiple adjusted image processing models are obtained, and the target image processing model with the highest image processing accuracy is determined. In this way, the target intermediate layer that needs to be adjusted in the image processing model can be determined by the gradient heat map, and the target intermediate layer that needs to be adjusted can be adjusted in a targeted manner without the need for traversal adjustment of the network structure of the model, saving time and effort.

[0040] Regarding S110 , after the first image sample is processed by the image processing model, a first processing result may be output. The first processing result may be different from a reference processing result corresponding to the first image sample.

[0041] The image processing model may be a trained model obtained after model training. After training, the image processing model may be tested based on multiple test samples. If the image processing model performs poorly when processing certain test samples and obtains incorrect processing results, the test samples that obtained the incorrect results may be used as the first sample images.

[0042] For example, if the image processing model is a target detection model, the erroneous processing result may be missed target detection; if the image processing model is a drivable area segmentation model, the erroneous processing result may be segmenting the drivable area into a non-drivable area, thereby forming a hole.

[0043] Of course, each model may have multiple types of error processing results. For example, the drivable area segmentation model may have error processing results such as label errors, missed areas, false detection of specific obstacles, and uneven edges.

[0044] The multiple first image samples may include samples that will produce various types of erroneous processing results after being processed by the image processing model. In this way, adjusting the image processing model based on the multiple first image samples can make multiple improvements to the image processing model and improve the performance of the image processing model in multiple aspects.

[0045] In S120, the plurality of intermediate layers may be layers between an input layer and an output layer in the image processing model. The second image sample may be any one of the plurality of first image samples.

[0046] In some implementations, in order to more accurately perform targeted adjustments on the image processing model, before S120 , the method may further include:

[0047] According to the preset correspondence between the model type and the model interpretability method, the target model interpretability method corresponding to the image processing model is determined to be a method based on the guidance of the intermediate layer heat map.

[0048] Here, there are multiple model interpretability methods, and different types of models are suitable for different model interpretability methods. Model interpretability methods corresponding to different model types can be pre-set, that is, a preset correspondence between model types and model interpretability methods. Then, based on this preset correspondence, the target model interpretability corresponding to the image processing model can be determined as a method guided by the intermediate layer heat map.

[0049] Subsequently, the image processing model can be processed based on the method guided by the intermediate layer heat map.

[0050] Although there are many model interpretability methods in existing research, there has been no research on applying model interpretability methods to the field of computer vision. In the embodiments of the present application, the output of the intermediate layer of the model is visualized using the model interpretability method, and the model ablation experiment design and model structure improvement work are directly guided by the visualization information.

[0051] In some embodiments, the model interpretability method may include a method based on guidance of an intermediate layer heat map, and the model interpretability method may also include at least one of a perturbed data sample method, a feature engineering-based method, and a sample local importance interpretation-based method.

[0052] In this way, through the above process, a model interpretation method that is more suitable for the image processing model can be determined, so as to better visualize the image processing model, thereby facilitating subsequent more accurate targeted adjustments to the image processing model.

[0053] Regarding S130 , the magnitude of the heat map value of the gradient heat map can reflect the gradient magnitude of the feature.

[0054] The target gradient heat map may include at least areas where the gradient is greater than a gradient threshold. The gradient threshold can be set according to actual needs.

[0055] If the gradient of a certain area is greater than the gradient threshold, it can indicate that the area is significantly different from other areas, indicating that there may be errors in the area.

[0056] If the gradient heatmap corresponding to an intermediate layer contains regions with gradients greater than the gradient threshold, it can indicate that there is an error in the processing of that intermediate layer. The image processing model may output incorrect results due to processing errors in this intermediate layer.

[0057] The magnitude of the gradient in a gradient heatmap can be intuitively reflected by the color depth in the gradient heatmap. For example, if the first image sample is an image of a drivable area, then the gradient heatmap corresponding to the intermediate layer should not show any gradient changes within the drivable area, nor should any color changes. However, if a color change occurs in the gradient heatmap corresponding to a particular intermediate layer, this indicates that a gradient change has occurred within the drivable area, indicating an error in the processing of that intermediate layer, resulting in a void.

[0058] In some embodiments, it can be preset to output only the gradient heat map of the area where the gradient is greater than the gradient threshold, so that only the target gradient heat map is obtained after gradient analysis, and the target gradient heat map only includes the area where the gradient is greater than the gradient threshold.

[0059] For example, segmentation models are prone to the problem of partial pixel segmentation errors. By pre-setting the gradient heat map to output only the areas where the gradient is greater than the gradient threshold, the target gradient heat map can be output only for the pixels where segmentation errors occur.

[0060] In some embodiments, to more accurately determine the target gradient thermal map, S130 may include:

[0061] Perform gradient analysis on the output results of multiple intermediate layers respectively to obtain the gradient heat maps corresponding to the multiple intermediate layers;

[0062] A target gradient heatmap is determined from multiple gradient heatmaps.

[0063] Here, we can first obtain the gradient heatmap corresponding to each intermediate layer, and then filter the target gradient heatmap from it. The target gradient heatmap here includes not only areas where the gradient is greater than the gradient threshold, but also other areas.

[0064] In this way, by first obtaining the gradient heat maps corresponding to the multiple intermediate layers, and then determining the target gradient heat map from the multiple gradient heat maps, the target gradient heat map can be determined more accurately.

[0065] In some embodiments, in order to determine an intermediate layer with a large error, determining a target gradient heat map from multiple gradient heat maps may include:

[0066] determining an area of ​​a first region included in each of the plurality of gradient heat maps;

[0067] A target gradient heatmap is determined from the plurality of gradient heatmaps, wherein the area of ​​the first region is greater than an area threshold.

[0068] Here, the first region may be a region where the gradient is greater than the gradient threshold. A larger area of ​​the first region may indicate more errors in the gradient heat map, and a larger error may occur in the processing of the intermediate layer corresponding to the gradient heat map.

[0069] In other words, the gradient heat map with more errors can be used as the target heat map.

[0070] The area threshold can be set according to actual needs.

[0071] In this way, the target gradient heat map with more errors can be determined through the above process, thereby facilitating the determination of the intermediate layer with larger errors.

[0072] Regarding S140, the target intermediate layer may be an intermediate layer with errors in the processing process. Therefore, the target intermediate layer may have a negative impact on the output of the image processing model, resulting in an erroneous processing result output by the image processing model. Therefore, in order to reduce the impact of the target intermediate layer on the output of the image processing model, the weight of the target intermediate layer can be reduced.

[0073] Here, the image processing model may be adjusted multiple times based on multiple first image samples, and an adjusted image processing model may be obtained after each adjustment, thereby obtaining multiple adjusted image processing models.

[0074] In some implementations, to more conveniently reduce the weight of the target intermediate layer, S140 may include:

[0075] Reduce the number of channels in the target middle layer.

[0076] Here, reducing the number of channels of the target intermediate layer can reduce the impact of the target intermediate layer on the output results of the image processing model.

[0077] In this way, by reducing the number of channels in the target middle layer, the weight of the target middle layer can be easily reduced.

[0078] In some implementations, to quickly reduce the weight of the target intermediate layer, S140 may include:

[0079] In the case that the target intermediate layer is located in the first branch, the first branch is deleted.

[0080] Here, there may be multiple parallel branches in the structure of the image processing model. The first branch can be any one of the multiple parallel branches. If the target intermediate layer is located in the first branch, the weight of the target intermediate layer can be directly reduced to 0 by deleting the first branch.

[0081] In this way, by deleting the branch where the target middle layer is located, the weight of the target middle layer can be directly reduced to 0, thereby quickly reducing the weight of the target middle layer.

[0082] In some implementations, to more flexibly reduce the weight of the target intermediate layer, S140 may include:

[0083] In the case that the target intermediate layer is located in the first branch, the weight of the first branch is reduced and the weight of the second branch is increased.

[0084] Here, the second branch may be a branch parallel to the first branch in the image processing model.

[0085] If the target intermediate layer is located in the first branch, when weakening the influence of the target intermediate layer on the output result of the image processing model, in addition to reducing the weight of the first branch, the weight of the second branch parallel to the first branch can also be increased.

[0086] In some implementations, when the target intermediate layer is located in the first branch, only the weight of the second branch may be increased without reducing the weight of the first branch.

[0087] In this way, through the above process, the weight of the target middle layer can be reduced more flexibly.

[0088] Regarding S150, after each adjustment of the image processing model to obtain the adjusted image processing model, the adjusted image processing model can be tested to obtain the image processing accuracy of each adjusted image processing model, and the adjusted image processing model with the highest image processing accuracy can be used as the final target image processing model, that is, the improved image processing model.

[0089] In some implementations, to more accurately determine the target image processing model with the highest image processing accuracy, S150 may include:

[0090] acquiring a plurality of third image samples;

[0091] performing an image processing test on each adjusted image processing model based on a plurality of third image samples to obtain image processing accuracy rates corresponding to the plurality of adjusted image processing models;

[0092] It is determined that the target image processing model is the one with the highest image processing accuracy among the multiple adjusted image processing models.

[0093] Here, after the third image sample is processed by the image processing model, a second processing result may be output, and the second processing result may be different from a reference processing result corresponding to the third image sample.

[0094] The multiple third image samples may be the multiple first image samples mentioned above, or may be other image samples whose error types of processing results obtained after being processed by the image processing model are the same as the error types of the processing results of the multiple first image samples.

[0095] Specifically, for each adjusted image processing model, multiple third image samples are respectively input into the adjusted image processing model. If the processing result output by the adjusted image processing model is the same as the reference processing result corresponding to the third image sample, it indicates that the adjusted image processing model processes the third image sample correctly; if the processing result output by the adjusted image processing model is different from the reference processing result corresponding to the third image sample, it indicates that there is an error in the processing of the third image sample by the adjusted image processing model.

[0096] The image processing accuracy may be a ratio of the number of third image samples processed correctly to the total number of third image samples.

[0097] After determining the image processing accuracy corresponding to each adjusted image processing model through the above process, the target image processing model can be determined as the one with the highest image processing accuracy.

[0098] In this way, through the above process, the target image processing model with the highest image processing accuracy can be determined more accurately.

[0099] In order to better describe the entire solution, based on the above embodiments, a specific example is given. Figure 2 As shown, the model improvement method may include S201-S210, which will be explained in detail below.

[0100] S201: Acquire multiple first image samples.

[0101] S202: According to a preset correspondence between the model type and the model interpretability method, it is determined that the target model interpretability method corresponding to the image processing model is a method based on guidance of an intermediate layer heat map.

[0102] S203: Input the second image sample into the image processing model to obtain output results corresponding to the multiple intermediate layers included in the image processing model.

[0103] S204: Perform gradient analysis on the output results of the multiple intermediate layers to obtain gradient heat maps corresponding to the multiple intermediate layers.

[0104] S205 : Determine the area of ​​a first region included in each of the multiple gradient heat maps.

[0105] S206 , determining a target gradient heatmap from the multiple gradient heatmaps, wherein the area of ​​the first region is greater than an area threshold.

[0106] S207, reducing the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, updating the second image sample to any unused first image sample, returning to execute based on the updated second image sample and the adjusted image processing model, inputting the second image sample into the image processing model, and obtaining output results corresponding to multiple intermediate layers included in the image processing model, until multiple first image samples are used, and multiple adjusted image processing models are obtained.

[0107] S208: Acquire multiple third image samples.

[0108] S209: Perform an image processing test on each adjusted image processing model based on the multiple third image samples to obtain image processing accuracy rates corresponding to the multiple adjusted image processing models.

[0109] S210: Determine the target image processing model with the highest image processing accuracy among the multiple adjusted image processing models.

[0110] The specific process can be found in the above embodiment and will not be described again here.

[0111] The embodiment of the present application combines the model explanation method with the model improvement, providing a more reasonable and time-saving and labor-saving model improvement method.

[0112] The model improvement method provided in the embodiment of the present application is able to design ablation experiments in a well-founded and targeted manner compared to the current methods of ergodic model structure ablation experiments, and fills the gap that the current model interpretability method is not directly applied to model structure improvement. With the improvement of computing power, various large models have gradually begun to show their superiority, and there are more directions to try for network adjustment of large models. If an ergodic exploration without basis is conducted, it will be time-consuming and labor-intensive, and the direction of model improvement may not be grasped. The embodiment of the present application can make model improvements in a well-founded and quick manner, and establish a connection between different layers of the model and different error types. In this way, after the model is improved, when a new type of error occurs in the model, the prior knowledge obtained based on the model interpretability heat map can be used to more quickly and directly lock the part of the model that needs to be improved corresponding to the error, shorten the model performance improvement cycle, and improve production efficiency.

[0113] Based on the same inventive concept, the present application embodiment also provides a model improvement device. Figure 3 The model improvement device provided in the embodiments of the present application is described in detail.

[0114] Figure 3 A schematic structural diagram of a model improvement device provided in one embodiment of the present application is shown.

[0115] like Figure 3 As shown, the model improvement device may include:

[0116] An acquisition module 301 is configured to acquire a plurality of first image samples, and output a first processing result after the first image samples are processed by an image processing model, wherein the first processing result is different from a reference processing result corresponding to the first image sample;

[0117] An input module 302 is configured to input a second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, where the plurality of intermediate layers are layers between the input layer and the output layer in the image processing model, and the second image sample is any one of the plurality of first image samples;

[0118] An analysis module 303 is configured to perform gradient analysis on the output results of the multiple intermediate layers to obtain a target gradient heat map, where the target gradient heat map includes at least areas where the gradient is greater than a gradient threshold.

[0119] The processing module 304 is configured to reduce the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, update the second image sample to any unused first image sample, and return to execute the second image sample input into the image processing model based on the updated second image sample and the adjusted image processing model to obtain output results corresponding to multiple intermediate layers included in the image processing model, until all the first image samples are used, thereby obtaining multiple adjusted image processing models.

[0120] The first determining module 305 is configured to determine the target image processing model having the highest image processing accuracy among the multiple adjusted image processing models.

[0121] Thus, the first image sample that is processed incorrectly by the image processing model can be input into the image processing model to obtain the output results corresponding to the multiple intermediate layers, and then the output results of the multiple intermediate layers are subjected to gradient analysis to obtain a target gradient heat map, which at least includes an area where the gradient is greater than the gradient threshold. Then, the weight of the target intermediate layer corresponding to the target gradient heat map is reduced to obtain an adjusted image processing model. After the multiple first image samples are used, multiple adjusted image processing models are obtained, and the target image processing model with the highest image processing accuracy is determined. In this way, the target intermediate layer that needs to be adjusted in the image processing model can be determined by the gradient heat map, and the target intermediate layer that needs to be adjusted can be adjusted in a targeted manner without the need for traversal adjustment of the network structure of the model, saving time and effort.

[0122] In some embodiments, in order to more accurately adjust the image processing model, the apparatus may further include:

[0123] The second determination module is used to determine that the target model explanatory method corresponding to the image processing model is a method based on the guidance of the intermediate layer heat map according to a preset correspondence between the model type and the model explanatory method before inputting the second image sample into the image processing model to obtain the output results corresponding to the multiple intermediate layers included in the image processing model.

[0124] In some embodiments, to more accurately determine the target gradient thermal map, the analysis module 303 may include:

[0125] The gradient analysis submodule is used to perform gradient analysis on the output results of multiple intermediate layers respectively to obtain the gradient heat maps corresponding to the multiple intermediate layers;

[0126] The first determination submodule is used to determine a target gradient heat map from multiple gradient heat maps.

[0127] In some implementations, to determine an intermediate layer with a significant error, the first determining submodule may include:

[0128] a first determining unit, configured to determine an area of ​​a first region included in each of the plurality of gradient heat maps, the first region being a region where a gradient is greater than a gradient threshold;

[0129] The second determining unit is configured to determine, from the plurality of gradient heat maps, a target gradient heat map in which the area of ​​the first region is greater than an area threshold.

[0130] In some implementations, to more conveniently reduce the weight of the target intermediate layer, the processing module 304 may include:

[0131] The reduction submodule is used to reduce the number of channels in the target middle layer.

[0132] In some implementations, to quickly reduce the weight of the target middle layer, the processing module 304 may include:

[0133] The deletion submodule is used to delete the first branch when the target intermediate layer is located in the first branch.

[0134] In some implementations, in order to more flexibly reduce the weight of the target intermediate layer, the processing module 304 may include:

[0135] A processing submodule is used to reduce the weight of the first branch and increase the weight of the second branch when the target intermediate layer is located in the first branch, and the second branch is a branch parallel to the first branch in the image processing model.

[0136] In some embodiments, in order to more accurately determine the target image processing model with the highest image processing accuracy, the first determining module 305 may include:

[0137] an acquisition submodule, configured to acquire a plurality of third image samples, and output a second processing result after the third image samples are processed by the image processing model, wherein the second processing result is different from a reference processing result corresponding to the third image samples;

[0138] a testing submodule, configured to perform an image processing test on each adjusted image processing model based on a plurality of third image samples, and obtain image processing accuracy rates corresponding to the plurality of adjusted image processing models;

[0139] The second determining submodule is used to determine that the target image processing model has the highest image processing accuracy among the multiple adjusted image processing models.

[0140] In some embodiments, the model interpretability method may include a method based on guidance of an intermediate layer heat map, and the model interpretability method may also include at least one of a perturbed data sample method, a feature engineering-based method, and a sample local importance interpretation-based method.

[0141] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown.

[0142] like Figure 4 As shown in FIG, the electronic device 4 is a block diagram of an exemplary hardware architecture of an electronic device capable of implementing the model improvement method and apparatus according to the embodiment of the present application. The electronic device may refer to the electronic device in the embodiment of the present application.

[0143] The electronic device 4 may include a processor 401 and a memory 402 storing computer program instructions.

[0144] Specifically, the processor 401 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0145] Memory 402 may include a large-capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In certain embodiments, memory 402 is non-volatile solid-state memory. In certain embodiments, memory 402 may include read-only memory (ROM), random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory 402 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0146] The processor 401 implements any one of the model improvement methods in the above embodiments by reading and executing computer program instructions stored in the memory 402 .

[0147] In one example, the electronic device may further include a communication interface 403 and a bus 404. Figure 4 As shown, the processor 401 , the memory 402 , and the communication interface 403 are connected via a bus 404 and communicate with each other.

[0148] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0149] Bus 404 comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 404 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0150] The electronic device can execute the model improvement method in the embodiment of the present application, thereby realizing the combination Figures 1 to 3 Described model improvement method and apparatus.

[0151] In addition, in combination with the model improvement method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, any one of the model improvement methods in the above embodiments is implemented.

[0152] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0153] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0154] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0155] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0156] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A model improvement method, characterized in that: The method comprises: Acquire a plurality of first image samples, and output a first processing result after the first image samples are processed by an image processing model, wherein the first processing result is different from a reference processing result corresponding to the first image sample; Inputting a second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, wherein the plurality of intermediate layers are layers between the input layer and the output layer in the image processing model, and the second image sample is any one of the plurality of first image samples; Performing gradient analysis on the output results of the multiple intermediate layers respectively to obtain a target gradient heat map, wherein the target gradient heat map at least includes an area where the gradient is greater than a gradient threshold; reducing the weight of the target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, updating the second image sample to any unused first image sample, returning to execute the second image sample to be input into the image processing model based on the updated second image sample and the adjusted image processing model, and obtaining output results corresponding to multiple intermediate layers included in the image processing model, until all the multiple first image samples are used, thereby obtaining multiple adjusted image processing models; Determine the target image processing model as the one with the highest image processing accuracy among the multiple adjusted image processing models.

2. The method according to claim 1, characterized in that Before inputting the second image sample into the image processing model to obtain output results corresponding to the plurality of intermediate layers included in the image processing model, the method further includes: According to the preset correspondence between the model type and the model interpretability method, it is determined that the target model interpretability method corresponding to the image processing model is a method based on the guidance of the intermediate layer heat map.

3. The method according to claim 1, characterized in that The performing gradient analysis on the output results of the multiple intermediate layers to obtain a target gradient heat map includes: Performing gradient analysis on the output results of the multiple intermediate layers respectively to obtain gradient heat maps corresponding to the multiple intermediate layers respectively; The target gradient thermodynamic map is determined from the plurality of gradient thermodynamic maps.

4. The method according to claim 3, characterized in that The determining the target gradient thermogram from the plurality of gradient thermograms includes: Determine an area of ​​a first region included in each of the plurality of gradient heat maps, where the first region is a region where the gradient is greater than a gradient threshold; The target gradient heat map having an area of ​​the first region greater than an area threshold is determined from the plurality of gradient heat maps.

5. The method according to claim 1, wherein Reducing the weight of the target intermediate layer corresponding to the target gradient heat map includes: Reduce the number of channels in the target middle layer.

6. The method according to claim 1, wherein Reducing the weight of the target intermediate layer corresponding to the target gradient heat map includes: In a case where the target intermediate layer is located in the first branch, the first branch is deleted.

7. The method according to claim 1, characterized in that Reducing the weight of the target intermediate layer corresponding to the target gradient heat map includes: In a case where the target intermediate layer is located in a first branch, the weight of the first branch is reduced and the weight of a second branch is increased, where the second branch is a branch in parallel with the first branch in the image processing model.

8. The method according to claim 1, characterized in that Determining the target image processing model as the one with the highest image processing accuracy among the multiple adjusted image processing models includes: Acquire a plurality of third image samples, wherein the third image samples are processed by the image processing model to output a second processing result, wherein the second processing result is different from a reference processing result corresponding to the third image samples; performing an image processing test on each adjusted image processing model based on a plurality of third image samples to obtain image processing accuracy rates corresponding to the plurality of adjusted image processing models; Determine that the target image processing model has the highest image processing accuracy among the multiple adjusted image processing models.

9. The method according to claim 2, characterized in that The model interpretability method includes a method based on guidance of an intermediate layer heat map, and the model interpretability method also includes at least one of a perturbed data sample method, a feature engineering-based method, and a sample local importance interpretation-based method.

10. A model improvement device, characterized in that: The device comprises: an acquisition module, configured to acquire a plurality of first image samples, wherein the first image samples are processed by an image processing model to output a first processing result, wherein the first processing result is different from a reference processing result corresponding to the first image sample; an input module, configured to input a second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, wherein the plurality of intermediate layers are layers between the input layer and the output layer in the image processing model, and the second image sample is any one of the plurality of first image samples; an analysis module, configured to perform gradient analysis on the output results of the plurality of intermediate layers respectively to obtain a target gradient heat map, wherein the target gradient heat map at least includes an area where a gradient is greater than a gradient threshold; a processing module, configured to reduce the weight of a target intermediate layer corresponding to the target gradient heat map to obtain an adjusted image processing model, update the second image sample to any unused first image sample, and return to execute, based on the updated second image sample and the adjusted image processing model, input the second image sample into the image processing model to obtain output results corresponding to a plurality of intermediate layers included in the image processing model, until all the plurality of first image samples are used, thereby obtaining a plurality of adjusted image processing models; The first determining module is configured to determine that the target image processing model is the one with the highest image processing accuracy among the multiple adjusted image processing models.

Images