A model attention visualization method and related device

Abstract

The present invention provides a model attention visualization method and related devices, which relate to the fields of artificial intelligence and computer vision technology. By obtaining a self-supervised model, and obtaining the category part data information that the self-supervised model pays attention to; the category part data is clipped to establish a classification data set; the backbone of the self-supervised model is analyzed and the phase part parameters corresponding to the backbone are frozen according to the analysis results; the self-supervised model is classified and trained according to the frozen backbone combined with the classification data set; the heat map attention corresponding to the self-supervised model is viewed through visualization and the analysis results are generated. Based on the self-supervised pre-training model, combined with the visual classification model, model fine-tuning, and model heat map visualization, the technical effect of visualizing the model training effect is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and computer vision technology, and in particular to a model attention visualization method and related devices. Background Art

[0002] With the development of technologies in the field of artificial intelligence vision, self-supervised learning, fine-tuning, and computer vision technologies have become increasingly mature and widely used. Self-supervised learning has shown advantages on large-scale unlabeled data sets. It not only greatly reduces the need for data labeling, but also has better robustness than supervised learning. Fine-tuning technology, as the key to improving model performance, plays a core role in various model optimization scenarios. The development of computer vision technology has made the decision-making process of neural networks more transparent. Through the visualization of feature maps, researchers can intuitively understand the model's attention allocation logic.

[0003] However, self-supervised learning is used to train the backbone, and it is necessary to view the generated results according to various strategies to infer the model performance. The quality of the model is somewhat random, and it is difficult to directly quantify the model quality. Although fine-tuning can specifically adjust or supplement model parameters to optimize the performance of specific tasks, when the model performance does not meet the standards, it is still complicated to diagnose the specific reasons, and solutions need to be explored through repeated experiments. Regarding feature visualization, although models based on classification tasks can better utilize backward gradients and forward features to generate attention maps, existing visualization frameworks are often difficult to cope with when faced with transformer architectures and complex detection models, and usually require researchers to perform tedious manual operations to extract feature maps.

[0004] Therefore, how to visualize the model training effect has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to realize the visualization of model training effects, the present application provides a model attention visualization method and related devices.

[0006] In the first aspect, a model attention visualization method provided by the present application adopts the following technical solution:

[0007] A model attention visualization method, comprising:

[0008] Obtaining a self-supervisory model and obtaining category and part data information that the self-supervisory model focuses on;

[0009] Cutting the category part data to establish a classification data set;

[0010] Analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results;

[0011] Performing classification training on the self-supervised model according to the frozen backbone combined with the classification data set;

[0012] The heat map attention corresponding to the self-supervised model is viewed visually and the analysis results are generated.

[0013] Optionally, the self-supervised model adopts the SIMMIM self-supervised learning framework in MMselfsup, the backbone part in the self-supervised model is a 4-layer stage, the dim output is 1536, and the self-supervised model is a SwinTransformer-large model.

[0014] Optionally, the step of clipping the category part data to establish a classification data set includes:

[0015] Performing part clipping on the category part data to scale them to a uniform size;

[0016] The trimmed category part data are classified according to data categories to establish a classification data set.

[0017] Optionally, the step of analyzing the backbone of the self-supervisory model and freezing some parameters of the stage corresponding to the backbone according to the analysis result includes:

[0018] Analyze the type information of the backbone of the self-supervisory model and the stage structure of the neural network model used by the backbone to generate analysis results;

[0019] Determine the number of stages that need to be frozen according to the analysis results;

[0020] Freeze some parameters of the stages corresponding to the backbone according to the number of stages that need to be frozen.

[0021] Optionally, after the step of determining the number of stages to be frozen according to the analysis result, the step further includes:

[0022] Build a classification network;

[0023] Analyze the corresponding output matrix dimension according to the backbone type information;

[0024] Determine the classification head input according to the output matrix dimension;

[0025] The classification network is updated according to the classification head input and the backbone.

[0026] Optionally, the step of performing classification training on the self-supervisory model according to the frozen backbone combined with the classification data set includes:

[0027] Obtaining an MMPretrain training framework, and configuring the classification data set on the MMPretrain training framework;

[0028] In the case of the frozen backbone, classification training is performed on the classification network in the self-supervised model.

[0029] Optionally, the step of visually viewing the heat map attention corresponding to the self-supervisory model and generating an analysis result includes:

[0030] Using the PyTorch-Grad-CAM visualization framework to view the heat map attention corresponding to the self-supervised model to generate viewing results;

[0031]

[0032] Among them, A represents the feature layer, k represents the kth channel of feature layer A, c represents category c, and Ak represents the data of channel k of feature layer A. Represents the weight for Ak;

[0033] An analysis result is generated through the review result.

[0034] In a second aspect, a model attention visualization system is provided, wherein the model attention visualization comprises:

[0035] An information acquisition module, used to acquire a self-supervisory model and obtain category and part data information that the self-supervisory model focuses on;

[0036] A classification data set module, used for cutting the category part data to establish a classification data set;

[0037] A freezing module, used to analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results;

[0038] A classification training module, used for performing classification training on the self-supervised model according to the frozen backbone combined with the classification data set;

[0039] The analysis result module is used to visualize the heat map attention corresponding to the self-supervised model and generate analysis results.

[0040] In a third aspect, the present application provides a model attention visualization device, which includes: a memory and a processor, and when the processor runs the model attention visualization program stored in the memory, it executes the method described above.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0042] In summary, the present application includes the following beneficial technical effects:

[0043] This application obtains a self-supervised model and the category part data information that the self-supervised model pays attention to; the category part data is cropped to establish a classification data set; the backbone of the self-supervised model is analyzed and the phase part parameters corresponding to the backbone are frozen according to the analysis results; the self-supervised model is classified and trained according to the frozen backbone combined with the classification data set; the heat map attention corresponding to the self-supervised model is viewed through visualization and the analysis results are generated. Based on the self-supervised pre-trained model, combined with the visual classification model, model fine-tuning, and model heat map visualization, the technical effect of visualizing the model training effect is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the structure of a model attention visualization device of a hardware operating environment involved in an embodiment of the present application;

[0045] Figure 2 It is a flowchart of the first embodiment of the model attention visualization method of the present application;

[0046] Figure 3 This is a schematic diagram of a specific implementation process of the first embodiment of the model attention visualization method of the present application;

[0047] Figure 4 It is a structural block diagram of the first embodiment of the model attention visualization system of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a model attention visualization device for the hardware operating environment involved in the embodiment of the present application.

[0050] like Figure 1As shown, the model attention visualization device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RandomAccess Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure shown in does not constitute a limitation of the model attention visualization device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a model attention visualization program.

[0053] exist Figure 1 In the model attention visualization device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the present application can be set in the model attention visualization device, and the model attention visualization device calls the model attention visualization program stored in the memory 1005 through the processor 1001, and executes the model attention visualization method provided in the embodiment of the present application.

[0054] The present application embodiment provides a model attention visualization method, referring to Figure 2 , Figure 2 This is a flow chart of the first embodiment of the model attention visualization method of the present application.

[0055] In this embodiment, the model attention visualization method includes the following steps:

[0056] Step S10: Obtain a self-supervisory model and obtain category part data information that the self-supervisory model focuses on.

[0057] It should be noted that the self-supervised model adopts the SIMMIM self-supervised learning framework in MMselfsup. The backbone part of the self-supervised model is a 4-layer stage, the dim output is 1536, and the self-supervised model is a SwinTransformer-large model.

[0058] It is understandable that the existing self-supervised models are trained on unlabeled data to learn the original image features. For example, the visualization of the self-supervised model in MMSelfsup is mainly mask reconstruction and t-SNE classification visualization, or checking the downstream task indicators of knn linear classification or other types of downstream task indicators. It is rare to visualize the model feature heat map before the downstream tasks in the training process, and there is a lack of understanding of the attention features learned by the self-supervised model.

[0059] Step S20: cropping the category part data to establish a classification data set.

[0060] It can be understood that the step of cropping the category part data to establish a classification data set includes: cropping the category part data to scale them to a uniform size; and classifying the cropped category part data according to data categories to establish a classification data set.

[0061] In the specific implementation, the category area of ​​interest in the samples during self-supervised training is cropped and scaled to a uniform size. The area should contain complete objects with as little background interference as possible; the cropped images are stored in the corresponding folder to generate a classification data set.

[0062] Step S30: Analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results.

[0063] It should be noted that the self-supervised model determines the backbone type based on its training framework, that is, whether the model is CNN or Transformer, and whether it is a single-stage model or a multi-stage model. If it is a multi-stage model, the number of stages is.

[0064] It can be understood that the step of analyzing the backbone of the self-supervised model and freezing some parameters of the stages corresponding to the backbone according to the analysis results includes: analyzing the type information of the backbone of the self-supervised model and the stage structure of the neural network model used by the backbone to generate the analysis results; determining the number of stages to be frozen according to the analysis results; and freezing some parameters of the stages corresponding to the backbone according to the number of stages to be frozen.

[0065] In a specific implementation, after the step of determining the number of stages that need to be frozen according to the analysis results, the method further includes: constructing a classification network; analyzing the corresponding output matrix dimension according to the type information of the backbone; determining the classification head input according to the output matrix dimension; and updating the classification network according to the classification head input and the backbone.

[0066] It should be noted that the step of performing classification training on the self-supervised model according to the frozen backbone combined with the classification data set includes: obtaining the MMPretrain training framework, configuring the classification data set on the MMPretrain training framework; and performing classification training on the classification network in the self-supervised model under the frozen backbone.

[0067] It is understandable that the classification model uses the MMPretrain training framework, combining the linear classification head with the backbone of SwinTransformer-large, and the dim of the backbone and the in_channels of the head are consistent.

[0068] It should be noted that in self-supervised learning, data is generally processed by mask occlusion, and then the mask area is reconstructed by the model to reflect the effect of model learning. In this process, the feature extraction part is the work of the backbone. This embodiment adopts the MMSelfsup self-supervised learning framework, the SIMMIM self-supervised learning method is the basis, and the backbone is the SwinTransformer-large model, such as Figure 3 As shown in the figure, by performing self-supervised learning on a large number of unlabeled porcelain bottle images (256x256), a large number of low-level features, high-level semantics, and generalization capabilities of porcelain bottles are learned, and the weight information of these features is stored in the backbone part.

[0069] The backbone used in this embodiment is trained by the above-mentioned SIMMIM self-supervised learning. By analyzing its structure through the SwinTransformer paper or source code, it can be known that it is a 4-stage structure. The dim output of the last stage is 1536, so the corresponding head in_channels must also correspond to 1536.

[0070] This example uses the MMPretrain training framework for classification training. In the MMPretrain training framework, there is a ready-made SwinTransformer classification training configuration, which uses the GlobalAveragePooling method for neck and the LinearClsHead (linear classification) method for head. Finally, the backbone selected in this example is SwinTransformer-large, the neck is GlobalAveragePooling, the head is LinearClsHead linear classification head, the final stage output dimension of the model is 1536, and the in_channels of the head is 1536.

[0071] Step S40: Classification training is performed on the self-supervised model based on the frozen backbone and the classification data set.

[0072] In the specific implementation, freezing the backbone parameters of SwinTransformer requires freezing the corresponding 4 stages. Edit the training configuration file of MMpretrain. In the configuration of its model, you can assemble the freezing parameter frozen_stages, starting with sequence number 0. If you configure it to 3, you can freeze all 4 stages.

[0073] Configure the backbone part of the model to be the backbone with the same parameters as the self-supervised learning, configure the pre-trained model to be the model pre-trained by self-supervision, configure the training dataset to be the classification dataset processed by S1, configure the dataset loader to perform various enhancements such as flipping and scaling, optimize the training iterations and loss function, configure the number of training iterations and verification cycle, etc., and finally configure the number of categories.

[0074] During training, set find_unused_parameters to True so that the unused parameters are not passed and only the weights with the same name as the backbone layer are loaded from the SIMMIM pre-trained model file.

[0075] This example freezes the backbone stage through the above configuration and passes in the SIMMIM pre-trained model file parameters to achieve the purpose of using the same set of backbone parameters. During training, only the neck and head parts are updated, and the backbone parameter weights do not change.

[0076] Step S50: View the heat map attention corresponding to the self-supervised model through visualization and generate analysis results.

[0077] In a specific implementation, the steps of visually viewing the heat map attention corresponding to the self-supervised model and generating analysis results include: using a PyTorch-Grad-CAM visualization framework to view the heat map attention corresponding to the self-supervised model to generate viewing results;

[0078]

[0079] Among them, A represents the feature layer, k represents the kth channel of feature layer A, c represents category c, and Ak represents the data of channel k of feature layer A. Represents the weight for Ak; the analysis result is generated by viewing the result.

[0080] It should be noted that this embodiment takes out the model and configuration file, uses the visual script integrated in the MMPretrain tool to configure it, loads the test image, and the script calls Grad-Cam to obtain the feature matrix calculated forward by the model through the hook, and then obtains the gradient matrix calculated backward through the hook, and generates a heat map through matrix operations.

[0081] When analyzing the heat map, observe whether the attention focus area is on the object of attention. If so, the effect is good. If not, the attention distraction effect is not good.

[0082] In the specific implementation, the library used for model heat map visualization is PyTorch-Grad-CAM, which was originally designed to explain the decision-making process of the model in image classification. It has been integrated in the MMPretrain framework, and the classification model and images are loaded through the visual script, the feature layer is specified, and the visual output freezes the backbone calculation attention heat map; by observing the distribution of the heat map, it can be known whether the model's attention focus area is focused on the object, and the model effect can be clearly understood.

[0083] It should be noted that in the visualization process of the Grad-CAM program integrated by MMPretain, the specified norm feature layer is first selected, usually the last norm layer of the backbone. By obtaining the matrix after the final norm output, the final feature matrix extracted by the model can be obtained. The principle formula of LayerNorm is as follows.

[0084]

[0085] During actual operation, GRAD_CAM will first register a hook, which will hook the forward and backward propagation processes of the model. When calculating each stage of the backbone, the hook will be used to obtain and record the calculated layer matrix information, and the recorded information will be saved in two corresponding lists, one is the feature matrix (activation output) list corresponding to the layer, and the other is the gradient matrix list corresponding to the layer.

[0086] When the head is classified, the final gradient matrix of the corresponding category is obtained by backpropagation, and the mean of the gradient matrix corresponding to each layer and channel is calculated to obtain the weight, and then it is multiplied with the feature matrix, merged, and scaled to generate the cam heat map corresponding to each layer.

[0087] Then the CAM of each layer is merged, the maximum value is processed to generate the final heat map, and finally the heat map is superimposed on the original image for visualization. The visualized color area is not a heat map, so the colors displayed from high to low are red, yellow, green, and blue.

[0088] The visualized heat map can help us understand the focus of the feature matrix and gradient matrix. By focusing on the focus area of ​​the heat map, blue is the area with weak attention, and red and yellow are the areas with strong attention. By observing the distribution of the attention heat map on objects and backgrounds, we can know the model's feature extraction ability for objects of this category. Depending on the situation, the model can be divided into the following situations:

[0089] The model has the best effect: the focused area (red and yellow area) fits the overall area of ​​the classified object (more than 90%), and the edges fit well, which means that the model has a very good feature extraction ability for objects of this color and shape.

[0090] The model works well: the focused area (red and yellow area) fits the overall area of ​​the classified object (60%-90%), but the edge only fits partially or not at all. This means that the model has learned most of the overall features, but not all of the edge features. You can consider adding samples or fine-tuning the annotations.

[0091] The model effect is average: the attention focus area (red and yellow area) fits the characteristics of some areas of the classified object (40%-60%). This situation means that the model has learned the characteristics of some areas with obvious characteristics, but has not learned the overall characteristics. Generally, it is necessary to slightly adjust the labeled data or replace the model backbone network; or the entire object is the attention focus area (-100%). This reverse situation usually occurs when the number of parameters in the model network layer is too large. The model has learned everything, but the sample size is insufficient, causing the model to directly focus on the surrounding features and infer the object. The accuracy is also relatively high, which is overfitting. However, when it appears in the training sample situation, it may lead to misjudgment. It is necessary to increase the sample diversity and sample size, or change to a model with fewer parameters.

[0092] Poor model effect: The attention focus (red and yellow) area is scattered (less than 40% and scattered), and the focus is not on the key feature area of ​​the object. This situation indicates that the model has not learned the key features of the object. Basically, this situation indicates that the quality of the labeled data is not good and needs to be re-labeled.

[0093] It is understandable that this embodiment provides a mechanism for evaluating and optimizing attention heat maps. By comparing the attention distribution before and after self-supervised learning, the improvement in the model's ability to identify important features can be quantified. This method is applicable to visual self-supervised models, which helps researchers and developers better understand and improve the performance of the model and provides a way for interpretive research on the model.

[0094] This embodiment obtains the self-supervised model and the category part data information that the self-supervised model pays attention to; the category part data is cropped to establish a classification data set; the backbone of the self-supervised model is analyzed and the phase part parameters corresponding to the backbone are frozen according to the analysis results; the self-supervised model is classified and trained according to the frozen backbone combined with the classification data set; the heat map attention corresponding to the self-supervised model is viewed through visualization and the analysis results are generated. Based on the self-supervised pre-training model, combined with the visual classification model, model fine-tuning, and model heat map visualization, the technical effect of visualizing the model training effect is achieved.

[0095] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which a program for model attention visualization is stored. When the program for model attention visualization is executed by a processor, the steps of the method for model attention visualization as described above are implemented.

[0096] Reference Figure 4 , Figure 4 This is a structural block diagram of the first embodiment of the model attention visualization system of this application.

[0097] like Figure 4As shown, the model attention visualization system proposed in the embodiment of the present application includes:

[0098] The information acquisition module 10 is used to acquire the self-supervisory model and obtain the category part data information that the self-supervisory model focuses on;

[0099] A classification data set module 20, for cutting the category part data to establish a classification data set;

[0100] A freezing module 30 is used to analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results;

[0101] A classification training module 40 is used to perform classification training on the self-supervised model according to the frozen backbone combined with the classification data set;

[0102] The analysis result module 50 is used to visualize the heat map attention corresponding to the self-supervised model and generate analysis results.

[0103] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any restrictions on this.

[0104] This embodiment obtains the self-supervised model and the category part data information that the self-supervised model pays attention to; the category part data is cropped to establish a classification data set; the backbone of the self-supervised model is analyzed and the phase part parameters corresponding to the backbone are frozen according to the analysis results; the self-supervised model is classified and trained according to the frozen backbone combined with the classification data set; the heat map attention corresponding to the self-supervised model is viewed through visualization and the analysis results are generated. Based on the self-supervised pre-training model, combined with the visual classification model, model fine-tuning, and model heat map visualization, the technical effect of visualizing the model training effect is achieved.

[0105] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0106] In addition, for technical details not described in detail in this embodiment, please refer to the model attention visualization method provided in any embodiment of the present application, which will not be repeated here.

[0107] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0108] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0110] The above are only preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A model attention visualization method, characterized in that: include: Obtaining a self-supervisory model and obtaining category and part data information that the self-supervisory model focuses on; Cutting the category part data to establish a classification data set; Analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results; Performing classification training on the self-supervised model according to the frozen backbone combined with the classification data set; View the heat map attention corresponding to the self-supervised model through visualization and generate analysis results; The self-supervised model adopts the SIMMIM self-supervised learning framework in MMselfsup, the backbone part of the self-supervised model is a 4-layer stage, the dim output is 1536, and the self-supervised model is a SwinTransformer-large model; The step of cutting the category part data to establish a classification data set includes: Performing part clipping on the category part data to scale them to a uniform size; Classifying the trimmed category part data according to data categories to establish a classification data set; The step of analyzing the backbone of the self-supervisory model and freezing some parameters of the stage corresponding to the backbone according to the analysis result includes: Analyze the type information of the backbone of the self-supervisory model and the stage structure of the neural network model used by the backbone to generate analysis results; Determine the number of stages that need to be frozen according to the analysis results; Freeze some parameters of the stages corresponding to the backbone according to the number of stages that need to be frozen; The step of visually viewing the heat map attention corresponding to the self-supervisory model and generating analysis results includes: Using the PyTorch-Grad-CAM visualization framework to view the heat map attention corresponding to the self-supervised model to generate viewing results; Among them, A represents the feature layer, k represents the kth channel in the feature layer A, and c represents category c. The data of channel k in feature layer A, Representative for The weight of , ReLU is a linear rectification function; An analysis result is generated through the review result.

2. The model attention visualization method according to claim 1, characterized in that After the step of determining the number of stages to be frozen according to the analysis result, the method further includes: Build a classification network; Analyze the corresponding output matrix dimension according to the backbone type information; Determine the classification head input according to the output matrix dimension; The classification network is updated according to the classification head input and the backbone.

3. The model attention visualization method according to claim 2, characterized in that: The step of performing classification training on the self-supervisory model according to the frozen backbone combined with the classification data set includes: Obtaining an MMPretrain training framework, and configuring the classification data set on the MMPretrain training framework; In the case of the frozen backbone, classification training is performed on the classification network in the self-supervised model.

4. A model attention visualization system, characterized in that: Executing the method according to claim 1, comprising: An information acquisition module, used to acquire a self-supervisory model and obtain category and part data information that the self-supervisory model focuses on; A classification data set module, used for cutting the category part data to establish a classification data set; A freezing module, used to analyze the backbone of the self-supervised model and freeze some parameters of the stage corresponding to the backbone according to the analysis results; A classification training module, used for performing classification training on the self-supervised model according to the frozen backbone combined with the classification data set; The analysis result module is used to visualize the heat map attention corresponding to the self-supervised model and generate analysis results.

5. A model attention visualization device, characterized in that: The device comprises: a memory and a processor, and the processor executes the method as described in any one of claims 1 to 3 when running the model attention visualization program stored in the memory.

6. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 3.

Images