Class activation graph generation method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence and research and development, and relates to a class activation graph generation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining each feature graph generated by a convolution layer; binarizing the pixel value of each pixel in the feature map by adopting a sign function to obtain a target binarized feature map; calculating a weight coefficient of the feature map according to the total number value of each target binary feature map; performing weighting processing on the target binary feature map according to the weight coefficient, performing superposition on each weighted feature map, and performing processing by adopting an activation function to obtain a class activation map; and performing up-sampling on the class activation graph to the original size of the input image, and superposing the up-sampled class activation graph on the input image to generate a class activation graph visualization result. In addition, the invention also relates to a block chain technology, and data such as a feature graph and a class activation graph can be stored in a block chain. According to the method, the influence of noise on class activation graph generation can be reduced, and the attention capability on important areas is enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and research and development technology, and in particular to a method, device, computer equipment and storage medium for generating a class activation map. Background Art

[0002] Class activation maps play an important role in the field of deep learning and computer vision. They are mainly used to visualize the areas of attention of deep learning models for specific categories. The generated class activation map can clearly show which parts of the image the model pays most attention to when making predictions. Video surveillance is widely used in various large business scenarios of enterprises. In these surveillance videos, class activation map technology can play an important role in helping to identify abnormal behavior or intrusion events. By visualizing the model's attention to specific areas, security personnel can locate potential threats more quickly. In addition, class activation maps can ensure that the monitoring system pays enough attention to sensitive areas, thereby further enhancing customer privacy protection. However, there is a significant problem with the current class activation map technology, that is, it is easily interfered by a large amount of indistinguishable noise, resulting in the lack of effectiveness of the generated heat map, especially for small or uncommon objects, which are often ignored.

[0003] Therefore, it is necessary to improve and optimize the class activation map technology to improve its anti-interference ability and accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a class activation map generation method, device, computer equipment and storage medium, which can reduce the impact of noise on the generation of class activation maps and enhance the ability to focus on important areas, thereby improving the anti-interference ability of class activation map generation.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a method for generating a class activation map, which adopts the following technical solution:

[0006] Obtain each feature map generated by a convolutional layer in a convolutional neural network model, wherein the feature map is obtained by extracting features of an input image based on the convolutional neural network model;

[0007] Using a predefined symbol function to binarize the pixel value of each pixel in the feature map to generate a target binary feature map;

[0008] Calculate the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps;

[0009] Performing weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map;

[0010] Each of the weighted feature maps is superimposed and processed using a preset activation function to obtain a class activation map.

[0011] In order to solve the above technical problems, the embodiment of the present application further provides a class activation map generation device, which adopts the following technical solution:

[0012] An acquisition module, used to acquire each feature map generated by a convolutional layer in a convolutional neural network model, wherein the feature map is obtained by extracting features of an input image based on the convolutional neural network model;

[0013] A processing module, used for binarizing the pixel value of each pixel in the feature map using a predefined symbol function to generate a target binary feature map;

[0014] A calculation module, used for calculating the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps;

[0015] A weighting module, used for performing weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map;

[0016] The superposition module is used to superimpose each of the weighted feature maps and process them using a preset activation function to obtain a class activation map.

[0017] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, which adopts the following technical solution: the computer device includes a memory and a processor, the memory stores computer-readable instructions, and the processor implements the steps of the class activation map generation method as described in any one of the above items when executing the computer-readable instructions.

[0018] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which adopts the following technical solution: the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the class activation map generation method as described in any one of the above items are implemented.

[0019] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0020] The embodiment scheme of the present application extracts features from the input image through the convolution layer of the convolutional neural network model to generate a series of feature maps. By using a symbol function to binarize each pixel value in the feature map, the complexity of the feature map can be reduced, and the anti-interference ability of the class activation map to noise can be enhanced. The weight coefficient of each feature map is calculated according to the total value of each target binary feature map, and the importance of the feature map in generating the class activation map can be reflected by the weight coefficient. By weighting each target binary feature map according to the weight coefficient, a weighted feature map is obtained, and these weighted feature maps are superimposed, and a preset activation function is used for processing, and the information of different feature maps can be integrated to generate a class activation map. The present application scheme can highlight the important features of the model by binarizing the feature map, reduce unnecessary details and noise, and improve the anti-interference ability and accuracy of the class activation map generation. The class activation map is generated by weighting, superimposing, etc. the target binary feature map, further enhancing the ability to focus on important areas in the class activation map, making the area of ​​the model more intuitive and easy to understand. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0023] Figure 2 It is a flowchart of an embodiment of the method for generating a class activation graph of the present application;

[0024] Figure 3 It is a structural schematic diagram of an embodiment of a class activation graph generating device of the present application;

[0025] Figure 4 It is a structural diagram of an embodiment of a computer device of the present application. DETAILED DESCRIPTION

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0027] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0029] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0030] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0031] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0032] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0033] It should be noted that the class activation map generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the class activation map generation device is generally set in the server / terminal device.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0035] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating a class activation map according to the present application. The class activation map generation method comprises the following steps:

[0036] Step S201, obtaining each feature map generated by the convolutional layer in the convolutional neural network model, wherein the feature map is obtained by extracting features of an input image based on the convolutional neural network model;

[0037] In this embodiment, the class activation map generation method is executed on the electronic device (eg Figure 1 The server / terminal device shown in the figure can obtain each feature map generated by the convolution layer in the convolutional neural network model through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 2G / 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0038] In this embodiment, a convolutional neural network (CNN) model is pre-constructed, a convolutional layer of a specific layer in the convolutional neural network model is defined as a convolutional layer for obtaining a feature map, and an identifier of the convolutional layer is recorded. The specific layer may be the last layer or another layer. The deep learning framework of the convolutional neural network model may be implemented using the symbolic mathematics system TensorFlow, the open source deep learning framework PyTorch, and the like.

[0039] Specifically, a pre-collected input image is obtained, the input image is forward propagated through a convolutional neural network model, and the input image is feature extracted through the convolutional layer in the convolutional neural network model to obtain a series of feature maps. The feature map is composed of a three-dimensional array (height × width × number of channels), and each element in the three-dimensional array represents the pixel value at the corresponding position in the feature map. The convolutional layer is located according to the recorded identifier, and the feature map is extracted from the convolutional layer through a preset hook function or a specific interface defined in the model.

[0040] For example, the acquired feature map can be expressed as A k , where k represents the index of the feature map. For the input image I, the feature map A output by the convolutional neural network model k for:

[0041] A k ∈I H×W ;

[0042] Among them, H is the height of the feature map and W is the width of the feature map.

[0043] Step S202, using a predefined sign function to binarize the pixel value of each pixel in the feature map to generate a target binary feature map;

[0044] In this embodiment, a predefined sign function is obtained, which is a function for binarizing an input value into a preset value. The preset value can be defined as any value among -1, 0 and 1. The sign function determines the preset value of the output by judging whether the input value exceeds a specific threshold. The pixel value of each pixel in all feature maps is traversed, the pixel value is input into the sign function, the input pixel value is compared with the specific threshold through the sign function, and the pixel value is converted into the corresponding preset value according to the comparison result to realize the binarization of the pixel value. After all pixel values ​​of all feature maps are binarized, a target binary feature map corresponding to each feature map is generated.

[0045] In some optional implementations, the above-mentioned use of a predefined sign function to binarize the pixel value of each pixel in the feature map to generate a target binary feature map may include the following steps:

[0046] Creating an initial binary feature map of the same size as the feature map;

[0047] Specifically, for each feature map, an initial binary feature map of exactly the same size as each feature map is created. The initial binary feature map is used to store the result after the binarization process. Initially, the initial binary feature map can be an all-zero matrix, or initialized to other values ​​according to specific requirements.

[0048] Obtaining a predefined symbolic function, where the symbolic function is a function used to binarize an input value into a preset value;

[0049] Specifically, a predefined sign function is obtained, which is a function used to binarize the input value (i.e., the pixel value in the feature map) into a preset value. The preset value can be defined as any value among -1, 0 and 1. The sign function determines the preset value of the output by judging whether the input value exceeds a specific threshold. For example, when the input value exceeds a specific threshold, the corresponding output preset value is 1; when the input value does not exceed a specific threshold, the corresponding output preset value is 0.

[0050] Determining a color type of the feature map, and setting a binarization processing channel according to the color type and the sign function;

[0051] Specifically, the color type of each feature map is determined, and the color type may include a grayscale map and a color map. The binarization processing channel is set according to the color type and the sign function of the feature map. For example, when the color type of the feature map is a grayscale map, the binarization processing channel is determined as a single channel, and the sign function is applied to the single channel; when the color type of the feature map is a color map, the binarization processing channel is determined as multiple channels, and the sign function is applied to each channel respectively.

[0052] Obtaining the pixel value of each pixel in the feature map by traversing;

[0053] Specifically, a traversal method is adopted to access the feature map one by one and obtain the pixel value of each pixel in the feature map.

[0054] The pixel value is input into the binarization processing channel, and the pixel value is stored in the corresponding position of the initial binarization feature map using the binarization processing channel to generate a target binarization feature map.

[0055] Specifically, for each pixel value traversed, the pixel value is input into the corresponding binarization processing channel, the pixel value is compared with a specific threshold value through the sign function set by the binarization processing channel, the pixel value is converted into a corresponding preset value according to the comparison result, and the converted preset value is stored in the corresponding position of the initial binarization feature map. When all pixels are binarized and stored in the initial binarization feature map, a target binarization feature map is obtained. The target binarization feature map contains a binarization representation of the original feature map.

[0056] For example, for feature map A k Each pixel in Apply the symbolic function to generate the target binary feature map B k :

[0057]

[0058] Among them, i is the feature map A k The i-th row in the graph, j is the feature map A k The jth column in is the preset value of the i-th row and j-th column in the target binary feature map, It is composed of feature map A k The pixel value of the i-th row and j-th column in is converted into a binary value. The sign function sgn(x) can be defined as:

[0059]

[0060] That is, the specific threshold in the sign function is set to 0; the value range of the preset value is -1, 0, 1.

[0061] The embodiment of the present application ensures the consistency of the output and input feature maps in the spatial dimension by creating an initial binary feature map of the same size as the feature map; the binary processing channel is set according to the color type and the sign function, and the independent or joint processing of different color channels can be realized, thereby enhancing the flexibility and accuracy of the processing. By combining the binary processing channel of the sign function, the pixel values ​​in the feature map are binarized to generate the target binary feature map, which helps to reduce data complexity while retaining key information.

[0062] Step S203, calculating a weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps;

[0063] In this embodiment, a traversal method is used to obtain all preset thresholds on the target binary feature map, and each preset threshold is added to obtain the total value of the target binary feature map. The same method is used to calculate all target binary feature maps to obtain the total value of each target binary feature map. Optionally, the total values ​​of each target binary feature map are collected to form a value list or array. According to the total values ​​of each target binary feature map, the weight coefficient of each feature map is calculated. For example, the weight coefficient of each feature map can be calculated by normalization method, average method or direct proportional method.

[0064] In some optional implementations, the calculation of the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps may include the following steps:

[0065] Obtaining the total value of each target binary feature map by traversal method;

[0066] Specifically, a traversal method is adopted to obtain all preset thresholds in the target binary feature map one by one, and each preset threshold is added to obtain the total value of the target binary feature map. The same method is used to calculate each target binary feature map to obtain the total value of each target binary feature map.

[0067] Optionally, the total values ​​of each target binary feature map are collected to form a list or array of values.

[0068] Adding the total values ​​of each target binary feature map to obtain the sum of the total values;

[0069] Specifically, the total values ​​of each target binary feature map collected in the value list or array are added together to obtain the sum of the total values.

[0070] Calculating the normalization factor corresponding to each target binary feature map according to the total value of each target binary feature map and the sum of the total value;

[0071] Specifically, according to the total value and the sum of the total values ​​of each target binary feature map, the normalization factor corresponding to each target binary feature map is calculated respectively. For example, the ratio of the total value of each target binary feature map to the sum of the total values ​​may be taken as the normalization factor corresponding to each target binary feature map.

[0072] According to the total value of each target binary feature map and the normalization factor, the weight coefficient of each feature map is calculated respectively.

[0073] Specifically, the weight coefficient of each feature map is calculated according to the total value of each target binary feature map and the normalization factor obtained through the normalization process. The weight coefficient is obtained by dividing the total value of each target binary feature map by the sum of the total values, and the weight coefficient reflects the importance of each feature map in the overall.

[0074] The embodiment of the present application adds up the total values ​​of all target binary feature maps to obtain the sum of the total values, so that the overall distribution of all feature maps can be known. By calculating the normalization factor corresponding to each feature map based on the total value and the sum of the total values ​​of each target binary feature map, the normalization process helps to eliminate the deviation caused by the difference in numerical ranges between different feature maps, making the calculation of weight coefficients more fair and accurate. By calculating the weight coefficient of each feature map based on the total value and normalization factor of each target binary feature map, a quantitative evaluation of the importance of multiple feature maps is achieved by calculating the weight coefficient, which helps to perform targeted processing or fusion according to the importance of the feature map in subsequent image processing tasks, thereby improving the efficiency and accuracy of image processing.

[0075] In some optional implementations, the above-mentioned calculation of the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps may further include the following steps:

[0076] Obtaining the total value of each target binary feature map by traversal method;

[0077] Specifically, a traversal method is adopted to obtain all preset thresholds in the target binary feature map one by one, and each preset threshold is added to obtain the total value of the target binary feature map. The same method is used to calculate each target binary feature map to obtain the total value of each target binary feature map.

[0078] Optionally, the total values ​​of each target binary feature map are collected to form a list or array of values.

[0079] Obtaining the total number of pixels of each of the feature maps;

[0080] Specifically, the number of all pixels in each feature map is obtained as the total number of pixels. Alternatively, the size information of each feature map is obtained, and the total number of pixels of the feature map is calculated according to the size information.

[0081] The total value of each target binary feature map is divided by the total number of pixels of each feature map to obtain an average value as the weight coefficient of each feature map.

[0082] Specifically, for each target binary feature map, the total value of the target binary feature map is divided by the total number of pixels of the target binary feature map, and the average value is calculated. The average value is used as the weight coefficient of each feature map, and the weight coefficient reflects the density or proportion of activated pixels in the feature map. The higher the weight coefficient of a feature map, the higher the proportion of its activated pixels, which means that the feature map is more important in the overall feature representation.

[0083] For example, by binarizing the target feature map B k All preset values ​​are averaged to calculate the weight coefficient of each feature map

[0084]

[0085] Among them, Z is the total number of pixels in the feature map.

[0086] The embodiment of the present application traverses the total value of each target binary feature map, obtains the total number of pixels of each feature map, divides the total value of each target binary feature map by the total number of pixels of the corresponding feature map, and obtains the average value as the weight coefficient, thereby realizing the conversion from absolute value to relative proportion, so that the weight coefficient can more accurately reflect the density or proportion of activated pixels in the feature map.

[0087] Step S204, performing weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map;

[0088] In this embodiment, each target binary feature map is weighted according to a weight coefficient. For example, for each target binary feature map, each element (ie, a preset value) therein is multiplied by a weight coefficient to obtain a weighted feature map.

[0089] In some optional implementations, the calculation formula of the weighted feature map includes:

[0090]

[0091] Among them, A k is the feature map, Weighted A k is the weighted feature map, is the weight coefficient of the feature map, B k is the target binary feature map.

[0092] Specifically, the weight coefficient of each feature map is multiplied by each target binary feature map to perform weighted processing on each target binary feature map to obtain a weighted feature map, wherein the weight coefficient matches the dimension and number of the target binary feature map.

[0093] Step S205, superimposing each of the weighted feature maps, and processing them using a preset activation function to obtain a class activation map;

[0094] In this embodiment, each weighted feature map is superimposed, and the superposition operation is performed on the spatial dimension (height × width) of the weighted feature map, that is, the values ​​of each weighted feature map at the same spatial position (i.e., the values ​​at the same height and width) are summed to obtain a superimposed comprehensive feature map. The superimposed comprehensive feature map is processed using a preset activation function to generate a class activation map. Among them, the role of the activation function is to perform a nonlinear transformation on the superimposed comprehensive feature map to generate a class activation map. The activation function may include a ReLU (Rectified Linear Unit) function, a Sigmoid function, or a Tanh function, etc.

[0095] In some optional implementations, the above-mentioned superposition of each weighted feature map and processing with a preset activation function to obtain a class activation map may include the following steps:

[0096] Superimposing the values ​​of each weighted feature map at the same spatial position to obtain a comprehensive feature map;

[0097] Specifically, for each spatial position (ie, each combination of height × width), the values ​​of each weighted feature map at the spatial position are traversed, and the values ​​of all weighted feature maps at the same spatial position are superimposed to obtain a comprehensive feature map.

[0098] A preset activation function is obtained, the comprehensive feature map is input into the activation function, and the comprehensive feature map is nonlinearly transformed using the activation function to obtain a class activation map.

[0099] Specifically, a preset activation function is obtained, which may include a ReLU (Rectified Linear Unit) function, a Sigmoid function, or a Tanh function, etc. The comprehensive feature map is passed as input to the activation function, and each value in the comprehensive feature map is traversed by the activation function, and each value is nonlinearly transformed to obtain a class activation map. The class activation map is composed of two-dimensional values ​​(or matrices), and its size is the same as that of the comprehensive feature map. Each value in the two-dimensional array is the result of processing by the activation function, which represents the degree of activation of each spatial position for a specific category.

[0100] The embodiment of the present application superimposes the values ​​of each weighted feature map at the same spatial position, and uses an activation function to perform a nonlinear transformation on the superimposed comprehensive feature map to obtain a class activation map, thereby mapping the input features to the output space through the activation function and introducing nonlinear factors, thereby enhancing the expression and generalization capabilities of the model. In this embodiment, by performing a nonlinear transformation on the comprehensive feature map through a preset activation function, a more abstract and advanced feature representation can be further extracted.

[0101] In some optional implementations, after the step of superimposing each of the weighted feature maps and processing them using a preset activation function to obtain a class activation map, the following steps may be included:

[0102] Obtaining the original size of the input image;

[0103] Using a predefined upsampling algorithm, the size of the class activation map is adjusted to match the original size of the input image, so as to obtain an upsampled class activation map;

[0104] In this embodiment, the original size of the input image is obtained, and the original size includes width and height. Since the class activation map is generated on the feature map (the output after processing by the convolutional neural network), the size of the feature map and the class activation map is usually smaller than the original size of the input image. In order to align the class activation map with the input image, a predefined upsampling algorithm is used to adjust the size of the class activation map to match the original size of the input image, and obtain the upsampled class activation map. Among them, the upsampling algorithm can include bilinear interpolation, bicubic interpolation, nearest neighbor interpolation, etc.

[0105] An image overlay strategy is determined, and the upsampled class activation map is overlaid with the input image according to the image overlay strategy to obtain a class activation map visualization result.

[0106] Specifically, an image overlay strategy is determined, and the image overlay strategy may include a color mapping strategy and a transparency setting strategy. The color mapping strategy is to map the values ​​in the class activation map to a color code, such as a heatmap, in which higher values ​​are represented by warmer colors and lower values ​​are represented by cooler colors. The transparency setting strategy is used to set the transparency of the class activation map to simultaneously display the content of the input image and the activation area of ​​the class activation map. According to the determined image overlay strategy, the upsampled class activation map is superimposed with the input image to obtain a class activation map visualization result, for example, each value in the upsampled class activation map is converted to a corresponding color according to the color mapping strategy; the transparency of the upsampled class activation map is converted according to the transparency setting strategy; after the color and transparency conversion, the upsampled class activation map is aligned and superimposed with the input image to obtain a class activation map visualization result.

[0107] The embodiment of the present application ensures the consistency in size between the class activation map and the input image by upsampling the preliminary class activation map to the original size of the input image, and generates a class activation map visualization result by superimposing the upsampled class activation map on the input image, which can intuitively display the image area that the model focuses on when identifying specific categories.

[0108] The embodiment of the present application uses the above scheme, specifically extracts features from the input image through the convolution layer of the convolutional neural network model, generates a series of feature maps, and uses a symbol function to binarize each pixel value in the feature map, which can reduce the complexity of the feature map and enhance the anti-interference of the class activation map to noise. The weight coefficient of each feature map is calculated according to the total value of each target binary feature map, and the importance of the feature map in generating the class activation map can be reflected by the weight coefficient. By weighting each target binary feature map according to the weight coefficient, a weighted feature map is obtained, and these weighted feature maps are superimposed, and a preset activation function is used for processing, and the information of different feature maps can be integrated to generate a class activation map. The present application scheme can highlight the important features of the model by binarizing the feature map, reduce unnecessary details and noise, and improve the anti-interference ability and accuracy of the class activation map generation. The class activation map is generated by weighting and superimposing the target binary feature map, which further enhances the ability to focus on important areas in the class activation map, making the area of ​​the model more intuitive and easy to understand.

[0109] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned input images, feature maps, class activation maps and other information as well as other related data, the above-mentioned input images, feature maps, class activation maps and other information as well as other related data can also be stored in a node of a blockchain.

[0110] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0112] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0113] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a class activation map generation device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0114] like Figure 3 As shown, the class activation map generating device 400 described in this embodiment includes: an acquisition module 401, a processing module 402, a calculation module 403, a weighting module 404 and a superposition module 405.

[0115] in:

[0116] An acquisition module 401 is used to acquire each feature map generated by a convolutional layer in a convolutional neural network model, wherein the feature map is obtained by extracting features from an input image based on the convolutional neural network model;

[0117] The processing module 402 is used to perform binarization processing on the pixel value of each pixel in the feature map using a predefined sign function to generate a target binary feature map;

[0118] A calculation module 403 is used to calculate a weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps;

[0119] A weighting module 404 is used to perform weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map;

[0120] The superposition module 405 is used to superimpose each of the weighted feature maps and process them using a preset activation function to obtain a class activation map.

[0121] In this embodiment, a convolutional neural network (CNN) model is pre-constructed, a convolutional layer of a specific layer in the convolutional neural network model is defined as a convolutional layer for obtaining a feature map, and an identifier of the convolutional layer is recorded. The specific layer may be the last layer or another layer. The deep learning framework of the convolutional neural network model may be implemented using the symbolic mathematics system TensorFlow, the open source deep learning framework PyTorch, and the like.

[0122] Specifically, the acquisition module 401 is used to acquire a pre-collected input image, forward propagate the input image through a convolutional neural network model, and extract features of the input image through the convolutional layer in the convolutional neural network model to obtain a series of feature maps. The feature map is composed of a three-dimensional array (height × width × number of channels), and each element in the three-dimensional array represents the pixel value of the corresponding position in the feature map. The convolutional layer is located according to the recorded identifier, and the feature map is extracted from the convolutional layer through a preset hook function or a specific interface defined in the model.

[0123] For example, the acquired feature map can be expressed as A k , where k represents the index of the feature map. For the input image I, the feature map A output by the convolutional neural network model k for:

[0124] A k ∈I H×W ;

[0125] Among them, H is the height of the feature map and W is the width of the feature map.

[0126] Specifically, the processing module 402 is used to obtain a predefined sign function, which is a function used to binarize the input value into a preset value. The preset value can be defined as any value among -1, 0 and 1. The sign function determines the preset value of the output by judging whether the input value exceeds a specific threshold. The pixel value of each pixel in all feature maps is traversed, the pixel value is input to the sign function, the input pixel value is compared with the specific threshold through the sign function, and the pixel value is converted into the corresponding preset value according to the comparison result to realize the binarization of the pixel value. After all pixel values ​​of all feature maps are binarized, a target binary feature map corresponding to each feature map is generated.

[0127] Specifically, the calculation module 403 is used to obtain all preset thresholds on the target binary feature map by traversal, and each preset threshold is added to obtain the total value of the target binary feature map. The same method is used for all target binary feature maps to obtain the total value of each target binary feature map. Optionally, the total values ​​of each target binary feature map are collected to form a value list or array. According to the total values ​​of each target binary feature map, the weight coefficient of each feature map is calculated, for example, the weight coefficient of each feature map can be calculated by normalization method, average value method or direct proportional method.

[0128] Specifically, the weighting module 404 is used to perform weighted processing on each target binary feature map according to the weight coefficient. For example, for each target binary feature map, each element (ie, a preset value) therein is multiplied by the weight coefficient to obtain a weighted feature map.

[0129] Specifically, the superposition module 405 is used to superimpose each weighted feature map, and the superposition operation is performed on the spatial dimension (height × width) of the weighted feature map, that is, the values ​​of each weighted feature map at the same spatial position (i.e., the values ​​at the same height and width) are summed to obtain a superimposed comprehensive feature map. The superimposed comprehensive feature map is processed using a preset activation function to generate a class activation map. Among them, the role of the activation function is to perform a nonlinear transformation on the superimposed comprehensive feature map to generate a class activation map. The activation function may include a ReLU (Rectified Linear Unit) function, a Sigmoid function, or a Tanh function, etc.

[0130] The class activation map generation device 400 of the embodiment of the present application extracts features from the input image through the convolution layer of the convolutional neural network model to generate a series of feature maps. By using a sign function to binarize each pixel value in the feature map, the complexity of the feature map can be reduced, and the anti-interference ability of the class activation map to noise can be enhanced. The weight coefficient of each feature map is calculated according to the total value of each target binary feature map, and the importance of the feature map in generating the class activation map can be reflected by the weight coefficient. By weighting each target binary feature map according to the weight coefficient, a weighted feature map is obtained, and these weighted feature maps are superimposed, and processed by a preset activation function, the information of different feature maps can be integrated to generate a class activation map. The present application scheme can highlight the important features of the model by binarizing the feature map, reduce unnecessary details and noise, and improve the anti-interference ability and accuracy of the class activation map generation. The class activation map is generated by weighting and superimposing the target binary feature map, which further enhances the ability to focus on important areas in the class activation map, making the area of ​​the model more intuitive and easy to understand.

[0131] In some optional implementations of this embodiment, the processing module 402 may include a creation submodule, a function acquisition submodule, a setting submodule, a first traversal submodule, and a generation submodule.

[0132] Creating a submodule for creating an initial binary feature map of the same size as the feature map;

[0133] A function acquisition submodule, used to acquire a predefined symbolic function, wherein the symbolic function is a function used to binarize an input value into a preset value;

[0134] Setting a submodule, used to determine the color type of the feature map, and setting a binarization processing channel according to the color type and the sign function;

[0135] The first traversal submodule is used to obtain the pixel value of each pixel in the feature map by traversal;

[0136] A generating submodule is used to input the pixel value into the binarization processing channel, use the binarization processing channel to store the pixel value correspondingly to the corresponding position of the initial binarization feature map, and generate a target binarization feature map.

[0137] Specifically, a submodule is created for creating, for each feature map, an initial binary feature map of exactly the same size as each feature map. The initial binary feature map is used to store the result after the binary processing. Initially, the initial binary feature map can be an all-zero matrix, or initialized to other values ​​according to specific requirements.

[0138] Specifically, the function acquisition submodule is used to obtain a predefined symbol function, which is a function used to binarize the input value (i.e., the pixel value in the feature map) into a preset value. The preset value can be defined as any value between -1, 0 and 1. The symbol function determines the preset value of the output by judging whether the input value exceeds a specific threshold. For example, when the input value exceeds a specific threshold, the corresponding output preset value is 1; when the input value does not exceed a specific threshold, the corresponding output preset value is 0.

[0139] Specifically, a submodule is set to determine the color type of each feature map, and the color type may include a grayscale map and a color map. A binarization processing channel is set according to the color type and the sign function of the feature map. For example, when the color type of the feature map is a grayscale map, the binarization processing channel is determined as a single channel, and the sign function is applied to the single channel; when the color type of the feature map is a color map, the binarization processing channel is determined as multiple channels, and the sign function is applied to each channel respectively.

[0140] Specifically, the first traversal submodule is used to access the feature map one by one in a traversal manner and obtain the pixel value of each pixel in the feature map.

[0141] Specifically, the generation submodule is used to input each traversed pixel value into the corresponding binarization processing channel, compare the pixel value with a specific threshold value through the sign function set by the binarization processing channel, convert the pixel value into a corresponding preset value according to the comparison result, and store the converted preset value in the corresponding position of the initial binarization feature map. When all pixels are binarized and stored in the initial binarization feature map, a target binarization feature map is obtained. The target binarization feature map contains a binarization representation of the original feature map.

[0142] For example, for feature map A k Each pixel in Apply the symbolic function to generate the target binary feature map B k :

[0143]

[0144] Among them, i is the feature map A k The i-th row in the graph, j is the feature map A k The jth column in is the preset value of the i-th row and j-th column in the target binary feature map, It is composed of feature map A k The pixel value of the i-th row and j-th column in is converted into a binary value. The sign function sgn(x) can be defined as:

[0145]

[0146] That is, the specific threshold in the sign function is set to 0; the value range of the preset value is -1, 0, 1.

[0147] The processing module 402 of the embodiment of the present application ensures the consistency of the output and input feature maps in the spatial dimension by creating an initial binary feature map of the same size as the feature map; the binary processing channel is set according to the color type and the sign function, and the independent or joint processing of different color channels can be realized, thereby enhancing the flexibility and accuracy of the processing. By combining the binary processing channel of the sign function, the pixel values ​​in the feature map are binarized to generate a target binary feature map, which helps to reduce data complexity while retaining key information.

[0148] In some optional implementations of this embodiment, the calculation module 403 may include a second traversal submodule, an addition submodule, a normalization submodule, and a weight calculation submodule.

[0149] A second traversal submodule is used to obtain the total value of each target binary feature map by traversal;

[0150] An adding submodule, used for adding the total values ​​of each target binary feature map to obtain the sum of the total values;

[0151] A normalization submodule, used to calculate the normalization factor corresponding to each target binary feature map according to the total value of each target binary feature map and the sum of the total value;

[0152] The weight calculation submodule is used to calculate the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps and the normalization factor.

[0153] Specifically, the second traversal submodule is used to obtain all preset thresholds in the target binary feature map one by one by traversing, and add up each preset threshold to obtain the total value of the target binary feature map. The same method is used to calculate each target binary feature map to obtain the total value of each target binary feature map.

[0154] Optionally, the total values ​​of each target binary feature map are collected to form a list or array of values.

[0155] Specifically, the addition submodule is used to add the total values ​​of each target binary feature map collected in the value list or array to obtain the sum of the total values.

[0156] Specifically, the normalization submodule is used to calculate the normalization factor corresponding to each target binary feature map according to the total value and the sum of the total values ​​of each target binary feature map. For example, the ratio of the total value of each target binary feature map to the sum of the total values ​​can be used as the normalization factor corresponding to each target binary feature map.

[0157] Specifically, the weight calculation submodule is used to calculate the weight coefficient of each feature map according to the total value of each target binary feature map and the normalization factor obtained through the normalization process. The weight coefficient is obtained by dividing the total value of each target binary feature map by the sum of the total values, and the weight coefficient reflects the importance of each feature map in the overall.

[0158] The calculation module 403 of the embodiment of the present application can obtain the overall distribution of all feature maps by adding the total values ​​of all target binary feature maps to obtain the sum of the total values. By calculating the normalization factor corresponding to each feature map based on the total value and the sum of the total values ​​of each target binary feature map, the normalization process helps to eliminate the deviation caused by the difference in numerical ranges between different feature maps, making the calculation of the weight coefficient more fair and accurate. By calculating the weight coefficient of each feature map based on the total value and the normalization factor of each target binary feature map, a quantitative evaluation of the importance of multiple feature maps is achieved by calculating the weight coefficient, which helps to perform targeted processing or fusion according to the importance of the feature map in subsequent image processing tasks, thereby improving the efficiency and accuracy of image processing.

[0159] In some optional implementations of this embodiment, the calculation module 403 may include a third traversal submodule, a total number acquisition submodule, and an average calculation submodule.

[0160] A third traversal submodule is used to obtain the total value of each target binary feature map by traversal;

[0161] A total number acquisition submodule, used to obtain the total number of pixels of each feature map;

[0162] The average calculation submodule is used to divide the total value of each target binary feature map by the total number of pixels in each feature map to obtain an average value as the weight coefficient of each feature map.

[0163] Specifically, the third traversal submodule is used to obtain all preset thresholds in the target binary feature map one by one by using a traversal method, and add each preset threshold to obtain the total value of the target binary feature map. The same method is used to calculate each target binary feature map to obtain the total value of each target binary feature map.

[0164] Optionally, the total values ​​of each target binary feature map are collected to form a list or array of values.

[0165] Specifically, the total number acquisition submodule is used to obtain the number of all pixels in each feature map as the total number of pixels. Alternatively, the size information of each feature map is obtained, and the total number of pixels of the feature map is calculated according to the size information.

[0166] Specifically, the average calculation submodule is used to divide the total value of each target binary feature map by the total number of pixels of the target binary feature map to calculate the average value, and use the average value as the weight coefficient of each feature map, which reflects the density or proportion of activated pixels in the feature map. The higher the weight coefficient of a feature map, the higher the proportion of its activated pixels, which means that the feature map is more important in the overall feature representation.

[0167] For example, by binarizing the target feature map B k All preset values ​​are averaged to calculate the weight coefficient of each feature map

[0168]

[0169] Among them, Z is the total number of pixels in the feature map.

[0170] The calculation module 403 of the embodiment of the present application traverses the total value of each target binary feature map, obtains the total number of pixels of each feature map, divides the total value of each target binary feature map by the total number of pixels of the corresponding feature map, and obtains the average value as the weight coefficient, thereby realizing the conversion from absolute value to relative proportion, so that the weight coefficient can more accurately reflect the density or proportion of activated pixels in the feature map.

[0171] In some optional implementations of this embodiment, the calculation formula of the weighted feature map includes:

[0172]

[0173] Among them, A k is the feature map, Weighted A k is the weighted feature map, is the weight coefficient of the feature map, B k is the target binary feature map.

[0174] Specifically, the weight coefficient of each feature map is multiplied by each target binary feature map to perform weighted processing on each target binary feature map to obtain a weighted feature map, wherein the weight coefficient matches the dimension and number of the target binary feature map.

[0175] In some optional implementations of this embodiment, the superposition module 405 may include a third traversal submodule, a total number acquisition submodule, and an average calculation submodule.

[0176] A superposition submodule, used for superimposing the values ​​of each weighted feature map at the same spatial position to obtain a comprehensive feature map;

[0177] The transformation submodule is used to obtain a preset activation function, input the comprehensive feature map into the activation function, and use the activation function to perform a nonlinear transformation on the comprehensive feature map to obtain a class activation map.

[0178] Specifically, the superposition submodule is used to traverse the values ​​of each weighted feature map at each spatial position (i.e., each combination of height × width) and superimpose the values ​​of all weighted feature maps at the same spatial position to obtain a comprehensive feature map.

[0179] Specifically, the transformation submodule is used to obtain a preset activation function, which may include a ReLU (Rectified Linear Unit) function, a Sigmoid function, or a Tanh function. The comprehensive feature map is passed as input to the activation function, and each value in the comprehensive feature map is traversed by the activation function, and each value is nonlinearly transformed to obtain a class activation map. The class activation map is composed of two-dimensional values ​​(or matrices), and its size is the same as that of the comprehensive feature map. Each value in the two-dimensional array is the result of processing by the activation function, which represents the degree of activation of each spatial position to a specific category.

[0180] The superposition module 405 of the embodiment of the present application superimposes the values ​​of each weighted feature map at the same spatial position, and uses an activation function to perform a nonlinear transformation on the superimposed comprehensive feature map to obtain a class activation map, thereby realizing the mapping of input features to the output space through the activation function and introducing nonlinear factors, thereby enhancing the expression ability and generalization ability of the model. In this embodiment, by performing a nonlinear transformation on the comprehensive feature map through a preset activation function, a more abstract and advanced feature representation can be further extracted.

[0181] In some optional implementations of this embodiment, the class activation map generation device 400 may further include a visualization module. The visualization module is used to obtain the original size of the input image;

[0182] Using a predefined upsampling algorithm, the size of the class activation map is adjusted to match the original size of the input image, so as to obtain an upsampled class activation map;

[0183] An image overlay strategy is determined, and the upsampled class activation map is overlaid with the input image according to the image overlay strategy to obtain a class activation map visualization result.

[0184] Specifically, the visualization module is used to obtain the original size of the input image, which includes width and height. Since the class activation map is generated on the feature map (the output after processing by the convolutional neural network), the size of the feature map and the class activation map is usually smaller than the original size of the input image. In order to align the class activation map with the input image, a predefined upsampling algorithm is used to adjust the size of the class activation map to match the original size of the input image, and obtain the upsampled class activation map. Among them, the upsampling algorithm can include bilinear interpolation, bicubic interpolation, nearest neighbor interpolation, etc.

[0185] Specifically, an image overlay strategy is determined, and the image overlay strategy may include a color mapping strategy and a transparency setting strategy. The color mapping strategy is to map the values ​​in the class activation map to a color code, such as a heatmap, in which higher values ​​are represented by warmer colors and lower values ​​are represented by cooler colors. The transparency setting strategy is used to set the transparency of the class activation map to simultaneously display the content of the input image and the activation area of ​​the class activation map. According to the determined image overlay strategy, the upsampled class activation map is superimposed with the input image to obtain a class activation map visualization result, for example, each value in the upsampled class activation map is converted to a corresponding color according to the color mapping strategy; the transparency of the upsampled class activation map is converted according to the transparency setting strategy; after the color and transparency conversion, the upsampled class activation map is aligned and superimposed with the input image to obtain a class activation map visualization result.

[0186] The class activation map generating device 400 of the embodiment of the present application ensures the consistency in size between the class activation map and the input image by upsampling the preliminary class activation map to the original size of the input image, and generates a class activation map visualization result by superimposing the upsampled class activation map on the input image, which can intuitively display the image area that the model focuses on when identifying specific categories.

[0187] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0188] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 with a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0189] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0190] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the class activation map generation method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0191] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the computer-readable instructions stored in the memory 61 or process data, such as running the computer-readable instructions of the class activation map generation method.

[0192] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0193] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the class activation map generation method as described above.

[0194] The computer device, computer-readable storage medium and computer-readable instructions provided by the embodiment of the present application are executed by the processor through the convolution layer of the convolutional neural network model to extract the features of the input image, generate a series of feature maps, and use the symbol function to binarize each pixel value in the feature map, so as to reduce the complexity of the feature map and enhance the anti-interference of the class activation map to noise. The weight coefficient of each feature map is calculated according to the total value of each target binary feature map, and the importance of the feature map in generating the class activation map can be reflected by the weight coefficient. By weighting each target binary feature map according to the weight coefficient, a weighted feature map is obtained, and these weighted feature maps are superimposed, and the preset activation function is used for processing, so that the information of different feature maps can be integrated to generate a class activation map. The present application scheme can highlight the important features concerned by the model by binarizing the feature map, reduce unnecessary details and noise, and improve the anti-interference ability and accuracy of the class activation map generation. The class activation map is generated by weighting, superimposing and other methods of the target binary feature map, which further enhances the ability to focus on important areas in the class activation map, making the area concerned by the model more intuitive and easy to understand.

[0195] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0196] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is also within the scope of patent protection of this application.

[0197] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A method for generating a class activation map, characterized in that: The steps include: Obtain each feature map generated by a convolutional layer in a convolutional neural network model, wherein the feature map is obtained by extracting features of an input image based on the convolutional neural network model; Using a predefined symbol function to binarize the pixel value of each pixel in the feature map to generate a target binary feature map; Calculate the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps; Performing weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map; Each of the weighted feature maps is superimposed and processed using a preset activation function to obtain a class activation map.

2. The class activation map generation method according to claim 1, characterized in that: The step of using a predefined sign function to binarize the pixel value of each pixel in the feature map to generate a target binary feature map comprises: Creating an initial binary feature map of the same size as the feature map; Obtaining a predefined symbolic function, where the symbolic function is a function used to binarize an input value into a preset value; Determining a color type of the feature map, and setting a binarization processing channel according to the color type and the sign function; Obtaining the pixel value of each pixel in the feature map by traversing; The pixel value is input into the binarization processing channel, and the pixel value is stored in the corresponding position of the initial binarization feature map using the binarization processing channel to generate a target binarization feature map.

3. The class activation map generation method according to claim 1, characterized in that: The step of calculating the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps comprises: Obtaining the total value of each target binary feature map by traversal method; Adding the total values ​​of each target binary feature map to obtain the sum of the total values; Calculating the normalization factor corresponding to each target binary feature map according to the total value of each target binary feature map and the sum of the total value; According to the total value of each target binary feature map and the normalization factor, the weight coefficient of each feature map is calculated respectively.

4. The class activation map generation method according to claim 1, characterized in that: The step of calculating the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps comprises: Obtaining the total value of each target binary feature map by traversal method; Obtaining the total number of pixels of each of the feature maps; The total value of each target binary feature map is divided by the total number of pixels of each feature map to obtain an average value as the weight coefficient of each feature map.

5. The class activation map generation method according to claim 1, characterized in that: The calculation formula of the weighted feature map includes: Among them, A k is the feature map, Weighted A k is the weighted feature map, is the weight coefficient of the feature map, B k is the target binary feature map.

6. The class activation map generation method according to claim 1, characterized in that: The step of superimposing each of the weighted feature maps and processing them using a preset activation function to obtain a class activation map includes: Superimposing the values ​​of each weighted feature map at the same spatial position to obtain a comprehensive feature map; A preset activation function is obtained, the comprehensive feature map is input into the activation function, and the comprehensive feature map is nonlinearly transformed using the activation function to obtain a class activation map.

7. The class activation map generation method according to claim 1, characterized in that: After the step of superimposing each of the weighted feature maps and processing them using a preset activation function to obtain a class activation map, the method further includes: Obtaining the original size of the input image; Using a predefined upsampling algorithm, the size of the class activation map is adjusted to match the original size of the input image, so as to obtain an upsampled class activation map; An image overlay strategy is determined, and the upsampled class activation map is overlaid with the input image according to the image overlay strategy to obtain a class activation map visualization result.

8. A class activation map generating device, characterized in that: The device comprises: An acquisition module, used to acquire each feature map generated by a convolutional layer in a convolutional neural network model, wherein the feature map is obtained by extracting features of an input image based on the convolutional neural network model; A processing module, used for binarizing the pixel value of each pixel in the feature map using a predefined symbol function to generate a target binary feature map; A calculation module, used for calculating the weight coefficient of each of the feature maps according to the total value of each of the target binary feature maps; A weighting module, used for performing weighted processing on each of the target binary feature maps according to the weight coefficient to obtain a weighted feature map; The superposition module is used to superimpose each of the weighted feature maps and process them using a preset activation function to obtain a class activation map.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the class activation map generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the class activation map generation method according to any one of claims 1 to 7 are implemented.

Images