A self-enhancing method and apparatus for single-class target perception based on gradient-guided feature activation

Abstract

This invention discloses a gradient-guided feature activation-based self-enhanced method for single-class target perception, comprising: determining the target category to be detected in a target image; inputting the target image into a target detection network for a first forward propagation to obtain the prediction score of the target category corresponding to the prediction layer of the target detection network; performing gradient backpropagation on the prediction scores to obtain the gradient information of the feature map of the target image corresponding to each prediction layer; determining the channel weights and correlation coefficients of the feature map corresponding to each prediction layer based on the gradient information; determining the final channel weights of the feature map based on the channel weights and correlation coefficients; weighting the feature map corresponding to the prediction layer using the final channel weights to obtain the enhanced feature map; and inputting the enhanced feature map into the target detection network for a second forward propagation to obtain the enhanced target detection result. Thus, without retraining the network, the method improves the ability to recognize specific target categories while reducing the additional cost of retraining the network.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, specifically relating to a target detection method, device, electronic device, and computer-readable storage medium. Background Technology

[0002] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that incorporate convolutional computations and have a deep structure. They are one of the representative algorithms in deep learning. Due to their excellent feature extraction capabilities, CNNs have become the primary choice for current object detection algorithms. During the training of multi-class object detection networks, CNNs typically focus on features from multiple classes simultaneously. However, when performing detection tasks, in order to balance the performance of all-class detection, the feature information of certain single classes is often suppressed by the feature information of other classes. This leads to a decrease in the network's detection performance when focusing on a specific class.

[0003] Current methods mainly involve attention-based adjustment strategies and class balancing techniques, or data augmentation of the training set. However, existing solutions all require adjusting the network structure and retraining, inevitably incurring additional costs and exhibiting low efficiency in identifying specific target categories. Summary of the Invention

[0004] To address the aforementioned problems in related technologies, this invention provides a single-class target perception self-enhancement method based on gradient-guided feature activation. The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] In a first aspect, the present invention provides a single-class target perception self-enhancement method based on gradient-guided feature activation, comprising: determining the target category to be detected in a target image; inputting the target image into a target detection network for a first forward propagation to obtain the prediction score of the target category corresponding to the prediction layer of the target detection network; wherein the target detection network includes multiple prediction layers; performing gradient backpropagation on the prediction scores to obtain gradient information of the feature map of the target image corresponding to each prediction layer; determining the channel weights and correlation coefficients of the feature map corresponding to each prediction layer based on the gradient information; determining the final channel weights of the feature map based on the channel weights and correlation coefficients; weighting the feature map corresponding to the prediction layer using the final channel weights to obtain an enhanced feature map; and inputting the enhanced feature map into the target detection network for a second forward propagation to obtain an enhanced target detection result.

[0006] In some possible implementations, before backpropagating the predicted scores to obtain the gradient information of the feature map of the target image corresponding to the prediction layer, the method further includes: obtaining candidate boxes and confidence scores of the candidate boxes output by the first forward propagation for locating the feature map; selecting target candidate boxes from the candidate boxes according to the target category; and selecting enhancement candidate boxes from the target candidate boxes according to the number of target candidate boxes and / or the confidence scores of the target candidate boxes.

[0007] In some possible implementations, gradient backpropagation is performed on the predicted scores to obtain gradient information of the feature map of the target image corresponding to the prediction layer. This includes: performing gradient backpropagation on the predicted scores of the target categories in the enhanced candidate boxes according to the chain rule to obtain gradient information of the feature map of the target image corresponding to the prediction layer.

[0008] In some possible implementations, the channel weights of the feature map corresponding to each prediction layer and the correlation coefficient of each channel weight are determined based on the gradient information, including: determining the mean of the gradient information of the feature map corresponding to each prediction layer in each channel; determining the channel weights of the feature map corresponding to each prediction layer based on each mean of the feature map corresponding to each prediction layer; and normalizing the maximum value of each channel weight by a preset normalization function to obtain the correlation coefficient of each channel weight.

[0009] In some possible implementations, the final channel weights are expressed as:

[0010]

[0011] Where norm() is the maximum and minimum value normalization function. The most Let ε1 be the channel weights of the feature maps corresponding to the n prediction layers, and ε2 be the channel weights of the feature maps corresponding to the n prediction layers. n Let be the correlation coefficient of the n channel weights, where n is the number of prediction layers.

[0012] In some possible implementations, before weighting the feature map corresponding to the prediction layer with the final channel weights to obtain the enhanced feature map, the method further includes: enhancing the final channel weights with a preset enhancement coefficient to obtain the enhanced final channel weights.

[0013] In some possible implementations, the feature map corresponding to the prediction layer is weighted by the final channel weights to obtain the enhanced feature map, including: enhancing each channel of the feature map by the enhanced final channel weights to obtain the enhanced feature map.

[0014] Secondly, the present invention provides a single-class target perception self-enhancing device with gradient-guided feature activation, comprising: a first determining module for determining the target category to be detected in a target image; a first forward propagation module for inputting the target image into a target detection network for a first forward propagation to obtain the prediction score of the target category corresponding to the prediction layer of the target detection network; wherein the target detection network includes multiple prediction layers; a gradient backpropagation module for performing gradient backpropagation on the prediction scores to obtain the gradient information of the feature map of the target image corresponding to each prediction layer; a second determining module for determining the channel weights and correlation coefficients of the feature map corresponding to each prediction layer based on the gradient information; a third determining module for determining the final channel weights of the feature map based on the channel weights and correlation coefficients; a weighting module for weighting the feature map corresponding to the prediction layer using the final channel weights to obtain the enhanced feature map; and a second forward propagation module for inputting the enhanced feature map into the target detection network for a second forward propagation to obtain the enhanced target detection result.

[0015] In some possible implementations, the apparatus further includes: a first acquisition module for acquiring candidate boxes and confidence scores of the candidate boxes from the first forward propagation output for locating feature maps; a first selection module for selecting target candidate boxes from the candidate boxes based on the target category; and a second selection module for selecting enhanced candidate boxes from the target candidate boxes based on the number of target candidate boxes and / or the confidence scores of the target candidate boxes.

[0016] In some possible implementations, the gradient backpropagation module is also used to backpropagate the predicted scores of the target categories in the enhanced candidate boxes according to the chain rule, so as to obtain the gradient information of the feature map of the target image corresponding to the prediction layer.

[0017] In some possible implementations, the second determining module is further configured to determine the mean of gradient information of the feature map corresponding to each prediction layer in each channel; determine the channel weight of the feature map corresponding to each prediction layer based on each mean of the feature map corresponding to each prediction layer; and normalize the maximum value of each channel weight by a preset normalization function to obtain the correlation coefficient of each channel weight.

[0018] In some possible implementations, the device further includes an enhancement module for enhancing the final channel weights using a preset enhancement coefficient to obtain enhanced final channel weights.

[0019] In some possible implementations, the weighting module is also used to enhance each channel of the feature map separately using the enhanced final channel weights to obtain an enhanced feature map.

[0020] Thirdly, the present invention provides an electronic device comprising: a memory storing computer-executable instructions; and a processor connected to the memory for executing the computer-executable instructions to implement the method as described in the first aspect of the present invention and its possible embodiments.

[0021] Fourthly, the present invention provides a computer storage medium storing computer-executable instructions, which, when executed by a processor, enable the implementation of the method as described in the first aspect of the present invention and its possible embodiments.

[0022] The beneficial effects of the technical solution provided by the embodiments of the present invention compared with the prior art are as follows:

[0023] In this invention, gradient information of the feature map of the target image corresponding to each prediction layer is obtained by backpropagating the predicted score of the target category using gradients. This gradient information is then processed to weight the feature maps corresponding to each prediction layer, resulting in enhanced feature maps. These enhanced feature maps are then input into the target detection network to obtain enhanced target detection results. Thus, by enhancing the existing network feature maps, the network structure does not need to be changed. This improves the ability to recognize specific target categories without retraining, while also reducing the additional cost of retraining the network.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a gradient-guided feature activation-based self-enhancing method for single-class target perception provided in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of a single-class target perception self-enhancement method based on gradient-guided feature activation provided in an embodiment of the present invention;

[0027] Figure 3a A feature map heatmap for object detection using existing object detection networks;

[0028] Figure 3b This is a feature map heatmap for target detection using the gradient-guided feature activation-based single-class target perception self-enhancing method provided by this invention.

[0029] Figure 4 This is a schematic diagram of the structure of a single-class target perception self-enhancing device with gradient-guided feature activation according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of an electronic device structure according to an embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0032] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0033] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0034] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0035] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0036] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that incorporate convolutional computations and have a deep structure. They are one of the representative algorithms in deep learning. Due to their excellent feature extraction capabilities, CNNs have become the primary choice for current object detection algorithms. During the training of multi-class object detection networks, CNNs typically focus on features from multiple classes simultaneously. However, when performing detection tasks, in order to balance the performance of all-class detection, the feature information of certain single classes is often suppressed by the feature information of other classes. This leads to a decrease in the network's detection performance when focusing on a specific class.

[0037] Current methods mainly involve attention-based adjustment strategies and class balancing techniques, or data augmentation of the training set. However, existing solutions all require adjusting the network structure and retraining, inevitably incurring additional costs and exhibiting low efficiency in identifying specific target categories.

[0038] In view of this, embodiments of the present invention provide a single-class target perception self-enhancement method based on gradient-guided feature activation to solve the above problems.

[0039] See Figure 1 As shown, Figure 1 This is a flowchart illustrating a gradient-guided feature activation-based single-class target perception self-enhancement method provided in an embodiment of the present invention. The following will be combined with... Figure 1 The steps shown are explained.

[0040] S101, Determine the category of the target object to be detected in the target image;

[0041] The target category refers to the category of content contained in the target image. The selection of the target category can be based on the business requirements in the actual application process. For example, if the target image contains content such as people, trees, and cars, then if the goal is to detect and identify people in the target image, the target category could be people; if the goal is to detect and identify cars in the target image, then the target category could be cars.

[0042] S102, the target image is input into the target detection network for the first forward propagation to obtain the prediction score of the target category corresponding to the prediction layer of the target detection network; wherein, the target detection network includes multiple prediction layers;

[0043] It should be noted that forward propagation is the process of obtaining output from input data through a neural network model. In this process, the input data passes through the input layer, hidden layers, and finally reaches the output layer. Each layer's computation processes the data to obtain the input for the next layer. In this embodiment of the invention, the object detection network can be a trained deep neural network. During object detection of input image data, the object detection network of this invention processes the target image data layer by layer into the neural network model. The prediction layer of the object detection network obtains feature maps of the input image data, and during forward propagation, a series of candidate boxes and their corresponding confidence scores are obtained on each feature map. The object detection network can output data including regression prediction parameters, the confidence scores of each candidate box, and prediction scores for each category within the candidate boxes. In other words, obtaining the prediction score for the target category is equivalent to obtaining the prediction score for the target category corresponding to each candidate box.

[0044] Understandably, the role of candidate boxes is to provide bounding boxes containing the target content in the feature map for subsequent classification and localization refinement. For each category of the target image, each candidate box has a corresponding prediction score; therefore, the category with the highest prediction score in each candidate box indicates that the candidate box is more likely to represent a feature of that category. In this embodiment of the invention, to identify the target category, only the candidate box with the highest prediction score for that target category can be processed.

[0045] Based on this, in some embodiments, before performing step S103 above, the method may further include the following steps:

[0046] S1021, Obtain the candidate bounding boxes and the confidence scores of the candidate bounding boxes output by the first forward propagation for locating the feature map;

[0047] S1022, Select a target candidate box from the candidate box according to the target category;

[0048] Specifically, each candidate box will obtain a prediction score for each category. Selecting a target candidate box from the candidate boxes can be done by selecting the candidate box with the highest prediction score for the target category.

[0049] S1023, Select enhanced candidate boxes from the target candidate boxes based on the number of target candidate boxes and / or the confidence level of the target candidate boxes.

[0050] Understandably, the confidence score of a candidate box refers to the degree of matching between the candidate box and the true category. The higher the confidence score, the more likely the candidate box is to be the true target category. Therefore, to increase the probability that the target candidate box is the target category, candidate boxes with higher confidence scores can be selected from the target candidate boxes as augmented candidate boxes for subsequent data processing.

[0051] For example, selecting augmented candidate boxes from the target candidate boxes could involve selecting the target candidate boxes with the highest confidence levels (top 85%) as augmented candidate boxes. Alternatively, to improve data accuracy, the target candidate boxes with the highest confidence levels (top 70%) could be selected as augmented candidate boxes.

[0052] In some embodiments, when the confidence scores of all target candidate boxes are relatively high, it indicates that each target candidate box is highly likely to be the target category. In this case, all target candidate boxes can be used as augmentation candidates. For example, if the lowest confidence score among all target candidate boxes is greater than 0.4, all target candidate boxes are used as augmentation candidates.

[0053] In addition, to reduce the possibility of inaccurate final object detection results due to an insufficient number of augmentation candidate boxes, the number of target candidate boxes can also be considered when selecting augmentation candidate boxes.

[0054] For example, in order to ensure the number of augmentation candidate boxes, when the number of target candidate boxes is less than 10, all target candidate boxes are used as augmentation candidate boxes.

[0055] S103, perform gradient backpropagation on the predicted scores to obtain the gradient information of the feature map of the target image corresponding to each prediction layer;

[0056] In some embodiments, step S103 above may include:

[0057] S1031, according to the chain rule, the predicted score of the target category in the enhanced candidate box is backpropagated to obtain the gradient information of the feature map of the target image corresponding to the prediction layer.

[0058] Understandably, in neural networks, the predicted score is calculated through forward propagation. During forward propagation, the input data is processed through a series of layers in the network. Each layer performs some calculations on the input data and then passes the result to the next layer. This process continues until a predicted score is generated. Once the predicted score is obtained, the prediction error, i.e., the loss, can be calculated. Then, using the chain rule, this prediction error is backpropagated to every layer in the network, thus obtaining the gradient of each layer—this is backpropagation.

[0059] In some embodiments, the gradient information of the feature map of the target image corresponding to the prediction layer can be represented by formula (1):

[0060]

[0061] Among them, A' n Let i represent the gradient information of feature map A corresponding to prediction layer n, where i is the height of feature map A, j is the width of feature map A, C is the number of channels in the feature map, and k is the target class. To find the sign of the partial derivative, Let be the prediction score of the k-th class in the prediction results of prediction layer n. Let i be a feature map with height i and width j.

[0062] S104, Determine the channel weights of the feature maps corresponding to each prediction layer and the correlation coefficient of each channel weight based on the gradient information;

[0063] In some embodiments, step S104 above may include:

[0064] S1041, Determine the mean gradient information of the feature map corresponding to each prediction layer in each channel;

[0065] S1042, Determine the channel weights of the feature maps corresponding to each prediction layer based on each mean of the feature maps corresponding to each prediction layer;

[0066] S1043, normalize the maximum value of each channel weight using a preset normalization function to obtain the correlation coefficient of each channel weight.

[0067] Understandably, each feature map can include multiple channels, and each channel can correspond to a category on the feature map. By averaging the gradient information of the corresponding feature map for each channel, we can obtain the average values ​​for multiple channels. Thus, the weight of each channel can include the average values ​​of multiple channels, and these average values ​​form the weight vector of the channel weights.

[0068] In some embodiments, the channel weights of the feature map corresponding to each prediction layer can be represented by formula (2):

[0069]

[0070] in, to These are the channel weights corresponding to the n prediction layers, where n is the number of prediction layers.

[0071] After obtaining the channel weights corresponding to each prediction layer through the above formula (2), the maximum value of the weight vector corresponding to each channel weight is taken, and the correlation coefficient of each channel weight is obtained by normalizing the multiple maximum values ​​through the normalization function.

[0072] In some embodiments, the normalization function can be the softmax function, and the correlation coefficient of each channel weight can be expressed by formula (3):

[0073]

[0074] in, to The softmax function, which represents the maximum channel weights of the feature maps corresponding to the n prediction layers, can be expressed by formula (4):

[0075]

[0076] Where, x1 to x n The parameters of the softmax function are x1 to x in formula (3). n That is to

[0077] S105, Determine the final channel weights of the feature map based on the channel weights and correlation coefficients;

[0078] In some embodiments, the channel weights and correlation coefficients can be weighted using the norm function to obtain the final channel weights of the feature map. The norm function can be expressed by formula (5):

[0079]

[0080] Where norm() is the maximum and minimum value normalization function, y is the parameter of the norm function, min(y) is the minimum value of y, and max(y) is the maximum value of y.

[0081] In some embodiments, the final channel weights of the feature map can be represented by formula (6):

[0082]

[0083] in, The most Let ε1 be the channel weights of the feature maps corresponding to the n prediction layers, and ε2 be the channel weights of the feature maps corresponding to the n prediction layers. n Let be the correlation coefficient of the n channel weights, where n is the number of prediction layers.

[0084] S106, The feature map corresponding to the prediction layer is weighted by the final channel weight to obtain the enhanced feature map;

[0085] Understandably, in order to increase the convergence speed of the object detection network, hyperparameters can be used as enhancement coefficients to enhance the final channel weights.

[0086] Based on this, in some embodiments, before performing the above step S106, the final channel weight can be enhanced by a preset enhancement coefficient to obtain the enhanced final channel weight.

[0087] For example, the enhanced final channel weights can be expressed by formula (7):

[0088]

[0089] Wherein, γ is a hyperparameter, and the value of γ ranges from 1 to 2.

[0090] Based on this, step S106 above may include:

[0091] Step S1061: Enhance each channel of the feature map using the enhanced final channel weights to obtain the enhanced feature map.

[0092] In some embodiments, each channel of the enhanced feature map can be represented by formula (8):

[0093]

[0094] Where C is the number of channels. Let A be the weight corresponding to channel C. C This refers to the Cth channel of feature map A.

[0095] S107, the enhanced feature map is input into the object detection network for a second forward propagation to obtain the enhanced object detection result.

[0096] In this embodiment of the invention, gradient information of the feature map of the target image corresponding to each prediction layer is obtained by backpropagating the predicted score of the target category. This gradient information is then processed to weight the feature map corresponding to each prediction layer, resulting in an enhanced feature map. This enhanced feature map is then input into the target detection network to obtain the enhanced target detection result. Thus, by enhancing the existing network feature map, the network structure does not need to be changed. This improves the ability to recognize specific target categories without retraining, while also reducing the additional cost of retraining the network.

[0097] The above method is illustrated below through a specific embodiment:

[0098] Figure 2 This is a schematic diagram of the structure of a single-class target perception self-enhancement method based on gradient-guided feature activation provided in an embodiment of the present invention. (See attached diagram.) Figure 2As shown: The target detection network includes a feature extraction backbone network and a multi-scale target detection head. After the target image is input into the target detection network, the backbone network extracts the type features of the target image, which are then fed into the target detection head for prediction, resulting in the outputs of n prediction layers, which is the first forward propagation process in step S102. Afterwards, gradient backpropagation is performed on the target category k corresponding to each prediction layer to obtain n feature maps A1 to A2. n gradient information A1' to A' n Then, the channel weights of the feature maps corresponding to the n prediction layers are used. The most and the correlation coefficients ε1 to ε of the n channel weights n Obtain the final channel weights The final channel weights are determined by the hyperparameter γ. Enhancement is performed to obtain the final enhanced channel weights. Then through Each channel of the feature map is enhanced to obtain an enhanced feature map. Finally, the enhanced feature map is input into the target detection head for a second forward propagation to obtain the enhanced detection result of the target image.

[0099] Figure 3a To perform feature map heatmaps for object detection using existing object detection networks, Figure 3b This is a feature map heatmap for target detection using the gradient-guided feature activation single-class target perception self-enhancement method provided by this invention. Figure 3b In this case, the enhancement coefficient γ is set to 1.5. (This is achieved through comparison.) Figure 3a and Figure 3b As can be seen intuitively, after the target detection method provided by this invention is enhanced, the network's overall attention is more focused on the area around the target category to be detected. The results with higher confidence are less affected, and the enhancement effect is obvious for target categories with confidence near the threshold. Moreover, after enhancement, the target categories that the network noticed but did not select in the final prediction result are restored.

[0100] In this embodiment of the invention, gradient information of the feature map of the target image corresponding to each prediction layer is obtained by backpropagating the predicted score of the target category. This gradient information is then processed to weight the feature map corresponding to each prediction layer, resulting in an enhanced feature map. This enhanced feature map is then input into the target detection network to obtain the enhanced target detection result. Thus, by enhancing the existing network feature map, the network structure does not need to be changed. This improves the ability to recognize specific target categories without retraining, while also reducing the additional cost of retraining the network.

[0101] Figure 4This is a schematic diagram of a single-class target perception self-enhancing device with gradient-guided feature activation according to an embodiment of the present invention. See also... Figure 4 As shown, the device 400 may include:

[0102] The first determining module 401 is used to determine the category of the target to be detected in the target image;

[0103] The first forward propagation module 402 is used to input the target image into the target detection network for the first forward propagation and obtain the prediction score of the target category corresponding to the prediction layer of the target detection network; wherein, the target detection network includes multiple prediction layers;

[0104] The gradient backpropagation module 403 is used to backpropagate the predicted scores to obtain the gradient information of the feature map of the target image corresponding to each prediction layer.

[0105] The second determining module 404 is used to determine the channel weights of the feature maps corresponding to each prediction layer and the correlation coefficient of each channel weight based on the gradient information.

[0106] The third determining module 405 is used to determine the final channel weights of the feature map based on the channel weights and correlation coefficients.

[0107] The weighting module 406 is used to weight the feature map corresponding to the prediction layer using the final channel weights to obtain the enhanced feature map;

[0108] The second forward propagation module 407 is used to input the enhanced feature map into the target detection network for a second forward propagation to obtain the enhanced target detection result.

[0109] In some possible implementations, the apparatus further includes: a first acquisition module for acquiring candidate boxes and confidence scores of the candidate boxes from the first forward propagation output for locating feature maps; a first selection module for selecting target candidate boxes from the candidate boxes based on the target category; and a second selection module for selecting enhanced candidate boxes from the target candidate boxes based on the number of target candidate boxes and / or the confidence scores of the target candidate boxes.

[0110] In some possible implementations, the gradient backpropagation module is also used to backpropagate the predicted scores of the target categories in the enhanced candidate boxes according to the chain rule, so as to obtain the gradient information of the feature map of the target image corresponding to the prediction layer.

[0111] In some possible implementations, the second determining module is further configured to determine the mean of gradient information of the feature map corresponding to each prediction layer in each channel; determine the channel weight of the feature map corresponding to each prediction layer based on each mean of the feature map corresponding to each prediction layer; and normalize the maximum value of each channel weight by a preset normalization function to obtain the correlation coefficient of each channel weight.

[0112] In some possible implementations, the device further includes an enhancement module for enhancing the final channel weights using a preset enhancement coefficient to obtain enhanced final channel weights.

[0113] In some possible implementations, the weighting module is also used to enhance each channel of the feature map separately using the enhanced final channel weights to obtain an enhanced feature map.

[0114] Based on the same inventive concept, embodiments of the present invention provide an electronic device that can be consistent with the gradient-guided feature activation single-class target perception self-enhancement method described in one or more of the above embodiments. Figure 5 This is a schematic diagram of an electronic device structure according to an embodiment of the present invention. See also... Figure 5 As shown, the electronic device 500 can use general-purpose computer hardware, including a processor 501 and a memory 502.

[0115] Based on the same inventive concept, the present invention provides a computer storage medium storing computer-executable instructions. After being executed by a processor, the computer-executable instructions can realize the single-class target perception self-enhancement method of gradient-guided feature activation as described in one or more of the above embodiments.

[0116] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A single-class target perception self-enhancement method of gradient-guided feature activation, characterized in that, The method comprises: determining a target class of a target image to be detected; inputting the target image into a target detection network for first forward propagation, to obtain a prediction score of the target class corresponding to a prediction layer of the target detection network; wherein the target detection network comprises a plurality of prediction layers; obtaining a confidence of a candidate box of a feature map of the target image output by the first forward propagation; selecting a target candidate box from the candidate box according to the target class; selecting an enhanced candidate box from the target candidate box according to a number of the target candidate box and / or the confidence of the target candidate box; performing gradient backpropagation on the prediction score to obtain gradient information of a feature map of the target image corresponding to each of the prediction layers; the gradient backpropagation on the prediction score to obtain the gradient information of the feature map of the target image corresponding to the prediction layer comprises: performing gradient backpropagation on the prediction score of the target class in the enhanced candidate box according to a chain rule of differentiation, to obtain the gradient information of the feature map of the target image corresponding to the prediction layer; An expression of gradient information of a feature map of the target image corresponding to the prediction layer is: wherein, is a prediction layer n is gradient information of a corresponding feature map A, i is height of the feature map A, j is width of the feature map A, C is a channel number of the feature map A, and k is a target class, is a partial derivative symbol, is a prediction layer n is a prediction score of the k-th class in a prediction result of the prediction layer k is a feature map with height i j is a feature map with width​​ determining a channel weight of the feature map corresponding to each of the prediction layers and a correlation coefficient of each of the channel weights according to the gradient information; determining a final channel weight value of the feature map according to the channel weight and the correlation coefficient; weighting the feature map corresponding to the prediction layer by the final channel weight value to obtain an enhanced feature map; inputting the enhanced feature map into the target detection network for second forward propagation, to obtain an enhanced target detection result.

2. The self-reinforcing method of claim 1, wherein, the determination of the channel weight of the feature map corresponding to each of the prediction layers and the correlation coefficient of each of the channel weights according to the gradient information comprises: determining a mean value of the gradient information on each channel of the feature map corresponding to each of the prediction layers; determining the channel weight of the feature map corresponding to each of the prediction layers according to each of the mean values of the feature map corresponding to each of the prediction layers; normalizing a maximum value of each of the channel weights by a preset normalization function to obtain the correlation coefficient of each of the channel weights.

3. The self-reinforcing method according to claim 2, characterized in that, the final channel weight value is expressed as: wherein, is a max-min normalization function, to be is n a channel weight of a feature map corresponding to the prediction layer, to be is n a correlation coefficient of the channel weights, n is the number of prediction layers.

4. The self-reinforcing method of claim 3, wherein, before the weighting of the feature map corresponding to the prediction layer by the final channel weight value to obtain the enhanced feature map, the method further comprises: enhancing the final channel weight value by a preset enhancement coefficient to obtain an enhanced final channel weight value.

5. The self-reinforcing method of claim 4, wherein, the weighting of the feature map corresponding to the prediction layer by the final channel weight value to obtain the enhanced feature map comprises: enhancing each channel of the feature map by the enhanced final channel weight value respectively to obtain the enhanced feature map.

6. A single-class target perception self-enhancement device with gradient-guided feature activation, characterized in that, The method comprises: a first determination module configured to determine a target class of a target image to be detected; a first forward propagation module configured to input the target image into a target detection network for first forward propagation, to obtain a prediction score of the target class corresponding to a prediction layer of the target detection network; wherein the target detection network comprises a plurality of prediction layers; obtain a confidence of the candidate box; select a target candidate box from the candidate box according to the target category; select an enhanced candidate box from the target candidate box according to a number of the target candidate box and / or a confidence of the target candidate box; a gradient back propagation module, configured to perform gradient back propagation on the prediction score to obtain gradient information of the feature map of the target image corresponding to each of the prediction layers; the gradient back propagation on the prediction score to obtain the gradient information of the feature map of the target image corresponding to the prediction layer includes: perform gradient back propagation on the prediction score of the target category in the enhanced candidate box according to a chain rule to obtain the gradient information of the feature map of the target image corresponding to the prediction layer; An expression of gradient information of a feature map of the target image corresponding to the prediction layer is: wherein, is a prediction layer n corresponding gradient information of the feature map A, i is a height of the feature map A, j is a width of the feature map A, C is a channel number of the feature map A, and k is a target class, is a partial derivative symbol, is a prediction layer n is a prediction score of the k-th class in a prediction result of the prediction layer k is a feature map with a height of i a width of j ​​ a second determination module, configured to determine a channel weight of the feature map corresponding to each of the prediction layers and a correlation coefficient of each of the channel weights according to the gradient information; a third determination module, configured to determine a final channel weight of the feature map according to the channel weight and the correlation coefficient; a weighting module, configured to weight the feature map corresponding to the prediction layer by the final channel weight to obtain an enhanced feature map; a second forward propagation module, configured to input the enhanced feature map into the target detection network to perform second forward propagation to obtain an enhanced target detection result.

7. An electronic device, comprising: comprise: a memory, which stores computer executable instructions; a processor, which is connected with the memory, and is configured to execute the computer executable instructions to implement the method in any one of claims 1 to 5.

8. A computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the method in any one of claims 1 to 5.

Images