Weakly supervised object detection method based on candidate box weight redistribution

Abstract

The application discloses a weakly supervised target detection method based on candidate frame weight redistribution, comprising the following steps: step one, picture acquisition processing; step two, model training; step three, convergence verification; step four, model reasoning; the application starts from the weak supervision, learns the target detection by using the weakly labeled picture only in the image level, uses Grad-CAM information to dig out the potential target position in the feature map, then converts the Grad-CAM information into an edge-foreground response map to further analyze the spatial position relationship between the candidate frame and the high activation area in the picture, gives greater learning weight to the candidate frame correctly positioned in the complete contour position of the target, suppresses the weight value of the candidate frame falling in the target interior, and alleviates the problem that the weakly supervised target detection model is usually prone to falling into a local optimum problem due to the loss of the reference frame information, effectively improves the original weakly supervised target detection model, and improves the detection performance and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a weakly supervised object detection method based on candidate box weight redistribution. BACKGROUND

[0002] In recent years, with the improvement of computer hardware and computing power, computer vision and digital image processing technology has developed rapidly. As a basic research direction in the field of computer vision, object detection has been a research hotspot in the field of artificial intelligence and machine learning in recent years, and its related core technology has been widely used in many fields such as intelligent construction site, rail transit, highway maintenance, intelligent parking, intelligent security, etc. In the field of identity recognition, object detection is one of the key links of human pose estimation, object segmentation, face recognition and other visual tasks, and its detection result will directly affect the target tracking, recognition and behavior description results of a series of related visual tasks. However, object detection is still an open research problem and there is still a lot of room for improvement and potential. The research of object detection technology is still a very challenging topic, and has great theoretical research value and wide industrial application prospect.

[0003] With the rapid development of convolutional neural networks, object detection technology has made many progress under the support of large-scale data sets. Usually, the samples of these data sets contain accurate target class and position and other strong annotation information. However, it takes a lot of human resources and time cost to obtain such strong annotation information, which is difficult to meet the actual application needs of various fields. In contrast, it is easier to obtain image-level class label information, which can be obtained from the Internet through a picture search engine to obtain a large number of samples with image-level class annotation information. Based on this situation, the weakly supervised object detection task has become a research topic with practical application and promotion value. The weakly supervised object detection model only uses image class labels as supervision information in the training stage. The trained model can output the target position and corresponding class information in the image in inference. At present, many mainstream weakly supervised object detection methods are based on the framework of multiple instance learning. Multiple instance learning method regards an image as a bag and all candidate boxes as multiple instances. If a bag contains at least one positive instance, the bag is called a positive bag. Conversely, if all instances in the bag are negative instances, the bag is called a negative bag. The object detection model (instance classifier) is trained under the constraint of multiple instances. Of course, with the great progress of convolutional neural networks, convolutional neural networks have incomparable advantages in feature abstraction and description ability compared with handcrafted features. Therefore, recent weakly supervised detection research work is mostly to combine multiple instance learning and convolutional neural networks to obtain better weakly supervised object detection performance.

[0004] Current weakly supervised object detection algorithms usually rely on traditional image edge detection algorithms to generate candidate regions, and the performance is not satisfactory, and the lack of benchmark box information often leads to the problem that the weakly supervised object detection model is easy to fall into local optimum. SUMMARY

[0005] The present application aims to provide a weakly supervised object detection method based on candidate box weight redistribution to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a weakly supervised object detection method based on candidate box weight redistribution, comprising the following steps: step one, picture acquisition processing; step two, model training; step three, convergence verification; step four, model reasoning;

[0007] In the above step one, first, collect the pictures used for training as a training set, then define the pictures in the training set, divide the picture I into R candidate boxes, and the predicted class number is C;

[0008] In the above step two, after the picture acquisition processing in step one is completed, the model is trained, and the training steps include:

[0009] 1) Initialize the convolutional neural network;

[0010] 2) Obtain the feature map of the image by forward propagation of the neural network;

[0011] 3) Forward propagation of the classification branch and obtain Grad-CAM information ;

[0012] 4) Assign a class pseudo label to each pixel value in the feature map according to the Grad-CAM information, let is the activation value of the spatial position (i,j) with respect to the class , wherein C is the class set, c is the class index, and c=0 represents the background class;

[0013] 5) Scan the above Grad-CAM through a sliding window with size w, let is the number of pixel points with class c contained in the window corresponding to the center position (i,j), then the statistical quantity calculation formula of each sliding window region is:

[0014] ;

[0015] 6) If the center pixel point of the sliding window satisfies the following conditions, it is considered as a target edge point: first, there are enough pixel points of the same class in the sliding window; second, the number of foreground class pixels and background class pixels contained in the sliding window is similar, define and respectively correspond to the threshold values in the two conditions;

[0016] 7) If the sliding window center pixel point meets the following conditions, it is considered as a target foreground point: first, the sliding window contains enough same category pixel points; second, the number difference of foreground and background category pixels in the sliding window is large enough, defined as and respectively correspond to the threshold values in the two conditions;

[0017] 8) Calculate the edge-foreground response weight of each candidate box through the edge label and foreground label assigned by the pixel points corresponding to the four edge positions of the candidate box;

[0018] 9) Calculate the loss of the target detection branch combined with the weight of the candidate box, and obtain the loss result;

[0019] In the above step three, when the loss result in step two is obtained, the random gradient descent algorithm is used to update the parameters, and then the 2)-9) sub-steps in step two are repeated until the model converges to the position, and then the trained model is brought into the test, and the model training is completed after passing the test;

[0020] In the above step four, when the model training in step three is completed, the trained model is used to infer and detect the target. First, initialize the convolutional neural network, then obtain the feature map of the image through the forward propagation of the neural network, then the quality analysis module obtains the candidate box stability score and performs candidate box filtering, and finally the target detection branch is forward propagated and the target detection result is obtained.

[0021] Preferably, in the step two, the corresponding category pseudo label The calculation formula is:

[0022] ;

[0023] Where 0 and -1 represent background label and uncertain class label, and respectively correspond to the foreground and background threshold values.

[0024] Preferably, in the step two, the edge label of the pixel point (i,j) The calculation formula is as follows:

[0025]

[0026] Preferably, in the step two, the foreground label of the pixel point (i,j) The calculation formula is as follows:

[0027]

[0028] Preferably, in the step two, the calculation formula of the edge-foreground response weight is as follows:

[0029] ;

[0030] wherein, is a weight factor.

[0031] Preferably, in the step two, the loss function is:

[0032] .

[0033] Compared with the prior art, the beneficial effects of the present application are: the present application starts from the weak supervision angle, learns the target detection by using the weakly labeled picture with only the image level, excavates the potential target position in the feature map by using the Grad-CAM information, then converts the Grad-CAM information into the edge-foreground response map to further analyze the spatial position relationship between the candidate box and the high activation area in the picture, gives greater learning weight to the candidate box correctly positioned at the target complete contour position, and suppresses the weight value of the candidate box falling in the target interior, which makes up for the poor precision candidate area generated by relying on the traditional image edge detection algorithm in the weakly supervised target detection algorithm, and alleviates the problem that the weakly supervised target detection model is usually prone to local optimum due to the lack of baseline box information, effectively improves the original weakly supervised target detection model, and improves the detection performance and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 is the object positioning map of the weakly supervised target detection of the present application;

[0035] Fig. 2 is the method flowchart of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0037] Please refer to Figs. 1-2 , the present application provides an embodiment: a weakly supervised target detection method based on candidate box weight redistribution, comprising the following steps: step one, picture acquisition processing; step two, model training; step three, convergence verification; step four, model reasoning;

[0038] Wherein in the above step one, first collect the picture used for training as a training set, then define the picture of the training set, divide the picture I into R candidate boxes, and predict the class number as C;

[0039] Wherein in the above step two, when the picture collection process of step one is completed, model training is performed, and the training steps include:

[0040] 1) initialize the convolutional neural network;

[0041] 2) forward propagation of the neural network to obtain the feature map of the image;

[0042] 3) forward propagation of the classification branch and obtain Grad-CAM information ;

[0043] 4) assign a class pseudo label to each pixel value in the feature map according to the Grad-CAM information, let is the activation value of the spatial position (i,j) with respect to the class , the corresponding class pseudo label of The calculation formula is:

[0044] ;

[0045] Where 0 and -1 represent the background label and the uncertain class label respectively, and correspond to the foreground and background thresholds respectively;

[0046] 5) scan the above Grad-CAM through a sliding window with size w, let is the number of pixel points with class c contained in the window with center position (i,j), then the statistical quantity calculation formula of each sliding window region is:

[0047] ;

[0048] 6) if the sliding window center pixel point satisfies the following conditions, it is regarded as the target edge point: first, there are enough pixel points of the same class contained in the sliding window; second, the number of foreground class pixels and background class pixels contained in the sliding window is similar, define and correspond to the thresholds in the two conditions respectively, the edge label of the pixel point (i,j) The calculation formula is as follows:

[0049] ;

[0050] 7) If the sliding window center pixel point meets the following conditions, it is considered as a target foreground point: first, the sliding window contains enough same category pixel points; second, the number difference of foreground category pixels and background category pixels contained in the sliding window is large enough, defined as and respectively corresponding to the threshold values in the two conditions, the foreground label of the pixel point (i, j) The calculation formula is as follows:

[0051]

[0052] 8) The edge-foreground response weight of each candidate box is calculated through the edge label and the foreground label assigned by the pixel points corresponding to the four edge positions of the candidate box, and the calculation formula of the edge-foreground response weight is as follows:

[0053] ;

[0054] wherein, is a weight factor;

[0055] 9) The loss of the target detection branch is calculated in combination with the weight of the candidate box, and the loss result is obtained, and the loss function is:

[0056] ;

[0057] Wherein, in the above step three, when the loss result in step two is obtained, the random gradient descent algorithm is used to update the parameters, and then the 2)-9) sub-steps in step two are repeated until the model converges to the position, and then the trained model is brought into the test, and the model training is completed after passing the test;

[0058] Wherein, in the above step four, when the model training in step three is completed, the trained model is used to infer and detect the target, first, the convolutional neural network is initialized, then the neural network forward propagation obtains the feature map of the image, then the quality analysis module obtains the candidate box stability score and performs candidate box filtering, and finally the target detection branch forward propagation and obtains the target detection result.

[0059] Based on the above, the advantages of the present application are that, when the application is used, from the perspective of weak supervision, the target detection is learned by using the weakly labeled picture only at the image level, the Grad-CAM information is used to excavate the potential target position in the feature map, then the Grad-CAM information is converted into an edge-foreground response map to further analyze the spatial position relationship between the candidate box and the high activation region in the image, the candidate box correctly positioned at the complete contour position of the target is given a greater learning weight, and the weight value of the candidate box falling inside the target is suppressed, which greatly improves the detection effect and accuracy of the detection model.

[0060] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the present application can be implemented in other particular forms without departing from the spirit or essential characteristics of the present application. The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference signs in the claims should be considered as limiting the scope of the claims with respect to the figures of the patent document.

Claims

1. A weakly supervised object detection method based on candidate box weight redistribution, comprising the steps of: The method comprises the following steps: step one, picture collection processing; step two, model training; step three, convergence verification; and step four, model reasoning. In the step one, pictures used for training are collected as a training set, and then the pictures in the training set are defined, the picture I is divided into R candidate frames, and the number of predicted categories is C. In the step two, after the picture collection processing in the step one is completed, the model is trained, and the training steps comprise the following steps: 1) initializing a convolutional neural network; 2) obtaining a feature map of an image through forward propagation of the neural network; 3) Class branch forward propagation and obtain Grad-CAM information ; 4) Assign class pseudo-labels to each pixel value in the feature map according to Grad-CAM information, let be the activation value of class c at spatial location (i,j), where C is the set of classes, c is the class index, and c=0 represents the background class. be the activation value of class c at spatial location (i,j), where C is the set of classes, c is the class index, and c=0 represents the background class. 5) Scan the above Grad-CAM by a sliding window of size w, let Let n(i,j) be the number of pixels of class c in the window corresponding to the center position (i,j), then the statistic of each sliding window is calculated by the following formula: ； 6) If the center pixel of the sliding window satisfies the following conditions, it is considered as a target edge point: first, the sliding window contains enough pixels of the same class; second, the number of foreground and background pixels contained in the sliding window is similar, defined as and correspond to the threshold values in the two conditions, respectively; 7) If the center pixel of the sliding window satisfies the following conditions, it is considered as a target foreground pixel: first, the sliding window contains enough pixels of the same class; second, the difference between the number of foreground class pixels and the number of background class pixels in the sliding window is large enough, defined as and correspond to the threshold values in the two conditions, respectively; 8) calculating an edge-foreground response weight of each candidate frame through edge labels and foreground labels of pixel points corresponding to four edge frames of the candidate frame; 9) combining the weight of the candidate frame to calculate a loss of the target detection branch, and obtaining a loss result; In the step three, after the loss result in the step two is obtained, a random gradient descent algorithm is used to update parameters, and then the steps 2)-9) in the step two are repeated until the model converges, and then the trained model is used for testing, and the training of the model is completed after the model is qualified; In the step four, after the model training in the step three is completed, the trained model is used for reasoning and detecting a target, the convolutional neural network is initialized, the feature map of the image is obtained through forward propagation of the neural network, the quality analysis module obtains a candidate frame stability score and performs candidate frame filtering, and finally the target detection branch is forward propagated to obtain a target detection result.

2. The weakly supervised object detection method based on reweighting bounding boxes according to claim 1, wherein: In the step two, The corresponding category pseudo label The calculation formula is: ； where 0 and -1 represent the background label and the uncertain class label, respectively, and correspond to the foreground and background thresholds, respectively.

3. The weakly supervised object detection method based on reweighting bounding boxes according to claim 1, wherein: In the step two, the edge label of the pixel point (i, j) The calculation formula is as follows:

4. The weakly supervised object detection method based on reweighting bounding boxes according to claim 3, wherein: In the step two, the foreground label of the pixel point (i, j) The calculation formula is as follows:

5. The weakly supervised object detection method based on reweighting bounding boxes according to claim 4, wherein: According to the method of claim 1, the edge-foreground response weight is calculated according to the following formula in the step two: ； wherein is a weight factor.

6. The weakly supervised object detection method based on reweighting bounding boxes according to claim 5, wherein: In the step two, the loss function is: 。

Images