An online object detection improvement algorithm based on self-supervision and similarity suppression
By using self-supervision and similarity suppression algorithms in label-free sample detection, the accuracy of pseudo-labels is improved, the shortcomings of label-free samples in target detection are solved, and high-precision online target detection is achieved.
Patent Information
- Application Number
- CN202210284288.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-22
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-03-22
AI Technical Summary
In the prior art, there is a lack of methods to improve the accuracy of pseudo-labels for unlabeled samples, resulting in poor performance of online object detection models, especially in complex scenarios where the number of labeled samples is limited, such as in unmanned boat collaborative systems, it is difficult to achieve high-precision detection. .
The online object detection and improvement algorithm based on self-supervision and similarity suppression is adopted to improve the accuracy of the pseudo-label through dynamic confidence suppression, non-maximum suppression and similarity suppression algorithms, thereby improving the detection accuracy of the model.
It effectively improves the role of labelless samples in object detection tasks, improves model performance, solves the problem of low object detection accuracy, and ensures online improvement of prediction accuracy and stability of training.
Smart Images

Figure CN114676771B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image processing, and in particular relates to an online target detection enhancement algorithm based on self-supervision and similarity suppression. Background Art
[0002] Object detection is to detect possible object categories and their coordinate information in an image. It is a basic task in the field of image processing and computer vision. In the supervised learning mode based on deep learning, the training process of the object detection model requires a large number of labeled samples, and each sample needs to be manually labeled with the coordinate information and category information of each object. However, the formation of such a large number of labeled samples requires a lot of manpower and material resources, and is time-consuming and costly, so often only a small number of labeled samples and a large number of unlabeled samples can be obtained. A large number of unlabeled samples cannot participate in model training due to the lack of label information. The model can only rely on a small number of labeled samples for training, resulting in poor detection performance, which cannot reach the level of practical application and also causes a waste of resources. Therefore, in order to solve the above problems, the online object detection boosting algorithm has received more extensive attention and research in recent years. It can be described as mining the information of a large number of unlabeled samples through algorithms, thereby improving the performance of the object detection model trained with a small number of labeled samples. At present, it has been widely developed and applied in some object detection application fields with complex scenes and limited number of labeled samples, such as autonomous driving, intelligent monitoring, urban planning, and unmanned boat operation systems at sea.
[0003] In recent years, there are currently two solutions to the problem of how to use unlabeled samples to improve the performance of target detection models:
[0004] (1) Online object detection improvement algorithm based on consistency loss: Unlabeled samples are perturbed by data augmentation and other methods, and the model is constrained to achieve consistency in the prediction results of the data before and after the enhancement to improve the robustness and generalization of the model for unlabeled samples. In the process, labeled samples are combined to constrain the model to have effective predictions. Finally, a detection model with better performance is trained by combining labeled and unlabeled samples. For example, CSD (Jeong J, Lee S, Kim J, et al. Consistency-based semi-supervised learning for object detection [J]. Advances in neural information processing systems, 2019, 32: 10759-10768.), ISD (Jeong J, Verma V, Hyun M, et al. Interpolation-based semi-supervised learning for object detection [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 11602-11611.). Since this method requires mixing the original labeled samples for training, it cannot achieve the purpose of online improvement of the model.
[0005] (2) Online object detection improvement algorithm based on self-supervised learning. Self-supervised learning is to use auxiliary tasks to mine a large amount of self-information from unlabeled data. The network is trained through this constructed supervised information, so that it can learn representations that are valuable for downstream tasks. The method is to use labeled samples for model training and unlabeled samples for model update, and improve model performance and recognition accuracy in iterations. The online object detection improvement algorithm based on self-supervised learning can be described as follows: first, a small number of labeled samples are used to train the offline model, and the offline model is used to pseudo-label the unlabeled samples, and the unlabeled samples are perturbed by data augmentation and other methods. The constraint model is trained using its pseudo-labels and perturbed data to improve the performance of the visual detection model online. For example, STAC (Sohn K, Zhang Z, Li CL, et al. A simple semi-supervised learning framework for object detection [J]. arXiv preprint arXiv: 2005.04757, 2020.), Unbiased Teacher (Liu YC, Ma CY, He Z, et al. Unbiased teacher for semi-supervised object detection [J]. arXiv preprintarXiv:2102.09480, 2021.).
[0006] Both of the above methods only apply perturbations to unlabeled samples to perform online boosting training, thereby improving the recognition accuracy of the target detection model. However, they lack the process of improving the accuracy of pseudo-labels. Due to the large number of missed detections and false detections in pseudo-labels, biased training will occur during the online boosting training iteration, resulting in poor performance of the target detection model and easy model collapse. The training process is dominated by difficult samples or erroneous pseudo-labels. In some practical scenarios, for example, in an unmanned boat collaborative system, when a surface unmanned boat cluster performs missions in a complex marine environment, due to the combined effects of weather, sea conditions, lighting and other conditions on sea targets such as ships, there are fewer labeled samples and more difficult samples. The above methods are difficult to extract high-quality pseudo-labels, and the accuracy of pseudo-labels needs to be further improved to meet online boosting training, thereby achieving high-precision detection of targets. Summary of the invention
[0007] The purpose of the present invention is to provide an online target detection improvement algorithm based on self-supervision and similarity suppression, which solves the problem in the prior art that a large number of unlabeled samples cannot be used for model training in actual applications due to lack of label information, resulting in waste of resources, and makes the model detection accuracy low and cannot be applied to actual scenarios.
[0008] In order to achieve the above object, the technical solution adopted by the present invention is:
[0009] An online object detection enhancement algorithm based on self-supervision and similarity suppression includes the following steps:
[0010] (1) Collecting image samples of sea level targets acquired by an unmanned boat in a simulation environment, and dividing the collected image samples into a labeled sample set and an unlabeled sample set; performing data enhancement on each unlabeled sample in the unlabeled sample set, and performing online boosting training on the initially trained model using the enhanced unlabeled samples and pseudo labels after similarity suppression, and obtaining a model with improved detection accuracy after online boosting training;
[0011] (2) Input the image to be detected into the model with improved detection accuracy to achieve online target detection.
[0012] A further improvement of the present invention is that the pseudo-label after similarity suppression Obtained through the following process:
[0013] Step 1: collect image samples of sea level targets acquired by unmanned boats in a simulation environment, and divide the collected image samples into a labeled sample set S and an unlabeled sample set U; use the labeled sample set S to perform initial training on the two-stage target detection model F to obtain the model F′ after initial training;
[0014] Step 2: Use the initially trained model F′ to train each unlabeled sample x in the unlabeled sample set U u Perform pseudo-label prediction, and obtain the unlabeled sample x by sequentially passing the dynamic confidence suppression algorithm, non-maximum suppression algorithm, and similarity suppression algorithm u The corresponding similarity suppression pseudo-label
[0015] A further improvement of the present invention is that in step 1, the total loss during initial training is l s for:
[0016]
[0017] Among them, L cls is the category loss function, L reg is the position loss function, x s is an image sample in the labeled sample set S, p* is the category label of the target in the image, t * is its location label, b is the index of the target real box, i is the index of the anchor, and p i is the probability value of the i-th anchor predicting that there is a target, is the binary label of the i-th anchor relative to the b-th ground truth box, t i is the coordinate value predicted by the i-th anchor, is the location label of the bth ground-truth box, is the total number of positive and negative sampling points during the initial training process, It is only the number of positive sampling points, and λ1 is the proportion of regression loss in the initial training.
[0018] A further improvement of the present invention is that the specific process of step 2 is:
[0019] Step 2.1: Use model F′ to train each unlabeled sample x in the unlabeled sample set U u Perform inference and get each unlabeled sample x u The corresponding pseudo label P u ;
[0020] Step 2.2: Use the dynamic confidence suppression algorithm to suppress the pseudo label P according to the confidence u Filter and obtain the corresponding maximum hard false label
[0021] Step 2.3: Use non-maximum suppression algorithm to suppress maximum hard false labels Perform non-maximum suppression to obtain non-maximum suppression pseudo-label
[0022] Step 2.4: Use similarity suppression algorithm to suppress maximum hard false labels Perform similarity suppression to obtain unlabeled samples x u The corresponding similarity suppression pseudo-label
[0023] A further improvement of the present invention is that in step 2.2, the pseudo label P is calculated by the following formula: u Filter and obtain the corresponding maximum hard false label
[0024]
[0025] Among them, ε is the confidence threshold, E is the total number of iterations, and e is the current iteration.
[0026] A further improvement of the present invention is that the specific process of step 2.3 is as follows:
[0027] Step 2.3.1: Extracting maximum hard false labels The detection result predicted as the kth class in, is the probability score of the k-th detection box category, for The corresponding position prediction results, k∈1,2,…K, K is the total number of categories;
[0028] Step 2.3.2: Test results According to the k-th detection box category probability score Arrange in descending order to get the sorted k-th category detection results in, is the descending order of the probability scores of the k-th detection box category, for The corresponding position prediction result;
[0029] Step 2.3.3: For the position prediction results The prediction result of the i-th detection box position in And the j-th detection box position prediction result calculate and The intersection ratio i∈I, j≠i, I is The total number of detection results, if the intersection over union (IoU) is greater than the intersection over union threshold τ, then Eliminate
[0030] Step 2.3.4: For maximum hard false labels For each category k in , repeat steps 2.3.1 to 2.3.3 to obtain unlabeled samples x u The corresponding non-maximum suppression pseudo-label
[0031] A further improvement of the present invention is that the specific steps of step 2.4 are:
[0032] Step 2.4.1: Based on the labeled sample set S in step 1, extract the mean image block of the k-th target, denoted as M k (S);
[0033] Step 2.4.2: Extract non-maximum suppression pseudo-labels middle for The position result of the detection of the kth category in ;
[0034] Step 2.4.3: Detection location results of the kth category Includes several location results for Each of Crop unlabeled samples x u , get the image area of the corresponding area Expand and shrink the image area by interpolation and other methods To a fixed scale, calculate the mean image block M of the image area after the fixed scale and the k-th target k (S), if the similarity is less than the similarity threshold π k , then from Remove
[0035] Step 2.4.4: For maximum hard false labels For each k in , repeat steps 2.4.1-2.4.3 to obtain the unlabeled sample x u The corresponding similarity suppression pseudo-label
[0036] A further improvement of the present invention is that the data enhancement method in step (1) is color transformation, left-right flipping or scale change.
[0037] A further improvement of the present invention is that the online boosting training loss l is used for the online boosting training of the model F′ after the initial training. u for:
[0038]
[0039] Among them, L cls is the category loss function, L reg is the position loss function, for The category labels in for The location label in, b is the index of the real box, i is the index of the anchor, c i is the category probability predicted by the i-th anchor, yes The i-th anchor corresponds to the category label of the b-th real box, t i is the predicted position of the i-th anchor, yes The location label of the bth ground-truth box in , is the binary label of the i-th anchor relative to the b-th ground truth box, is the total number of positive and negative sampling points during online boosting training, is only the number of positive sampling points, and λ2 is the proportion of regression loss during online boosting training.
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] The present invention can utilize a large number of unlabeled samples to improve the detection accuracy of a model trained with only a small number of labeled samples, giving full play to the role of unlabeled sample sets in target detection tasks, improving the performance of the model, and further improving the online target detection accuracy, solving the problem of low target detection accuracy caused by only being able to use a small number of labeled sample sets for training in actual applications.
[0042] Furthermore, the dynamic confidence suppression algorithm proposed in the present invention can dynamically adjust the confidence suppression threshold. Due to the limited detection capability in the early stage of online boosting training iteration, there are a large number of pseudo-labels with low certainty. At this time, a low threshold is required to retain most of the correct pseudo-labels. In the later stage, the detection capability of the model is improved, and the threshold is dynamically increased, which is conducive to filtering erroneous pseudo-labels, improving the quality of pseudo-labels, and facilitating the model online boosting training process.
[0043] Furthermore, the non-maximum suppression algorithm used in the present invention can remove repeated prediction frames with a high overlap rate and filter out low-quality pseudo-labels during the online boosting training process, thereby preventing low-quality pseudo-labels from generating misleading information to the model during the online boosting training process.
[0044] Furthermore, the similarity suppression algorithm proposed in the present invention can mine the feature similarities of similar detection results, remove difficult samples with low similarity, improve the quality of pseudo-labels in the iterative process of online boosting training, and ensure the correctness and stability of predicted pseudo-labels during online boosting training. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flow chart of the present invention.
[0046] Figure 2 This is the network structure diagram of the two-stage target detection model Faster R-CNN in the present invention.
[0047] Figure 3 This is a diagram showing the detection effect of the present invention on the task of improving online target detection on a simulation dataset. DETAILED DESCRIPTION
[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0049] Step 1: collect image samples of sea level targets acquired by unmanned boats in a simulation environment, and divide the collected image samples into a labeled sample set S and an unlabeled sample set U. The labeled sample set S is used to perform initial training on the two-stage target detection model F. The training is stopped after the two-stage target detection model F obtains a certain detection accuracy (for example, the detection accuracy reaches more than 70%), and a model F′ of the two-stage target detection model F after initial training is obtained.
[0050] Preferably, the two-stage target detection model F in step 1 is Faster R-CNN (Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks [J]. Advances in neural information processing systems, 2015, 28: 91-99.), and its total loss during initial training is l s for:
[0051]
[0052] Among them, the total loss l s Contains category loss and position loss, L cls is the category loss function, that is, the cross entropy function, L reg is the position loss function, i.e., the Smooth-L1 function. s is an image sample in the labeled sample set S, p * is the category label of the target in the image, t * is its location label. Specifically, b is the index of the target ground truth box, and i is the index of the anchor. i is the probability value of the i-th anchor predicting that there is a target, is the binary label of the i-th anchor relative to the b-th ground truth box, t i It is the coordinate value predicted by the i-th anchor. is the location label of the bth ground-truth box. is the total number of positive and negative sampling points during the initial training process, It is only the number of positive sampling points, and λ1 is the proportion of regression loss in the initial training.
[0053] Step 2: For each unlabeled sample x in the unlabeled sample set U u , using model F′ for unlabeled samples x uPerform pseudo-label prediction, and obtain the unlabeled sample x by sequentially passing the dynamic confidence suppression algorithm, non-maximum suppression algorithm, and similarity suppression algorithm u The corresponding similarity suppression pseudo-label The specific steps include:
[0054] Step 2.1: Use model F′ for each unlabeled sample x u Perform inference and get each unlabeled sample x u The corresponding pseudo label P u .
[0055] Step 2.2: Use the dynamic confidence suppression algorithm to suppress the pseudo label P according to the confidence u Filter and obtain the corresponding maximum hard false label The specific calculation process of this algorithm is: Among them, ε is the confidence threshold, which is dynamically adjusted according to the total number of iterations E and the current iteration e, and is calculated as follows:
[0056] Step 2.3: Use the non-maximum suppression algorithm to Perform non-maximum suppression to remove similar pseudo-labels with high position overlap. Obtain non-maximum suppressed pseudo-labels The algorithm is completed through the following steps:
[0057] Step 2.3.1: Extraction The detection result predicted as the kth class in, is the probability score of the k-th detection box category, for The corresponding position prediction result. k∈1,2,…K, K is the total number of categories.
[0058] Step 2.3.2: according to Arrange in descending order to obtain the detection results of the kth category after sorting in, is the descending order of the probability scores of the k-th detection box category, for The corresponding position prediction result.
[0059] Step 2.3.3: For the position prediction results The prediction result of the i-th detection box position in And the j-th detection box position prediction result calculate and The intersection ratio i∈I, j≠i, I is If IoU is greater than the intersection-over-union threshold τ, then Eliminate
[0060] Step 2.3.4: For For each category k in, repeat steps 2.3.1-2.3.3 to obtain unlabeled samples x u The corresponding non-maximum suppression pseudo-label
[0061] Step 2.4: Use similarity suppression algorithm to Perform similarity suppression to obtain unlabeled samples x u The corresponding similarity suppression pseudo-label Among them, the specific steps of the similarity suppression algorithm are:
[0062] Step 2.4.1: Based on the labeled sample set S in step 1, extract the mean image block of the k-th class target, denoted as M k (S).
[0063] Step 2.4.2: Extract non-maximum suppression pseudo-labels middle for The detection position result of the kth category in .
[0064] Step 2.4.3: Including several for Each of Crop unlabeled samples x u Get the image area corresponding to the area Expand and shrink the image area by interpolation and other methods To a fixed scale, calculate the mean image block M of the image area after the fixed scale and the k-th target k (S), if the similarity is less than the similarity threshold π k , then from Remove
[0065] Step 2.4.2: For For each k in , repeat steps 2.4.1 to 2.4.3 to obtain Unlabeled samples x obtained by similarity suppression algorithm u The corresponding similarity suppression pseudo-label
[0066] Step 3: For each unlabeled sample x in the unlabeled sample set U u Perform data enhancement and use the enhanced unlabeled samples xu and the corresponding pseudo-label after similarity suppression obtained in step 2 The model F′ after initial training is subjected to online boosting training. After E rounds of online boosting training, a model F″ with improved detection accuracy is obtained. Model F″ is a model that is trained by boosting the model F′ that was initially trained using the unlabeled sample set U. The role of the unlabeled sample set U in the target detection task is fully utilized. The detection accuracy of F″ will be better than that of the model F′ that was initially trained only with the labeled sample set S. This solves the problem that in actual applications, only a small amount of labeled sample sets S can be used for training, resulting in poor performance of the target detection model.
[0067] Preferably, the data enhancement method in step 3 is color transformation, left-right flipping, scale change, etc. For the unlabeled sample x u , the enhanced sample is recorded as A(x u ), A is the data enhancement method. Using A(x u )and The online boosting training loss l when performing online boosting training on the initially trained model F′ i for:
[0068]
[0069] Among them, the online improvement training loss l u Contains online training category loss and online training position loss, L cls is the category loss function, that is, the cross entropy function, L reg is the position loss function, that is, the Smooth-L1 function. for The category labels in for The location label in. b is the index of the ground-truth box, and i is the index of the anchor. c i is the category probability predicted by the i-th anchor, yes The i-th anchor corresponds to the category label of the b-th ground-truth box. i is the predicted position of the i-th anchor, yes The location label of the bth ground-truth box in . is the binary label of the i-th anchor relative to the b-th ground truth box, is the total number of positive and negative sampling points during online boosting training, is only the number of positive sampling points, and λ2 is the proportion of regression loss during online boosting training.
[0070] Step 4: Input the image to be detected into the model F″ with improved detection accuracy to achieve high-precision online target detection tasks.
[0071] Example 1
[0072] like Figure 1 As shown, an online target detection improvement algorithm based on self-supervision and similarity suppression includes the following steps:
[0073] Step 1: Divide the data set of sea level targets obtained by the unmanned boat in the simulation environment into a training set, a validation set, and a test set. Sea level targets include three categories: merchant ships, floats, and fishing boats. Randomly divide them according to the training set 0.75, the validation set 0.2, and the test set 0.05. Then divide the labeled sample set S and the unlabeled sample set U from the training set, and the ratio of the number of samples contained in S and U is 1:3. Build a two-stage target detection model Faster R-CNN, denoted as F.
[0074] The construction process is as follows Figure 2 As shown in the figure, F is divided into twenty-two layers, namely convolutional layer 1-2, pooling layer 1, convolutional layer 5-7, pooling layer 2, convolutional layer 8-10, pooling layer 3, convolutional layer 11-13, RPN candidate frame extraction layer, ROI candidate frame region feature extraction layer, fully connected layer 1-2, and classification regression layer. Among them, the parameters of each deep convolution kernel pooling layer are: the number of channels of the input two-dimensional image is 3, and the resolution is 600*600. The convolution kernel size of convolutional layer 1-2 is 3*3, and the number of output channels is 64. The convolution kernel size of convolutional layer 3-4 is 3*3, and the number of output channels is 128. The convolution kernel size of convolutional layer 5-7 is 3*3, and the number of output channels is 256. The convolution kernel size of convolutional layer 8-13 is 3*3, and the number of output channels is 512. The pooling range of pooling layer 1-4 is 2*2. The resolution of the classification layer feature map in the RPN candidate box extraction layer is 38*38, and the number of channels is 18. The resolution of the regression layer feature map is 38*38, and the number of channels is 36. The resolution of the feature map of the ROI pooling layer is 7*7, and the number of channels is 512. The number of channels of the feature maps of the fully connected layers 1-2 is 4096. The number of channels of the classification layer feature map in the classification regression layer is 4, and the number of channels of the regression layer feature map is 4.
[0075] Use the labeled sample set S to perform initial training on F, and the total loss during initial training is l s for: Among them, the total loss l s Contains category loss and position loss, L cls is the category loss function, that is, the cross entropy function, L reg is the position loss function, i.e., the Smooth-L1 function. sis an image sample in the labeled sample set S, p * is the category label of the target in the image, t * is its location label. Specifically, b is the index of the target ground truth box, and i is the index of the anchor. i is the probability value of the i-th anchor predicting that there is a target, is the binary label of the i-th anchor relative to the b-th ground truth box, t i It is the coordinate value predicted by the i-th anchor. is the location label of the bth ground-truth box. is the total number of positive and negative sampling points during the initial training process, It is only the number of positive sampling points, and λ1 is the proportion of regression loss in the initial training, which is 1.
[0076] Initial training is stopped after F obtains a certain detection accuracy, for example, the detection accuracy reaches more than 70%. This step obtains model F′ after the initial training of model F.
[0077] Step 2: For each unlabeled sample x in the unlabeled sample set U u , follow these steps:
[0078] Step 2.1: Use model F′ to first u Perform inference and get each unlabeled sample x u The corresponding pseudo label P u .
[0079] Step 2.2: Use the dynamic confidence suppression algorithm to suppress the pseudo label P according to the confidence u Filter and obtain the corresponding maximum hard false label Specifically Among them, ε is the confidence threshold, which is dynamically adjusted according to the total number of iterations E and the current iteration e, and is calculated as follows: E is usually set to 35.
[0080] Step 2.3: Use the non-maximum suppression algorithm to Perform non-maximum suppression to remove similar pseudo-labels with high position overlap. Obtain non-maximum suppressed pseudo-labels The non-maximum suppression steps are:
[0081] Step 2.3.1: Extraction The detection result predicted as the kth class in, is the probability score of the k-th detection box category, for The corresponding position prediction result. k∈1,2,…K, K is the total number of categories.
[0082] Step 2.3.2: according to Arrange in descending order to get the sorted k-th category detection results in, is the descending order of the probability scores of the k-th detection box category, for The corresponding position prediction result.
[0083] Step 2.3.3: For The prediction result of the i-th detection box position in calculate and The intersection ratio i∈I, j≠i, I is If IoU is greater than the intersection-over-union threshold τ, τ = 0.5, then Eliminate
[0084] Step 2.3.4: For For each category k in , repeat steps 2.3.1 to 2.3.3 to obtain unlabeled samples x u The corresponding non-maximum suppression pseudo-label
[0085] Step 2.4: Use similarity suppression algorithm to Perform similarity suppression to obtain unlabeled samples x u The corresponding similarity suppression pseudo-label The algorithm steps are:
[0086] Step 2.4.1: Based on the labeled sample set S in step 1, extract the mean image block of the k-th class target, denoted as M k (S).
[0087] Step 2.4.2: Extract non-maximum suppression pseudo-labels middle in, for The detection position result of the kth category in .
[0088] Step 2.4.3: Including several for Each of Crop unlabeled samples x u Get the image area corresponding to the area Expand and shrink the image area by interpolation and other methods To a fixed scale, calculate the mean image block M of the image area after the fixed scale and the k-th target k (S) similarity, the similarity uses cosine similarity, if the similarity is less than the similarity threshold π k , π k =0.3, then from Remove
[0089] Step 2.4.4: For For each k in , repeat the above steps 2.4.1 to 2.4.3 to obtain Unlabeled samples x obtained by similarity suppression algorithm u The corresponding similarity suppression pseudo-label
[0090] Step 3: For each unlabeled sample x in the unlabeled sample set U u Perform data enhancement and use the enhanced unlabeled samples x u and the corresponding pseudo-label after similarity suppression obtained in step 2 The model F′ after initial training is trained online. After E rounds of online training, a model F″ with improved detection accuracy is obtained.
[0091] The loss of online boosting training is: Among them, the online improvement training loss l u Contains online training category loss and online training position loss, L cls is the category loss function, that is, the cross entropy function, L reg is the position loss function, that is, the Smooth-L1 function. for The category labels in for The location label in. b is the index of the ground-truth box, and i is the index of the anchor. c i is the category probability predicted by the i-th anchor, yes The i-th anchor corresponds to the category label of the b-th ground-truth box. i is the predicted position of the i-th anchor, yes The location label of the bth ground-truth box in . is the binary label of the i-th anchor relative to the b-th ground truth box, is the total number of positive and negative sampling points during online boosting training, is only the number of positive sampling points, and λ2 is the proportion of regression loss during online boosting training.
[0092] Step 4: Input the image to be detected into the model F″ with improved detection accuracy to achieve high-precision target detection tasks.
[0093] The effect of this embodiment can be further verified by the following experiments:
[0094] 1. Experimental hardware platform conditions: Intel CPU i7-9700, 32GB memory, graphics card NVIDIA GTX-1080Ti. Software conditions: machine learning framework PyTorch1.1.
[0095] 2. Experimental content and results:
[0096] 2.1 Experimental samples: The simulation data set obtained by the unmanned boat at sea includes three types: merchant ships, buoys and fishing boats.
[0097] 2.2 Experimental content: The simulation data set model acquired by the unmanned boat at sea is trained based on the training algorithm of the prior art Faster R-CNN and the online boosting algorithm based on self-supervision and similarity suppression proposed in the present invention, and the detection accuracy of the target detection model under the two algorithms is compared, namely, the AP and average AP of the three types of targets.
[0098] 2.3 Experimental basic hyperparameter settings, see Table 2:
[0099] Table 2 Basic parameters for setting
[0100] Hyperparameters value Batch size 4 Input image basic size 600*600 Maximum number of rounds 35 Pre-trained parameter model VGG16 Optimizer SGD Learning Rate 0.001 attenuation 20 / 25 rounds
[0101] 2.4 Experimental results are shown in Table 3:
[0102] Table 3 Experimental results
[0103]
[0104]
[0105] The experimental verification results show that the online boosting training algorithm of the present invention can use a large number of unlabeled samples to further improve the detection accuracy of the model trained with only a small number of labeled samples. Some detection results are as follows: Figure 3 The present invention gives full play to the role of unlabeled sample sets in target detection tasks, and solves the problem that only a small number of labeled sample sets S can be used for training in practical applications, resulting in poor performance of target detection models.
Claims
1. An online object detection improvement algorithm based on self-supervision and similarity suppression, characterized in that: The following steps are involved: (1) Collecting image samples of sea level targets acquired by an unmanned boat in a simulation environment, and dividing the collected image samples into a labeled sample set and an unlabeled sample set; Perform data enhancement on each unlabeled sample in the unlabeled sample set, and use the enhanced unlabeled samples and similarity suppressed pseudo labels to perform online boosting training on the initially trained model. After online boosting training, a model with improved detection accuracy is obtained. (2) Input the image to be detected into the model with improved detection accuracy to achieve online target detection; Pseudo-labeling after similarity suppression Obtained through the following process: Step 1: Collect image samples of sea level targets acquired by unmanned boats in a simulation environment, and divide the collected image samples into a labeled sample set S and an unlabeled sample set U; use the labeled sample set S to perform initial training on the two-stage target detection model F to obtain the model F after initial training. ′ ; Step 2: Use the initially trained model F ′ For each unlabeled sample x in the unlabeled sample set U u Perform pseudo-label prediction, and obtain the unlabeled sample x by sequentially passing the dynamic confidence suppression algorithm, non-maximum suppression algorithm, and similarity suppression algorithm u The corresponding similarity suppression pseudo-label The specific process of step 2 is: Step 2.1: Using Model F ′ For each unlabeled sample x in the unlabeled sample set U u Perform inference and get each unlabeled sample x u The corresponding pseudo label P u ; Step 2.2: Use the dynamic confidence suppression algorithm to suppress the pseudo label P according to the confidence u Filter and obtain the corresponding maximum hard false label Step 2.3: Use non-maximum suppression algorithm to suppress maximum hard false labels Perform non-maximum suppression to obtain non-maximum suppression pseudo-label Step 2.4: Use similarity suppression algorithm to suppress maximum hard false labels Perform similarity suppression to obtain unlabeled samples x u The corresponding similarity suppression pseudo-label In step 2.2, the pseudo label P is given by u Filter and obtain the corresponding maximum hard false label Among them, ε is the confidence threshold, E is the total number of iterations, and e is the current iteration; The specific steps of step 2.4 are: Step 2.4.1: Based on the labeled sample set S in step 1, extract the mean image block of the k-th target, denoted as M k (S); Step 2.4.2: Extract non-maximum suppression pseudo-labels middle for The position result of the detection of the kth category in ; Step 2.4.3: Detection location results of the kth category Includes several location results for Each of Crop unlabeled samples x u , get the image area of the corresponding area Expand and shrink the image area by interpolation and other methods To a fixed scale, calculate the mean image block M of the image area after the fixed scale and the k-th target k (S), if the similarity is less than the similarity threshold π k , then from Remove Step 2.4.4: For maximum hard false labels For each k in , repeat steps 2.4.1-2.4.3 to obtain the unlabeled sample x u The corresponding similarity suppression pseudo-label 2. The online target detection improvement algorithm based on self-supervision and similarity suppression according to claim 1, characterized in that: In step 1, the total loss during initial training is l s for: Among them, L cls is the category loss function, L reg is the position loss function, x s is an image sample in the labeled sample set S, p * is the category label of the target in the image, t * is its location label, b is the index of the target real box, i is the index of the anchor, and p i is the probability value of the i-th anchor predicting that there is a target, is the binary label of the i-th anchor relative to the b-th ground truth box, t i is the coordinate value predicted by the i-th anchor, is the location label of the bth ground-truth box, is the total number of positive and negative sampling points during the initial training process, It is only the number of positive sampling points, and λ1 is the proportion of regression loss in the initial training.
3. The online target detection improvement algorithm based on self-supervision and similarity suppression according to claim 1, characterized in that: The specific process of step 2.3 is as follows: Step 2.3.1: Extracting maximum hard false labels The detection result predicted as the kth class in, is the probability score of the k-th detection box category, for The corresponding position prediction results, k∈1,2,…K, K is the total number of categories; Step 2.3.2: Test results According to the k-th detection box category probability score Arrange in descending order to get the sorted k-th category detection results in, is the descending order of the probability scores of the k-th detection box category, for The corresponding position prediction result; Step 2.3.3: For the position prediction results The prediction result of the i-th detection box position in And the j-th detection box position prediction result calculate and The intersection ratio i∈I, j≠i, I is The total number of detection results, if the intersection over union (IoU) is greater than the intersection over union threshold τ, then Eliminate Step 2.3.4: For maximum hard false labels For each category k in , repeat steps 2.3.1 to 2.3.3 to obtain unlabeled samples x u The corresponding non-maximum suppression pseudo-label 4. The online target detection improvement algorithm based on self-supervision and similarity suppression according to claim 1, characterized in that: The data augmentation method in step (1) is color transformation, left-right flipping or scale change.
5. The online target detection improvement algorithm based on self-supervision and similarity suppression according to claim 1, characterized in that: After initial training, the model F ′ Online boosting training loss l when performing online boosting training u for: Among them, L cls is the category loss function, L reg is the position loss function, for The category labels in for The location label in, b is the index of the real box, i is the index of the anchor, c i is the category probability predicted by the i-th anchor, yes The i-th anchor corresponds to the category label of the b-th real box, t i is the predicted position of the i-th anchor, yes The location label of the bth ground-truth box in , is the binary label of the i-th anchor relative to the b-th ground truth box, is the total number of positive and negative sampling points during the online boosting training process, is only the number of positive sampling points, and λ2 is the proportion of regression loss during online boosting training.
