A sar image target detection method combined with high-trust knowledge distillation

Abstract

The application discloses a SAR image target detection method combined with high-trust knowledge distillation, belongs to the technical field of SAR image target detection, and comprises four steps of training of a teacher model, re-labeling of a data set, training of a student model and reasoning of the student model. The application applies the thought of knowledge distillation to a SAR image target detection task, and proposes a label reuse mechanism, can directly save soft labels generated by the teacher model, and uses the labels again when training the student model, so that repeated labeling is not needed, time cost is saved, a calculation formula of distillation regression loss is optimized, an originally used strategy of taking the teacher model as a learning object is improved into selection of a learning object according to the size of L2 distance, this is more conducive to learning of richer features, and a prediction box fusion strategy which can be directly embedded after non-maximum suppression is proposed, the strategy can reasonably optimize the boundary of a prediction box according to the size of confidence, and the target is covered as completely as possible.

Description

Technical Field

[0001] This invention relates to the field of SAR image target detection technology, and specifically to a SAR image target detection method that combines high-reliability knowledge distillation. Background Technology

[0002] my country's aerospace reconnaissance equipment has acquired multi-source, multi-band, multi-mode, multi-application, and high-resolution ground imaging capabilities. How to process and interpret image information more quickly and effectively on embedded platforms has become a problem to be solved. While existing neural network models can filter out usable intelligence information from SAR images, their parameter count is often large, making efficient portability to embedded platforms difficult. Furthermore, the large number of model parameters is a critical issue. Most importantly, SAR image datasets are frequently updated; using a small model leads to insufficient fitting ability, affecting recognition performance, while using a large model results in a significant consumption of embedded platform computing resources due to its massive parameter count. Large models also affect message generation speed, failing to meet the high timeliness requirements of reconnaissance. Therefore, it is necessary to minimize the number of model parameters while ensuring a high detection rate.

[0003] The aforementioned problems urgently need to be solved. To address this, a SAR image target detection method combining high-reliability knowledge distillation is proposed. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to solve the problem of excessive model parameters when the SAR image target detection model is mounted on an embedded device, and reduce the number of model parameters by improving the knowledge distillation method while ensuring high reliability of the recognition effect, thereby shortening the inference time. This invention provides a SAR image target detection method that combines high reliability knowledge distillation.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0006] S1: Training Teacher Model

[0007] Collect a SAR image dataset and label the SAR images in it. Build a teacher model. The teacher model outputs soft labels and hard labels. Use a portion of the image data in the SAR image dataset as the training dataset, and then use hard labels as supervision information to train the teacher model until the model converges.

[0008] S2: Relabel the training dataset

[0009] All training data from the training dataset are fed into the trained teacher model to obtain soft label values, which means the training data is re-labeled. Both hard and soft label values ​​are saved for use in training the student model.

[0010] S3: Training the student model

[0011] A student model is constructed, and the soft predictions output by the student model are made as close as possible to the soft labels inferred by the teacher model through the distillation loss function, so that the student model has the ability to output both soft and hard predictions.

[0012] S4: Reasoning in the Student Model

[0013] After the student model is trained, the inference process of the student model is carried out using the prediction box fusion strategy.

[0014] Furthermore, in step S1, the annotation method is to annotate the x and y axis coordinates of the center point of each target, the target length w and width h, and the target category.

[0015] Furthermore, in step S1, the soft label includes the x and y coordinates of the target's center point, the target's length w and width h, and the probability value for each category; the hard label includes the x and y coordinates of the target's center point, the target's length w and width h, and the target's category and confidence level.

[0016] Furthermore, in step S3, the formula for the total loss is as follows:

[0017] L loss =(1-α)L hard (S,T)+αL soft (s,t)

[0018] Where L hard The loss is the hard-label loss, i.e., the prediction loss, L. soft α is the soft label loss, also known as distillation loss, and α is the weighting coefficient; S and T correspond to the hard labels of the student model's predicted hard values ​​and the true values, respectively, and s and t correspond to the soft labels of the student model's predicted soft values ​​and the soft labels inferred by the teacher model, respectively.

[0019] Furthermore, the formula for calculating the predicted loss is as follows:

[0020] L hard =L cls (S,T)+L reg (S,T)

[0021] Among them, L cls (S,T) represents the classification prediction loss, and L... reg (S,T) represents the predicted regression loss.

[0022] Furthermore, the formula for calculating distillation loss is as follows:

[0023] L soft =Lsoft_cls (s,t)+L soft_reg (s,t)

[0024] Among them, L soft_cls (s,t) represents the distillation separation loss, L soft_reg (s,t) represents the distillation regression loss.

[0025] Furthermore, the formula for calculating distillation separation loss is as follows:

[0026] L soft_cls (s,t)=T 2 *KL div (C t C s )

[0027] Where T is the temperature parameter, KL div Let C be the KL divergence. t =softmax(Z) t / T), C s =softmax(Z) s / T), Z t The soft tag obtained in step S2, after being smoothed by the temperature parameter T and subjected to softmax, results in a new tag denoted as C. t Similarly, Z s The soft predicted value obtained by the student model, after being smoothed by the temperature parameter T and softmaxed, is denoted as the new label C. s .

[0028] Furthermore, the formula for calculating the distillation regression loss is as follows:

[0029]

[0030] Among them, b s For the student model prediction box, b t For the teacher model prediction box, b gt The soft tag obtained in step S2, 2 This represents the L2 distance.

[0031] Furthermore, in step S4, the processing flow of the prediction box fusion strategy is as follows:

[0032] S41: Perform nonmaximum suppression on the detection results;

[0033] S42: For each class, select the prediction box with the highest confidence, and calculate the intersection ratio with the subsequent prediction boxes accordingly. Set two thresholds a and b. If the intersection ratio of a prediction box is greater than b, it is set as box A, and the intersection ratio of another prediction box is less than a, it is set as box B. Then proceed to step S43. Otherwise, directly compare the next prediction box.

[0034] S43: Keep the center point of box A unchanged, scale the length and width according to the ratio of its confidence level to the confidence level of box B to obtain the scaled box M. Then, find the minimum bounding rectangle N between box M and box B as the final prediction box, and use the confidence level of box B as the confidence level of box N.

[0035] S44: Starting from the second-highest confidence bounding box in each class, repeat steps S42 and S43 in sequence until all the bounding boxes have been traversed.

[0036] S45: The SAR image target detection process ends after all prediction boxes have been traversed.

[0037] Furthermore, in step S43, if the confidence ratio of box A and box B is greater than a set value, it is increased; if it is less than a set value, it is decreased.

[0038] Compared with existing technologies, this invention has the following advantages: This SAR image target detection method, which combines high-confidence knowledge distillation, applies the idea of ​​knowledge distillation to SAR image target detection tasks and proposes a label reuse mechanism. It can directly save the soft labels generated by the teacher model and reuse these labels when training the student model, eliminating the need for repeated labeling and saving time. The calculation formula for distillation regression loss is optimized, improving the original strategy of using the teacher model as the learning object to selecting the learning object based on the magnitude of the L2 distance, which is more conducive to learning richer features. A prediction box fusion strategy that can be directly embedded in non-maximum suppression is proposed. This strategy can reasonably optimize the boundaries of the prediction boxes according to the confidence level, achieving the most complete target coverage possible. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the SAR image target detection method combining high-confidence knowledge distillation in an embodiment of the present invention.

[0040] Figure 2a This is a schematic diagram of a normal situation in an embodiment of the present invention, where the intersection-union ratio can accurately measure the overlap of the prediction boxes;

[0041] Figure 2b This is a schematic diagram of an extreme crossover ratio in an embodiment of the present invention. At this time, the difference in size between the two prediction boxes is too large, and the crossover ratio cannot effectively reflect the overlap.

[0042] Figure 3 This is a schematic diagram of the prediction box fusion strategy processing flow in an embodiment of the present invention. Detailed Implementation

[0043] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0044] This embodiment provides a technical solution: a SAR image target detection method combining high-confidence knowledge distillation, including the following specific steps:

[0045] (1) Training the teacher model: First, a relatively complete SAR image dataset is collected, including images and annotations. The annotation method is to annotate the x and y coordinates of the center point of each target, the length w and width h of the target, and the category of the target. The teacher model usually selects a backbone with a large number of parameters, such as YOLOv5X or DenseNet121, and outputs soft labels (x and y coordinates of the center point of the target, the length w and width h of the target, and the probability value of each category. Therefore, the length of the soft label of each target is 4+c, where c is the number of categories. See Table 1 for specific examples) and hard labels (x and y coordinates of the center point of the target, the length w and width h of the target, and the category and confidence of the target. The length of the hard label of each target is 6. See Table 2 for specific examples). Then, the teacher model is trained with the hard labels as supervision information until the model converges.

[0046] (2) Relabeling of training dataset: All training data are fed into the trained teacher model to obtain soft label values, which is equivalent to relabeling the training data. Both hard and soft label values ​​are saved for use in training the student model.

[0047] (3) Training the student model: The backbone of the student model is generally a model with a small number of parameters, such as YOLOv5S. The student model also has the ability to output both soft and hard predictions. Besides supervised learning with hard labels to generate normal prediction loss, the soft predictions of the student model are also used to calculate the loss function with the soft labels obtained in the previous step; this is the origin of distillation loss. Furthermore, to ensure the accuracy of the prediction boxes, the regression method for the student model's prediction boxes is determined based on the actual situation. The final loss function is calculated by summing the prediction loss and the distillation loss.

[0048] (4) Inference of the student model: After step (3) is completed, the student model often encounters the problem of inaccurate prediction boxes during inference. This paper proposes a novel prediction box fusion strategy that can obtain more accurate and reasonable prediction boxes.

[0049] In step (2), examples of generated soft tags are shown in the table below (assuming the number of categories is c):

[0050] Table 1 Examples of soft tags generated by the teacher model

[0051]

[0052] Examples of hard labels used in step (2) are shown in the table below:

[0053] Table 2 Examples of Hard Labels

[0054]

[0055]

[0056] It can be seen that the difference between hard labels and soft labels lies in the fact that hard labels select the category with the highest probability from the soft labels as the class to which the model belongs, while all other items remain the same. Because the teacher model has a very large number of parameters, the accuracy of the obtained soft label values ​​is very high, and can be regarded as close to the true value, thus saving the cost of manual annotation to a certain extent.

[0057] In step (3), the total loss formula is as follows:

[0058] L loss =(1-α)L hard (S,T)+αL soft (s,t)

[0059] Where L hard The loss is the hard-label loss, i.e., the prediction loss, L. soft The loss is the soft label loss, also known as the distillation loss, and α is the weight coefficient. S and T correspond to the hard predicted values ​​and hard labels from the true values ​​of the student model, respectively, while s and t correspond to the soft predicted values ​​and soft labels inferred by the teacher model, respectively. By updating the weights of the student model through the backpropagation algorithm and continuously reducing the loss value, the output of the student model can gradually approach that of the teacher model.

[0060] The prediction loss is calculated similarly to the normal YOLO series loss, including classification loss and regression loss, and its calculation formula is as follows:

[0061] L hard =L cls (S,T)+L reg (S,T)

[0062] Among them, L cls (S,T) represents the classification prediction loss, and L... reg (S,T) represents the predicted regression loss.

[0063] The distillation loss is essentially the loss between the soft labels output by the teacher network in step (2) and the soft predictions output by the student model. The distillation loss can be considered to consist of distillation classification loss and distillation regression loss.

[0064] Lsoft =L soft_cls (s,t)+L soft_reg (s,t)

[0065] Among them, L soft_cls (s,t) represents the distillation separation loss, L soft_reg (s,t) represents the distillation regression loss.

[0066] Distillation classification loss utilizes soft labels to make the output probability distribution of the student model as close as possible to that of the teacher model. Typically, a temperature parameter T is introduced into the output layers of both the teacher and student models. The larger the value of T, the smoother the output target class probability distribution, and the higher the degree of label softening. Thus, soft labels can be transformed into:

[0067] C t =softmax(Z) t / T)

[0068] C s =softmax(Z) s / T)

[0069] Among them, Z t The new label obtained after smoothing the temperature parameters and applying softmax to the soft label obtained in step (2) is denoted as C. t Similarly, Z s The soft predicted values ​​obtained for the student model, after being smoothed by temperature parameters and softmax, are denoted as the new label C. s Therefore, the distillation classification loss can be summarized as follows:

[0070] L soft_cls (s,t)=T 2 *KL div (C t C s )

[0071] Among them, KL div KL divergence is used to measure the difference between the classification outputs of the teacher model and the student model. The smaller the KL divergence value, the more similar the probability distributions of the two outputs are.

[0072] The calculation method for distillation regression loss differs from conventional regression calculation. Conventional regression directly calculates the L2 distance between the predicted bounding boxes inferred by the student model and those inferred by the teacher model, using this distance as the regression loss. This method only focuses on learning the output of the teacher model and ignores the directional nature of the truth. In this invention, the learning object is dynamically selected based on the actual situation. When the L2 distance between the student model's predicted bounding box and the soft label obtained in step (2) is larger, the soft label is used as the learning object; otherwise, the L2 distance is calculated between the student model's predicted bounding box and the teacher model's predicted bounding box. The resulting method for calculating distillation regression loss is as follows:

[0073]

[0074] Among them, b s For the student model prediction box, b t For the teacher model prediction box, b gt The soft tag obtained in step (2), 2 This represents the L2 distance.

[0075] In step (4), in order to obtain more accurate prediction boxes, nonmaximum suppression is generally used to eliminate redundant prediction boxes. The steps of nonmaximum suppression are as follows:

[0076] 1) Calculate the confidence scores of the predicted bounding boxes for all categories and sort them from highest to lowest;

[0077] 2) For each class, select the prediction box with the highest confidence and calculate the intersection and union ratio (IoU) with the subsequent prediction boxes. If the IoU is greater than the set threshold, remove the prediction box with the lower confidence; otherwise, keep the two prediction boxes.

[0078] 3) Starting from the second-highest confidence bounding box in each category, repeat step 2) until all the bounding boxes have been traversed.

[0079] Examples of the Intersection over Union (IOU) involved include: Figure 2a The figure shows the ratio of the overlap area to the union area of ​​two similar prediction boxes. The calculation formula is as follows:

[0080]

[0081] While nonmaximum suppression works well in eliminating redundant bounding boxes in conventional target detection tasks, the process of calculating the intersection-union ratio (IU) in SAR image recognition is prone to problems. This is because the unique characteristics of SAR images often result in anomalous aspect ratios of targets, such as... Figure 2a As shown, under normal aspect ratios, the intersection-over-union ratio (IoU) can effectively describe the overlap level of two predicted boxes; however, once the aspect ratio becomes abnormal, such as... Figure 2bAs shown, the intersection-over-union (IoU) ratio is significantly small. Following the conventional approach of nonmaximum suppression, both predicted bounding boxes should be retained. However, in reality, the smaller predicted bounding box often only records local information about the target and needs to be suppressed. If a small IoU threshold is simply chosen, most of the smaller predicted bounding boxes will be suppressed, resulting in the predicted bounding boxes failing to adequately cover the entire target.

[0082] like Figure 3 As shown, this invention proposes a novel predicted bounding box fusion strategy that can be directly applied after nonmaximum suppression, providing a plug-and-play solution; and it yields more suitable regression boxes. The processing flow of the predicted bounding box fusion strategy is as follows:

[0083] 1) Nonmaximum suppression is applied to the detection results;

[0084] 2) For each class, select the prediction box with the highest confidence and calculate the intersection ratio with the subsequent prediction boxes. Set two thresholds a and b (which can be set to 0.2 and 0.7 for SAR targets). If the intersection ratio of one prediction box is greater than b (set as box A) and the intersection ratio of another prediction box is less than a (set as box B), then proceed to step 3). Otherwise, directly compare the next prediction box.

[0085] 3) First, keep the center point of box A unchanged. Scale the length and width according to the ratio of its confidence level to the confidence level of box B (scale up if the confidence level ratio is greater than 1, shrink if it is less than 1) to obtain the scaled box M. Then, calculate the minimum bounding rectangle N between M and B, which is the final predicted box. Use the confidence level of box B as the confidence level of box N.

[0086] 4) Starting from the second-highest confidence bounding box in each category, repeat steps 2) and 3) in sequence until all the bounding boxes have been traversed.

[0087] 5) The SAR image target detection process ends after all prediction boxes have been traversed.

[0088] In summary, the SAR image target detection method combined with high-confidence knowledge distillation in the above embodiments applies the idea of ​​knowledge distillation to the SAR image target detection task and proposes a label reuse mechanism that can directly save the soft labels generated by the teacher model and reuse them when training the student model, without repeating the labeling, thus saving time and costs. The calculation formula of the distillation regression loss is optimized, and the original strategy of using the teacher model as the learning object is improved to select the learning object based on the magnitude of the L2 distance, which is more conducive to learning richer features. A prediction box fusion strategy that can be directly embedded in the non-maximum suppression is proposed. This strategy can reasonably optimize the boundary of the prediction box according to the confidence level, so as to cover the target as completely as possible.

[0089] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A SAR image target detection method combining high-reliability knowledge distillation, characterized in that, Includes the following steps: S1: Training Teacher Model Collect a SAR image dataset and label the SAR images in it. Build a teacher model. The teacher model outputs soft labels and hard labels. Use a portion of the image data in the SAR image dataset as the training dataset, and then use hard labels as supervision information to train the teacher model until the model converges. S2: Relabel the training dataset All training data from the training dataset are fed into the trained teacher model to obtain soft label values, which means the training data is re-labeled. Both hard and soft label values ​​are saved for use in training the student model. S3: Training the student model A student model is constructed, and the soft predictions output by the student model are made as close as possible to the soft labels inferred by the teacher model through the distillation loss function, so that the student model has the ability to output both soft and hard predictions. In step S3, the formula for the total loss is as follows: ； in This is the hard-label loss, i.e., the prediction loss. This is the soft label loss, i.e., distillation loss. These are the weighting coefficients; These correspond to the hard labels of the hard predicted values ​​and the true values ​​predicted by the student model, respectively. These correspond to the soft predictions made by the student model and the soft labels inferred by the teacher model, respectively. The formula for calculating the predicted loss is as follows: ； in, To predict classification loss, To predict regression loss; The formula for calculating distillation loss is as follows: ； in, For distillation classification loss, This represents the loss due to distillation regression. The formula for calculating the distillation regression loss is as follows: ； in, Predicted bounding boxes for student models. For the teacher model prediction box, The predicted bounding boxes in the soft labels output by the teacher model. Indicates L2 distance; S4: Reasoning in the Student Model After the student model is trained, the inference process of the student model is carried out using the prediction box fusion strategy.

2. The SAR image target detection method combining high-confidence knowledge distillation according to claim 1, characterized in that: In step S1, the annotation method is to annotate the x and y axis coordinates of the center point of each target, the target length w and width h, and the target category.

3. The SAR image target detection method combining high-confidence knowledge distillation according to claim 1, characterized in that: In step S1, the soft label includes the x and y coordinates of the target's center point, the target's length w and width h, and the probability value for each category; the hard label includes the x and y coordinates of the target's center point, the target's length w and width h, and the target's category and confidence level.

4. The SAR image target detection method combining high-confidence knowledge distillation according to claim 1, characterized in that: The formula for calculating distillation separation loss is as follows: ； Where T is the temperature parameter. Let KL divergence be the KL divergence. , , The new label obtained from the soft label acquired in step S2, after smoothing with temperature parameter T and softmax, is denoted as [label name missing]. Similarly The soft predicted values ​​obtained from the student model, after being smoothed by the temperature parameter T and softmaxed, are denoted as the new labels. .

5. The SAR image target detection method combining high-confidence knowledge distillation according to claim 1, characterized in that: In step S4, the processing flow of the prediction box fusion strategy is as follows: S41: Perform nonmaximum suppression on the detection results; S42: For each class, select the prediction box with the highest confidence, and calculate the intersection ratio with the subsequent prediction boxes accordingly. Set two thresholds a and b. If the intersection ratio of a prediction box is greater than b, it is set as box A, and the intersection ratio of another prediction box is less than a, it is set as box B. Then proceed to step S43. Otherwise, directly compare the next prediction box. S43: Keep the center point of box A unchanged, scale the length and width according to the ratio of its confidence level to the confidence level of box B to obtain the scaled box M. Then, find the minimum bounding rectangle N between box M and box B as the final prediction box, and use the confidence level of box B as the confidence level of box N. S44: Starting from the second-highest confidence bounding box in each class, repeat steps S42 and S43 in sequence until all the bounding boxes have been traversed. S45: The SAR image target detection process ends after all prediction boxes have been traversed.

6. The SAR image target detection method combining high-confidence knowledge distillation according to claim 5, characterized in that: In step S43, if the confidence ratio of box A and box B is greater than a set value, it is increased; if it is less than a set value, it is decreased.

Images