Target detection method based on grouping Hungary matching and loU perception

Abstract

The invention provides a target detection method based on grouping Hungary matching and l oU perception, and relates to the technical field of image detection. According to the target detection method based on grouping Hungary matching and l oU perception, multi-scale Hungary matching and multi-scale query selection are adopted, so that the situation that the final prediction target is biased to a large target is avoided, and the precision of a small target is improved; meanwhile, an IoU branch is added in the training stage, the combination of the IoU confidence coefficient and the classification confidence coefficient is used as a comprehensive confidence coefficient evaluation index, and in the Topk stage, an initial anchor frame with higher quality can be selected, so that the model can be fitted more quickly, and higher precision improvement is achieved when 12 epoch is reached.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a target detection method based on group Hungarian matching and loU perception. Background Art

[0002] Object detection is one of the basic tasks in computer vision, which is to identify and locate objects in images. There are two main types of modern object detection algorithms: one is based on CNN, and the other is based on transformer. However, the detection algorithms based on CNN need to manually set thresholds due to the need for NMS post-processing, and the training epoch is relatively long. The transformer-based object detection algorithm solves the above problems of CNN by optimizing the model structure. The DETR series is a model structure paradigm of the transformer architecture. At present, the DETR series has the problem of low accuracy in detecting small targets.

[0003] The study found the following two problems: 1. Most of the receptive fields of DETR come from deep feature maps, which causes the final predicted targets to be biased towards large targets, resulting in low detection accuracy for small targets; 2. The quality of Detr's initial anchor box is low (the classification confidence and IoU confidence of the anchor box are inconsistent), resulting in more training rounds to make the model more accurate.

[0004] To this end, we developed a new target detection method based on group Hungarian matching and loU awareness. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a target detection method based on group Hungarian matching and loU perception, which solves the problem that the current DETR series in the prior art has low small target detection accuracy.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: a target detection method based on group Hungarian matching and loU perception, the algorithm model is based on the DINO model and is a DETR paradigm, including the following specific steps:

[0009] The image is subjected to four-layer feature extraction from the backbone, and the positional encoding fusion is input into the encoder for simultaneous feature fusion within and across scales;

[0010] The tokens for predicting objects of different sizes are obtained at different scales in different proportions. That is, the topK tokens are grouped according to the proportions at different scales to select the tokens with the highest confidence at each scale. The tokens are passed through three branch prediction heads, including the bbox branch prediction head, the category branch prediction head, and the IoU branch prediction head. The IoU branch prediction value and the category confidence are combined as a new evaluation criterion.

[0011] Then the multi-scale query selection module is executed, and the category prediction head and IoU prediction head are used to form the category score and IoU score respectively. The obtained class score and IoU score are combined as a new confidence judgment indicator, and then topK are performed within different groups to filter out the token with the highest confidence. The token is sent to the box prediction head to generate an initialized prediction box, and then the prediction box corresponding to the token is used as the input data of the initial box and decoder. The prediction box is sent to the decoder part and fused with the position encoding to form a query, and memory-related features are further extracted, and then multi-scale label allocation training is performed.

[0012] Furthermore, an algorithm is provided to determine GT boxes of different scales and tokens of different scales for Hungarian group matching:

[0013] α i ∈[0,1], i=1...m, the value range is between 0 and 1,

[0014]

[0015]

[0016]

[0017]

[0018] POS ij ,neg ij =P(Q i ,GT j ), i=1...m, j=1...n (5)

[0019] pos=[pos1,...,pos m ]neg=[neg1,...,neg m ] (6)

[0020] Among them, n is the number of GT boxes and m is the number of groups.

[0021] Furthermore, an algorithm for a multi-scale selection module is provided, comprising:

[0022] Yes The form after flattening in the spatial direction is used as the input data of the module as shown in the following formula:

[0023] T i =Flatten(F i ), i=1...m (7)

[0024] is the i-th layer feature map, with a scale of (hi, wi), m is the number of layers of the feature map, PredClassHead is the category prediction head, Ti is flattened and concatenated as the overall input, and the category score C∈R of each token is obtained through PredClassHead 1×u×k As shown in the following formula:

[0025] C=PredClassHead(T1,T2,...,T m ) (8)

[0026] D∈R 1×u It is the confidence level of the maximum value of C in the last dimension, representing the maximum probability of the final predicted category of the token;

[0027] A token will predict the probability of the object belonging to different categories. Finally, the maximum value is taken to obtain the probability of the maximum category prediction, which is the confidence level as shown in the following formula:

[0028] D=max(C,dim=-1) (9)

[0029] Where dim is the dimension of C. In DINO, feature maps at different scales are concatenated together, and then the confidence at all scales is predicted through the category prediction branch. The top N tokens are sorted according to the confidence, and the position information indices predicted by the token is used as the position information (x, y, w, h) initialized by the downstream query. 1×N The subscripts of the top N confidences in D ranked by topK also represent the token numbers as shown in the following formula:

[0030] indices = topK(D,N) (10)

[0031] N is the number of topK selected. In the model, the corresponding number is first divided according to the ratio of the number of tokens in different feature maps to the number of all tokens in the feature maps. Finally, the number of top ranked tokens to be selected is n. i , as shown in the following formula:

[0032]

[0033] (hi, wi) represents the size of the feature map of the i-th layer. There are m layers of feature maps in total. The flattening operation is performed like DINO, and the feature maps of different layers are flattened separately. The flattened form of different feature maps is shown in the following formula:

[0034] T i =Flatten(F i ), i=1...m (12)

[0035] Represents the feature map of the i-th layer, m represents the number of scale layers, and a single scale is flattened and independently enters the category prediction head for category confidence prediction. As shown in the following announcement:

[0036] C i =PredClassHead(T i ), i=1...m (13)

[0037] m: the number of layers of the feature map, Ti: the expanded form of the feature maps of different layers, and the maximum value of the confidence of the corresponding category of each layer token is obtained at each scale As shown in the following formula:

[0038] D i =max(C i , dim=-1) (14)

[0039] Where dim: is the number of categories, is the category probability predicted by each of the above tokens. TopK is performed on each scale to obtain the top n categories on each scale. i The location information corresponding to the token: As shown in the following formula:

[0040] indices i =topK(D i ,n i ), i=1...m (15)

[0041] Finally, the position index information at all scales is concatenated together to obtain indices∈R 1×N , as shown in the following formula:

[0042] indices=[indices1,...,indices m ] (16).

[0043] Furthermore, an IoU branch prediction value algorithm is provided, including:

[0044] u∈R B×N×1 For the predicted IoU value, the IoU value corresponding to the predicted box and the real box is taken as the real information and trained through L2 loss, as shown in the following formula:

[0045] u=Linear(token) (17)

[0046] Where token∈R B×N×C For feature points, in the topK stage, the classification and IoU value are combined as a comprehensive evaluation index for screening. In the inference stage, this combination is used as a threshold evaluation index to filter the box, as shown in the following formula:

[0047] t=s α *u 1-α (18)

[0048] Among them, s is the predicted classification score and u is the predicted IoU score.

[0049] (III) Beneficial effects

[0050] The present invention provides a target detection method based on group Hungarian matching and loU perception, which has the following beneficial effects:

[0051] 1. This target detection method based on grouped Hungarian matching and loU perception adopts multi-scale Hungarian matching and multi-scale query selection, which avoids the situation that the final predicted targets are biased towards large targets and improves the accuracy of small targets;

[0052] 2. This target detection method based on grouped Hungarian matching and loU perception adds an IoU branch in the training stage, and combines IoU confidence and classification confidence as a comprehensive confidence evaluation indicator. In the Topk stage, it can select higher quality initial anchor frames, allowing the model to fit faster, thereby achieving higher accuracy improvement in 12 epochs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the overall model detection method flow of the present invention;

[0054] Figure 2 It is a model structure diagram of the present invention;

[0055] Figure 3 A diagram of the grouped Hungarian matching method of the present invention;

[0056] Figure 4 This is a diagram of the multi-scale selection module of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Example

[0059] like Figure 1 - Figure 4 As shown, the embodiment of the present invention provides a target detection method based on group Hungarian matching and loU perception, the algorithm model is based on the DINO model and is a DETR paradigm, and includes the following specific steps:

[0060] Step 1: Extract four layers of features from the backbone image, input the positional encoding fusion into the encoder, and perform feature fusion within and across scales simultaneously;

[0061] Step 2: Obtain tokens for predicting objects of different sizes at different scales in different proportions. That is, group the topK tokens at different scales according to the proportions to select the tokens with the highest confidence at each scale. Pass the tokens through three branch prediction heads, including the bbox branch prediction head, the category branch prediction head, and the IoU branch prediction head. Combine the IoU branch prediction value and the category confidence as a new evaluation criterion.

[0062] Among them, considering that different queries should be responsible for finding objects of different sizes, the idea of ​​grouping is adopted to perform Hungarian group matching on GT boxes of different scales and tokens of different scales. Subsequently, GT boxes of different scales and queries of different groups can also be matched in Hungarian groups, thereby guiding tokens of different scales and queries of different groups to pay attention to objects of different sizes.

[0063] Specifically, an algorithm is provided to determine GT boxes of different scales and tokens of different scales for Hungarian group matching:

[0064] α i ∈[0,1], i=1...m, this set of parameters is a threshold division interval, which needs to be determined manually. It is a relative threshold with a value range between 0 and 1. Next, the relative threshold is converted into an absolute threshold as shown in the following formula (1):

[0065]

[0066] In order to determine which group the GT box is divided into, where n is the number of GT boxes, we will obtain the absolute area of ​​the GT box as shown in the following formula (2):

[0067]

[0068] if Then the GT frame is assigned to the i-th group, and the anchor frame of the i-th group is subjected to position encoding and MLP to obtain the position embedding information as shown in the following formula (3):

[0069]

[0070] Then, a learnable content information is initialized, and the location information and content information are concatenated to obtain the initialization query corresponding to the i-th group as follows (4):

[0071]

[0072] The query generated by the above i-th group and the corresponding GT box are subjected to Hungarian matching to obtain the position index of the positive and negative samples corresponding to the i-th group as shown in the following formula (5):

[0073] POS ij ,neg ij =P(Q i ,GT j ), i=1...m, j=1...n (5)

[0074] Where m is the number of groups and n is the number of ground truth boxes. Finally, the indexes of positive and negative samples of all groups are concatenated together to form the final positive and negative samples, as shown in the following formula (6):

[0075] pos=[pos1,...,pos m ]neg=[neg1,...,neg m ] (6);

[0076] Step 3: Then execute the multi-scale query selection module, use the category prediction head and IoU prediction head to form the category score and IoU score respectively, combine the obtained category score and IoU score together as a new confidence judgment indicator, and then perform topK screening in different groups to select the token with the highest confidence, send the token to the box prediction head to generate the initialized prediction box, and then use the prediction box corresponding to the token as the input data of the initial box and decoder, send the prediction box to the decoder part and the position encoding fusion to form a query, further extract memory-related features, and then perform multi-scale label allocation training;

[0077] Specifically, an algorithm for a multi-scale selection module is provided, including:

[0078] Yes The form after flattening in the spatial direction is used as the input data of the module as shown in the following formula:

[0079] T i =Flatten(F i ), i=1...m (7)

[0080] is the i-th layer feature map, with a scale of (hi, wi), m is the number of layers of the feature map, PredClassHead is the category prediction head, Ti is flattened and concatenated as the overall input, and the category score C∈R of each token is obtained through PredClassHead 1×u×80 As shown in the following formula:

[0081] C=PredClassHead(T1,T2,...,T m ) (8)

[0082] D∈R 1×u It is the confidence level of the maximum value of C in the last dimension, representing the maximum probability of the final predicted category of the token;

[0083] A token will predict the probability of the object belonging to different categories. Finally, the maximum value is taken to obtain the probability of the maximum category prediction, which is the confidence level as shown in the following formula:

[0084] D=max(C,dim=-1) (9)

[0085] Where dim is the dimension of C. In DINO, feature maps at different scales are concatenated together, and then the confidence at all scales is predicted through the category prediction branch. The top N tokens are sorted according to the confidence, and the position information indices predicted by the token is used as the position information (x, y, w, h) initialized by the downstream query. 1×N The subscripts of the top N confidences in D ranked by topK also represent the token numbers as shown in the following formula:

[0086] indices = topK(D,N) (10)

[0087] N is the number of topK selected. In the model, the corresponding number is first divided according to the ratio of the number of tokens in different feature maps to the number of all tokens in the feature maps. Finally, the number of top ranked tokens to be selected is n. i, as shown in the following formula:

[0088]

[0089] (hi, wi) represents the size of the feature map of the i-th layer. There are m layers of feature maps in total. The flattening operation is performed like DINO, and the feature maps of different layers are flattened separately. The flattened form of different feature maps is shown in the following formula:

[0090] T i =Flatten(F i ), i=1...m (12)

[0091] Represents the feature map of the i-th layer, m represents the number of scale layers, and a single scale is flattened and independently enters the category prediction head for category confidence prediction. As shown in the following announcement:

[0092] C i =PredClassHead(T i ), i=1...m (13)

[0093] m: the number of layers of feature maps, Ti: the form of feature maps of different layers after display, and the maximum value of the confidence of the corresponding category of each layer token at each scale As shown in the following formula:

[0094] D i =max(C i , dim=-1) (14)

[0095] Where dim: is the number of categories, is the category probability predicted by each of the above tokens. TopK is performed on each scale to obtain the top n categories on each scale. i The location information corresponding to the token: As shown in the following formula:

[0096] indices i =topK(D i ,n i ), i=1...m (15)

[0097] Finally, the position index information at all scales is concatenated together to obtain indices∈R 1×N , as shown in the following formula:

[0098] indices=[indices1,...,indices m ] (16);

[0099] In DINO, top-k selects the predicted position information of 900 tokens from the tokens as the query position information. However, it can be found that most of the selected tokens have relatively large receptive fields, which will cause most of the initialized anchor boxes to float around large objects. This will cause the anchor boxes responsible for predicting small objects to be around large objects or in the background, making it difficult to optimize towards small objects. Therefore, we use different proportions of the number of tokens to obtain tokens that predict objects of different sizes at different scales.

[0100] Based on these characteristics, the token is passed through the three branch prediction heads respectively. Furthermore, it is observed that the classification confidence and IoU confidence of the anchor boxes in the topK stage are quite different, that is, the classification confidence and IoU confidence of a certain anchor box are not uniform. There are some low-quality boxes with high category scores but low IoU scores, and some low-quality boxes with low category scores and high IoU scores. The anchor boxes with good quality have high category scores and high IoU scores. Not only the predicted category of the box is accurate, but the positioning of the box is also more accurate.

[0101] After counting the category confidence and IoU confidence of the predicted box generated by the token before topK in DINO, it is found that most of the blue points are distributed in the upper left and lower right of the image. If the points are close to the straight line in the middle, the quality of the box is better. In order to solve the above phenomenon, an independent fully connected branch will be used to perceive the IoU value of each predicted box, and an algorithm for IoU branch prediction value will be provided, including:

[0102] u∈R B×N×1 To predict the IoU value, we take the IoU value corresponding to the predicted box and the real box as the real information and train it through L2 loss, as shown in the following formula (11):

[0103] u=Linear(token) (17)

[0104] Where token∈R B×N×C For feature points, in the topK stage, we no longer simply use the classification score as the ranking standard, but use the classification and IoU value as a comprehensive evaluation indicator for screening. In the inference stage, we will also use this combination as a threshold evaluation indicator to filter boxes. As shown in the following formula (18):

[0105] t=s α *u 1-α (18)

[0106] s is the predicted classification score and u is the predicted IoU score.

[0107] Ablation experiment

[0108]

[0109] Table 1: Ablation experiment of each component epoch = 12, where SLM: scalelabelmatch module, SQS: scalequeryselection module, IoU: IoUbranch module

[0110] The ablation study of the designed components was conducted under the condition of 12 epochs. These two parts are indispensable. The SLM part is to hierarchically separate and distinguish tokens with different receptive fields, and provide grouping ratios for subsequent query grouping, so that the initial distribution of subsequent queries is exactly distributed around the predictions of each token, which can further improve the position convergence speed of the model. Different groups of queries can predict objects of different scales by adaptively obtaining feature information of objects of different scales in cross-attention, so the competition between queries responsible for predicting different scales is reduced. The competition between queries of the same group at the same scale is also reduced because the number of queries in the same group is greatly reduced. Through the combination of SLM and SQS, we have greatly improved the model's improvement on small and medium targets, with an AP improvement of 1.8% for small targets and 0.2% for medium targets. We can find that adding only the SLM part will have a great impact on the model. This is because only grouping tokens, but not grouping queries in the same way, will affect the query's judgment of the target scale, which is harmful to the model accuracy.

[0111] By adding the IoU branch, it can be clearly found that the accuracy of AP75 is further improved by 0.2% AP, and the accuracy of medium and large targets is also improved by 0.8% and 0.5% respectively. We guess that this is because the semantic information and target information of medium and large targets are sufficient, so the query responsible for predicting medium and large targets converges faster. However, it is interesting that only adding the IoU branch will reduce the performance of AP50 below the original value of DINO. This may be because when the IoU threshold is low, the shape of the box is very different, which will affect the suppression and convergence of the downstream box, but our previous method can effectively alleviate this problem;

[0112]

[0113] Table 2: Ablation of various ratio comparisons of grouping in scalelabelmatch module epochs = 12

[0114] The grouping method of the GT box in the SLM module is scaled to ensure that the GT boxes of different scales are matched one-to-one with the query responsible for predicting the group, so as to divide the correct positive and negative samples. We use 4 scale feature layers, so we need to divide the query into 4 groups. When the area of ​​the GT box belongs to a certain threshold range mentioned above, the label is assigned to the query within the group. Therefore, the division of the threshold is sensitive to the performance of the model. We tried different combinations, some of which will lead to a decrease in performance. After multiple groups of ablation studies, we finally adopted the parameter combination of [0.2, 0.6, 0.8, 1]. In the end, we used the square of this combination, that is, [0.04, 0.36, 0.64, 1] as the threshold combination division. Under the condition of epochs = 12, the accuracy can reach 49.4% AP. We can see that this combination is also the best for large, medium and small targets.

[0115] Experimental Results

[0116] Comparison results of using DINO as the baseline and other models with the model of this application. The model of this application uses 900query like DINO. The models shown in * are all built based on the mmdetection framework, and the model data are all trained on a single card 3090:

[0117] We use DINO as a baseline for experiments. The baseline uses deformable-transformer as the backbone and uses CDN, lookforward-twice and other methods to improve performance. We follow DINO's method and adopt its techniques.

[0118]

[0119] Table 3: Main results

[0120] For backbone, we use 4 scales of resnet50 backbone network as input. The results are shown in Table 2. Under the condition of epochs=12, our method improves 2.3% points for small targets and 1.2% for medium targets, but it is found that the performance of large targets decreases by 0.5%. According to our experience, the model we proposed may slow down the convergence speed of large targets in terms of the partition ratio parameter setting. When our model is at epochs=24 and epochs=36, the accuracy of large targets also increases. It can be seen that our model will pay more attention to the small target area and improve the convergence speed of small targets.

[0121] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The target detection method based on group Hungarian matching and loU perception, the algorithm model is based on the DINO model and is a DETR paradigm, characterized by: The specific steps include: The image is subjected to four-layer feature extraction from the backbone, and the positional encoding fusion is input into the encoder for simultaneous feature fusion within and across scales; At different scales, we group topK tokens according to their proportions to select the tokens with the highest confidence at each scale. We pass the tokens through three branch prediction heads, including the bbox branch prediction head, the category branch prediction head, and the IoU branch prediction head. We combine the IoU branch prediction value and the category confidence as a new evaluation criterion. Execute the multi-scale query selection module, use the category prediction head and IoU prediction head to form the category score and IoU score respectively, combine the obtained category score and IoU score together as a new confidence judgment indicator, and then perform topK screening in different groups to select the token with the highest confidence, send the token to the box prediction head to generate the initialized prediction box, and then use the prediction box corresponding to the token as the input data of the initial box and decoder, send the prediction box to the decoder part and the position encoding fusion to form a query, further extract memory-related features, and then perform multi-scale label allocation training.

2. The target detection method based on group Hungarian matching and loU perception according to claim 1 is characterized in that: Provide an algorithm to determine GT boxes of different scales and tokens of different scales for Hungarian group matching: α i ∈[0,1], i=1...m, the value range is between 0 and 1, pos ij ,neg ij =P(Q i ,GT j ),i=1...m,j=1...n(5) Where n is the number of GT frames, m is the number of groups, and multiple queries in the i-th group and multiple GT frames divided into the i-th group are Hungarian matched to form the result of the i-th group Hungarian matching. The number of positive samples formed by matching in this group = the total number of GTs, and the number of negative samples = the total number of queries - the number of positive samples; pos=[pos1,...,pos m ]neg=[neg1,...,neg m ] (6) The final result is a set of m groups of positive samples merged together and a set of m groups of negative samples merged together to form positive and negative samples respectively.

3. The target detection method based on group Hungarian matching and loU perception according to claim 2 is characterized in that: An algorithm for multi-scale selection module is provided, including: Yes The form after flattening in the spatial direction is used as the input data of the module as shown in the following formula: T i =Flatten(F i ),i=1...m (7) is the i-th layer feature map, the scale is (hi, wi), m is the number of layers of the feature map, PredClassHead is the category prediction head. Ti is flattened and concatenated as the overall input and then passed through PredClassHead to obtain the category score C∈R of each token. 1×u×80 As shown in the following formula: C=PredClassHead(T1,T2,...,T m ) (8) D∈R 1×u It is the confidence level of the maximum value of C in the last dimension, representing the maximum probability of the final predicted category of the token; A token will predict the probability of the object belonging to different categories. Finally, the maximum value is taken to obtain the probability of the maximum category prediction, which is the confidence level as shown in the following formula: D=max(C,dim=-1) (9) Where dim is the dimension of C. In DINO, feature maps at different scales are concatenated together, and then the confidence at all scales is predicted through the category prediction branch. The top N tokens are sorted according to the confidence, and the position information indices predicted by the token is used as the position information (x, y, w, h) initialized by the downstream query. 1×N The subscripts of the top N confidences in D ranked by topK also represent the token numbers as shown in the following formula: indices = topK(D,N) (10) N is the number of topK selected. In the model, the corresponding number is first divided according to the ratio of the number of tokens in different feature maps to the number of all tokens in the feature maps. Finally, the number of top ranked tokens to be selected is n. i , as shown in the following formula: (hi, wi) represents the size of the feature map of the i-th layer. There are m layers of feature maps in total. The flattening operation is performed like DINO, and the feature maps of different layers are flattened separately. The flattened form of different feature maps is shown in the following formula: T i =Flatten(F i ),i=1...m (12) Represents the feature map of the i-th layer, m represents the number of scale layers, and a single scale is flattened and independently enters the category prediction head for category confidence prediction. As shown in the following formula: C i =PredClassHead(T i ),i=1...m (13) m: the number of layers of the feature map, Ti: the expanded form of the feature maps of different layers, and the maximum value of the confidence of the corresponding category of each layer token is obtained at each scale As shown in the following formula: D i =max(C i ,dim=-1) (14) Where dim: is the number of categories, is the category probability predicted by each of the above tokens. TopK is performed on each scale to obtain the top n categories on each scale. i The location information corresponding to the token: As shown in the following formula: indices i =topK(D i ,n i ),i=1...m (15) Finally, the position index information at all scales is concatenated together to obtain indices∈R 1×N , as shown in the following formula: indices=[indices1,...,indices m ] (16).

4. The target detection method based on group Hungarian matching and loU perception according to claim 3 is characterized in that: An algorithm for calculating an IoU branch prediction value is provided, including: u∈R B×N×1 For the predicted IoU value, the IoU value corresponding to the predicted box and the real box is taken as the real information and trained through L2 loss, as shown in the following formula: u=Linear(token) (17) Where token∈R B×N×C For feature points, in the topK stage, the classification and IoU value are combined as a comprehensive evaluation index for screening. In the inference stage, this combination is used as a threshold evaluation index to filter the box, as shown in the following formula: t=s α *u 1-α (18) Among them, s is the predicted classification score and u is the predicted IoU score.

Images