An Anchor-based Object Detection Method for the Entire Transformer Framework

Abstract

The present invention discloses an anchor-based object detection method for the full Transformer framework, comprising the following steps: using the hierarchical encoder of Swin Transformer as the backbone network of the object detection model and training it; inputting the image to be detected into the trained model, and using the hierarchical encoder of the model to extract image features; fusing features of different scales through a convolution-free feature fusion module; the fused feature layers then pass through a decoder, where a fixed number of anchor points / anchor boxes are assigned to each feature layer, and the center point coordinates and length and width values of the object bounding boxes are predicted for the anchor points / anchor boxes; positive samples are assigned to the prediction results, and conflict resolution and adjustment are performed. The detection method disclosed by the present invention achieves performance comparable to or even better than that of convolutional neural networks by only using the Transformer framework model without any convolutional layer operations.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to an anchor-based object detection method for a full Transformer framework. Background Art

[0002] With the in-depth research on deep learning and computer vision, Transformer has partially exceeded or even completely exceeded the performance of convolutional neural networks in many directions in the field of computer vision. Object detection is an important research direction in the field of computer vision and has a very broad market in many important fields such as surveillance deployment, violation detection, and medical image analysis. However, due to the development of convolutional neural networks in recent years, the structures and frameworks of current mainstream object detectors are relatively single, mainly adopting fully convolutional neural network structures or structures combining convolutional neural networks and Transformer, lacking the existence of object detectors with a full Transformer structure. Since Transformer exhibits more excellent potential and performance than convolutional neural networks, the exploration of an object detection network with a full Transformer architecture in the present invention helps to further promote the progress of Transformer methods. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides an anchor-based object detection method for a full Transformer framework, constructs an object detection model with a full Transformer framework, expands the application of Transformer in object detection and also achieves good results.

[0004] To achieve the above object, the technical solution of the present invention is as follows:

[0005] An anchor-based object detection method for a full Transformer framework, comprising the following steps:

[0006] Step 1, using the hierarchical encoder of Swin Transformer as the backbone network of the full Transformer object detection model and training it;

[0007] Step 2, inputting the image to be detected into the trained model, and using the hierarchical encoder of the model to extract image features;

[0008] Step 3, fusing features of different scales through a non-convolutional feature fusion module for top-down and bottom-up fusion of feature layers of different scales;

[0009] Step 4: The fused feature layer then passes through the decoder. A fixed number of anchor points / anchor boxes are assigned to each feature layer, and the center coordinates, width, and height values of the target bounding boxes are predicted for the anchor points / anchor boxes to obtain the prediction results;

[0010] Step 5: Positive samples are assigned to the prediction results of the decoder, and conflict resolution and adjustment are performed on the repeatedly assigned identical samples.

[0011] In the above solution, in Step 1, during the training process of the model, the category loss uses the cross-entropy loss. The cross-entropy loss CE formula is as follows:

[0012]

[0013] where N represents the total number of samples in the current batch, M represents the total number of categories, i represents the i-th sample, c represents the current category, y ic represents the sign function 0 or 1, taking 1 if the true category of sample i is equal to c, otherwise taking 0, and p ic represents the probability that sample i belongs to the c-th category, and ic represents that sample i belongs to the c-th category;

[0014] Use Distance to represent the center point distance between all anchor boxes and all target bounding boxes, and x bbox , y bbox represent the relative x and y coordinates of the center point of the target bounding box, and x ap , y ap represent the relative x and y coordinates of the anchor point;

[0015] The point score P formula is as follows, and the point score loss is the mean square error loss:

[0016] P = G(F(Distance))

[0017] where,

[0018] The target bounding box loss uses the complete intersection over union loss, and the complete intersection over union loss is as follows:

[0019]

[0020] where v represents the similarity of the aspect ratio, gt represents the target true value, w represents the predicted target width, h represents the predicted target height, w gt represents the target true value width, and h gt represents the target true value height, α represents the parameter weight of v to adjust the weight of v, and IOU represents the intersection over union;

[0021] The confidence loss uses the binary cross-entropy loss, and the binary cross-entropy loss BCE formula is as follows:

[0022]

[0023] Among them, y i represents the class label of the i-th sample. The positive sample is 1, and the negative sample is 0. p i represents the probability that the i-th sample is a positive sample.

[0024] In the above solution, in step two, the implementation method of the hierarchical encoder is as follows:

[0025] Use a small window to slice the feature layer. First, perform local attention calculation inside each small window, and then shift the window to the right and downwards so that the attention of the original adjacent four windows can be propagated when calculating the attention in the new window; the window attention calculation formula is as follows:

[0026]

[0027] Among them, Q represents the query, K represents the key, V represents the value, B is the relative position bias, and d represents the dimension of K;

[0028] For a window with length and width M, first subtract the reference position coordinates from the absolute position coordinates to obtain the relative position coordinates, and then add M - 1 to the relative coordinates so that the relative position starts from 0. Multiply the row index by 2M - 1 and then add the vertical coordinate to obtain the relative position index. The relative position in the window obtains the relative position bias from the relative position bias table according to the relative position index.

[0029] In the above solution, in step three, the implementation method of the non-convolutional feature fusion module is as follows:

[0030] The upper-layer feature layer is transformed through block fusion to double the dimension and halve the width and height. The current feature layer undergoes dimension changes through a feed-forward neural network. The two are fused through a concatenation operation and then fused with the block fusion result of the upper-layer feature layer for top-down fusion; the feature layer after top-down fusion of this layer and the top-down feature layer of the previous layer are subjected to block fusion downsampling and then added to obtain the bottom-up fusion process. The entire process is composed of a combination of concatenation and linear transformation of the dimension and size of the feature layer, without using any convolutional layers.

[0031] In the above solution, the specific method of step four is as follows:

[0032] Each layer of the detection head consists of a group of stacked decoders. The input in the decoder is a fixed-position query, and each query only predicts a single region. All the regions represented by the queries cover the entire feature map. Each query is responsible for making a one-to-one fixed prediction for the region it represents, and each region is represented by an anchor point. The predicted results are divided into parallel branches: a class branch and a bounding box branch. In the class branch, the result first passes through a feed-forward network and then is divided into two branches: a class branch and a point score branch. Each branch outputs the prediction result through a fully connected layer. The class branch is the score for each class, and the number is the same as the number of classes in the dataset. The point score branch is the mapping of the distance between the predicted point of each anchor and the center point of the target. The bounding box branch is also divided into two branches, and each branch also passes through a fully connected layer to output the corresponding result. The two branches are the bounding box parameters of the target and the existence target score corresponding to each predicted bounding box.

[0033] A certain number of anchor points are assigned to each feature layer. The intervals between the anchor points of each feature layer are different, and the intervals increase as the downsampling factor of the feature layer increases. The number of horizontal and vertical coordinates of the anchor points is generated according to the following formula:

[0034]

[0035] where feature layer represents the serial number of the feature layer, width and height represent the width and height of the feature layer, and nx and ny represent the number of horizontal anchor points and the number of vertical anchor points;

[0036] Each feature layer determines the number of anchor points of the current feature layer according to its serial number, width, and height, and generates the corresponding grid according to the number of horizontal and vertical coordinates. Since the grid starts from (0, 0), the grids of all feature layers are offset 0.5 units to the right and down.

[0037] The anchor method directly predicts the coordinate parameters of the target x, y, w, and h according to the anchor points. t x and t y are the direct prediction results of the network regarding x and y. b x , b y are the final prediction results regarding x and y. The specific formula is as follows:

[0038]

[0039] where σ is the Sigmoid function, 2×[σ(t) - 0.5] is the offset of the relative coordinates of the target center point and the assigned anchor point, and the prediction method for the anchor w and h is to directly predict the relative values of the width and height of the current feature layer;

[0040] The anchor box method is based on anchor points. Different ratios of anchor boxes are assigned to each anchor point. The anchor point serves as the center of the anchor box. The ratios of the anchor boxes are 1:1, 1:2, and 2:1, representing targets with different widths and heights. The anchor boxes are obtained by clustering the lengths and widths of the bounding boxes assigned by different feature layers. The anchor point prediction method directly predicts the relative offset of the coordinates of the anchor point to the center of the target point and the relative values of the width and height based on the feature layer. The prediction method of the anchor box has the same center point offset as the anchor point method, and the width and height are predicted based on multiples of the size of the assigned anchor box.

[0041] The sizes of the targets predicted by each feature layer are different. The size of the target bounding box is assigned to the corresponding feature layer according to the following formula:

[0042]

[0043] In the above solution, the specific method of step five is as follows:

[0044] In the positive sample assignment strategy, positive samples are assigned to the anchor points near the center point of the target bounding box for matching. Specifically, the anchor point closest to the center point of the target bounding box is assigned as the center point of the positive sample. Then, three offset points are determined based on the center point of the positive sample and the center point of the target bounding box. The positive samples of the anchor box are determined based on the intersection-over-union ratio of the anchor box and the target bounding box on the basis of the positive sample of the anchor point. If the intersection-over-union value exceeds 0.5, the anchor box is considered a positive sample.

[0045] The conflict resolution strategy needs to be carried out according to the priority of the sample points. The priority of the center point of the positive sample is the highest, and the priorities of the offset points are the same. When there is a conflict at the center point of the positive sample, the nearest center point of the positive sample is retained according to the distance. When there is a conflict between the center point of the positive sample and the offset point, the offset point is directly removed from the positive sample and the center point is retained. When there is a conflict between the offset points, the nearest offset point is retained according to the distance.

[0046] Through the above technical solution, a target detection method based on an anchor-based full Transformer framework provided by the present invention has the following beneficial effects:

[0047] (1) The present invention fully considers the advantages and potential of the Transformer model in target detection, designs a target detection method with a full Transformer framework, and expands the target detection method based on Transformer. The Swin Transformer backbone network extracts features, and the non-convolutional feature fusion module Layer Merging achieves an effect equivalent to that of the convolutional feature fusion module. The one-to-one fixed region and the side head add position representations and can accelerate convergence.

[0048] (2) Without using any convolutional layer operations, the present invention achieves performance equivalent to or even better than that of the convolutional neural network only using the Transformer framework model. Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art.

[0050] Figure 1 Schematic flow chart of an anchor-based full Transformer framework object detection method disclosed in an embodiment of the present invention.

[0051] Figure 2 Structure diagram of a convolution-free feature fusion module disclosed in the present invention.

[0052] Figure 3 Schematic flow chart of the decoder prediction head prediction disclosed in the present invention. Detailed Embodiments

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

[0054] Without using any convolution operations, the present invention constructs an object detection model with a full Transformer framework. The object detection model composed of a hierarchical encoder, a one-to-one fixed-region decoder, a convolution-free feature fusion module composed of concatenation and linear transformation, and an anchor point / anchor box-based object detection method expands the application of Transformer in object detection and also achieves good results. The specific embodiments are as follows:

[0055] An anchor-based full Transformer framework object detection method, as Figure 1 shown, includes the following steps:

[0056] Step 1, use the hierarchical encoder of Swin Transformer as the backbone network of the full Transformer object detection model and train it;

[0057] During the training process of the model, the class loss is the same as that of other object detection methods, and the cross-entropy loss is adopted. The cross-entropy loss CE formula is as follows:

[0058]

[0059] where N represents the total number of samples in the current batch, M represents the total number of classes, i represents the i-th sample, c represents the current class, y ic represents the sign function 0 or 1, which takes 1 if the true class of sample i is equal to c, otherwise takes 0, and p icIndicates the probability that sample i belongs to the c-th category, where ic indicates that sample i belongs to the c-th category;

[0060] Use Distance to represent the center point distance between all anchor boxes and all target bounding boxes, x bbox , y bbox Represents the relative x and y coordinates of the center point of the target bounding box, x ap , y ap Represents the relative x and y coordinates of the anchor point;

[0061] The point score P formula is as follows, and the point score loss is the mean squared error loss:

[0062] P = G(F(Distance))

[0063] Among them,

[0064] The target bounding box loss uses the complete intersection over union loss. The complete intersection over union loss comprehensively considers the overlapping area, the distance of the center point, and the aspect ratio, which can accelerate the convergence of the network and achieve better results. The complete intersection over union loss is as follows:

[0065]

[0066] Among them, v represents the similarity of the aspect ratio, gt represents the ground truth, w represents the predicted target width, h represents the predicted target height, w gt Represents the ground truth width, h gt Represents the ground truth height, α represents the parameter weight of v to adjust the weight of v, and IOU represents the intersection over union;

[0067] The confidence loss uses the binary cross-entropy loss. The target score of the positive sample should all be the intersection over union value with the target. When the predicted target completely overlaps with the true target, the predicted value should be 1. The binary cross-entropy loss BCE formula is as follows:

[0068]

[0069] Among them, y i Represents the class label of the i-th sample, 1 for positive samples and 0 for negative samples, p i Represents the probability that the i-th sample is a positive sample.

[0070] Step 2: Input the image to be detected into the trained model, and use the hierarchical encoder of the model to extract image features.

[0071] The implementation method of the hierarchical encoder is as follows:

[0072] For the feature layer, the global attention mechanism is no longer calculated. Instead, the feature layer is segmented using small windows. First, local attention is calculated within each small window, which can significantly reduce the computational complexity of global attention calculation. Then, the window is shifted to the right and downwards, enabling the attention of the original adjacent four windows to be propagated when calculating the attention of the new window. This can not only reduce the computational complexity but also achieve the effect of global attention through attention propagation.

[0073] The window attention calculation formula is as follows:

[0074]

[0075] Among them, Q represents the query, K represents the key, V represents the value, B is the relative position bias, and d represents the dimension of K;

[0076] For a window with length and width M, first subtract the reference position coordinates from the absolute position coordinates to obtain the relative position coordinates, then add M - 1 to the relative coordinates so that the relative position starts counting from 0. Multiply the row index by 2M - 1 and add the column coordinate to obtain the relative position index. The relative position in the window obtains the relative position bias from the relative position bias table according to the relative position index. The relative position bias is trainable and the same for each window.

[0077] Step three, features of different scales are fused between feature layers of different scales in a top-down and bottom-up manner through a non-convolutional feature fusion module.

[0078] As Figure 2 shown, the implementation method of the non-convolutional feature fusion module is as follows:

[0079] A feature fusion module is constructed without using any convolutional operations to replace the role of the Feature Pyramid Network / Path Aggregation Network in the non-convolutional object detection module. The specific process is as follows:

[0080] The upper feature layer is transformed through block fusion to double the dimension and halve the width and height. The current feature layer undergoes dimensional changes through a feed-forward neural network. The two are fused with the result of the block fusion of the upper feature layer through a concatenation operation to achieve top-down fusion; the feature layer after top-down fusion of this layer and the top-down feature layer of the previous layer undergo block fusion downsampling and then are added together to obtain the bottom-up fusion process. The entire process is composed of a combination of concatenation and linear transformation of the dimension and size of the feature layer, without using any convolutional layers, thus achieving the role of replacing the convolutional layer feature fusion module.

[0081] Step four, the fused feature layer then passes through the decoder, where a fixed number of anchor points / anchor boxes are assigned to each feature layer, and the center point coordinates and length and width values of the target bounding box are predicted for the anchor points / anchor boxes to obtain the prediction results.

[0082] As Figure 3As shown below, the specific method is as follows:

[0083] Each layer of the detection head consists of a group of stacked decoders. The input in the decoder is a fixed-position query, and each query only predicts one region. The regions represented by all the queries cover the entire feature map. Each query is responsible for making a one-to-one fixed prediction for the region it represents, and each region is represented by an anchor point.

[0084] The predicted results are divided into parallel branches: the class branch and the bounding box branch. In the class branch, the results first pass through a feed-forward network and then are divided into two branches: the class branch and the point score branch. Each branch outputs the prediction results through a fully connected layer. The class branch is the score for each class, and the number is the same as the number of classes in the dataset. The point score branch is the mapping of the distance between the predicted point of each anchor and the center point of the target. The bounding box branch is also divided into two branches, and each branch also passes through a fully connected layer to output the corresponding results. The two branches are the bounding box parameters of the target and the existence target score corresponding to each predicted bounding box.

[0085] A certain number of anchor points are assigned to each feature layer. The anchor point intervals of each feature layer are different, and the interval increases as the downsampling factor of the feature layer increases. The number of horizontal and vertical coordinates of the anchor points is generated according to the following formula:

[0086]

[0087] where, feature layer represents the serial number of the feature layer, widtg and height represent the width and height of the feature layer, and nx and ny represent the number of horizontal anchor points and the number of vertical anchor points;

[0088] Each feature layer determines the number of anchor points of the current feature layer according to its serial number, width, and height, and generates the corresponding grid according to the number of horizontal and vertical coordinates. Since the grid starts from (0, 0), the grids of all feature layers are offset 0.5 units to the right and down;

[0089] The anchor point method directly predicts the coordinate parameters of the target x, y, w, and h according to the anchor points. t x and t y are the direct prediction results of the network regarding x and y. b x and b y are the final prediction results regarding x and y. The specific formula is as follows:

[0090]

[0091] where, σ is the Sigmoid function, 2×[σ(t) - 0.5] is the offset of the relative coordinates of the target center point and the assigned anchor point, and the prediction method for the anchor point w and h is to directly predict the relative values of the width and height of the current feature layer;

[0092] The anchor box method is based on anchor points. Different ratios of anchor boxes are assigned to each anchor point. The anchor point serves as the center of the anchor box. The ratios of the anchor boxes are 1:1, 1:2, and 2:1, representing targets with different widths and heights. The anchor boxes are obtained by clustering the lengths and widths of the bounding boxes assigned by different feature layers. The anchor point prediction method directly predicts the relative offset of the coordinates of the anchor point to the center of the target point and the relative values of the width and height based on the feature layer. The prediction method of the anchor box has the same center point offset as the anchor point method, and the width and height are predicted based on multiples of the size of the assigned anchor box.

[0093] The sizes of the targets predicted by each feature layer are different. The size of the target bounding box is assigned to the corresponding feature layer according to the following formula:

[0094]

[0095] Step 5: Assign positive samples to the prediction results of the decoder, and resolve and adjust the conflicts of the same samples that are repeatedly assigned.

[0096] The specific method is as follows:

[0097] In the positive sample assignment strategy, positive samples are assigned to the anchor points near the center point of the target bounding box for matching. Specifically, the anchor point closest to the center point of the target bounding box is assigned as the center point of the positive sample. Then, three offset points are determined based on the center point of the positive sample and the center point of the target bounding box. For example, if the center point of the positive sample is in the upper right of the center point of the target bounding box, offset point 1 is the anchor point above the center point of the positive sample, offset point 2 is the anchor point on the right, and offset point 3 is the anchor point in the upper right. The positive sample of the anchor box is determined based on the intersection-over-union ratio of the anchor box and the target bounding box on the basis of the positive sample of the anchor point. If the intersection-over-union value exceeds 0.5, the anchor box is considered a positive sample.

[0098] The conflict resolution strategy needs to be based on the priority of the sample points. The priority of the center point of the positive sample is the highest, and the priorities of the offset points are the same. When there is a conflict at the center point of the positive sample, the nearest center point of the positive sample is retained according to the distance. When there is a conflict between the center point of the positive sample and the offset point, the offset point is directly removed from the positive sample and the center point is retained. When there is a conflict between the offset points, the nearest offset point is retained according to the distance.

[0099] To verify the effectiveness and advancement of the model proposed by the present invention, the following experiments are carried out: The experimental dataset uses VOC0712. The VOC0712 dataset is an integration of the VOC07 dataset and the VOC12 dataset. Among them, the VOC12 dataset contains 17,125 images, and VOC07 contains 9,963 images.

[0100] The experimental results of the proposed model and the baseline DETR model are shown in Table 1. The backbone network of DETR uses a convolutional neural network represented by ResNet50, and the detection head uses a combination of an encoder and a decoder and only a single feature layer.

[0101] Table 1 Experimental Results

[0102] Model Image Size Training Epochs Convergence Epochs mAP DETR-R50 600，1000 200 137 70.5 DETR-R50 800，1333 200 141 71.3 DETR-R101 600，1000 200 150 75.6 Ours-SwinT 600，1000 50 32 76.7 Ours-SwinT 800，1333 50 36 78.5 Ours-SwinS 600，1000 50 39 81.2

[0103] As can be seen from Table 1, the model of the present invention is superior to DETR in terms of both accuracy and convergence speed in different image sizes and backbone networks with higher numbers of layers.

[0104] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An anchor-based object detection method for the full Transformer framework, characterized in that It includes the following steps: Step 1: Use the hierarchical encoder of Swin Transformer as the backbone network of the full Transformer object detection model and train it; Step 2: Input the image to be detected into the trained model and use the hierarchical encoder of the model to extract image features; Step 3: Features of different scales are fused between feature layers of different scales in a top-down and bottom-up manner through a convolution-free feature fusion module; Step 4: The fused feature layers pass through the decoder, a fixed number of anchor points / anchor boxes are assigned to each feature layer, and the center point coordinates and length and width values of the target bounding box are predicted for the anchor points / anchor boxes to obtain the prediction results; Step 5: Positive samples are assigned to the prediction results of the decoder, and conflict resolution and adjustment are performed on the repeatedly assigned same samples; In Step 3, the implementation method of the convolution-free feature fusion module is as follows: The upper-layer feature layer is transformed through block fusion to double the dimension and halve the width and height, and the current feature layer undergoes dimensional change through a feed-forward neural network. The two are fused with the block fusion result of the upper-layer feature layer through a concatenation operation to form top-down fusion; the feature layer after top-down fusion of this layer and the top-down feature layer of the previous layer undergo block fusion downsampling and then are added together to obtain the bottom-up fusion process. The entire process is composed of a combination of concatenation and linear transformation of the dimension and size of the feature layer, without using any convolutional layers.

2. The method for object detection based on an anchor-based full Transformer framework according to claim 1, wherein In Step 1, during the training process of the model, the categorical loss uses cross-entropy loss, and the cross-entropy loss is as follows: ； Where N represents the total number of samples in the current batch, M represents the total number of categories, i represents the i-th sample, and c represents the current category. represents the sign function 0 or 1, taking 1 if the true category of sample i is equal to c, otherwise taking 0. represents the probability that sample i belongs to the c-th category. represents that sample i belongs to the c-th category. Use Distance to represent the distance from the center points of all anchor boxes to all target bounding boxes. Represent the relative x and y coordinates of the center point of the target bounding box. Represent the relative x and y coordinates of the anchor point. Point score The formula is as follows, and the point score loss is the mean squared error loss: ； Among them, ; The complete intersection over union loss is used for the target bounding box loss, and the complete intersection over union loss is as follows: ； Among them, represents the similarity of the aspect ratio, represents the true value of the target, represents the predicted target width, represents the predicted target height, represents the true value width of the target, represents the true value height of the target, represents the parameter weight of, to adjust the weight of, represents the intersection over union; The confidence loss uses binary cross-entropy loss, and the binary cross-entropy loss is as follows: ； where, represents the class label of the i-th sample, with the positive sample being 1 and the negative sample being 0, represents the probability that the i-th sample is a positive sample.

3. The method for object detection based on an anchor-based full Transformer framework according to claim 1, wherein, In Step 2, the implementation method of the hierarchical encoder is as follows: The feature layer is sliced using small windows. First, local attention calculation is performed inside each small window, and then the window is shifted to the right and downwards so that the attention of the original adjacent four windows can be propagated when calculating attention in the new window; the window attention calculation formula is as follows: ； Among them, Q represents the query, K represents the key, V represents the value, and B is the relative position bias. represents the dimension of K; For a window with length and width M, first subtract the reference position coordinates from the absolute position coordinates to obtain the relative position coordinates, then add M - 1 to the relative coordinates so that the relative position starts from 0. The row index is multiplied by 2M - 1 and then added to the column coordinate to obtain the relative position index. The relative position in the window obtains the relative position bias from the relative position bias table according to the relative position index.

4. The object detection method based on an anchor-based full Transformer framework according to claim 1, wherein, The specific method of Step 4 is as follows: Each layer of the detection head consists of a group of stacked decoders. The input in the decoder is a fixed-position query. Each query only predicts one region. The regions represented by all the queries connect the entire feature map. Each query is responsible for making a one-to-one fixed prediction for the region it represents. Each region is represented by an anchor point; the predicted results are divided into parallel branches: a class branch and a bounding box branch; In the class branch, the results first pass through a feed-forward network and then are divided into two branches: the class branch and the point score branch. Each branch outputs the prediction results through a fully connected layer. The class branch is the score for each class, and the number is the same as the number of classes in the dataset. The point score branch is the mapping of each anchor predicting the distance between the point and the target center point; the bounding box branch is also divided into two branches, and each also passes through a fully connected layer to output the corresponding results. The two branches are the bounding box parameters of the target and the corresponding presence target score for each predicted bounding box; A certain number of anchor points are assigned to each feature layer. The anchor point intervals of each feature layer are different, and the intervals increase as the downsampling factor of the feature layer increases. The number of horizontal and vertical coordinates of the anchor points is generated according to the following formula: ； Among them, represents the serial number of the feature layer, and represent the width and height of the feature layer, and represent the number of horizontal anchor points and the number of vertical anchor points; Each feature layer determines the number of anchor points of the current feature layer according to its serial number and length and width, and generates the corresponding grid according to the number of horizontal and vertical coordinates. Since the grid starts from (0, 0), the grids of all feature layers are offset 0.5 units to the right and down; The anchor method directly predicts the coordinate parameters of the target x, y, w, h based on the anchor points. and is the direct prediction result of the network regarding x and y. , is the final prediction result regarding x and y. The specific formula is as follows: ； Among them, is the Sigmoid function, is the offset of the relative coordinates of the target center point and the assigned anchor point. The prediction method for the anchor point w and h is to directly predict the relative values of the width and height of the current feature layer; The anchor box method is based on anchor points. Different ratios of anchor boxes are assigned to each anchor point. The anchor point is the center of the anchor box. The ratios of the anchor boxes are 1:1, 1:2, and 2:1, representing targets with different widths and heights. The anchor boxes are obtained by clustering the lengths and widths of the bounding boxes assigned by different feature layers; the anchor point prediction method directly predicts the relative offset of the coordinates of the anchor point to the target point center and the relative values of the width and height based on the feature layer; the prediction method of the anchor box has the same center point offset as the anchor point method, and the width and height are predicted according to the multiples of the size of the assigned anchor box; The sizes of the targets predicted by each feature layer are different. The sizes of the target bounding boxes are assigned to the corresponding feature layers according to the following formula: 。 5. The object detection method of an anchor-based full Transformer framework according to claim 1, characterized in that, The specific method of Step Five is as follows: In the positive sample assignment strategy, positive samples are assigned to the anchor points near the center point of the target bounding box for matching. Specifically, the anchor point closest to the center point of the target bounding box is assigned as the center point of the positive sample, and then three offset points are determined according to the center point of the positive sample and the center point of the target bounding box. The positive samples of the anchor box are determined according to the intersection-over-union ratio of the anchor box and the target bounding box based on the positive sample of the anchor point. If the intersection-over-union value exceeds 0.5, the anchor box is considered a positive sample; The conflict resolution strategy needs to be carried out according to the priority of the sample points. The priority of the center point of the positive sample is the highest, and the priorities of the offset points are the same. When there is a conflict at the center point of the positive sample, the nearest center point of the positive sample is retained according to the distance. When there is a conflict between the center point of the positive sample and the offset point, the offset point is directly removed from the positive sample and the center point is retained. When there is a conflict between the offset points, the nearest offset point is retained according to the distance.

Images