Automatic driving target detection method based on improved RT-DETR

Abstract

The invention discloses an automatic driving target detection method based on improved RT-DETR, and relates to the technical field of target detection and automatic driving, and the method comprises the steps: obtaining a disclosed automatic driving target detection data set SODA10M; a backbone network of an FSSD-DETR algorithm is constructed, and a BaseBlock optimization feature extraction process in a P5 layer is replaced with an FDC module; a small target detection layer is introduced into the neck network of the RT-DETR algorithm to improve the detection performance of a remote small target; a neck network of an FSSD-DETR algorithm is constructed, and a Zoomcat module and a ScalSeq module are used to optimize a feature fusion process; an up-sampling operator of the FSSD-DETR algorithm is constructed, and a DySample up-sampling operator is used to replace a nearest neighbor interpolation method so as to improve the image quality; therefore, good performance is maintained when complex traffic scenes such as small targets, dense vehicles and shielding are processed.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and autonomous driving, and in particular to an autonomous driving target detection method based on improved RT-DETR. Background Art

[0002] With the acceleration of urbanization, the number of traffic vehicles has increased sharply, resulting in increasingly complex road environments. This change has put forward more stringent requirements on the target detection algorithm in autonomous driving technology. Target detection technology must not only cope with conventional traffic scenarios, but also maintain efficiency and accuracy under challenges such as extreme weather, complex lighting, and peak hours. In the perception system of autonomous vehicles, target detection algorithms need to quickly and accurately parse continuous image data streams to achieve real-time decision-making and response. This requires not only that the algorithm has excellent processing capabilities, but also that it must meet real-time standards.

[0003] At present, object detection technology is mainly divided into two categories: detectors based on convolutional neural networks (CNNs) and detectors based on Transformers. CNN-based detectors are further divided into the R-CNN series and the YOLO series. Although CNN-based detectors have been widely studied and applied in the past few years, the sacrifice of real-time performance of the R-CNN series algorithms and the limitations of the YOLO series in small and dense target detection have limited their application scope in complex scenarios.

[0004] RT-DETR (Real-Time Detection Transformer) is a real-time end-to-end object detection model based on Transformer. Figure 1 As shown in the figure, the model consists of a backbone network, an encoder and a decoder. RT-DETR uses the convolutional neural network ResNet18 as the backbone network for feature extraction to extract different scale features in the image layer by layer. These features represent the visual information in the image, such as edges, textures, and colors. RT-DETR uses an efficient hybrid encoder to encode feature maps into global context representations, thereby modeling long-distance dependencies between features. At the same time, the encoder separates intra-scale feature interactions from inter-scale feature fusion, significantly improving the operating efficiency of the model. The RT-DETR decoder gradually predicts the bounding box and category of the target by receiving the features generated by the encoder and the initial query. It optimizes the query quality through the IoU perception mechanism to ensure more accurate positioning and classification of detected objects. RT-DETR not only performs well in processing capabilities, but also maintains real-time performance.

[0005] However, the original RT-DETR algorithm has a high missed detection rate when dealing with distant or small-sized targets, and it is difficult to effectively use the underlying features for precise positioning and identification. At the same time, in the process of multi-scale feature fusion, simple splicing or summation operations in RT-DETR may ignore the detailed information in large-size features, resulting in poor detection of overlapping or dense targets in complex scenes. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides an automatic driving target detection method based on improved RT-DETR, comprising the following steps:

[0007] S1. Obtain a public autonomous driving target detection dataset;

[0008] S2. Build the FSSD-DETR algorithm based on the original RT-DETR algorithm, build the backbone network of the FSSD-DETR algorithm, and use the FDC module to replace the BasicBlock in the P5 layer of the original RT-DETR algorithm. The FDC module includes a frequency selection module, an adaptive convolution kernel module, and an adaptive expansion rate module. The frequency selection module extracts high-frequency components and low-frequency components by performing frequency decomposition on the input feature map, and applies adaptive weights according to different frequency characteristics. The adaptive convolution kernel module decomposes the convolution kernel parameters to process low-frequency components and high-frequency components respectively. The adaptive expansion rate module generates a convolution expansion rate by performing frequency selection and offset calculation on the input feature map.

[0009] S3, construct the neck network of the FSSD-DETR algorithm, and introduce the small target detection layer P2 into the neck network of the original RT-DETR algorithm;

[0010] S4, further construct the neck network of the FSSD-DETR algorithm, adding the Zoom_cat module and the Scal Seq module to optimize the feature fusion process;

[0011] S5. Construct an upsampling operator for the FSSD-DETR algorithm and use the DySample upsampling operator to replace the nearest neighbor interpolation method of the original RT-DETR algorithm.

[0012] S6. Set the FSSD-DETR algorithm training environment and hyperparameters, train and optimize the FSSD-DETR algorithm, and compare and analyze the experimental results.

[0013] The technical solution further defined in the present invention is:

[0014] Furthermore, in step S1, the public autonomous driving target detection dataset is set to the SODA10M dataset.

[0015] In the aforementioned automatic driving target detection method based on improved RT-DETR, in step S1, 5000 labeled training sets and 5000 validation sets are obtained from the SODA10M dataset for training and verification respectively.

[0016] In the above-mentioned automatic driving target detection method based on improved RT-DETR, in step S2, the frequency selection module extracts high-frequency and low-frequency components by frequency decomposing the input feature map, and applies adaptive weights according to different frequency characteristics; different frequency components are processed separately and recombined to enhance the feature map, and the formula is as follows:

[0017]

[0018] Among them, X represents the input feature map, f represents the frequency, and M f represents the frequency weight generated by the convolutional layer, F(X) represents the components of different frequencies, and X′ is the representation of the feature map in the frequency domain.

[0019] In the aforementioned automatic driving target detection method based on improved RT-DETR, in the adaptive convolution kernel module of step S2, the low-frequency and high-frequency convolution kernel weights are first calculated; then the weight ratio of these two parts is dynamically adjusted according to the input features, and the formula is as follows:

[0020] W adaptive =W low α+W high (1-α)

[0021] W final =mean(W adaptive )·a low +(W adaptive -mean(W adaptive ))·a high

[0022] Among them, W low Represents the convolution kernel weight of the low-frequency part, W high represents the convolution kernel weight of the high-frequency part, α represents the dynamically adjusted weight ratio, and W adaptive represents the adaptive convolution kernel weight, mean represents the mean, a low Represents the dynamic adjustment coefficient of low-frequency weight, a high Represents the dynamic adjustment coefficient of high-frequency weight, W final Represents the final convolution kernel weight.

[0023] In the above-mentioned automatic driving target detection method based on improved RT-DETR, in step S2, the adaptive dilation rate module is used to process input images with different spatial scales and dynamically adjust the dilation rate according to the input content; the offset Δp and the mask M are used to dynamically change the receptive field of the convolution, and the formula is as follows:

[0024]

[0025] Among them, p represents a certain position of the input feature map, Ω represents the sampling point set of the convolution kernel, W(p) represents the weight of the convolution kernel at position p, the convolution kernel weights the input feature by the weight W(p), X(p) represents the value of the input feature map X at p, M(p) represents the mask value at position p, and Y represents the output feature map.

[0026] In the aforementioned automatic driving target detection method based on improved RT-DETR, in step S3, a two-dimensional convolution and batch normalization operation is performed on the image of size 160×160 in the backbone network to form P2, and then sent to the neck network for feature fusion.

[0027] In the above-mentioned automatic driving target detection method based on improved RT-DETR, in step S4, the Zoom_cat module splits the features of three different sizes, large, medium and small, in descending order, downsamples the large-size feature map l to the same spatial size as the medium-size feature map m through adaptive maximum pooling and adaptive average pooling, and then adds the two pooling results; upsamples the small-size feature map s to make its spatial size consistent with m; splices the adjusted large-size feature map l′, the unadjusted medium-size feature map m, and the adjusted small-size feature map s′ in the feature dimension to form a comprehensive feature map lms that integrates multi-scale information, and its formula is as follows:

[0028] l′=AdaptiveMaxPool2d(l)+AdaptiveAvgPool2d(l)

[0029] s'=Interpolate(s, mode='DySample')

[0030] lms=Concat([l′, m, s′], dim=1)

[0031] Among them, l, m, s represent feature maps of large, medium and small sizes respectively, AdaptiveMaxPool2d and AdaptiveAvgPool2d represent adaptive maximum pooling and average pooling operations on the input feature map l, Interpolate represents the interpolation operation, Concat represents the connection operation, l′ represents the feature map combined with adaptive maximum pooling and average pooling, and s′ represents the feature map interpolated by the DySample upsampling method.

[0032] In the above-mentioned automatic driving target detection method based on improved RT-DETR, in step S4, the ScalSeq module receives three feature maps of different sizes, and merges them in the new time dimension after adjusting the number of channels and sizes; the merged feature map is then processed by 3D convolution, batch normalization and LeakyReLU activation function, and finally dimensionality reduction is performed in the time dimension by 3D maximum pooling, and finally the merged and processed feature map is output, and its formula is as follows:

[0033] x′ 1 =Unsqueeze(x 1 , dim=-3)

[0034] x′ 2 =Unsqueeze(UpSample(x 2 ,size=(H 1 , W 1 )), dim = -3)

[0035] x′ 3 =Unsqueeze(UpSample(x 3 ,size=(H 1 , W 1 )), dim = -3)

[0036] x conv3d =Conv 3D(Concat([x′ 1 , x′ 2 , x′ 3 ],dim=-3))

[0037] x final =Squeeze(MaxPool 3D(x conv3d ), dim=-3)

[0038] Among them, x′ 1 , x′ 2 , x′ 3They represent three feature maps of different sizes of the input respectively. Dim=-3 means concatenating the feature maps on the newly added time dimension. The newly added time dimension is the third dimension. Unsqueeze means adding a new dimension on the specified dimension. The specified dimension is the first 3 dimensions, i.e., the third dimension. UpSample means upsampling operation, Concat means concatenation operation, Conv 3D means 3D convolution operation, MaxPool 3D means 3D pooling operation, Squeeze means compression on the -3rd dimension, x final Represents the feature map of the final output.

[0039] In the above-mentioned automatic driving target detection method based on improved RT-DETR, in step S5, the DySample upsampling operator uses a point sampling method to realize dynamic upsampling, and transforms the input feature map of a given size of C×H×W into a sampling set S of a size of 2×sH×sW through a sampling point generator, where the first dimension 2 in S represents the x and y coordinates; then the input feature is resampled using the grid_sample function to obtain an upsampled feature map of a size of C×sH×sW; the sampling point generator generates an offset O by combining a linear layer, a dynamic range factor, and a pixel shuffle sub-pixel convolution, and then adds it to the original grid position G to obtain the final sampling set S:

[0040] X'=grid_sample(X,G+0.5sigmoid(Linear 1 (X))×Linear 2 (X)

[0041] Among them, X represents the input feature map; G represents the original grid position, which is used to represent the standard coordinates on the feature map; sigmoid represents the activation function; Linear represents a linear transformation of the input feature map X to generate a new feature representation; 0.5 is used to scale the offset coefficient; grid_sample represents the sampling operation; X′ represents the output feature map generated after sampling by the grid_sample function.

[0042] The beneficial effects of the present invention are:

[0043] In the present invention, an FDC module is added to the backbone network of the model to replace the BasicBlock in the P5 layer, thereby improving the ability of the model to extract features; the detection performance of the model for small targets is improved by adding a small target detection layer; the advantages of the Zoom_cat module and the ScalSeq module are used to re-improve the feature fusion process, so that the model can capture key information in the image more comprehensively; the shortcomings of the nearest neighbor interpolation method are improved by adopting the DySample upsampling operator; the improved model FSSD-DETR has an mAP50 increase of 3.6%, an accuracy increase of 3.4%, and a recall rate increase of 2.1% compared with the original RT-DETR algorithm, thereby achieving a significant improvement in accuracy by adding a small amount of parameters and meeting real-time requirements; in practical applications, this balance between accuracy and speed has high practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural diagram of the RT-DETR algorithm in the background technology of the present invention;

[0045] Figure 2 It is a schematic diagram of the overall process of the present invention;

[0046] Figure 3 FSSD-DETR algorithm structure diagram in an embodiment of the present invention;

[0047] Figure 4 FIG. 4 is a structural diagram of an FDC module in an embodiment of the present invention;

[0048] Figure 5 It is a structural diagram of the Zoom_cat module in an embodiment of the present invention;

[0049] Figure 6 It is a structural diagram of the ScalSeq module in an embodiment of the present invention;

[0050] Figure 7 4 is a structural diagram of a Dysample upsampling operator in an embodiment of the present invention;

[0051] Figure 8 This is a comparison chart of the effects of the FSSD-DETR algorithm and the original RT-DETR algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] This embodiment provides an automatic driving target detection method based on improved RT-DETR, such as Figure 2 As shown, the following steps are included:

[0053] S1. Download the public autonomous driving target detection dataset SODA10M, obtain 5000 labeled training sets and 5000 validation sets from the SODA10M dataset for training and verification. Since the labels of the test set are not public, the performance of the FSSD-DETR algorithm is verified on the validation set. At the same time, resize all images in the dataset to a fixed resolution of 640×640 to ensure that the image size of the input model is consistent, facilitate batch processing of the model, improve the training efficiency and inference speed of the model, and reduce the problem of inconsistent feature extraction caused by different image sizes.

[0054] S2, based on the original RT-DETR algorithm Figure 3 The FSSD-DETR (FSSD Detectiin Transformer) algorithm shown in the figure builds the backbone network of the FSSD-DETR algorithm, and uses the FDC (Frequency-Adaptive Dilated Convilution) module to replace the BasicBlock in the P5 layer of the original RT-DETR algorithm. The BasicBlock is mainly composed of two relatively simple convolutional layers. When dealing with feature extraction tasks in complex driving scenes, there is a problem of losing important features. The FDC module can help the model more accurately capture the edges, textures, and multi-scale features of objects in the image, improve the ability to recognize targets in complex backgrounds, thereby optimizing the feature extraction process and enhancing the detection performance of the model in different scales and complex scenes.

[0055] like Figure 4 As shown, the FDC module includes a frequency selection module, an adaptive convolution kernel module and an adaptive dilation rate module; the FDC module was originally used to process pixel-level image features in semantic segmentation tasks, and it can dynamically adjust the dilation rate according to the local frequency components of the image, so as to more flexibly capture features of different scales; in order to better extract the features of objects of various scales in the autonomous driving scene without adding a large number of parameters, and to help the model better capture detailed information such as the edges and textures of objects in the image, this embodiment only uses the FDC module to replace BasicB1ock in the P5 layer.

[0056] The frequency selection module extracts high-frequency and low-frequency components by frequency decomposing the input feature map, and applies adaptive weights according to different frequency characteristics; different frequency components are processed separately and recombined to enhance the feature map. The formula is as follows:

[0057]

[0058] Among them, X represents the input feature map, f represents the frequency, and M frepresents the frequency weight generated by the convolutional layer, F(X) represents the components of different frequencies, and X′ is the representation of the feature map in the frequency domain.

[0059] The adaptive convolution kernel module decomposes the convolution kernel parameters to process low-frequency components and high-frequency components respectively. In the adaptive convolution kernel module, the weights of low-frequency and high-frequency convolution kernels are first calculated. Then, the weight ratio of these two parts is dynamically adjusted according to the input features. The formula is as follows:

[0060] W adaptive =W low α+W high (1-α)

[0061] W final =mean(W adaptive )·a low +(W adaptive -mean(W adaptive ))·a high

[0062] Among them, W low Represents the convolution kernel weight of the low-frequency part, W high represents the convolution kernel weight of the high-frequency part, α represents the dynamically adjusted weight ratio, and W adaptive represents the adaptive convolution kernel weight, mean represents the mean, a l represents the dynamic adjustment coefficient of low-frequency weight, ahigh represents the dynamic adjustment coefficient of high-frequency weight, W final Represents the final convolution kernel weight.

[0063] Through the above adaptive method, the convolution kernel can not only flexibly capture the different frequency information in the input features and enhance the network's perception of details and overall structure, but also dynamically balance between high-frequency details and low-frequency global context according to scene requirements, thereby enhancing the network's perception of details and overall structure and improving the precise positioning and recognition performance of targets in complex scenes.

[0064] The adaptive dilation rate module generates an adaptable convolution dilation rate by performing frequency selection and offset calculation on the input feature map. It can adaptively adjust the receptive field according to the frequency information of the input features, ensuring that the key information of the image can be effectively extracted at different frequencies and scales.

[0065] The adaptive dilation rate module can process input images with different spatial scales and dynamically adjust the dilation rate according to the input content; the offset Δp and mask M are used to dynamically change the receptive field of the convolution, and the formula is as follows:

[0066]

[0067] Among them, p represents a certain position of the input feature map, Ω represents the sampling point set of the convolution kernel, W(p) represents the weight of the convolution kernel at position p, the convolution kernel weights the input feature by the weight W(p), X(p) represents the value of the input feature map X at p, M(p) represents the mask value at position p, and Y represents the output feature map.

[0068] The offset Δp and mask M are the core of adaptive calculation. The offset Δp dynamically adjusts the position of the convolution operation so that the convolution kernel can flexibly adapt to the local features of the input image, thereby enhancing the ability to capture the target. The mask M is responsible for selectively enhancing or suppressing the convolution response of certain areas according to the frequency and spatial information of the input, ensuring that the model can effectively extract useful features at different scales and frequencies. Combining the above two, fine control of the receptive field is achieved to adapt to feature changes in different complex scenes.

[0069] S3. Construct the neck network of the FSSD-DETR algorithm, introduce the small target detection layer P2 into the neck network of the original RT-DETR algorithm, perform a two-dimensional convolution and batch normalization operation on the image of size 160×160 in the backbone network to form P2, and send it to the neck network for feature fusion, so that the model can better utilize high resolution and underlying features to improve the detection performance of small targets in the distance.

[0070] The original RT-DETR algorithm uses a feature map of 80×80 size to detect small-sized targets. For an image with an input size of 640×640, the receptive field size of a grid in the feature map is 8×8. It is difficult for the network to learn the feature information of targets less than 8 pixels high or wide in the original image, resulting in excessive missed detection rates. Missing a pedestrian or vehicle may lead to serious consequences. Therefore, this embodiment can effectively utilize images of 160×160 size by introducing a small target detection layer to utilize richer underlying feature information, thereby improving the detection capability of small targets.

[0071] S4. Further construct the neck network of the FSSD-DETR algorithm, add the Zoom_cat module and the ScalSeq module to optimize the feature fusion process, improve the feature fusion capability of multi-scale features, effectively combine the feature information of large, medium and small scale targets, and enhance the detection accuracy and robustness of the model for targets of different sizes in complex scenarios.

[0072] like Figure 5As shown in the figure, the Zoom_cat module splits the features of three different sizes, large, medium and small (for example, input features of 640*640, 320*320, 160*160, 80*80, 40*40 or smaller sizes, where the three adjacent sizes from large to small correspond to large, medium and small respectively), and downsamples the large-size feature map l to the same spatial size as the medium-size feature map m through adaptive maximum pooling and adaptive average pooling, and then adds the two pooling results; upsamples the small-size feature map s to make its spatial size consistent with m; splices the adjusted large-size feature map l′, the unadjusted medium-size feature map m and the adjusted small-size feature map s′ in the feature dimension to form a comprehensive feature map lms that integrates multi-scale information, and its formula is as follows:

[0073] l′=AdaptiveMaxPool2d(l)+AdaptiveAvgPool2d(l)

[0074] s'=Interpolate(s, mode='DySample')

[0075] lms=Concat([l′, m, s′], dim=1)

[0076] Among them, l, m, s represent feature maps of large, medium and small sizes respectively, AdaptiveMaxPool2d and AdaptiveAvgPool2d represent adaptive maximum pooling and average pooling operations on the input feature map l, Interpolate represents the interpolation operation, Concat represents the connection operation, l′ represents the feature map combined with adaptive maximum pooling and average pooling, and s′ represents the feature map interpolated by the DySample upsampling method.

[0077] like Figure 6 As shown in the figure, the ScalSeq module receives three feature maps of different sizes, and merges them in the new time dimension after adjusting the number of channels and sizes; the merged feature map is then processed by 3D convolution, batch normalization and LeakyReLU activation function, and finally dimensionality reduction is performed in the time dimension through 3D maximum pooling, and finally the merged and processed feature map is output. The formula is as follows:

[0078] x′ 1 =Unsqueeze(x 1 , dim=-3)

[0079] x′ 2 =Unsqueeze(UpSample(x 2 ,size=(H 1, W 1 )), dim = -3)

[0080] x′ 3 =Unsqueeze(UpSample(x 3 ,size=(H 1 , W 1 )), dim = -3)

[0081] x conv3d =Conv3D(Concat([x′ 1 , x′ 2 , x′ 3 ],dim=-3))

[0082] x final =Squeeze(MaxPool3D(x conv3d ), dim=-3)

[0083] Among them, x′ 1 , x′ 2 , x′ 3 They represent three feature maps of different sizes of the input respectively. Dim=-3 means concatenating the feature maps on the newly added time dimension (the third dimension). Unsqueeze means adding a new dimension on the specified dimension (the first 3 dimensions, i.e., the third dimension). The specified dimension is the first 3 dimensions, i.e., the third dimension. UpSample means upsampling operation, Concat means concatenation operation, Conv3D means 3D convolution operation, MaxPool 3D means 3D pooling operation, Squeeze means compression on the first 3 dimensions, x final Represents the feature map of the final output.

[0084] S5. Construct an upsampling operator for the FSSD-DETR algorithm and use the DySample upsampling operator to replace the nearest neighbor interpolation method of the original RT-DETR algorithm. This improves the problems of detail loss, jagged edges, and image distortion that may be caused by the nearest neighbor interpolation method during the upsampling process, and improves the upsampling accuracy and image quality, thereby better preserving the target's detailed information, especially maintaining the integrity and accuracy of the image when detecting small or distant targets.

[0085] The DySample upsampling operator uses point sampling to achieve dynamic upsampling. It transforms the input feature map of a given size of C×H×W into a sampling set S of size 2×sH×sW through the sampling point generator, where the first dimension 2 in S represents the x and y coordinates. The input feature is then resampled using the grid_sample function to obtain an upsampled feature map of size C×sH×sW. The sampling point generator generates an offset O by combining a linear layer, a dynamic range factor, and a pixel shuffle sub-pixel convolution, which is then added to the original grid position G to obtain the final sampling set S:

[0086] X'=grid_sample(X,G+0.5sigmoid(Linear 1 (X))×Linear 2 (X)

[0087] Among them, X represents the input feature map; G represents the original grid position, which is used to represent the standard coordinates on the feature map; sigmoid represents the activation function; Linear represents a linear transformation of the input feature map X to generate a new feature representation; 0.5 is used to scale the offset coefficient to ensure that the offset is not too large; grid_sample represents the sampling operation; X′ represents the output feature map generated after sampling by the grid_sample function.

[0088] The original RT-DETR algorithm uses the nearest neighbor interpolation method to upsample the image. This method only focuses on the spatial position relationship of pixels, but does not fully consider the semantic information of the image content. For objects such as distant vehicles and pedestrians, problems such as loss of details, jagged edges, and image distortion may occur. Figure 7 As shown in the figure, DySample uses point sampling method to achieve dynamic upsampling, which can improve the quality of image upsampling. At the same time, DySample avoids the complex dynamic convolution and sub-network construction process commonly seen in traditional kernel-based dynamic upsampling methods, significantly reducing the number of parameters.

[0089] S6. Set the FSSD-DETR algorithm training environment and hyperparameters, train and optimize the FSSD-DETR algorithm, compare and analyze the experimental results, and use the improved FSSD-DETR model to train on the training set processed in step S1.

[0090] In this embodiment, all experiments are carried out under the Ubuntu 20.04 operating system, the GPU is NVIDIA GeForce GTX4090, the video memory is 24 GB, the CPU is Intel (R) Xeon (R) Gold 5418Y, the deep learning architecture is Pytorch 1.11.0, the programming language is Python 3.8, and the accelerated computing architecture is CUDA 11.3; during the training process, the initial learning rate is set to 0.0001, the AdamW optimizer with a weight decay of 0.0001 is used, the batch size is set to 8, and the training rounds are set to 150; all training does not use pre-trained weights; other strategies and hyperparameters follow the original RT-DETR algorithm; after the training is completed, the optimal weight is used to test the validation set to evaluate the performance of the model.

[0091] After obtaining the experimental results, a comparison and analysis were performed. The experimental part used precision, recall, average precision mAP50, parameter quantity and transmission frame rate per second FPS as evaluation indicators; in order to verify the superiority of the improved strategy proposed in this embodiment over the original RT-DETR algorithm and the current mainstream target detection algorithm, a comparative experiment was carried out. The experimental results are shown in Table 1 below.

[0092] Table 1 Comparison results of different models on the SODA10M dataset

[0093]

[0094] As can be seen from Table 1, the mAP50 of the FSSD-DETR algorithm proposed in this embodiment reaches 59.5%, surpassing all YOLO series models involved in the comparison; compared with the highest YOLOv9c in the YOLO series, the FSSD-DETR algorithm is 6.7% higher with fewer parameters; in addition, the algorithm in this embodiment achieves the highest precision of 73.4% and the highest recall of 54%, reducing the possibility of missed detection.

[0095] The effect comparison between the algorithm in this embodiment and the original RT-DETR algorithm is shown in the figure below. Figure 8 As shown, the left side is the effect before improvement, and the right side is the effect after improvement; it can be clearly seen from the figure that the FSSD-DETR algorithm can maintain stable performance in changeable weather conditions and diverse road scenes; in contrast, RT-DETR often misses detections when dealing with small target detection; this comparison fully demonstrates the significant effectiveness of the FSSD-DETR algorithm in reducing the missed detection rate, and its efficient detection capability can improve the safety of the autonomous driving system. The algorithm in this embodiment achieves a significant improvement in accuracy by adding a small amount of parameters, and meets real-time requirements; in practical applications, this balance between accuracy and speed has high practical value.

[0096] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.

Claims

1. An automatic driving target detection method based on improved RT-DETR, characterized by: The following steps are involved: S1. Obtain a public autonomous driving target detection dataset; S2. Build the FSSD-DETR algorithm based on the original RT-DETR algorithm, build the backbone network of the FSSD-DETR algorithm, and use the FDC module to replace the Basic Block in the P5 layer of the original RT-DETR algorithm. The FDC module includes a frequency selection module, an adaptive convolution kernel module, and an adaptive expansion rate module. The frequency selection module extracts high-frequency components and low-frequency components by performing frequency decomposition on the input feature map, and applies adaptive weights according to different frequency characteristics. The adaptive convolution kernel module processes low-frequency components and high-frequency components separately by decomposing the convolution kernel parameters; The adaptive dilation rate module generates the convolution dilation rate by performing frequency selection and offset calculation on the input feature map; S3, construct the neck network of the FSSD-DETR algorithm, and introduce the small target detection layer P2 into the neck network of the original RT-DETR algorithm; S4, further construct the neck network of the FSSD-DETR algorithm, adding the Zoom_cat module and the ScalSeq module to optimize the feature fusion process; S5. Construct an upsampling operator for the FSSD-DETR algorithm and use the DySample upsampling operator to replace the nearest neighbor interpolation method of the original RT-DETR algorithm. S6. Set the FSSD-DETR algorithm training environment and hyperparameters, train and optimize the FSSD-DETR algorithm, and compare and analyze the experimental results.

2. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S1, the public autonomous driving target detection dataset is set to the SODA10M dataset.

3. The automatic driving target detection method based on improved RT-DETR according to claim 2, characterized in that: In the step S1, 5000 labeled training sets and 5000 labeled validation sets are obtained from the SODA10M dataset for training and validation.

4. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S2, the frequency selection module extracts high-frequency and low-frequency components by frequency decomposing the input feature map, and applies adaptive weights according to different frequency characteristics; different frequency components are processed separately and recombined to enhance the feature map, and the formula is as follows: Among them, X represents the input feature map, f represents the frequency, and M f represents the frequency weight generated by the convolutional layer, F(X) represents the components of different frequencies, and X′ is the representation of the feature map in the frequency domain.

5. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In the adaptive convolution kernel module of step S2, the low-frequency and high-frequency convolution kernel weights are first calculated; then the weight ratio of the two parts is dynamically adjusted according to the input features, and the formula is as follows: W adaptive =W low ·a+W high ·(1-a) W final =mean(W adaptive )·a low +(W adaptive -mean(W adaptive ))·a high Among them, W low Represents the convolution kernel weight of the low-frequency part, W high represents the convolution kernel weight of the high-frequency part, α represents the dynamically adjusted weight ratio, and W adaptive represents the adaptive convolution kernel weight, mean represents the mean, a low Represents the dynamic adjustment coefficient of low-frequency weight, a high Represents the dynamic adjustment coefficient of high-frequency weight, W final Represents the final convolution kernel weight.

6. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S2, the adaptive expansion rate module is used to process input images with different spatial scales and dynamically adjust the expansion rate according to the input content; the offset Δp and the mask M are used to dynamically change the receptive field of the convolution, and the formula is as follows: Among them, p represents a certain position of the input feature map, Ω represents the sampling point set of the convolution kernel, W(p) represents the weight of the convolution kernel at position p, the convolution kernel weights the input feature by the weight W(p), X(p) represents the value of the input feature map X at p, M(p) represents the mask value at position p, and Y represents the output feature map.

7. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S3, a two-dimensional convolution and batch normalization operation is performed on the image of size 160×160 in the backbone network to form P2, and the image is sent to the neck network for feature fusion.

8. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S4, the Zoom_cat module splits the features of three different sizes, large, medium and small, with decreasing sizes in turn, downsamples the large-size feature map l to the same spatial size as the medium-size feature map m through adaptive maximum pooling and adaptive average pooling, and then adds the two pooling results; upsamples the small-size feature map s to make its spatial size consistent with m; splices the adjusted large-size feature map l′, the unadjusted medium-size feature map m, and the adjusted small-size feature map s′ in the feature dimension to form a comprehensive feature map lms that integrates multi-scale information, and its formula is as follows: l′=AdaptiveMaxPool2d(l)+AdaptiveAvgPool2d(l) s'=Interpolate(s, mode='DySample') lms=Concat([l′, m, s′], dim=1) Among them, l, m, s represent feature maps of large, medium and small sizes respectively, AdaptiveMaxPool2d and AdaptiveAvgPool2d represent adaptive maximum pooling and average pooling operations on the input feature map l, Interpolate represents the interpolation operation, Concat represents the connection operation, l′ represents the feature map combined with adaptive maximum pooling and average pooling, and s′ represents the feature map interpolated by the DySample upsampling method.

9. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S4, the ScalSeq module receives three feature maps of different sizes, and merges them in the new time dimension after adjusting the number of channels and sizes; the merged feature map is then processed by 3D convolution, batch normalization and LeakyReLU activation function, and finally dimensionality reduction is performed in the time dimension by 3D maximum pooling, and finally the merged and processed feature map is output, and its formula is as follows: x'1=Unsqueeze(x1,dim=-3) x′2=Unsqueeze(UpSample(x2, size=(H1, W1)), dim=-3) x′3=Unsqueeze(UpSample(x3, size=(H1, W1)), dim=-3) x conv3d =Conv 3D(Concat([x′1,x′2,x′3],dim=-3)) x final =Squeeze(MaxPool 3D(x conv3d ),dim=-3) Among them, x′1, x′2, and x′3 represent three feature maps of different sizes of input, respectively. Dim=-3 means concatenating feature maps on the newly added time dimension, and the newly added time dimension is the third dimension. Unsqueeze means adding a new dimension to the specified dimension, and the specified dimension is the first 3 dimensions, that is, the third dimension. UpSample means upsampling operation, Concat means concatenation operation, Conv 3D means 3D convolution operation, MaxPool 3D means 3D pooling operation, Squeeze means compression on the first 3 dimensions, and x final Represents the feature map of the final output.

10. The automatic driving target detection method based on improved RT-DETR according to claim 1, characterized in that: In step S5, the DySample upsampling operator uses a point sampling method to implement dynamic upsampling, and converts the input feature map of a given size of C×H×W into a sampling set S of a size of 2×sH×sW through a sampling point generator, where the first dimension 2 in S represents the x and y coordinates; then the input feature is resampled using the grid_sample function to obtain an upsampled feature map of a size of C×sH×sW; the sampling point generator generates an offset O by combining a linear layer, a dynamic range factor, and a pixelshuffle sub-pixel convolution, and then adds it to the original grid position G to obtain the final sampling set S: X′=grid_sample(X,G+0.5sigmoid(Linear1(X))×Linear2(X)) Among them, X represents the input feature map; G represents the original grid position, which is used to represent the standard coordinates on the feature map; sigmoid represents the activation function; Linear represents a linear transformation of the input feature map X to generate a new feature representation; 0.5 is used to scale the offset coefficient; grid_sample represents the sampling operation; X′ represents the output feature map generated after sampling by the grid_sample function.

Images