A vehicle target detection method based on MKAN-DETR in a complex scene

Abstract

A vehicle target detection method based on MKAN-DETR in complex scenes includes: acquiring a dataset containing clear images of vehicle targets in complex scenes, and dividing the dataset into a training set and a test set; constructing a vehicle target detection model (MKAN-DETR) for complex scenes; training the constructed vehicle target detection model (MKAN-DETR) using the training set to obtain a trained vehicle target detection model; and testing the trained vehicle target detection model on the test set to obtain vehicle target detection results. This invention provides a vehicle target detection method based on MKAN-DETR in complex scenes, improving the detection capability for occluded vehicles and dense traffic flow. It can accurately detect distant small vehicles and partially occluded vehicle targets in complex road environments, making it suitable for applications with high timeliness requirements such as roadside perception and real-time traffic monitoring.

Description

Technical Field

[0001] This invention relates to the field of vehicle target detection technology, and specifically to a vehicle target detection method based on MKAN-DETR in complex scenarios. Background Technology

[0002] In recent years, with the rapid increase in global car ownership, integrated transportation solutions centered on intelligent driving and vehicle-to-infrastructure (V2I) communication have received widespread attention. In this system, the vehicle target detection capability of roadside perception units is crucial for achieving full-domain trajectory tracking, collision warning, and traffic flow scheduling. However, in real-world urban roadside deployments, perception systems often face highly challenging non-ideal scenarios: severe vehicle overlap and occlusion caused by dense congestion, as well as image noise and boundary blurring caused by low light at night, strong exposure, and rain or snow. Ensuring the detection accuracy of distant small targets and heavily occluded targets in these specific scenarios, while meeting the real-time and post-processing requirements of edge devices, is a pressing practical challenge that needs to be addressed.

[0003] Early solutions were mainly divided into two-stage models (such as Faster R-CNN) and single-stage models (such as the YOLO series). To address the aforementioned challenges, researchers have made numerous improvements to CNN architectures, exemplified by YOLO. However, single-stage models often employ convolutional architectures, whose local connectivity is limited to capturing local topological information. In low-light or severely occluded scenes at night, the limited receptive field often leads to a break in the global contextual cues of the vehicle, resulting in missed detections. Furthermore, CNN architectures have relatively weak generalization capabilities, often exhibiting a sharp performance drop when faced with new environments with drastic changes in lighting.

[0004] To overcome the limitations of CNNs, introducing Transformers into object detection has become a new trend to improve generalization in complex scenes. Although Transformer-based algorithms perform well in feature aggregation and noise resistance, the complex attention operations lead to extremely high computational redundancy, making it difficult to meet the real-time deployment requirements of roadside edge computing devices. Baidu's RT-DETR model combines the advantages of CNNs and Transformers, and eliminates the NMS post-processing step, enabling direct object localization and classification within the model itself. Although improved studies based on RT-DETR have further demonstrated its potential, its limitations are still exposed when dealing with specific scenarios involving dense overlap and complex lighting: First, existing models lack an efficient global-local feature dynamic screening mechanism when handling noise at night or in rainy weather, leading to environmental noise being mistaken for target features; Second, when facing multiple overlapping vehicles on congested roads, conventional neck networks struggle to retain high-resolution sharp boundaries, easily resulting in missed detections of "multiple vehicles predicted as one vehicle"; Third, existing decoders mostly rely on traditional multilayer perceptrons (MLPs) for bounding box regression, which has insufficient linear mapping capability when dealing with complex targets whose appearance is deformed due to occlusion or whose only part is exposed, resulting in a significant decrease in localization accuracy. Summary of the Invention

[0005] To address the technical challenges of existing roadside technologies, such as strong background noise interference, boundary feature aliasing due to congestion and occlusion, and low regression accuracy of partially obscured targets in real-world roadside scenarios, this invention provides a vehicle target detection method based on MKAN-DETR for complex scenarios. This method improves the detection capability for occluded vehicles and dense traffic flow, accurately detecting distant small vehicles and partially occluded vehicle targets in complex road environments. It is suitable for applications with high timeliness requirements, such as roadside perception and real-time traffic monitoring.

[0006] The technical solution adopted in this invention is as follows: A vehicle target detection method based on MKAN-DETR in complex scenarios, characterized by the following steps: Step 1: Obtain a dataset containing clear images of vehicle targets in complex scenes, and divide the dataset into training and testing sets; Step 2: Construct a vehicle target detection model for complex scenarios (MKAN-DETR); Step 3: Use the training set to train the vehicle target detection model (MKAN-DETR) for complex scenarios constructed in Step 2 to obtain the trained vehicle target detection model; Step 4: Use the vehicle target detection model trained in Step 3 to test the test set and obtain the vehicle target detection results.

[0007] In step 1, a vehicle target detection dataset for training is constructed: an experimental dataset is built by extracting one image every 10 frames from the open-source UA-DETRAC vehicle detection dataset, containing 8229 training images and 5625 test images as the original data source; then, based on the coordinate information of the ignored regions provided by the dataset, pixel-level masking is performed on the dataset images to set the pixel values ​​in the ignored regions to zero. This is shown in Table 1 below.

[0008] In step 2, the vehicle target detection model includes a backbone network, an enhanced hybrid encoder, and a Transformer decoder. The backbone network is used for multi-scale feature extraction of the image, the enhanced hybrid encoder is used to enhance the model's ability to perceive the global context, and the Transformer decoder is used to enhance the target localization accuracy. Specifically, it includes: The original HGNetV2 backbone of RT-DETR was replaced with the structurally reparameterized RepViT backbone network. An enhanced hybrid encoder was constructed by improving the original efficient hybrid encoder of RT-DETR: a multi-scale dual-path fusion (MDPF) module was integrated into the hybrid encoder to enhance the model's ability to capture occluded targets and subtle features; the neck network was reconstructed into an adaptive context feature fusion (ACFF) network by using a cascaded multi-dilation context (CMDC) module and an attention-guided downsample (Adown) module to achieve cross-scale semantic information transfer and spatial detail supplementation; finally, a KAN-Decoder based on KANs network was introduced to realize vehicle target detection in occluded and small target scenarios.

[0009] Step 2 includes: 2.1: Constructing the RepViT backbone network for extracting multi-scale features from images; 2.2: Construct a multi-scale dual-path fusion module to enhance the capture capability of occluded targets and subtle features; 2.3: Construct a cross-scale contextual feature fusion network to enhance the robustness and discriminative power of feature extraction; 2.4: Constructing the KAN-Decoder to improve the bounding box regression fitting ability.

[0010] 2.1 includes: The RepViT backbone network adopts a four-stage hierarchical architecture, extracting multi-scale features through progressive downsampling: After the network receives an input image, it first performs initial feature extraction through a convolutional layer with a stride of 2, followed by four stages of feature extraction modules. Each stage consists of multiple stacked RepViT basic modules, which integrate depthwise separable convolution and a token mixing mechanism. Each RepViT basic module consists of a token mixer and a channel mixer connected in series. (1); in: For the input feature map, This is the final output feature map for the module. and These represent the Token mixer and the Channel mixer, respectively. The structures of the two mixers are as follows: When the stride of the token mixer is 1, multi-branch deep convolution is used to fuse multi-scale spatial information: (2); When the stride of the token mixer is 2, it degenerates into a standard depthwise separable convolution: (3); In the above formula: For 1×1 standard convolution: It is a 3×3 standard convolution; This is the output feature map of the Token mixer; It is a 3×3 depthwise convolution. It is a 1×1 depthwise convolution; For Squeeze-and-Excitation channel attention modules; It is a 1×1 pointwise convolution.

[0011] The channel mixer employs an inverse bottleneck FFN structure with residual connections: (4); in: To increase the number of channels from Expand to Pointwise convolution; The activation function for the Gaussian error linear unit; To increase the number of channels from Compress back Pointwise convolution.

[0012] The different stages are connected by downsampling layers, with the feature map resolution decreasing sequentially and the number of channels gradually increasing, forming four feature layers of different scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32. While maintaining a lightweight architecture, the network uses reparameterization techniques to enhance expressive power during training with a multi-branch structure, and merges these into a single-path convolution during inference to improve computational efficiency. The first two stages focus on local feature extraction, while the latter two emphasize global semantic understanding. The output multi-level feature maps can be directly interfaced with core networks such as FPN and PAN.

[0013] 2.2 includes: 2.2.1: Construct a dilated convolution with different dilation rates to build a multi-scale attention (MSA) layer with multiple receptive fields: Multi-scale attention (MSA) mainly consists of two parallel multi-scale dilated branches (MSDB), which are applied to the input features. , Let represent a third-order real tensor, where Indicates the number of channels. Indicates the height of the feature map. Indicates the width of the feature map. A single MSDB branch represents the input features. During processing, firstly, 1×1 convolution dimensionality reduction is required, and then the features are divided into 3 independent subgroups in the channel dimension as shown in equation (5). , , and These correspond to the front, middle, and back of the dimensionality-reduced feature maps, respectively. The characteristic responses of each channel. Each subgroup is passed through channels with a specific expansion rate. The 3×3 depthwise convolution is processed to extract spatial features in parallel under different receptive fields, thereby obtaining the depthwise convolution output features of each subgroup as shown in Equation (6). , , and They are respectively , and After a specific expansion rate The depthwise convolution output features are then concatenated and followed by a 1×1 convolution to recover the number of channels, resulting in a single branch output. As shown in equation (7): (5); (6); (7); In the above formula: For the segmentation function; ,in, For a 1×1 convolution, For batch normalization; For activation functions; Corresponding to a specific expansion rate A 3×3 depthwise convolution; the two MSDB branches take... and , ; This is a concatenation function.

[0014] The outputs of the two branches are then summed element-wise to fuse multi-scale information, and a spatial attention map is generated using the Sigmoid function. As shown in equation (8), spatial attention maps are used. Enhanced features are obtained by recalibrating the original input. : (8); (9); In the above formula: For a 1×1 convolution, This represents the Sigmoid function. This is a spatial attention map. This indicates element-wise multiplication. For output features.

[0015] 2.2.2: Constructing a multi-level attention neural operator: The Multi-Pole Attention Neural Operator (MANO) does not use separate network layers to process different scales, but instead defines a convolutional kernel with shared weights. As a recursive operator to construct multi-scale feature representations, such as Figure 5 As shown. From the original high-resolution input start, To represent a third-order real tensor, Indicates the height of the feature map. Indicates the width of the feature map. This represents the feature embedding dimension, i.e., the number of channels. This is achieved by repeatedly applying this convolutional kernel. Level downsampling generates a series of feature map sequences with half the resolution. For any level , Its feature map From the previous level feature map via convolution kernel Transformed from, that is The convolution kernel thus acts as a scale-independent projector, ensuring that features at different levels remain consistent in the semantic space.

[0016] In obtaining feature maps at each level Subsequently, to capture features while maintaining linear computational complexity, the model divides the feature map of each layer into potentially overlapping sliding windows and independently computes self-attention within each window. For the ... Extract queries from each window of the layer. ,key Sum Calculate the standard attention map As shown in equation (10). Since the calculation is restricted to a local window, redundancy in full-image calculation is avoided. At the same time, cross-window information interaction can be achieved by utilizing the overlap and hierarchical mixing of sliding windows, thereby obtaining weighted features. .

[0017] (10); (11); In the above formula: for Normalization function, The first The query, key, and value matrix extracted from the layer. These represent the channel dimensions of the feature map. For the first Standard attention map of the layer, For the first Features of the layer after attention weighting.

[0018] To distribute at different resolution levels Attention features The aggregation results in a final full-resolution output. The Multi-Pole Attention Neural Operator (MANO) also introduces a convolution with downsampling. Corresponding shared transpose convolution As shown in equation (12). For any level The algorithm applies recursively. The upsampling operator restores the attention features to the original resolution, where layer 0 directly maintains the identity mapping; the final output The sum of features recovered from all levels is fused by linear stacking, integrating all information from fine details to the global context. (12); (13); In the formula: To restore to the original resolution Layer features, For shared transpose convolution as an upsampling operator, Indicates the upsampling operator Recursive application Second-rate, For attention computation operations, This is the final full-resolution output feature.

[0019] 2.2.3: Building a multi-scale dual-path fusion module: The AIFI module is part of the efficient hybrid encoder in the original RT-DETR model. It retains the AIFI from the original RT-DETR model, concatenates MSA at its front end, and simultaneously uses the multi-level attention neural operator (MANO) in parallel to construct the multi-scale dual-path fusion module MDPF. Assuming the input features... The high-level feature maps of the RepViT backbone network, and the feature output results of the multi-scale dual-path fusion module MDPF are as follows: (14); In the formula: Processed by the MSA module. The output of the MSA module. Processed by the AIFI module. Processed by the MANO module. This is the final output feature of the MDPF module.

[0020] The features enhanced by MSA are fed into a parallel structure consisting of AIFI and MANO. The AIFI branch continues to capture pixel-based global correlations, while the MANO branch is responsible for long-distance dependencies and dynamically adjusting channel weights, thereby enhancing the model's ability to capture occluded targets and subtle features, and improving detection accuracy and robustness under a lightweight architecture.

[0021] 2.3 includes: 2.3.1: Build a Cascaded Multi-Expansion Context Module (CMDC); The CMDC module cascades multiple Multi-DilationContextBlock (MDCB) modules. This module employs a three-way parallel branch structure, configuring different dilation rates to respectively extract local details, mesoscale context, and large-scale semantic information. This effectively constructs multi-layered receptive fields while maintaining low computational cost, enhancing the model's ability to perceive targets at different scales. Simultaneously, global average pooling and channel attention mechanisms are introduced during the feature fusion stage to adaptively refine the joint features, enabling the model to dynamically adjust the response intensity of each channel based on the global context.

[0022] The entire cascaded multiple dilatational context module (CMDC) is built around a multiple dilatational context module (MDCB). The computation process of the MDCB mainly includes three stages: feature dimensionality reduction, multi-scale extraction, and fusion mapping. Assume the input features are... , This represents the number of input feature channels. First, the number of channels is adjusted using a 1×1 convolution. To obtain intermediate features As shown in equation (15), Subsequently, the system simultaneously enters three parallel 3×3 depthwise convolution branches with different dilation rates. After concatenating the three features to obtain 3n channels, normalized activation is performed, and the channel number is mapped back through a 1×1 convolution. Finally, as shown in Equation (16), the fused features are obtained by performing a residual connection with the input features. : (15); (16); In the formula: This corresponds to a depthwise convolution with a dilation rate of 3×3; No. The expansion rate is Branching characteristics; , ; It is a 1×1 convolution; For batch normalization; For activation functions; This is a concatenation function; express Branch features after depthwise convolution with a dilation rate of 1, express Branch features after depthwise convolution with a dilation rate of 2, express Branch features after depthwise convolution with a dilation rate of 3.

[0023] By cascading three multiple dilatational context modules (MDCBs), cascaded fused features are obtained. As shown in equation (18), next, for An adaptive average pooling operation is performed to compress the spatial dimension, thereby aggregating global spatial information to generate channel descriptors. This descriptor is then processed through an FFN structure, first reducing dimensionality to decrease computation, and then restoring it to the original channel dimension; as shown in the following equation: (17); In the above formula: for Output features after adaptive average pooling; To reduce the weights of the fully connected layer, the number of channels is reduced from... Compress to ; For compression ratio, take ; For activation functions; To increase the weights of the fully connected layer, the number of channels is increased from... Restore to ; Finally, the normalized channel weights are generated using the Sigmoid activation function and compared with the original input. Element-wise multiplication is performed to achieve adaptive recalibration of key channel features. As shown in Equation (19), due to the input features of the cascaded multi-expansion context module (CMDC) It also needs to undergo a 1×1 convolution, and then... The features are added together and then fused through a 1×1 convolution to obtain the output features. : (18); (19); In the formula: To cascade three MDCBs and recalibrate branch characteristics. It is a feedforward neural network. This represents the Sigmoid function. This indicates element-wise multiplication. , For 1×1 convolution, For batch normalization.

[0024] 2.3.2: Setting up the Adown module: The Adown (Attention-guided Downsample) module, proposed by YOLOv9, replaces the original downsampling step in the CCFF of RT-DETR. This module first uniformly divides the input feature map into two parts along the channel dimension; it applies a 3×3 convolutional downsampling operation to the first part of the feature map; it then applies a 2×2 average pooling operation to the second part of the feature map, followed by a 1×1 convolution to adjust the number of channels; finally, it concatenates and merges the feature maps processed by the two branches along the channel dimension, as shown below. Figure 9 As shown, this achieves the goal of minimizing the loss of detail information while reducing resolution.

[0025] 2.3.3: Constructing a cross-scale contextual feature fusion network: The three feature maps at different scales output by the backbone network are obtained as input to the cross-scale context-aware feature fusion network (Adaptive Context Feature Fusion, ACFF), and are labeled as shallow feature maps in descending order of spatial resolution. Mid-layer feature map Deep feature map The three feature maps have three independent data paths: Perform top-down feature fusion. First, process the deep feature map... Perform an upsampling operation to double its resolution, and then combine it with the mid-level feature map. The feature maps are concatenated along the channel dimension. The concatenated feature maps serve as the input to the first CMDC module. After processing by this module, the output is a medium-scale intermediate feature map with the same resolution as the input, denoted as . ; For intermediate feature maps at medium scale Perform an upsampling operation to double its resolution, and then compare it with the shallow feature map. The feature maps are concatenated along the channel dimension. The concatenated feature maps serve as input to the second CMDC module. After processing by this module, the output is a highest-resolution feature map that incorporates top-level semantic information, denoted as... The As the first output path of the network; Perform bottom-up feature fusion. Combine feature maps... As input to the Adown module for downsampling, a feature map with half the resolution is obtained after the downsampling operation; this downsampled feature map is then compared with an intermediate feature map at a medium scale. The data is concatenated along the channel dimension and then input into the third CMDC module. After processing by this module, a medium-resolution feature map is output, denoted as... The As the second output path of the network; feature map As input to the Adown module's downsampling, a feature map with its resolution halved is obtained after the downsampling operation; this downsampled feature map is then compared with the deep feature map. The data is concatenated along the channel dimension and input into the fourth CMDC module. After processing by this module, the lowest resolution feature map with the strongest global receptive field is output, denoted as... The As the third output path of the network.

[0026] After the bidirectional feature fusion process described above, the ACFF structure finally outputs enhanced feature maps at three scales. These feature maps contain rich semantic information and fine spatial details, which can improve the detection performance of the model in complex backgrounds and small target scenes while maintaining real-time performance.

[0027] 2.4 includes: 2.4.1: Building the RT-DETR decoder; RT-DETR's decoder is based on the Deformable Transformer architecture, consisting of six cascaded decoder layers. Its self-attention mechanism and parallel computing capabilities effectively capture long-range dependencies in the input feature map, thereby improving the detection performance and inference speed for distant targets. Within each decoder layer, the query vector first interacts through a multi-head self-attention mechanism to capture contextual dependencies. Each attention head is computed as follows: (20); In the above formula: Indicates the first One point of attention; Use the Softmax activation function; , , These are respectively query, key, and value vectors; For the first The projection matrix of each attention head; Feature dimensions for each head; This is a scaling factor to prevent the dot product result from becoming too large.

[0028] The module then enters the multi-scale deformable cross-attention module, using the center of the reference bounding box as the sampling reference point, and performs sparse sampling on the multi-scale feature map output by the encoder. The multi-scale deformable cross-attention is shown in the following equation: (twenty one); In the above formula: For query vector; The coordinates of the center point of the reference bounding box are given, where The horizontal coordinates of the center point of the bounding box. The vertical coordinates of the center point of the bounding box; For feature layer-level indexing, a total of One scale; For the sampling point index, sampling is performed at each scale. One point; The learnable sampling offset predicted from the query vector; In the first Sampling features at the offset position on the layer feature map; Let be the attention weights for the corresponding sampling points, and satisfy . By aggregating features from only key sampling points within the neighborhood, local visual information can be extracted at a lower computational cost. Finally, the updated features are output via a feedforward neural network and residual connections for the next stage of refinement.

[0029] 2.4.2: Setting up KANs: Kolmogorov-Arnold Networks (KANs), based on the Kolmogorov-Arnold representation theorem, propose an innovative architecture that transfers the activation function from network nodes to connection edges and uses learnable B-spline functions for parameterization. In this architecture, each connection edge is equipped with an independent activation function that can be optimized through training, enabling the network to adaptively learn the optimal nonlinear transformation. The core component of KANs is the KAN Linear Layer, whose final output is obtained by adding the computation results of the base branch and the spline branch. The base branch retains the structural characteristics of traditional neural networks, applying the base activation function to the input and performing a linear transformation in conjunction with the weight matrix, primarily responsible for extracting the global linear features of the data. The spline branch is key to KAN's ability to implement learnable activation functions; this branch first computes the B-spline basis function values ​​of the input at predefined spline grid nodes. B-spline basis functions are a class of piecewise polynomials with local support, defined recursively as follows: (twenty two); (twenty three); In the formula: For nodes Starting with the 0th order B-spline basis function, For the node sequence Nodes , For the node sequence 1 node For input variables. Represented by node Starting from B-order spline basis functions; For the node sequence 1 node For nodes Starting from B-order spline basis functions, For the node sequence 1 node For nodes Starting from B-order spline basis functions.

[0030] The calculated basis function values ​​are then linearly combined with the learnable spline weight matrix. By optimizing these trainable weights, the network can fit a univariate function curve of arbitrary shape, thereby achieving local nonlinear refinement of the input, as shown in equation (24). (twenty four); In the formula: This represents the output features of the KAN linear layer; and These are the trainable weight matrices acting on the basic branch and the spline branch, respectively; the basic activation function is the SiLU function. , This represents the Sigmoid activation function; Let B-spline basis function vectors be used. Indicates the 0th node Starting from k B-order spline basis functions; Indicates the first n 1 node n Starting from k B-order spline basis functions; This represents the transpose of a vector.

[0031] 2.4.3: Setting up KAN-Decoder: KAN is introduced into the DeformableTransformer prediction head of the RT-DETR decoder, replacing the original Multilayer Perceptron (MLP). In the iterative bounding box refinement strategy, the encoder bounding box head first uses KAN to generate initial reference boxes. As a starting point for optimization: (25); In the above formula: For encoder KAN prediction head; For encoder output features; The initial reference box is represented as ,in: The horizontal coordinates of the center point of the bounding box. The vertical coordinates of the center point of the bounding box. The width of the bounding box. The height of the bounding box.

[0032] Before each subsequent layer of decoding computation, the system uses the reference box of the previous layer to generate the position embedding of the current layer through the query position header constructed by MLP, and injects the latest geometric information into the query feature: (26); In the above formula: Embed an MLP generator for the location; For the first The reference box output by the layer; For the injection of the first Location embedding of layer query features.

[0033] The output features of the current layer are ultimately fed into the class score head and the bounding box regression head built based on KAN. This regression head leverages KAN's powerful non-linear mapping capabilities to calculate the corrected offset relative to the current reference box, enhancing the model's localization accuracy when handling small targets and occluded samples. That is: (27); Iterative updates based on the current reference frame: (28); In the above formula: For the first The output query features of the layer decoder; These are the relative correction offsets predicted by KAN, corresponding to the correction values ​​for the center point coordinates and width and height of the bounding box, respectively. This is the reference box output from the previous layer; This is the updated bounding box prediction result for the current layer.

[0034] In step 3, during training, the vehicle target detection model MKAN-DETR uses a joint loss function composed of classification loss and bounding box regression loss. The classification loss constrains the accuracy of target category prediction, while the bounding box regression loss constrains the accuracy of target localization. The classification loss is calculated using FocalLoss and is defined as follows: (29); In the formula: For the first Each match predicts the confidence probability of the true class; As a category balance factor; For focusing parameters; To accurately label the target quantity.

[0035] The bounding box regression loss consists of two parts: L1 loss and GIoU loss, defined as follows: (30); In the formula: For the first One true labeled bounding box; The predicted bounding box that matches it; and This is the balancing weighting coefficient for the two losses.

[0036] L1 loss is calculated as the sum of the absolute differences between the four-dimensional coordinates of the predicted bounding box and the ground truth bounding box: (31); GIoU loss introduces a minimum closed region penalty term on top of IoU, defined as: (32); In the formula: This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. It is the smallest closed rectangular region that simultaneously contains both the predicted bounding box and the ground truth bounding box; This is the area within the smallest closed region that is not covered by any box, used to penalize cases where two boxes are too far apart.

[0037] The overall training loss function of RT-DETR is a weighted sum of the classification loss and the bounding box regression loss: (33); In the formula: and The loss weight coefficient is used to balance the contribution ratio of the two tasks, class prediction and bounding box localization, to model training.

[0038] During training, the AdamW optimizer, combined with the backpropagation algorithm, continuously adjusts the weight parameters in the network, gradually reducing the calculated joint loss function. Backpropagation calculates the loss for each parameter using the chain rule. gradient: (34); The AdamW parameter update rule is as follows: (35); In the above formula: The gradient of the current parameters; This is a first-moment estimate of the gradient; This is the second moment estimate of the gradient; The learning rate; It is a numerically stable term; This is the weight decay coefficient; the second term The third term represents the adaptive gradient update amount. As a weight decay term, both factors jointly drive the parameter updates in the direction of loss reduction, making... It gradually decreases until it converges.

[0039] After multiple rounds of iterative training, the model gradually converges, ultimately achieving optimal object detection performance. After convergence, mAP is used as the comprehensive evaluation metric for the optimal detection performance. (36); In the above formula: For the first The class has a recall rate of The corresponding precision; the average precision for each class is obtained by integrating the precision-recall curve. Then take the average value of all categories to get This serves as a comprehensive measure of the final detection performance.

[0040] Step 3 includes the following steps: To begin training the model, the training image containing the target to be detected is input into the vehicle target detection model MKAN-DETR. The image first passes through the RepViT backbone network to extract multi-scale features, then passes through the MDPF module to suppress background noise, and then passes through the ACFF structure to perform cross-scale context-aware feature fusion to obtain an enhanced feature map. The fused features are then fed into the Transformer encoder for global feature modeling, and finally the decoder generates the target's category prediction and bounding box coordinate prediction.

[0041] Decoder Layer output query features through Generate confidence scores for each category: (37); In the above formula: ,in To query the number of vectors, Total number of target categories; Map the output to An interval represents the confidence probability for each category.

[0042] The predicted corrected offset relative to the reference box is iteratively updated to obtain the final bounding box: (38); In the above formula: , and These consist of the center point coordinates and the four-dimensional parameters of width and height of the bounding box prediction result; This is the reference bounding box output from the previous layer; For the first The layer decoder outputs query features. After... After layer iterations, the output of the last layer is taken as the final prediction result: (39); In the above formula: For the first Category prediction for each query; For the first Bounding box prediction for each query; This represents the number of query vectors.

[0043] The network's prediction results are matched with the corresponding ground truth annotations. For each prediction-ground truth annotation pair... Calculate the matching cost: (40); In the above formula: - Category cost: the predicted bounding box relative to the true category. The confidence level is negative; - L1 regression cost, a measure of the predicted bounding box With real frame The absolute difference of the coordinates; - GIoU cost, a measure of the predicted bounding box With real frame The degree of geometric overlap between them; For generalized intersection-union-ratio, it is defined as: (41); in: This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. It is the smallest closed rectangular region that simultaneously contains both the predicted bounding box and the ground truth bounding box; It represents the area within the smallest closed region that is not covered by any frame.

[0044] - : The balance weight coefficient of each cost item.

[0045] The optimal binary matching between predicted and ground truth bounding boxes is achieved using the Hungarian algorithm. Classification loss and bounding box regression loss are calculated based on the matching results. The RT-DETR decoder output is then used. Each prediction result Includes category confidence probability and bounding box coordinates Let the set of true annotations be... One target, one real-world label Includes real category labels and the actual bounding box coordinates The Hungarian algorithm finds the match by minimizing the matching cost. The prediction results and The optimal one-to-one correspondence between each real label, and the optimal matching. Defined as: (42); In the above formula: For the first A real label With the Prediction results The matching cost between them; For all possible matching permutations and combinations; In order to be with the first An index of prediction results that match real labels.

[0046] Network parameters are updated using the backpropagation algorithm, based on the joint loss obtained from forward inference. The gradients of each parameter are calculated by backpropagating layer by layer using the chain rule: (43); Update network parameters using the AdamW optimizer: (44); The gradient starts from the loss layer and flows sequentially through the decoder → encoder → ACFF → MDPF → RepViT backbone, driving the parameters of each module to be updated in the direction of loss reduction.

[0047] Repeat the iterative learning process of forward inference and parameter optimization described above until the model converges.

[0048] Each iteration performs forward inference and parameter updates; the loss increases with the number of iterations. Monotonically decreasing: (45); Training terminates when the convergence condition is met. (46); In the above formula: This represents the optimal loss value after convergence. The threshold for determining convergence. Model parameters after convergence. These are the optimal parameters, corresponding to the optimal target detection performance.

[0049] In step 4, the road traffic image requiring vehicle detection is input into the trained vehicle target detection model MKAN-DETR. The model ultimately outputs the category information and bounding box coordinates of each vehicle target in the image, achieving accurate detection and localization of vehicle targets in the road scene. See Table 2 below.

[0050]

[0051] This invention provides a vehicle target detection method based on MKAN-DETR in complex scenarios, with the following technical advantages: 1) This invention introduces RepViT to replace the original RT-DETR backbone network, which reduces computational overhead while effectively taking into account both lightweight design and multi-scale feature representation capabilities. This allows the model to significantly reduce the number of parameters and inference time while maintaining high detection accuracy, making it suitable for deployment on edge devices and embedded platforms with limited computing resources.

[0052] 2) This invention integrates the MDPF module and ACFF structure, utilizing the contextual mechanisms of MSA and CMDC to significantly enhance the activation level of weak signals such as vehicle outlines in low-light environments while suppressing invalid background noise such as rain and glare. This enables more accurate perception of the spatial distribution of vehicle targets at different scales and in dense scenes, improving the detection capabilities for occluded vehicles and dense traffic flow.

[0053] 3) This invention introduces KAN-Decoder to construct an adaptive nonlinear regression head, which integrates low-level detailed features with high-level semantic features, effectively improving the localization accuracy of occluded targets and small targets. It can accurately detect distant small vehicles and partially occluded vehicle targets in complex road environments.

[0054] 4) This invention adopts an end-to-end detection framework design, which directly learns the mapping relationship between the input image and the target detection result. It does not require complicated post-processing operations, has a fast detection speed and short response time, and is suitable for application scenarios with high timeliness requirements such as road test perception and real-time traffic monitoring. Attached Figure Description

[0055] The present invention will be further described below with reference to the accompanying drawings and examples; Figure 1 This is a flowchart of the present invention.

[0056] Figure 2 This is a diagram of the MKAN-DETR network structure of the present invention.

[0057] Figure 3 This is a diagram of the RepViT backbone network structure of the present invention.

[0058] Figure 4 This is a diagram of the multi-scale attention structure of the present invention.

[0059] Figure 5 This is a structural diagram of the multi-scale dual-path fusion module of the present invention.

[0060] Figure 6 This is a structural diagram of the cascaded multi-expansion context module of the present invention.

[0061] Figure 7 This is a diagram of the adaptive context feature fusion network structure of the present invention.

[0062] Figure 8 This is a structural diagram of the KAN-Decoder of the present invention.

[0063] Figure 9 This is a structural diagram of the Adown (Attention-guided Downsample) module. Detailed Implementation

[0064] like Figures 1-8 As shown, this embodiment of the invention provides a vehicle target detection method based on MKAN-DETR in complex environments, including the following steps: S1: Constructing a vehicle target detection dataset for training: An experimental dataset is constructed by extracting one image every 10 frames from the open-source UA-DETRAC vehicle detection dataset. This dataset contains 8229 training images and 5625 test images as the original data source. Then, based on the coordinate information of the ignored regions provided by the dataset, pixel-level masking is performed on the dataset images to set the pixel values ​​within the ignored regions to zero.

[0065] S2: The original HGNetV2 backbone network of RT-DETR is replaced with the structurally reparameterized RepViT. An enhanced hybrid encoder is constructed by improving the original efficient hybrid encoder of RT-DETR: a Multi-Scale Dual-Path Fusion (MDPF) module is integrated into the hybrid encoder to enhance the model's ability to capture occluded targets and subtle features. Then, a Cascaded Multi-Dilation Context (CMDC) module and an Attention-guided Downsample (Adown) module are used to reconstruct the neck network into an Adaptive Context Feature Fusion (ACFF) network to achieve cross-scale semantic information transfer and spatial detail supplementation. Finally, a KAN-Decoder based on KANs networks is introduced to enable vehicle target detection in occluded and small target scenarios. The specific steps are as follows: S2.1: Constructing the RepViT backbone network for extracting multi-scale features from images: An input image with a spatial resolution of 128×128 and 3 channels is fed into the PatchEmbedding module of the RepViT network. First, it passes through a Conv2d_BN convolutional layer with a kernel size of 3×3, a stride of 2, and 24 output channels, and is then processed by the GELU activation function to obtain a feature map with a spatial resolution of 64×64 and 24 channels. This feature map is then fed into a Conv2d_BN convolutional layer with a kernel size of 3×3, a stride of 2, and 48 output channels to obtain a feature map F1 with a spatial resolution of 32×32 and 48 channels.

[0066] The feature map F1 with a spatial resolution of 32×32 and 48 channels is input into the first RepViTBlock module, which has a stride of 1 and includes two sub-structures: TokenMixer and ChannelMixer. The TokenMixer uses the RepVGGDW depthwise separable convolutional structure and the Squeeze-Excite attention module for feature extraction. The ChannelMixer uses 1×1 pointwise convolution for channel expansion and compression, and adds the input features and output features through residual connections to obtain the feature map F2 with a spatial resolution of 32×32 and 48 channels.

[0067] The feature map F2 is input into the second and third RepViTBlock modules in sequence. The structure of the two modules is the same as that of the first RepViTBlock, and the step size is 1. Spatial feature mixing is performed through TokenMixer, channel information interaction is performed through ChannelMixer, and information transmission is maintained through residual connection. Finally, the feature map F3 with a spatial resolution of 32×32 and 48 channels is obtained.

[0068] The feature map F3 is input into the fourth RepViTBlock module, which has a stride of 2. First, spatial downsampling is achieved through depthwise convolution, then the number of channels is adjusted to 96 through 1×1 convolution, and channel transformation is performed using ChannelMixer to obtain the feature map F4 with a spatial resolution of 16×16 and 96 channels.

[0069] The feature map F4 with a spatial resolution of 16×16 and 96 channels is sequentially input into the 5th, 6th and 7th RepViTBlock modules. The stride of each module is 1. Feature extraction is performed through the RepVGGDW depthwise convolution and channel hybrid structure, and residual connections are used to enhance the feature representation capability. Finally, the feature map F5 with a spatial resolution of 16×16 and 96 channels is obtained.

[0070] The feature map F5 is input into the 8th RepViTBlock module, which has a stride of 2. It performs downsampling through depthwise convolution and expands the number of channels to 192 through pointwise convolution, resulting in a feature map F6 with a spatial resolution of 8×8 and 192 channels.

[0071] The feature map F6 with an 8×8 spatial resolution and 192 channels is sequentially input into multiple RepViTBlock modules, each with a stride of 1. Spatial information is mixed through a TokenMixer structure, and channel information is exchanged through a ChannelMixer structure. Residual connections are introduced within the modules to enable the network to learn residual features, thereby obtaining a deep feature map F7 with an 8×8 spatial resolution and 192 channels.

[0072] The feature map F7 is input into the RepViTBlock module with a stride of 2. Further downsampling is achieved through depthwise convolution, and the number of channels is expanded to 384 through pointwise convolution, resulting in a feature map F8 with a spatial resolution of 4×4 and 384 channels.

[0073] The feature map F8 is sequentially input into two subsequent RepViTBlock modules with a stride of 1. Deep semantic information is further extracted using the TokenMixer and ChannelMixer structures, while maintaining the residual connection structure. The final output feature map F9, with a spatial resolution of 4×4 and 384 channels, is obtained. This feature map represents the high-level semantic output features of the RepViT backbone network.

[0074] In this embodiment, convolution is the core computational operation in convolutional neural networks. Spatial information of input features is extracted by setting the convolution kernel size, stride, and padding method. Batch normalization is used to standardize the feature distribution to improve the stability of network training and speed up convergence. The GELU activation function is used to enhance the nonlinear expressive power of the model, enabling the network to learn more complex feature representations.

[0075] In this embodiment, the RepViTBlock module adopts a structure combining TokenMixer and ChannelMixer. TokenMixer achieves feature interaction in the spatial dimension through depthwise separable convolution and attention mechanism, while ChannelMixer achieves information fusion in the channel dimension through pointwise convolution and adds input features and output features through residual connections, enabling the network to learn residual mapping, thereby alleviating the gradient vanishing problem in deep network training and improving the model's expressive power and training stability.

[0076] S2.2: The specific steps for constructing the Multi-Scale Dual-Path Fusion Module (MDPF) to enhance the capture capability of occluded targets and subtle features are as follows: Input a feature map with a spatial resolution of 20×20 and 256 channels into the AIFI module. First, save the feature map as the input feature map Identity for the residual branch. The feature map input is fed into the multi-scale attention module (MSA). This module consists of multiple parallel multi-scale dilated convolutional branches with different dilation rates. Each branch includes a 1×1 convolution, a depthwise separable convolution (with different dilation rates), and another 1×1 convolution structure. The output features from multiple branches are element-wise summed and fused, then channel fusion is performed through a 1×1 convolution, and attention weights are generated using a sigmoid activation function. These attention weights are then element-wise multiplied with the original input Identity to obtain an attention-enhanced feature map with a spatial resolution of 20×20 and 256 channels. .

[0077] feature map Perform a dimensional transformation, flattening the shape [B,256,20,20] and replacing it with a sequence of features of shape [B,400,256]. A two-dimensional sine and cosine position code is generated based on a width of 20, a height of 20, and a channel number of 256. The code has a shape of [1, 400, 256]. Element-wise addition with two-dimensional sine and cosine position codes yields sequence features containing positional information. .

[0078] sequence features The input is fed into the Transformer encoder layer, where a multi-head self-attention mechanism models the correlations between different positions in the sequence, thereby capturing global contextual information and obtaining the attention output feature. This feature is then processed by Dropout and added to the input feature residuals, and normalized by the first layer to obtain the final feature. ;Then the features The input is a feedforward neural network consisting of two fully connected layers and a GELU activation function. First, a first linear transformation maps the channel dimension from 256 to 2048, followed by GELU activation and Dropout processing. Then, a second linear transformation restores the channel dimension to 256. The output is then compared with... The residuals are summed and then normalized in a second layer to obtain the features of the Transformer output sequence. Its shape is [B,400,256].

[0079] sequence features By reshaping the dimension and permuting and reshaping the feature map into shape [B,256,20,20], we obtain the Transformer output feature map. Simultaneously, the residual branch feature map Identity is input into the MANO module. This module first performs channel mapping on the feature map and converts it into a spatial sequence representation of [B, 20, 20, 256]. Multi-scale local attention is then calculated using a multipole attention structure to extract local contextual information at different sampling scales. Multi-scale feature fusion is performed through layer-by-layer downsampling and upsampling structures, and feature enhancement is combined with a feedforward network to finally generate a feature map with local enhancement capabilities. Its spatial resolution is 20×20.

[0080] Finally, the Transformer outputs feature maps. Output characteristics of the MANO module Element-wise addition and fusion are performed to obtain the final output feature map of the AIFI module, with a spatial resolution of 20×20 and 256 channels.

[0081] In this embodiment, specifically, convolution operations are used to extract spatial information and map channels. Dilated convolution expands the receptive field by setting different dilation rates, enabling the model to acquire multi-scale contextual information while maintaining spatial resolution. Batch normalization is used to standardize convolutional features, thereby accelerating network convergence and improving training stability. The GELU activation function provides smooth nonlinear transformation capabilities, enabling the neural network to learn more complex data distributions.

[0082] Furthermore, the multi-head self-attention mechanism in the Transformer encoder layer can simultaneously model different feature relationships in multiple subspaces, thereby capturing long-distance dependency information; the feedforward neural network enhances the information interaction capability between channels through two layers of linear transformation and nonlinear activation; layer normalization performs normalization operations on the feature dimension, which helps stabilize the training process; residual connections, by directly adding the input features to the output features, enable the network to learn residual mappings, thereby alleviating the gradient vanishing problem in deep network training and improving network performance.

[0083] Two-dimensional sine and cosine positional encoding provides explicit spatial positional information for the Transformer, enabling the model to retain two-dimensional spatial structure during sequence modeling. The MANO module enhances local structural features through multi-scale local attention and channel mapping structure, complementing the global context information extracted by the Transformer. It further enhances feature representation capabilities through residual fusion, thereby achieving global-local collaborative modeling of input features.

[0084] S2.3: The specific steps for constructing a cross-scale context-aware feature fusion network to enhance the robustness and discriminative power of feature extraction are as follows: The three feature maps at different scales output by the backbone network are obtained and then convolved with a 1×1 convolution to a single number of 256, resulting in shallow feature maps with a spatial resolution of 80×80. 40×40 spatial resolution mid-layer feature map 20×20 spatial resolution deep feature map ; A deep feature map with a spatial resolution of 20×20 and 256 channels. After upsampling, a feature map with a spatial resolution of 40×40 and 256 channels is obtained. ; The feature map with a spatial resolution of 40×40 and 256 channels and mid-layer feature map The feature map with 512 channels is obtained by concatenating the features along the channel dimension. This feature map is then input into the first CMDC module to obtain a feature map with a spatial resolution of 40×40 and 256 channels. ; The feature map with a spatial resolution of 40×40 and 256 channels After upsampling, a feature map with a spatial resolution of 80×80 and 256 channels is obtained. ; The feature map with 80×80 spatial resolution and 256 channels and shallow feature map The feature map with 512 channels is obtained by concatenating the features along the channel dimension. This feature map is then input into the second CMDC module to obtain a feature map with an 80×80 spatial resolution and 256 channels. The feature map As the first output path of the cross-scale feature fusion network; The feature map with 80×80 spatial resolution and 256 channels The input is fed into the Adown module, and after downsampling, a feature map with a spatial resolution of 40×40 and 256 channels is obtained. ; The feature map with a spatial resolution of 40×40 and 256 channels and feature map The feature map with 512 channels is obtained by concatenating the features along the channel dimension. This feature map is then input into the third CMDC module to obtain a feature map with a spatial resolution of 40×40 and 256 channels. The feature map As the second output path of the cross-scale feature fusion network; The feature map with a spatial resolution of 40×40 and 256 channels The input is fed into the Adown module, and after downsampling, a feature map with a spatial resolution of 20×20 and 256 channels is obtained. ; A feature map with a spatial resolution of 20×20 and 256 channels and deep feature maps The feature map with 512 channels is obtained by concatenating the features along the channel dimension. This feature map is then input into the fourth CMDC module to obtain a feature map with a spatial resolution of 20×20 and 256 channels. The feature map This serves as the third output path of the cross-scale feature fusion network.

[0085] S2.4: The specific steps for constructing the KAN-Decoder to improve the fitting ability of bounding box regression are as follows: Obtain three feature maps at different resolutions from the cross-scale context-aware feature fusion network output, specifically feature maps at 80×80 spatial resolution. 40×40 spatial resolution feature map and feature maps with 20×20 spatial resolution ; The feature map , as well as The inputs are fed into an input projection module containing 1×1 convolution and batch normalization layers for unified channel mapping. Then, they are flattened in the spatial dimension and spliced ​​along the sequence dimension to obtain a multi-scale feature sequence with 256 channels. The multi-scale feature sequence with 256 channels is input into the encoder output module consisting of a linear layer and layer normalization to obtain an enhanced encoder feature sequence with 256 channels. The encoder feature sequence is input into the linear encoder score head to calculate the initial target category score for each feature point; the top-ranked features are then selected based on the initial target category scores. (like The feature points of the feature points are used to extract the corresponding feature vectors with 256 channels to form the Top-K query features, and the corresponding spatial grid coordinates are generated as the Top-K anchor box sequence. The Top-K query features with 256 channels are input into the bounding box header of an encoder with three network layers built on KAN (Kolmogorov-Arnold Networks). The original multilayer perceptron is replaced by the powerful nonlinear mapping capability of the KAN network. The Top-K query features are nonlinearly mapped using KAN to generate a coordinate offset sequence, which is added to the Top-K anchor box sequence to obtain the initial reference box. Add the coordinate offset sequence to the Top-K anchor frame sequence bit by bit to obtain the sequence containing... An initial reference box sequence of coordinate information is used as the starting point for the optimization of the decoder's iterative bounding box refinement strategy. In each layer of the KAN-Decoder's forward decoding computation, a query position header containing a multilayer perceptron architecture is used to generate a position embedding vector with 256 channels for the current reference box sequence. The position embedding vector is added bitwise to the current query feature sequence (the initial layer is the Top-K query features or a learnable initialization vector) and fused to obtain a query feature sequence with 256 channels that have been injected with the latest geometric information. The query feature sequence with 256 channels injected with the latest geometric information, together with the multi-scale feature sequence, is input into the deformable Transformer decoding layer of the current layer for feature interaction and attention aggregation calculation, to obtain the decoding output feature sequence with 256 channels of the current layer. The decoded output feature sequence with 256 channels in the current layer is input in parallel into the linear decoder fraction head and the bounding box regression head of the decoder with two network layers based on KAN. The linear decoder fraction head performs linear mapping processing on the current layer's decoded output feature sequence, outputting the current layer's contained features. A sequence of target category probabilities for each prediction result; The KAN-based decoder bounding box regression head performs nonlinear processing on the current layer's decoded output feature sequence, and the output includes... A sequence of corrected offsets relative to the current reference box sequence for each prediction result; The corrected offset sequence is superimposed on the current reference box sequence to obtain the updated current layer refined reference box sequence; The above decoding process is iterated sequentially, with the refined reference box sequence and query feature sequence output from the previous layer as the input to the next layer, until the forward propagation of all deformable decoding layers in the KAN-Decoder is completed. The target class probability sequence and refined reference box sequence output from the final layer are used as the final prediction result of target detection.

[0086] S3: The specific steps for training the vehicle target detection model MKAN-DETR built in S2 using the training dataset in S1 are as follows: Initialize the hyperparameters required for training the model, including batch size, training epochs, and initial learning rate. Begin training the model by inputting training images containing the target to be detected into the MKAN-DETR network. The images first pass through the backbone network RepViT to extract multi-scale features, then undergo cross-scale context-aware feature fusion using an ACFF structure to obtain enhanced feature maps. The fused features are then fed into the Transformer encoder for global feature modeling, and finally, the decoder generates target category predictions and bounding box coordinate predictions. Match the network's predictions with the corresponding ground truth annotations. Optimal binary matching between predicted and ground truth bounding boxes is achieved using the Hungarian algorithm. Classification loss and bounding box regression loss are calculated based on the matching results. Backpropagation is used to update the network parameters, and this iterative learning process of forward inference and parameter optimization is repeated until the model converges.

[0087] S4: Input the road traffic image that needs to be detected into the trained MKAN-DETR object detection network, and finally output the category information and bounding box coordinates of each vehicle target in the image to achieve accurate detection and localization of vehicle targets in the road scene.

[0088] Verification Example: To verify the performance of the proposed MKAN-DETR-based vehicle target detection method in complex scenes, this invention utilizes the publicly available vehicle detection dataset UA-DETRAC for comparative experiments. It selects mainstream YOLO series and Transformer architecture target detection algorithms for comparison, including YOLOv8, YOLOv9, YOLOv10, YOLOv11, Anchor-DETR, RT-DETR-R50, LW-DETR, D-FINE, RT-DETRv2, and UAV-DETR—a total of 10 algorithms—for comprehensive evaluation. Since the comparison models have different dependencies on input data formats and underlying evaluation code, this experiment uses both YOLO and COCO formats for training and testing. To ensure consistency in the experimental comparison, this experiment primarily selects mAP50, mAP50-95, number of parameters, and floating-point operations (FLOPs) common to all models as the main performance indicators. Precision and recall are listed only as reference data.

[0089] Let the number of true positive samples that are correctly predicted be the true number ( The number of negative samples that are incorrectly predicted as positive samples is called false positives. The number of positive samples that are incorrectly predicted as negative samples is called false negatives. Accuracy and recall rate Defined as: (47); (48); In the formula, the accuracy rate Recall represents the proportion of results predicted to be positive that are actually positive; This represents the proportion of correctly predicted positive samples out of all actual positive samples. The prediction box is used to determine whether it represents a true positive sample based on a set Intersection over Union (IoU) threshold. .

[0090] Average Precision AP The area under the precision-recall curve (PR curve) is defined as: (49); For including For a target category detection task, the average precision across all categories (mAP) is defined as the average precision of all categories. The mean of the values. When the Intersection over Union (IoU) threshold is strictly set to 0.5, the calculated average precision across all classes is denoted as mAP. 50 The calculation formula is as follows: (50); In the formula, This indicates that when the IoU threshold is 0.5, the first... i The average precision of each category. This represents the precision of the first category when the IoU threshold is 0.5. i Average precision for each category.

[0091] mAP 50-95 This refers to calculating the mean average (mAP) of 10 different thresholds (0.50, 0.55, 0.60, ..., 0.95) with a step size of 0.05 within the Intersection over Union (IoU) threshold range of 0.50 to 0.95, and then averaging these 10 mAP values. The formula is as follows: (50); In the formula, the set This metric can more rigorously and comprehensively reflect the model's accurate positioning capability under high overlap requirements.

[0092] Based on the performance comparison of different algorithm models in Table 3, it can be seen that although the YOLO series achieves lower computational load due to its smaller parameter count, the detection accuracy of the YOLO series is still lower than that of the method of this invention by 65.1%, while the mAP50-95 of the method of this invention reaches 45.9%. Compared with other mainstream algorithms based on the Transformer architecture, the method of this invention reduces the computational burden while maintaining high accuracy. Compared with RT-DETR-R50 and RT-DETRv2, these two not only have a parameter count as high as 42.0M and 36.0M respectively, and FLOPs both exceed 100G, but their mAP50 index is significantly lower than that of the method of this invention. Compared with Anchor-DETR, LW-DETR and D-FINE, the method of this invention shows a higher accuracy advantage. Although UAV-DETR is close to the method of this invention in terms of accuracy, the method of this invention reduces the parameter count by 3.4M and the FLOPs by 31.1G.

[0093]

[0094] Compared with other mainstream models, this invention is more suitable for deployment on vehicle-mounted edge devices or roadside units with limited hardware resources, and meets the real-time requirements of intelligent transportation systems for environmental perception tasks.

Claims

1. A vehicle target detection method based on MKAN-DETR in complex scenarios, characterized in that... Includes the following steps: Step 1: Obtain a dataset containing clear images of vehicle targets in complex scenes, and divide the dataset into training and testing sets; Step 2: Construct a vehicle target detection model for complex scenarios (MKAN-DETR); Step 3: Use the training set to train the vehicle target detection model (MKAN-DETR) for complex scenarios constructed in Step 2 to obtain the trained vehicle target detection model; Step 4: Use the vehicle target detection model trained in Step 3 to test the test set and obtain the vehicle target detection results.

2. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 1, characterized in that: In step 1, a vehicle target detection dataset for training is constructed: an experimental dataset is constructed by extracting one image every 10 frames from the open-source UA-DETRAC vehicle detection dataset, which contains 8229 training images and 5625 test images as the original data source; then, according to the coordinate information of the ignored regions provided by the dataset, pixel-level masking is performed on the dataset images to set the pixel values ​​in the ignored regions to zero.

3. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 2, characterized in that: In step 2, the vehicle target detection model includes a backbone network, an enhanced hybrid encoder, and a Transformer decoder; the backbone network is used for multi-scale feature extraction of images, the enhanced hybrid encoder is used to enhance the model's ability to perceive global context, and the Transformer decoder is used to enhance the target localization accuracy. Specifically, it includes: The original HGNetV2 backbone of RT-DETR was replaced with the structurally reparameterized RepViT backbone network. An enhanced hybrid encoder was constructed by improving the original efficient hybrid encoder of RT-DETR: a multi-scale dual-path fusion (MDPF) module was integrated into the hybrid encoder to enhance the model's ability to capture occluded targets and subtle features; the neck network was reconstructed into an adaptive context feature fusion (ACFF) network by using a cascaded multi-dilation context (CMDC) module and an attention-guided downsample (Adown) module to achieve cross-scale semantic information transfer and spatial detail supplementation; finally, a KAN-Decoder based on KANs network was introduced to realize vehicle target detection in occluded and small target scenarios.

4. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 3, characterized in that: Step 2 includes: 2.1: Constructing the RepViT backbone network for extracting multi-scale features from images; 2.2: Construct a multi-scale dual-path fusion module to enhance the capture capability of occluded targets and subtle features; 2.3: Construct a cross-scale contextual feature fusion network to enhance the robustness and discriminative power of feature extraction; 2.4: Constructing the KAN-Decoder to improve the bounding box regression fitting ability.

5. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 4, characterized in that: 2.1 includes: The RepViT backbone network adopts a four-stage hierarchical architecture design, extracting multi-scale features through progressive downsampling: After the network receives an input image, it first performs initial feature extraction through a convolutional layer with a stride of 2, followed by four stages of feature extraction modules. Each stage consists of multiple stacked RepViT basic modules, which integrate depthwise separable convolution and a token mixing mechanism. Each RepViT basic module consists of a token mixer and a channel mixer connected in series. （1）； in: For the input feature map, The module outputs a feature map as its final output. and These represent the Token mixer and the Channel mixer, respectively. The structures of the two mixers are as follows: When the stride of the token mixer is 1, multi-branch deep convolution is used to fuse multi-scale spatial information: （2）； When the stride of the token mixer is 2, it degenerates into a standard depthwise separable convolution: （3）； In the above formula: For 1×1 standard convolution: It is a 3×3 standard convolution; This is the output feature map of the Token mixer; It is a 3×3 depthwise convolution. It is a 1×1 depthwise convolution; For Squeeze-and-Excitation channel attention modules; It is a 1×1 pointwise convolution; The channel mixer employs an inverse bottleneck FFN structure with residual connections: （4）； in: To increase the number of channels from Expand to Pointwise convolution; The activation function for the Gaussian error linear unit; To increase the number of channels from Compress back Pointwise convolution; Each stage is connected by a downsampling layer, with the feature map resolution decreasing sequentially and the number of channels increasing gradually, forming four feature layers of different scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32. While maintaining a lightweight design, the network uses a multi-branch structure to enhance expressive power during the training stage through reparameterization techniques, and merges into a single-path convolution during the inference stage to improve computational efficiency. The output multi-level feature maps can be directly connected to the neck network.

6. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 5, characterized in that: 2.2 includes: 2.2.1: Construct a dilated convolution with different dilation rates to build a multi-scale attention (MSA) layer with multiple receptive fields: Multi-scale attention (MSA) consists of two parallel multi-scale dilated branches (MSDB), for input features , Let represent a third-order real tensor, where Indicates the number of channels. Indicates the height of the feature map. Indicates the width of the feature map; a single MSDB branch pairs with the input features. During processing, firstly, the features are reduced in dimensionality by 1×1 convolution, and then the features are divided into 3 independent subgroups in the channel dimension as shown in equation (5). , , and These correspond to the front, middle, and back of the dimensionality-reduced feature maps, respectively. The characteristic response of each channel; each subgroup is passed through a specific expansion rate. The 3×3 depthwise convolution is processed to extract spatial features in parallel under different receptive fields, thereby obtaining the depthwise convolution output features of each subgroup as shown in Equation (6). , , and They are respectively , and After a specific expansion rate The depthwise convolution output features are then concatenated and followed by a 1×1 convolution to restore the number of channels, resulting in a single branch output. As shown in equation (7): （5）； （6）； （7）； In the above formula: For the segmentation function; ,in, For 1×1 convolution, For batch normalization; For activation functions; Corresponding to a specific expansion rate A 3×3 depthwise convolution; the two MSDB branches take... and , ; This is a concatenation function; The outputs of the two branches are then summed element-wise to fuse multi-scale information, and a spatial attention map is generated using the Sigmoid function. As shown in equation (8), spatial attention maps are used. Enhanced features are obtained by recalibrating the original input. : （8）； （9）； In the above formula: For 1×1 convolution, This represents the Sigmoid function. This is a spatial attention map. This indicates element-wise multiplication. For output features; 2.2.2: Constructing a multi-level attention neural operator: The Multipolar Attention Neural Operator (MANO) defines a convolutional kernel with shared weights. As a recursive operator to construct multi-scale feature representations from the original high-resolution input start, To represent a third-order real tensor, Indicates the height of the feature map. Indicates the width of the feature map. This represents the feature embedding dimension, i.e., the number of channels; this is achieved by repeatedly applying the convolutional kernel. Level downsampling generates a series of feature map sequences with half the resolution; for any level , Its feature map From the previous level feature map via convolution kernel Transformed from, that is ; In obtaining feature maps at each level Subsequently, to capture features while maintaining linear computational complexity, the model divides the feature map of each layer into potentially overlapping sliding windows and independently computes self-attention within each window; for the Extract queries from each window of the layer. ,key Sum Calculate the standard attention map As shown in equation (10); cross-window information interaction is achieved by utilizing the overlapping and hierarchical mixing of sliding windows, thereby obtaining weighted features. ; （10）； （11）； In the above formula: for Normalization function, The first The query, key, and value matrix extracted from the layer. These represent the channel dimensions of the feature map. For the first Standard attention map of the layer, For the first Features of the layer after attention weighting; To distribute at different resolution levels Attention features The aggregation results in a final full-resolution output. The Multi-Pole Attention Neural Operator (MANO) also introduces a convolution with downsampling. Corresponding shared transpose convolution As shown in equation (12); for any level The algorithm applies recursively. The upsampling operator restores the attention features to the original resolution, where layer 0 directly maintains the identity mapping; the final output The sum of features recovered from all levels is fused by linear stacking, integrating all information from fine details to the global context. （12）； （13）； In the formula: To restore the original resolution Layer features, For shared transpose convolution as an upsampling operator, Indicates the upsampling operator Recursive application Second-rate, For attention computation operations, For the final full-resolution output features; 2.2.3: Building a multi-scale dual-path fusion module: The AIFI of the original RT-DETR model is retained, and MSA is concatenated at its front end. Simultaneously, the multi-scale dual-path fusion module MDPF is constructed by parallelizing the multi-level attention neural operator (MANO); assuming the input features... The high-level feature maps of the RepViT backbone network, and the feature output results of the multi-scale dual-path fusion module MDPF are as follows: （14）； In the formula: Processed by the MSA module. The output of the MSA module. Processed by the AIFI module. Processed by the MANO module. This is the final output feature of the MDPF module.

7. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 6, characterized in that: 2.3 includes: 2.3.1: Build a Cascaded Multi-Expansion Context Module (CMDC); The Cascaded Multiple Inflatable Context Module (CMDC) is built around the Multiple Inflatable Context Module (MDCB). The computation process of the MDCB includes three stages: feature dimensionality reduction, multi-scale extraction, and fusion mapping. Assuming the input features are... , This represents the number of input feature channels; first, the number of channels is adjusted using a 1×1 convolution. To obtain intermediate features As shown in equation (15), Subsequently, the system simultaneously enters three parallel 3×3 depthwise convolution branches with different dilation rates; after concatenating the three features to obtain 3n channels, it performs normalized activation and then maps the channel number back through a 1×1 convolution. Finally, as shown in Equation (16), the fused features are obtained by performing a residual connection with the input features. : （15）； （16）； In the formula: This corresponds to a depthwise convolution with a dilation rate of 3×3; No. The expansion rate is Branching characteristics; , ; It is a 1×1 convolution; For batch normalization; For activation functions; This is a concatenation function; express Branch features after depthwise convolution with a dilation rate of 1, express Branch features after depthwise convolution with a dilation rate of 2, express Branch features after depthwise convolution with a dilation rate of 3; By cascading three multiple dilatational context modules (MDCBs), cascaded fused features are obtained. As shown in equation (18), next, for An adaptive average pooling operation is performed to compress the spatial dimension, thereby aggregating global spatial information to generate channel descriptors. These descriptors are then processed through an FFN structure, first reducing dimensionality to decrease computation, and then restoring them to the original channel dimension, as shown in the following equation: （17）； In the above formula: for Output features after adaptive average pooling; To reduce the weights of the fully connected layer, the number of channels is reduced from... Compress to ; This refers to the compression ratio; For activation functions; To increase the weights of the fully connected layer, the number of channels is increased from... Restore to ; Finally, the normalized channel weights are generated using the Sigmoid activation function and compared with the original input. Element-wise multiplication is performed to achieve adaptive recalibration of key channel features; as shown in equation (19), due to the input features of the cascaded multi-expansion context module (CMDC) It also needs to undergo a 1×1 convolution, and then... The features are added together and then fused through a 1×1 convolution to obtain the output features. : （18）； （19）； In the formula: To cascade three MDCBs and recalibrate branch characteristics. It is a feedforward neural network. This represents the Sigmoid function. This indicates element-wise multiplication. , For 1×1 convolution, For batch normalization; 2.3.2: Setting up the Adown module: The Adown (Attention-guided Downsample) module replaces the original downsampling step in the CCFF of RT-DETR. This module first divides the input feature map into two parts evenly in the channel dimension; it applies a 3×3 convolutional downsampling operation with a stride of 2 to the first part of the feature map; it applies a 2×2 average pooling operation to the second part of the feature map, and then adjusts the number of channels through a 1×1 convolution; finally, it concatenates and merges the feature maps processed by the above two branches in the channel dimension. 2.3.3: Constructing a cross-scale contextual feature fusion network: The three feature maps at different scales output by the backbone network are obtained as input to the cross-scale context-aware feature fusion network (Adaptive Context Feature Fusion, ACFF), and are labeled as shallow feature maps in descending order of spatial resolution. Mid-layer feature map Deep feature map The three feature maps have three independent data paths: Perform top-down feature fusion; first, process the deep feature map. Perform an upsampling operation to double its resolution, and then combine it with the mid-level feature map. The feature maps are concatenated along the channel dimension. The concatenated feature maps serve as the input to the first CMDC module. After processing by this module, the output is a medium-scale intermediate feature map with the same resolution as the input, denoted as . ; For intermediate feature maps at medium scale Perform an upsampling operation to double its resolution, and then compare it with the shallow feature map. The feature maps are concatenated along the channel dimension. The concatenated feature maps serve as input to the second CMDC module. After processing by this module, the output is a highest-resolution feature map that incorporates top-level semantic information, denoted as... The As the first output path of the network; Perform bottom-up feature fusion; combine feature maps As input to the Adown module's downsampling, a feature map with half the resolution is obtained after the downsampling operation; this downsampled feature map is then compared with an intermediate feature map at a medium scale. The data is concatenated along the channel dimension and then input into the third CMDC module. After processing by this module, a medium-resolution feature map is output, denoted as... The As the second output path of the network; feature map As input to the Adown module's downsampling, a feature map with its resolution halved is obtained after the downsampling operation; this downsampled feature map is then compared with the deep feature map. The data is concatenated along the channel dimension and input into the fourth CMDC module. After processing by this module, the lowest resolution feature map with the strongest global receptive field is output, denoted as... The As the third output path of the network; After the bidirectional feature fusion process described above, the ACFF structure finally outputs enhanced feature maps at three scales.

8. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 7, characterized in that: 2.4 includes: 2.4.1: Building the RT-DETR decoder; RT-DETR's decoder is based on the Deformable Transformer architecture, consisting of six cascaded decoder layers. Within each decoder layer, the query vector first interacts through a multi-head self-attention mechanism to capture contextual dependencies; where each attention head is computed as follows: （20）； In the above formula: Indicates the first One point of attention; Use the Softmax activation function; , , These are respectively query, key, and value vectors; For the first The projection matrix of each attention head; Feature dimensions for each head; This is a scaling factor to prevent the dot product result from becoming too large; Next, the multi-scale deformable cross-attention module is entered, using the center of the reference bounding box as the sampling reference point, and sparse sampling is performed on the multi-scale feature map output by the encoder; the multi-scale deformable cross-attention is shown in the following equation: （21）； In the above formula: For query vector; The coordinates of the center point of the reference bounding box are given, where The horizontal coordinates of the center point of the bounding box. The vertical coordinates of the center point of the bounding box; For feature layer-level indexing, a total of One scale; For the sampling point index, sampling is performed at each scale. One point; The learnable sampling offset predicted from the query vector; In the first Sampling features at the offset position on the layer feature map; Let be the attention weights for the corresponding sampling points, and satisfy . ; 2.4.2: Setting up KANs: KANs consist of a KAN Linear Layer, whose final output is obtained by adding the calculation results of the base branch and the spline branch. The base branch retains the structural features of traditional neural networks, and is responsible for extracting the global linear features of the data by applying the basic activation function to the input and performing a linear transformation in combination with the weight matrix. The spline branch first calculates the B-spline basis function values ​​of the input at the predefined spline grid nodes. The B-spline basis function is defined recursively: （22）； （23）； In the formula: For nodes Starting with the 0th order B-spline basis function, For the node sequence Nodes , For the node sequence 1 node For input variables; Represented by node Starting from B-order spline basis functions; For the node sequence 1 node For nodes Starting from B-order spline basis functions, For the node sequence 1 node For nodes Starting from B-order spline basis functions; The calculated basis function values ​​are then linearly combined with the learnable spline weight matrix. By optimizing these trainable weights, the network can fit a univariate function curve of arbitrary shape, thereby achieving local nonlinear refinement of the input, as shown in equation (24). （24）； In the formula: This represents the output features of the KAN linear layer; and These are the trainable weight matrices acting on the basic branch and the spline branch, respectively; the basic activation function is the SiLU function. , This represents the Sigmoid activation function; Let B-spline basis function vectors be used. Indicates the 0th node Starting from k B-order spline basis functions; Indicates the first n 1 node n Starting from k B-order spline basis functions; Indicates vector transpose; 2.4.3: Setting up KAN-Decoder: KAN is introduced into the DeformableTransformer prediction head of the RT-DETR decoder, replacing the original Multilayer Perceptron (MLP). In the iterative bounding box refinement strategy, the encoder bounding box head first uses KAN to generate initial reference boxes. As a starting point for optimization: （25）； In the above formula: For encoder KAN prediction head; For encoder output features; The initial reference box is represented as ,in: The horizontal coordinates of the center point of the bounding box. The vertical coordinates of the center point of the bounding box. The width of the bounding box. The height of the bounding box; Before each subsequent layer of decoding computation, the system uses the reference box of the previous layer to generate the position embedding of the current layer through the query position header constructed by MLP, and injects the latest geometric information into the query feature: （26）； In the above formula: Embed an MLP generator for the location; For the first The reference box output by the layer; For the injection of the first Location embedding of layer query features; The output features of the current layer are ultimately fed into the class score head and the bounding box regression head built based on KAN. This regression head utilizes the powerful non-linear mapping capability of KAN to calculate the corrected offset relative to the current reference box, enhancing the model's localization accuracy when handling small targets and occluded samples; that is: （27）； Iterative updates based on the current reference frame: （28）； In the above formula: For the first The output query features of the layer decoder; These are the relative correction offsets predicted by KAN, corresponding to the correction values ​​for the center point coordinates and width and height of the bounding box, respectively. This is the reference box output from the previous layer; This is the updated bounding box prediction result for the current layer.

9. The vehicle target detection method based on MKAN-DETR in complex scenarios according to claim 8, characterized in that: In step 3, during training, the vehicle target detection model MKAN-DETR uses a joint loss function composed of classification loss and bounding box regression loss. The classification loss constrains the accuracy of target category prediction, while the bounding box regression loss constrains the accuracy of target localization. The classification loss is calculated using FocalLoss and is defined as follows: （29）； In the formula: For the first Each match predicts the confidence probability of the true class; As a category balance factor; For focusing parameters; To accurately label the target quantity; The bounding box regression loss consists of two parts: L1 loss and GIoU loss, defined as follows: （30）； In the formula: For the first One true labeled bounding box; The predicted bounding box that matches it; and The balancing weighting coefficients for the two losses; L1 loss is calculated as the sum of the absolute differences between the four-dimensional coordinates of the predicted bounding box and the ground truth bounding box: （31）； GIoU loss introduces a minimum closed region penalty term on top of IoU, defined as: （32）； In the formula: This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. It is the smallest closed rectangular region that simultaneously contains both the predicted bounding box and the ground truth bounding box; This is the area within the smallest closed region that is not covered by any box, used to penalize cases where two boxes are too far apart; The overall training loss function of RT-DETR is a weighted sum of the classification loss and the bounding box regression loss: （33）； In the formula: and The loss weight coefficient is used to balance the contribution ratio of the two tasks, class prediction and bounding box localization, to model training. During training, the AdamW optimizer, combined with the backpropagation algorithm, continuously adjusts the weight parameters in the network, gradually reducing the calculated joint loss function. Backpropagation calculates the loss for each parameter using the chain rule. gradient: （34）； The AdamW parameter update rule is as follows: （35）； In the above formula: The gradient of the current parameters; This is a first-moment estimate of the gradient; This is the second moment estimate of the gradient; The learning rate; It is a numerically stable term; This is the weight decay coefficient; the second term The third term represents the adaptive gradient update amount. As a weight decay term, both factors jointly drive the parameter updates in the direction of loss reduction, making... Gradually decrease until convergence; After multiple rounds of iterative training, the model gradually converges, ultimately achieving the optimal object detection performance. After convergence, mAP is used as the comprehensive evaluation metric for the optimal detection performance. （36）； In the above formula: For the first The class has a recall rate of The corresponding precision; the average precision for each class is obtained by integrating the precision-recall curve. Then take the average value of all categories to get This serves as a comprehensive measure of the final detection performance.

10. The vehicle target detection method based on MKAN-DETR in complex scenes according to claim 9, characterized in that: Step 3 includes the following steps: To begin training the model, the training image containing the target to be detected is input into the vehicle target detection model MKAN-DETR. The image first passes through the RepViT backbone network to extract multi-scale features, then passes through the MDPF module to suppress background noise, and then passes through the ACFF structure to perform cross-scale context-aware feature fusion to obtain an enhanced feature map. The fused features are then fed into the Transformer encoder for global feature modeling, and finally the decoder generates the target's category prediction and bounding box coordinate prediction. Decoder Layer output query features through Generate confidence scores for each category: （37）； In the above formula: ,in To query the number of vectors, Total number of target categories; Map the output to An interval represents the confidence probability for each category; The predicted corrected offset relative to the reference box is iteratively updated to obtain the final bounding box: （38）； In the above formula: , and These consist of the center point coordinates and the four-dimensional parameters of width and height of the bounding box prediction result; This is the reference bounding box output from the previous layer; For the first The output query features of the layer decoder; go through After layer iterations, the output of the last layer is taken as the final prediction result: ；（39）； In the above formula: For the first Category prediction for each query; For the first Bounding box prediction for each query; This represents the number of query vectors. The network's prediction results are matched with the corresponding ground truth annotations. For each prediction-ground truth annotation pair... Calculate the matching cost: （40）； In the above formula: - Category cost, which is the predicted bounding box relative to the true category. The confidence level is negative; - L1 regression cost, a measure of the predicted bounding box With real frame The absolute difference of the coordinates; - GIoU cost, a measure of the predicted bounding box With real frame The degree of geometric overlap between them; For the generalized intersection-union-ratio, it is defined as: （41）； in: This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. It is the smallest closed rectangular region that simultaneously contains both the predicted bounding box and the ground truth bounding box; It represents the area within the smallest closed region that is not covered by any frame; - The balancing weight coefficients for each cost item; The optimal binary matching between the predicted bounding box and the ground truth bounding box is achieved using the Hungarian algorithm, and the classification loss and bounding box regression loss are calculated based on the matching results. Network parameters are updated using the backpropagation algorithm, based on the joint loss obtained from forward inference. The gradients of each parameter are calculated by backpropagating layer by layer using the chain rule: （43）； Update network parameters using the AdamW optimizer: （44）； The gradient starts from the loss layer and flows sequentially through the decoder → encoder → ACFF → MDPF → RepViT backbone, driving the parameters of each module to be updated in the direction of loss reduction. Repeat the iterative learning process of forward inference and parameter optimization described above until the model converges; Each iteration performs forward inference and parameter updates; the loss increases with the number of iterations. Monotonically decreasing: （45）； Training terminates when the convergence condition is met. （46）； In the above formula: This represents the optimal loss value after convergence. The convergence threshold; the model parameters after convergence. These are the optimal parameters, corresponding to the optimal target detection performance.

Images