Intelligent traffic target detection method and system based on large visual model and enhanced detection model

Abstract

The application discloses a smart traffic target detection method and system based on a large visual model and an enhanced detection model. The method comprises the following steps: inputting an input image into a pre-trained detection framework DAIR model, wherein the detection framework DAIR model comprises a double-branch backbone network, a structure perception feature adapter and a self-adaptive cross-source interaction routing module; a large visual model and an enhanced detection model in the double-branch backbone network are used to extract detection features and basic representations of the input image respectively; the structure perception feature adapter is used to convert the basic representations into a representation space compatible with a detection task, thereby generating aligned features; the self-adaptive cross-source interaction routing module is used to perform self-adaptive cross-source interaction routing between the aligned features and the detection features, thereby obtaining fusion representations; and the fusion representations are input into a detection head of an encoder-decoder of the enhanced detection model in the double-branch backbone network, so that target classification and bounding box regression are performed, and target prediction results are obtained.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation target detection technology, and mainly to an intelligent transportation target detection method and system based on a large visual model and an enhanced detection model. Background Technology

[0002] Currently, intelligent transportation detection systems generally employ deep learning-based object detection models to perceive and identify urban traffic elements such as vehicles and pedestrians. The mainstream detection frameworks can be divided into two main categories: one is based on Convolutional Neural Networks (CNNs), represented by the YOLO series, which achieves rapid detection through predefined anchor boxes and regression methods; the other is based on Transformer architectures, represented by the DETR (Detection Transformer) series, which utilizes attention mechanisms and query objects for end-to-end prediction. RT-DETR (Real-Time Detection Transformer) and its improved version RT-DETRv2, as advanced representatives, have achieved initial applications in the field of intelligent transportation due to their balance between real-time performance and high accuracy.

[0003] However, existing detection algorithms heavily rely on large-scale, high-quality labeled data for learning. In complex urban governance scenarios, relying solely on a single target detection model and limited data distribution makes it difficult to simultaneously guarantee high precision and high recall. Their technical bottlenecks mainly manifest in the following two categories: First, the feature extraction capability is insufficient. Traditional CNN or Transformer backbone networks are not sufficient in feature extraction when faced with complex lighting, partial occlusion, and drastic changes in target scale, which can easily lead to missed detection of small targets. At the same time, when the appearance features of different types of targets (such as pedestrians and cyclists) are similar, the model has difficulty capturing subtle discriminative features, which can lead to misclassification and reduce detection accuracy.

[0004] Secondly, the data suffers from a long-tail distribution and poor scene generalization. In intelligent transportation scenarios, there are abundant samples of common objects (such as cars), while samples of special or rare objects (such as construction vehicles, animals that have entered the area, and specific traffic signs) are scarce. This long-tail data problem makes it difficult for the model to learn robust scene-related features from limited samples, resulting in a significant drop in detection performance in vertical domains or zero-sample scenarios, failing to meet the reliability requirements of actual deployment.

[0005] Existing technologies suffer from insufficient target detection accuracy in complex intelligent transportation scenarios. Especially given the diverse needs of urban governance, reliable and accurate object recognition is crucial for improving the intelligence level of key areas such as traffic management and public safety.

[0006] In summary, existing intelligent traffic detection systems primarily rely on data-driven methods to train single or ensemble object detection models. While these methods have achieved good results on standard datasets, their core drawback lies in their failure to effectively utilize the generalized semantic priors of large-scale visual foundational models to enhance the feature representation capabilities of the detection models. Specifically: 1. Semantic Gap Problem: Existing models such as RT-DETR have limited parameter size in their backbone networks (such as convolutional or traditional Transformers). The features they learn are mainly geared towards the detection task itself, lacking general visual semantic knowledge obtained from larger-scale, unsupervised pre-training. This makes the models unable to "see clearly" or "distinguish clearly" when faced with out-of-domain samples or complex semantic scenes.

[0007] 2. Low feature fusion efficiency: Although some works have attempted to introduce additional features, they mostly use simple splicing or addition methods and have failed to design a dedicated module to bridge the distribution gap between "general semantic features" and "detection-specific features". This results in a conflict between the two during the optimization process, and the semantic prior is overwhelmed by the local localization requirements of the detection task, failing to leverage their complementary advantages.

[0008] 3. Insufficient adaptability: Existing models use a fixed feature fusion strategy for all input images, which cannot dynamically adjust the dependence ratio of semantic and spatial information according to the complexity of the scene (such as occlusion, density, long distance, etc.), resulting in poor robustness in complex and ever-changing urban scenes. Summary of the Invention

[0009] This invention overcomes the shortcomings of existing technologies and provides a smart transportation target detection method and system based on a large visual model and an enhanced detection model. It leverages the feature extraction and generalization capabilities of the large visual model to help the detection model focus on extracting and representing effective feature information of the detected object. By combining context-aware expert routing with residual semantic refinement, ACEF (Adaptive Cross-source Exchange Forwarding) effectively bridges the gap between semantic abstraction and localization sensitivity, achieving more efficient utilization of heterogeneous representations and significantly improving multi-scale target perception, occlusion robustness, and localization accuracy.

[0010] To achieve the above objectives, the technical solution adopted by this invention is: a smart traffic target detection method based on a large visual model and an enhanced detection model, comprising the following steps: Obtain the input image; The input image is fed into the pre-trained detection framework DAIR model, which includes a dual-branch backbone network, a structure-aware feature adapter, and an adaptive cross-source interactive routing module. The dual-branch backbone network includes a large vision model and an enhanced detection model. The detection features and basic representations of the input image are extracted by the large vision model and the enhanced detection model in the dual-branch backbone network, respectively. The basic representation is transformed into a representation space compatible with the detection task by using a structure-aware feature adapter, generating aligned features; The adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between aligned features and detected features to obtain a fused representation. The fused features are input into the encoder-decoder head of the enhanced detection model in the dual-branch backbone network to perform target classification and bounding box regression, thereby obtaining target prediction results.

[0011] In a preferred embodiment of the present invention, one branch of the dual-branch backbone network includes: using a pre-trained ViT of a large vision model pre-trained on a test dataset to extract a basic representation for semantic generalization, wherein the basic representation is: , ; d represents the basic features of the i-th layer of the large visual model; the superscript i indicates the network layer number, i.e., the index of the feature of the i-th layer, i∈[3,4,5], and the subscript d represents the original heterogeneous features, i.e., the basic features; I is the input image. The encoder representing the large visual model, This indicates that the input image is fed into the encoder of the large visual model for forward computation; Another branch of the dual-branch backbone network includes a backbone network employing an enhanced detection model to generate multi-scale detection features sensitive to localization. , , This represents the backbone network of the enhanced detection model; This means that the image is fed into the backbone network of the enhanced detection model for forward computation, and the output is the detection features; The subscript r represents the detection feature of the i-th layer-wise detection of the enhanced detection model.

[0012] In a preferred embodiment of the present invention, the structure-aware feature adapter includes: a linear calibration module, a nonlinear alignment module, and a spatial awareness alignment module; The linear calibration module is used to perform linear projection alignment, and includes a 3×3 convolutional layer and a synchronous batch normalization layer, as follows: , For the linearly calibrated features of the i-th layer, a unified embedding space is established to coordinate the feature distribution and for heterogeneous representations; where, It is a 3×3 convolution. For batch normalization; the subscript l indicates linear calibration characteristics; The nonlinear alignment module adds the GELU activation function to the linear calibration module. ,in, For GELU activation function, The i-th layer is the non-linear aligned feature; the subscript n indicates the non-linear aligned feature. The spatially aware alignment module adds depthwise convolution to the nonlinear alignment module to recover the spatial inductive bias, including: ,in, This represents a 3×3 depthwise convolution. represents the i-th layer of basic features after spatial awareness alignment; the subscript s indicates the spatial awareness aligned feature.

[0013] In a preferred embodiment of the present invention, the adaptive cross-source expert fusion module projects the aligned basic features and the detection features output by the enhanced detection model into a unified interaction space. A lightweight gating network is used to predict expert routing weights based on global context descriptors. A sparse Top-k routing mechanism is used to activate the most relevant experts to perform heterogeneous feature interaction. The selected expert outputs are aggregated by weighted summation and further combined with the original representation through residual semantic refinement to generate the final fused features.

[0014] In a preferred embodiment of the present invention, projecting onto a unified interactive space includes the following steps: The alignment of basic features and detection features is projected into a unified interaction space. The algorithm includes: , Lightweight projection bottlenecks are addressed to construct a unified interactive space. The algorithm includes: , in, This represents a transition function consisting of 1×1 convolutions. For splicing features, Features projected onto a unified interaction space; the superscript i indicates the network layer number; Indicates will and Direct channel splicing.

[0015] In a preferred embodiment of the present invention, context-aware routing includes: extracting data from the projected representation using global average pooling. The global context descriptor is extracted, and based on the global context descriptor, a lightweight gated network is used to estimate the expert route distribution. The algorithm includes: , The Softmax function is represented by GAP(·); GAP(·) represents adaptive global average pooling. Let W1 and W2 represent the gating score of the j-th expert under the i-th layer features; W1 and W2 represent the learnable weight matrices of the two fully connected network layers. A sparse Top-k routing strategy is used to activate only the most relevant experts: ,in This represents the set of experts selected at the i-th feature; the superscript i indicates the network layer number, and the subscript j indicates the index of the j-th expert; Topk represents the sparse Top-k routing function. This means selecting the top k experts with the highest weights from all expert weights; k represents the preset hyperparameter, meaning that only k experts are activated each time.

[0016] In a preferred embodiment of the present invention, expert interactive learning includes: Each expert network is implemented as a lightweight bottleneck structure, and the algorithm includes: , Only call the respective conversion functions The Top-k experts are used to refine the features. Experts' enhanced statement The result is obtained by weighted summation of the activated experts: ;in, For the features of the j-th expert under the i-th layer of features, This represents the set of activated experts obtained from the Top-k selection in the previous step; This indicates the fusion features enhanced by the expert; This represents the output feature of the j-th expert unit at the i-th scale, where the input feature x is fed in.

[0017] In a preferred embodiment of the present invention, residual semantic refinement includes: The expert-adjusted features are then reintegrated with the original projective embeddings. The algorithm is as follows: , This represents the final fused representation passed to the subsequent Transformer encoder-decoder architecture.

[0018] In a preferred embodiment of the present invention, the establishment and training of the detection framework DAIR model includes: Collect images of targets for intelligent transportation detection and label the detection boxes and corresponding categories; A detection framework DAIR model is constructed, and a structure-aware feature adapter and an adaptive cross-source interactive routing module are established. The backbone features of the large visual model are embedded into the enhanced detection model as a dual-branch backbone network. The main structure of the large visual model of the DAIR detection framework is frozen, and the collected data is cross-trained to obtain a smart traffic detection task scenario model and obtain weights. This enables pre-training of the model; The trained model Deployed to the intelligent transportation cloud server.

[0019] In a preferred embodiment of the present invention, an intelligent traffic target detection system based on a large visual model and an augmented detection model includes: The data acquisition module uses a camera to capture input images; The data processing module is used to input the input image into a pre-trained detection framework DAIR model. The DAIR model includes a two-branch backbone network, a structure-aware feature adapter, and an adaptive cross-source interactive routing module. The two-branch backbone network includes a large vision model and an enhanced detection model. Detection features and basic representations of the input image are extracted by the large vision model and the enhanced detection model in the two-branch backbone network, respectively. The structure-aware feature adapter transforms the basic representations into a representation space compatible with the detection task, generating aligned features. The adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between the aligned features and the detection features to obtain a fused representation. The fused features are then input into the encoder-decoder detection head of the enhanced detection model in the two-branch backbone network for target classification and bounding box regression to obtain target prediction results. The data output module is used to output the target prediction results.

[0020] This invention addresses the deficiencies in the technical background, and the beneficial technical effects of this invention are: This invention leverages the generalization advantages of the large visual model (DINOv3) to enhance the performance of the detection model (RT-DETRv2). It sets up a lightweight structure-aware feature adapter to effectively align the base model with detection features while preserving semantic priors. An adaptive cross-source expert fusion module is constructed to mine complementary semantic and spatial information between heterogeneous data sources through a dynamic routing mechanism. By introducing the feature extraction capabilities of the large visual model into the detection framework, accurate and robust city-level target detection tasks are achieved in the cloud computing environment of smart cities.

[0021] 1. A Decoupled Alignment and Interactive Routing (DAIR) framework is proposed to integrate the visual base model into a Transformer-based object detector. DAIR establishes a principled representation bridging paradigm, effectively mitigating conflicts caused by joint optimization by decoupling the "representation alignment" and "interactive routing" processes. This enables the semantic priors of the base model (DINOv3) to be efficiently and stably transferred to the feature space of the detection task, laying the foundation for subsequent modules.

[0022] 2. A Structure-Aware Feature Adapter (SAFA) is proposed, a lightweight spatial perception representation alignment module. While preserving the transferable semantic priors of large-scale visual models, SAFA effectively bridges the distribution gap between general-purpose base model representations and features for detection tasks by introducing a spatial perception mechanism consisting of linear calibration, nonlinear activation, and deep convolution. It significantly improves the model's feature representation ability in complex urban scenes with occlusion and dramatic scale changes, while maintaining extremely low computational overhead.

[0023] 3. An Adaptive Cross-Source Expert Fusion (ACEF) module was introduced, redefining the heterogeneous feature fusion problem as a dynamic interactive routing task. The ACEF module abandons static concatenation or attention fusion strategies, employing a context-aware sparse Top-k routing mechanism. This allows the model to adaptively select the most relevant "expert" branch for feature interaction based on the specific features of the input scene (such as density and occlusion). The ACEF module achieves selective and efficient utilization of DINOv3 semantic information and RT-DETRv2 localization information, maximizing feature fusion performance while ensuring computational efficiency. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Figure 1 This is a schematic diagram of the structure of the intelligent transportation target detection method based on a large visual model and an enhanced detection model in a preferred embodiment of the present invention (including a dual-branch backbone network, SAFA (Structured Aware Feature Adapter), and ACEF (Adaptive Cross-Source Interactive Routing) module).

[0026] Figure 2 This is a structural perception feature adapter in the intelligent transportation target detection method based on a large visual model and an enhanced detection model, as described in a preferred embodiment of the present invention.

[0027] Figure 3This is a schematic diagram of the ACEF structure in the intelligent transportation target detection method based on a large vision model and an enhanced detection model in a preferred embodiment of the present invention; Figure 4 This is a qualitative comparison of RT-DETRv2 and the DAIR method in Example 1 in various visual scenes on the COCO dataset with different detector sizes (s, m, l, x). Figure 5 This is a comparison of Grad-CAM heatmaps of RT-DETRv2 and DAIR on the COCO dataset; Figure 6 This is a qualitative comparison of RT-DETRv2 and DAIR under complex traffic scenarios on the KITTI dataset with different detector sizes (s, m, l, x). Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention and therefore only show the components relevant to the present invention.

[0029] It should be noted that if directional indicators (such as up, down, bottom, top, etc.) are involved in the embodiments of the present invention, these directional indicators are only used to explain the relative positional relationship and movement of the components in a specific posture. If the specific posture changes, the directional indicators will also change accordingly. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of that feature. Unless otherwise explicitly specified and limited, the terms "set," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances.

[0030] Example 1: A smart traffic target detection method based on a large visual model and an augmented detection model, comprising the following steps: Obtain the input image; The input image is fed into the pre-trained detection framework DAIR model, which includes a dual-branch backbone network, a structure-aware feature adapter (SAFA), and an adaptive cross-source interactive routing module (ACEF). The dual-branch backbone network includes a large vision model and an enhanced detection model. The detection features and basic representations of the input image are extracted by the large vision model (DINOv3) and the augmented detection model (RT-DETRv2) in the dual-branch backbone network, respectively. The basic representation is transformed into a representation space compatible with the detection task by using a structure-aware feature adapter, generating aligned features; The adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between aligned features and detected features to obtain a fused representation. The fused features are input into the encoder-decoder head of the enhanced detection model in the dual-branch backbone network to perform target classification and bounding box regression, thereby obtaining target prediction results.

[0031] Specifically, one branch of the dual-branch backbone network includes: using a pre-trained ViT on a large vision model pre-trained on the test dataset to extract the basic representation for semantic generalization, which is: , ; I represents the basic features of the i-th layer of the large visual model; I is the input image. The encoder representing the large visual model, This indicates that the input image is fed into the encoder of the large visual model for forward computation; the other branch of the dual-branch backbone network includes: a backbone network employing an enhanced detection model to generate multi-scale detection features sensitive to localization, the detection features being... , , This represents the backbone network of the enhanced detection model. The detection features of the i-th layer of the enhanced detection model are extracted.

[0032] Furthermore, the large-vision model and the augmented detection model of the dual-branch backbone network extract generalizable semantic knowledge from large-scale self-supervised pre-training and dataset-specific detection representations learned from downstream object detection tasks, respectively, to provide heterogeneous but complementary representations for bridging and interactive learning. DAIR does not directly aggregate these heterogeneous representations but retains their domain-specific features and performs subsequent decoupled representation matching and interactive learning; this avoids premature semantic interference between heterogeneous representation spaces and lays a stable foundation for subsequent cross-source integration.

[0033] Figure 1 In the diagram, flames are marked as trainable / fine-tunable branches; snowflakes are marked as frozen pre-trained weight branches. The three-layer multi-scale detection features output by the enhanced detection model of the dual-branch backbone network are... , , Where b is the batch size, c is the number of channels, h is the feature map height, w is the feature map width, and F... r The features are the detection features, i.e. the features output by the enhanced detection model (RT-DETRv2). The superscripts 3, 4, and 5 represent multi-scale features, indicating 3 high-resolution fine-grained features and 5 low-resolution high-level semantic features. F d The basic representation, i.e., the features output by the large visual model (DINOv3); F s The aligned features are the output of SAFA. The backbone weights of the large visual model are frozen and do not participate in gradient updates; they are only used for feature extraction.

[0034] Specifically, such as Figure 2 As shown, the structure-aware feature adapter includes: a linear calibration module, a nonlinear alignment module, and a spatial awareness alignment module; The linear calibration module performs linear projection alignment and consists of a 3×3 convolutional layer and a SyncBN (Synchronous Batch Normalization) layer, as follows: This coordinates feature distribution and establishes a unified embedding space for heterogeneous representations; where, It is a 3×3 convolution. For batch normalization; linear calibration module such as Figure 2 As shown in Figure (a); The nonlinear alignment module adds the GELU activation function to the linear calibration module. ,in, For GELU activation function; non-linear alignment module such as Figure 2 As shown in Figure (b); The spatially aware alignment module adds depthwise convolution to the nonlinear alignment module to recover the spatial inductive bias, including: ,in, Represents a 3×3 depthwise convolution; spatially aware alignment module, such as Figure 2 As shown in Figure (c).

[0035] Furthermore, the linear calibration module coordinates the feature distribution and establishes a unified embedding space for heterogeneous representations. The structure-aware feature adapter employs deep convolution as a lightweight structure adapter; its inherent locality bias complements the globally abstracted basic representation by emphasizing object-aware spatial patterns, thereby reducing the discrepancy between basic features and detector-oriented representations with minimal computational overhead.

[0036] Figure 1 In the structure-aware feature adapter, each scale has its own independent SAFA module. Each feature is fed into its own structure-aware feature adapter, not a shared single module. The channels and spatial dimensions of the DINOv3 output features are aligned to the corresponding dimensions c1, c2, c3 of RT-DETRv2. Spatial induction bias is supplemented through depthwise convolution to compensate for the lack of spatial awareness in DINOv3 for detection tasks. The aligned features are then output. Dimensions and RT-DETRv2 One-to-one correspondence, and then sent to the ACEF fusion module.

[0037] Furthermore, Figure 1 The diagram illustrates the overall architecture of the dual-branch feature extraction network; the input image is fed into the RT-DETRv2 backbone network and the DINOv3 backbone network respectively; the RT-DETRv2 backbone outputs multi-scale detection features. DINOv3 backbone output multi-scale semantic features The DINOv3 backbone parameters are frozen; the output features of DINOv3 at each scale are input into independent SAFA structure-aware feature adapters. Within the SAFA, features are sequentially processed through 1×1 convolution, depthwise separable convolution, normalization, and GELU activation, outputting dimensionally aligned features. The flame icon indicates that network parameters are involved in training and updating, while the snowflake icon indicates that network parameters are frozen.

[0038] Specifically, the adaptive cross-source expert fusion module projects aligned base features and detection features output by the enhanced detection model into a unified interaction space (i.e., Unified Feature Space). This maps heterogeneous features from two different sources to the same dimensional space, eliminating feature distribution differences and preparing for subsequent routing. A lightweight gating network predicts expert routing weights based on global context description. A sparse Top-k routing mechanism activates the most relevant experts for heterogeneous feature interaction. The selected expert outputs are aggregated through weighted summation and further combined with the original representation through residual semantic refinement to generate the final fused features. Furthermore, the channels are 384, h1, w. 1； h1 and w1 are the output height and width of the large visual model. A single bilinear interpolation transforms 384, h1, w1 into 384, h2, w2. Then, a Structure-Aware Feature Adapter (SAFA) is applied. The output channels after SAFA are 256, h2, w2. h2 and w2 are the output height and width of RT-DETRv2, representing the channel dimensions after Conv1×1 projection in the adaptive cross-source expert fusion module. The adaptive cross-source expert fusion module previously performed cat, resulting in 2*256; after Conv1*1, it becomes 256, h2, w2. The channel dimensions within each expert are all 256.

[0039] Projecting onto a unified interactive space includes the following steps: The alignment of basic features and detection features is projected into a unified interaction space. The algorithm includes: , Lightweight projection bottlenecks are addressed to construct a unified interactive space. The algorithm includes: The detection features corresponding to the i-th layer Alignment semantic features corresponding to the i-th layer By concatenating the features along the channel dimension, a unified feature space of the projection layer output is obtained; among them, This represents a transition function consisting of 1×1 convolutions. Features after splicing Synchronized Batch Normalization is the normalization of synchronization statistics across multiple GPUs in a multi-GPU training scenario.

[0040] Specifically, the projection stage reduces the distribution inconsistency among heterogeneous feature sources and establishes a stable representation manifold for subsequent expert interactive learning.

[0041] Context-aware routing includes: from the projected representation using global average pooling. The global context descriptor is extracted, and based on the global context descriptor, a lightweight gated network is used to estimate the expert route distribution. The algorithm includes: Where, the superscript i represents the network layer number, i.e., the index of the feature of the i-th layer (multi-scale index); j is the expert number; Let be the routing weight of the j-th expert at the i-th scale; Features projected onto a unified interaction space are the unified feature space features output by the Projection layer; GAP stands for adaptive global average pooling, which compresses the H×W feature map into a one-dimensional vector. Learnable weight matrix of a two-layer fully connected network; This represents the Softmax function, which normalizes all expert outputs to obtain the probability distribution weights of each expert. The Gaussian Error Linear Unit (GELU) activation function is used. The gated structure consists of a 1×1 convolution, SyncBN normalization, and the GELU activation function.

[0042] The workflow is as follows: Projection features Global average pooling (GAP) is performed to compress the two-dimensional feature map into a one-dimensional feature vector. This vector is then linearly transformed using the first layer of learnable weights W1, followed by a non-linear transformation using the GELU activation function. After a second layer of linear transformation using learnable weights W2, the output score vector corresponds to the number of experts. Finally, the vector is normalized using the Softmax function to obtain the routing weights for each expert at the i-th scale. Perform Top-K filtering, retain the K experts with the highest weights, and reset the weights of the remaining experts to 0, thus achieving sparse activation MoE (i.e., expert sparse activation).

[0043] Among them, a sparse Top-k routing strategy is adopted, which activates only the most relevant experts: ,in This represents the set of experts selected at the i-th feature.

[0044] In a specific embodiment of the present invention, in order to maximize the computational efficiency of the system and meet the real-time requirements of intelligent transportation scenarios while ensuring sufficient interaction between heterogeneous features (semantic features of the large visual model and spatial features of the enhanced detection model), the value of k in the sparse Top-k routing strategy is 2 (i.e., k=2). Specifically, at each feature scale, the lightweight gating network outputs the original routing weights of all candidate expert networks. When performing the Top-2 selection operation, the system selects only the two experts with the largest routing weights (e.g., an expert focusing on global semantic reasoning and an expert focusing on local spatial refinement) to form the active expert set T, and forces the routing weights of the remaining unselected experts to 0. Subsequently, the routing weights of these two activated experts are optionally renormalized (so that their sum is 1), and forward computation and feature refinement are performed only on these two expert units. By setting k=2, the DAIR model in this embodiment can achieve flexible dynamic combination and complementary advantages between two different interaction modes: "semantic abstraction" and "location sensitivity". This setup avoids the limitations in information representation and routing oscillations that may result from using a single expert (k=1), while also effectively preventing computational redundancy and parameter waste caused by activating too many experts (e.g., k≥3). Thus, it achieves the best balance between detection accuracy and inference speed in complex urban traffic scenarios (e.g., dense occlusion, distant small targets).

[0045] Expert interactive learning includes: Each expert is implemented as a lightweight bottleneck structure, and the algorithm includes: , , To represent a 1×1 convolution, only the respective transformation functions are called. The Top-k experts are used to refine the features. Experts' enhanced statement The result is obtained by weighted summation of the activated experts: ,in, This represents the fusion feature after expert enhancement, specifically the feature output by the MoE expert modules after fusion at the i-th scale. For the features of the j-th expert under the i-th layer of features, This represents the set of activated experts obtained from the Top-k selection in the previous step. Iterate through each selected and activated expert. The expert route weight at the i-th scale after Top-K masking This represents the output feature of the j-th expert unit at the i-th scale, where the input feature x is fed in.

[0046] The original routing weights of each expert output by the gated network Perform the Top-K selection operation: select the K experts with the largest route weights to form the active expert set. ; will not be in the set The routing rights corresponding to the experts within the set are reset to 0; optionally, the routing rights for the set are also reset. The routing weights of the K experts retained within the pool are renormalized so that the sum of the weights of the activated experts equals 1; only the activated set is activated. The expert units within the pool perform forward computation, while unselected experts do not participate in the computation; subsequently, the outputs of the activated experts are weighted and summed according to a formula to obtain the expert fusion output features. .

[0047] Furthermore, the adaptive routing mechanism enables the model to dynamically emphasize different interaction modes based on scene features. For example, in complex contexts, experts focused on semantic reasoning may receive greater routing weights; while for small targets or severely occluded instances, experts emphasizing spatial refinement can be prioritized for activation. Compared to fusion mechanisms based on dense attention, expert routing promotes representation specialization and adaptive computation while avoiding unnecessary interactions between heterogeneous features.

[0048] Among them, residual semantic refinement includes: The expert-adjusted features are then reintegrated with the original projective embeddings. The algorithm is as follows: , This represents the final fused representation passed to the subsequent Transformer encoder-decoder architecture. Furthermore, the residual formula ensures that the detector backbone remains the primary source of localization information, while the base representation provides complementary semantic guidance. This design improves optimization stability and reduces the risk of semantic over-dominance common in direct fusion strategies.

[0049] Specifically, ACEF reformulates feature fusion as adaptive heterogeneous interactive learning, rather than static feature aggregation. By combining context-aware expert routing with residual semantic refinement, ACEF effectively bridges the gap between semantic abstraction and localization sensitivity, enabling more efficient utilization of heterogeneous representations and significantly improving multi-scale object perception, occlusion robustness, and localization accuracy. From a representation learning perspective, SAFA and ACEF address two fundamentally different challenges in VFM (Vision Base Model) augmented object detection: SAFA minimizes cross-domain representation differences, while ACEF maximizes complementary information exchange. Their decoupling optimizations allow DAIR to simultaneously maintain semantic transferability and localization sensitivity, thereby establishing an effective representation bridging mechanism between the base model and the object detector.

[0050] Example 2, based on Example 1, the establishment and training of the DAIR detection framework model includes: Images of targets for intelligent transportation detection are collected, and detection frames and corresponding categories are labeled. Specifically, in this embodiment, a 2-megapixel (1920×1080) high-definition camera, based on existing technology, is used. The distance between the detection area and the camera is less than 10 meters and greater than 1 meter. The camera is IPX6 waterproof. Input images are collected.

[0051] A detection framework DAIR model is constructed, and a structure-aware feature adapter and an adaptive cross-source interactive routing module are established.

[0052] The backbone features of the large visual model are embedded into the enhanced detection model as a two-branch backbone network.

[0053] The main structure of the large visual model of the DAIR detection framework is frozen, and the collected data is cross-trained to obtain a smart traffic detection task scenario model and obtain weights. This enables the pre-training of the model.

[0054] The trained model The system is deployed to a smart transportation cloud server. Specifically, in this embodiment, the smart transportation cloud server uses an NVIDIA RTX 409 server with 24GB of video memory and at least 32GB of RAM.

[0055] Example 3: A smart traffic target detection system based on a large visual model and an augmented detection model, comprising: The data acquisition module uses a camera to capture input images; The data processing module is used to input the input image into a pre-trained detection framework DAIR model. The DAIR model includes a two-branch backbone network, a structure-aware feature adapter, and an adaptive cross-source interactive routing module. The two-branch backbone network includes a large vision model and an enhanced detection model. Detection features and basic representations of the input image are extracted by the large vision model and the enhanced detection model in the two-branch backbone network, respectively. The structure-aware feature adapter transforms the basic representations into a representation space compatible with the detection task, generating aligned features. The adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between the aligned features and the detection features to obtain a fused representation. The fused features are then input into the encoder-decoder detection head of the enhanced detection model in the two-branch backbone network for target classification and bounding box regression to obtain target prediction results. The data output module is used to output the target prediction results.

[0056] The steps are for implementing a smart transportation target detection method based on a large visual model and an enhanced detection model in Embodiment 1.

[0057] Example 4, based on Example 1, Figure 4 The results show qualitative comparisons of RT-DETRv2 and the DAIR method in Example 1 in various visual scenes on the MSCOCO dataset with different detector sizes (s, m, l, x). The DAIR algorithm can output detection results with more accurate spatial localization and more reliable semantics, and this improvement remains stable under different model parameter scales.

[0058] In simple, single-target scenarios (such as birds flying in a forest), DAIR generates bounding boxes with higher fit and more reliable confidence estimates. This is evidenced by the improvement in target confidence from 0.91 to 0.95 using the s-scale model. The better matching between the algorithm's predicted bounding boxes and the target object's outline indicates that the transferred base feature representation improves target perception accuracy, rather than simply increasing classification confidence.

[0059] In complex scenarios with occlusion and multi-target interaction (such as giraffes), DAIR achieves a better balance between confidence and localization accuracy: in m-scale models, although the confidence of some occluded targets decreases (0.84 vs. 0.95), the predicted bounding boxes can more accurately select the visible areas of the targets, avoiding the problem of high confidence but inaccurate spatial localization. As the model size increases, the performance gap between the two methods narrows; even so, DAIR still has more stable instance-level detection performance, demonstrating enhanced robustness of feature representation learning.

[0060] The advantages of DAIR in Example 1 are particularly evident in densely packed small target scenes (such as distant birds). This method not only achieved competitive confidence scores (e.g., 0.42 vs. 0.38; 0.40 vs. 0.38), but also successfully detected distant target instances that RT-DETRv2 completely missed, demonstrating its higher sensitivity to fine-grained visual cues and improved detection completeness. Although quantitative results show that the method's performance improvement on medium and large targets is greater than that on small targets, visualization results indicate that DAIR has better recall and perceived quality in densely distributed small target areas. This suggests that quantitative metrics may underestimate the actual effectiveness of this method in such challenging scenes.

[0061] In summary, the above phenomena indicate that the performance improvement of DAIR does not stem from an increase in the number of model parameters, but rather from more efficient feature representation learning. Specifically, the framework proposed in this invention decouples the Structure-Aware Alignment Module (SAFA) from the Adaptive Cross-Source Interaction Module (ACEF), establishing a connection between semantic abstraction and spatial structure, thereby achieving more robust, accurate, and complete object detection.

[0062] Figure 4 In the image, the prediction results of RT-DETRv2 and DAIR on the COCO dataset are visualized. Orange bounding boxes represent DAIR's predictions, and blue bounding boxes represent RT-DETRv2's predictions. The labels s, m, l, and x represent different model sizes. To further evaluate DAIR's feature representation learning ability, Figure 5 Grad-CAM heatmaps of RT-DETRv2 and DAIR were compared at different detector scales (s, m, l, x). Unlike the qualitative results of detection, this visualization can show the spatial distribution and organization of feature responses, explaining the performance differences between the two on the COCO dataset from the feature representation level. Furthermore, RT-DETRv2 and DAIR can display the confidence scores of the corresponding detected targets at the bounding boxes of the prediction results on the COCO dataset, according to actual usage requirements.

[0063] First, the DAIR of this invention can effectively suppress background interference. For example... Figure 5 As shown, in a partially occluded bird scene (column 1), the activation region of RT-DETRv2 extends beyond the target boundary into the background area; while the activation range of DAIR is limited to the visible area of ​​the target. A similar phenomenon is observed in a dense flock of sheep scene (column 11): RT-DETRv2 produces scattered feature responses in irrelevant areas, while the activation region of DAIR is more concentrated. This indicates that DAIR reduces activation leakage and improves the ability to distinguish between foreground and background.

[0064] Secondly, DAIR possesses stronger goal-oriented attention and instance-level discrimination capabilities. For example... Figure 5 As shown, in the occluded giraffe scene (column 6), RT-DETRv2 exhibits overlapping activation regions on adjacent target instances, while DAIR achieves clearer instance segmentation. This advantage is further demonstrated in the dense sheep scene (column 11): DAIR retains continuous feature responses centered on the target, while RT-DETRv2 displays a fragmented response pattern. These results indicate that under conditions of occlusion and dense target arrangement, DAIR improves structural consistency and instance-level feature representation capabilities.

[0065] Furthermore, DAIR demonstrates significant advantages in highly complex visual scenarios. For example... Figure 5 As shown, whether in occluded scenes (columns 1 and 6) or dense target scenes (column 11), the activation map generated by DAIR has a more regular structure, a higher degree of alignment with the target, and less noise; while RT-DETRv2 is still easily affected by background clutter and other instances. This indicates that DAIR has superior robustness in handling complex real-world scenarios such as occlusion and dense target distribution.

[0066] Overall, Grad-CAM visualization results show that, compared to RT-DETRv2, DAIR can generate more target-focused feature responses, suppress background interference, and improve instance-level discrimination.

[0067] Figure 5 This section compares the Grad-CAM heatmaps of RT-DETRv2 and DAIR on the COCO dataset. Note: Columns 1, 6, and 11 are the ground truth images (Gt); columns 2, 7, and 12 are the Grad-CAM heatmaps generated by DAIR; columns 3, 8, and 13 are the fused visualizations obtained by overlaying the DAIR Grad-CAM heatmaps onto the ground truth images. Similarly, columns 4, 9, and 14 are the Grad-CAM heatmaps generated by RT-DETRv2; columns 5, 10, and 15 are the fused visualizations obtained by overlaying the RT-DETRv2 Grad-CAM heatmaps with the ground truth images. The labels s, m, l, and x represent different model sizes.

[0068] Figure 6 This presents a qualitative comparison of RT-DETRv2 and DAIR with different detector sizes (s, m, l, x) in complex traffic scenarios on the KITTI dataset.

[0069] like Figure 6 As shown, in long-distance scenes (i.e.) Figure 6In the left column, DAIR improves the confidence and detection reliability of distant targets: the confidence of a cyclist in the s-scale model increases from 0.80 to 0.84, and in the x-scale model from 0.83 to 0.85. It also slightly optimizes the localization of nearby vehicles (e.g., from 0.93 to 0.94). Simultaneously, this method effectively suppresses false detections caused by the background in RT-DETRv2, indicating that the model's ability to distinguish between foreground and background and its long-distance perception capabilities are both improved.

[0070] like Figure 6 As shown, in scenarios where targets are densely packed and occluded (i.e. Figure 6 In the middle column (of the model), DAIR achieved more stable and complete detection results. For a partially occluded pedestrian, the confidence score of the m-scale model increased from 0.39 to 0.57, while the x-scale model successfully detected one missed target instance (confidence score 0.43). Although the confidence score for nearby cyclists decreased slightly (0.90 vs. 0.87 in the l-scale model), DAIR was able to detect more distant targets and had better localization consistency, demonstrating a more balanced ability to represent scene features.

[0071] like Figure 6 As shown, in scenes with heavy occlusion and shadow interference (i.e. Figure 6 In the right column, DAIR's robustness is further improved. The confidence level for distant targets is significantly improved (e.g., l-scale model: 0.82 to 0.88; x-scale model: 0.61 to 0.84), while recovering several missed distant targets (e.g., a target with a confidence level of 0.43 in the m-scale model). Occluded or shadowed vehicles also show improved performance (s-scale model: 0.72 to 0.82; l-scale model: 0.67 to 0.81). Although the confidence level for a certain target decreases slightly in the x-scale model (from 0.70 to 0.67), DAIR provides more accurate localization results, indicating that this method achieves a better trade-off between confidence and localization accuracy.

[0072] Overall, in complex driving scenarios, DAIR continuously improves long-distance perception, dense target discrimination, and robustness against occlusion. These results confirm that the SAFA and ACEF modules proposed in this invention effectively link semantic and spatial features, enabling the model to achieve more reliable detection in real-world environments.

[0073] Figure 6 Visualize the prediction results of RT-DETRv2 and DAIR on the KITTI dataset. The label "o" represents the original input image; the labels s, m, l, and x correspond to different detector sizes. Figure 6In the diagram, the orange bounding box represents the prediction result output by DAIR, and the blue bounding box represents the prediction result output by RT-DETRv2. However, this is not the only limitation; depending on the actual usage requirements, the confidence levels of the corresponding detected targets by DAIR and RT-DETRv2 can be displayed at the bounding boxes.

[0074] Working principle: like Figure 1 As shown, the framework consists of three components: a dual-branch backbone network, SAFA (Structure-Aware Feature Adapter), and ACEF (Adaptive Cross-Source Interactive Routing). Given an input image, the DINOv3 branch extracts basic representations, while the RT-DETRv2 backbone network generates proprietary detection features. Subsequently, SAFA transforms the detection features into a detection-compatible representation space, generating aligned features. Based on the aligned representations, ACEF further performs adaptive cross-source interactive routing between the basic representations and the aligned features, generating a fused representation Z for use in the subsequent Transformer encoder-decoder detection pipeline. Unlike existing methods that jointly optimize feature fusion during the unification process, DAIR decouples representation alignment from interactive routing, thereby mitigating optimization conflicts and enabling more efficient transfer of basic semantic priors to the detection-oriented feature space.

[0075] like Figure 2 As shown, it transfers fundamental semantic priors to the feature space oriented towards the detector through structure-aware alignment. Specifically, SAFA integrates linear, spatial, and channel-aware modules to explore different adaptation strategies and identify effective mechanisms for aligning heterogeneous representations.

[0076] like Figure 3 As shown, the Adaptive Cross-Source Expert Fusion (ACEF) module reformulates heterogeneous feature fusion as an adaptive interactive routing problem. ACEF dynamically routes information between the base representation and the detector-oriented representation through context-aware expert selection, thereby achieving adaptive utilization of complementary cross-source knowledge.

[0077] This invention does not simply stack models or improve post-processing of predicted boxes, but rather innovatively proposes a Decoupled Alignment and Interactive Routing (DAIR) framework, starting from the source of feature representation. The core of this framework lies in the first-ever deep coupling of a large vision model with strong generalization capabilities (DINOv3) with a high-performance real-time detector (RT-DETRv2).

[0078] Specifically, this invention designs a Structure-Aware Feature Adapter (SAFA) to effectively align the rich semantic priors of DINOv3 with the features of the detection task while preserving them. Furthermore, it uses an Adaptive Cross-Source Expert Fusion Module (ACEF) to dynamically and scene-adaptively mine complementary information between the two. This solution fundamentally solves the problems of insufficient feature extraction, missing semantic priors, and conflicts in heterogeneous feature fusion in existing technologies, resulting in bottlenecks in classification accuracy and weak generalization ability. While maintaining real-time performance, it significantly improves the detection performance for occluded, dense, long-distance, and long-tailed targets, providing accurate and reliable visual analysis capabilities for efficient intelligent transportation supervision.

[0079] The above specific embodiments are specific support for the concept proposed in this invention, and should not be used to limit the scope of protection of this invention. Any equivalent changes or modifications made on the basis of this technical solution in accordance with the technical concept proposed in this invention shall still fall within the scope of protection of this invention.

Claims

1. A smart traffic target detection method based on a large visual model and an augmented detection model, characterized in that, Includes the following steps: Obtain the input image. The input image is fed into the pre-trained detection framework DAIR model, which includes a dual-branch backbone network, a structure-aware feature adapter, and an adaptive cross-source interactive routing module. The dual-branch backbone network includes a large vision model and an enhanced detection model. The detection features and basic representations of the input image are extracted by the large vision model and the enhanced detection model in the dual-branch backbone network, respectively. The basic representation is transformed into a representation space compatible with the detection task by using a structure-aware feature adapter, generating aligned features; The adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between aligned features and detected features to obtain a fused representation. The fused features are input into the encoder-decoder head of the enhanced detection model in the dual-branch backbone network to perform target classification and bounding box regression, thereby obtaining target prediction results.

2. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 1, characterized in that: One branch of the dual-branch backbone network includes: using a pre-trained ViT on a large vision model pre-trained on the test dataset to extract the base representation for semantic generalization, which is: , ; d represents the basic features of the i-th layer of the large visual model, i.e., the original heterogeneous features of the input; the superscript i indicates the network layer number, i.e., the index of the feature of the i-th layer, i.e., the superscript i∈[3,4,5], and the subscript d represents the original heterogeneous features, i.e., the basic features; I is the input image. The encoder representing the large visual model, This indicates that the input image is fed into the encoder of the large visual model for forward computation; Another branch of the dual-branch backbone network includes a backbone network employing an enhanced detection model to generate multi-scale detection features sensitive to localization. , , This represents the backbone network of the enhanced detection model; This means that the image is fed into the backbone network of the enhanced detection model for forward computation, and the output is the detection features; The subscript r represents the detection feature of the i-th layer-wise detection of the enhanced detection model.

3. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 2, characterized in that: The structure-aware feature adapter includes: a linear calibration module, a nonlinear alignment module, and a spatially-aware alignment module; The linear calibration module is used to perform linear projection alignment, and includes a 3×3 convolutional layer and a synchronous batch normalization layer, as follows: , For the linearly calibrated features of the i-th layer, a unified embedding space is established to coordinate the feature distribution and for heterogeneous representations; where, It is a 3×3 convolution. For batch normalization; the subscript l indicates linear calibration characteristics; The nonlinear alignment module adds the GELU activation function to the linear calibration module. ,in, For GELU activation function, The i-th layer is the non-linear aligned feature; the subscript n indicates the non-linear aligned feature. The spatially aware alignment module adds depthwise convolution to the nonlinear alignment module to recover the spatial inductive bias, including: ,in, This represents a 3×3 depthwise convolution. represents the i-th layer of basic features after spatial awareness alignment; the subscript s indicates the spatial awareness aligned feature.

4. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 3, characterized in that: The adaptive cross-source expert fusion module projects the aligned basic features and the detection features output by the enhanced detection model into a unified interaction space. It uses a lightweight gating network to predict expert routing weights based on global context descriptors. A sparse Top-k routing mechanism activates the most relevant experts to perform heterogeneous feature interaction. The selected expert outputs are aggregated through weighted summation and further combined with the original representation through residual semantic refinement to generate the final fused features.

5. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 4, characterized in that: Projecting onto a unified interactive space involves the following steps: The alignment of basic features and detection features is projected into a unified interaction space. The algorithm includes: , Lightweight projection bottlenecks are addressed to construct a unified interactive space. The algorithm includes: ， in, This represents a transition function consisting of 1×1 convolutions. For splicing features, Features projected onto a unified interaction space; the superscript i indicates the network layer number; Indicates will and Direct channel splicing.

6. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 5, characterized in that: Context-aware routing includes: from the projected representation via global average pooling The global context descriptor is extracted, and based on the global context descriptor, a lightweight gated network is used to estimate the expert route distribution. The algorithm includes: , The Softmax function is represented by GAP(·); GAP(·) represents adaptive global average pooling. Let W1 and W2 represent the gating score of the j-th expert under the i-th layer features; W1 and W2 represent the learnable weight matrices of the two fully connected network layers. A sparse Top-k routing strategy is used to activate only the most relevant experts: ,in This represents the set of experts selected at the i-th feature; the superscript i indicates the network layer number, and the subscript j indicates the index of the j-th expert; Topk represents the sparse Top-k routing function. This means selecting the top k experts with the highest weights from all expert weights; k represents the preset hyperparameter, meaning that only k experts are activated each time.

7. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 6, characterized in that: Expert interactive learning includes: Each expert network is implemented as a lightweight bottleneck structure, and the algorithm includes: , Only call the respective conversion functions The Top-k experts are used to refine the features. Experts' enhanced statement The result is obtained by weighted summation of the activated experts: ; in, For the features of the j-th expert under the i-th layer of features, This represents the set of activated experts obtained from the Top-k selection in the previous step; This indicates the fusion features enhanced by the expert; This represents the output feature of the j-th expert unit at the i-th scale, where the input feature x is fed in.

8. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 7, characterized in that: Residual semantic refinement includes: The expert-adjusted features are then reintegrated with the original projective embeddings. The algorithm is as follows: , This represents the final fused representation passed to the subsequent Transformer encoder-decoder architecture.

9. The intelligent transportation target detection method based on a large visual model and an enhanced detection model according to claim 8, characterized in that: The establishment and training of the DAIR detection framework model includes: Collect images of targets for intelligent transportation detection and label the detection boxes and corresponding categories; A detection framework DAIR model is constructed, and a structure-aware feature adapter and an adaptive cross-source interactive routing module are established. The backbone features of the large visual model are embedded into the enhanced detection model as a dual-branch backbone network. The main structure of the large visual model of the DAIR detection framework is frozen, and the collected data is cross-trained to obtain a smart traffic detection task scenario model and obtain weights. This enables pre-training of the model; The trained model Deployed to the intelligent transportation cloud server.

10. A smart traffic target detection system based on a large visual model and an augmented detection model, characterized in that, include: The data acquisition module uses a camera to capture input images; The data processing module is used to input the input image into the pre-trained detection framework DAIR model. The detection framework DAIR model includes a dual-branch backbone network, a structure-aware feature adapter, and an adaptive cross-source interactive routing module. The dual-branch backbone network includes a large vision model and an enhanced detection model. The detection features and basic representations of the input image are extracted by the large vision model and the enhanced detection model in the dual-branch backbone network, respectively. The basic representation is transformed into a representation space compatible with the detection task by using a structure-aware feature adapter, generating aligned features; An adaptive cross-source interactive routing module performs adaptive cross-source interactive routing between aligned features and detection features to obtain a fused representation. The fused features are then input into the detection head of the encoder-decoder of the enhanced detection model in the dual-branch backbone network to perform target classification and bounding box regression, thereby obtaining the target prediction result. The data output module is used to output the target prediction results; This method is used to implement the intelligent transportation target detection method based on a large visual model and an enhanced detection model as described in any one of claims 1 to 9.

Images