Small target detection and tracking method based on traffic scene

Abstract

The invention provides a small target detection and tracking method based on a traffic scene, and relates to the technical field of small target detection, and the method comprises the steps: obtaining continuous video frames of the traffic scene; inputting each frame of image into the integrated model; performing small target detection on each frame of image by using the target detection sub-network, and outputting a detection result set; constructing a dynamic graph based on the detection result set; and inputting the dynamic graph into the dynamic graph tracking sub-network for spatio-temporal feature interaction modeling, updating node spatio-temporal features, performing target association and trajectory generation based on the updated node spatio-temporal features, and outputting a continuous target trajectory. According to the method and the device, the technical problem of high detection omission ratio of the small target in a complex traffic scene due to small size and frequent shielding of the small target in the traffic scene in the prior art is solved, and through small target detection and tracking, the accuracy of small target detection is improved, and the traffic safety is improved.

Description

Technical Field

[0001] This application relates to the field of small target detection technology, specifically to a method for small target detection and tracking based on traffic scenarios. Background Technology

[0002] In recent years, high-position video technology has developed rapidly. By installing high-position video cameras on the roadside to collect data and using visual algorithms to analyze the collected data in a comprehensive manner, it is beneficial to achieve more refined and accurate management of roadside parking. Accurate detection and tracking of small targets such as pedestrians, cyclists, and motorcycles is also crucial for improving road safety and the decision-making of intelligent transportation systems. In roadside high-position video surveillance scenarios, through small target detection and tracking technology, traffic management departments can monitor non-motorized vehicles, pedestrians, and other targets on the road in real time, promptly detect traffic congestion, accidents, and other anomalies, collect traffic flow statistics, help analyze traffic conditions, optimize traffic light control strategies, and improve road traffic efficiency.

[0003] However, due to challenges such as target occlusion in complex roadside scenarios, low imaging resolution, and variable lighting, existing methods still lack sufficient accuracy and stability in perceiving small targets, which can easily lead to missed detections, false detections, and identity switching, directly restricting traffic safety.

[0004] In summary, existing technologies suffer from the technical problem that small targets are small in size and often obscured in traffic scenarios, resulting in a high rate of missed detection in complex traffic situations, which further affects traffic safety. Summary of the Invention

[0005] The purpose of this application is to provide a small target detection and tracking method based on traffic scenarios, in order to solve the technical problem in the prior art that small targets are small in size and often occluded in traffic scenarios, resulting in a high false negative rate in complex traffic scenarios, which further affects traffic safety.

[0006] To achieve the above objectives, this application provides a method for small target detection and tracking based on traffic scenes. The method includes: acquiring continuous video frames of a traffic scene; inputting each frame of the continuous video frames into an integrated model, the integrated model including a cascaded target detection subnetwork and a dynamic graph tracking subnetwork; using the target detection subnetwork to perform small target detection on each frame of the continuous video frames, outputting a detection result set, wherein the target detection subnetwork integrates a multi-scale feature fusion module and an attention mechanism; constructing a dynamic graph based on the detection result set, where nodes in the dynamic graph correspond to detected targets, and edges in the dynamic graph represent the interaction relationship between adjacent nodes; inputting the dynamic graph into the dynamic graph tracking subnetwork for spatiotemporal feature interaction modeling, updating the spatiotemporal features of the nodes, performing target association and trajectory generation based on the updated spatiotemporal features of the nodes, and outputting a continuous target trajectory, wherein the continuous target trajectory has a target identity ID.

[0007] Optionally, a traffic scene training dataset is obtained, wherein the traffic scene training dataset includes a traffic video dataset and an open-source traffic scene dataset, and the traffic scene dataset has target identity ID identifiers; an initial model is constructed, including the target detection sub-network and the dynamic graph tracking sub-network, wherein the target detection sub-network is an improved YOLO target detection network, and the dynamic graph tracking sub-network is based on a dynamic graph neural network; using the traffic scene dataset, the target identity ID is used as the training label, and the initial model is trained end-to-end using a joint loss function combining ID classification loss and detection box regression loss to obtain the integrated model.

[0008] Optionally, the traffic scene is acquired by acquiring the original video sequence of the traffic scene through a visual sensor, and target detection and tracking annotation are performed on each frame of the original video sequence to obtain the traffic video dataset, wherein the annotation information includes at least the category, unique ID and two-dimensional detection box coordinates of each target; the open source traffic scene dataset is obtained, which includes at least the KITTI dataset; the traffic video dataset and the open source traffic scene dataset are fused to obtain the traffic scene dataset.

[0009] Optionally, an object detection sub-network is constructed, which includes at least a backbone network, a neck structure, and a decoupled detection head. The backbone network is implemented using multiple stacked convolutional neural network layers to extract multi-scale features of details. The neck structure employs a multi-scale feature pyramid structure to capture multi-scale contextual information and utilizes a cross-stage spatial attention mechanism to enhance the localization capability of small targets. The decoupled detection head adopts a decoupled detection head design to separate classification and regression tasks.

[0010] Optionally, each detected target in the detection result set is mapped to a node in the dynamic graph, and the node features contain spatial and temporal feature information; the association between adjacent detected targets in the detection result set is mapped to the edges of the dynamic graph; the structure of the dynamic graph is encoded into an adjacency matrix, where the matrix elements represent the weight values ​​of the edges between adjacent detected targets; the structure of the dynamic graph is incrementally updated according to the traffic scene video sequence to complete the construction of the dynamic graph.

[0011] Optionally, if a new node appears after the traffic scene video sequence undergoes small target detection through the target detection subnetwork, a new node is created in the dynamic graph; if a node disappears after the traffic scene video sequence undergoes small target detection through the target detection subnetwork, a new node is removed from the dynamic graph; and the edge weights of the adjacency matrix are dynamically adjusted based on the small target detection results of the latest video frame.

[0012] Optionally, the weight values ​​of the edges between adjacent detected targets are calculated after normalization by spatial proximity, velocity similarity, and appearance similarity. The spatial proximity is calculated by calculating the Euclidean distance between the center points of the targets, the velocity similarity is calculated by calculating the difference between the vectors, and the appearance similarity is calculated by calculating the cosine similarity between the features of the target detection boxes.

[0013] Optionally, spatial features are obtained by extracting the geometric features and appearance information of each target using the target detection subnetwork; temporal features are obtained by determining the changes in the target position using the dynamic graph tracking subnetwork; and the spatial features and temporal features are fused to obtain the spatiotemporal features of the node.

[0014] The technical solution provided in this application has at least the following technical effects or advantages:

[0015] By acquiring continuous video frames of traffic scenes, each frame of the continuous video frames is input into an integrated model, which includes a cascaded target detection subnetwork and a dynamic graph tracking subnetwork. The target detection subnetwork performs small target detection on each frame of the continuous video frames, outputting a detection result set. The target detection subnetwork integrates a multi-scale feature fusion module and an attention mechanism. Based on the detection result set, a dynamic graph is constructed, where nodes correspond to detected targets, and edges represent the interaction relationships between adjacent nodes. The dynamic graph is input into the dynamic graph tracking subnetwork for spatiotemporal feature interaction modeling, updating the spatiotemporal features of nodes. Based on the updated spatiotemporal features, target association and trajectory generation are performed, outputting a continuous target trajectory, where the continuous target trajectory has a target identity ID. In other words, by optimizing small target detection through multi-scale feature fusion and an attention mechanism, and by utilizing the spatial modeling capabilities of the dynamic graph neural network to optimize small target tracking, the detection and tracking performance of small targets in complex and diverse traffic scenarios is improved, providing accurate and real-time small target parsing results for intelligent transportation systems.

[0016] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a small target detection and tracking method based on a traffic scenario according to this application.

[0019] Figure 2 This is a schematic diagram illustrating the process of constructing an integrated model in a small target detection and tracking method based on traffic scenarios according to this application. Detailed Implementation

[0020] This application provides a method for small target detection and tracking in traffic scenarios, addressing the technical problem in existing technologies where small targets, due to their small size and frequent occlusion in traffic scenes, suffer from high false negative rates in complex traffic environments, further impacting traffic safety. The method optimizes small target detection through multi-scale feature fusion and attention mechanisms, and improves tracking by leveraging the spatial modeling capabilities of dynamic graph neural networks. This enhances the detection and tracking performance of small targets in complex and diverse traffic scenarios, providing accurate and real-time small target parsing results for intelligent transportation systems.

[0021] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0022] For examples, please refer to the appendix. Figure 1 This application provides a method for small target detection and tracking in traffic scenarios, wherein the method specifically includes the following steps:

[0023] S100: Acquire continuous video frames of traffic scenes.

[0024] Specifically, this typically involves cameras installed at high locations, such as roadside high-position video cameras, capturing wide traffic scenes and continuously recording video at high resolution and a fixed frame rate. The installation height, tilt angle, and focal length of the roadside high-position video camera need to be adjusted on-site to ensure that its monitoring field of view can completely cover the target area, such as a road area approximately 80 meters long and four lanes wide, and minimize the obstruction of distant targets by nearby vehicles. After the equipment is deployed and powered on, the camera begins to work continuously according to preset parameters, converting light signals into digital signals and transmitting continuous video images in real time to the backend server in the form of a stable data stream via wired or wireless networks.

[0025] Traffic scene continuous video frames are a series of static images captured by roadside high-position video cameras. Each image is called a frame, and the number of frames captured per unit time is called the frame rate. The frequency is essentially formed by rapidly playing these frames in chronological order. By using a high-position viewpoint and 4K high resolution, the monitoring range of a single camera is expanded, and the number of image pixels for small targets at a distance is significantly increased.

[0026] S200: Input each frame of the continuous video frames of the traffic scene into the integrated model, which includes a cascaded target detection subnetwork and a dynamic graph tracking subnetwork.

[0027] Further details are attached. Figure 2 As shown, S200 of this application includes:

[0028] A traffic scene training dataset is obtained, which includes a traffic video dataset and an open-source traffic scene dataset, and the traffic scene dataset has target identity IDs. An initial model is constructed, including the target detection sub-network and the dynamic graph tracking sub-network, wherein the target detection sub-network is an improved YOLO target detection network, and the dynamic graph tracking sub-network is based on a dynamic graph neural network. Using the traffic scene dataset, the target identity ID is used as the training label, and the initial model is trained end-to-end using a joint loss function combining ID classification loss and detection box regression loss to obtain the integrated model.

[0029] Furthermore, this application also includes the following steps: acquiring original video sequences of traffic scenes through a visual sensor, and performing target detection and tracking annotation on each frame of the original video sequence to obtain the traffic video dataset, wherein the annotation information includes at least the category, unique ID, and two-dimensional detection box coordinates of each target; obtaining the open-source traffic scene dataset, wherein the open-source traffic scene dataset includes at least the KITTI dataset; and fusing the traffic video dataset and the open-source traffic scene dataset to obtain the traffic scene dataset.

[0030] Specifically, in building the integrated model, the first step is to collect raw video sequences of traffic scenes using visual sensors. This involves setting up high-position video cameras and other sensor devices to acquire video image data of traffic scenes. Small target types such as pedestrians and non-motorized vehicles are then detected, tracked, and labeled, including the category, ID, and two-dimensional detection box for each target. The aforementioned continuous video frames of the traffic scene are real-time video streams, while the traffic video dataset consists of historically collected video data used as training samples. For each traffic participant in the frame, such as people and non-motorized vehicles, a precise bounding box, category, and a unique ID throughout their appearance are assigned, thereby constructing a private traffic video dataset that meets the needs of practical applications.

[0031] Object detection annotation refers to marking the location of each target of interest (such as a car or pedestrian) in each frame of an image using a rectangular bounding box (a two-dimensional detection box), defined by the x and y coordinates of its top-left vertex and the box's width and height. Each box is then assigned a category label, such as "car" or "pedestrian." Tracking annotation refers to assigning the same unique ID to targets belonging to the same real-world context across multiple consecutive frames. For example, if pedestrian A appearing in the first frame is labeled ID001, then in all subsequent frames, pedestrian A's detection box should be assigned ID001 until it leaves the field of view.

[0032] To compensate for the potential limitations of private data in terms of scale and scene diversity, authoritative open-source datasets are introduced as supplementary resources, including but not limited to KITTI and MOT Challenge datasets. The KITTI dataset is one of the most influential benchmark datasets in the fields of autonomous driving and computer vision. It collects rich urban, rural, and highway scene data by equipping vehicles with cameras, LiDAR, and GPS, and provides detailed annotations for various tasks such as 2D / 3D object detection, tracking, and road segmentation. The MOT Challenge dataset is a public benchmark platform focused on multi-object tracking, providing a series of challenging video sequences, including complex situations such as dense crowds, frequent interactions, and occlusion. It provides accurate bounding boxes and cross-frame identity ID annotations, specifically designed for training and evaluating the performance of multi-object tracking algorithms. KITTI and MOT Challenge datasets are both publicly available datasets. Traffic video datasets and open-source traffic scene datasets are merged to form a hybrid training dataset, namely the traffic scene dataset.

[0033] On the model building side, a two-part initial model is designed. The front end is a target detection sub-network based on the classic YOLO architecture with targeted improvements, specifically enhancing its small target perception capabilities. The back end is a tracking sub-network with a dynamic graph neural network at its core, responsible for spatiotemporal correlation reasoning of detected targets. The overall model architecture is achieved by integrating the real-time target detection advantages of YOLO11 and the spatiotemporal modeling capabilities of DGNN. YOLO11 improves small target detection performance by utilizing multi-scale feature fusion and attention mechanisms, while DGNN improves small target tracking performance by updating the relationships between nodes (targets) and edges (interactions) through a dynamic graph structure.

[0034] The improved YOLO object detection network is a very popular single-stage object detection algorithm, known for its speed. The improvement means that targeted optimizations have been made to its basic architecture, such as adding shallow feature outputs that are more suitable for small object detection, introducing attention mechanisms, or optimizing the feature pyramid structure to improve the detection capability of small objects in traffic scenes.

[0035] Dynamic Graph Neural Networks (DGNNs) are deep learning models used to process graph-structured data. Nodes in a dynamic graph are the detected target instances in each frame, and edges represent potential spatial (within the same frame) or temporal (across frames) relationships between targets. DGNNs use a message-passing mechanism to allow nodes to exchange information, thereby learning and updating the feature representation of each target, thus encoding rich contextual and spatiotemporal relational information.

[0036] Using a prepared traffic scene dataset, the provided target identity IDs are used as key supervision signals. A joint loss function, which integrates ID classification loss and detection box regression loss, is employed to train the entire initial model end-to-end. Through extensive iterative optimization, the model not only learns to accurately locate targets in each frame but also learns to continuously and correctly identify the identity of each target amidst complex temporal changes, resulting in a mature integrated model capable of outputting ID-enabled trajectories.

[0037] Model optimization and parameter updates are performed using a joint loss function. This joint loss function comprises two parts: a classification loss function and a bounding box position regression loss function. The classification loss function uses the target's ID as the training label to determine whether two targets are the same target. It is calculated using the cross-entropy loss function, expressed as: ;in, `L` represents the predicted ID category, and `G` represents the ground truth value. The target box location regression loss function uses the IoU loss function. IoU Loss is based on the Intersection over Union (IoU) ratio between the predicted box and the ground truth box, where the predicted box is denoted as `P` and the labeled box as `G`. The corresponding IoU can be expressed as: IoU Loss is the ratio of the intersection to the union of two bounding boxes; it is defined as: .

[0038] Using a joint loss function for end-to-end training enables the model to learn both object detection and tracking tasks simultaneously, improving the accuracy and real-time performance of detection and tracking. This not only enhances the intelligence of traffic management but also strengthens traffic safety and optimizes the allocation of traffic resources.

[0039] Furthermore, this application also includes the following steps: constructing an object detection sub-network, wherein the object detection sub-network includes at least a backbone network, a neck structure, and a decoupled detection head; wherein the backbone network is implemented using multiple stacked convolutional neural network layers to extract multi-scale features of details; the neck structure adopts a multi-scale feature pyramid structure to capture multi-scale contextual information and employs a cross-stage spatial attention mechanism to enhance the localization ability of small targets; the decoupled detection head adopts a decoupled detection head design to separate classification and regression tasks.

[0040] Specifically, the object detection sub-network includes at least a backbone network, a neck structure, and a decoupled detection head. The backbone network is the main feature extractor of the object detection model, typically composed of a series of convolutional layers, pooling layers, and other basic operations. Similar to the human spine, it is responsible for abstracting and extracting multi-level, multi-semantic feature maps from the original input image layer by layer. Shallow features are rich in detail and suitable for capturing edges and textures; deep features are semantically strong and suitable for identifying object categories.

[0041] The neck structure, located between the backbone network and the detection head, is primarily responsible for fusing and enhancing the multi-scale features extracted by the backbone network. A common structure is the multi-scale feature pyramid, which aligns and fuses feature maps from different levels in terms of spatial size and semantic information through upsampling and downsampling operations. This ensures that the output features of each layer possess rich details and high-level semantics, thereby improving the model's ability to detect targets of different sizes, especially small targets.

[0042] Traditional detection heads typically use the same convolutional network branch to simultaneously predict the object's category (classification) and bounding box location (regression). Decoupled detection heads, on the other hand, separate these two tasks, using two independent and lightweight branch networks to process them separately. This reduces the inherent conflict and interference between classification and regression tasks, allowing each branch to focus more on its own target, thereby improving overall detection accuracy.

[0043] A proven deep convolutional network with powerful feature extraction capabilities is used as the foundation. After inputting a high-resolution image, a series of convolutions and downsampling operations output a set of multi-scale feature maps. A key neck structure is constructed; to effectively handle targets with significant scale differences in traffic scenes, an enhanced multi-scale feature pyramid structure is employed. A top-down path upsamples deep, high-semantic features and fuses them with shallow, high-resolution features. Simultaneously, a bottom-up path downsamples shallow, fine-grained features and fuses them with deep features, achieving bidirectional feature flow and full interaction. During this process, a cross-stage spatial attention mechanism is embedded, calculating joint attention weights from feature maps of different stages and applying them to the features before fusion. This allows the network to actively focus on image regions containing small or difficult-to-distinguish targets during fusion, significantly enhancing the localization signal for these small targets. A decoupled detection head is designed. On each scale-fused feature map obtained from the neck structure, two small convolutional network branches are connected in parallel: a classification branch is responsible for outputting the probability that each preset anchor point belongs to each target category; the other regression branch is responsible for independently outputting the precise coordinate offset and size adjustment parameters of the bounding box corresponding to each anchor point. This separate design allows the two tasks to optimize independently without hindering each other, jointly improving the accuracy of the detection boxes and the confidence of the category judgment.

[0044] Through an improved neck structure and attention mechanism, the network can fully utilize all feature information from low to high layers, ensuring that even deep features retain the spatial details needed to locate small targets, fundamentally improving the feature representation quality of small targets. The target detection sub-network can more effectively extract and utilize multi-scale features in images, improving the detection capability for small targets. The multi-scale feature pyramid structure and cross-stage spatial attention mechanism significantly enhance the network's detection performance for multi-scale targets. The decoupled detection head design further improves the accuracy of classification and regression tasks.

[0045] S300: The target detection sub-network is used to perform small target detection on each frame of the continuous video frames of the traffic scene, and outputs a detection result set. The target detection sub-network integrates a multi-scale feature fusion module and an attention mechanism.

[0046] Specifically, the object detection subnetwork takes each frame of a continuous video frame of a traffic scene as input and outputs an identifier Dt={(Bi, Ci, Li)}, where i represents target i; Bi=[x,y,w,h] represents the detection box information of target i, x and y represent the center point coordinates of the detection box, w and h represent the width and height of the detection box; Ci represents the confidence score of target i; and Li represents the category of target i.

[0047] Specifically, the acquired continuous video frames of the traffic scene are input frame by frame in chronological order into a pre-trained object detection sub-network. The network first performs depthwise convolution on each high-resolution image frame through its backbone network, extracting multi-scale feature maps rich in information. These maps are then fed into the neck structure, which integrates a multi-scale feature fusion module. Features from deep and shallow layers of the backbone network are bidirectionally fused through upsampling, downsampling, and weighted summation to generate fused features at each layer that simultaneously possess precise location information and high-level category information. An attention mechanism is applied to these fused features, automatically calculating a spatial weight mask to enhance regions in the fused feature map that have weak responses but may contain small or edge targets, while suppressing background noise. The processed features are then fed into a decoupled detection head, where the classification and regression branches work independently, predicting the class probability and bounding box fine-tuning parameters for each preset anchor point, respectively. After confidence threshold filtering (retaining only predictions with a confidence greater than 0.5) and maximum suppression to eliminate redundant boxes, the network outputs the detection result set for the current frame.

[0048] The detection result set is a structured data set output by the object detection sub-network after processing a frame of image. Each entry in the detection result set typically includes: object category (such as pedestrian), confidence score (ranging from 0 to 1), and two-dimensional bounding box coordinates (usually in pixels, in the format of [top left x, top left y, width, height] or [center x, center y, width, height]).

[0049] By using a target detection subnetwork that integrates a multi-scale feature fusion module and an attention mechanism, the detection accuracy for small targets is significantly improved, addressing the problem of small target detection in traffic scenarios and providing accurate basic data for subsequent target tracking and behavior analysis.

[0050] S400: Based on the detection result set, construct a dynamic graph, where the nodes of the dynamic graph correspond to the detection targets, and the edges of the dynamic graph represent the interaction relationship between two adjacent nodes.

[0051] Furthermore, S400 of this application includes: mapping each detected target in the detection result set to a node of the dynamic graph, wherein the node features contain spatial and temporal feature information; mapping the association between adjacent detected targets in the detection result set to edges of the dynamic graph; encoding the structure of the dynamic graph into an adjacency matrix, wherein the matrix elements represent the weight values ​​of the edges between adjacent detected targets; and incrementally updating the structure of the dynamic graph according to the traffic scene video sequence to complete the construction of the dynamic graph.

[0052] Furthermore, this application also includes the following steps: if a new node is found after the traffic scene video sequence undergoes small object detection through the target detection subnetwork, a new node is created in the dynamic graph; if a node disappears after the traffic scene video sequence undergoes small object detection through the target detection subnetwork, a new node is removed from the dynamic graph; and the edge weights of the adjacency matrix are dynamically adjusted based on the small object detection results of the latest video frame.

[0053] Furthermore, this application also includes the following steps: the weight values ​​of the edges between adjacent detection targets are calculated after normalization by spatial proximity, velocity similarity, and appearance similarity, wherein the spatial proximity is calculated by calculating the Euclidean distance between the center points of the targets, the velocity similarity is calculated by calculating the difference between vectors, and the appearance similarity is calculated by calculating the cosine similarity between the features of the target detection boxes.

[0054] Specifically, the discrete and disordered sequence of frame-by-frame detection boxes output by the object detection subnetwork is intelligently organized into a structured dynamic graph containing rich spatiotemporal relationships. Each detected target in the detection result set is created as a new node, and a graph that integrates spatial and temporal features is initialized for it.

[0055] A time edge is established by using a data association algorithm to connect the nodes of the current frame with the nodes of the same physical target in the previous frame's dynamic graph, forming a cross-frame trajectory chain. On the other hand, based on the geometric relationship and motion pattern between targets in the current frame, such as calculating the Euclidean distance and the angle between velocity vectors between each pair, a spatial edge is established between node pairs that meet specific conditions to characterize the real-time interaction relationship between targets.

[0056] The constructed graph structure is encoded as an adjacency matrix. The value of each element in the matrix is ​​calculated by the type and association strength of the edges between the corresponding node pairs, such as the matching confidence of temporal edges and the interaction strength score of spatial edges.

[0057] As the video sequence progresses, the dynamic graph is incrementally updated: if a detected target cannot be associated with any existing node in the graph, it is considered a newly appeared target, and a new node is created in the graph; conversely, if a node in the graph fails to be associated with any detection box in consecutive frames, it is considered that the target has left the scene or is persistently occluded, and the node and all its edges are removed from the graph after reaching a lifetime threshold. Each frame recalculates and adjusts the edge weights in the adjacency matrix based on the latest detection results and node states. For example, spatial edge weights are updated based on the latest observations to reflect changes in interaction relationships. Incremental updates are an efficient strategy for processing streaming data. Instead of completely reconstructing the entire graph every frame, it only locally modifies the existing graph structure based on the detection results of the new frame—adding new nodes, removing missing nodes, and adjusting edge connections and weights—significantly reducing computational complexity.

[0058] In other words, for each time t, the dynamic graph is represented as Gt=(Nt,Et), where Nt represents a set of nodes, each node representing a detection target, and Et represents a set of edges, each edge representing the interaction relationship between two adjacent nodes. For each node i, a feature vector xi is used, containing spatial and temporal feature information; for Et, the relationship between adjacent nodes (i.e., detection targets) is encoded as an adjacency matrix At, where Aij represents the weight of the edge between node i and node j; the graph structure is dynamically evolved through inter-frame updates: for newly added detection targets, corresponding nodes are created, while for corresponding disappearing targets, the corresponding nodes are removed, and the edge weights are adjusted in real time according to the latest interaction relationship, thus realizing the construction of the dynamic graph.

[0059] For Dynamic Graph Neural Networks (DGNNs), information between adjacent nodes is aggregated by utilizing graph neural network layers. For each graph neural network layer l, it can be represented as: H(l+1) = σ(AtH). （l） W （l） ), where H （l） W represents the node features of the l-th layer. （l） H represents the learnable weight parameters of the l-th layer. （l+1）Let At represent the node features of layer (l+1), σ() represent the activation function (ReLU can be used), and xi represent each node i using a feature vector, specifically xi = [Fs, Ft], where Fs represents the spatial features extracted using YOLO11, and Ft represents the temporal features calculated using the DGNN dynamic graph neural network. The relationship (weights) between adjacent nodes (i.e., detected targets) is calculated based on three factors: spatial proximity is calculated by the Euclidean distance between the target center points; velocity similarity is calculated by the difference between vectors; and appearance similarity is calculated by the cosine similarity between the target detection box features. In other words, spatial proximity is the degree of physical proximity between two targets in the image space, and it is usually calculated by the Euclidean distance between the center points of the two targets. The closer the distance, the higher the spatial proximity, and the larger the edge weight. Velocity similarity is the consistency of the motion states of two targets, and it is calculated by comparing the velocity vectors of the two targets, usually by calculating the difference between these two vectors. If the velocity vectors of two targets are similar, their velocity similarity is high, and the weight of the edge will increase accordingly. Appearance similarity refers to the degree of similarity between two targets in their appearance features. Appearance similarity is calculated by calculating the cosine similarity between the features of the two target detection boxes. The higher the cosine similarity, the more similar the two targets are in appearance, and the larger the weight of the edge.

[0060] After calculating spatial proximity, velocity similarity, and appearance similarity, they need to be normalized to ensure they have equal importance when calculating edge weights. The normalized values ​​are then combined according to a predetermined weighting strategy to ultimately calculate the edge weights.

[0061] By mapping detected targets to nodes that fuse spatiotemporal features, targets are no longer isolated observation boxes, but rather intelligent agents carrying historical and current state information, resulting in more discriminative and temporally smooth feature representations. By constructing temporal and spatial edges, temporal edges maintain the continuity of target identity and form the basis for trajectory generation; spatial edges capture real-time interactions between targets, providing direct evidence for understanding group behavior. Through incremental updates and dynamic adjustments to the adjacency matrix, the entire graph can respond to scene changes in real time with extremely low computational overhead, ensuring efficient adaptation to dynamic traffic environments.

[0062] S500: Input the dynamic graph into the dynamic graph tracking subnetwork to perform spatiotemporal feature interaction modeling, update the spatiotemporal features of the nodes, perform target association and trajectory generation based on the updated spatiotemporal features of the nodes, and output continuous target trajectories, wherein the continuous target trajectories have target identity ID identifiers.

[0063] Furthermore, S500 of this application includes: extracting and capturing the geometric features and appearance information of each target using a target detection sub-network to obtain spatial features; determining the change of target position using a dynamic graph tracking sub-network to obtain temporal features; and fusing the spatial features and the temporal features to obtain the spatiotemporal features of the node.

[0064] Specifically, the small target detection and tracking model based on YOLO11 and DGNN achieves effective small target tracking by utilizing temporal-spatial feature interactions. It mainly consists of two parts: spatial features Fs, extracted using YOLO11, capture the geometric features and appearance information of each target, enabling accurate target localization and recognition; temporal features Ft, calculated using a dynamic graph neural network (DGNN), represent changes in target position; furthermore, edge weights are adjusted in real-time based on the latest interaction relationships to construct a dynamic graph, ensuring robust target tracking in complex and rapidly changing traffic environments.

[0065] For each detection box, features are extracted from the P3 layer (the neck layer of the network) at 8x downsampling resolution. Bilinear interpolation yields a 256x7x7 feature block, which is then global average pooled to obtain a 256-dimensional appearance vector. Geometric features include normalized center coordinates (x, y), width (w), height (h), aspect ratio (w / h), and area (w*h), totaling 6 dimensions. These are concatenated and passed through a 128-dimensional fully connected layer to output the final 128-dimensional spatial features.

[0066] For each tracked target, a historical state queue of 10 frames (approximately 0.33 seconds) is maintained, containing the position and spatial features of each frame. A single-layer GRU is used to process this sequence, with a hidden state dimension of 64. Simultaneously, the average velocity and acceleration of the target over the past 5 frames are calculated. The GRU hidden states are concatenated with the motion features, and then passed through a 32-dimensional fully connected layer to output the final 32-dimensional temporal features. A channel attention module is used for fusion. The 128-dimensional spatial features and 32-dimensional temporal features are mapped to 64 dimensions respectively and then concatenated to obtain a 128-dimensional vector. Then, channel weights are generated through an SE block for recalibration, outputting the final 128-dimensional spatiotemporal features of the nodes.

[0067] By fusing spatial and temporal features into node spatiotemporal features, targets in traffic scenarios can be represented more accurately, giving each node in the dynamic graph richer information and improving the accuracy of target tracking and behavior analysis.

[0068] In summary, the small target detection and tracking method based on traffic scenarios provided in this application has the following technical advantages:

[0069] By acquiring continuous video frames of traffic scenes, each frame of the continuous video frames is input into an integrated model, which includes a cascaded target detection subnetwork and a dynamic graph tracking subnetwork. The target detection subnetwork performs small target detection on each frame of the continuous video frames, outputting a detection result set. The target detection subnetwork integrates a multi-scale feature fusion module and an attention mechanism. Based on the detection result set, a dynamic graph is constructed, where nodes correspond to detected targets, and edges represent the interaction relationships between adjacent nodes. The dynamic graph is input into the dynamic graph tracking subnetwork for spatiotemporal feature interaction modeling, updating the spatiotemporal features of nodes. Based on the updated spatiotemporal features, target association and trajectory generation are performed, outputting a continuous target trajectory, where the continuous target trajectory has a target identity ID. In other words, by optimizing small target detection through multi-scale feature fusion and an attention mechanism, and by utilizing the spatial modeling capabilities of the dynamic graph neural network to optimize small target tracking, the detection and tracking performance of small targets in complex and diverse traffic scenarios is improved, providing accurate and real-time small target parsing results for intelligent transportation systems.

[0070] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0071] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for small target detection and tracking in traffic scenarios, characterized in that, include: Acquire continuous video frames of traffic scenes; Each frame of the continuous video frame of the traffic scene is input into the integrated model, which includes a cascaded target detection subnetwork and a dynamic graph tracking subnetwork. The target detection subnetwork is used to perform small target detection on each frame of the continuous video frames of the traffic scene, and outputs a detection result set. The target detection subnetwork integrates a multi-scale feature fusion module and an attention mechanism. Based on the detection result set, a dynamic graph is constructed, where the nodes of the dynamic graph correspond to the detection targets, and the edges of the dynamic graph represent the interaction relationship between two adjacent nodes. The dynamic graph is input into the dynamic graph tracking subnetwork for spatiotemporal feature interaction modeling, the spatiotemporal features of the nodes are updated, the target association and trajectory generation are performed based on the updated spatiotemporal features of the nodes, and the continuous target trajectory is output, wherein the continuous target trajectory has a target identity ID identifier.

2. The method for small target detection and tracking in traffic scenarios as described in claim 1, characterized in that, Building an integrated model includes: Obtain a traffic scene training dataset, wherein the traffic scene training dataset includes a traffic video dataset and an open-source traffic scene dataset, and the traffic scene dataset has a target identity ID identifier; An initial model is constructed, including the object detection subnetwork and the dynamic graph tracking subnetwork, wherein the object detection subnetwork is an improved YOLO object detection network, and the dynamic graph tracking subnetwork is based on a dynamic graph neural network. Using the traffic scene dataset, the target identity ID is used as the training label. The initial model is trained end-to-end by combining the joint loss function of ID classification loss and detection box regression loss to obtain the integrated model.

3. The method for small target detection and tracking in traffic scenarios as described in claim 2, characterized in that, Obtain traffic scene datasets, including: The traffic video dataset is obtained by acquiring raw video sequences of traffic scenes through a visual sensor and performing target detection and tracking annotation on each frame of the raw video sequence. The annotation information includes at least the category, unique ID, and coordinates of the two-dimensional detection box for each target. Obtain the open-source traffic scenario dataset, which includes at least the KITTI dataset; The traffic video dataset and the open-source traffic scene dataset are merged to obtain the traffic scene dataset.

4. The method for small target detection and tracking in traffic scenarios as described in claim 2, characterized in that, Construct a target detection subnetwork, which includes at least a backbone network, a neck structure, and a decoupled detection head; The backbone network is implemented using multiple stacked convolutional neural network layers to extract multi-scale features of details. The neck structure adopts a multi-scale feature pyramid structure to capture multi-scale contextual information and uses a cross-stage spatial attention mechanism to enhance the localization ability of small targets. The decoupled detection head adopts a decoupled detection head design to separate classification and regression tasks.

5. The method for small target detection and tracking in traffic scenarios as described in claim 2, characterized in that, Based on the detection result set, a dynamic graph is constructed, including: Each detected target in the detection result set is mapped to a node in the dynamic graph, and the node features contain spatial and temporal feature information; The correlation between adjacent detected targets in the detection result set is mapped to the edges of the dynamic graph; The structure of the dynamic graph is encoded into an adjacency matrix, where the matrix elements represent the weight values ​​of the edges between adjacent detected targets; The structure of the dynamic graph is incrementally updated based on the traffic scene video sequence to complete the construction of the dynamic graph.

6. The method for small target detection and tracking in a traffic scenario as described in claim 5, characterized in that, The structure of the dynamic graph is incrementally updated based on the traffic scene video sequence, including: If a new node is found after the traffic scene video sequence undergoes small target detection through the target detection subnetwork, a new node is created in the dynamic graph. If a node disappears after the traffic scene video sequence is processed by the target detection subnetwork for small target detection, a new node is removed from the dynamic graph. The edge weights of the adjacency matrix are dynamically adjusted based on the small target detection results of the latest video frame.

7. The method for small target detection and tracking in traffic scenarios as described in claim 5, characterized in that, The structure of the dynamic graph is encoded into an adjacency matrix, where the matrix elements represent the weight values ​​of the edges between adjacent detected targets, including: The weight values ​​of the edges between adjacent detected targets are calculated after normalization by spatial proximity, velocity similarity, and appearance similarity. Specifically, spatial proximity is calculated by Euclidean distance between the center points of the targets, velocity similarity is calculated by difference between vectors, and appearance similarity is calculated by cosine similarity between the features of the target detection boxes.

8. The method for small target detection and tracking in traffic scenarios as described in claim 1, characterized in that, Spatial features are obtained by extracting the geometric features and appearance information of each target using the target detection sub-network; By using a dynamic graph to track subnetworks, changes in the target's location are determined, and temporal characteristics are obtained. The spatial features and the temporal features are fused to obtain the spatiotemporal features of the node.

Images