A low-latency moving target detection method and system for navigation

Abstract

The present invention relates to the field of autonomous driving technology, and specifically discloses a low-latency moving target detection method and system for navigation, wherein the method comprises: collecting an event stream through a vehicle-mounted event camera; adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain an event image; inputting the event image into a pulse neural network to obtain an output feature map; detecting the feature map through a feature pyramid network and a detection head to obtain a motion trajectory stream of the target; and performing nonlinear model predictive control based on the motion trajectory stream to obtain an optimized motion trajectory stream of the target.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a low-latency moving target detection method and system for navigation. Background Art

[0002] Existing autonomous driving systems primarily use natural image cameras and millimeter-wave radars to perceive and predict environmental objects, enabling navigation path planning, obstacle avoidance, and emergency response. These systems often struggle to effectively handle unexpected events, such as pedestrians and motorcycles crossing the road, vehicles cutting in, and objects suddenly appearing after being obscured. These systems also face limitations in their perception mechanisms, such as strong backlighting and sudden changes in tunnel brightness.

[0003] Due to their fixed frame rates and reliance on exposure mechanisms, onboard natural image cameras are prone to high detection latency due to motion blur in high-speed scenarios. Their limited dynamic range makes it difficult to capture target outlines in environments like strong backlight and sudden changes in brightness in tunnels. While millimeter-wave radar can measure speed, its sparse point cloud data cannot identify target morphological features, leading to misjudgments and further compromising target trajectory prediction. Multi-sensor fusion solutions also suffer from compounding delays due to data alignment and processing complexity, making it difficult to respond promptly to emergency situations.

[0004] Currently, major event camera object detection methods focus primarily on object detection accuracy and recognition rate. However, research on low-latency detection methods for event cameras remains limited. Furthermore, there is currently a lack of solutions to the issues of event sparsity and temporal position offset in event object detection. Summary of the Invention

[0005] In response to the above problems, the purpose of the present invention is to provide a low-latency moving target detection method and system for navigation, which can achieve ultra-low-latency target detection and trajectory extrapolation, significantly improve the real-time response capability to high-speed and sudden moving targets and the detection robustness in complex lighting environments.

[0006] The present invention provides a low-latency moving target detection method for navigation, comprising:

[0007] Collect event streams through vehicle-mounted event cameras;

[0008] The time window length of the event stream is adjusted in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain an event image;

[0009] Inputting the event image into a pulse neural network to obtain an output feature map;

[0010] The feature map is detected by a feature pyramid network and a detection head to obtain a motion trajectory flow of the target;

[0011] Nonlinear model predictive control is performed according to the motion trajectory flow to obtain the target optimized motion trajectory flow.

[0012] In a possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0013] The time window length is adjusted according to the following formula:

[0014] ;

[0015] in, is the time window length, is the upper limit of the time window, is the lower limit length of the time window, is the upper threshold of speed, is the lower threshold of speed, is the upper threshold of acceleration, is the lower threshold of acceleration, is the two-norm of the spatial gradient of the event, is the time gradient of the event, is the scaling factor, It is a parameter that balances space and time.

[0016] In a possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0017] The spatial gradient of events is calculated according to the following formula :

[0018] ;

[0019] ;

[0020] in, is the time gradient of the event, For the direction of the event, The polar coordinate code for each event point.

[0021] In a possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0022] The temporal gradient of an event is calculated according to the following formula :

[0023] ;

[0024] in, Code the time of the event, For events and time The second derivative of is the differential of time.

[0025] In one possible implementation, the spiking neural network includes four cascaded convolution modules; and the feature map output by inputting the event image into the spiking neural network includes:

[0026] Input the event image at the current moment into a convolution module and obtain the features of the current moment;

[0027] Decomposing the features at the current moment into low-frequency features and high-frequency features through a wavelet attention mechanism, and concatenating the low-frequency features and high-frequency features to obtain the output features at the current moment;

[0028] The features of the previous moment are convolved through the spatial displacement time attention mechanism to obtain the output features of the previous moment;

[0029] The output features of the current moment are fused with the output features of the previous moment through splicing and convolution modules to obtain a feature map.

[0030] In one possible implementation, the detection head comprises three convolutional layers, which respectively output the detected category, confidence, coordinates, and offset of the bounding box. The process of detecting the feature map using the feature pyramid network and the detection head to obtain the target's motion trajectory stream includes:

[0031] Upsampling the feature map step by step to obtain image interpolation;

[0032] Fusing the image interpolation with the channel-aligned bilinear interpolation to obtain a fused feature map;

[0033] The fusion feature map is optimized by the detection head to obtain the target's motion trajectory flow.

[0034] In a possible implementation, optimizing the fused feature map by the detection head to obtain a motion trajectory flow of the target includes:

[0035] The fused feature map is optimized according to the following formula:

[0036] ;

[0037] in, The target trajectory flow, is the weight of the coordinates and the offset of the bounding box, is the weight of the coordinates and the confidence of the bounding box, is the weight of the coordinate and the category loss of the bounding box, is the coordinate and the offset of the bounding box, is the coordinate and confidence of the bounding box, is the category loss of coordinates and bounding boxes.

[0038] In a possible implementation, optimizing the fused feature map by the detection head to obtain a motion trajectory flow of the target includes:

[0039] Calculate the coordinates and the offset of the bounding box according to the following formula :

[0040] ;

[0041] in, is the intersection-over-union ratio of the predicted target bounding box and the true target bounding box, is the Euclidean distance between the center point of the predicted target bounding box and the center point of the true target bounding box, is the diagonal length of the minimum predicted target bounding box and the true target bounding box, To predict the target bounding box, is the true target bounding box;

[0042] Calculate the coordinates and confidence of the bounding box according to the following formula :

[0043] ;

[0044] in, is the predicted grid resolution, The number of prediction targets for each grid, is the true target confidence, is the target confidence of the prediction, and is the coordinate index of the grid;

[0045] The coordinates and the category loss of the bounding box are calculated according to the following formula :

[0046] ;

[0047] in, is the number of categories, For category index, For the predicted category, For the real category, is the focus parameter.

[0048] In a possible implementation, performing nonlinear model predictive control according to the motion trajectory flow to obtain a target optimized motion trajectory flow includes:

[0049] Dynamically adjust the time window length of the motion trajectory flow according to the jerkiness and sliding overlap mechanism;

[0050] Constructing a nonlinear motion model based on a cubic polynomial according to the time window length; determining model parameters of the nonlinear motion model by integrating physical dynamics constraints and data-driven optimization;

[0051] Predicting the motion trajectory flow according to the nonlinear motion model to obtain a predicted trajectory;

[0052] The predicted trajectory is corrected according to the Kalman filter to obtain an optimized motion trajectory flow of the target.

[0053] The present invention also provides a low-latency moving target detection system for navigation, characterized in that it is applied to any of the above-mentioned low-latency moving target detection methods, including:

[0054] The acquisition module is used to collect event streams through the vehicle-mounted event camera;

[0055] The time window adjustment module is used to adjust the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain the event image;

[0056] A feature map extraction module, configured to input the event image into a pulse neural network to obtain an output feature map;

[0057] A motion trajectory flow detection module is used to detect the feature map through a feature pyramid network and a detection head to obtain the motion trajectory flow of the target;

[0058] The motion trajectory flow optimization module is used to perform nonlinear model predictive control according to the motion trajectory flow to obtain the target optimized motion trajectory flow.

[0059] The low-latency moving target detection method and system for navigation provided by the present invention targets the high-density and directional continuity spatiotemporal characteristics (such as the aggregation of event clusters along the motion trajectory and the density gradient change caused by acceleration) of fast-moving targets (such as vehicles that suddenly change lanes, pedestrians crossing, high-speed motorcycles, and obstacles that suddenly appear after being obscured) in short-time event slices. An extremely lightweight spiking neural network (SNN) is designed to directly process the asynchronous spatiotemporal signals of the event stream, avoiding the computational redundancy of traditional image format conversion and complex models. In addition, a dynamic prediction model is constructed in combination with the physical laws of short-term trajectories (such as uniform speed / uniform acceleration patterns and steering angle continuity) to achieve ultra-low-latency target detection and trajectory extrapolation, significantly improving the real-time response capability to high-speed and sudden moving targets and the detection robustness in complex lighting environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of a flow chart of a low-latency moving target detection method provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of the operation of a spiking neural network according to an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of a simulation of the model training process provided by an embodiment of the present invention;

[0063] Figure 4 Schematic diagram of experimental results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following detailed description of the embodiments of the present invention is provided in conjunction with the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are intended to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention. That is, the present invention is not limited to the preferred embodiments described, and the scope of the present invention is defined by the claims.

[0065] In the description of the present invention, it should be noted that, unless otherwise specified, “plurality” means two or more; the terms “first”, “second”, etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance; for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0066] Figure 1 A flow chart of a low-latency moving target detection method provided by an embodiment of the present invention is shown as follows: Figure 1 As shown, the present invention provides a low-latency moving target detection method for navigation, comprising:

[0067] Step S1, collecting event streams through a vehicle-mounted event camera;

[0068] In one possible implementation, traditional object detection often uses a fixed time window, such as 30ms, to accumulate events. This can lead to blurred features for high-speed targets or insufficient signal-to-noise ratio for low-speed targets. For example, a window that is too long can cause motion artifacts, while a window that is too short can result in too few valid events. The asynchronous event stream captured by the vehicle-mounted event camera of the present invention has microsecond-level temporal resolution.

[0069] Step S2, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain an event image;

[0070] In one possible implementation, the target's velocity is calculated based on the event density gradient of the vehicle-mounted event camera. A velocity-window nonlinear mapping function is constructed to achieve adaptive adjustment of the time window length.

[0071] In one possible implementation, a double threshold method is used to truncate the window time. Soft thresholding or no thresholding can also be used. For example, sigmoid is used to normalize the time window to between 0 and 1, and then mapped back to the upper and lower time limits.

[0072] The speed-driven time window adaptation mechanism of this invention dynamically adjusts the window length by sensing the target's motion speed in real time. This solves the dual contradictions of traditional fixed-window methods, which suffer from motion blur in high-speed scenes and insufficient signal-to-noise ratio in low-speed scenes. It effectively reduces window response time, trajectory positioning errors caused by motion smear, and slow-moving scenes.

[0073] In one possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0074] Adjust the time window length according to the following formula:

[0075] ;

[0076] This formula means that when the speed is too fast or the acceleration is too large, the minimum time window is taken, when the speed is too slow and the acceleration is too slow, the maximum time window is taken, and in other cases, the acceleration and speed are calculated together;

[0077] in, is the time window length, The upper limit of the time window is 10ms. The lower limit of the time window is 50ms. is the upper threshold of speed, is the lower threshold of speed, is the upper threshold of acceleration, is the lower threshold of acceleration, is the two-norm of the spatial gradient of the event, is the time gradient of the event, is the scaling factor, It is a parameter that balances space and time.

[0078] In one possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0079] The spatial gradient of events is calculated according to the following formula :

[0080] ;

[0081] ;

[0082] in, is the time gradient of the event, For the direction of the event, The polar coordinate code for each event point.

[0083] In one possible implementation, adjusting the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration, and event density gradient includes:

[0084] The temporal gradient of an event is calculated according to the following formula :

[0085] ;

[0086] in, Code the time of the event, For events and time The second derivative of is the differential of time.

[0087] Step S3, inputting the event image into the spiking neural network to obtain an output feature map;

[0088] In one possible implementation, the spiking neural network includes four cascaded SNN blocks, i.e., convolution modules. Figure 2 A schematic diagram of the operation of a spiking neural network provided by an embodiment of the present invention.

[0089] Each convolution module simultaneously fuses the output of the previous moment to construct the temporal features of the target feature.

[0090] Specifically, the event image at the current moment is input into a convolution module, and the features of the current moment are obtained;

[0091] To align the frequency distribution of features in different environments, this paper proposes a wavelet domain cross-attention mechanism. The wavelet attention mechanism decomposes the features at the current moment into low-frequency features and high-frequency features, and then concatenates the low-frequency features and high-frequency features to obtain the output features at the current moment.

[0092] In order to align the features in time sequence, the features of the previous moment are convolved through the spatial displacement time attention mechanism to obtain the output features of the previous moment;

[0093] The output features of the current moment are fused with the output features of the previous moment through splicing and convolution modules to obtain a feature map.

[0094] Specifically, the wavelet domain cross-attention mechanism decomposes the input feature map F into low-frequency features Flf and high-frequency features Fhf through the wavelet transform (DWT). Because low-frequency features contain more background noise, while high-frequency features contain more discrete noise, this study uses a self-attention mechanism to extract important information from low-frequency and high-frequency features, respectively, and concatenates these outputs to obtain the final output through the inverse wavelet transform (IDWT).

[0095] The self-attention mechanism consists of three convolutions, mapping the input features to Q, K, and V, representing different feature distributions. Q is then transposed, multiplied by K, and softmaxed to produce an attention map. The attention map is further multiplied by V to produce the attention result, which is summed with the input to form the output.

[0096] Specifically, each channel of the input feature map will pass through a convolution to obtain a spatial offset, which will be encoded as a spatial offset coordinate and applied to the convolution to obtain an offset feature. An attention mechanism is then applied to the offset feature to obtain the final model output.

[0097] The wavelet-domain cross-attention mechanism proposed in this paper calculates a channel-space cross-attention matrix in the wavelet domain, enhances the response weights of sudden motion features, and aligns the sparsity of event images. By collaboratively calculating multi-scale wavelet decomposition and spatiotemporal attention weights, it overcomes the bottleneck of traditional neural networks' sensitivity to sudden motion features and effectively improves the signal-to-noise ratio of motion edge features in sudden acceleration / braking scenarios. In actual use, a discrete Fourier transform can also be used to optimize the density differences of event streams.

[0098] The present invention adopts the spatial displacement temporal attention mechanism through spatiotemporal position encoding and dynamic association modeling, which significantly improves the robustness of trajectory prediction and the accuracy of time series prediction in complex scenarios.

[0099] Step S4, obtaining the target's motion trajectory flow by detecting the feature map through the feature pyramid network and the detection head;

[0100] In one possible implementation, the detection head consists of three convolutional layers, which respectively output the detected category, confidence, coordinates, and offset of the bounding box; the feature map is upsampled step by step to obtain image interpolation; the image interpolation is fused with bilinear interpolation aligned with the channel to obtain a fused feature map; the fused feature map is optimized by the detection head to obtain the target's motion trajectory flow.

[0101] Among them, the feature pyramid uses a top-down path and horizontal cross-layer connections to gradually upsample and fuse the multi-scale feature maps output by multiple SNN Blocks with bilinear interpolation of channel alignment. The high-level semantic features are element-wise added with shallow high-resolution features through 2× nearest neighbor interpolation, and a 1×1 convolution kernel is introduced to achieve cross-channel information interaction. The final output fusion feature map aggregates the global context while retaining spatial details, forming a unified representation space with multi-scale perception capabilities, effectively solving the size sensitivity problem in target detection tasks.

[0102] In one possible implementation, the detection head optimizes the fused feature map to obtain the target's motion trajectory flow, including:

[0103] The fused feature map is optimized according to the following formula:

[0104] ;

[0105] in, The target trajectory flow, is the weight of the coordinates and the offset of the bounding box, is the weight of the coordinates and the confidence of the bounding box, is the weight of the coordinate and the category loss of the bounding box, is the coordinate and the offset of the bounding box, is the coordinate and confidence of the bounding box, is the category loss of coordinates and bounding boxes.

[0106] In one possible implementation, the detection head optimizes the fused feature map to obtain the target's motion trajectory flow, including:

[0107] Calculate the coordinates and the offset of the bounding box according to the following formula :

[0108] ;

[0109] in, is the intersection-over-union ratio of the predicted target bounding box and the true target bounding box, is the Euclidean distance between the center point of the predicted target bounding box and the center point of the true target bounding box, is the diagonal length of the minimum predicted target bounding box and the true target bounding box, To predict the target bounding box, is the true target bounding box;

[0110] Calculate the coordinates and confidence of the bounding box according to the following formula :

[0111] ;

[0112] in, is the predicted grid resolution, The number of prediction targets for each grid, is the true target confidence, is the target confidence of the prediction, and is the coordinate index of the grid;

[0113] The coordinates and the category loss of the bounding box are calculated according to the following formula :

[0114] ;

[0115] in, is the number of categories, For category index, For the predicted category, For the real category, is the focus parameter.

[0116] Step S5: Perform nonlinear model predictive control based on the motion trajectory flow to obtain the target optimized motion trajectory flow.

[0117] In complex dynamic scenarios, nonlinear modeling of short-term motion trajectories requires a balance between real-time performance, accuracy, and robustness. Traditional linear models struggle to capture sudden motion changes such as sudden acceleration and sudden turns, while global nonlinear models, due to their high computational complexity, struggle to meet real-time requirements. To address this, this paper proposes a solution that integrates hierarchical modeling with dynamic correction. This solution achieves highly accurate, adaptive trajectory modeling through four core steps.

[0118] In one possible implementation, the time window length of the motion trajectory flow is dynamically adjusted according to the jerk, i.e., the acceleration change rate, and the sliding overlap mechanism, targeting the local nonlinear characteristics of the motion trajectory;

[0119] Specifically, the traditional fixed time window performs stably in uniform speed or uniform acceleration scenarios, but cannot quickly respond to instantaneous behaviors such as emergency braking and sudden changes in direction. The present invention dynamically adjusts the length of the time window by calculating the jerk in real time: when the jerk increases significantly (such as emergency obstacle avoidance of a vehicle), the window length is automatically shortened to capture instantaneous motion details; when the motion tends to be stable (such as drone cruising), the window is extended to reduce the computational load. At the same time, a sliding overlap mechanism is introduced to ensure the continuity of the trajectory when the window is switched, avoiding data breakage problems caused by segmented modeling. This dynamic partitioning strategy can not only ensure a sensitive response to sudden movements, but also effectively balance computational efficiency when hardware resources are limited.

[0120] Construct a nonlinear motion model based on a cubic polynomial according to the length of the time window;

[0121] Specifically, based on dynamic window partitioning, a nonlinear motion model based on cubic polynomials is constructed. This cubic polynomial model mathematically characterizes the time-varying acceleration characteristics (such as the nonuniform acceleration of a vehicle during a sharp turn), and its degrees of freedom provide sufficient flexibility for trajectory fitting. The model parameters of the nonlinear motion model are determined by integrating physical dynamic constraints with data-driven optimization. Firstly, theoretical acceleration derived from Newton's laws of motion is introduced as a priori constraints to avoid physical implausibilities caused by purely data-driven optimization (such as vehicle instantaneous speed exceeding the limit). Secondly, a regularized least squares method is used to fit the observed data to mitigate the risk of overfitting caused by sensor noise. This hybrid physics-data modeling approach preserves the complex nonlinear characteristics of actual motion while ensuring the model's dynamic feasibility.

[0122] This paper, based on a nonlinear modeling approach using cubic polynomials and physical dynamic constraints, overcomes the problem of inaccurate predictions from traditional linear models for short-term, sudden trajectories. The coefficient matrix is ​​optimized and solved using the Newton-Raphson iterative method. A dual gain adjustment mechanism for model prediction and sensor observation is designed, automatically switching to observation-dominated mode under unusual disturbances to achieve alignment of spatial coordinates over time.

[0123] Predict the motion trajectory flow according to the nonlinear motion model to obtain the predicted trajectory;

[0124] Specifically, a multi-level optimization strategy is used in the trajectory fitting phase to enhance modeling robustness. First, the raw sensor data (such as GPS position and IMU acceleration) is spatiotemporally aligned and outliers are filtered to eliminate hardware transmission delays and outlier interference. Model parameter estimation is then performed using iterative weighted least squares, assigning greater weight to high-confidence data points (such as high-precision position measurements from lidar). High-frequency jitter noise is also suppressed using L1 regularization constraints. To address kinematic parameter jumps (such as sudden robot stops), a sliding window-based local parameter smoother is designed. While preserving the characteristics of motion mutations, the model parameters of adjacent windows are optimized for consistency, avoiding trajectory step-like phenomena caused by independent window modeling.

[0125] In addition to the nonlinear motion model, the present invention can also use a multi-layer neural network for modeling as a trajectory fitting solution.

[0126] To overcome the contradiction between the cumulative error of the model's long-term prediction and the instantaneous noise of the sensor, a dual-modal Kalman filter correction framework is designed. The predicted trajectory is corrected according to the Kalman filter to obtain the optimized motion trajectory flow of the target.

[0127] In normal motion mode, the predicted output of the aforementioned nonlinear model serves as the Kalman filter's prior estimate. Multi-source sensor observation data (such as relative pose from the visual odometry and velocity from the millimeter-wave radar) is dynamically integrated through a covariance adaptive adjustment mechanism. The Kalman gain matrix balances the weighting of model predictions with real-time data. When a dramatic motion disturbance is detected (such as a sudden increase in model residuals due to sudden braking), the system automatically switches to observation-dominated mode, temporarily de-weighting the model predictions and prioritizing state updates based on high-frequency sensor data to prevent trajectory divergence caused by model mismatch. Furthermore, a kinematic plausibility check module is introduced to verify the corrected trajectory for physical rules such as velocity continuity and acceleration bounds, further ensuring the reliability of the output trajectory.

[0128] Example 1

[0129] The experimental environment runs on Windows 10 (64-bit), an Intel Core i7-8750H CPU, 32GB of RAM, a 932GB mechanical hard drive, and an Nvidia RTX 3090 graphics card. Python 3.6.0 is used as the development language, and the machine learning environment includes numpy 1.19.5, tensorflow 1.2.0, pandas 1.1.5, torch 1.7.1, and Scikit-learn 1.0.

[0130] The simulation uses the Gen 1 event camera autonomous driving dataset. The Gen 1 dataset was recorded using a PROPHESEE GEN1 sensor mounted on the vehicle dashboard, with a resolution of 304 × 240 pixels. Labels were manually obtained using the grayscale estimation function of the ATIS camera. The Gen 1 dataset contains 39 hours of open road footage and a variety of driving scenarios, including urban, highway, suburban, and rural areas. The parameter settings are shown in Table 1.

[0131] Table 1

[0132]

[0133] Figure 3 A schematic diagram of the simulation of the model training process provided by an embodiment of the present invention, Figure 3 In it, Total Loss is the convergence of the loss function, Cls Loss is the loss of target classification, IoU Loss is the bounding box regression loss, and Obj Loss is the target discrimination loss.

[0134] Figure 4 The following diagram shows experimental results from an embodiment of the present invention, showing a PR curve of the prediction results. This curve is intended to evaluate the changes in precision and recall at different thresholds. The vertical axis represents precision, the horizontal axis represents recall, and the area enclosed by the PR curve represents the AP metric. Table 2 shows a comparison of the model of the present invention with other models.

[0135] Table 2

[0136]

[0137] In the experiment, we use Average Precision, Precision, Recall, and Running Time as the criteria for judging the performance of the model.

[0138] Among them, YOLOX: open-sourced by Megvii Technology and others, integrates progress in target detection such as decoupling head, data enhancement, anchor-free, and label classification with YOLO. Experiments on various datasets have proved that YOLOX has excellent generalization and efficiency.

[0139] SNN-YOLO: Proposed in 2024, it is the SNN version of YOLO, used for target detection in event cameras, with excellent computing speed, but lower performance than the original YOLO.

[0140] The model of the present invention integrates the speed advantage of SNN and significantly improves detectability through multiple improvements.

[0141] The present invention also provides a low-latency moving target detection system for navigation, characterized in that it is applied to any of the above-mentioned low-latency moving target detection methods, including:

[0142] The acquisition module is used to collect event streams through the vehicle-mounted event camera;

[0143] The time window adjustment module is used to adjust the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain the event image;

[0144] A feature map extraction module is used to input the event image into the pulse neural network to obtain the output feature map;

[0145] The motion trajectory flow detection module is used to obtain the target's motion trajectory flow through the feature pyramid network and the detection head detection feature map;

[0146] The motion trajectory flow optimization module is used to perform nonlinear model predictive control based on the motion trajectory flow to obtain the target optimized motion trajectory flow.

[0147] The low-latency moving target detection method and system for navigation provided by the present invention addresses the problem of sparsity and variability of event streams. The event stream collected by the on-board event camera is extracted into an event window by a speed-driven time window adaptation module in combination with the vehicle's motion speed and constructed as an event image. In order to construct the feature map, the present invention proposes that the event image will be gradually input into 4 SNN Blocks, and each SNN Block simultaneously fuses the output of the previous moment to construct the temporal correlation of the target features. In order to align the frequency domain and time series features, the present invention proposes a wavelet domain cross-attention mechanism and a spatial displacement time attention mechanism used in the SNN Block. The trajectory of the target is predicted by a feature pyramid and a detection head. In order to optimize the predicted trajectory of the target, the present invention further introduces nonlinear modeling of short-term motion trajectories, ultimately achieving low-latency and high-precision time series tracking of fast moving targets.

[0148] This paper designs an extremely lightweight spiking neural network (SNN) to directly process the asynchronous spatiotemporal signals of event streams, avoiding the computational redundancy of traditional image format conversion and complex models. This method combines the physical laws of short-term trajectories (such as uniform speed / uniform acceleration patterns and steering angle continuity) to construct a dynamic prediction model, achieving ultra-low-latency target detection and trajectory extrapolation, significantly improving the real-time response capability to high-speed and sudden-moving targets and the detection robustness in complex lighting environments.

[0149] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A low-latency moving target detection method for navigation, characterized in that: include: Collect event streams through vehicle-mounted event cameras; The time window length of the event stream is adjusted in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain an event image; Inputting the event image into a pulse neural network to obtain an output feature map; The feature map is detected by a feature pyramid network and a detection head to obtain a motion trajectory flow of the target; Performing nonlinear model predictive control according to the motion trajectory flow to obtain an optimized motion trajectory flow of the target; The real-time adjustment of the time window length of the event stream according to the vehicle speed, vehicle acceleration and event density gradient includes: The time window length is adjusted according to the following formula: Where ΔT is the time window length, T min is the upper limit length of the time window, T max is the lower limit length of the time window, τ high is the upper threshold of speed, τ low is the lower threshold of speed, τ accel is the upper threshold of acceleration, τ decel is the lower threshold of acceleration, is the two-norm of the spatial gradient of the event, is the temporal gradient of the event, k is the scaling factor, and α is the parameter balancing space and time; The spiking neural network includes four cascaded convolution modules; the feature map obtained by inputting the event image into the spiking neural network includes: Input the event image at the current moment into a convolution module and obtain the features of the current moment; Decomposing the features at the current moment into low-frequency features and high-frequency features through a wavelet attention mechanism, and concatenating the low-frequency features and high-frequency features to obtain the output features at the current moment; The features of the previous moment are convolved through the spatial displacement time attention mechanism to obtain the output features of the previous moment; The output features of the current moment are fused with the output features of the previous moment through splicing and convolution modules to obtain a feature map.

2. The low-latency moving target detection method according to claim 1, wherein: The real-time adjustment of the time window length of the event stream according to the vehicle speed, vehicle acceleration and event density gradient includes: The spatial gradient of events is calculated according to the following formula in, is the time gradient of the event, θ is the event direction, and N(θ) is the polar coordinate encoding of each event point.

3. The low-latency moving target detection method according to claim 1, wherein: The real-time adjustment of the time window length of the event stream according to the vehicle speed, vehicle acceleration and event density gradient includes: The temporal gradient of an event is calculated according to the following formula Where E(t) is the time code of the event, is the second-order derivative of the event with respect to time t, and δt is the differential of time.

4. The low-latency moving target detection method according to claim 1, wherein: The detection head consists of three convolutional layers, which output the detected category, confidence, coordinates, and offset of the bounding box respectively; The detecting the feature map by using the feature pyramid network and the detection head to obtain the target's motion trajectory flow includes: Upsampling the feature map step by step to obtain image interpolation; Fusing the image interpolation with the channel-aligned bilinear interpolation to obtain a fused feature map; The fusion feature map is optimized by the detection head to obtain the target's motion trajectory flow.

5. The low-latency moving target detection method according to claim 4, wherein: The optimizing the fused feature map by the detection head to obtain the target motion trajectory flow includes: The fused feature map is optimized according to the following formula: L total =λ coord ·L bbox +λ obj ·L confidence +λ class ·L classification Among them, L total is the target trajectory flow, λ coord is the weight of the coordinates and the offset of the bounding box, λ obj is the weight of the coordinates and the confidence of the bounding box, λ class is the weight of the coordinate and the category loss of the bounding box, L bbox is the coordinate and the offset of the bounding box, L confidence is the coordinate and confidence of the bounding box, L classification is the category loss of coordinates and bounding boxes.

6. The low-latency moving target detection method according to claim 5, characterized in that: The optimizing the fused feature map by the detection head to obtain the target motion trajectory flow includes: Calculate the coordinates and the offset L of the bounding box according to the following formula bbox : Among them, IoU is the intersection-over-union ratio of the predicted target bounding box and the true target bounding box, ρ 2 is the Euclidean distance between the center point of the predicted target bounding box and the center point of the true target bounding box, c is the diagonal length of the minimum predicted target bounding box and the true target bounding box, b pred is the predicted target bounding box, b gt is the true target bounding box; Calculate the coordinates and the confidence L of the bounding box according to the following formula confidence : Among them, S is the predicted grid resolution, B is the number of predicted targets in each grid, and C ij is the true target confidence, is the predicted target confidence, i and j are the coordinate indices of the grid; The coordinates and the category loss L of the bounding box are calculated according to the following formula classification : Among them, K is the number of categories, h is the category index, is the predicted category, p h is the true category and γ is the focusing parameter.

7. The low-latency moving target detection method according to claim 1, wherein: The performing of nonlinear model predictive control according to the motion trajectory flow to obtain the target optimized motion trajectory flow includes: Dynamically adjust the time window length of the motion trajectory flow according to the jerkiness and sliding overlap mechanism; A nonlinear motion model based on a cubic polynomial is constructed according to the time window length; model parameters of the nonlinear motion model are determined by integrating physical dynamics constraints with data-driven optimization: Predicting the motion trajectory flow according to the nonlinear motion model to obtain a predicted trajectory; The predicted trajectory is corrected according to the Kalman filter to obtain an optimized motion trajectory flow of the target.

8. A low-latency moving target detection system for navigation, characterized in that: The low-latency moving target detection method according to any one of claims 1 to 7 comprises: The acquisition module is used to collect event streams through the vehicle-mounted event camera; The time window adjustment module is used to adjust the time window length of the event stream in real time according to the vehicle speed, vehicle acceleration and event density gradient to obtain the event image; A feature map extraction module, configured to input the event image into a pulse neural network to obtain an output feature map; A motion trajectory flow detection module is used to detect the feature map through a feature pyramid network and a detection head to obtain the motion trajectory flow of the target; A motion trajectory flow optimization module is used to perform nonlinear model predictive control based on the motion trajectory flow to obtain an optimized motion trajectory flow of the target; The time window adjustment module is further configured to adjust the time window length according to the following formula: Where ΔT is the time window length, T min is the upper limit length of the time window, T max is the lower limit length of the time window, τ high is the upper threshold of speed, τ low is the lower threshold of speed, τ accel is the upper threshold of acceleration, τ decel is the lower threshold of acceleration, is the two-norm of the spatial gradient of the event, is the temporal gradient of the event, k is the scaling factor, and α is the parameter balancing space and time; The feature map extraction module is further used to input the event image at the current moment into a convolution module and obtain the features of the current moment; Decomposing the features at the current moment into low-frequency features and high-frequency features through a wavelet attention mechanism, and concatenating the low-frequency features and high-frequency features to obtain the output features at the current moment; The features of the previous moment are convolved through the spatial displacement time attention mechanism to obtain the output features of the previous moment; The output features of the current moment are fused with the output features of the previous moment through splicing and convolution modules to obtain a feature map.

Images