A method and system for detecting small moving targets of an event camera based on topological constraints

Abstract

The application discloses a kind of event camera motion small target detection method and system based on topological constraint, it is related to computer vision technical field, including: event stream is discretized and voxelized processing, generates three-dimensional sparse tensor;Space topology network including embedding topological learning module and space-time consistency module is constructed, and event point cloud is adaptively key point screening using grid sampling unit;Local topological features are extracted by graph attention, space-time consistency enhancement and feature interpolation unit, and then global context extraction, local space attention generation and feature fusion obtain space-time consistency enhancement features;Predictive probability map is generated based on the features, and with real label extraction predicted point cloud and real point cloud to construct target loss function, optimize network after forward inference, and output motion small target detection result;Motion small target refers to the target that size meets preset small target determination condition, improve the efficiency of motion small target detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for detecting small moving targets using an event camera based on topological constraints. Background Technology

[0002] Currently, traditional frame-based cameras, limited by fixed frame rates and finite dynamic range, are prone to motion blur and information loss when capturing high-speed targets, making them unsuitable for high-speed, small-target detection. Event cameras, as a novel visual sensor, can capture pixel brightness changes and output asynchronous event streams with microsecond-level response times. They possess significant advantages such as high temporal resolution, high dynamic range, and no motion blur, making them valuable for applications in high-speed moving target detection and low-light environment perception. Currently, target detection technologies based on event cameras are mainly divided into two major directions: dense representation and sparse representation. The current development status of each direction is as follows: Early techniques accumulated events into frame-like representations, directly reusing mature image detection algorithms, but this approach lost the original temporal precision of the events. Subsequent developments included voxel grid representation techniques, which generated multi-channel tensors by quantizing the temporal dimension, preserving temporal information to some extent, but still limited to short time windows and lacking sufficient temporal context information. To capture long-term temporal dependencies, researchers introduced recurrent neural networks (RNNs) to model temporal dynamics; recent Transformer-based methods have further enhanced long-distance correlation modeling capabilities through self-attention mechanisms. However, dense representation techniques have inherent drawbacks: the densification process inevitably sacrifices the original microsecond-level temporal precision of events, introducing motion blur; simultaneously, converting sparse events into dense tensors introduces redundant background data, significantly increasing computational overhead.

[0003] To preserve the sparsity and high temporal resolution of events, researchers proposed a technical approach based on graphs, spiking neural networks (SNNs), and point clouds. Graph-based methods model events as dynamic spatiotemporal graphs and aggregate local features through message passing; SNNs directly process asynchronous impulses to reduce power consumption, but rely on dedicated hardware and have limited performance in complex scenarios; point cloud-based methods model events as 3D spatiotemporal point clouds and utilize sparse convolutions to avoid denser overhead, but only focus on local neighborhood aggregation and lack explicit modeling of global trajectory continuity, leading to frequent missed detections of targets at turning points or in occluded scenarios, and serious discontinuities in the detected trajectory.

[0004] As can be seen from the above, how to improve the efficiency of detecting small moving targets in the process of event camera moving targets based on topological constraints is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method and system for detecting moving small targets using an event camera based on topological constraints, which can improve the efficiency of detecting moving small targets in the process of detecting moving small targets using an event camera based on topological constraints. The specific solution is as follows: Firstly, this application provides a method for detecting small moving targets using an event camera based on topological constraints, including: Discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and voxelize each event block to obtain the corresponding three-dimensional sparse tensor. A spatial topology network, including an embedded topology learning module and a spatiotemporal consistency module, is constructed based on a sparse convolutional encoder-decoder. The grid sampling unit in the embedded topology learning module is used to adaptively filter key points of the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The key event point set is used to extract features by the graph attention unit, spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module to obtain local topological features. The local topological features are then processed by the global context extraction branch, local spatial attention generation branch and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. A predicted probability map is generated using the spatial topology network and based on the spatiotemporal consistency enhancement features. Predicted point clouds and real point clouds are extracted from the predicted probability map and the corresponding real labels based on preset thresholds. Then, a target loss function is determined based on the predicted point clouds and the real point clouds. The spatial topology network is optimized based on the target loss function. The optimized network is then used to perform forward inference on the asynchronous event stream to obtain the detection results of small moving targets in the asynchronous event stream. The small moving targets are moving targets whose size meets the preset small size judgment conditions.

[0006] Optionally, feature extraction is performed on the set of key event points using the graph attention unit in the embedded topology learning module, including: The graph attention unit in the embedded topology learning module is determined, and the three-dimensional sparse tensor is converted into a vector to be processed, including a query vector, a key vector, and a value vector, using the linear projection layer in the graph attention unit and based on a preset shared weight matrix. The edge weight encoding layer in the graph attention unit is used to determine the inverse distance weights corresponding to each adjacent event point in the key event point set. Then, a preset two-layer perceptron is used to map each adjacent event point based on the inverse distance weights to obtain geometric embedding features that satisfy the preset high-dimensional conditions. The attention weight calculation layer in the graph attention unit is used to fuse the vector to be processed and the geometric embedding features to obtain a fusion result, and the attention weight for edge perception is determined based on the fusion result.

[0007] Optionally, the spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module are used to extract features from the set of key event points to obtain local topological features, including: The attention weights and the value vectors are weighted and aggregated to obtain the aggregation result. Then, the aggregation result is subjected to layer normalization and residual connection operations to obtain the topology enhancement feature. The spatial consistency enhancement unit is used to perform spatial consistency and correlation enhancement operations on the topology enhancement features to obtain the enhancement result. The feature interpolation unit is then used to map the enhancement result to the resolution of the original event point cloud to obtain local topology features.

[0008] Optionally, the local topological features are processed using the global context extraction branch and the local spatial attention generation branch in the spatiotemporal consistency module, including: The global context extraction branch in the spatiotemporal consistency module is used to perform global average pooling on the local topological features to obtain the corresponding global context features. Local spatial attention is used to generate branches to determine a first learnable weight matrix and a second learnable weight matrix corresponding to the local topological features. Then, the Sigmoid activation function and ReLU activation are used to determine the data to be processed based on the local topological features, the first learnable weight matrix, and the second learnable weight matrix.

[0009] Optionally, the local topological features are processed using the feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features, including: The data to be processed is multiplied element-wise with the global context feature to obtain the scaling result. The scaling result is then scaled based on a preset scaling factor to obtain spatial enhancement features. The global context features and the spatial enhancement features are fused using a feature fusion unit to obtain a fusion result. The fusion result is then residually connected with the local topological features using the feature fusion unit to obtain spatiotemporal consistency enhancement features.

[0010] Optionally, the step of generating a predicted probability map using the spatial topology network and based on the spatiotemporal consistency enhancement features, extracting predicted point clouds and real point clouds from the predicted probability map and the corresponding real labels based on preset thresholds, and then determining a target loss function based on the predicted point clouds and the real point clouds, includes: The spatiotemporal consistency enhancement features are processed using the spatial topology network to obtain a prediction probability map, and the corresponding prediction point cloud is extracted from the prediction probability map based on a preset prediction threshold. Based on a preset real threshold, the corresponding real point cloud is extracted from the real label corresponding to the predicted probability map. Then, a target loss function is constructed based on the predicted point cloud and the real point cloud using a topological similarity loss layer and a spatiotemporal continuity loss layer.

[0011] Optionally, optimizing the spatial topology network based on the target loss function includes: The topological similarity loss layer is used to determine the zero-dimensional and one-dimensional topological features between the predicted point cloud and the real point cloud, and a topological loss is constructed based on the zero-dimensional and one-dimensional topological features; the topological loss includes the Betty number loss for constraining the consistency of the number of topological features and the Wasserstein distance loss for constraining the consistency of the distribution of topological features. The spatiotemporal continuity loss layer is used to constrain the continuity of the predicted point cloud in the temporal and spatial dimensions to obtain the continuity constraint result. The binary cross-entropy loss between the predicted point cloud and the true label is determined. Then, the target loss function is used and the total model loss is determined based on the binary cross-entropy loss and the topology loss, so as to optimize the spatial topology network using the total model loss.

[0012] Secondly, this application provides a small moving target detection system based on topological constraints for event cameras, comprising: The event stream processing module is used to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and to voxelize each event block to obtain the corresponding three-dimensional sparse tensor. A spatial topology network construction module is used to construct a spatial topology network based on a sparse convolutional encoder-decoder, including an embedded topology learning module and a spatiotemporal consistency module, so as to use the grid sampling unit in the embedded topology learning module to adaptively filter the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The topology feature extraction module is used to extract features from the key event point set using the graph attention unit, spatial consistency enhancement unit, and feature interpolation unit in the embedded topology learning module to obtain local topology features. The local topology features are then processed using the global context extraction branch, local spatial attention generation branch, and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. The detection result generation module is used to generate a predicted probability map using the spatial topology network and based on the spatiotemporal consistency enhancement features. It then extracts predicted point clouds and real point clouds from the predicted probability map and their corresponding real labels based on preset thresholds. Next, it determines a target loss function based on the predicted point clouds and the real point clouds, optimizes the spatial topology network based on the target loss function, and uses the optimized network to perform forward inference on the asynchronous event stream to obtain the detection results of small moving targets in the asynchronous event stream. The small moving targets are moving targets whose size meets preset small size criteria.

[0013] As can be seen from the above, before performing small moving target detection by an event camera based on topological constraints, this application needs to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and then voxelize each event block to obtain the corresponding three-dimensional sparse tensor. A spatial topology network, including an embedded topology learning module and a spatiotemporal consistency module, is constructed based on a sparse convolutional encoder-decoder. The grid sampling unit in the embedded topology learning module is used to adaptively filter key points of the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The graph attention unit, spatial consistency enhancement unit, and feature interpolation unit in the embedded topology learning module are then used to perform key point filtering on the event point cloud. Feature extraction is performed on the key event point set to obtain local topological features. These local topological features are then processed by the global context extraction branch, local spatial attention generation branch, and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. A predicted probability map is generated using the spatial topology network and based on the spatiotemporal consistency enhanced features. Predicted point clouds and real point clouds are extracted from the predicted probability map and the corresponding real labels based on preset thresholds. A target loss function is then determined based on the predicted point clouds and real point clouds. The spatial topology network is optimized based on the target loss function, and the optimized network is used for forward inference on the asynchronous event stream to obtain the detection results of moving small targets in the asynchronous event stream.

[0014] Therefore, this application first needs to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and then voxelize each event block to obtain the corresponding three-dimensional sparse tensor. Secondly, a spatial topology network including an embedded topology learning module and a spatiotemporal consistency module is constructed based on a sparse convolutional encoder-decoder. The mesh sampling unit in the embedded topology learning module is used to adaptively filter key points in the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. Then, the graph attention unit, spatial consistency enhancement unit, and feature interpolation unit in the embedded topology learning module are used to perform feature interpolation on the set of key event points. The process involves extracting local topological features and processing these features using the global context extraction branch, local spatial attention generation branch, and feature fusion unit within the spatiotemporal consistency module to obtain enhanced spatiotemporal consistency features. Next, a predicted probability map is generated using a spatial topology network and these enhanced features. Predicted point clouds and real point clouds are extracted from the predicted probability map and their corresponding ground truth labels based on preset thresholds. A target loss function is then determined based on the predicted and real point clouds. Finally, the spatial topology network is optimized using this target loss function, and the optimized network is used for forward inference on the asynchronous event stream to obtain the detection results for moving small targets within the asynchronous event stream. This improves the efficiency of moving small target detection in topology-constrained event camera detection, thereby enhancing the user experience. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a flowchart of an event camera moving small target detection method based on topological constraints disclosed in this application; Figure 2 This is a schematic diagram illustrating the principle of a specific event camera-based moving small target detection method disclosed in this application, which is based on topological constraints. Figure 3 This is a flowchart illustrating the construction process of a specific SpTopoNet network disclosed in this application; Figure 4 This is a schematic diagram of a specific network architecture disclosed in this application; Figure 5 This is a schematic diagram of the structure of a specific topology learning module disclosed in this application; Figure 6This is a flowchart illustrating a specific spatiotemporal consistency module disclosed in this application. Figure 7 This is a schematic diagram of the structure of an event camera moving small target detection system based on topological constraints disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Currently, traditional frame cameras are limited by fixed frame rates and limited dynamic range, which can easily lead to motion blur and information loss when shooting high-speed targets, making it difficult to meet the needs of high-speed small target detection.

[0019] Early techniques accumulated events into frame-based representations, directly reusing mature image detection algorithms, but this approach lost the original temporal precision of the events. Subsequent developments included voxel mesh representation techniques, which generated multi-channel tensors by quantizing the temporal dimension, preserving temporal information to some extent, but still limited to short time windows and lacking sufficient temporal context information. Therefore, this application provides a topological constraint-based method for detecting small moving targets in event cameras, which can improve the efficiency of detecting small moving targets in topological constraint-based event camera moving target detection processes.

[0020] See Figure 1 As shown, this embodiment of the invention discloses a method for detecting small moving targets using an event camera based on topological constraints, including: Step S11: Discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and voxelize each event block to obtain the corresponding three-dimensional sparse tensor.

[0021] In this embodiment, the aim of this application is to construct a detection model that combines high temporal accuracy, trajectory continuity, and robustness in complex scenarios. It is worth noting that the core network of this application embodiment is SpTopoNet, which is built based on sparse convolution. The Topological Learning Module (TLM) and the Spatio-Temporal Consistency Module (SCM) are key functional modules embedded in the network architecture. They work together to extract and optimize the features of moving small targets: the TLM segments the event point cloud and captures local spatio-temporal patterns within trajectory segments through explicit spatio-temporal structure learning, extracting local topological features of the small targets; the SCM models long-distance dependencies between different trajectory segments through sparse convolution, extracting global spatial correlation features of the small targets; finally, the learning process of the two types of features is constrained by the topological loss function (EvTopoLoss), penalizing structural discontinuities in the event point cloud, forcing global topological consistency of the trajectory, and combining the efficient computational power of sparse convolution to achieve high-precision and continuous detection of moving small targets.

[0022] It is worth mentioning that the method principle diagram of this application embodiment is as follows: Figure 2 As shown, it includes four main steps: (1) event data preprocessing; (2) SpTopoNet network construction; (3) model training and optimization; and (4) object detection inference.

[0023] Specifically: (1) Event data preprocessing: The asynchronous event stream output by the event camera is preprocessed, discretized into event blocks according to a fixed time window, and voxelized into three-dimensional sparse tensors to adapt to the network input. Conventional data augmentation operations are performed on the training set data to improve generalization ability. The corresponding small target dataset of the event camera is used and divided into training set, validation set and test set according to the proportion to complete the data preparation.

[0024] (2) SpTopoNet network construction: The construction flowchart of the SpTopoNet network is as follows. Figure 3 As shown: The framework is based on a sparse convolutional encoder-decoder, embedding a topology learning module (TLM) and a spatiotemporal consistency module (SCM). The TLM is responsible for extracting local topological features, filtering key event points through sampling, and calculating attention weights by fusing feature similarity and spatial proximity. After feature aggregation and post-processing, the output features are presented. The SCM is concatenated after the TLM and extracts global context and local attention weights through a two-branch structure. After fusion, the global correlation features are enhanced.

[0025] (3) Model training optimization: An adaptive optimizer and learning rate decay strategy are adopted, and the EvTopoLoss total loss is constructed by combining the binary classification cross-entropy loss and the topology loss function to adjust the strength of the topology constraint. A reasonable training batch and number of iterations are set, and the model weights are saved periodically during the process. The weights with the best performance on the validation set are selected as the final training model.

[0026] (4) Target detection reasoning: Input the preprocessed test set data into the trained model, extract features, fuse and decode the prediction results, and output the final small target detection results (detection rate, false alarm rate).

[0027] Step S12: Construct a spatial topology network based on a sparse convolutional encoder-decoder, including an embedded topology learning module and a spatiotemporal consistency module, so as to use the grid sampling unit in the embedded topology learning module to adaptively filter the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points.

[0028] In this embodiment, Figure 4 The diagram illustrates the overall network architecture of this application embodiment, comprising three core components: first, a Topology Learning Module (TLM); second, a Spatiotemporal Consistency Module (SCM); and third, a Topology Loss Function (EvTopoLoss). The TLM and SCM are core functional modules embedded in the network infrastructure, working together to extract local topological features of moving small targets and model global spatial correlation features, respectively. The EvTopoLoss function serves as an optimization constraint during model training, penalizing structural discontinuities in the event point cloud to enforce global topological consistency of the detected trajectory.

[0029] The Topology Learning Module (TLM) is used for local topological feature extraction. Figure 5 This is a schematic diagram of the structure corresponding to the topology learning module. Considering the representation of the local spatiotemporal topology of the motion trajectory, this embodiment constructs a topology learning module (TLM) to extract local topological features from sparse, non-uniform event point clouds. Notably, the topology learning module (TLM) consists of a grid sampling unit, a graph attention unit, a spatial consistency enhancement unit, and a feature interpolation unit. The graph attention unit is the core unit of the topology learning module, consisting of a linear projection layer, an edge weight encoding layer, an attention weight calculation layer, and a feature aggregation layer.

[0030] Furthermore, the network can first use grid sampling units to adaptively filter sparse and non-uniform event point clouds, thereby achieving accurate extraction of key event points.

[0031] Step S13: Use the graph attention unit, spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module to extract features from the key event point set to obtain local topological features. Then use the global context extraction branch, local spatial attention generation branch and feature fusion unit in the spatiotemporal consistency module to process the local topological features to obtain spatiotemporal consistency enhanced features.

[0032] In this embodiment, the input features are processed through a linear projection layer and then through a shared weight matrix. Convert to query vector Key vector Sum value vector Simultaneously, the inverse distance weights of adjacent points are calculated using the edge weight encoding layer. And through two layers of sensors This is mapped to high-dimensional geometric features; subsequently, the attention weights are obtained by fusing feature similarity and spatial proximity through an attention weight calculation layer. The calculation formula is as follows: ; in, Inverse distance encoding for adjacent event points Representative point of Nearest neighbor set A two-layer perceptron is used to convert edge weights into high-dimensional features. Then, through weighted aggregation, combined with layer normalization and residual connections, topologically enhanced features are obtained, calculated as follows: ; It is worth mentioning that the edge-aware attention mechanism can enhance the representation ability of topological structure by jointly considering feature similarity and geometric proximity, enabling the network to effectively learn the spatiotemporal topological relationships between event points in local trajectories.

[0033] Furthermore, after completing the graph attention feature calculation, the module can further perform spatial consistency enhancement operations and strengthen the correlation and consistency of features within the neighborhood. Finally, through the interpolation processing of the feature interpolation unit, the enhanced topological features are mapped back to the original resolution of the event point cloud, ensuring the adaptability of the feature dimension to subsequent modules and providing fine-grained local topological feature support for global trajectory modeling.

[0034] Specifically, feature extraction of the key event point set using the graph attention unit in the embedded topology learning module can include: determining the graph attention unit in the embedded topology learning module, and using the linear projection layer in the graph attention unit and based on a preset shared weight matrix to convert the three-dimensional sparse tensor into a vector to be processed including a query vector, a key vector, and a value vector; using the edge weight encoding layer in the graph attention unit to determine the inverse distance weights corresponding to each adjacent event point in the key event point set, so as to map each adjacent event point using a preset two-layer perceptron and based on each inverse distance weight to obtain geometric embedding features that meet preset high-dimensional conditions; using the attention weight calculation layer in the graph attention unit to fuse the vector to be processed and the geometric embedding features to obtain a fusion result, and determining the attention weights for edge perception based on the fusion result.

[0035] Furthermore, in this embodiment, the spatial consistency enhancement unit and feature interpolation unit embedded in the topology learning module are used to extract features from the key event point set to obtain local topological features. This can include: weighted aggregation of attention weights and value vectors to obtain aggregation results, and layer normalization and residual connection operations on the aggregation results to obtain topological enhancement features; spatial consistency and correlation enhancement operations are performed on the topological enhancement features using the spatial consistency enhancement unit to obtain enhancement results, and the enhancement results are mapped to the resolution of the original event point cloud using the feature interpolation unit to obtain local topological features.

[0036] In this embodiment, the flowchart corresponding to the Spatiotemporal Consistency Module (SCM) is as follows: Figure 6 As shown: From the perspective of long-term topological constraints and global spatiotemporal consistency of motion trajectories, this application embodiment constructs a spatiotemporal consistency module (SCM) to model long-distance dependencies between different trajectory segments.

[0037] The Spatiotemporal Consistency Module (SCM) consists of a global context extraction branch, a local spatial attention generation branch, and a feature fusion unit. The feature fusion unit is the core unit of the SCM and consists of a residual connection layer and an element-wise multiplication layer.

[0038] Subsequently, the local topological features output by the topology learning module are input into the dual parallel branches; the upper branch extracts global contextual features through a global average pooling operation, calculated as shown in the following formula: ; The lower branch generates pointwise spatial attention weights through a sequential process of "linear transformation → ReLU activation → linear transformation → Sigmoid activation" (i.e., the L→R→L→S process shown in the diagram). The final spatial augmentation features are calculated using the following formula: ; in, and For learnable weight matrix, The Sigmoid activation function is used to implement adaptive feature modulation. This represents element-wise multiplication. This is the scaling factor.

[0039] It is worth mentioning that the above mechanism enables each event point to selectively fuse global context information based on its own local features, while preserving input feature details through residual connections and effectively suppressing the interference of spatial outliers, thereby improving the spatiotemporal consistency of features across the entire spatiotemporal range. Understandably, the above module completes global feature correlation modeling based on sparse convolution, ensuring computational efficiency while overcoming the locality limitations of traditional sparse detection methods. Working synergistically with the local topological features output by TLM, it significantly enhances the coherence of long-term trajectories of small moving targets and improves detection stability in complex scenarios such as occlusion and high-speed motion.

[0040] Specifically, processing local topological features using the global context extraction branch and the local spatial attention generation branch in the spatiotemporal consistency module can include: performing global average pooling on the local topological features using the global context extraction branch in the spatiotemporal consistency module to obtain the corresponding global context features; and using the local spatial attention generation branch to determine the first and second learnable weight matrices corresponding to the local topological features, so as to determine the data to be processed based on the local topological features, the first and second learnable weight matrices, and the Sigmoid activation function and ReLU activation.

[0041] Furthermore, the feature fusion unit in the spatiotemporal consistency module processes local topological features to obtain spatiotemporal consistency enhanced features. This can include: multiplying the data to be processed element-wise with the global context features to obtain the scaling result, scaling the scaling result based on a preset scaling factor to obtain spatial enhanced features; fusing the global context features and spatial enhanced features using the feature fusion unit to obtain the fusion result, and then using the feature fusion unit to perform residual connection between the fusion result and the local topological features to obtain spatiotemporal consistency enhanced features.

[0042] Step S14: Utilize the spatial topology network and generate a predicted probability map based on the spatiotemporal consistency enhancement feature. Extract predicted point clouds and real point clouds from the predicted probability map and their corresponding real labels based on preset thresholds. Then, determine a target loss function based on the predicted point clouds and the real point clouds. Optimize the spatial topology network based on the target loss function. Use the optimized network to perform forward inference on the asynchronous event stream to obtain the detection results of small moving targets in the asynchronous event stream. The small moving target is a moving target whose size meets a preset small size determination condition.

[0043] In this embodiment, considering the model optimization objective, the present application constructs a topology loss function, EvTopoLoss, to achieve global topological consistency constraints on the trajectory of detected moving small targets. The topology loss function EvTopoLoss consists of a topological structure similarity loss layer and a spatiotemporal continuity loss layer, which work together to constrain the trajectory topology.

[0044] To further enhance the scientific validity and effectiveness of topology constraints, this application introduces a topology-aware mechanism based on persistent coherence and explicitly regularizes the network learning process to ensure that the model retains the correct topology for the motion trajectory.

[0045] The aforementioned topological loss function is applied to the spatiotemporal point cloud generated from the prediction results and the ground truth labels, and is specifically defined as follows: ; in, Indicates the predicted probability. Indicates the true label, and These are threshold parameters for the predicted results and the true labels, respectively, used to filter valid event points and construct a spatiotemporal point cloud. (Predicted point cloud) and (Real point cloud).

[0046] Specifically, a predicted probability map is generated using a spatial topology network and based on spatiotemporal consistency enhancement features. Predicted point clouds and real point clouds are extracted from the predicted probability map and their corresponding ground truth labels based on preset thresholds. Then, a target loss function is determined based on the predicted and real point clouds. This process may include: processing the spatiotemporal consistency enhancement features using a spatial topology network to obtain the predicted probability map, and extracting the corresponding predicted point clouds from the predicted probability map based on a preset prediction threshold; extracting the corresponding real point clouds from the ground truth labels corresponding to the predicted probability map based on a preset ground truth threshold; and then constructing the target loss function using a topological similarity loss layer and a spatiotemporal continuity loss layer based on the predicted and real point clouds.

[0047] In this embodiment, the EvTopoLoss topological loss function includes two complementary loss components, and the overall expression is as follows: ; in, and These correspond to 0-dimensional and 1-dimensional topological features, respectively. For Betty's losses, For Wasserstein distance loss, it is worth mentioning that Betty number loss is used to constrain... To ensure the correctness of the number of topological features, Wasserstein distance loss compares the predicted and the persistent graph of the real point cloud. This not only guarantees the correctness of the number of topological features, but also achieves accurate matching of the distribution of feature importance. The two work together to achieve comprehensive constraints on the topological structure.

[0048] To balance point-by-point detection accuracy with global topology consistency, this application embodiment constructs a topology loss function EvTopoLoss in conjunction with binary cross-entropy loss (BCE): ; in, This is the cross-entropy loss for binary classification, used to ensure the accuracy of point-by-point detection; This is the regularization coefficient, used to adjust the strength of topological constraints and achieve a dynamic balance between detection accuracy and topological consistency.

[0049] It is worth mentioning that the above loss function, together with the TLM and SCM modules, forms a complete technical link of "feature extraction - global association - topological constraint", which together ensures the continuity and correctness of the detection trajectory from the two dimensions of feature representation and model optimization, and significantly improves the detection rate and trajectory integrity of small moving targets in complex scenes.

[0050] Specifically, optimizing the spatial topology network based on the objective loss function can include: using a topology similarity loss layer to determine the zero-dimensional and one-dimensional topology features between the predicted point cloud and the real point cloud, and constructing a topology loss based on the zero-dimensional and one-dimensional topology features; the topology loss includes the Betty number loss for constraining the consistency of the number of topology features and the Wasserstein distance loss for constraining the consistency of the distribution of topology features; using a spatiotemporal continuity loss layer to constrain the continuity of the predicted point cloud in the temporal and spatial dimensions, obtaining the continuity constraint result, and determining the binary cross-entropy loss between the predicted point cloud and the real label; then using the objective loss function and based on the binary cross-entropy loss and the topology loss to determine the total model loss, so as to optimize the spatial topology network using the total model loss.

[0051] As can be seen from the above, the embodiments of this application first need to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and then voxelize each event block to obtain the corresponding three-dimensional sparse tensor; secondly, a spatial topology network including an embedded topology learning module and a spatiotemporal consistency module is constructed based on a sparse convolutional encoder-decoder, so as to use the grid sampling unit in the embedded topology learning module to adaptively filter the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points; then, the graph attention unit, spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module are used to perform feature interpolation on the set of key event points. The process involves extracting local topological features and then processing these features using the global context extraction branch, local spatial attention generation branch, and feature fusion unit within the spatiotemporal consistency module to obtain enhanced spatiotemporal consistency features. Next, a predicted probability map is generated using the spatial topology network and these enhanced features. Predicted point clouds and real point clouds are extracted from the predicted probability map and their corresponding ground truth labels based on preset thresholds. A target loss function is then determined based on the predicted and real point clouds. Finally, the spatial topology network is optimized using the target loss function, and the optimized network is used for forward inference on the asynchronous event stream to obtain the detection results of small moving targets within the asynchronous event stream. This improves the efficiency of small moving target detection in topology-constrained event camera detection, thereby enhancing the user experience.

[0052] Accordingly, see Figure 7 As shown, this application also provides an event camera moving small target detection system based on topological constraints, including: The event stream processing module 11 is used to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain a number of corresponding event blocks, and to voxelize each event block to obtain a corresponding three-dimensional sparse tensor. The spatial topology network construction module 12 is used to construct a spatial topology network including an embedded topology learning module and a spatiotemporal consistency module based on a sparse convolutional encoder-decoder, so as to use the grid sampling unit in the embedded topology learning module to adaptively filter the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The topology feature extraction module 13 is used to extract features from the key event point set using the graph attention unit, spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module to obtain local topology features, and to process the local topology features using the global context extraction branch, local spatial attention generation branch and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. The detection result generation module 14 is used to generate a predicted probability map using the spatial topology network and based on the spatiotemporal consistency enhancement feature, and to extract predicted point clouds and real point clouds from the predicted probability map and the corresponding real labels based on preset thresholds, respectively. Then, a target loss function is determined based on the predicted point cloud and the real point cloud, and the spatial topology network is optimized based on the target loss function. The optimized network is then used to perform forward inference on the asynchronous event stream to obtain the detection result of the moving small target in the asynchronous event stream; the moving small target is a moving target whose size meets the preset small size judgment condition.

[0053] In some specific embodiments, the topological feature extraction module 13 may specifically include: The graph attention unit determination unit is used to determine the graph attention unit in the embedded topology learning module, and use the linear projection layer in the graph attention unit and a preset shared weight matrix to convert the three-dimensional sparse tensor into a vector to be processed, including a query vector, a key vector and a value vector. The inverse distance weight determination unit is used to determine the inverse distance weights corresponding to each adjacent event point in the key event point set using the edge weight encoding layer in the graph attention unit, so as to use a preset two-layer perceptron and map each adjacent event point based on each inverse distance weight to obtain geometric embedding features that satisfy the preset high-dimensional conditions. The fusion result determination unit is used to fuse the vector to be processed and the geometric embedding features using the attention weight calculation layer in the graph attention unit to obtain the fusion result, so as to determine the attention weight for edge perception based on the fusion result.

[0054] In some specific embodiments, the topological feature extraction module 13 may specifically include: The topology enhancement feature generation unit is used to perform weighted aggregation of the attention weights and the value vector to obtain the aggregation result, and to perform layer normalization and residual connection operations on the aggregation result to obtain the topology enhancement feature; The enhancement result generation unit is used to perform spatial consistency and correlation enhancement operations on the topology enhancement features using the spatial consistency enhancement unit to obtain the enhancement result, and to use the feature interpolation unit to map the enhancement result to the resolution of the original event point cloud to obtain local topology features.

[0055] In some specific embodiments, the topological feature extraction module 13 may specifically include: The global context feature generation unit is used to perform global average pooling on the local topological features using the global context extraction branch in the spatiotemporal consistency module to obtain the corresponding global context features. The weight matrix determination unit is used to generate branches using local spatial attention to determine a first learnable weight matrix and a second learnable weight matrix corresponding to the local topological features, so as to determine the data to be processed based on the local topological features, the first learnable weight matrix and the second learnable weight matrix using the Sigmoid activation function and ReLU activation.

[0056] In some specific embodiments, the topological feature extraction module 13 may specifically include: The spatial enhancement feature generation unit is used to multiply the data to be processed with the global context feature element by element to obtain the scaling result, and to scale the scaling result based on a preset scaling factor to obtain the spatial enhancement feature. The fusion result generation unit is used to fuse the global context features and the spatial enhancement features using the feature fusion unit to obtain the fusion result, and to perform residual connection between the fusion result and the local topological features using the feature fusion unit to obtain the spatiotemporal consistency enhancement features.

[0057] In some specific embodiments, the detection result generation module 14 may specifically include: The prediction point cloud generation unit is used to process the spatiotemporal consistency enhancement features using the spatial topology network to obtain a prediction probability map, and extract the corresponding prediction point cloud from the prediction probability map based on a preset prediction threshold. The target loss function construction unit is used to extract the corresponding real point cloud from the real label corresponding to the predicted probability map based on a preset real threshold, and then construct the target loss function based on the predicted point cloud and the real point cloud using a topological similarity loss layer and a spatiotemporal continuity loss layer.

[0058] In some specific embodiments, the detection result generation module 14 may specifically include: The topology loss construction unit is used to determine the zero-dimensional and one-dimensional topological features between the predicted point cloud and the real point cloud using the topology similarity loss layer, and to construct a topology loss based on the zero-dimensional and one-dimensional topological features; the topology loss includes the Betty number loss for constraining the consistency of the number of topological features and the Wasserstein distance loss for constraining the consistency of the distribution of topological features. The network optimization unit is used to constrain the continuity of the predicted point cloud in the time and space dimensions using the spatiotemporal continuity loss layer, obtain the continuity constraint result, determine the binary cross-entropy loss between the predicted point cloud and the true label, and then use the target loss function and the binary cross-entropy loss and the topology loss to determine the total model loss, so as to optimize the spatial topology network using the total model loss.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0060] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0062] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0063] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting small moving targets using an event camera based on topological constraints, characterized in that, include: Discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and voxelize each event block to obtain the corresponding three-dimensional sparse tensor. A spatial topology network, including an embedded topology learning module and a spatiotemporal consistency module, is constructed based on a sparse convolutional encoder-decoder. The grid sampling unit in the embedded topology learning module is used to adaptively filter key points of the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The key event point set is used to extract features by the graph attention unit, spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module to obtain local topological features. The local topological features are then processed by the global context extraction branch, local spatial attention generation branch and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. A predicted probability map is generated using the spatial topology network and based on the spatiotemporal consistency enhancement features. Predicted point clouds and real point clouds are extracted from the predicted probability map and their corresponding real labels based on preset thresholds. A target loss function is then determined based on the predicted and real point clouds. The spatial topology network is optimized based on this target loss function, and the optimized network is used for forward inference on the asynchronous event stream to obtain the detection results of small moving targets in the asynchronous event stream. The small moving targets are those whose size meets a preset small-size determination condition. The process of extracting features from the key event point set using the graph attention unit in the embedded topology learning module includes: determining the graph attention unit in the embedded topology learning module, and using the linear projection layer in the graph attention unit and based on a preset shared weight matrix to convert the three-dimensional sparse tensor into a vector to be processed including a query vector, a key vector, and a value vector; using the edge weight encoding layer in the graph attention unit to determine the inverse distance weights corresponding to each adjacent event point in the key event point set, so as to map each adjacent event point using a preset two-layer perceptron and based on each inverse distance weight to obtain geometric embedding features that satisfy preset high-dimensional conditions; and using the attention weight calculation layer in the graph attention unit to fuse the vector to be processed and the geometric embedding features to obtain a fusion result, so as to determine the attention weights for edge perception based on the fusion result. Specifically, the spatial consistency enhancement unit and feature interpolation unit in the embedded topology learning module are used to extract features from the key event point set to obtain local topological features. This includes: weighting and aggregating the attention weights and the value vectors to obtain an aggregation result, and performing layer normalization and residual connection operations on the aggregation result to obtain topological enhancement features; using the spatial consistency enhancement unit to perform spatial consistency and correlation enhancement operations on the topological enhancement features to obtain an enhancement result, and using the feature interpolation unit to map the enhancement result to the resolution of the original event point cloud to obtain local topological features.

2. The method for detecting small moving targets using an event camera based on topological constraints according to claim 1, characterized in that, The local topological features are processed using the global context extraction branch and the local spatial attention generation branch in the spatiotemporal consistency module, including: The global context extraction branch in the spatiotemporal consistency module is used to perform global average pooling on the local topological features to obtain the corresponding global context features. Local spatial attention is used to generate branches to determine a first learnable weight matrix and a second learnable weight matrix corresponding to the local topological features. Then, the Sigmoid activation function and ReLU activation are used to determine the data to be processed based on the local topological features, the first learnable weight matrix, and the second learnable weight matrix.

3. The method for detecting small moving targets using an event camera based on topological constraints according to claim 2, characterized in that, The local topological features are processed by the feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features, including: The data to be processed is multiplied element-wise with the global context feature to obtain the scaling result. The scaling result is then scaled based on a preset scaling factor to obtain spatial enhancement features. The global context features and the spatial enhancement features are fused using a feature fusion unit to obtain a fusion result. The fusion result is then residually connected with the local topological features using the feature fusion unit to obtain spatiotemporal consistency enhancement features.

4. The event camera moving small target detection method based on topological constraints according to any one of claims 1 to 3, characterized in that, The process of generating a predicted probability map using the spatial topology network and based on the spatiotemporal consistency enhancement features, extracting predicted point clouds and real point clouds from the predicted probability map and corresponding ground truth labels based on preset thresholds, and then determining a target loss function based on the predicted point clouds and the real point clouds, includes: The spatiotemporal consistency enhancement features are processed using the spatial topology network to obtain a prediction probability map, and the corresponding prediction point cloud is extracted from the prediction probability map based on a preset prediction threshold. Based on a preset real threshold, the corresponding real point cloud is extracted from the real label corresponding to the predicted probability map. Then, a target loss function is constructed based on the predicted point cloud and the real point cloud using a topological similarity loss layer and a spatiotemporal continuity loss layer.

5. The method for detecting small moving targets using an event camera based on topological constraints according to claim 4, characterized in that, The optimization of the spatial topology network based on the target loss function includes: The topological similarity loss layer is used to determine the zero-dimensional and one-dimensional topological features between the predicted point cloud and the real point cloud, and a topological loss is constructed based on the zero-dimensional and one-dimensional topological features; the topological loss includes the Betty number loss for constraining the consistency of the number of topological features and the Wasserstein distance loss for constraining the consistency of the distribution of topological features. The spatiotemporal continuity loss layer is used to constrain the continuity of the predicted point cloud in the temporal and spatial dimensions to obtain the continuity constraint result. The binary cross-entropy loss between the predicted point cloud and the true label is determined. Then, the target loss function is used and the total model loss is determined based on the binary cross-entropy loss and the topology loss, so as to optimize the spatial topology network using the total model loss.

6. A small moving target detection system based on topological constraints using an event camera, characterized in that, include: The event stream processing module is used to discretize the asynchronous event stream output by the event camera based on a preset fixed time window to obtain several corresponding event blocks, and to voxelize each event block to obtain the corresponding three-dimensional sparse tensor. A spatial topology network construction module is used to construct a spatial topology network based on a sparse convolutional encoder-decoder, including an embedded topology learning module and a spatiotemporal consistency module, so as to use the grid sampling unit in the embedded topology learning module to adaptively filter the event point cloud corresponding to the three-dimensional sparse tensor to obtain a set of key event points. The topology feature extraction module is used to extract features from the key event point set using the graph attention unit, spatial consistency enhancement unit, and feature interpolation unit in the embedded topology learning module to obtain local topology features. The local topology features are then processed using the global context extraction branch, local spatial attention generation branch, and feature fusion unit in the spatiotemporal consistency module to obtain spatiotemporal consistency enhanced features. The detection result generation module is used to generate a predicted probability map using the spatial topology network and based on the spatiotemporal consistency enhancement features. It then extracts predicted point clouds and real point clouds from the predicted probability map and their corresponding real labels based on preset thresholds. Next, it determines a target loss function based on the predicted point clouds and the real point clouds, optimizes the spatial topology network based on the target loss function, and uses the optimized network to perform forward inference on the asynchronous event stream to obtain the detection results of small moving targets in the asynchronous event stream. The small moving targets are moving targets whose size meets preset small size criteria. Specifically, the topological feature extraction module includes: The graph attention unit determination unit is used to determine the graph attention unit in the embedded topology learning module, and use the linear projection layer in the graph attention unit and a preset shared weight matrix to convert the three-dimensional sparse tensor into a vector to be processed, including a query vector, a key vector and a value vector. The inverse distance weight determination unit is used to determine the inverse distance weights corresponding to each adjacent event point in the key event point set using the edge weight encoding layer in the graph attention unit, so as to use a preset two-layer perceptron and map each adjacent event point based on each inverse distance weight to obtain geometric embedding features that satisfy the preset high-dimensional conditions. The fusion result determination unit is used to fuse the vector to be processed and the geometric embedding features using the attention weight calculation layer in the graph attention unit to obtain the fusion result, so as to determine the attention weight for edge perception based on the fusion result; Specifically, the topological feature extraction module includes: The topology enhancement feature generation unit is used to perform weighted aggregation of the attention weights and the value vector to obtain the aggregation result, and to perform layer normalization and residual connection operations on the aggregation result to obtain the topology enhancement feature; The enhancement result generation unit is used to perform spatial consistency and correlation enhancement operations on the topology enhancement features using the spatial consistency enhancement unit to obtain the enhancement result, and to use the feature interpolation unit to map the enhancement result to the resolution of the original event point cloud to obtain local topology features.

Images