A low-energy 3D point cloud single target tracking method based on spiking neural network

Abstract

The present invention proposes a low-energy 3D point cloud single target tracking method based on a pulse neural network, including: obtaining 3D point cloud data of the previous frame and the current frame of the target, designing a voxel grid module, dividing the point cloud into 3D voxels; designing a feature extraction module, using a three-stage pulse convolution block as a feature extractor to extract voxelized point cloud features, and adding a Transformer encoder layer to capture the global context information of the point cloud through a self-attention mechanism; designing a part-to-part motion modeling module, modeling and encoding the continuous point cloud features on the time series part by part through a two-dimensional pulse convolution block, and obtaining the feature information changes of the previous frame and the current frame in the corresponding spatial part; designing a regression prediction module, and inferring the relative motion of the target through the pulse regression prediction module. While reducing energy consumption and improving efficiency, the present invention ensures that the network can accurately track the target in a variety of scenes and motion states.

Description

Technical Field

[0001] The present invention relates to the technical field of single target tracking, and in particular to a low-energy consumption three-dimensional point cloud single target tracking method based on a pulse neural network. Background Art

[0002] In recent years, in the fields of intelligent driving, drone navigation, robotics, etc., 3D point cloud data, as a high-dimensional and accurate form of spatial information expression, has received widespread attention. As an important means of environmental perception, 3D point cloud can accurately capture the spatial position, size and shape of the target object, and has strong environmental adaptability, especially in low light and complex weather conditions, which is more advantageous than traditional image perception methods. Single target tracking, as a key and core part of 3D point cloud data processing, can help the system track the position changes of specific targets in real time and provide reliable data information for autonomous decision-making and behavior control. However, existing 3D point cloud single target tracking methods generally rely on traditional artificial neural networks (ANN). Although they have made significant progress in accuracy and performance, their high computational complexity and high power consumption limit their application on resource-constrained embedded devices, especially in scenarios with low power consumption and high real-time requirements, they still face great challenges.

[0003] In contrast, the spiking neural network (SNN) is a neural network model that simulates the activity of human brain neurons. Its calculation process is event-driven, that is, it only performs calculations when there is information transmission, so it has a natural low-power consumption characteristic. In addition, SNN can effectively reduce the complexity and energy consumption of the model by using pulse signals for information transmission, and has significant real-time and high-efficiency advantages. In recent years, spiking neural networks have gradually become one of the frontier directions of deep learning, but their application in three-dimensional point cloud perception is still relatively small. Three-dimensional point cloud data has the characteristics of sparsity, high dimensionality, and unstructuredness. How to combine SNN's pulse processing mode to efficiently perceive and process point cloud data has become an important and challenging research topic. This study will promote the application of SNN in three-dimensional point cloud perception by exploring the single-target tracking method of three-dimensional point cloud based on SNN, which will help deepen the understanding of spiking neural networks and their application prospects. Summary of the invention

[0004] In view of the problems of low efficiency and high energy consumption of single target tracking in the prior art, the present invention proposes a low-energy three-dimensional point cloud single target tracking method based on a pulse neural network. The method is based on a pulse neural network, and only needs to add the synaptic weight to the neuron membrane potential when a pulse occurs, so as to avoid a large number of high-precision floating-point multiplication operations in traditional neural networks, thereby achieving extremely low energy consumption during the tracking process.

[0005] In order to achieve the above technical objectives, the present invention provides the following technical solutions:

[0006] A low-energy 3D point cloud single target tracking method based on a spiking neural network, which specifically includes:

[0007] S1, point cloud data acquisition and voxel grid construction module; acquiring three-dimensional point cloud data through a sensor, and voxelizing the acquired three-dimensional point cloud data based on a grid method to divide it into 3D voxels; the 3D voxels include the voxelized three-dimensional point cloud data and voxel coordinates;

[0008] S2, constructing a feature extraction module; taking the 3D voxels obtained in step S1 as input, designing a three-stage pulse convolution block as a feature extractor, and constructing a pulse neural network to extract features from the voxelized three-dimensional point cloud data; and adding a layer of Transformer encoder after the pulse neural network, capturing the global context information of the point cloud data through the self-attention mechanism, and finally outputting continuous point cloud features;

[0009] S3, build a part-to-part motion modeling module; model the continuous point cloud features on the time series part by part through a two-dimensional pulse convolution block to capture the fine-grained motion changes of the point cloud at different spatial positions; and encode the information changes of each corresponding spatial part;

[0010] S4, construct a regression prediction module; predict the four-degree-of-freedom relative motion of the target based on the encoded information obtained by the part-to-part motion modeling module, and perform a rigid transformation on the 3D bounding box of the previous frame to obtain the 3D bounding box of the current frame to achieve target tracking.

[0011] Furthermore, the point cloud data acquisition in step S1 is specifically as follows:

[0012] For the target object in the scene, the position of the 3D bounding box in the first frame is known, and the initial target position information is determined; at any time during the tracking process , based on the LiDAR sensor, the 3D point cloud data of the previous frame and the current frame are acquired, and the acquired 3D point cloud data are appropriately cropped according to the position of the 3D bounding box of the previous frame and the defined point cloud range to reduce the interference of irrelevant environmental information.

[0013] Furthermore, the voxel grid construction module in step S1 specifically includes:

[0014] S11, dividing the acquired three-dimensional point cloud data according to a preset voxel size;

[0015] S12, traverse each point, calculate its position coordinates in the voxel grid, and assign a corresponding voxel coordinate to each point;

[0016] S13, marking the point cloud data in batches, storing the voxel coordinates of each point as batch coordinates, and performing batch processing on the point cloud data;

[0017] S14, after processing, output 3D data, which is a matrix containing voxelized three-dimensional point cloud data and voxel coordinates, which is the input of the subsequent voxel feature extraction module.

[0018] Furthermore, each layer of spiking neurons in the spiking neural network in step S2 works based on a voltage update mechanism and a pulse emission mechanism, which are specifically:

[0019] Voltage update mechanism: Spiking neurons accumulate and update the membrane voltage according to the input pulses in the time series; suppose that at time t, the spiking neurons in this layer receive the input signal from the spiking neurons in the previous layer. , then the membrane voltage V(t) of the spiking neuron in this layer is updated to:

[0020] ;

[0021] in, is the membrane voltage of the spiking neuron in the previous layer;

[0022] It is represented by the weighted sum of the pulses emitted by the previous layer after passing through the synaptic weights:

[0023] ;

[0024] in, is the synaptic weight coming from the i-th neuron in the previous layer, Is whether the i-th neuron in the previous layer emits a pulse at time t, 0 means no pulse, 1 means pulse, and b is the bias term of the neuron;

[0025] The pulse release mechanism includes a release mechanism and a reset mechanism;

[0026] Discharge mechanism: setting membrane voltage threshold ;when When the neuron fires , and trigger the reset mechanism; pulse The formula is:

[0027] ;

[0028] in, is a step function, that is ;

[0029] Reset mechanism: After a pulse is emitted, the membrane voltage is reset to the preset reset voltage value ,Right now .

[0030] Furthermore, the structure of the spiking neural network in step S2 specifically includes:

[0031] The voxel feature encoding module is used to encode the point features within the voxel, average the features of the points within the voxel, and obtain the overall features of the voxel;

[0032] Pulse convolution feature extraction, including a three-dimensional sparse convolution layer, a batch normalization layer and a pulse emission neuron; the three-dimensional sparse convolution layer uses a 3×3×3 convolution kernel; the pulse emission neuron suppresses low activation values ​​by setting a trigger threshold to achieve feature screening based on the pulse emission mechanism;

[0033] The multi-level pulse convolution extraction module includes multiple sparse convolution coding layers, each of which is further divided into multiple sub-convolution units. Each sub-convolution unit contains a three-dimensional sparse convolution layer, a batch normalization layer, and a pulse emission neuron. By expanding the receptive field layer by layer, feature information of different scales is extracted, and the number of channels in each coding layer is gradually increased to improve the feature expression capability.

[0034] The output feature aggregation module includes a two-dimensional convolutional layer, a batch normalization layer, and a pulse-discharging neuron. The two-dimensional convolutional layer reduces the number of channels of the encoded features and further suppresses the low activation value output through the pulse-discharging neuron to extract the compressed feature representation, which is ready to be output to the subsequent Transformer encoder module.

[0035] More specifically, the structure of the Transformer encoder module specifically includes: multiple Transformer encoder layers; each Transformer encoder layer includes a self-attention mechanism and a feedforward network, the feedforward network includes two linear layers and an activation function, and each Transformer encoder layer also includes a normalization layer and a Dropout layer to prevent overfitting.

[0036] Further, in step S3:

[0037] The part-to-part motion modeling module specifically includes:

[0038] The feature fusion unit is used to receive the continuous point cloud features output by step S2, and splice them along the channel dimension to generate a feature expression with an explicit spatial structure. The new feature contains the feature information of the point cloud data from the previous frame and the current frame at the corresponding spatial position.

[0039] Multi-layer 2D pulse convolution blocks, each of which consists of a convolution layer, a batch normalization layer, and a pulse-emitting neuron; the first three layers of convolution blocks maintain a fixed number of channels and spatial resolution, the middle three layers of convolution blocks gradually expand the number of channels and reduce the spatial resolution, and the last three layers further increase the number of channels and reduce the spatial size; the batch normalization layer is used to stabilize the feature distribution; the pulse-emitting neuron uses the pulse-emitting mechanism to achieve fine-grained motion modeling by setting the trigger threshold;

[0040] The two-dimensional convolution operation acts on the fused feature map, and a weighted sum operation is performed on each local area in the feature map, thereby extracting local feature information at different spatial positions, and transforming and encoding the local feature information; by encoding the changes in feature information within each corresponding spatial part through two-dimensional convolution, the target motion can be effectively modeled;

[0041] Adaptive max pooling layer, used to downsample feature maps to a fixed size and unify the size of different input features;

[0042] The flattening layer flattens the pooled features to form comprehensive feature information as the input of the regression prediction module in step S4; the comprehensive feature information includes: the feature information changes in each corresponding spatial part of the previous frame and the current frame.

[0043] Further, in step S4: the output of step S3 is used as the input of the regression prediction module, the regression prediction module includes a plurality of fully connected layers, each of which is activated by spiking neurons; step S4 specifically includes:

[0044] S41, the feature information first enters the first fully connected layer, and after the spike neurons are activated, the feature information is preliminarily processed and converted, highlighting the part related to the target transient motion;

[0045] S42, the activated features are continuously passed to the next fully connected layer, and the feature information is deeply processed through multiple fully connected layers, and regression prediction is performed by continuously refining and optimizing the feature vector of the output feature dimension;

[0046] S43, the final output layer accurately predicts the position change and posture change based on the deeply processed features, that is, the obtained four-degree-of-freedom relative motion, which is expressed as:

[0047] ;

[0048] in, It is a relative motion with four degrees of freedom; Indicates the change in the target's coordinates in three-dimensional space, that is, the change in position; It is the change in rotation angle in three-dimensional space, that is, the change in posture;

[0049] S44. Combine the four-degree-of-freedom relative motion information, perform a rigid transformation on the position of the 3D bounding box of the previous frame, obtain the position of the 3D bounding box of the current frame, and achieve target tracking.

[0050] Based on the above technical solution, the present invention has at least the following beneficial effects:

[0051] The present invention is a low-energy three-dimensional point cloud single target tracking method based on a pulse neural network. First, SNN uses a pulse-triggered computing mechanism to significantly reduce computing energy consumption while maintaining efficient time series information processing, and is suitable for real-time application scenarios with limited resources. Secondly, SNN has significant advantages in capturing and modeling the dynamic characteristics of sequence data, which helps to more accurately describe the subtle motion changes and local characteristics of point cloud targets. Its pulse activation function can realize the encoding of continuous data in the time dimension, and improve the robustness and stability of tracking. Overall, the introduction of SNN into point cloud single target tracking not only improves energy efficiency, but also enhances the target detection and tracking performance in complex scenes, and is expected to achieve a real-time three-dimensional target tracking system with lower latency and higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a low-energy three-dimensional point cloud single target tracking method based on a pulse neural network proposed by the present invention;

[0053] Figure 2 It is a schematic diagram of the part-to-part motion change in the present invention, which shows the information change of the corresponding position part of the previous frame and the current frame;

[0054] Figure 3 It is a network structure diagram of the feature extraction module in the present invention;

[0055] Figure 4 : is a network structure diagram of a single-layer Transformer encoder in the Transformer encoder module of the present invention;

[0056] Figure 5 A network structure diagram of the motion modeling module in the present invention;

[0057] Figure 6 This is a graph of the tracking visualization results in the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.

[0060] like Figure 1-6 As shown, the present invention provides a low-energy three-dimensional point cloud single target tracking method based on a pulse neural network. Compared with the traditional point cloud single target tracking method based on an artificial neural network, SNN can effectively reduce the complexity and energy consumption of the model by using pulse signals for information transmission, and has significant real-time and high efficiency advantages.

[0061] like Figure 1 As shown, the method proposed by the present invention specifically comprises the following steps:

[0062] S1, point cloud data acquisition and voxel grid construction module; acquiring three-dimensional point cloud data through a sensor, and voxelizing the acquired three-dimensional point cloud data based on a grid method to divide it into 3D voxels; the 3D voxels include the voxelized three-dimensional point cloud data and voxel coordinates;

[0063] As a preferred implementation, the point cloud data acquisition in step S1 is specifically as follows:

[0064] For the target object in the scene, the position of the 3D bounding box in the first frame is known, and the initial target position information is determined; at any time during the tracking process , based on the LiDAR sensor, the 3D point cloud data of the previous frame and the current frame are obtained, and the obtained 3D point cloud data are appropriately cropped according to the position of the 3D bounding box of the previous frame and the defined point cloud range to reduce the interference of irrelevant environmental information; Figure 2 As shown, , They are the 3D point cloud data obtained after cropping the previous frame and the current frame;

[0065] The construction of the voxel network module specifically includes:

[0066] S11, dividing the acquired three-dimensional point cloud data according to a preset voxel size;

[0067] S12, traverse each point, calculate its position coordinates in the voxel grid, and assign a corresponding voxel coordinate to each point;

[0068] S13, marking the point cloud data in batches, storing the voxel coordinates of each point as batch coordinates, and performing batch processing on the point cloud data;

[0069] S14, outputting 3D data after processing, which is a matrix containing voxelized 3D point cloud data and voxel coordinates, i.e., the input of the subsequent voxel feature extraction module;

[0070] S2, construct feature extraction module; Figure 3 As shown, the 3D voxels obtained in step S1 are used as input, a three-stage pulse convolution block is designed as a feature extractor, and a pulse neural network is constructed to extract features from the voxelized three-dimensional point cloud data; and a layer of Transformer encoder is added after the pulse neural network to capture the global context information of the point cloud data through a self-attention mechanism, and finally output continuous point cloud features;

[0071] As a preferred embodiment, each layer of spiking neurons in the spiking neural network in step S2 works based on a voltage update mechanism and a pulse emission mechanism, and these two mechanisms are specifically:

[0072] Voltage update mechanism: Spiking neurons accumulate and update the membrane voltage according to the input pulses in the time series; suppose that at time t, the spiking neurons in this layer receive the input signal from the spiking neurons in the previous layer. , then the membrane voltage V(t) of the spiking neuron in this layer is updated to:

[0073] ;

[0074] in, is the membrane voltage of the spiking neuron in the previous layer;

[0075] It is represented by the weighted sum of the pulses emitted by the previous layer after passing through the synaptic weights:

[0076] ;

[0077] in, is the synaptic weight coming from the i-th neuron in the previous layer, Is whether the i-th neuron in the previous layer emits a pulse at time t, 0 means no pulse, 1 means pulse, and b is the bias term of the neuron;

[0078] The pulse release mechanism includes a release mechanism and a reset mechanism;

[0079] Discharge mechanism: setting membrane voltage threshold ;when When the neuron fires , and trigger the reset mechanism; pulse The formula is expressed as:

[0080] ;

[0081] in, is a step function, that is ;

[0082] Reset mechanism: After a pulse is emitted, the membrane voltage is reset to the preset reset voltage value ,Right now .

[0083] Furthermore, if Figure 3 As shown, the structure of the pulse neural network in step S2 specifically includes:

[0084] The voxel feature encoding module is used to encode the point features within the voxel, average the features of the points within the voxel, and obtain the overall features of the voxel;

[0085] Pulse convolution feature extraction, including a three-dimensional sparse convolution layer, a batch normalization layer and a pulse emission neuron; the three-dimensional sparse convolution layer uses a 3×3×3 convolution kernel; the pulse emission neuron suppresses low activation values ​​by setting a trigger threshold to achieve feature screening based on the pulse emission mechanism;

[0086] The multi-level pulse convolution extraction module includes multiple sparse convolution coding layers, each of which is further divided into multiple sub-convolution units. Each sub-convolution unit contains a three-dimensional sparse convolution layer, a batch normalization layer and a pulse emission neuron. By expanding the receptive field layer by layer and extracting feature information of different scales, each coding layer gradually increases the number of channels to improve the feature expression capability.

[0087] The output feature aggregation module includes a 2D convolutional layer, a batch normalization layer, and a spike neuron. The 2D convolutional layer reduces the number of channels of the encoded features and further suppresses low activation value outputs through spike neurons to extract compressed feature representations for output to the subsequent Transformer encoder module.

[0088] More specifically, the Transformer encoder module includes multiple Transformer encoder layers; Figure 4As shown, each Transformer encoder layer contains a self-attention mechanism and a feedforward network. The feedforward network includes two linear layers and an activation function. Each Transformer encoder layer also includes a normalization layer and a Dropout layer to prevent overfitting. After the Transformer encoder module is extracted, the continuous point cloud features of the previous frame and the current frame in the time series are obtained, which are recorded as , ;

[0089] S3, build part-to-part motion modeling module; transform continuous point cloud features , Part-by-part modeling is performed through 2D pulse convolution blocks to capture the fine-grained motion changes of the point cloud at different spatial locations; and the information changes of each corresponding spatial part are encoded;

[0090] As a preferred embodiment, Figure 5 As shown, the part-to-part motion modeling module specifically includes:

[0091] Feature fusion unit, used to receive continuous 3D point cloud features in time series , and concatenate along the channel dimension to generate a feature expression with an explicit spatial structure. The new feature contains information from the corresponding parts of the point clouds in two consecutive time periods;

[0092] Multi-layer 2D pulse convolution blocks, each of which consists of a convolution layer, a batch normalization layer, and a pulse-emitting neuron; the first three layers of convolution blocks maintain a fixed number of channels and spatial resolution, the middle three layers of convolution blocks gradually expand the number of channels and reduce the spatial resolution, and the last three layers further increase the number of channels and reduce the spatial size; the batch normalization layer is used to stabilize the feature distribution; the pulse-emitting neuron uses the pulse-emitting mechanism to achieve fine-grained motion modeling by setting the trigger threshold;

[0093] The two-dimensional convolution operation acts on the fused feature map, and a weighted sum operation is performed on each local area in the feature map, thereby extracting local feature information at different spatial positions, and transforming and encoding the local feature information; by encoding the changes in feature information within each corresponding spatial part through two-dimensional convolution, the target motion can be effectively modeled;

[0094] Adaptive max pooling layer, used to downsample feature maps to a fixed size and unify the size of different input features;

[0095] A flattening layer flattens the pooled features to form comprehensive feature information as input to the regression prediction module in step S4; the comprehensive feature information includes: feature information changes in each corresponding spatial part of the previous frame and the current frame;

[0096] S4, constructing a regression prediction module; predicting the four-degree-of-freedom relative motion of the target based on the encoded information obtained by the part-to-part motion modeling module, and performing a rigid transformation according to the 3D bounding box of the previous frame to obtain the 3D bounding box of the current frame, so as to track the target; as a preferred implementation, in step S4:

[0097] The output of step S3 is used as the input of the regression prediction module, wherein the regression prediction module includes a plurality of fully connected layers, each of which is activated by spiking neurons; step S4 specifically includes:

[0098] S41, the feature information first enters the first fully connected layer, and after the spike neurons are activated, the feature information is preliminarily processed and converted, highlighting the part related to the target transient motion;

[0099] S42, the activated features are continuously passed to the next fully connected layer, and the feature information is deeply processed through multiple fully connected layers, that is, continuously refined and optimized;

[0100] S43, the final output layer accurately predicts the position change and posture change based on these deeply processed features, that is, the four-degree-of-freedom relative motion, which is expressed as:

[0101] ;

[0102] in, It is a relative motion with four degrees of freedom; Indicates the change in the target's coordinates in three-dimensional space, that is, the change in position; It is the change in rotation angle in three-dimensional space, that is, the change in posture;

[0103] S44. Combine the four-degree-of-freedom relative motion information, perform a rigid transformation on the position of the 3D bounding box of the previous frame, obtain the position of the 3D bounding box of the current frame, and achieve target tracking.

[0104] In this embodiment, the position of the 3D bounding box of the target object is given and known in the first frame. In the subsequent tracking process, the bounding box predicted by each frame is used as the known bounding box of the next frame, and so on. For the current frame, the 3D bounding box of the previous frame is also considered to be known. However, when calculating the tracking accuracy and success rate, the accuracy and success rate are calculated by comparing with the actual bounding box of the target object that has been marked in the data set.

[0105] The final result is as follows Figure 6As shown: This figure shows the tracking status of the cyclist class (i.e., a person riding a bicycle) at different times starting from T=0 in the KITTI dataset. The green truth box indicates the actual position of the target object, and the red prediction box indicates the predicted position of the target object. It can be seen from the figure that the method proposed in the present invention can still achieve good tracking effect as time changes.

[0106] In summary, the method proposed in the present invention introduces pulse neural networks into point cloud tracking, replacing the artificial neural network in the traditional process, which not only improves energy efficiency, but also enhances the target detection and tracking performance in complex scenes, and is expected to achieve a real-time three-dimensional target tracking system with lower latency and higher accuracy.

[0107] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0108] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A low-energy three-dimensional point cloud single target tracking method based on a pulse neural network, characterized in that: The specific steps include: S1, point cloud data acquisition and voxel grid construction module; acquiring three-dimensional point cloud data through a sensor, and voxelizing the acquired three-dimensional point cloud data based on a grid method to divide it into 3D voxels; the 3D voxels include the voxelized three-dimensional point cloud data and voxel coordinates; S2, construct feature extraction module; Taking the 3D voxels obtained in step S1 as input, designing a three-stage pulse convolution block as a feature extractor, and constructing a pulse neural network to extract features from the voxelized 3D point cloud data; and adding a layer of Transformer encoder module after the pulse neural network to capture the global context information of the point cloud data through a self-attention mechanism, and finally outputting continuous point cloud features; S3, build a part-to-part motion modeling module; model the continuous point cloud features on the time series part by part through a two-dimensional pulse convolution block to capture the fine-grained motion changes of the point cloud at different spatial positions; and encode the information changes of each corresponding spatial part; S4, construct a regression prediction module; predict the four-degree-of-freedom relative motion of the target based on the encoded information obtained by the part-to-part motion modeling module, and perform a rigid transformation on the 3D bounding box of the previous frame to obtain the 3D bounding box of the current frame to achieve target tracking.

2. The low-energy 3D point cloud single target tracking method based on pulse neural network according to claim 1 is characterized in that: The point cloud data acquisition in step S1 is specifically as follows: For the target object in the scene, the position of the 3D bounding box in the first frame is known, and the initial target position information is determined; at any time during the tracking process , based on the LiDAR sensor, the 3D point cloud data of the previous frame and the current frame are acquired, and the acquired 3D point cloud data are appropriately cropped according to the position of the 3D bounding box of the previous frame and the defined point cloud range to reduce the interference of irrelevant environmental information.

3. The low-energy consumption three-dimensional point cloud single target tracking method based on pulse neural network according to claim 1 is characterized in that: The voxel grid construction module in step S1 specifically includes: S11, dividing the acquired three-dimensional point cloud data according to a preset voxel size; S12, traverse each point, calculate its position coordinates in the voxel grid, and assign a corresponding voxel coordinate to each point; S13, marking the point cloud data in batches, storing the voxel coordinates of each point as batch coordinates, and performing batch processing on the point cloud data; S14, after processing, output 3D data, which is a matrix containing voxelized three-dimensional point cloud data and voxel coordinates, which is the input of the subsequent voxel feature extraction module.

4. The low-energy 3D point cloud single target tracking method based on pulse neural network according to claim 1 is characterized in that: Each layer of spiking neurons in the spiking neural network in step S2 works based on a voltage update mechanism and a pulse emission mechanism, which are specifically: Voltage update mechanism: Spiking neurons accumulate and update the membrane voltage according to the input pulses in the time series; suppose that at time t, the spiking neurons in this layer receive the input signal from the spiking neurons in the previous layer. , then the membrane voltage V(t) of the spiking neuron in this layer is updated to: ； in, is the membrane voltage of the spiking neuron in the previous layer; It is represented by the weighted sum of the pulses emitted by the previous layer after passing through the synaptic weights: ； in, is the synaptic weight coming from the i-th neuron in the previous layer, Is whether the i-th neuron in the previous layer emits a pulse at time t, 0 means no pulse, 1 means pulse, and b is the bias term of the neuron; The pulse release mechanism includes a release mechanism and a reset mechanism; Discharge mechanism: setting membrane voltage threshold ;when When the neuron fires , and trigger the reset mechanism; pulse The formula is: ； in, is a step function, that is ; Reset mechanism: After a pulse is emitted, the membrane voltage is reset to the preset reset voltage value ,Right now .

5. The low-energy consumption three-dimensional point cloud single target tracking method based on pulse neural network according to claim 4 is characterized in that: The structure of the spiking neural network in step S2 specifically includes: The voxel feature encoding module is used to encode the point features within the voxel, average the features of the points within the voxel, and obtain the overall features of the voxel; Pulse convolution feature extraction, including a three-dimensional sparse convolution layer, a batch normalization layer and a pulse emission neuron; the three-dimensional sparse convolution layer uses a 3×3×3 convolution kernel; the pulse emission neuron suppresses low activation values ​​by setting a trigger threshold to achieve feature screening based on the pulse emission mechanism; The multi-level pulse convolution extraction module includes multiple sparse convolution coding layers, each of which is further divided into multiple sub-convolution units. Each sub-convolution unit contains a three-dimensional sparse convolution layer, a batch normalization layer, and a pulse emission neuron. By expanding the receptive field layer by layer, feature information of different scales is extracted, and the number of channels in each coding layer is gradually increased to improve the feature expression capability. The output feature aggregation module includes a two-dimensional convolutional layer, a batch normalization layer, and a pulse-discharging neuron. The two-dimensional convolutional layer reduces the number of channels of the encoded features and further suppresses the low activation value output through the pulse-discharging neuron to extract the compressed feature representation, which is ready to be output to the subsequent Transformer encoder module.

6. The low-energy consumption three-dimensional point cloud single target tracking method based on pulse neural network according to claim 1 is characterized in that: The structure of the Transformer encoder module in step S2 specifically includes: multiple Transformer encoder layers; each Transformer encoder layer includes a self-attention mechanism and a feedforward network, the feedforward network includes two linear layers and an activation function, and each Transformer encoder layer also includes a normalization layer and a Dropout layer to prevent overfitting.

7. The low-energy 3D point cloud single target tracking method based on pulse neural network according to claim 1, characterized in that: In step S3: The part-to-part motion modeling module specifically includes: The feature fusion unit is used to receive the continuous point cloud features output by step S2, and splice them along the channel dimension to generate a feature expression with an explicit spatial structure. The new feature contains the feature information of the point cloud data from the previous frame and the current frame at the corresponding spatial position. Multi-layer 2D pulse convolution blocks, each of which consists of a convolution layer, a batch normalization layer, and a pulse-emitting neuron; the first three layers of convolution blocks maintain a fixed number of channels and spatial resolution, the middle three layers of convolution blocks gradually expand the number of channels and reduce the spatial resolution, and the last three layers further increase the number of channels and reduce the spatial size; the batch normalization layer is used to stabilize the feature distribution; the pulse-emitting neuron uses the pulse-emitting mechanism to achieve fine-grained motion modeling by setting the trigger threshold; The two-dimensional convolution operation acts on the fused feature map, and a weighted sum operation is performed on each local area in the feature map, thereby extracting local feature information at different spatial positions, and transforming and encoding the local feature information; by encoding the changes in feature information within each corresponding spatial part through two-dimensional convolution, the target motion can be effectively modeled; Adaptive max pooling layer, used to downsample feature maps to a fixed size and unify the size of different input features; The flattening layer flattens the pooled features to form comprehensive feature information as the input of the regression prediction module in step S4; the comprehensive feature information includes: the feature information changes in each corresponding spatial part of the previous frame and the current frame.

8. The low-energy consumption three-dimensional point cloud single target tracking method based on pulse neural network according to claim 1 is characterized in that: In step S4: the output of step S3 is used as the input of the regression prediction module, the regression prediction module includes a plurality of fully connected layers, each of which is activated by spiking neurons; step S4 specifically includes: S41, the feature information first enters the first fully connected layer, and after the spike neurons are activated, the feature information is preliminarily processed and converted, highlighting the part related to the target transient motion; S42, the activated features are continuously passed to the next fully connected layer, and the feature information is deeply processed through multiple fully connected layers, and regression prediction is performed by continuously refining and optimizing the feature vector of the output feature dimension; S43, the final output layer accurately predicts the position change and posture change based on the deeply processed features, that is, the obtained four-degree-of-freedom relative motion, which is expressed as: ； in, It is a relative motion with four degrees of freedom; Indicates the change in the target's coordinates in three-dimensional space, that is, the change in position; It is the change in rotation angle in three-dimensional space, that is, the change in posture; S44. Combine the four-degree-of-freedom relative motion information, perform a rigid transformation on the position of the 3D bounding box of the previous frame, obtain the position of the 3D bounding box of the current frame, and achieve target tracking.

Images