Target identification method and system based on event stream adaptive slicing

Abstract

The invention discloses a target identification method and system based on event stream adaptive slicing, and the method comprises the steps: obtaining a time-space pulse event stream asynchronously outputted by an event camera, and the time-space pulse event stream comprises a plurality of time-space pulse events; according to the target identification method and system based on event stream adaptive slicing, a time-space pulse event stream is dynamically segmented by adopting a neuronal membrane voltage waveform, noise and redundant information are effectively eliminated, and more complete capture of action fragments is realized; a dynamic termination strategy is introduced, the strategy evaluates a Tempotron neuron issuing sequence of a slice in real time, and processing of a subsequent event stream is stopped when a reasoning result tends to be stable in a segment-by-segment reasoning mode, so that rapid identification is performed under the condition that the whole event stream is not processed, and the calculation overhead is remarkably reduced while the accuracy is kept.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition of neuromorphic vision, and in particular relates to a target recognition method and system based on event stream adaptive slicing. Background Art

[0002] Traditional computer vision systems rely on frame-based cameras, whose fixed frame rates are significantly insufficient in highly dynamic or complex lighting scenes, leading to problems such as motion blur, high redundancy, and low dynamic range. To address this issue, event cameras have emerged. Event cameras are neuromorphic devices inspired by the human retina and differ from traditional frame-based sensors. Instead of capturing visual input from all pixels at a fixed rate to form an image, the output of an event camera follows an Addressed Event Representation (AER) protocol, in which each pixel independently emits an event when it detects a significant change in light intensity, resulting in an asynchronous, sparse stream of events representing the scene. Event cameras offer significant advantages over traditional cameras, including high temporal resolution, low power consumption, and extremely high dynamic range (up to 140 dB). Commonly used AER cameras include asynchronous time-based image sensors (ATIS), dynamic vision sensors (DVS), and dynamic and active pixel vision sensors (DAVIS).

[0003] Spiking Neural Networks (SNNs) are brain-inspired computing models that process information by simulating the spiking discharges of neurons, making them naturally suited for event-driven algorithms. However, existing SNN-based event stream recognition methods typically require processing the entire event stream. Because event cameras are highly sensitive to changes within their visual receptive fields, the generated event streams are susceptible to noise, which can affect classification accuracy. To alleviate this problem, some research has focused on segmenting event streams to extract valid information and reduce the impact of noise. For example, the existing paper (Baby et al., "Dynamic vision sensors for human activity recognition," ACPR, 2017) segments the event stream into frames at fixed time intervals. However, this method is highly sensitive to the choice of interval, and segmentation performance is significantly affected by the speed of the target's motion. Existing literature Ghosh et al., "Real-time gesture recognition using dynamic vision sensors," J. Real-Time Image Process., 2014, proposed a segmentation method based on a fixed number of events. This method effectively eliminates the influence of motion speed on target shape capture. However, when the number of events increases, each slice may contain redundant information; and when the number of events decreases, the slice may not be able to represent the complete action.

[0004] In addition, the existing literature Chen et al., "Efficient event stream segmentation for noise-robust recognition," IEEE Trans. Neural Netw. Learn. Syst., 2021. proposed a noise-robust adaptive event stream segmentation method, which uses a motion sign detection (MSD) module to identify areas with high event density (such as the existing literature Peng et al., "Bag of motion words for action recognition," CVPR, 2016.), which usually correspond to periods of intense movement, and determines the timing of segmentation by detecting the local maximum of membrane voltage. However, the main problem with this method is that intense movement segments may not represent complete movements, and threshold-based methods are less effective when facing movements of different speeds due to fluctuations in membrane voltage peaks.

[0005] Therefore, existing technologies for event stream processing suffer from noise sensitivity, excessive redundant information, and poor adaptability of segmentation methods, making it difficult to effectively achieve adaptive event stream segmentation and rapid recognition. Therefore, a more efficient method is needed to address these shortcomings and improve the accuracy and efficiency of event stream processing. Summary of the Invention

[0006] The purpose of the present invention is to solve the problems raised in the background technology and to propose a target recognition method and system based on adaptive slicing of event streams.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The present invention proposes a target recognition method based on event stream adaptive slicing, comprising:

[0009] Acquire a spatiotemporal pulse event stream asynchronously output by an event camera, wherein the spatiotemporal pulse event stream includes a plurality of spatiotemporal pulse events;

[0010] The spatiotemporal pulse event stream is transferred to the spiking neuron to capture the membrane voltage change waveform of the spiking neuron;

[0011] Adaptively use each peak interval of the membrane voltage change waveform as a slice;

[0012] Each slice is sequentially input into a trained target recognition model to obtain the corresponding Tempotron neuron firing sequence for each slice. The target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal encoding layer, a second spatiotemporal encoding layer, a pooling layer, and a population encoding Tempotron neuron.

[0013] Each time a slice's Tempotron neuron firing sequence is obtained, the similarity between the current Tempotron neuron firing sequence and the previous adjacent Tempotron neuron firing sequence is calculated. When the similarity of multiple consecutive adjacent Tempotron neuron firing sequences is greater than a first threshold, the subsequent slices are stopped from being input into the trained target recognition model.

[0014] For each Tempotron neuron firing sequence corresponding to the slice input into the trained target recognition model, the category with the largest number of pulses fired in each Tempotron neuron firing sequence is taken as the final target category of each slice.

[0015] Preferably, the spatiotemporal pulse event stream is represented as E={ e i ∣ e i =[ x i , y i , t i , p i ] T ,i∈[1,N]} ,in represents the number of spatiotemporal pulse events in the spatiotemporal pulse event stream, represents transpose, Indicates the The pixel abscissa of a spatiotemporal pulse event, Indicates the The pixel ordinate of a spatiotemporal pulse event, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

[0016] Preferably, after capturing the membrane voltage change waveform of the pulse neuron, the membrane voltage change waveform is smoothed;

[0017] In the process of adaptively taking each peak interval of the membrane voltage change waveform as a slice, each peak interval of the smoothed membrane voltage change waveform is sequentially taken as a candidate slice, and the number of spatiotemporal pulse events corresponding to each candidate slice is compared with a preset second threshold;

[0018] When it is greater than or equal to the second threshold, the current candidate slice is taken as a slice;

[0019] If the number of spatiotemporal pulse events is less than the second threshold, the next candidate slice is integrated with the current candidate slice to obtain a new candidate slice, and the number of spatiotemporal pulse events corresponding to the new candidate slice is compared with the preset second threshold. If the number of spatiotemporal pulse events corresponding to the new candidate slice is greater than or equal to the second threshold, the current new candidate slice is taken as a slice.

[0020] The peak interval is the interval between two adjacent troughs in the membrane voltage change waveform.

[0021] Preferably, after the slice is input into the trained target recognition model, the first spatiotemporal coding layer adopts the spatial neighborhood radius and the time decay constant Calculate each space-time pulse event in the slice in sequence, and obtain the first harmonic mean time surface corresponding to each space-time pulse event in sequence;

[0022] The first harmonic mean time surfaces are sequentially matched with all time surface prototypes in the first spatiotemporal coding layer for the first similarity, and the time surface prototype with the largest first similarity is used as the time surface prototype of each corresponding spatiotemporal pulse event. The polarity of the light intensity change of the pixel point corresponding to the corresponding spatiotemporal pulse event is modified to the time surface prototype with the largest first similarity, and the modified spatiotemporal pulse event is obtained, which is expressed as ,in The first harmonic mean time surface corresponding to the current space-time pulse event has the largest first similarity. A time surface prototype, k 1 ∈ [1, K 1 ] , is the number of temporal surface prototypes in the first spatiotemporal coding layer;

[0023] Each modified spatiotemporal pulse event is sequentially used as the input of the second spatiotemporal coding layer, and the second spatiotemporal coding layer adopts the spatial neighborhood radius and the time decay constant Calculating each modified space-time pulse event in sequence, and obtaining a second harmonic mean time surface corresponding to each modified space-time pulse event in sequence;

[0024] Perform a second similarity match on each second harmonic mean time surface with all time surface prototypes in the second space-time coding layer in turn, use the time surface prototype with the largest second similarity as the time surface prototype of the modified space-time pulse event, and add the time surface prototype with the largest second similarity to the corresponding modified space-time pulse event to obtain the added space-time pulse event, which is expressed as e i ''=[ x i , y i , t i , p k 1 , p k 2 ] T ,in The second largest similarity of the second harmonic mean time surface corresponding to the current space-time pulse event A time surface prototype, k 2 ∈ [1, K 2 ] , is the number of temporal surface prototypes in the second spatiotemporal coding layer;

[0025] Each added spatiotemporal pulse event is then passed through the pooling layer and the single-layer pulse neural network of the group encoding Tempotron neurons to obtain the Tempotron neuron firing sequence of the current slice.

[0026] Preferably, the first spatiotemporal coding layer adopts a spatial neighborhood radius and the time decay constant Each spatiotemporal impulse event in the slice is calculated in sequence, and the first harmonic mean time surface corresponding to each spatiotemporal impulse event is obtained in sequence. The specific calculation formula of the first harmonic mean time surface is as follows: ;

[0027] in, ; ;

[0028] in, Indicates the The first harmonic mean time surface corresponding to the space-time pulse event, Indicates the current The past of a space-time pulse event A space-time pulse event, is the offset, u= [ u x , u y ] T ,and u x ∈ [ - R 1 , R 1 ] , u y ∈ [ - R 1 , R 1 ] , For the The offset pixels within the receptive field of a spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of a spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Indicates the The timestamp of the space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal pulse event in the spatiotemporal pulse event stream Past The time of the most recent historical space-time pulse event, Indicates the pixels of spatiotemporal impulse events, and , Represents pixels The time set of all past space-time pulse events, Indicates taking The most recent historical space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

[0029] Preferably, the second spatiotemporal coding layer adopts a spatial neighborhood radius and the time decay constant Calculate each modified space-time pulse event in sequence to obtain the second harmonic mean time surface corresponding to each modified space-time pulse event. The specific calculation formula of the second harmonic mean time surface is as follows: ; in, ; ;

[0030] in, Indicates the The second harmonic mean time surface corresponding to the modified space-time impulse event, is the offset, u'= [ u x ' , u y ' ] T ,and u x ' ∈ [ - R 2 , R 2 ] , u y ' ∈ [ - R 2 , R 2 ] , For the The offset pixels within the receptive field of the modified spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of the modified spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal spike event in the modified spatiotemporal spike event stream Past The time of the most recent historical space-time pulse event, Represents pixels The event set of all past spacetime pulse events.

[0031] Preferably, during the training phase of the target recognition model, the training set includes There are multiple categories in each category, and each sample is a spatiotemporal pulse event stream. All samples in the training set are shuffled and input into the first spatiotemporal coding layer of the training stage in sequence. The first spatiotemporal coding layer calculates the spatiotemporal pulse events in each sample in sequence, and obtains the first harmonic mean time surface of the training stage corresponding to each spatiotemporal pulse event in sequence;

[0032] Before removal The first harmonic mean time surface of each training stage is used as the time surface prototype of the first spatiotemporal coding layer of the training stage in turn;

[0033] Then the remaining first harmonic mean time surfaces in the training phase are compared with The third similarity matching is performed on the time surface prototypes to obtain the time surface prototype with the largest third similarity of the remaining first harmonic mean time surfaces in each training stage, and the time surface prototype with the largest third similarity is updated. The update formula of the time surface prototype of the first spatiotemporal coding layer in the training stage is as follows: ;

[0034] in, ;

[0035] in, represents the first spatiotemporal coding layer after the update in the training phase. A time surface prototype, represents the first spatiotemporal coding layer in the training phase. A time surface prototype, represents the remaining training phase The first harmonic mean time surface, express and The third similarity calculation is a cosine similarity calculation. represents the update amplitude of the time surface prototype of the first spatiotemporal coding layer, The remaining first harmonic mean time surface in the training phase has been The number of temporal surface prototypes that match the first spatiotemporal coding layer, It is a customized preset value;

[0036] The final updated The temporal surface prototypes are used as the temporal surface prototypes of the first spatiotemporal coding layer that have been trained;

[0037] Modify the polarity of the light intensity change of the pixel point corresponding to each spatiotemporal pulse event in each sample of the training set to the time surface prototype corresponding to the third largest similarity, and use the modified samples as the input of the second spatiotemporal coding layer in the training stage;

[0038] The second spatiotemporal coding layer sequentially calculates the spatiotemporal impulse events in each modified sample, and sequentially obtains the second harmonic mean time surface of the training phase corresponding to each spatiotemporal impulse event;

[0039] Before removal The second harmonic mean time surface of each training stage is used as the time surface prototype of the second spatiotemporal coding layer of the training stage in turn;

[0040] Then the remaining second harmonic mean time surfaces in the training phase are compared with The time surface prototype is matched with the fourth similarity to obtain the time surface prototype with the largest fourth similarity of the remaining second harmonic mean time surface in each training stage, and the time surface prototype with the largest fourth similarity is updated. The update formula of the time surface prototype of the second spatiotemporal coding layer in the training stage is as follows: ;

[0041] in, ;

[0042] in, represents the updated second spatiotemporal coding layer in the training phase. A time surface prototype, represents the second spatiotemporal coding layer in the training stage. A time surface prototype, represents the remaining training phase The second harmonic mean time surface, express and The fourth similarity calculation is a cosine similarity calculation, represents the update amplitude of the temporal surface prototype of the second spatiotemporal coding layer, Indicates that the remaining second harmonic mean time surface in the training phase has been The number of temporal surface prototypes matched in the second spatiotemporal coding layer;

[0043] The final updated The temporal surface prototypes are used as the temporal surface prototypes of the trained second spatiotemporal coding layer;

[0044] The temporal surface prototype with the fourth largest similarity is added to the spatiotemporal pulse events of each modified sample, and the added samples are used as the input of the pooling layer in the training stage in turn. The output of the pooling layer in the training stage is then used as the input of the single-layer pulse neural network of the population encoding Tempotron neurons in the training stage to realize the training of the entire target recognition model.

[0045] Preferably, the single-layer pulse neural network of the population encoding Tempotron neurons includes Category associated Tempotron neurons;

[0046] When the output of the pooling layer corresponding to each slice is input to the single-layer spiking neural network of the population coding Tempotron neurons, the output of the single-layer spiking neural network of the population coding Tempotron neurons is the same as the firing sequence of the Tempotron neurons corresponding to each slice, and the firing sequence of the Tempotron neurons is expressed as [ z 1 , z 2 ,..., z c ,..., z C ] ,in Indicates the Category associated The number of spikes fired by a Tempotron neuron, z c ∈ [1,Z] ;

[0047] The category with the largest number of pulses in the Tempotron neuron firing sequence corresponding to each slice is taken as the final target category of each slice.

[0048] An object recognition system based on event stream adaptive slicing, comprising:

[0049] A space-time pulse event stream receiving module is used to obtain the space-time pulse event stream asynchronously output by the event camera, wherein the space-time pulse event stream includes multiple space-time pulse events;

[0050] The capture module is used to transmit the spatiotemporal pulse event stream into the spiking neuron and capture the membrane voltage change waveform of the spiking neuron;

[0051] An adaptive segmentation module is used to adaptively slice each peak interval of the membrane voltage waveform;

[0052] An input module is used to sequentially input each slice into a trained target recognition model to obtain the Tempotron neuron firing sequence corresponding to each slice. The target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal coding layer, a second spatiotemporal coding layer, a pooling layer, and a population coding Tempotron neuron.

[0053] A dynamic termination module is used to calculate the similarity between the Tempotron neuron firing sequence of each slice and the previous adjacent Tempotron neuron firing sequence. When the similarity of multiple consecutive adjacent Tempotron neuron firing sequences is greater than a first threshold, the subsequent slices are stopped from being input into the trained target recognition model.

[0054] The recognition module is used to determine the category with the largest number of pulses in each Tempotron neuron firing sequence corresponding to each slice input into the trained target recognition model as the final target category of each slice.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] This target recognition method and system based on adaptive slicing of event streams uses neuron membrane voltage waveforms to dynamically segment spatiotemporal pulse event streams, effectively eliminating noise and redundant information and achieving more complete capture of action fragments. It also introduces a dynamic termination strategy that evaluates the sliced ​​Tempotron neuron firing sequence in real time. By reasoning segment by segment, it stops processing subsequent event streams when the reasoning results tend to be stable, achieving rapid recognition without processing the entire event stream, significantly reducing computational overhead while maintaining accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the process of the target recognition method based on event stream adaptive slicing of the present invention;

[0058] Figure 2 Schematic diagram of the modules of the target recognition method based on adaptive slicing of event streams of the present invention;

[0059] Figure 3 Schematic diagram of the structure of the target recognition model of the present invention;

[0060] Figure 4 A comparison of the accuracy and average time percentage of the target recognition model using slices and without slices in the DVS128Gesture training set;

[0061] Figure 5 A comparison of the accuracy and average time percentage of the target recognition model using slices and unsliced ​​objects in the MINIST-DVS training set. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0063] In one embodiment, Figure 1-Figure 3 As shown, a target recognition method based on event stream adaptive slicing is provided, including:

[0064] Step 1: Obtain a spatiotemporal pulse event stream asynchronously output by an event camera, wherein the spatiotemporal pulse event stream includes multiple spatiotemporal pulse events;

[0065] Among them, the spatiotemporal pulse event flow is expressed as E={ e i ∣ e i =[ x i , y i , t i , p i ] T ,i∈[1,N]} ,in represents the number of space-time pulse events in the space-time pulse event stream (a positive integer), represents transpose, Indicates the The pixel abscissa of a spatiotemporal pulse event, Indicates the The pixel ordinate of a spatiotemporal pulse event, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

[0066] It should be noted that the event camera is a neuromorphic device inspired by the human retina and is different from traditional frame-based sensors. Instead of capturing visual input from all pixels at a fixed rate to form an image, the output of the event camera follows an address event representation protocol, in which each pixel independently emits an event when it detects a significant change in light intensity, resulting in an asynchronous sparse event stream representing the scene. Compared to traditional cameras, AER cameras have significant advantages, including high temporal resolution, low power consumption, and extremely high dynamic range (up to 140dB).

[0067] Step 2: The spatiotemporal pulse event stream is transferred to the spiking neuron to capture the membrane voltage change waveform of the spiking neuron;

[0068] The spiking neuron is the Lif (Leaky Integrate and Fire) neuron;

[0069] The calculation formula for determining the membrane voltage of the spiking neuron is: ;

[0070] in, ;

[0071] in, is a double exponential kernel function, Indicates the current time The membrane voltage, and are the decay time constants of membrane integration and synaptic current, respectively, and are coefficients that ensure that the maximum value of the postsynaptic potential kernel is normalized to 1.

[0072] Step 3: Adaptively taking each peak interval of the membrane voltage waveform as a slice;

[0073] Before adapting each peak interval of the membrane voltage change waveform as a slice, after capturing the membrane voltage change waveform of the spiking neuron, the membrane voltage change waveform is smoothed (a Savitzky-Golay filter is used to apply a smoothing process to the membrane potential, thereby effectively capturing the potential trend in the membrane potential waveform);

[0074] In the process of adaptively using each peak interval of the membrane voltage change waveform as a slice, each peak interval of the smoothed membrane voltage change waveform is sequentially used as a candidate slice (wherein the peak interval is: the interval between two adjacent troughs in the membrane voltage change waveform), and the number of spatiotemporal pulse events corresponding to each candidate slice is compared with a preset second threshold (wherein the second threshold is the minimum number of spatiotemporal pulse events contained in the preset slice);

[0075] When it is greater than or equal to the second threshold, the current candidate slice is taken as a slice;

[0076] If the number of spatiotemporal pulse events is less than the second threshold, the next candidate slice is integrated with the current candidate slice to obtain a new candidate slice, and the number of spatiotemporal pulse events corresponding to the new candidate slice is compared with the preset second threshold until the number of spatiotemporal pulse events corresponding to the new candidate slice is greater than or equal to the second threshold, and the current new candidate slice is taken as a slice (that is, as long as the number of spatiotemporal pulse events contained in the integrated new candidate slice is less than the second threshold, the next candidate slice is continuously integrated until the number of spatiotemporal pulse events corresponding to the new candidate slice is greater than or equal to the second threshold, and the new candidate slice is taken as a slice);

[0077] Each slice is represented by { e i ∣ e i =[ x i , y i , t i , p i ] T ,i∈[1,M]} ,in represents the number of spatiotemporal pulse events in a slice, and the number of spatiotemporal pulse events in each slice is equal or unequal, and is greater than or equal to a second threshold, and .

[0078] Step 4: Input each slice into the trained target recognition model in sequence to obtain the Tempotron neuron firing sequence corresponding to each slice in sequence, where the target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal coding layer, a second spatiotemporal coding layer, a pooling layer, and a group coding Tempotron neuron connected in sequence;

[0079] In step 4.1, after the slice is input into the trained target recognition model, the first spatiotemporal coding layer uses the spatial neighborhood radius and the time decay constant Calculate each space-time pulse event in the slice in sequence, and obtain the first harmonic mean time surface corresponding to each space-time pulse event in sequence;

[0080] The specific calculation formula of the first harmonic mean time surface is as follows: ;

[0081] in, ; ;

[0082] in, Indicates the The first harmonic mean time surface corresponding to the space-time pulse event, Indicates the current The past of a space-time pulse event A space-time pulse event, is the offset, u= [ u x , u y ] T ,and u x ∈ [ - R 1 , R 1 ] , u y ∈ [ - R 1 , R 1 ] , For the The offset pixels within the receptive field of a spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of a spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Indicates the The timestamp of the space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal pulse event in the spatiotemporal pulse event stream Past The time of the most recent historical space-time pulse event, Indicates the pixels of spatiotemporal impulse events, and , Represents pixels The event set of all past spacetime pulse events, Indicates taking The most recent historical space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

[0083] Step 4.2: Perform first similarity matching (using cosine similarity calculation) on each first harmonic mean time surface and all time surface prototypes in the first spatiotemporal coding layer in turn, and use the time surface prototype with the largest first similarity as the time surface prototype of each corresponding spatiotemporal pulse event. Modify the polarity of the light intensity change of the pixel point corresponding to the corresponding spatiotemporal pulse event to the time surface prototype with the largest first similarity, and obtain the modified spatiotemporal pulse event, which is expressed as e i '=[ x i , y i , t i , p k 1 ] T ,in The first harmonic mean time surface corresponding to the current space-time pulse event has the largest first similarity. A time surface prototype, k 1 ∈ [1, K 1 ] , is the number of temporal surface prototypes in the first spatiotemporal coding layer;

[0084] Step 4.3: Use each modified spatiotemporal pulse event as the input of the second spatiotemporal coding layer in turn, and the second spatiotemporal coding layer adopts the spatial neighborhood radius and the time decay constant Calculating each modified space-time pulse event in sequence, and obtaining a second harmonic mean time surface corresponding to each modified space-time pulse event in sequence;

[0085] The specific calculation formula of the second harmonic mean time surface is as follows: ;

[0086] in, ; ;

[0087] in, Indicates the The second harmonic mean time surface corresponding to the modified space-time impulse event, is the offset, u'= [ u x ' , u y ' ] T ,and u x ' ∈ [ - R 2 , R 2 ] , u y ' ∈ [ - R 2 , R 2 ] , For the The offset pixels within the receptive field of the modified spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of the modified spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal spike event in the modified spatiotemporal spike event stream Past The time of the most recent historical space-time pulse event, Represents pixels The event set of all past spacetime pulse events.

[0088] Step 4.4: Perform a second similarity match (using cosine similarity calculation) on each second harmonic mean time surface and all time surface prototypes in the second space-time coding layer in turn, and use the time surface prototype with the largest second similarity as the time surface prototype of the modified space-time pulse event. Then, add the time surface prototype with the largest second similarity to the corresponding modified space-time pulse event to obtain the added space-time pulse event, which is expressed as e i ''= [ x i , y i , t i , p k 1 , p k 2 ] T ,in The second largest similarity of the second harmonic mean time surface corresponding to the current space-time pulse event A time surface prototype, k 2 ∈ [1, K 2 ] , is the number of temporal surface prototypes in the second spatiotemporal coding layer;

[0089] Step 4.5: Pass each added spatiotemporal pulse event through the pooling layer and the single-layer pulse neural network of the group encoding Tempotron neurons in turn to obtain the Tempotron neuron firing sequence of the current slice.

[0090] Step 5: For each slice's Tempotron neuron firing sequence, calculate the similarity between the current Tempotron neuron firing sequence and the previous adjacent Tempotron neuron firing sequence (using cosine similarity). When the similarity of multiple consecutive (e.g., in this embodiment, three consecutive) adjacent Tempotron neuron firing sequences is greater than a first threshold, stop inputting subsequent slices into the trained target recognition model (Step 5 is a dynamic termination strategy that can significantly reduce computational redundancy and improve the efficiency of event stream processing. This strategy evaluates the slice's Tempotron neuron firing sequence in real time and, through segment-by-segment reasoning, stops processing subsequent event streams when the inference results stabilize, achieving rapid recognition without processing the entire event stream, significantly reducing computational overhead while maintaining accuracy).

[0091] It should be noted that, after obtaining the Tempotron neuron firing sequence of the first slice, the Tempotron neuron firing sequence of the second slice, the Tempotron neuron firing sequence of the third slice, and the Tempotron neuron firing sequence of the fourth slice, and among the four slices, the Tempotron neuron firing sequence similarity between two adjacent slices is less than the first threshold, when the Tempotron neuron firing sequence of the fifth slice is obtained, the similarity between the Tempotron neuron firing sequence of the fifth slice and the Tempotron neuron firing sequence of the fourth slice is calculated. If it is greater than the first threshold, then The sixth slice is input into the trained target recognition model to obtain the Tempotron neuron firing sequence of the sixth slice, and the similarity between the Tempotron neuron firing sequence of the sixth slice and the Tempotron neuron firing sequence of the fifth slice is calculated. When the similarity of three consecutive adjacent Tempotron neuron firing sequences is greater than the first threshold, that is, the similarity between the Tempotron neuron firing sequence of the seventh slice and the Tempotron neuron firing sequence of the sixth slice is greater than the first threshold, the subsequent slices are stopped from being input into the trained target recognition model, and the Tempotron neuron firing sequences of seven slices are obtained.

[0092] Step 6: For each Tempotron neuron firing sequence corresponding to the slice input into the trained target recognition model, the category with the largest number of pulses fired in the Tempotron neuron firing sequence is selected as the final target category of each slice (i.e., in this embodiment, for the Tempotron neuron firing sequences corresponding to the seven slices, the category with the largest number of pulses fired from each Tempotron neuron firing sequence is selected as the final target category of each slice, thereby obtaining all target categories of the spatiotemporal pulse event stream).

[0093] It should be noted that the single-layer pulse neural network of population coding Tempotron neurons includes categories (the The categories are the number of categories contained in the training set of spatiotemporal pulse event streams in the training phase) Tempotron neurons;

[0094] When the output of the pooling layer corresponding to each slice is input to the single-layer spiking neural network of the population coding Tempotron neurons, the output of the single-layer spiking neural network of the population coding Tempotron neurons is the same as the firing sequence of the Tempotron neurons corresponding to each slice, and the firing sequence of the Tempotron neurons is expressed as [ z 1 , z 2 ,..., z c ,..., z C ] ,in Indicates the Category associated The number of spikes fired by a Tempotron neuron, z c ∈ [1,Z] ; That is, the firing sequence of Tempotron neurons is The sequence of the number of pulses emitted corresponds to each category.

[0095] The category with the largest number of pulses in the Tempotron neuron firing sequence corresponding to each slice is taken as the final target category of each slice.

[0096] Among them, the spiking neural network is a brain-inspired computing model. Unlike traditional neural networks, the spiking neural network transmits and processes information by simulating the pulse discharge of neurons. The event camera records visual input through asynchronous discrete events and can well adapt to the asynchronous spike communication model of the spiking neural network. The event-driven Tempotron learning rule is used to classify the extracted spatiotemporal features.

[0097] Tempotron is a supervised learning algorithm for spiking neural networks. A single Tempotron neuron can effectively complete a binary classification task by simply marking the neuron as firing or not firing. For multi-classification tasks, a combination of multiple Tempotron neurons is required. Tempotron uses the Leaky Integrate-and-Fire (LIF) model as its neuron model. is the weighted sum of the postsynaptic voltages generated by the presynaptic spikes of each tempotron neuron, where the peak amplitude and shape of the postsynaptic voltage are determined by its corresponding synaptic weight and The membrane voltage of the single-layer pulse neural network of the population coding Tempotron neurons is determined to be the weighted sum of the postsynaptic potentials generated by all presynaptic Tempotron neurons: ;

[0098] in, ;

[0099] in, Indicates the The synaptic weight value of the presynaptic Tempotron neuron, Indicates the The timestamp of the presynaptic Tempotron neuron, represents the resting potential of the Tempotron neuron;

[0100] The Tempotron learning rule updates the synaptic connection weight by calculating the difference between the peak value of the neuron membrane voltage and the pulse firing threshold. The calculation formula of this variable of the synaptic weight value is as follows: ;

[0101] in, Indicates the The update amount of synaptic weights, Indicates the moment when the membrane voltage of the Tempotron neuron reaches its peak, Represents the learning rate.

[0102] In this method, a pulse neural network composed of Tempotron neurons is used as the classifier of the system. The input pulse sequence is the event stream of the feature event stream output by the spatiotemporal feature extraction in the previous stage, which includes three variables: timestamp, weight address index, and category. Considering that a single neuron is easily interfered with, this method uses a group encoding method to associate multiple Tempotron neurons for each category, and expresses the output signal through the joint activity of multiple neurons to enhance the neuron's expressive ability and robustness against noise. Classification task, each category is associated with Tempotron neurons are associated, so the output layer has neuronal groups and × Tempotron neurons. During the learning phase of the spiking neural network, only the Tempotron neurons in the corresponding category group need to fire, while Tempotron neurons in other groups do not. During the classification prediction phase of the spiking neural network, a majority voting method is used. That is, the Tempotron neuron group with the largest number of pulses predicts the pulse pattern as the corresponding category, thereby achieving target object recognition. The spiking neural network in this model uses an event-driven simulation method, which has higher recognition efficiency.

[0103] In this embodiment, the target recognition model is in the training phase, and the training set includes Categories (such as the MNIST-DVS training set or the DVS128 Gesture training set), each category contains multiple samples, and each sample is a spatiotemporal pulse event stream. All samples in the training set are shuffled and input into the first spatiotemporal coding layer of the training stage in sequence. The first spatiotemporal coding layer calculates the spatiotemporal pulse events in each sample in sequence, and obtains the first harmonic mean time surface of the training stage corresponding to each spatiotemporal pulse event in sequence;

[0104] Before removal The first harmonic mean time surface of each training stage is used as the time surface prototype of the first spatiotemporal coding layer of the training stage in turn;

[0105] Then take out the remaining The first harmonic mean time surface or front After the second harmonic mean time surface, all the remaining harmonic mean time surfaces) the first harmonic mean time surfaces are respectively The third similarity matching is performed on the time surface prototypes to obtain the time surface prototype with the largest third similarity of the remaining first harmonic mean time surfaces in each training stage, and the time surface prototype with the largest third similarity is updated. The update formula of the time surface prototype of the first spatiotemporal coding layer in the training stage is as follows: ;

[0106] in, ;

[0107] in, represents the first spatiotemporal coding layer after the update in the training phase. A time surface prototype, represents the first spatiotemporal coding layer in the training phase. A time surface prototype, represents the remaining training phase The first harmonic mean time surface, express and The third similarity calculation is a cosine similarity calculation. represents the update amplitude of the time surface prototype of the first spatiotemporal coding layer, The remaining first harmonic mean time surface in the training phase has been The number of temporal surface prototypes that match the first spatiotemporal coding layer, and The larger the value, the smaller the update amplitude. It is a custom preset value (obtained based on experience);

[0108] The final updated The temporal surface prototypes are used as the temporal surface prototypes of the first spatiotemporal coding layer that have been trained;

[0109] Modify the polarity of the light intensity change of the pixel point corresponding to each spatiotemporal pulse event in each sample of the training set to the time surface prototype corresponding to the third largest similarity, and use the modified samples as the input of the second spatiotemporal coding layer in the training stage;

[0110] The second spatiotemporal coding layer sequentially calculates the spatiotemporal impulse events in each modified sample, and sequentially obtains the second harmonic mean time surface of the training phase corresponding to each spatiotemporal impulse event;

[0111] Before removal The second harmonic mean time surface of each training stage is used as the time surface prototype of the second spatiotemporal coding layer of the training stage in turn;

[0112] Then the remaining second harmonic mean time surfaces in the training phase are compared with The time surface prototype is matched with the fourth similarity to obtain the time surface prototype with the largest fourth similarity of the remaining second harmonic mean time surface in each training stage, and the time surface prototype with the largest fourth similarity is updated. The update formula of the time surface prototype of the second spatiotemporal coding layer in the training stage is as follows: ;

[0113] in, ;

[0114] in, represents the updated second spatiotemporal coding layer in the training phase. A time surface prototype, represents the second spatiotemporal coding layer in the training stage. A time surface prototype, represents the remaining training phase The second harmonic mean time surface, express and The fourth similarity calculation is a cosine similarity calculation, represents the update amplitude of the temporal surface prototype of the second spatiotemporal coding layer, Indicates that the remaining second harmonic mean time surface in the training phase has been The number of temporal surface prototypes matched in the second spatiotemporal coding layer;

[0115] The final updated The temporal surface prototypes are used as the temporal surface prototypes of the trained second spatiotemporal coding layer;

[0116] The temporal surface prototype with the fourth largest similarity is added to the spatiotemporal pulse events of each modified sample, and the added samples are used as the input of the pooling layer in the training stage in turn. The output of the pooling layer in the training stage is then used as the input of the single-layer pulse neural network of the population encoding Tempotron neurons in the training stage to realize the training of the entire target recognition model.

[0117] In another embodiment, based on a method for target recognition based on adaptive slicing of event streams, the present application further provides a target recognition system based on adaptive slicing of event streams, including:

[0118] A space-time pulse event stream receiving module is used to obtain the space-time pulse event stream asynchronously output by the event camera, and the space-time pulse event stream contains multiple space-time pulse events;

[0119] The capture module is used to transmit the spatiotemporal pulse event stream into the spiking neuron and capture the membrane voltage change waveform of the spiking neuron;

[0120] An adaptive segmentation module is used to adaptively slice each peak interval of the membrane voltage waveform;

[0121] An input module is used to sequentially input each slice into a trained target recognition model to obtain the Tempotron neuron firing sequence corresponding to each slice. The target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal coding layer, a second spatiotemporal coding layer, a pooling layer, and a population coding Tempotron neuron.

[0122] A dynamic termination module is used to calculate the similarity between the Tempotron neuron firing sequence of each slice and the previous adjacent Tempotron neuron firing sequence. When the similarity of multiple consecutive adjacent Tempotron neuron firing sequences is greater than a first threshold, the subsequent slices are stopped from being input into the trained target recognition model.

[0123] The recognition module is used to determine the category with the largest number of pulses in each Tempotron neuron firing sequence corresponding to each slice input into the trained target recognition model as the final target category of each slice.

[0124] For the specific limitations of the target recognition system based on adaptive slicing of event streams, please refer to the limitations of the target recognition method based on adaptive slicing of event streams mentioned above, which will not be repeated here. The various modules in the above-mentioned target recognition system based on adaptive slicing of event streams can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0125] The following specific experimental data demonstrates the advantages of the target recognition method based on event stream adaptive slicing:

[0126] This method achieves high recognition accuracy on neuromorphic training sets (DVS128Gesture, N-MNIST, and MNIST-DVS). This method effectively extracts spatiotemporal features from spatiotemporal pulse event streams and the dynamic activity of past events within the receptive field of each incoming event, thereby capturing scene dynamics and achieving high recognition accuracy on the neuromorphic action training set. Table 1 compares the recognition accuracy of this method with that of existing techniques on the neuromorphic training set (where existing techniques refer to Comparative Examples 1 through 7 in Table 1):

[0127] Table 1

[0128] Among them, Comparative Example 1: R. Xiao, H. Tang, Y. Ma, R. Yan, and G. Orchard, “An event-driven categorization model for aer image sensors using multispike encoding and learning,” IEEE transactions on neural networks and learning systems, vol. 31, no. 9, pp. 3649–3657, 2019.

[0129] Comparative Example 2: Q. Liu, H. Ruan, D. Xing, H. Tang, and G. Pan, “Effective aer object classification using segmented probability-maximization learning in spiking neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 1308–1315, 2020.

[0130] Comparative Example 3: M. Su, P. Yang, R. Jiang, and R. Yan, “Event-based object recognition using feature fusion and spiking neural networks,” in International Conference on Neural Information Processing, pp. 470–482, Springer, 2023.

[0131] Comparative Example 4: X. Lagorce, G. Orchard, F. Galluppi, BE Shi, and RBBenosman, “Hots: a hierarchy of event-based time-surfaces for pattern recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 7, pp. 1346–1359, 2016.

[0132] Comparative Example 5: SB Shrestha and G. Orchard, "Slayer: Spike layer errorreassignment in time," Advances in neural information processing systems, vol.31, 2018.

[0133] Comparative Example 6: W. He, Y. Wu, L. Deng, G. Li, H. Wang, Y. Tian, ​​W. Ding, W. Wang, and Y. Xie, “Comparing snns and rnns on neuromorphic vision datasets: Similarities and differences,” Neural Networks, vol. 132, pp. 108–120, 2020.

[0134] Comparative Example 7: A. Grimaldi, V. Boutin, S.-H. Ieng, R. Benosman, and LUPerrinet, “A robust event-driven approach to always-on object recognition,” Neural Networks, p. 106415, 2024.

[0135] This method can dynamically adjust the segmentation strategy of the pulse event stream to adapt to different scenarios and event densities, and has greater robustness and flexibility. Table 2 compares the common event stream segmentation methods and this method on the MNIST-DVS training set and the DVS128Gesture training set:

[0136] Table 2

[0137] Among them, Comparative Example 8: Y. Chen, W. Bai, Q. Huang, and J. Xiao, "Efficientmotion symbol detection and multikernel learning for aer object recognition," IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 4, pp.1544–1552, 2021.

[0138] Comparative Example 9: M. Su, P. Yang, R. Jiang, and R. Yan, “Event-based object recognition using feature fusion and spiking neural networks,” in International Conference on Neural Information Processing, pp. 470–482, Springer, 2023.

[0139] This method can achieve the highest accuracy by using only part of the information in the acquired spatiotemporal pulse event stream. Through the adaptive segmentation method based on the membrane voltage waveform and the dynamic termination strategy, it can significantly reduce computational redundancy and improve the efficiency of event stream processing. It is particularly suitable for application scenarios that require real-time response. Figure 4 As shown in the figure, in the DVS128Gesture training set, when the target recognition model received only 32% of the spatiotemporal pulse event stream data, the accuracy improved by 2.14% when using slicing processing compared to no slicing. Moreover, after receiving 61.26% of the data, our target recognition model achieved a maximum accuracy of 95.19%. The accuracy remained stable over time, even exceeding the 94.65% achieved when 100% of the unprocessed data was input. Figure 5 As shown in Figure 3, on the MINIST-DVS training set, sliced ​​data also shows higher accuracy than unsliced ​​data.

[0140] This method leverages the precise temporal information of spatiotemporal spike events in an event-driven manner. The spiking neural network event-driven algorithm extracts and preserves as much precise temporal information as possible when processing the spatiotemporal spike events output by the event camera. The event camera asynchronously encodes visual information as a stream of spatiotemporal spike events, with each pixel immediately emitting a pulse when light intensity changes, enabling real-time response to changes in the visual scene. The spiking neural network event-driven algorithm effectively utilizes these spatiotemporal spike events without converting them into traditional fixed-frame-rate images. In contrast, traditional frame-driven algorithms require converting the event stream into fixed-frame-rate images for preprocessing and classification learning. In this process, a large amount of event information is compressed into the fixed-frame-rate images, losing the precise temporal information of the original events. This frame conversion preprocessing results in information loss and redundancy, potentially reducing the accuracy and efficiency of the algorithm.

[0141] This method, through direct processing of spatiotemporal spike events in an event-driven manner, can better utilize the asynchronous operation of event cameras and preserve the precise timing information of the original events as much as possible. This enables the algorithm to adapt more flexibly and efficiently to complex visual scenes and achieve better performance and accuracy in various visual perception tasks. Therefore, the event-driven form of spiking neural network algorithms has a clear advantage in processing spatiotemporal spike events output by event cameras.

[0142] This target recognition method and system based on adaptive slicing of event streams uses neuron membrane voltage waveforms to dynamically segment spatiotemporal pulse event streams, effectively eliminating noise and redundant information and achieving more complete capture of action fragments. It also introduces a dynamic termination strategy that evaluates the sliced ​​Tempotron neuron firing sequence in real time. By reasoning segment by segment, it stops processing subsequent event streams when the reasoning results tend to be stable, achieving rapid recognition without processing the entire event stream, significantly reducing computational overhead while maintaining accuracy.

[0143] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A target recognition method based on adaptive slicing of event streams, characterized by: The target recognition method based on event stream adaptive slicing includes: Acquire a spatiotemporal pulse event stream asynchronously output by an event camera, wherein the spatiotemporal pulse event stream includes a plurality of spatiotemporal pulse events; The spatiotemporal pulse event stream is transferred to the spiking neuron to capture the membrane voltage change waveform of the spiking neuron; Adaptively use each peak interval of the membrane voltage change waveform as a slice; Each slice is sequentially input into a trained target recognition model to obtain the corresponding Tempotron neuron firing sequence for each slice. The target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal encoding layer, a second spatiotemporal encoding layer, a pooling layer, and a population encoding Tempotron neuron. Each time a slice's Tempotron neuron firing sequence is obtained, the similarity between the current Tempotron neuron firing sequence and the previous adjacent Tempotron neuron firing sequence is calculated. When the similarity of multiple consecutive adjacent Tempotron neuron firing sequences is greater than a first threshold, the subsequent slices are stopped from being input into the trained target recognition model. For each Tempotron neuron firing sequence corresponding to the slice input into the trained target recognition model, the category with the largest number of pulses fired in each Tempotron neuron firing sequence is taken as the final target category of each slice.

2. The target recognition method based on event stream adaptive slicing according to claim 1, characterized in that: The spatiotemporal pulse event stream is expressed as ,in represents the number of spatiotemporal pulse events in the spatiotemporal pulse event stream, represents transpose, Indicates the The pixel abscissa of a spatiotemporal pulse event, Indicates the The pixel ordinate of a spatiotemporal pulse event, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

3. The target recognition method based on event stream adaptive slicing according to claim 2, characterized in that: After capturing the membrane voltage change waveform of the pulse neuron, the membrane voltage change waveform is smoothed; In the process of adaptively taking each peak interval of the membrane voltage change waveform as a slice, each peak interval of the smoothed membrane voltage change waveform is sequentially taken as a candidate slice, and the number of spatiotemporal pulse events corresponding to each candidate slice is compared with a preset second threshold; When it is greater than or equal to the second threshold, the current candidate slice is taken as a slice; If the number of spatiotemporal pulse events is less than the second threshold, the next candidate slice is integrated with the current candidate slice to obtain a new candidate slice, and the number of spatiotemporal pulse events corresponding to the new candidate slice is compared with the preset second threshold. If the number of spatiotemporal pulse events corresponding to the new candidate slice is greater than or equal to the second threshold, the current new candidate slice is taken as a slice. The peak interval is the interval between two adjacent troughs in the membrane voltage change waveform.

4. The target recognition method based on event stream adaptive slicing according to claim 3, characterized in that: After the slice is input into the trained target recognition model, the first spatiotemporal coding layer uses the spatial neighborhood radius and the time decay constant Calculate each space-time pulse event in the slice in sequence, and obtain the first harmonic mean time surface corresponding to each space-time pulse event in sequence; The first harmonic mean time surfaces are sequentially matched with all time surface prototypes in the first spatiotemporal coding layer for the first similarity, and the time surface prototype with the largest first similarity is used as the time surface prototype of each corresponding spatiotemporal pulse event. The polarity of the light intensity change of the pixel point corresponding to the corresponding spatiotemporal pulse event is modified to the time surface prototype with the largest first similarity, and the modified spatiotemporal pulse event is obtained, which is expressed as ,in The first harmonic mean time surface corresponding to the current space-time pulse event has the largest first similarity. A time surface prototype, , is the number of temporal surface prototypes in the first spatiotemporal coding layer; Each modified spatiotemporal pulse event is sequentially used as the input of the second spatiotemporal coding layer, and the second spatiotemporal coding layer adopts the spatial neighborhood radius and the time decay constant Calculating each modified space-time pulse event in sequence, and obtaining a second harmonic mean time surface corresponding to each modified space-time pulse event in sequence; Perform a second similarity match on each second harmonic mean time surface with all time surface prototypes in the second space-time coding layer in turn, use the time surface prototype with the largest second similarity as the time surface prototype of the modified space-time pulse event, and add the time surface prototype with the largest second similarity to the corresponding modified space-time pulse event to obtain the added space-time pulse event, which is expressed as ,in The second largest similarity of the second harmonic mean time surface corresponding to the current space-time pulse event A time surface prototype, , is the number of temporal surface prototypes in the second spatiotemporal coding layer; Each added spatiotemporal pulse event is then passed through the pooling layer and the single-layer pulse neural network of the group encoding Tempotron neurons to obtain the Tempotron neuron firing sequence of the current slice.

5. The target recognition method based on event stream adaptive slicing according to claim 4, characterized in that: The first spatiotemporal coding layer uses a spatial neighborhood radius and the time decay constant Each spatiotemporal impulse event in the slice is calculated in sequence, and the first harmonic mean time surface corresponding to each spatiotemporal impulse event is obtained in sequence. The specific calculation formula of the first harmonic mean time surface is as follows: ； in, ； ； in, Indicates the The first harmonic mean time surface corresponding to the space-time pulse event, Indicates the current The past of a space-time pulse event A space-time pulse event, is the offset, ,and , , For the The offset pixels within the receptive field of a spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of a spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Indicates the The timestamp of the space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal pulse event in the spatiotemporal pulse event stream Past The time of the most recent historical space-time pulse event, Indicates the pixels of spatiotemporal impulse events, and , Represents pixels The time set of all past space-time pulse events, Indicates taking The most recent historical space-time pulse event, Indicates the The polarity of light intensity change at the pixel corresponding to each spatiotemporal pulse event.

6. The target recognition method based on event stream adaptive slicing according to claim 5, characterized in that: The second spatiotemporal coding layer uses a spatial neighborhood radius and the time decay constant Calculate each modified space-time pulse event in sequence to obtain the second harmonic mean time surface corresponding to each modified space-time pulse event. The specific calculation formula of the second harmonic mean time surface is as follows: ； in, ； ； in, Indicates the The second harmonic mean time surface corresponding to the modified space-time impulse event, is the offset, ,and , , For the The offset pixels within the receptive field of the modified spatiotemporal spike event The time surface of the harmonic mean past time, and the time surface of the harmonic mean past time represents the pixel past The overall impact of a spatiotemporal pulse event on the current event, For the The offset pixels within the receptive field of the modified spatiotemporal spike event The harmonic mean past time, and the harmonic mean past time is the harmonic pixel past The time of the most recent historical space-time pulse event, Represents the offset pixel within the receptive field of the spatiotemporal spike event in the modified spatiotemporal spike event stream Past The time of the most recent historical space-time pulse event, Represents pixels The event set of all past spacetime pulse events.

7. The target recognition method based on event stream adaptive slicing according to claim 6, characterized in that: The target recognition model is in the training stage, and the training set contains There are multiple categories in each category, and each sample is a spatiotemporal pulse event stream. All samples in the training set are shuffled and input into the first spatiotemporal coding layer of the training stage in sequence. The first spatiotemporal coding layer calculates the spatiotemporal pulse events in each sample in sequence, and obtains the first harmonic mean time surface of the training stage corresponding to each spatiotemporal pulse event in sequence; Before removal The first harmonic mean time surface of each training stage is used as the time surface prototype of the first spatiotemporal coding layer of the training stage in turn; Then the remaining first harmonic mean time surfaces in the training phase are compared with The third similarity matching is performed on the time surface prototypes to obtain the time surface prototype with the largest third similarity of the remaining first harmonic mean time surfaces in each training stage, and the time surface prototype with the largest third similarity is updated. The update formula of the time surface prototype of the first spatiotemporal coding layer in the training stage is as follows: ； in, ； in, represents the first spatiotemporal coding layer after the update in the training phase. A time surface prototype, represents the first spatiotemporal coding layer in the training phase. A time surface prototype, represents the remaining training phase The first harmonic mean time surface, express and The third similarity calculation is a cosine similarity calculation. represents the update amplitude of the time surface prototype of the first spatiotemporal coding layer, The remaining first harmonic mean time surface in the training phase has been The number of temporal surface prototypes that match the first spatiotemporal coding layer, It is a customized preset value; The final updated The temporal surface prototypes are used as the temporal surface prototypes of the first spatiotemporal coding layer that have been trained; Modify the polarity of the light intensity change of the pixel point corresponding to each spatiotemporal pulse event in each sample of the training set to the time surface prototype corresponding to the third largest similarity, and use the modified samples as the input of the second spatiotemporal coding layer in the training stage; The second spatiotemporal coding layer sequentially calculates the spatiotemporal impulse events in each modified sample, and sequentially obtains the second harmonic mean time surface of the training phase corresponding to each spatiotemporal impulse event; Before removal The second harmonic mean time surface of each training stage is used as the time surface prototype of the second spatiotemporal coding layer of the training stage in turn; The retraining phase combines the remaining second harmonic mean time surfaces with The time surface prototype is matched with the fourth similarity to obtain the time surface prototype with the largest fourth similarity of the remaining second harmonic mean time surface in each training stage, and the time surface prototype with the largest fourth similarity is updated. The update formula of the time surface prototype of the second spatiotemporal coding layer in the training stage is as follows: ； in, ； in, represents the updated second spatiotemporal coding layer in the training phase. A time surface prototype, represents the second spatiotemporal coding layer in the training stage. A time surface prototype, represents the remaining training phase The second harmonic mean time surface, express and The fourth similarity calculation is a cosine similarity calculation, represents the update amplitude of the temporal surface prototype of the second spatiotemporal coding layer, Indicates that the remaining second harmonic mean time surface in the training phase has been The number of temporal surface prototypes matched in the second spatiotemporal coding layer; The final updated The temporal surface prototypes are used as the temporal surface prototypes of the trained second spatiotemporal coding layer; The temporal surface prototype with the fourth largest similarity is added to the spatiotemporal pulse events of each modified sample, and the added samples are used as the input of the pooling layer in the training stage in turn. The output of the pooling layer in the training stage is then used as the input of the single-layer pulse neural network of the population encoding Tempotron neurons in the training stage to realize the training of the entire target recognition model.

8. The target recognition method based on event stream adaptive slicing according to claim 7, characterized in that: The single-layer pulse neural network of the population coding Tempotron neurons includes Category associated Tempotron neurons; When the output of the pooling layer corresponding to each slice is input to the single-layer spiking neural network of the population coding Tempotron neurons, the output of the single-layer spiking neural network of the population coding Tempotron neurons is the same as the firing sequence of the Tempotron neurons corresponding to each slice, and the firing sequence of the Tempotron neurons is expressed as ,in Indicates the Category associated The number of spikes fired by a Tempotron neuron, ; The category with the largest number of pulses in the Tempotron neuron firing sequence corresponding to each slice is taken as the final target category of each slice.

9. An object recognition system based on adaptive slicing of event streams, characterized by: The target recognition system based on event stream adaptive slicing includes: A space-time pulse event stream receiving module is used to obtain the space-time pulse event stream asynchronously output by the event camera, wherein the space-time pulse event stream includes multiple space-time pulse events; The capture module is used to transmit the spatiotemporal pulse event stream into the spiking neuron and capture the membrane voltage change waveform of the spiking neuron; An adaptive segmentation module is used to adaptively slice each peak interval of the membrane voltage waveform; An input module is used to sequentially input each slice into a trained target recognition model to obtain the Tempotron neuron firing sequence corresponding to each slice. The target recognition model includes a single-layer spiking neural network consisting of a first spatiotemporal coding layer, a second spatiotemporal coding layer, a pooling layer, and a population coding Tempotron neuron. A dynamic termination module is used to calculate the similarity between the Tempotron neuron firing sequence of each slice and the previous adjacent Tempotron neuron firing sequence. When the similarity of multiple consecutive adjacent Tempotron neuron firing sequences is greater than a first threshold, the subsequent slices are stopped from being input into the trained target recognition model. The recognition module is used to determine the category with the largest number of pulses in each Tempotron neuron firing sequence corresponding to each slice input into the trained target recognition model as the final target category of each slice.

Images