Neuromorphic visual high-energy-efficiency pulse pedestrian re-identification method and system

Abstract

The invention belongs to the field of pedestrian re-identification of neuromorphic vision, and discloses a high-energy-efficiency pulse pedestrian re-identification method and system of neuromorphic vision, and the method comprises the steps: obtaining a time-space pulse event stream asynchronously outputted by an event camera; sampling and aggregating the space-time pulse event stream to obtain a continuous dense aggregated event stream, and classifying the continuous dense aggregated event stream into a reference sample, a positive sample and a negative sample; using a pulse neural network to respectively extract features of the samples as deep space-time input synaptic currents, obtaining membrane voltage-based pedestrian depth space-time features according to the deep space-time input synaptic currents, and performing compression aggregation on the membrane voltage-based pedestrian depth space-time features corresponding to each slice according to the time dimension, and obtaining a reference sample compression feature, a positive sample compression feature and a negative sample compression feature, and training and updating the pulse neural network. The method can effectively improve the accuracy rate of pedestrian re-identification based on neuromorphic vision, and can significantly reduce the energy consumption at the same time.

Description

Technical Field

[0001] The present invention belongs to the field of pedestrian re-identification based on neuromorphic vision, and specifically relates to a high-energy-efficiency pulse pedestrian re-identification method and system based on neuromorphic vision. It is a technology for pedestrian re-identification that directly trains low-energy pulse neural networks end-to-end based on event stream data. Background Art

[0002] Person re-identification (Re-ID) aims to identify specific individuals across multiple cameras with different viewpoints and perspectives, and is widely used in fields such as video surveillance and tracking. However, the widespread use of video and image data in these applications brings several challenges. The long-term collection of such data, especially using frame-based sensors, leads to high energy consumption for recording and storage, as well as potential privacy and security issues. While there are some person re-identification methods based on visible infrared patterns, most existing models still rely on traditional frame-based sensors.

[0003] Inspired by the principles of biological vision systems, event cameras offer a novel approach to dynamic visual perception. Unlike traditional frame-based cameras, event cameras do not capture images at fixed intervals. Instead, they respond to changes in the brightness of individual pixels, recording and generating "events" only when pixel intensity changes significantly. This design offers several advantages, including high temporal resolution (>10k fps), low power consumption (<10mW), and a wide dynamic range (>120dB). These attributes make event cameras particularly well-suited for re-identification tasks in environments with limited energy, rapid motion, extreme lighting conditions, and motion blur. Furthermore, unlike frame-based sensors, event cameras do not capture detailed image information of objects, which can help protect personal privacy. However, because event cameras do not provide detailed video frame-based content, event-driven processing emphasizes capturing the outlines of objects and their dynamic motion over time.

[0004] Spiking Neural Networks (SNNs) have attracted widespread attention due to their asynchronous computational properties, biological plausibility, and low energy consumption. In recent years, numerous specialized network modules and architectures have been developed as part of a concerted effort to narrow the performance gap between SNNs and artificial neural networks across various tasks. However, most direct training methods for SNNs focus primarily on classification tasks, and spike-based pedestrian feature matching suffers from significant ambiguity in the field of person re-identification. Consequently, existing event-stream-based person re-identification techniques have numerous shortcomings, necessitating a new, energy-efficient, and accurate SNN-based person re-identification solution to meet the demands of autonomous systems for person re-identification in complex environments. Summary of the Invention

[0005] The purpose of the present invention is to provide a high-energy-efficiency pulse pedestrian re-identification method and system for neuromorphic vision, which can effectively improve the accuracy of neuromorphic vision pedestrian re-identification while significantly reducing energy consumption.

[0006] To achieve the above object, the technical solution adopted by the present invention is: In a first aspect, a high-energy-efficiency pulsed person re-identification method based on neuromorphic vision is provided, comprising: Get the spatiotemporal pulse event stream asynchronously output by the event camera; Sampling the spatiotemporal pulse event stream to obtain a preliminary sampling event stream, aggregating the preliminary sampling event stream to obtain a continuous dense aggregated event stream, wherein the continuous dense aggregated event stream is composed of multiple slices; A continuous dense aggregated event stream is used as a benchmark sample, a continuous dense aggregated event stream with the same pedestrian category as the benchmark sample is used as a positive sample, and a continuous dense aggregated event stream with a different pedestrian category from the benchmark sample is used as a negative sample; A spiking neural network is used to extract the features of reference samples, positive samples, and negative samples as deep spatiotemporal input synaptic currents, and the deep spatiotemporal features of pedestrians based on membrane voltage are obtained according to the deep spatiotemporal input synaptic currents. The pedestrian deep spatiotemporal features based on membrane voltage corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression features, positive sample compression features and negative sample compression features; The triplet loss is calculated based on the distance between the compressed features of the benchmark sample and the compressed features of the positive sample, as well as the distance between the compressed features of the benchmark sample and the compressed features of the negative sample. The category prediction loss is calculated based on the pedestrian re-identification results corresponding to the compressed features of the benchmark sample. The triplet loss and category prediction loss are combined to update the spiking neural network. The updated spiking neural network is used to obtain the compressed features of the spatiotemporal spiking event stream to be identified, and the pedestrian re-identification results are obtained based on the compressed features.

[0007] Preferably, sampling the spatiotemporal pulse event stream to obtain a preliminary sampled event stream, and aggregating the preliminary sampled event stream to obtain a continuous dense aggregated event stream, comprises: Take the spatiotemporal pulse event stream, i.e. a set of space-time pulse event stream , Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events in intervals for the discrete time dimension, scaling the timestamps to , and generates the event body as follows: .

[0008] Spatiotemporal pulse event stream Perform sampling to obtain a preliminary sampling event flow , is the index of the initial sampling event stream, represents the total number of spatiotemporal pulse events in the preliminary sampling event stream; The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream Slice at the moment, aggregate all preliminary sampled event streams to get a shape of A continuous dense aggregate event stream, where is the number of slices of the continuous densely aggregated event stream, , Indicates the batch size, represents the total number of channel dimensions, Represents the height of a continuous dense aggregate event stream, Indicates the width of a continuous dense aggregate event stream.

[0009] Preferably, the method of using a spiking neural network to extract the features of the reference sample, the positive sample, and the negative sample as the deep spatiotemporal input synaptic current, and obtaining the deep spatiotemporal features of the pedestrian based on the membrane voltage according to the deep spatiotemporal input synaptic current, includes: Take the reference sample, positive sample or negative sample as the current sample and use the pulse neural network to extract the current sample The characteristics of the slice at the moment are obtained Deep spatiotemporal input synaptic currents at time ; Deep spatiotemporal input synaptic currents , calculate the deep spatiotemporal features of pedestrians based on membrane voltage: Where, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, is the time decay factor, is the membrane time constant.

[0010] Preferably, the membrane voltage-based pedestrian deep spatiotemporal features corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression features, the positive sample compression features and the negative sample compression features, including: Pick The deep spatiotemporal characteristics of pedestrians based on membrane voltage corresponding to the slice at the moment , and get the compressed features: Where, is the compression feature, where the one obtained for the benchmark sample is the benchmark sample compression feature, the one obtained for the positive sample is the positive sample compression feature, and the one obtained for the negative sample is the negative sample compression feature. is the number of slices of the continuous densely aggregated event stream, for The adaptive compression matrix at the moment, the dynamic update method of the adaptive compression matrix is ​​as follows: initialization The weight matrix at the moment , and based on the weight matrix Get the adaptive compression matrix: Where, for The weight matrix at time t, It is an exponential function with the natural constant e as its base.

[0011] Preferably, the triplet loss is calculated for each set of reference samples, positive samples, and negative samples, and the calculation formula is as follows: Where, is the triplet loss value, To obtain the maximum value, represents the compressed features of the benchmark samples, represents the positive sample compression feature, represents the negative sample compression feature, is the distance metric function, is a constant; Compress features for each benchmark sample , use the classifier to predict the pedestrian category probability, and calculate the cross entropy loss as the category prediction loss based on the pedestrian category probability; All triplet losses and class prediction losses are combined as the total loss, and the spiking neural network is updated based on the total loss.

[0012] Secondly, a high-energy-efficiency pulsed person re-identification system based on neuromorphic vision is provided, comprising: The event stream receiving module is used to obtain the spatiotemporal pulse event stream asynchronously output by the event camera; An event stream sampling and aggregation module, configured to sample the spatiotemporal pulse event stream to obtain a preliminary sampled event stream, and aggregate the preliminary sampled event stream to obtain a continuous dense aggregated event stream, wherein the continuous dense aggregated event stream is composed of a plurality of slices; Based on the membrane voltage feature characterization module, it is used to use a continuous dense aggregate event stream as a reference sample, a continuous dense aggregate event stream with the same pedestrian category as the reference sample as a positive sample, and a continuous dense aggregate event stream with a different pedestrian category from the reference sample as a negative sample. A spiking neural network is used to extract the features of the reference sample, positive sample, and negative sample as deep spatiotemporal input synaptic currents, and the deep spatiotemporal features of pedestrians based on membrane voltage are obtained based on the deep spatiotemporal input synaptic currents. Adaptive time-dimensional feature aggregation module, used to compress and aggregate the membrane voltage-based pedestrian deep spatiotemporal features corresponding to each slice according to the time dimension to obtain the baseline sample compression feature, positive sample compression feature and negative sample compression feature; The feature discrimination matching and prediction module is used to calculate the triplet loss based on the distance between the compressed features of the benchmark sample and the compressed features of the positive sample, as well as the distance between the compressed features of the benchmark sample and the compressed features of the negative sample, calculate the category prediction loss based on the pedestrian re-identification results corresponding to the compressed features of the benchmark sample, and update the pulse neural network by combining the triplet loss and category prediction loss.

[0013] Preferably, the sampling of the spatiotemporal pulse event stream to obtain a preliminary sampled event stream, and aggregating the preliminary sampled event stream to obtain a continuous dense aggregated event stream, comprises performing the following operations: Take the space-time pulse event stream , Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events; Spatiotemporal pulse event stream Perform sampling to obtain a preliminary sampling event flow , is the index of the initial sampling event stream, represents the total number of spatiotemporal pulse events in the preliminary sampling event stream; The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream Slice at the moment, aggregate all preliminary sampled event streams to get a shape of A continuous dense aggregate event stream, where is the number of slices of the continuous densely aggregated event stream, , Indicates the batch size, represents the total number of channel dimensions, Represents the height of a continuous dense aggregate event stream, Indicates the width of a continuous dense aggregate event stream.

[0014] Preferably, the pulse neural network is used to extract the features of the reference sample, the positive sample and the negative sample respectively as the deep spatiotemporal input synaptic current, and the deep spatiotemporal features of the pedestrian based on the membrane voltage are obtained according to the deep spatiotemporal input synaptic current, and the following operations are performed: Take the reference sample, positive sample or negative sample as the current sample and use the pulse neural network to extract the current sample The characteristics of the slice at the moment are obtained Deep spatiotemporal input synaptic currents at time ; Deep spatiotemporal input synaptic currents , calculate the deep spatiotemporal features of pedestrians based on membrane voltage: Where, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, is the time decay factor, is the membrane time constant.

[0015] Preferably, the pedestrian deep spatiotemporal features based on membrane voltage corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression features, positive sample compression features and negative sample compression features, and the following operations are performed: Pick The deep spatiotemporal characteristics of pedestrians based on membrane voltage corresponding to the slice at the moment , and get the compressed features: Where, is the compression feature, where the one obtained for the benchmark sample is the benchmark sample compression feature, the one obtained for the positive sample is the positive sample compression feature, and the one obtained for the negative sample is the negative sample compression feature. is the number of slices of the continuous densely aggregated event stream, for The adaptive compression matrix at the moment, the dynamic update method of the adaptive compression matrix is ​​as follows: initialization The weight matrix at the moment , and based on the weight matrix Get the adaptive compression matrix: Where, for The weight matrix at time t, It is an exponential function with the natural constant e as its base.

[0016] Preferably, the triplet loss is calculated for each set of reference samples, positive samples, and negative samples, and the calculation formula is as follows: Where, is the triplet loss value, To obtain the maximum value, represents the compressed features of the benchmark samples, represents the positive sample compression feature, represents the negative sample compression feature, is the distance metric function, is a constant; Compress features for each benchmark sample , use the classifier to predict the pedestrian category probability, and calculate the cross entropy loss as the category prediction loss based on the pedestrian category probability; All triplet losses and class prediction losses are combined as the total loss, and the spiking neural network is updated based on the total loss.

[0017] This invention provides a highly energy-efficient, pulsed person re-identification method and system based on neuromorphic vision. Addressing challenges faced by traditional video frame methods in poor lighting, motion blur, and privacy concerns, this invention leverages the asynchronous nature and high temporal resolution of event cameras to respond to illumination changes in specific scenes by recording an event stream of light intensity changes. A specialized spiking neural network model is constructed by applying a spiking neural network to feature extraction of the event stream input. Spatiotemporal features are extracted and fused from the neuromorphic data input. To address the issues of matching ambiguity and poor feature differentiation in pulse-based feature matching, a method based on membrane voltage feature representation and adaptive time-dimensional feature aggregation is proposed, effectively improving feature differentiation and matching accuracy. Leveraging the asynchronous event-driven mechanism of spiking neural networks and efficient temporal feature matching technology, this invention demonstrates higher recognition accuracy and superior energy efficiency than existing methods, addressing the energy consumption and redundant storage issues of traditional video-based methods. This system not only improves person re-identification accuracy but also provides a new solution for low-energy, real-time pedestrian monitoring applications, demonstrating broad practicality and potential for application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a high-energy-efficiency pulsed pedestrian re-identification method based on neuromorphic vision according to the present invention; Figure 2 Schematic diagram of the structure of a high-energy-efficiency pulse pedestrian re-identification system based on neuromorphic vision of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] It should be noted that when a component is referred to as being “connected” to another component, it may be directly connected to the other component or there may be a component in the middle; when a component is referred to as being “fixed” to another component, it may be directly fixed to the other component or there may be a component in the middle.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0022] Example 1: This embodiment proposes a highly energy-efficient pulsed person re-identification method based on neuromorphic vision. Based on membrane voltage adaptive feature matching, it effectively addresses the significant ambiguity in pulsed person feature matching in existing technologies. Furthermore, this embodiment proposes an end-to-end directly trained pulsed person re-identification model for privacy-preserving, low-energy person re-identification.

[0023] An event camera is a biomimetic visual perception sensor that employs a novel paradigm for acquiring visual information. Unlike traditional visual sensors that synchronously capture images at a fixed frame rate, an event camera asynchronously encodes visual information about the external scene, forming a continuous stream of spatiotemporal pulse events, expressed using an address event representation protocol. Each pixel in an event camera independently and asynchronously responds to changes in light intensity in the visual scene. Once the light intensity change at a pixel exceeds a preset threshold, the event camera immediately outputs a precise spatiotemporal pulse event. Event cameras offer numerous advantages, such as low power consumption, low information redundancy, and a high dynamic range, making them promising for broad application in the field of visual perception.

[0024] like Figure 1 As shown, this embodiment proposes a high-energy-efficiency pulse pedestrian re-identification method using neuromorphic vision, including the following steps: Step S101: Acquire a spatiotemporal pulse event stream asynchronously output by an event camera. The spatiotemporal pulse event stream includes multiple spatiotemporal pulse events, which are described using an address event expression protocol.

[0025] Using the formula Determine a spatiotemporal impulse event stream; where Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, +1 / -1 represents light intensity increase and decrease respectively. Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events in . Using the formula Sure ,in is a symbolic function, The pixel coordinates are and The point is The brightness of the moment, The brightness trigger threshold is determined by the inherent hardware configuration.

[0026] Step S102: Generate a preliminary sampled spatiotemporal pulse event stream (hereinafter referred to as the preliminary sampled event stream). Determine preliminary sampled spatiotemporal pulse event streams for spatiotemporal pulse events under constant sampling count based on the spatiotemporal pulse event streams. The preliminary sampled spatiotemporal pulse event streams are preliminary aggregated spatiotemporal pulse event streams determined at fixed sampling intervals.

[0027] Using the formula Determine the preliminary sampling space-time pulse event flow; is the index of the initial sampling event stream, Represents the total number of spatiotemporal pulse events in the preliminary sampling event stream, that is, the number of sampling events.

[0028] Step S103: Generate a continuous densely aggregated spatiotemporal pulse event stream (hereinafter referred to as the continuous densely aggregated event stream). This stream is constructed based on the initial sampled spatiotemporal pulse event stream. This stream is a dense input with a fixed shape and serves as the basic data for subsequent feature extraction. This addresses the issue of GPUs' poor support for sparse spike-driven computations.

[0029] The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream The slice of time is expressed as Determine a continuous dense aggregate spatiotemporal impulse event stream; where The time step of the corresponding spiking neuron is given by The initial sampling spatiotemporal pulse event streams are aggregated, so that the aggregation time dimension corresponds to the pulse neuron simulation time step dimension. The slice at time contains the initial sampled spatiotemporal impulse event stream All events in , for The minimum timestamp in the slice of moments, is the channel dimension, which continuously and densely aggregates the spatiotemporal pulse event stream All are taken from the preliminary sampling space-time pulse event stream , represents the Dirac delta function, only when The value is 1 only when

[0030] This aggregation method can be used to explicitly retain the time dimension and the shape is fixed to A continuous dense aggregated spatiotemporal pulse event stream of shape, where represents the number of slices of a continuous densely aggregated spatiotemporal spike event stream, and the number of slices corresponds to the total number of simulation moments of the spike neuron. Indicates the batch size, represents the channel dimension, Represents the height of the continuous dense aggregated spatiotemporal pulse event stream of the input, Represents the width of the continuous dense aggregated spatiotemporal pulse event stream of the input.

[0031] Step S104: Pedestrian deep spatiotemporal representation based on membrane voltage. The pedestrian deep spatiotemporal representation based on membrane voltage is extracted from a continuous, densely aggregated spatiotemporal pulse event stream in the pulse pedestrian re-identification model. Each pedestrian's continuous, densely aggregated spatiotemporal pulse event stream is mapped to its own unique features, and pedestrians are distinguished based on these features.

[0032] The continuous densely aggregated spatiotemporal pulse event stream is input as positive and negative samples. The positive and negative samples represent the spatiotemporal pulse event streams of pedestrians of the same category under different camera perspectives and the spatiotemporal pulse event streams of pedestrians of different categories (regardless of the camera perspective). The pulse neural network is used to extract the high-dimensional vector output to obtain the benchmark sample features. , positive sample features , negative sample features . That is, using the formula The final extracted deep spatiotemporal feature input synaptic current is obtained, where Represents the deep spatiotemporal input synaptic current, corresponding to the benchmark sample features , positive sample features , negative sample features , represents a spiking neural network.

[0033] Using the formula The deep spatiotemporal characteristics of pedestrians based on membrane voltage are obtained, where for The deep spatiotemporal characteristics of pedestrians represented by the membrane voltage at each moment, express The membrane voltage at the moment represents the deep spatiotemporal characteristics of the pedestrian, where is 0, The time decay factor, using the formula You can get the corresponding ,in is the membrane time constant. Representing the deep spatiotemporal features of pedestrians in the form of this real-valued continuous membrane voltage can avoid the problems of fuzzy matching and poor feature distinction during feature matching.

[0034] Step S105: Adaptive time-dimensional feature aggregation: Perform spatiotemporal fusion processing on the extracted deep spatiotemporal representation of pedestrians based on membrane voltage to generate features containing rich spatiotemporal information, ensuring the spatial structure integrity and temporal continuity of the features.

[0035] Using the formula The features of the corresponding pedestrian after compression of the time dimension are obtained. The features after compression of the time dimension fully integrate the spatiotemporal representation of the pedestrian's depth features. It is an adaptive time-dimensional compression matrix that can be dynamically updated according to different goals. The dynamic update process is as follows: first randomly initialize The weight matrix at the moment , used to capture the changes in time-dimensional features at different time steps, and use the formula The initial weight matrix is ​​mapped and adjusted, and then participates in aggregation. The entire model will adaptively determine the importance of the features of the corresponding time step, and perform feature matching and differentiation in this way. , positive sample features , negative sample features Perform adaptive time-dimensional feature aggregation to obtain the benchmark sample compression features , positive sample compression feature , negative sample compression feature .

[0036] Step S106: Pulse neural network training and prediction: The features obtained based on the adaptive time-dimensional feature aggregation are optimized using triplet loss and cross entropy loss to fully learn the features of different pedestrians, thereby improving the recognition accuracy of the system.

[0037] The triplet loss is calculated for each set of benchmark samples, positive samples and negative samples. The triplet loss formula is ,in In order to control the constant of the final optimized distance, the distance between the benchmark sample and the positive sample is shortened, and the distance between the benchmark sample and the negative sample is increased, thereby achieving the distinction between the same pedestrian features and different pedestrian features.

[0038] If there are two pedestrians X and Y in the pedestrian re-identification process, and each pedestrian has two event streams obtained from two different perspectives, then continuous dense aggregated event streams X1 and X2, Y1 and Y2 are obtained.

[0039] Taking the continuous dense aggregate event stream X1 as the benchmark sample, X2 is the positive sample, Y1 and Y2 are the negative samples, and the triplet loss is calculated as ; In the triplet loss calculation, the sign before the distance between the reference sample and the positive sample is positive, and the sign before the distance between the reference sample and the negative sample is negative; Taking the continuous dense aggregate event stream X2 as the benchmark sample, X1 is the positive sample, Y1 and Y2 are the negative samples, and the triple loss is calculated as ; Taking the continuous dense aggregate event stream Y1 as the benchmark sample, Y2 is the positive sample, X1 and X2 are the negative samples, and the triplet loss is calculated as ; Taking the continuous dense aggregate event stream Y2 as the benchmark sample, Y1 is the positive sample, X1 and X2 are the negative samples, and the triple loss is calculated as . 、 and Subscript The compressed features of the corresponding samples.

[0040] In addition, the benchmark sample compression features obtained by adaptive time-dimensional feature aggregation Using the formula Get the corresponding pedestrian prediction category, where represents the final predicted pedestrian category probability, Indicates that the pulse classifier classification function is used to classify the final predicted Match with the real label to calculate the cross entropy loss. According to the above example, it is necessary to compress the features of the benchmark samples respectively. 、 、 and Calculate the cross entropy loss.

[0041] Finally, the two-part loss is used to optimize the network to improve the correct prediction of the target pedestrian while distinguishing the pedestrian from other pedestrians.

[0042] After training is complete, the updated spiking neural network is used to obtain compressed features of the spatiotemporal pulse event stream to be identified, and the person re-identification result is obtained based on the compressed features. In this embodiment, steps S101-S105 are considered as a pulse person re-identification model. The spatiotemporal pulse event stream to be identified is first collected based on step S101, then adopted based on step S102, aggregated based on step S103, and characterized based on step S104. In the inference application, there is no need to distinguish between baseline samples, positive samples, and negative samples. Aggregation is performed based on step S105 to obtain compressed features, and then the person re-identification result is obtained based on the compressed features.

[0043] Example 2: like Figure 2 As shown, this embodiment provides a high-energy-efficiency pulse pedestrian re-identification system based on neuromorphic vision, including an event stream receiving module, an event stream sampling and aggregation module, a membrane voltage feature representation module, an adaptive time-dimensional feature aggregation module, and a feature differentiation matching and prediction module.

[0044] (1) Event stream receiving module, used to obtain the spatiotemporal pulse event stream output asynchronously by the event camera. The spatiotemporal pulse event stream includes multiple spatiotemporal pulse events, which are described using the address event expression protocol.

[0045] Using the formula Determine a spatiotemporal impulse event stream; where Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, +1 / -1 represents light intensity increase and decrease respectively. Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events in . Using the formula Sure ,in is a symbolic function, The pixel coordinates are and The point is The brightness of the moment, The brightness trigger threshold is determined by the inherent hardware configuration.

[0046] (2) An event stream sampling and aggregation module, configured to sample the spatiotemporal pulse event stream to obtain a preliminary sampling event stream, and aggregate the preliminary sampling event stream to obtain a continuous dense aggregate event stream, wherein the continuous dense aggregate event stream is composed of a plurality of slices.

[0047] According to the spatiotemporal pulse event stream, preliminary sampling spatiotemporal pulse event streams of the spatiotemporal pulse event under constant count sampling are determined respectively. The preliminary sampling spatiotemporal pulse event stream is a preliminary aggregated spatiotemporal pulse event stream determined at a fixed sampling interval.

[0048] Using the formula Determine the preliminary sampling space-time pulse event flow; is the index of the initial sampling event stream, Represents the total number of spatiotemporal pulse events in the preliminary sampling event stream, that is, the number of sampling events.

[0049] Based on the initial sampled spatiotemporal pulse event stream, a continuous densely aggregated spatiotemporal pulse event stream is constructed. This stream is a dense input with a fixed shape and serves as the basic data for subsequent feature extraction. This solves the problem of GPUs' poor support for sparse spike-driven computation.

[0050] The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream The slice of time is expressed as Determine a continuous dense aggregate spatiotemporal impulse event stream; where The time step of the corresponding spiking neuron is given by The initial sampling spatiotemporal pulse event streams are aggregated, so that the aggregation time dimension corresponds to the pulse neuron simulation time step dimension. The slice at time contains the initial sampled spatiotemporal impulse event stream All events in , for The minimum timestamp in the slice of moments, is the channel dimension, which continuously and densely aggregates the spatiotemporal pulse event stream All are taken from the preliminary sampling space-time pulse event stream , represents the Dirac delta function, only when The value is 1 only when

[0051] This aggregation method can be used to explicitly retain the time dimension and the shape is fixed to A continuous dense aggregated spatiotemporal pulse event stream of shape, where represents the number of slices of a continuous densely aggregated spatiotemporal spike event stream, and the number of slices corresponds to the total number of simulation moments of the spike neuron. Indicates the batch size, represents the channel dimension, Represents the height of the continuous dense aggregated spatiotemporal pulse event stream of the input, Represents the width of the continuous dense aggregated spatiotemporal pulse event stream of the input.

[0052] (3) A membrane voltage feature characterization module is used to use a continuous dense aggregate event stream as a reference sample, a continuous dense aggregate event stream with the same pedestrian category as the reference sample as a positive sample, and a continuous dense aggregate event stream with a different pedestrian category from the reference sample as a negative sample; a pulse neural network is used to extract the features of the reference sample, positive sample, and negative sample as deep spatiotemporal input synaptic current, and the deep spatiotemporal features of pedestrians based on membrane voltage are obtained according to the deep spatiotemporal input synaptic current.

[0053] The deep spatiotemporal representation of pedestrians based on membrane voltage is extracted as a continuous, densely aggregated spatiotemporal pulse event stream in the pulse pedestrian re-identification model. The continuous, densely aggregated spatiotemporal pulse event stream of each pedestrian will be mapped to its own unique features, and pedestrians will be distinguished based on these features.

[0054] The continuous densely aggregated spatiotemporal pulse event stream is input as positive and negative samples. The positive and negative samples represent the spatiotemporal pulse event streams of pedestrians of the same category under different camera perspectives and the spatiotemporal pulse event streams of pedestrians of different categories (regardless of the camera perspective). The pulse neural network is used to extract the high-dimensional vector output to obtain the benchmark sample features. , positive sample features , negative sample features . That is, using the formula The final extracted deep spatiotemporal feature input synaptic current is obtained, where Represents the deep spatiotemporal input synaptic current, corresponding to the benchmark sample features , positive sample features , negative sample features , represents a spiking neural network.

[0055] When mentioning features, it specifically involves pulse space compression and channel expansion units, membrane voltage-based residual connection units, and multi-layer perceptron units. The pulse space compression and channel expansion units are used to compress the height and width dimensions of the feature map while expanding the channel dimension of the feature map to compensate for the information that may be lost in the pulse space compression unit and further enhance the expressiveness of the features. The pulse space compression and channel expansion units fully retain and enhance the expressiveness of the features while reducing the spatial resolution of the feature map, providing high-quality input for subsequent units. The membrane voltage-based residual connection unit is used to retain the captured fine-grained spatial and temporal details from the continuous and densely aggregated spatiotemporal pulse event stream, achieve a certain identity mapping, alleviate the training instability and back-propagation gradient vanishing problem caused by excessive superposition between units, and ensure the normal flow of information within the entire system; the multi-layer perceptron unit is used to further assist in the deep mining of implicit features in the data to enhance the effect of feature extraction and provide richer and more discriminative feature information for subsequent processing.

[0056] Using the formula The deep spatiotemporal characteristics of pedestrians based on membrane voltage are obtained, where for The deep spatiotemporal characteristics of pedestrians represented by the membrane voltage at each moment, express The membrane voltage at the moment represents the deep spatiotemporal characteristics of the pedestrian, where is 0, The time decay factor, using the formula You can get the corresponding ,in is the membrane time constant. Representing the deep spatiotemporal features of pedestrians in the form of this real-valued continuous membrane voltage can avoid the problems of fuzzy matching and poor feature distinction during feature matching.

[0057] (4) Adaptive time-dimensional feature aggregation module, which is used to compress and aggregate the membrane voltage-based pedestrian deep spatiotemporal features corresponding to each slice according to the time dimension to obtain the baseline sample compression feature, positive sample compression feature and negative sample compression feature.

[0058] The extracted membrane voltage-based pedestrian deep spatiotemporal representation is subjected to spatiotemporal fusion processing to generate features containing rich spatiotemporal information, ensuring the spatial structure integrity and temporal continuity of the features.

[0059] Using the formula The features of the corresponding pedestrian after compression of the time dimension are obtained. The features after compression of the time dimension fully integrate the spatiotemporal representation of the pedestrian's depth features. It is an adaptive time-dimensional compression matrix that can be dynamically updated according to different goals. The dynamic update process is as follows: first randomly initialize The weight matrix at the moment , used to capture the changes in time-dimensional features at different time steps, and use the formula The initial weight matrix is ​​mapped and adjusted, and then participates in aggregation. The entire model will adaptively determine the importance of the features of the corresponding time step, and perform feature matching and differentiation in this way. , positive sample features , negative sample features Perform adaptive time-dimensional feature aggregation to obtain the benchmark sample compression features , positive sample compression feature , negative sample compression feature .

[0060] (5) Feature discrimination matching and prediction module, which is used to calculate the triplet loss based on the distance between the compressed features of the benchmark sample and the compressed features of the positive sample, and the distance between the compressed features of the benchmark sample and the compressed features of the negative sample, calculate the category prediction loss based on the pedestrian re-identification results corresponding to the compressed features of the benchmark sample, and update the pulse neural network by combining the triplet loss and category prediction loss.

[0061] The features obtained based on adaptive time-dimensional feature aggregation are optimized using triplet loss and cross entropy loss to fully learn the characteristics of different pedestrians, thereby improving the recognition accuracy of the system.

[0062] The triplet loss is calculated for each set of benchmark samples, positive samples and negative samples. The triplet loss formula is ,in In order to control the constant of the final optimized distance, the distance between the benchmark sample and the positive sample is shortened, and the distance between the benchmark sample and the negative sample is increased, thereby achieving the distinction between the same pedestrian features and different pedestrian features.

[0063] If there are two pedestrians X and Y in the pedestrian re-identification process, and each pedestrian has two event streams obtained from two different perspectives, then continuous dense aggregated event streams X1 and X2, Y1 and Y2 are obtained.

[0064] Taking the continuous dense aggregate event stream X1 as the benchmark sample, X2 is the positive sample, Y1 and Y2 are the negative samples, and the triplet loss is calculated as ; In the triplet loss calculation, the sign before the distance between the reference sample and the positive sample is positive, and the sign before the distance between the reference sample and the negative sample is negative; Taking the continuous dense aggregate event stream X2 as the benchmark sample, X1 is the positive sample, Y1 and Y2 are the negative samples, and the triple loss is calculated as ; Taking the continuous dense aggregate event stream Y1 as the benchmark sample, Y2 is the positive sample, X1 and X2 are the negative samples, and the triplet loss is calculated as ; Taking the continuous dense aggregate event stream Y2 as the benchmark sample, Y1 is the positive sample, X1 and X2 are the negative samples, and the triple loss is calculated as . 、 and Subscript The compressed features of the corresponding samples.

[0065] In addition, the benchmark sample compression features obtained by adaptive time-dimensional feature aggregation Using the formula Get the corresponding pedestrian prediction category, where represents the final predicted pedestrian category probability, Indicates that the pulse classifier classification function is used to classify the final predicted Match with the real label to calculate the cross entropy loss. According to the above example, it is necessary to compress the features of the benchmark samples respectively. 、 、 and Calculate the cross entropy loss.

[0066] Finally, the two-part loss is used to optimize the network to improve the correct prediction of the target pedestrian while distinguishing the pedestrian from other pedestrians.

[0067] After training is complete, the updated spiking neural network is used to obtain compressed features of the spatiotemporal pulse event stream to be identified, and the person re-identification result is obtained based on the compressed features. In this embodiment, steps S101-S105 are considered as a pulse person re-identification model. The spatiotemporal pulse event stream to be identified is first collected based on step S101, then adopted based on step S102, aggregated based on step S103, and characterized based on step S104. In the inference application, there is no need to distinguish between baseline samples, positive samples, and negative samples. Aggregation is performed based on step S105 to obtain compressed features, and then the person re-identification result is obtained based on the compressed features.

[0068] The present invention proposes a high-energy-efficiency pulsed pedestrian re-identification method and system based on neuromorphic vision, which has the following advantages: 1. This paper proposes an end-to-end person re-identification method and system based on direct training of a spiking neural network based on event data. This method fully utilizes the spatiotemporal processing capabilities of spiking neural networks and exploits the precise timing information of spatiotemporal spiking events in an event-driven manner.

[0069] A novel solution addresses the challenges of traditional video frame methods, such as poor lighting, motion blur, and privacy concerns. This method leverages the asynchronous nature and high temporal resolution of event cameras to respond to lighting changes in a specific scene by recording an event stream of varying light intensity. By applying spiking neural networks to feature extraction from event stream input, this method achieves the first pulsed person re-identification based on event stream data.

[0070] Aiming at the problems of matching ambiguity and poor feature discrimination in pulse-based feature matching, a spatiotemporal feature representation and adaptive time-dimensional feature aggregation method based on membrane potential is proposed, which effectively improves the feature discrimination and matching accuracy.

[0071] In addition, the present invention, based on the event-driven mechanism and discrete pulse transfer characteristics of spiking neural networks, significantly reduces energy consumption while achieving performance comparable to or even better than existing technologies. Existing person re-identification methods are mainly based on artificial neural networks, which have low computational efficiency and high energy consumption, severely restricting the deployment and promotion of such algorithms in practical scenarios. The present invention significantly alleviates this problem and has important application value. Currently, event-based person re-identification is in its infancy, and there are not many reference methods. The present invention provides a new solution for event-based person re-identification.

[0072] 2. The mAP (Mean Average Precision) and Rank-1 accuracy of the present invention in the real-world person re-identification task of 33 identities from four different camera angles are significantly higher than those of existing methods: By adopting an end-to-end training framework based on a spiking neural network, this method can efficiently extract and fuse pedestrian information features with different visual and temporal dynamics in asynchronous event streams. Compared with traditional methods, the adaptability of this method in complex environments has been greatly improved, thereby achieving higher mAP and Rank-1 in dynamic and changing real-world scenes. Table 1 shows the mAP and Rank-1 comparison results of the present method and existing technologies on the Ev-ReId dataset.

[0073] Table 1 Comparison of Rank-1 and mAP on the event-based person re-identification dataset Ev-ReId dataset Comparative Example 1: S. Ahmad, G. Scarpellini, P. Morerio, and A. Del Bue, “Event-driven re-id: A new benchmark and method towards privacy-preserving personre-identification,” in Proceedings of the IEEE / CVF winter conference on applications of computer vision, 2022, pp. 459–468. Comparative Example 2: S. Ahmad, P. Morerio, and A. Del Bue, “Person re-identification without identification via event anonymization,” inProceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp. 11 132–11 141. Comparative Example 3: S. Ahmad, P. Morerio, and A. Del Bue, “Person re-identification without identification via event anonymization,” in Proceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp. 11 132–11 141. Using the ResNet34 architecture 3. In the event frame person re-identification task of 385 and 749 identities from two non-overlapping cameras, with event frame lengths ranging from 5 to 675 frames, the proposed method achieves significantly higher mAP and Rank-1 than existing methods. By leveraging the spatiotemporal characteristics of spiking neural networks, the proposed method effectively captures key information about pedestrians in dynamic and complex environments, improving the accuracy of person re-identification. Table 2 shows the Rank-1 and mAP comparison results of the proposed method and existing methods on the Event-PRID-2011 dataset.

[0074] Table 2 Comparison of Rank-1 and mAP on the Event-PRID-2011 dataset for event-based person re-identification Comparative Example 1: C. Eom, G. Lee, J. Lee, and B. Ham, “Video-based personreidentification with spatial and temporal memory networks,” in Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 12036–12 045. Comparative Example 2: H. Wang, L. Jiao, S. Yang, L. Li, and Z. Wang, “Simple and effective: Spatial rescaling for person reidentification,” IEEE Transactionson neural networks and learning systems, vol. 33, no. 1, pp. 145–156, 2020. Comparative Example 3: S. Ahmad, P. Morerio, and A. Del Bue, “Person re-identification without identification via event anonymization,” inProceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp. 11 132–11 141. 4. This method demonstrates exceptional low energy consumption in real-world person re-identification tasks: Thanks to the event-driven and asynchronous computing architecture of spiking neural networks, this model achieves extremely low energy consumption while maintaining high accuracy and real-time performance. Compared to traditional artificial neural network-based solutions, its energy consumption is significantly reduced, making it more environmentally adaptable and practically deployable. Table 3 shows the energy consumption of this method compared to existing technologies on an event-based person re-identification dataset.

[0075] Table 3 Comparison of energy consumption on event-based person re-identification dataset Comparative Example 1: H. Wang, L. Jiao, S. Yang, L. Li, and Z. Wang, “Simple and effective: Spatial rescaling for person reidentification,” IEEE Transactionson neural networks and learning systems, vol. 33, no. 1, pp. 145–156, 2020. Comparative Example 2: S. Ahmad, P. Morerio, and A. Del Bue, “Person re-identification without identification via event anonymization,” inProceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp. 11 132–11 141. Comparative Example 3: C. Eom, G. Lee, J. Lee, and B. Ham, “Video-based personreidentification with spatial and temporal memory networks,” in Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 12036–12 045. Comparative Example 4: S. Ahmad, P. Morerio, and A. Del Bue, “Person re-identification without identification via event anonymization,” in Proceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp. 11 132–11 141. Uses the ResNet34 architecture.

[0076] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A high-energy-efficiency impulse person re-identification method based on neuromorphic vision, characterized by: The energy-efficient pulsed person re-identification method based on neuromorphic vision comprises: Get the spatiotemporal pulse event stream asynchronously output by the event camera; Sampling the spatiotemporal pulse event stream to obtain a preliminary sampling event stream, aggregating the preliminary sampling event stream to obtain a continuous dense aggregated event stream, wherein the continuous dense aggregated event stream is composed of multiple slices; A continuous dense aggregated event stream is used as a benchmark sample, a continuous dense aggregated event stream with the same pedestrian category as the benchmark sample is used as a positive sample, and a continuous dense aggregated event stream with a different pedestrian category from the benchmark sample is used as a negative sample; A spiking neural network is used to extract the features of reference samples, positive samples, and negative samples as deep spatiotemporal input synaptic currents, and the deep spatiotemporal features of pedestrians based on membrane voltage are obtained according to the deep spatiotemporal input synaptic currents. The pedestrian deep spatiotemporal features based on membrane voltage corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression features, positive sample compression features and negative sample compression features; The triplet loss is calculated based on the distance between the compressed features of the benchmark sample and the compressed features of the positive sample, as well as the distance between the compressed features of the benchmark sample and the compressed features of the negative sample. The category prediction loss is calculated based on the pedestrian re-identification results corresponding to the compressed features of the benchmark sample. The triplet loss and category prediction loss are combined to update the spiking neural network. The updated spiking neural network is used to obtain the compressed features of the spatiotemporal spiking event stream to be identified, and the pedestrian re-identification results are obtained based on the compressed features.

2. The energy-efficient pulsed person re-identification method based on neuromorphic vision according to claim 1, characterized in that: The step of sampling the spatiotemporal pulse event stream to obtain a preliminary sampling event stream, and aggregating the preliminary sampling event stream to obtain a continuous dense aggregated event stream includes: Take the space-time pulse event stream , Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events; Spatiotemporal pulse event stream Perform sampling to obtain a preliminary sampling event flow , is the index of the initial sampling event stream, represents the total number of spatiotemporal pulse events in the preliminary sampling event stream; The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream Slice at the moment, aggregate all preliminary sampled event streams to get a shape of A continuous dense aggregate event stream, where is the number of slices of the continuous densely aggregated event stream, , Indicates the batch size, represents the total number of channel dimensions, Represents the height of a continuous dense aggregate event stream, Indicates the width of a continuous dense aggregate event stream.

3. The energy-efficient pulsed person re-identification method based on neuromorphic vision according to claim 1, characterized in that: The method uses a spiking neural network to extract the features of the reference sample, the positive sample, and the negative sample as the deep spatiotemporal input synaptic current, and obtains the deep spatiotemporal features of the pedestrian based on the membrane voltage according to the deep spatiotemporal input synaptic current, including: Take the reference sample, positive sample or negative sample as the current sample and use the pulse neural network to extract the current sample The characteristics of the slice at the moment are obtained Deep spatiotemporal input synaptic currents at time ; Deep spatiotemporal input synaptic currents , calculate the deep spatiotemporal features of pedestrians based on membrane voltage: Where, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, is the time decay factor, is the membrane time constant.

4. The energy-efficient pulsed person re-identification method based on neuromorphic vision according to claim 1, characterized in that: The pedestrian deep spatiotemporal features based on membrane voltage corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression features, positive sample compression features and negative sample compression features, including: Pick The deep spatiotemporal characteristics of pedestrians based on membrane voltage corresponding to the slice at the moment , and get the compressed features: Where, is the compression feature, where the one obtained for the benchmark sample is the benchmark sample compression feature, the one obtained for the positive sample is the positive sample compression feature, and the one obtained for the negative sample is the negative sample compression feature. is the number of slices of the continuous densely aggregated event stream, for The adaptive compression matrix at the moment, the dynamic update method of the adaptive compression matrix is ​​as follows: initialization The weight matrix at the moment , and based on the weight matrix Get the adaptive compression matrix: Where, for The weight matrix at time t, It is an exponential function with the natural constant e as its base.

5. The energy-efficient pulsed person re-identification method based on neuromorphic vision according to claim 1, characterized in that: The triplet loss is calculated for each set of benchmark samples, positive samples, and negative samples, and the calculation formula is as follows: Where, is the triplet loss value, To obtain the maximum value, represents the compressed features of the benchmark samples, represents the positive sample compression feature, represents the negative sample compression feature, is the distance metric function, is a constant; Compress features for each benchmark sample , use the classifier to predict the pedestrian category probability, and calculate the cross entropy loss as the category prediction loss based on the pedestrian category probability; All triplet losses and class prediction losses are combined as the total loss, and the spiking neural network is updated based on the total loss.

6. A high-energy-efficiency impulse pedestrian re-identification system based on neuromorphic vision, characterized by: The energy-efficient pulsed person re-identification system based on neuromorphic vision comprises: The event stream receiving module is used to obtain the spatiotemporal pulse event stream asynchronously output by the event camera; An event stream sampling and aggregation module, configured to sample the spatiotemporal pulse event stream to obtain a preliminary sampled event stream, and aggregate the preliminary sampled event stream to obtain a continuous dense aggregated event stream, wherein the continuous dense aggregated event stream is composed of a plurality of slices; Based on the membrane voltage feature characterization module, it is used to use a continuous dense aggregate event stream as a reference sample, a continuous dense aggregate event stream with the same pedestrian category as the reference sample as a positive sample, and a continuous dense aggregate event stream with a different pedestrian category from the reference sample as a negative sample. A spiking neural network is used to extract the features of the reference sample, positive sample, and negative sample as deep spatiotemporal input synaptic currents, and the deep spatiotemporal features of pedestrians based on membrane voltage are obtained based on the deep spatiotemporal input synaptic currents. Adaptive time-dimensional feature aggregation module, used to compress and aggregate the membrane voltage-based pedestrian deep spatiotemporal features corresponding to each slice according to the time dimension to obtain the baseline sample compression feature, positive sample compression feature and negative sample compression feature; The feature discrimination matching and prediction module is used to calculate the triplet loss based on the distance between the compressed features of the benchmark sample and the compressed features of the positive sample, as well as the distance between the compressed features of the benchmark sample and the compressed features of the negative sample, calculate the category prediction loss based on the pedestrian re-identification results corresponding to the compressed features of the benchmark sample, and update the pulse neural network by combining the triplet loss and category prediction loss.

7. The energy-efficient pulsed person re-identification system based on neuromorphic vision according to claim 6, characterized in that: The sampling of the spatiotemporal pulse event stream to obtain a preliminary sampling event stream, and aggregating the preliminary sampling event stream to obtain a continuous dense aggregated event stream, perform the following operations: Take the space-time pulse event stream , Representing a stream of spatiotemporal pulse events The A space-time pulse event, and Indicates the The pixel coordinates of the spatiotemporal pulse events, Indicates the The timestamp of the space-time pulse event, Indicates the The polarity of light intensity change of each pixel corresponding to a spatiotemporal pulse event, Representing a stream of spatiotemporal pulse events The total number of spatiotemporal pulse events; Spatiotemporal pulse event stream Perform sampling to obtain a preliminary sampling event flow , is the index of the initial sampling event stream, represents the total number of spatiotemporal pulse events in the preliminary sampling event stream; The first Preliminary sampling event stream Aggregation as a continuous dense aggregation event stream Slice at the moment, aggregate all preliminary sampled event streams to get a shape of A continuous dense aggregate event stream, where is the number of slices of the continuous densely aggregated event stream, , Indicates the batch size, represents the total number of channel dimensions, Represents the height of a continuous dense aggregate event stream, Indicates the width of a continuous dense aggregate event stream.

8. The energy-efficient pulsed person re-identification system based on neuromorphic vision according to claim 6, characterized in that: The pulse neural network is used to extract the features of the reference sample, positive sample and negative sample respectively as the deep spatiotemporal input synaptic current, and the deep spatiotemporal features of the pedestrian based on the membrane voltage are obtained according to the deep spatiotemporal input synaptic current, and the following operations are performed: Take the reference sample, positive sample or negative sample as the current sample and use the pulse neural network to extract the current sample The characteristics of the slice at the moment are obtained Deep spatiotemporal input synaptic currents at time ; Deep spatiotemporal input synaptic currents , calculate the deep spatiotemporal features of pedestrians based on membrane voltage: Where, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, for The deep spatiotemporal characteristics of pedestrians based on membrane voltage at each moment, is the time decay factor, is the membrane time constant.

9. The energy-efficient pulsed person re-identification system based on neuromorphic vision according to claim 6, characterized in that: The pedestrian deep spatiotemporal features based on membrane voltage corresponding to each slice are compressed and aggregated according to the time dimension to obtain the baseline sample compression feature, the positive sample compression feature and the negative sample compression feature, and the following operations are performed: Pick The deep spatiotemporal characteristics of pedestrians based on membrane voltage corresponding to the slice at the moment , and get the compressed features: Where, is the compression feature, where the one obtained for the benchmark sample is the benchmark sample compression feature, the one obtained for the positive sample is the positive sample compression feature, and the one obtained for the negative sample is the negative sample compression feature. is the number of slices of the continuous densely aggregated event stream, for The adaptive compression matrix at the moment, the dynamic update method of the adaptive compression matrix is ​​as follows: initialization The weight matrix at the moment , and based on the weight matrix Get the adaptive compression matrix: Where, for The weight matrix at time t, It is an exponential function with the natural constant e as its base.

10. The energy-efficient pulsed person re-identification system based on neuromorphic vision according to claim 6, characterized in that: The triplet loss is calculated for each set of benchmark samples, positive samples, and negative samples, and the calculation formula is as follows: Where, is the triplet loss value, To obtain the maximum value, represents the compressed features of the benchmark samples, represents the positive sample compression feature, represents the negative sample compression feature, is the distance metric function, is a constant; Compress features for each benchmark sample , use the classifier to predict the pedestrian category probability, and calculate the cross entropy loss as the category prediction loss based on the pedestrian category probability; All triplet losses and class prediction losses are combined as the total loss, and the spiking neural network is updated based on the total loss.

Images