Accelerator based on pulse neural network, millimeter wave radar real-time target detection system and method

Abstract

The application provides an accelerator based on a pulse neural network, a millimeter wave radar real-time target detection system and method, the accelerator comprising: a multi-dimensional pulse time coding unit, a pulse neural network detection unit, an STDP online learning engine unit, a pulse routing and arbitration network unit and a target parameter decoder unit; the pulse neural network detection unit comprises a first distance feature extraction layer, a second Doppler-angle joint layer and a third detection decision layer; a mixed frequency intermediate frequency signal is subjected to multi-channel synchronous sampling, and is coded into a composite pulse event without performing a fast Fourier transform operation; the composite pulse event is input into a pulse neural network, and target parameters including distance, angle, speed and confidence are output; the application considers detection real-time performance, low power consumption and hardware integrability, and is suitable for being applied to a delay and power consumption sensitive embedded millimeter wave radar scene.

Description

Technical Field

[0001] This invention relates to the field of millimeter-wave radar technology, specifically to an accelerator based on a pulse neural network, a millimeter-wave radar real-time target detection system and method. Background Technology

[0002] Millimeter-wave radar, due to its all-weather operation, insensitivity to lighting and weather conditions, and low hardware cost, has become one of the core sensors in autonomous driving and intelligent perception systems. Current mainstream millimeter-wave radar target detection schemes generally employ the following signal processing pipeline: First, the echo intermediate frequency signal of each linear frequency modulated pulse (chirp) is sampled by an ADC. Then, a first-dimensional FFT (1D-FFT or Range FFT) is performed in the range dimension. Next, a second-dimensional FFT (2D-FFT or Doppler FFT) is performed between multiple chirs. For multiple-input multiple-output (MIMO) radars, a third-dimensional angle FFT (3D-FFT or Angle FFT) may also be performed. After completing the FFT, CFAR detection is performed on the range-Doppler spectrum or range-Doppler-angle cube to extract target points. Then, the target angle is obtained through DOA estimation (such as MUSIC, ESPRIT, or DBF). Finally, multiple reflection points belonging to the same target are aggregated using clustering algorithms (such as DBSCAN).

[0003] However, this traditional detection scheme has several major limitations. First, it has extremely high computational density. Three-dimensional FFT is typically computationally intensive, requiring billions of multiply-accumulate operations per second to process high-resolution radar data cubes (e.g., in scenarios with 512 range points, 256 Doppler points, and 12 angular channels). Real-time processing on embedded platforms typically requires high-performance DSPs or GPUs, leading to significant increases in power consumption and cost. Second, the pipeline is sequential, resulting in data bloat. FFT generates dense spectral data (all range-Doppler bins have values), but only a very small number of points actually contain the target. Storing and transferring the complete spectral data between FFT and CFAR leads to unnecessary storage bandwidth and access power consumption, particularly prominent in real-time embedded scenarios. Third, CFAR detection relies on statistical assumptions. Traditional CFAR (CA-CFAR, OS-CFAR, GO-CFAR, etc.) is based on certain statistical distribution assumptions about background noise and clutter (such as Gaussian distribution, Rayleigh distribution). When switching between different scenarios (such as highways, city streets, indoor parking lots), the fixed CFAR window and threshold strategy can easily lead to a decrease in detection rate or an increase in false alarm rate. Fourth, each module is optimized independently, lacking end-to-end collaboration. In traditional solutions, FFT, CFAR, DOA, and clustering are designed independently. Although each link can be optimized locally, the global optimum from the raw echo to the target output cannot be guaranteed, and the intermediate representations of each module are inconsistent, increasing the complexity of system integration.

[0004] In recent years, Spiking Neural Networks (SNNs), as third-generation neural networks, have attracted widespread attention for their event-driven, sparse computation, and natural ability to represent temporal information. SNNs express and transmit information in a sparse pulse time-series manner; neurons update their membrane potential and fire pulses only upon receiving input pulses, naturally exhibiting low power consumption. Synaptic Time-Dependent Plasticity (STDP), a biologically inspired rule, adjusts synaptic weights solely based on the time difference between presynaptic and postsynaptic pulses, eliminating the need for global error backpropagation and requiring only a small number of counters and comparators in hardware implementation. However, current applications of SNNs are mainly concentrated in traditional artificial intelligence tasks such as image classification and speech recognition. A complete technical path has not yet been found that can directly apply SNNs to millimeter-wave radar signal processing, particularly replacing FFT spectral analysis with pulse time coding and then replacing CFAR detection chains, DOA estimation, and target clustering with SNN inference. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides an accelerator, millimeter-wave radar real-time target detection system and method based on a pulse neural network, to reduce the computational load and power consumption of radar signal processing.

[0006] According to a first aspect of the present invention, an accelerator based on a spiking neural network is provided, deployed in a programmable logic device, comprising: A multi-dimensional pulse time coding unit is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and encode the intermediate frequency signal into a composite pulse event carrying distance information, Doppler information and angle information without performing a fast Fourier transform operation. The spiking neural network detection unit includes a first range feature extraction layer, a second Doppler-angle joint layer, and a third detection and decision layer, and the interlayer connections have a sparse constraint topology based on radar physical semantics; the first range feature extraction layer is used to receive the composite pulse event, output the pulse to pass to the next layer or make a target decision; The STDP online learning engine unit is used to adaptively update the synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer online based on the time difference between local presynaptic pulses and postsynaptic pulses during the operation of the accelerator. A pulse routing and arbitration network unit is used to transmit pulse event packets between the layers of the multi-dimensional pulse timing coding unit and the pulse neural network detection unit, and between the layers of the pulse neural network detection unit. The target parameter decoder unit is used to decode and output the target distance, angle, velocity parameters and confidence level based on the pulse firing rate and firing pattern of the output neurons in the third detection and decision layer.

[0007] According to a second aspect of the present invention, a real-time target detection system for millimeter-wave radar based on a pulse neural network is provided, comprising: The millimeter-wave radar RF front-end is used to transmit linear frequency modulated pulse chirp sequences, which are then mixed to obtain an intermediate frequency signal for a multi-channel ADC module. A multi-channel ADC module is used to synchronously sample the intermediate frequency signal of each receiving channel; An accelerator based on a pulse neural network is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and output target parameters including distance, velocity, angle and confidence level. The result output module is used to encapsulate the target parameters in frames and send them to the upper-layer system via CAN-FD or Ethernet interface.

[0008] According to a third aspect of the present invention, a real-time target detection method for millimeter-wave radar based on a pulse neural network is provided, the method comprising: The intermediate frequency signal obtained by mixing is sampled at a certain rate. f s Multi-channel synchronous sampling is performed, and the intermediate frequency signal is encoded into a composite pulse event carrying distance information, Doppler information, and angle information without performing fast Fourier transform operations; The composite pulse event is input into the first distance feature extraction layer of the spiking neural network, and the first pulse is output and passed to the second Doppler-angle joint layer. The second Doppler-angle joint layer outputs a second pulse and passes it to the third detection and decision layer. The third detection and decision layer outputs target parameters including distance, angle, velocity and confidence. The target parameters are encapsulated in frames and output through the communication interface; Specifically, based on the time difference between local presynaptic pulses and postsynaptic pulses, the synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer are adaptively updated online.

[0009] Compared with existing technologies, this invention has the following significant advantages: 1. It directly processes millimeter-wave radar ADC sampling data through a multi-dimensional pulse time coding unit, fundamentally reducing computational density and intermediate data storage requirements; 2. Through a multi-layered pulse neural network detection unit, end-to-end target detection in the time domain can be achieved; 3. By introducing an STDP online learning engine unit, the system can continuously adjust the weights of the pulse neural network during radar operation to adapt to noise and clutter environments in different scenarios, without the need for offline annotation of training data; 4. It adopts an event-based time step skipping mechanism and an efficient pulse routing arbitration strategy to skip idle time steps during pulse-sparse periods, reducing the dynamic power consumption of programmable logic devices. Attached Figure Description

[0010] Figure 1 A schematic diagram of an accelerator architecture based on a spiking neural network provided by the present invention;

[0011] Figure 2 A schematic diagram of a millimeter-wave radar real-time target detection system architecture based on a pulse neural network is provided for this invention.

[0012] Figure 3 This is a schematic diagram illustrating the principle of the multi-dimensional pulse time encoding unit in an embodiment of the present invention;

[0013] Figure 4 This is a schematic diagram of the three-layer architecture of the spiking neural network detection unit in an embodiment of the present invention;

[0014] Figure 5 This is a schematic diagram of the STDP online learning engine unit according to an embodiment of the present invention;

[0015] Figure 6This is a schematic diagram of the pulse routing and arbitration network unit according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0017] Example 1

[0018] refer to Figure 1 This embodiment provides an accelerator based on a spiking neural network, which can be deployed in a field-programmable gate array (FPGA). The device can simultaneously integrate programmable logic resources, a digital signal processing (DSP) acceleration core, and an embedded processor core. The spiking neural network-based accelerator can integrate a multi-dimensional pulse timing encoding unit, a spiking neural network detection unit, an STDP online learning engine unit, a pulse routing and arbitration network unit, and a target parameter decoder unit. These units can be interconnected via a high-bandwidth on-chip bus or a dedicated asynchronous pulse interface.

[0019] A multi-dimensional pulse time coding unit is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and encode the intermediate frequency signal into a composite pulse event carrying distance information, Doppler information, and angle information without performing a fast Fourier transform operation. The multi-dimensional pulse time coding unit may include a distance coding subunit, a Doppler coding subunit, an angle coding subunit, and a three-dimensional fusion subunit.

[0020] See Figure 3 The distance coding subunit is used to perform point-by-point amplitude calculation on the ADC sampling sequence within a single linear frequency modulated pulse chirp, and compare the calculated amplitude with a first preset threshold. After performing local nonmaximum suppression through a sliding window, a distance pulse event carrying a distance index, timestamp, and pulse intensity is generated. The first preset threshold is dynamically determined by the statistics of the amplitudes of all sampling points within the current linear frequency modulated pulse chirp.

[0021] Specifically, for a single linear frequency modulated pulse chirp N s ADC sampling sequence The distance pulse encoder generates a corresponding distance pulse sequence. Its core principle is that the farther the target is, the later the echo arrives, and the corresponding intermediate frequency signal has a higher amplitude within a specific sampling interval of the linear frequency modulated pulse chirp. The encoder processes the sampling sequence as follows:

[0022] Perform point-by-point amplitude calculation on the sampled sequence to obtain the amplitude value corresponding to each sample point. The amplitude can be calculated using modulo or squaring operations. The calculated amplitude value is then compared with a first preset threshold. Comparison. First preset threshold. It can be dynamically determined by the statistics of the amplitudes of all sampling points within the current linear frequency modulated pulse chirp. When the amplitude value of a certain sampling point... At that time, a candidate impulse event is initially generated. ,in i Indicates the sampling index (corresponding to the distance bin). t i Indicates the sampling time or the normalized timestamp. a i This represents the normalized pulse intensity level (which can be quantized to 2-4 bits).

[0023] To prevent a single strong target from generating too many pulses in adjacent bins, leading to saturation of the downstream spiking neural network (SNN), a local nonmaximum suppression mechanism can be introduced within the encoder. (In the sliding window...) W r Within the encoder, only the pulse event corresponding to the sampling point with the largest amplitude is retained, while the rest are suppressed. Finally, the encoder packages the distance pulse events into a pulse data packet of a uniform format, i.e., {type: distance pulse, distance index:} i Timestamp: t i ,strength: a i , Channel number: c}.

[0024] In some embodiments, to increase pulse sparsity, an cumulative firing mode can be adopted: a distance firing counter is maintained for each sampling point in the sampling sequence, and a pulse is fired and the counter is cleared when the cumulative distance exceeds a threshold, similar to the Integrate-and-Fire neuron model.

[0025] The Doppler coding subunit is used to calculate the differential phase between adjacent linear frequency modulated (LFM) pulse chirps across multiple LFM chirps in an ADC sampling sequence at the same distance bin, and to linearly map or symbolically approximate the differential phase as a pulse firing rate or pulse firing level to generate Doppler pulse events. Doppler information is typically contained in the phase changes of multiple consecutive LFM chirps at the same distance bin. Traditional schemes extract this information using Doppler FFT, while this application characterizes the Doppler information through pulse firing rate coding.

[0026] Specifically, radar transmission is set upN c A frame is formed by a linear frequency modulated pulse (chirp) for a given distance bin index. r Collect cross N c Sample values ​​of a linear frequency modulated pulse chirp This constitutes a slow time series of the distance bin. The differential phase between adjacent linear frequency modulated pulses (chirp) is then calculated from this slow time series. ,in This indicates the operation of calculating the argument. This represents the complex conjugate. The range of values ​​for the differential phase can be... The differential phase linear mapping or symbolic approximation is the pulse firing rate. f dop Or pulse delivery level. For example, using a linear mapping relationship. ,in f base Based on the distribution rate, k f The proportionality coefficient maps the magnitude of the differential phase to the pulse firing rate; at the same time, different pulse channels or pulse polarities can be used to distinguish between positive differential phase (representing the target is close to the radar) and negative differential phase (representing the target is far from the radar).

[0027] In another optional embodiment, to reduce hardware implementation complexity, the Doppler coding subunit employs a symbolic approximation mapping method, determining only the signs of the real and imaginary parts of the differential phase. This rapidly divides the differential phase into several quadrant regions, each corresponding to a different pulse firing rate level. Finally, a Doppler pulse event carrying a distance index, chirp sequence number, Doppler channel (representing velocity direction and velocity level), and timestamp is generated and sent to the 3D fusion subunit.

[0028] The angle coding subunit is used to arrange multi-channel sampled values ​​on the same range bin and the same linear frequency modulated pulse chirp into a vector, calculate the inter-channel phase difference between adjacent channels, and map the inter-channel phase difference to a pulse time difference to generate angle pulse events. Angle information is typically contained in the phase difference between multiple receive channels on the same range bin and the same linear frequency modulated pulse chirp. For MIMO radar configurations with multiple receive channels, the angle coding subunit utilizes the inter-channel phase difference to encode the target's direction.

[0029] Specifically, for a given distance bin and chirp index, N rx The sampled values ​​of each channel are arranged into a vector. Calculate the phase difference between adjacent channel pairs. The phase difference between channels is mapped to a pulse time difference. Specifically, the pulse emission time of a reference channel (such as channel 1) is set as a baseline. t ref , No. k The pulse firing time offset corresponding to the channel is ,in k angle This represents the time-angle mapping coefficient. When a target is present in a direction, the channels exhibit a regular phase progression relationship, and the corresponding pulse sequence has a regular time offset pattern. The direction-sensitive neurons in the subsequent layers of the spiking neural network detection unit infer the target angle by detecting this regular time offset pattern. Finally, an angle pulse event carrying the receiving channel index, distance index, time offset or time slot index, and timestamp is generated and sent to the 3D fusion submodule.

[0030] In another alternative embodiment, phase-pulse position encoding can be used to... The phase range is uniformly quantized as There are several intervals, each corresponding to a pulse transmission time slot. After phase-pulse position mapping, the sampling of different receiving channels at the same distance-doppler, the index of the time slot where the pulse is located reflects the phase information of that channel.

[0031] The 3D fusion subunit receives various pulse events from the range coding subunit, Doppler coding subunit, and angle coding subunit, and fuses them into a composite pulse event with a unified format. All events in the buffer can be sorted in ascending order according to their timestamps, and duplicate or redundant events occurring at the same time step and physical location can be eliminated. The fused composite pulse event packet can include timestamps. T global Distance Index r Doppler levels d Angular feature code ,strength a And source channel c, where A compact representation of the angular pulse encoding (such as quantized phase difference or time slot index). The sorted and deduplicated sequence of composite pulse events can be used as the first-layer input to the detection unit of a spiking neural network.

[0032] This invention utilizes a radar-specific multi-dimensional pulse time-coding unit to directly connect to ADC sampling data, while general-purpose SNN accelerators require externally provided pre-coded pulse input. The connection topology of this application is constrained by radar physical semantics, making it more efficient than the fully connected or general-purpose mesh topology of general-purpose accelerators. This invention also integrates a target parameter decoder unit, directly outputting radar detection results instead of classification labels from general-purpose SNNs. It solves the core problem of converting from synchronous dense ADC sampling to sparse pulse sequences and specifically designs a multi-dimensional pulse time-coding unit and a multi-layer spiking neural network detection unit for the three-dimensional physical dimensions of radar, filling a gap in the application of SNNs in radar signal processing.

[0033] The spiking neural network detection unit includes a first range feature extraction layer, a second Doppler-angle joint layer, and a third detection and decision layer, with inter-layer connections having a sparse constraint topology based on radar physical semantics; the first range feature extraction layer is used to receive the composite pulse event, output pulses to pass to the next layer, or make target decisions.

[0034] Specifically, the basic computational unit in the spiking neural network detection unit can be a leakage integral firing LIF neuron. LIF neurons can effectively simulate the membrane potential accumulation, leakage decay, and threshold firing behavior of biological neurons. In discrete time steps... k Above, the first i The membrane potential update equation for a LIF neuron can be expressed as: ;

[0035] in, Indicates the first i The neuron in the first k Membrane potential at each time step V rest This represents the resting potential and can be set to 0 or a configurable value. The leakage factor can take values ​​between 0 and 1, and can be implemented as a right shift operation; w ji Indicates from the first j The presynaptic neuron to the first i The synaptic weight of a postsynaptic neuron can be positive (representing an excitatory synapse) or negative (representing an inhibitory synapse). Indicates the first j The presynaptic neuron in the first k The pulse output of the time step can take the value of 0 (representing no pulse) or 1 (representing pulse). V exc Indicates the excitatory reversal potential; V inh This indicates an inhibitory reversal potential.

[0036] When the accumulated membrane potential Greater than or equal to a preset distribution threshold V th At that time, the neuron can fire an output pulse, that is, make Subsequently, its membrane potential was reset to V reset (can be equal to) V rest Furthermore, this neuron can enter a pre-defined refractory period. T ref During the refractory period T ref Within a time step (e.g., approximately 1 to 5 time steps), the neuron's membrane potential can remain at the resting potential, without accumulating any synaptic input.

[0037] In some embodiments, to achieve extremely high computational efficiency on the hardware platform, the membrane potential update equation of the LIF neuron can be computed using fixed-point arithmetic, and the leakage decay term is implemented as follows: Subtract its own rightward shift L leak The value after the bit, the synaptic input item is implemented as w ji The accumulation (when) (when) or skip (when) (Time). Because spiking neural networks are characterized by pulse sparsity, only a few presynaptic neurons fire pulses in most time steps. Therefore, the actual number of addition operations required is far less than the dense matrix multiplication and accumulation in fully connected neural networks.

[0038] The first distance feature extraction layer has several distance-sensitive neuron groups, each containing multiple neurons with different Doppler sensitivities. Each neuron only receives input pulse events from a preset local distance receptive field to achieve sparse constraints on local connections.

[0039] Specifically, the first distance feature extraction layer can receive a composite pulse event sequence from the output of a multi-dimensional pulse time coding unit. This layer contains... N r Each group of distance-sensitive neurons contains [number] groups. M d There are 10 neurons with different Doppler sensitivities, therefore the total number of neurons in the first layer is 10. N r × M dFor example, 512 distance bins × 8 Doppler sensitivity levels = 4096 neurons. The spatial arrangement of the distance-sensitive neuron groups corresponds linearly to the radar's distance detection range. Neurons within the same group have different synaptic weight distributions according to their Doppler sensitivity: neurons with high Doppler sensitivity have a greater weight for input from the fast Doppler channel, and vice versa.

[0040] refer to Figure 4 The connection between the first distance feature extraction layer and the input composite impulse event sequence can be achieved using a local distance receptive field connection, for a distance index of... r A distance-sensitive neuron group with a distance index of 0, wherein the neurons within it can receive signals only from input impulse events with a distance index in [...]. r 0 - R_half , r 0 + R_half Input pulse events within the range, where R_half The preset local range receptive field radius can be represented, and its size can be approximately 2 to 5 range bins. The Doppler receptive field covers all Doppler levels. This locally connected design conforms to the physical characteristics of radar echoes, since the reflected energy of a real target typically only leaks or is distributed within a few adjacent range bins, while also ensuring the sparsity of the connections, reducing hardware synaptic storage and computational overhead. The output pulse of the first range feature extraction layer... S 1_i [ k It is transmitted to the second Doppler-angle joint layer.

[0041] The second Doppler-angle joint layer has several groups of direction-velocity joint sensitive neurons, each group containing neurons corresponding to the combination of Doppler level and angular sector, and the second Doppler-angle joint layer has inhibitory connections for lateral inhibition and sharpening of the Doppler and angular dimensions.

[0042] Specifically, the second Doppler-angle joint layer includes P Each group of directional-velocity-sensitive neurons corresponds to a quantized (Doppler level) group. d Angular sector If the Doppler levels are divided into 8 levels and the angles into 16 sectors, then the total number of neurons in the second layer is 28. Compared with the first layer, the second layer undergoes significant dimensionality reduction and information compression in space. It can be understood as compressing the thousands of neurons in the first layer into a few hundred neurons in the second layer, with each neuron expressing a specific (velocity, direction) hypothesis.

[0043] The synaptic connection between the second Doppler-angle joint layer and the first distance feature extraction layer can have explicit physical semantic constraints, that is, for representing a specific (Doppler level) d 0 Angular sector a 0 The second layer of neurons corresponds to the Doppler sensitivity in the first layer. d 0 The neurons in the first layer establish strong excitatory synaptic connections with neurons of higher Doppler sensitivity, while establishing weaker excitatory or inhibitory connections with adjacent neurons of higher Doppler sensitivity. This achieves lateral inhibition and sharpening in the Doppler dimension. Connections in the angular dimension also exhibit sector selectivity. The pulse firing pattern in the second layer reflects the strength of evidence for the presence of a target under a specific (velocity, direction) hypothesis. If, within multiple linear frequency modulated pulse chirp frame periods, neurons representing a specific (velocity, direction) combination continuously fire at a higher rate, it indicates a higher probability of a target moving at that velocity in that direction.

[0044] The third detection and decision layer has several output neurons, each neuron corresponding to a set of target parameter candidates, and receives input from neurons in the second Doppler-angle joint layer, as well as cross-layer direct connections from neurons in the first distance feature extraction layer.

[0045] Specifically, the third detection and decision layer can serve as the decision output layer of the network, which can contain N_{out} output neurons. Each output neuron can correspond to a set of potential target parameter candidates (including range, velocity, and angle). The number and arrangement of output layer neurons can be configured according to the application requirements of the radar, with typical values ​​ranging from 32 to 128 output target slots.

[0046] The third detection and decision layer can receive output pulses from the second Doppler-angle joint layer. Simultaneously, to maintain the precision of the distance information, the third detection and decision layer can also receive fine distance pulses from the first distance feature extraction layer via a cross-layer direct connection, thus preserving the accuracy of the distance information. This cross-layer direct connection allows the neurons in the third layer to simultaneously acquire fine distance pulse patterns from the first layer and compressed Doppler-angle information from the second layer, thereby making a more accurate judgment on the existence of the target.

[0047] The target parameter decoder unit is used to decode and output the target distance, angle, velocity parameters and confidence level based on the pulse firing rate and firing pattern of the output neurons in the third detection and decision layer.

[0048] Specifically, the output pulses of the third detection and decision layer are fed into the target parameter decoder unit. The target parameter decoder unit can maintain a time-sliding window, the length of which can correspond to one frame period of the radar, for example, approximately 50 ms to 100 ms. Within the time-sliding window, the pulse firing rate (i.e., the total number of pulses fired) of each output neuron can be counted. When the pulse firing rate of a certain output neuron exceeds a preset detection threshold... At this point, the target parameter decoder unit can determine that a real target exists at the physical parameter location corresponding to the neuron. Then, based on the neuron's hardware index, it can directly look up the associated precise distance, velocity, and angle parameters in a table, and calculate a normalized confidence score based on the magnitude of its pulse firing rate. Finally, the target parameter decoder unit can decode and output a list of target parameters (including the distance, velocity, angle, and confidence score for each target).

[0049] refer to Figure 5 The STDP online learning engine unit is used to adaptively update the synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer online based on the time difference between local presynaptic pulses and postsynaptic pulses during the operation of the accelerator.

[0050] Specifically, the STDP online learning engine unit is used to update the synaptic weights between the first range feature extraction layer and the second Doppler-angle joint layer in real time during radar operation. STDP, as a biologically inspired local learning rule, is based on the core idea that the change in synaptic weights depends only on the time difference between the firing time of the presynaptic neuron and the firing time of the postsynaptic neuron. If the presynaptic pulse precedes the postsynaptic pulse, it indicates that the presynaptic input has made a positive contribution to the activation of the postsynaptic neuron, and therefore the weight of that synapse should be increased (i.e., long-term potentiation (LTP) occurs); conversely, if the postsynaptic pulse precedes the presynaptic pulse, the weight of that synapse should be decreased (i.e., long-term inhibition (LTD) occurs).

[0051] The STDP online learning engine unit may include a time difference calculation module. When both the presynaptic and postsynaptic neurons of a synapse connection fire pulses within the current time window, the time difference calculation module reads their respective pulse timestamps and calculates the time difference between them. .in, t pre It can indicate the timing of the presynaptic pulse. t post It can indicate the timing of the postsynaptic pulse.

[0052] The STDP online learning engine can also include a step quantization lookup table. This application quantizes a continuous exponentially decaying learning window into a step function and pre-stores it in a step quantization lookup table. For example, the step quantization lookup table can quantize time difference ranges (such as...) The time step is divided into several intervals, each interval corresponding to a fixed, pre-calculated weight change. The time difference calculation module calculates... Then, the time difference can be directly used as the address to retrieve the stepped quantization lookup table, thereby instantly obtaining the corresponding weight change. This lookup table-based implementation completely avoids complex mathematical function calculations, requiring only simple comparators and memory read operations, thus greatly improving hardware execution efficiency.

[0053] The STDP online learning engine can also include a weight saturation accumulator. The weight saturation accumulator can receive the weight changes from the output of the ladder quantization lookup table. and compare it with the currently stored synaptic weights. w ji Saturation addition or subtraction is performed to obtain the updated synaptic weights, thus completing the weight update. To prevent unbounded growth or excessive decay of synaptic weights during long-term online learning, which could lead to neuronal saturation or failure, the updated synaptic weights can be limited to a preset value. Within the interval, weight updates use saturated addition and subtraction, for example, it can be limited to the 8-bit integer range of [0, 255]. When the accumulated result exceeds this range, it is automatically truncated to the maximum or minimum value.

[0054] The STDP online learning engine can also include weight storage. Weight storage can be implemented using a dual-port Block RAM (BRAM) within the FPGA. One port can be configured for the spiking neural network detection core to read synaptic weights in real time during forward inference, while the other port can be configured for the STDP online learning engine to asynchronously write updated synaptic weights in the background. This dual-port shared storage design ensures that forward inference and online learning can proceed in parallel without interference, thus guaranteeing the real-time performance of radar target detection.

[0055] The pulse routing and arbitration network unit is used to transmit pulse event packets between the layers of the multi-dimensional pulse timing coding unit and the spiking neural network detection unit, and between the layers of the spiking neural network detection unit.

[0056] In some embodiments, the pulse routing and arbitration network unit may include a routing table. The routing table can be stored in the on-chip distributed RAM of the FPGA and is used to quickly retrieve the target layer range and corresponding target neuron group address allowed for forwarding of a pulse event based on the source layer number and source neuron ID in the pulse event packet. To support the local connectivity and multicast requirements in SNNs, the routing table can support multiple routing modes such as unicast, multicast, and broadcast. For example, when a neuron in the first layer fires a pulse, the routing network can, according to the routing table, simultaneously copy and distribute the pulse event to multiple neurons in the second layer that are within its local receptive field connection range, without needing to broadcast to all neurons in the second layer, thus greatly saving routing bandwidth.

[0057] refer to Figure 6 The pulse routing and arbitration network can also include a distributed arbitrator. When multiple source neurons from the same or different layers send pulse events to the same destination neuron or the same routing channel at the same time, multi-source contention occurs. To resolve this conflict, the distributed arbitrator can employ a weighted round-robin arbitration strategy based on pulse timestamps. Specifically, each pulse event packet is assigned a precise timestamp upon generation, representing the actual occurrence time of the pulse in the time domain. During arbitration, the distributed arbitrator prioritizes comparing the timestamps of competing pulse events. Pulse events with earlier timestamps (i.e., earlier occurrence times) receive higher forwarding priority, ensuring that the crucial timing information in the spiking neural network (SNN) is not reversed or corrupted during transmission. However, to prevent pulse events with later timestamps from experiencing starvation due to prolonged contention failures, the distributed arbitrator can also introduce a configurable age bias mechanism. This age bias mechanism can manifest as follows: for pulse events waiting to be forwarded in the input buffer, their virtual priority automatically increases by one priority level for each preset system clock cycle. This weighted round-robin arbitration ensures both the priority transmission of time-sensitive pulses and the overall fairness and high throughput of the routing network.

[0058] The pulse routing and arbitration network can also include a pulse FIFO buffer. This buffer can be deployed at the input and output of each routing node to temporarily queue and buffer pulse event packets during brief network congestion. The depth of the pulse FIFO buffer can be optimized based on the maximum possible burst density of pulses at each layer, for example, from approximately a threshold of 16 to 64. By appropriately setting the buffer depth, pulse burst peaks can be effectively absorbed, preventing pulse event loss due to buffer overflow.

[0059] Preferably, it also includes a RISC-V or other embedded processor core, used to configure the threshold parameters of the multi-dimensional pulse time coding unit, the neuron parameters of each layer of the spiking neural network detection unit, the learning parameters of the STDP online learning engine unit, and the detection threshold after the radar is powered on. The radar operating mode is switched according to instructions from the upper-level perception system, with different modes corresponding to different preset parameter sets. When pulse FIFO overflow, abnormal saturation of neuron membrane potential, or abnormal fluctuations in detection output are detected, adaptive parameter adjustment or reset is triggered; the target parameter decoder unit reads the target list detected in the current frame and sends it to the upper-level system via CAN-FD or Ethernet interface.

[0060] Example 2 See Figure 2 This invention provides a real-time target detection system for millimeter-wave radar based on a spiking neural network. The system may include a millimeter-wave radar RF front-end, a multi-channel ADC module, a spiking neural network-based accelerator, and a result output module. In some embodiments, the components of the system can be connected via high-bandwidth, low-latency physical interfaces to ensure that radar echo data can flow efficiently between processing stages in a streaming manner.

[0061] The millimeter-wave radar RF front-end is used to transmit linear frequency modulated pulse chirp sequences, which are then mixed to obtain an intermediate frequency signal for transmission to a multi-channel ADC module.

[0062] Specifically, the millimeter-wave radar RF front-end includes a transmitting antenna and a receiving antenna. The number of transmitting antenna channels can be approximately 1-3, and the number of receiving antenna channels can be approximately 4-12, thus forming a multiple-input multiple-output (MIMO) radar configuration. The operating frequency band of the millimeter-wave radar RF front-end can be approximately 76 GHz to 81 GHz. The mixer inside the RF front-end can mix the received high-frequency echo signal with the transmitted signal to output an intermediate-frequency signal containing target range and velocity information.

[0063] The multi-channel ADC module is used to synchronously sample the intermediate frequency signal of each receiving channel.

[0064] Specifically, the multi-channel ADC module can be connected to the output of the millimeter-wave radar RF front-end to synchronously sample the intermediate frequency (IF) signal output from each receiving channel. The sampling rate of the multi-channel ADC module can be approximately 10 MSPS to 50 MSPS, and the quantization bit width can be approximately 12 bits to 16 bits. The multi-channel digital IF sampling sequence output by the multi-channel ADC module can be fed into an accelerator in real time for subsequent digital signal processing.

[0065] An accelerator based on a spiking neural network is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and output target parameters including distance, velocity, angle, and confidence level. The accelerator serves as the core processing platform of this invention, internally integrating a multi-dimensional pulse time coding unit, a spiking neural network detection unit, an STDP online learning engine unit, a pulse routing and arbitration network unit, a target parameter decoder unit, and a RISC-V or other embedded processor core.

[0066] The accelerator based on spiking neural networks adopts an event-driven accelerator microarchitecture, including a neuron computation array, membrane potential state management, synaptic weight storage and access, and time window management and batch processing.

[0067] The neuron computing array is the core computing unit of the accelerator. In one embodiment, the array employs M parallel neuron processing units (NPUs), each of which can perform membrane potential updates, threshold comparisons, and pulse firing decisions for one target neuron in each system clock cycle.

[0068] The NPU's computational pipeline consists of the following stages:

[0069] (1) During the instruction fetching phase, retrieve the pulse event that arrives in the current clock cycle from the pulse event FIFO and parse its source neuron ID and timestamp;

[0070] (2) In the table lookup stage, the corresponding weight values ​​are read from the synaptic weight storage based on the source neuron ID and the current target neuron ID. w ji ;

[0071] (3) In the accumulation phase, the weights are... w ji Membrane potential registers accumulated to the target neuron V i ;

[0072] (4) During the leakage phase, leakage attenuation is performed on the membrane potential at intervals of the number of cycles. );

[0073] (5) Judgment stage, comparison V i With distribution threshold V th If the value exceeds the limit, an output pulse event is generated and the membrane potential is reset.

[0074] (6) Write-back phase: Write the updated membrane potential back to the status register and write the output pulse event to the routing network input FIFO.

[0075] To improve throughput, the neural network computation array is grouped into layers, allowing NPUs in different layers to run simultaneously. For example, within the current clock cycle, the first-layer NPU group processes the encoder-to-layer pulses, while the second-layer NPU group processes the layer-to-layer pulses. This pipelined parallelism creates a flowing, time-determined pulse processing pipeline between layers.

[0076] For large-scale spiking neural networks (SNNs) (such as 4096 neurons in the first layer), the membrane potential and state flags (refractory period flag, last firing timestamp, etc.) of each neuron need to be stored in the on-chip Block RAM (BRAM) or UltraRAM of the FPGA. The organization of the state memory affects access bandwidth and power consumption.

[0077] In one embodiment, a dual-port BRAM is used to store membrane potentials. The membrane potentials of N neurons are stored in two BRAM blocks according to their odd and even addresses, allowing the reading and writing of the membrane potentials of two different neurons within the same cycle. For the historical pulse timestamps that the STDP online learning engine unit needs to trace, a small circular buffer (such as a FIFO with a depth of 16-32) is used to maintain the firing timestamps of the most recent pulses for each neuron, which are then read by the STDP engine.

[0078] Synaptic weight storage and access employs the following strategy to compress storage requirements:

[0079] (1) Sparse connectivity: The neurons in the first layer are only connected to sources within the local receptive field range (e.g., 5-10 distance bins) of the input impulse event, rather than being fully connected. For 4096 neurons in the first layer, each connection has approximately 50-100 sources (5 distance bins × 10 Doppler / angle combinations), and the total number of synapses is approximately 200,000-400,000, which is far lower than the 16 million of fully connected synapses. The weight of each synapse can be quantized to 4-8 bits.

[0080] (2) Weight sharing: Neurons with different Doppler sensitivities within the same distance-sensitive neuron group share part of the distance receptive field weights, while only the Doppler-selective weights are stored independently. This further reduces storage requirements.

[0081] (3) Hierarchical storage: weights that are frequently accessed are stored in distributed RAM or LUT RAM, while weights that are accessed less frequently are stored in BRAM.

[0082] (4) Optional weight compression: For larger network configurations, run-length encoding or sparse matrix CSR format can be used to further compress weight storage.

[0083] Time window management and batch processing: The time-driven nature of spiking neural networks means that the membrane potentials of all neurons need to be updated (leaking) at every time step, even in time steps where no input pulses arrive. To reduce idle power consumption in pulseless time steps, this invention employs an event-based time step skipping mechanism:

[0084] Maintain a globally minimum next event timestamp register After completing all pulse processing and membrane potential updates for the current time step, check all pulse input FIFOs, the membrane potentials of all neurons (whether they are close to the threshold), and the STDP pending event queue to calculate the next time step number that must be processed. There is more than 1000 time steps between the current time step and the current time step. K skip idle time steps ( K skip If configurable (e.g., 3 or greater), then perform a batch leak update (rather than a time-step update) on the membrane potential within the idle segment, and directly jump to... Time step.

[0085] The formula for batch leak updates is: ;

[0086] In the hardware, Pre-compute and store as a lookup table When a batch leaks, the corresponding entry can be read and multiplied and added with the current membrane potential in one operation, thus avoiding iterative leakage operations step by step.

[0087] Preferably, a RISC-V or other embedded processor core is used for system configuration, mode control, and anomaly handling. After the radar is powered on, the threshold parameters of the pulse time encoder, the neuron parameters of each layer of the spiking neural network, the STDP learning parameters, and the detection threshold are configured. The radar operating mode (such as high-speed cruise mode, urban low-speed mode, and parking mode) is switched according to the instructions of the upper-level perception system. Different modes correspond to different preset parameter sets. The status registers of each unit inside the FPGA are monitored. When pulse FIFO overflow, abnormal saturation of neuron membrane potential, or abnormal fluctuation of detection output is detected, parameter adaptive adjustment or reset is triggered. The target parameter decoder unit reads the target list detected in the current frame and sends it to the upper-level system through CAN-FD or Ethernet interface.

[0088] The result output module is used to encapsulate the target parameters in frames and send them to the upper-layer system via CAN-FD or Ethernet interface.

[0089] Specifically, the output module can be connected to an accelerator based on a spiking neural network to receive the output target parameter information. This target parameter information may include the target's distance, velocity, angle, and confidence level. The output module can encapsulate these target parameters frame by frame and send them to the upper-level autonomous driving control system or perception and decision-making system via an in-vehicle network interface or industrial communication interface. The in-vehicle network interface may include a CAN-FD interface, a gigabit Ethernet interface, etc.

[0090] Example 3

[0091] This embodiment provides a real-time target detection method for millimeter-wave radar based on a pulse neural network, including the following steps:

[0092] S1, Mix the frequency to obtain the intermediate frequency signal and sample it at the following rate. f s Multi-channel synchronous sampling is performed, and the intermediate frequency signal is encoded into a composite pulse event carrying distance information, Doppler information, and angle information without performing fast Fourier transform operations.

[0093] The S101 millimeter-wave radar RF front-end transmits a linear frequency modulated continuous wave, and the reflected echoes are simultaneously received by multiple receiving antenna channels. The received echo signals are mixed with the transmitted signals and low-pass filtered to output multiple intermediate frequency analog signals. A multi-channel ADC module operates at a preset sampling rate. f s Synchronous digital sampling of intermediate frequency analog signals is performed to generate multi-channel raw sampling sequences.

[0094] Preferably, optional DC offset correction and amplitude normalization can be performed on each sampled data stream to obtain the multi-channel original sampling sequence. ,in ch The channel index is represented by n, the sample point index is represented by n within a chirp, and the chirp index is represented by c.

[0095] S102, the amplitude of the sampling points within a single chirp is , and the first preset threshold Comparison. First preset threshold. The statistical measure of the amplitude of all sampling points within the current linear frequency modulated pulse chirp (e.g., median multiplied by a coefficient) k th Dynamically determined. For The sampling point generates a distance pulse event. ,in n Indicates the sampling index. t n Indicates the sampling time. a n This represents the normalized pulse intensity level.

[0096] To prevent a single strong target from generating too many pulses in adjacent distance bins, which could saturate the downstream spiking neural network (SNN), only the pulse events corresponding to the sampling points with the largest amplitudes are retained within a sliding window (e.g., 3-5 sampling points), while the rest are suppressed. The distance pulse events are packaged into pulse data packets of a uniform format, i.e., {Type: distance pulse, distance index:} n Timestamp: t n ,strength: a n Channel number: c}

[0097] S103, Radar transmission set up N c A frame consists of chirps. For a given distance bin index... r Collect cross N c Sample values ​​of chirp This constitutes a slow time series for the distance bin. The differential phase between adjacent chirps is calculated from this slow time series. ,in This indicates the operation of calculating the argument. Represents complex conjugate. Maps the differential phase to the pulse firing rate. f dop Generate Doppler pulse events ,in r Indicates distance index, c Indicates the chirp sequence number. This indicates the Doppler channel.

[0098] S104, For a given distance bin r and chirp index c ,Will N rx The sampled values ​​of each channel are arranged into a vector. Calculate the phase difference between adjacent channel pairs. This maps the phase difference between channels to a pulse time difference. Specifically, the pulse emission time of a reference channel (such as channel 1) is set as a baseline. t ref , No. k The pulse firing time offset corresponding to the channel is ,in These are the time-angle mapping coefficients. When a target is present in a direction, the channels exhibit a regular phase progression relationship, and the corresponding pulse sequences have a regular time offset pattern. The direction-sensitive neurons in subsequent layers of the spiking neural network (SNN) infer the target angle by detecting this regular time offset pattern. This generates angular pulse events carrying spatial orientation information. .

[0099] S105. Fuse the above three-dimensional pulse events to generate a composite pulse event sequence with a unified format. Sort by timestamp and eliminate duplicate events. The unified pulse event format after fusion is: {timestamp: T global Distance index: r Doppler levels: d Angular feature code: ,strength: a Source channel: c}

[0100] S2. The composite pulse event is input into the first distance feature extraction layer of the spiking neural network, and the first pulse is output and passed to the second Doppler-angle joint layer. The second Doppler-angle joint layer outputs a second pulse and passes it to the third detection and decision layer. The third detection and decision layer outputs target parameters including distance, angle, velocity and confidence. The synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer are updated online adaptively based on the time difference between the local presynaptic pulse and postsynaptic pulse.

[0101] S201, each layer of the spiking neural network consists of multiple LIF (Leaky Integrate-and-Fire) neurons, with sparse, physically semantically constrained forward connections between layers. In discrete time steps... k Above, the first i The membrane potential update equation for a neuron is: ;

[0102] in, Indicates the first i The neuron in the first k Membrane potential at each time step V rest This represents the resting potential and can be set to 0 or a configurable value. The leakage factor can take values ​​between 0 and 1, and can be implemented as a right shift operation; w ji Indicates from the first j The presynaptic neuron to the first iThe synaptic weight of a postsynaptic neuron can be positive (representing an excitatory synapse) or negative (representing an inhibitory synapse). Indicates the first j The presynaptic neuron in the first k The pulse output of the time step can take the value of 0 (representing no pulse) or 1 (representing pulse). V exc This indicates an excitatory reversal potential (e.g., 1.0). V inh This indicates an inhibitory reversal potential (e.g., -0.2).

[0103] when Exceeding the threshold V th At this time, the neuron fires an output pulse. Subsequently, its membrane potential was reset to V reset (can be equal to) V rest Furthermore, this neuron can enter a pre-defined refractory period. T ref During the refractory period T ref Within a time step (e.g., approximately 1 to 5 time steps), the neuron's membrane potential can remain at the resting potential, without accumulating any synaptic input.

[0104] In the hardware implementation, membrane potential updates can be simplified to fixed-point operations: the leakage term is implemented as follows: Subtract its own rightward shift The value after the bit; the synaptic input item is implemented as The accumulation (when) (when) or skip (when) (Time). Because the characteristic of spiking neural networks (SNNs) is sparse pulses, only a few presynaptic neurons fire pulses in most time steps. Therefore, the actual number of addition operations required is far less than the dense matrix multiplication and accumulation in fully connected neural networks.

[0105] S202. Input the generated composite impulse event into the first distance feature extraction layer. The first layer contains features corresponding to the distance bin. Groups of distance-sensitive neurons, each group containing There are 10 neurons with different Doppler sensitivities, therefore the total number of neurons in the first layer is 10. indivual.

[0106] The spatial arrangement of distance-sensitive neuron groups corresponds linearly to the distance detection range of radar. Neurons within the same group have different synaptic weight distributions according to their Doppler sensitivity: neurons with high Doppler sensitivity have a greater weight for input from the fast Doppler channel, and vice versa.

[0107] The connection between the first layer of neurons and the input compound impulse events is a local receptive field connection. For the distance-sensitive neuron group with distance index r0, it receives signals from distance indices [ r 0 - R_half , r 0 + R_half Input pulse events within the range, where R _half The receptive field radius is defined as 2-5 range bins. The Doppler receptive field covers all Doppler levels. This locally connected design conforms to the physical characteristics of radar echoes (a target typically affects several adjacent range bins) while ensuring sparsity of connections, reducing hardware synaptic storage and computational overhead. The output pulse of the first layer is passed to the second Doppler-angle joint layer.

[0108] S203, the second Doppler-angle joint layer performs Doppler-angle joint feature extraction on the pulses output from the first distance feature extraction layer. The second layer contains P groups of direction-velocity joint sensitive neurons. Compared to the first layer, the second layer undergoes significant dimensionality reduction and information compression, compressing the number of neurons from thousands in the first layer to just over a hundred in the second layer. Each neuron expresses a specific (velocity, direction) hypothesis. Lateral inhibition connections are used to sharpen the Doppler and angular dimensions. The combination of neurons with velocity-direction selectivity exhibits a high co-firing rate in the presence of a moving target. The output pulses are then passed to the third detection and decision layer.

[0109] S204. The third detection and decision layer performs a target existence determination, decodes and outputs target parameters including distance, velocity, angle, and confidence level. The third layer contains... N out The third layer has three output neurons, each corresponding to a set of target parameter candidates (distance, velocity, angle). The third layer receives pulse input from the second layer and direct cross-layer connections from neurons in the first layer to maintain the accuracy of the distance information. These direct cross-layer connections enable the neurons in the third layer to simultaneously acquire fine-grained distance pulse patterns (from the first layer) and compressed Doppler-angle information (from the second layer), thereby making a more accurate determination of the target's presence.

[0110] The output pulses from the third layer are fed into the target parameter decoder unit. Each time an output pulse is fired, the target parameter decoder unit records the neuron index corresponding to the firing time and the distance-Doppler-angle parameter associated with that neuron, and calculates the confidence score by combining this with the pulse firing rate (or the number of firings). If the pulse firing rate of a certain output neuron exceeds a preset detection threshold... If the distance, velocity, and angle parameters corresponding to the neuron are determined to be valid targets, then the neuron is considered to have these parameters.

[0111] During operation, the S3 and STDP online learning engine units perform online adaptive updates of the synaptic weights between the first and second layers based solely on the time difference between local presynaptic and postsynaptic pulses. This is done after each radar frame is processed (or every K frames). The weight change for each applicable synapse is calculated. The weights are then accumulated to a saturation level. The weight change is determined by the time difference between the presynaptic and postsynaptic pulses using a laddered quantization lookup table. The updated weight constraints are within a preset range. Within the range.

[0112] S301, For connecting presynaptic neurons j and postsynaptic neurons i The weight changes of the synapse. Accumulated calculations are performed at each presynaptic and postsynaptic pulse event. The basic STDP rule is:

[0113] If the presynaptic pulse arrives before the postsynaptic pulse (i.e.) Time difference This enhances synaptic weights (long-term enhancement, LTP): ;

[0114] If the postsynaptic pulse arrives before the presynaptic pulse, the time difference... This weakens synaptic weights (long-term inhibition, LTD): ;

[0115] in,, A + and A - These represent the maximum amplitudes of LTP and LTD, respectively. and These represent the time constants of LTP and LTD, respectively.

[0116] Preferably, the continuous exponential decay window can be further quantized as a step function: the time difference range (e.g., Each time step is divided into several intervals, and each interval corresponds to a fixed weight change. This is stored in a stepped quantization lookup table. When a learning event is triggered, the corresponding interval is read from the lookup table. , with current weight Perform saturation addition or subtraction to update the weights.

[0117] S302, Configure Global Learning Rate The learning rate can be adaptively adjusted based on the quality of the detection output. When the number of new targets detected in consecutive frames is small and the output confidence is high, it indicates that the current environment is relatively stable, and the learning rate can be reduced. When significant changes in the environment (such as a vehicle entering a city street from a highway, causing changes in clutter characteristics) cause temporary fluctuations in the detection output, the learning rate can be temporarily increased to accelerate adaptation. The learning rate adjustment is periodically calculated by the RISC-V processor based on detection statistics (number of targets detected in this frame, average confidence, false alarm / new target ratio, etc.) and written to the learning engine configuration register.

[0118] Preferably, after a spike event enters the routing network, its SrcLayer field is used to look up the allowed target layer range for forwarding (e.g., encoder output can only reach the first layer, and the first layer output can only reach the second and third layers directly connected). Then, distributed arbitration forwarding is performed based on the spatial coordinates of the target neuron (distance index, etc.). For the local connection mode of the SNN (from the first layer to the second layer), the spike event only needs to reach the target neuron within the preset connection range of the source neuron. The routing network utilizes this characteristic to achieve region-based multicast: the spike event is simultaneously sent to the corresponding local neuron group in the target layer, rather than broadcasting to all neurons. This significantly reduces the routing bandwidth requirements.

[0119] Preferably, when multiple sources within the same target layer simultaneously send pulses to the same target neuron group, the routing network arbitrates. In one embodiment, a timestamp-based weighted round-robin arbitration can be used: pulse events with earlier timestamps receive higher priority, but to prevent later pulses from being starved for a long time, a configurable age bias is introduced, where each waiting period... Each cycle adds one priority level. This timestamp-aware arbitration differs from traditional fixed-priority or pure polling arbitration, and is more suitable for preserving impulse timing information in SNNs.

[0120] Preferably, since the SNN layers are connected in a feedforward unidirectional manner (encoder to the first layer, then to the second layer, and finally to the third layer), there are no feedback loops, so the routing network naturally does not have a deadlock problem. For the optional intra-layer lateral suppression connection (used for side suppression), a separate dedicated suppression channel is used for transmission, which does not pass through the main routing network, thus avoiding the generation of channel dependency loops.

[0121] S4. The target parameters are encapsulated in frames and output through the communication interface.

[0122] Specifically, the target parameter list (distance, velocity, angle, confidence level) is encapsulated frame by frame and output to the upper-layer system through the communication interface. Simultaneously, the RISC-V embedded processor evaluates the current detection quality based on statistical indicators such as the number of targets detected in the current frame, average confidence level, and false alarm rate, adjusting the pulse coding threshold as necessary. SNN distribution threshold or STDP learning rate This prepares the frame for processing.

[0123] In some embodiments, in order to further improve the system's adaptability and detection accuracy in different complex scenarios, this application may introduce a variety of optional modified embodiments and optimization mechanisms.

[0124] In an optional embodiment, to increase pulse sparsity, the distance coding subunit can employ a pulse coding method based on an integrated-and-fire (I&F) model, instead of the amplitude threshold-based coding method described above. Specifically, the distance coding subunit can maintain a hardware integrator for each distance bin. At each sampling time, the integrator accumulates the amplitude value of the current sampling point as the input current. When the accumulated value of the integrator exceeds a preset firing threshold, the distance bin fires a distance pulse and the integrator is reset to zero. This integrated-and-fire coding method can better preserve the energy accumulation effect of weak targets, thereby improving the detection sensitivity for distant or low-reflectivity weak targets.

[0125] In another optional embodiment, the spiking neural network detection unit can employ a four-layer feedforward structure instead of the aforementioned three-layer structure. Specifically, the four-layer structure may include a first distance feature extraction layer, a second Doppler feature extraction layer, a third angle feature extraction layer, and a fourth detection and decision layer. By spatially decoupling and layering Doppler feature extraction and angle feature extraction, the computational complexity of a single-layer neuron and the number of synaptic connections can be further reduced, making it more suitable for signal processing of ultra-large-scale MIMO radar arrays.

[0126] In summary, the accelerator-based millimeter-wave radar real-time target detection system and method provided in this application, through innovative multi-dimensional pulse time coding, hierarchical SNN networks, STDP online adaptive learning, and event-driven FPGA hardware architecture, successfully replaces the computationally intensive FFT-CFAR-DOA link in traditional radar signal processing with a single path of time-domain pulse processing. This application not only eliminates the problems of dense multiplication-accumulation operations and intermediate data storage expansion caused by multi-dimensional FFT at its root, achieving significant power savings and low-latency transmission, but also overcomes the dependence of traditional CFAR algorithms on static statistical assumptions through data-driven online adaptive learning, significantly improving the detection stability and robustness of radar in complex and variable clutter environments. It possesses extremely high engineering practical value and broad market application prospects.

[0127] All embodiments described above are not limited to the illustrated embodiments and can, of course, be combined with each other. Although the present invention has been described with reference to the embodiments shown in the accompanying drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments can be made therefrom.

Claims

1. An accelerator based on a spiking neural network, deployed in a programmable logic device, characterized in that, include: A multi-dimensional pulse time coding unit is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and encode the intermediate frequency signal into a composite pulse event carrying distance information, Doppler information and angle information without performing a fast Fourier transform operation. The spiking neural network detection unit includes a first range feature extraction layer, a second Doppler-angle joint layer, and a third detection and decision layer, and the interlayer connections have a sparse constraint topology based on radar physical semantics; the first range feature extraction layer is used to receive the composite pulse event, output the pulse to pass to the next layer or make a target decision; The STDP online learning engine unit is used to adaptively update the synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer online based on the time difference between local presynaptic pulses and postsynaptic pulses during the operation of the accelerator. A pulse routing and arbitration network unit is used to transmit pulse event packets between the layers of the multi-dimensional pulse timing coding unit and the pulse neural network detection unit, and between the layers of the pulse neural network detection unit. The target parameter decoder unit is used to decode and output the target distance, angle, velocity parameters and confidence level based on the pulse firing rate and firing pattern of the output neurons in the third detection and decision layer.

2. The accelerator according to claim 1, characterized in that, The multi-dimensional pulse time encoding unit includes a distance encoding subunit, which is used to perform point-by-point amplitude calculation on the ADC sampling sequence within a single linear frequency modulated pulse chirp, and compare the calculated amplitude with a first preset threshold. After performing local non-maximum suppression through a sliding window, a distance pulse event carrying a distance index, timestamp, and pulse intensity is generated. The first preset threshold is dynamically determined by the statistics of the amplitudes of all sampling points within the current linear frequency modulated pulse chirp.

3. The accelerator according to claim 1, characterized in that, The multi-dimensional pulse time coding unit includes a Doppler coding subunit, which is used to calculate the differential phase between adjacent linear frequency modulated pulses chirp for an ADC sampling sequence spanning multiple linear frequency modulated pulse chirp at the same distance bin, and to linearly map or symbolically approximate the differential phase as a pulse firing rate or pulse firing level to generate a Doppler pulse event.

4. The accelerator according to claim 1, characterized in that, The multi-dimensional pulse time encoding unit includes an angle encoding subunit, which is used to arrange the multi-channel sampled values ​​on the same distance bin and the same linear frequency modulated pulse chirp into a vector, calculate the inter-channel phase difference between adjacent channels, and map the inter-channel phase difference into a pulse time difference to generate an angle pulse event.

5. The accelerator according to claim 1, characterized in that, The first distance feature extraction layer has several groups of distance-sensitive neurons, each group containing multiple neurons with different Doppler sensitivities, and each neuron only receives input pulse events from a preset local distance receptive field to achieve sparse constraints on local connections. The second Doppler-angle joint layer has several groups of direction-velocity joint sensitive neurons, each group containing neurons corresponding to the combination of Doppler level and angular sector, and the second Doppler-angle joint layer has inhibitory connections for lateral inhibition and sharpening of the Doppler dimension and angular dimension; The third detection and decision layer has several output neurons, each neuron corresponding to a set of target parameter candidates, and receives input from neurons in the second Doppler-angle joint layer, and receives cross-layer direct connections from neurons in the first distance feature extraction layer.

6. The accelerator according to claim 1, characterized in that, The STDP online learning engine unit includes a time difference calculation module, a step quantization lookup table, and a weight saturation accumulator. The time difference calculation module is used to quantize the time difference between presynaptic pulses and postsynaptic pulses into a finite number of intervals, determine the fixed weight change of the corresponding interval by searching the step quantization lookup table, and perform saturation addition and subtraction update on the current synaptic weights through the weight saturation accumulator to limit the updated synaptic weights within a preset fixed number of points.

7. The accelerator according to claim 1, characterized in that, The spiking neural network detection unit is also used to execute an event-based time step jump mechanism: maintain a global minimum next event timestamp, and when the number of idle time steps between the current time step and the global minimum next event timestamp exceeds a second preset threshold, perform batch leak updates on the neuron membrane potential using a pre-calculated leak lookup table, and directly jump to the time step corresponding to the global minimum next event timestamp.

8. The accelerator according to claim 1, characterized in that, The pulse routing and arbitration network unit adopts a weighted round-robin arbitration strategy based on pulse timestamps, in which pulse events with earlier timestamps receive higher priority. A configurable age bias is introduced, and a priority level is increased every preset waiting period to prevent later-arriving pulse events from being starved for a long time.

9. A real-time target detection system for millimeter-wave radar based on a pulse neural network, characterized in that, include: The millimeter-wave radar RF front-end is used to transmit linear frequency modulated pulse chirp sequences, which are then mixed to obtain an intermediate frequency signal for a multi-channel ADC module. A multi-channel ADC module is used to synchronously sample the intermediate frequency signal of each receiving channel; An accelerator based on a pulse neural network is used to receive the intermediate frequency signal sampled by the multi-channel ADC module and output target parameters including distance, velocity, angle and confidence level. The result output module is used to encapsulate the target parameters in frames and send them to the upper-layer system via CAN-FD or Ethernet interface.

10. A real-time target detection method for millimeter-wave radar based on a pulse neural network, characterized in that, The method includes: The intermediate frequency signal obtained by mixing is sampled at a certain rate. f s Multi-channel synchronous sampling is performed, and the intermediate frequency signal is encoded into a composite pulse event carrying distance information, Doppler information, and angle information without performing fast Fourier transform operations; The composite pulse event is input into the first distance feature extraction layer of the spiking neural network, and the first pulse is output and passed to the second Doppler-angle joint layer. The second Doppler-angle joint layer outputs a second pulse and passes it to the third detection and decision layer. The third detection and decision layer outputs target parameters including distance, angle, velocity and confidence. The target parameters are encapsulated in frames and output through the communication interface; Specifically, based on the time difference between local presynaptic pulses and postsynaptic pulses, the synaptic weights of the first distance feature extraction layer and the second Doppler-angle joint layer are adaptively updated online.

Images