Sensing, storing and computing integrated signal processing method and system based on SNN and ANN hybrid architecture

Abstract

The invention discloses a sensing, storing and computing integrated signal processing method and system based on an SNN and ANN hybrid architecture. The system comprises a sensing module, an SNN processing unit, a storage module and an ANN processing unit, and integration of sensing, storage and brain-like calculation is realized through single-chip integration. The method comprises the steps that the sensing module generates event stream data; the SNN processing unit performs event-driven feature extraction; the storage module is used for realizing data caching and in-memory calculation; the ANN processing unit executes high-precision processing; and performing result feedback to optimize SNN parameters. According to the invention, the low-power-consumption event-driven mechanism of the SNN and the high-precision advantage of the ANN are utilized, and the in-storage calculation function of the storage module is combined, so that the energy consumption and delay are remarkably reduced, the recognition accuracy is ensured, and the method is suitable for real-time signal processing scenes such as the Internet of Things and automatic driving.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and signal processing technology, and in particular to a sensing-storage-computing integrated signal processing method and system based on a hybrid architecture of spiking neural network (SNN) and artificial neural network (ANN). In particular, it is a low-power, high-real-time signal processing method that integrates sensing, storage and brain-like computing functions, and is suitable for IoT terminals, edge computing devices and real-time monitoring systems. Background Technology

[0002] Traditional signal processing systems typically employ ANNs for data processing. However, ANNs rely on high-precision floating-point operations, resulting in high energy consumption and significant latency in real-time dynamic signal processing. For example, while patent CN110457892A (a CNN-based image processing method) improves recognition accuracy, it requires continuous power for data sampling and computation, leading to low energy efficiency. SNNs, which mimic biological neuron mechanisms, offer advantages such as event-driven and sparse coding, significantly reducing power consumption (as described in patent CN112115821B). However, their training is complex, computational accuracy is insufficient, and they struggle to independently process high-dimensional signals. Furthermore, existing technologies separate perception, storage, and computation functions (e.g., sensor outputs need to be transmitted via bus to independent memory and processors), resulting in high data transfer latency and low system integration. For instance, in autonomous driving scenarios, camera-perceived data needs to travel back and forth between memory and processor multiple times, causing response latency exceeding 50ms, which fails to meet real-time requirements. Therefore, there is an urgent need for an architecture that integrates the advantages of SNNs and ANNs and achieves a unified perception-storage-computing architecture to address the balance between energy efficiency, latency, and accuracy.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a signal processing method and system based on a hybrid architecture of SNN and ANN that integrates sensing, storage and computing. By integrating sensing, storage and brain-like computing functions at the hardware level, it solves the problems of high energy consumption, large latency and functional separation in the prior art, and achieves low power consumption and high real-time signal processing.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A smart signal processing method includes the following steps: 1. Sensing Phase: External signals are captured by the sensing module to generate event stream data; the event stream data is a sparse pulse sequence based on time coding; 2. SNN Processing Stage: The event stream data is input into the SNN processing unit for preliminary event-driven feature extraction; the SNN processing unit adopts the pulse temporal dependent plasticity (STDP) learning rule and outputs a low-dimensional feature pulse sequence. 3. Storage stage: The low-dimensional feature pulse sequence is dynamically written into the storage module; the storage module is based on a memristor array to realize in-memory computing function and supports in-situ caching and updating of data; 4. ANN Processing Stage: Read cached data from the storage module and input it into the ANN processing unit for high-precision classification or prediction; the ANN processing unit adopts a lightweight convolutional neural network and outputs the final processing result; 5. Feedback optimization: The ANN processing results are fed back to the SNN processing unit to adjust the STDP learning parameters and achieve collaborative optimization of the hybrid architecture.

[0006] An intelligent signal processing system, comprising: 1. Sensing module: Used to capture external signals and convert them into event stream data, including optical sensors or acoustic sensor arrays; 2. SNN processing unit: Directly connected to the perception module, it receives event stream data and performs event-driven processing, including a spiking neuron layer and STDP learning circuit; 3. Storage Module: Bidirectionally connected to the SNN processing unit and ANN processing unit, it realizes dynamic data storage and in-memory computation based on the cross-memristor array; 4. ANN processing unit: Connected to the storage module, it receives cached data and performs high-precision calculations, including lightweight convolutional layers and fully connected layers; The sensing module, SNN processing unit, storage module, and ANN processing unit are integrated through a system-on-a-chip (SoC) to form a single-chip architecture, eliminating data transfer delays.

[0007] Compared with the prior art, the present invention has the following technical effects: 1. Improved energy efficiency: The event-driven mechanism of SNN is activated only when the signal changes, significantly reducing standby power consumption; 2. Reduced latency: The in-memory computing function of the storage module avoids data movement, greatly reducing processing latency; 3. Accuracy Guarantee: The ANN processing unit compensates for the insufficient accuracy of the SNN, resulting in a significant improvement in recognition accuracy; 4. High integration: A single chip realizes a closed loop of sensing-storage-computing, which is suitable for resource-constrained edge devices.

[0008] The present invention will be further described below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a system architecture diagram of the present invention, showing the hardware connection relationship and data flow direction of the perception module, SNN processing unit, storage module and ANN processing unit.

[0011] Figure 2 This is a signal processing flowchart of the present invention, which shows the complete method steps from signal perception to result output and the key data conversion process. Detailed Implementation

[0012] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0013] Reference Figure 1The system of this invention is implemented using a single-chip SoC architecture. The perception module 101 consists of a CMOS image sensor array used to capture video stream signals; when pixel brightness changes exceed a threshold, it generates time-encoded event stream data (such as address event representation in AER format). The SNN processing unit 102 is directly physically connected to the output port of the perception module 101, and contains three layers of spiking neurons (input layer, hidden layer, and output layer), employing a Leaky Integrate-and-Fire (LIF) model; the event stream data triggers the accumulation of neuronal membrane potentials, and when a threshold is reached, it outputs a pulse, adjusting synaptic weights through an STDP circuit to output a low-dimensional feature pulse sequence. The storage module 103 is directly connected to the SNN processing unit 102 and the ANN processing unit 104; the low-dimensional feature pulse sequence is written to storage, achieving non-volatile data storage and in-situ addition operations (in-memory computation). The ANN processing unit 104 contains a lightweight MobileNetV2 structure, reads data from the storage module 103, performs convolution and fully connected operations, and outputs classification results (such as target recognition labels). The system synchronizes each module through a shared clock signal, and the ANN processing result is returned to the SNN processing unit 102 via feedback channel 105 to dynamically adjust the learning rate parameter.

[0014] Reference Figure 2 The implementation steps of the method of the present invention are as follows: Step S201: The sensing module 101 captures external signals (such as surveillance video) in real time. When a motion event is detected, it generates a sparse pulse sequence (event stream data) and transmits it to the SNN processing unit 102. Step S202: The SNN processing unit 102 performs event-driven processing on the event stream data: the input layer neurons encode time information, the hidden layer extracts motion features through the STDP rule, and the output layer generates a low-dimensional feature pulse sequence; Step S203: The low-dimensional feature pulse sequence is written into the storage module 103; at the same time, the storage module 103 performs an in-memory accumulation operation to preprocess some features; Step S204: The ANN processing unit 104 reads cached data from the storage module 103, converts it into a digital feature vector, extracts high-level semantics through a lightweight convolutional layer, and outputs the final result (such as a "pedestrian" or "vehicle" label). Step S205: If the confidence level of the ANN processing unit 104 is lower than the threshold (e.g., 80%), the SNN processing unit 102 is triggered through the feedback channel 105 to enhance the learning intensity and optimize subsequent processing.

[0015] In a specific application, taking a smart camera as an example: the camera sensing module captures 1080P@30fps video, the SNN processing unit processes only a small proportion of event pixels (moving areas), significantly reducing power consumption; the storage module caches feature data; the ANN processing unit completes target recognition, greatly improving accuracy and resulting in significant energy savings compared to a pure ANN solution. This embodiment verifies the comprehensive advantages of the present invention in terms of real-time performance, energy efficiency, and accuracy.

[0016] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any person skilled in the art can make many possible variations and modifications to the technical solution of the present invention, or modify it into equivalent embodiments, without departing from the scope of the present invention's technical solution. Therefore, all equivalent changes made based on the shape, structure, and principle of the present invention without departing from the scope of the present invention's technical solution should be covered within the protection scope of the present invention.

Claims

1. A signal processing method integrating sensing, storage, and computation based on a hybrid architecture of SNN and ANN, characterized in that, Includes the following steps: Step 1: Capture external signals through the sensing module (101) and generate a time-coded sparse pulse sequence as event stream data; Step 2: Input the event stream data into the SNN processing unit (102), use the pulse temporal dependence plasticity learning rule to perform event-driven feature extraction, and output a low-dimensional feature pulse sequence; Step 3: Dynamically write the low-dimensional feature pulse sequence into the storage module (103) to realize in-situ caching and in-memory computation of the data; Step 4: Read cached data from the storage module (103), input it into the ANN processing unit (105) for high-precision classification or prediction, and output the final processing result; Step 5: Feed the final processing result back to the SNN processing unit (102), adjust the learning parameters, and achieve collaborative optimization of the hybrid architecture.

2. The integrated signal processing method based on a hybrid SNN and ANN architecture as described in claim 1, characterized in that, In step 1, the event stream data is generated using the Address Event Representation (AER) format, and pulse output is triggered only when the signal change exceeds a preset threshold.

3. The integrated signal processing method based on a hybrid SNN and ANN architecture as described in claim 1, characterized in that, In step 2, the neuron model of the SNN processing unit (102) adopts the Leaky Integrate-and-Fire LIF model, and the dimension of the low-dimensional feature pulse sequence is compressed to 10%~30% of the original data.

4. The integrated signal processing method based on a hybrid SNN and ANN architecture as described in claim 1, characterized in that, In step 3, the storage module (103) stores the data and performs in-memory calculations.

5. The integrated sensing, storage, and computing signal processing method based on a hybrid SNN and ANN architecture as described in claim 1, characterized in that, In step 4, the ANN processing unit (104) adopts a lightweight convolutional neural network structure, which includes depth-separable convolutional layers.

6. A sensor-memory-computing integrated signal processing system based on a hybrid architecture of SNN and ANN, used to implement the method described in any one of claims 1-5, characterized in that, It includes a perception module (101), an SNN processing unit (102), a storage module (103), an ANN processing unit (104), and a feedback channel (105), wherein: The sensing module (101) is used to capture external signals and convert them into event stream data; The SNN processing unit (102) is directly connected to the perception module (101), receives event stream data and performs event-driven processing; The storage module (103) is bidirectionally connected to the SNN processing unit (102) and the ANN processing unit (104), supporting dynamic data storage and in-memory computation; The ANN processing unit (104) is connected to the storage module (103), receives cached data and performs high-precision calculations; The sensing module (101), SNN processing unit (102), storage module (103) and ANN processing unit (104) are integrated into a single-chip SoC, and data transfer delay is eliminated through physical interconnection.

7. The integrated signal processing system based on a hybrid SNN and ANN architecture as described in claim 6, characterized in that, The storage module (101) supports analog domain data processing and digital domain data processing.

8. The integrated signal processing system based on a hybrid SNN and ANN architecture as described in claim 6, characterized in that, The SNN processing unit (102) and the ANN processing unit (104) are synchronized by a shared clock signal, and the feedback channel is used to transmit the ANN processing results to adjust the learning rate.

Images