Time-domain signal processing method and apparatus, medium

Abstract

This disclosure relates to a time-domain signal processing method, apparatus, and medium. The method includes inputting a time-domain signal to be processed for a target task into heterogeneous neurons in a time-domain heterogeneous neuron layer. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time-scale information. At each heterogeneous neuron: each heterogeneous dendritic branch processes the input signal to be processed based on different first time factors to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each heterogeneous dendritic branch based on a second time factor to obtain a pulse output result; and a target result for the target task is determined based on the pulse output results of each heterogeneous neuron. The time-domain signal processing method, apparatus, and medium according to the embodiments of this disclosure can efficiently process multi-time-scale information, improve its ability to process multi-scale temporal information, and achieve high comprehensive performance in practical temporal tasks.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a time-domain signal processing method, apparatus, and medium. Background Technology

[0002] In the exploration of artificial general intelligence, neuromorphic computing has received widespread attention. Neuromorphic computing refers to breaking away from the von Neumann computing framework by drawing on and imitating the information processing mechanisms of the human brain, from multiple levels such as algorithms and hardware architecture.

[0003] Spiking Neural Networks (SNNs), a key algorithmic model in neuromorphic computing, are considered the third generation of neural networks. By mimicking the neurons and connections in the human brain, SNNs possess rich encoding capabilities, diverse spatiotemporal dynamic mechanisms, and event-driven characteristics. They can process complex spatiotemporal information and can be effectively deployed on neuromorphic hardware platforms, offering advantages such as high speed and low power consumption.

[0004] Currently, cutting-edge research on spiking neural networks primarily employs either the Integrate-and-Fire (IF) neuron model or the Leaky-Integrate-and-Fire (LIF) neuron model. The IF model directly sums the weighted synaptic inputs and then applies a threshold function to induce 0 / 1 pulses in the neuron. The LIF model, in addition to the IF model, incorporates a cell body membrane potential decay mechanism. Both of these neuron models can fit the real-valued activation functions of Artificial Neural Networks (ANNs) through frequency encoding at sufficiently long time steps. However, they are relatively simple abstract models of point neurons, merely simulating the evolutionary mechanism of neurons in the cell body of the human brain, and lack the ability to process multi-scale temporal information. In real-world environments, time-series sensing signals typically exhibit complex characteristics such as multiple time scales, long time durations, and noise. Existing spiking neural network models struggle to handle multi-time-scale information and suffer from drawbacks such as low computational efficiency, insufficient generalization and robustness, and difficulty in learning. Consequently, most current research on spiking neural networks utilizes feedforward network structures and applies them to the field of image recognition. Summary of the Invention

[0005] In view of this, this disclosure proposes a time-domain signal processing method, apparatus, and medium that can efficiently process information at multiple time scales, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale time-series information, and achieve high comprehensive performance in practical time-series tasks.

[0006] According to one aspect of this disclosure, a time-domain signal processing method is provided. The method utilizes a spiking neuron model for signal processing. The spiking neuron model includes a time-domain heterogeneous neuron layer, which comprises multiple heterogeneous neurons. Each heterogeneous neuron includes a cell body with multiple heterogeneous dendritic branches, and each heterogeneous dendritic branch has a different first time factor for processing signals transmitted to the heterogeneous dendritic branches. Each cell body has a different second time factor for processing signals transmitted to the cell body. The method includes:

[0007] The time-domain signal to be processed for the target task is input to each of the heterogeneous neurons in the time-domain heterogeneous neuron layer. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time scale information.

[0008] At each of the heterogeneous neurons: each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on a second time factor to obtain a pulse output result.

[0009] The target result for the target task is determined based on the pulse output results of each of the heterogeneous neurons.

[0010] Thus, the novel brain-inspired spiking neuron model constructed using sparse tree-like topology and dendritic and cell body dynamics can efficiently process information at multiple time scales, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale temporal information, and achieve high comprehensive performance in practical temporal tasks.

[0011] In one possible implementation, when the spiking neuron model includes multiple temporally heterogeneous neuron layers, the spiking output results obtained by each heterogeneous neuron in the previous temporally heterogeneous neuron layer are used as the input of the next temporally heterogeneous neuron layer, so that each heterogeneous neuron in the next temporally heterogeneous neuron layer receives each of the spiking output results; determining the target result for the target task based on the spiking output results of each heterogeneous neuron includes: determining the target result based on the spiking output results obtained by each heterogeneous neuron in the last temporally heterogeneous neuron layer.

[0012] In this way, by using a spiking neuron model with multiple temporally heterogeneous neuron layers for time-domain signal processing, we can better handle multi-scale time information with complex frequency domains and obtain higher accuracy, better robustness and generalization.

[0013] In one possible implementation, the spiking neuron model further includes an input layer and an output layer; the input layer is used to divide the time-domain signal to be processed to obtain the plurality of signals to be processed, and a plurality of first channels are provided between the input layer and the time-domain heterogeneous neuron layer, so that each signal to be processed is transmitted to each heterogeneous dendritic branch of the time-domain heterogeneous neuron through different first channels; a plurality of second channels are provided between the output layer and the time-domain heterogeneous neuron layer, so that the spiking output results of each heterogeneous neuron are transmitted to the output layer through different second channels, and the output layer is used to determine the target result based on the spiking output results of each heterogeneous neuron.

[0014] In this way, not only can the dimensionality reduction of high-dimensional signals be achieved, but it also helps to transmit the time-domain signal to be processed to the time-domain heterogeneous neuron layer through the first channel for signal processing. Furthermore, the output of the time-domain heterogeneous neuron layer, i.e. the pulse output result, is transmitted to the output layer through the second channel, so as to obtain the target result corresponding to the target task and realize the processing of the time-domain signal to be processed.

[0015] In one possible implementation, each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body, including: at each of the heterogeneous dendritic branches: the heterogeneous dendritic branch processes the input signal to be processed based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained by the heterogeneous dendritic branch at the previous time based on the first time factor to obtain a third output value, and uses the sum of the second output value at the current time and the third output value as the first output value.

[0016] Thus, a spiking neuron model is constructed based on the dendritic dynamics. Each heterogeneous neuron in this model is a complex neuron with a heterogeneous dendritic structure. Each heterogeneous dendritic branch has a different first time factor to process information of different frequencies. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0017] In one possible implementation, the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the second time factor to obtain a pulse output result, including: the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value, and uses the sum of the fourth output value at the current time and the fifth output value as the fourth output value at the current time; and determines the pulse output result based on the fourth output value at the current time and a preset threshold.

[0018] In this way, a spiking neuron model is constructed based on cell body dynamics. Each heterogeneous neuron in this model can have a different second time factor to process the information transmitted from the heterogeneous dendritic branches. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0019] In one possible implementation, the temporal heterogeneous neuron layer further includes a feedback channel, which transmits the pulse output result obtained by the cell body at the previous time to the heterogeneous dendritic branch through the feedback channel.

[0020] In this way, adding a feedback channel to the spiking neuron model can further enhance the spiking neuron model's ability to process signals, which is beneficial for better handling of multi-scale time information with complex frequency domains. It can achieve higher accuracy, better robustness and generalization in practical tasks involving multiple time information processing.

[0021] In one possible implementation, the target task includes any one of the following: image processing task, electroencephalogram (EEG) signal processing task, and speech signal processing task.

[0022] In this way, the spiking neuron model can be used to handle different signal processing tasks and obtain corresponding target results, making it more widely applicable.

[0023] According to another aspect of this disclosure, a time-domain signal processing apparatus is provided. The apparatus utilizes a spiking neuron model for signal processing. The spiking neuron model includes a time-domain heterogeneous neuron layer, which includes a plurality of heterogeneous neurons. Each heterogeneous neuron includes a cell body having a plurality of heterogeneous dendritic branches, and each heterogeneous dendritic branch has a different first time factor for processing signals transmitted to the heterogeneous dendritic branches. Each cell body has a different second time factor for processing signals transmitted to the cell body. The apparatus includes:

[0024] An input module is configured to input a time-domain signal to be processed for a target task into each of the heterogeneous neurons in the time-domain heterogeneous neuron layer. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time scale information.

[0025] The processing module is configured such that at each of the heterogeneous neurons: each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on a second time factor to obtain a pulse output result.

[0026] An output module is configured to determine a target result for the target task based on the pulse output results of each of the heterogeneous neurons.

[0027] Thus, the novel brain-inspired spiking neuron model constructed using sparse tree-like topology and dendritic and cell body dynamics can efficiently process information at multiple time scales, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale temporal information, and achieve high comprehensive performance in practical temporal tasks.

[0028] In one possible implementation, when the spiking neuron model includes multiple temporally heterogeneous neuron layers, the spiking output results obtained by each heterogeneous neuron in the previous temporally heterogeneous neuron layer are used as the input of the next temporally heterogeneous neuron layer, so that each heterogeneous neuron in the next temporally heterogeneous neuron layer receives each of the spiking output results; determining the target result for the target task based on the spiking output results of each heterogeneous neuron includes: determining the target result based on the spiking output results obtained by each heterogeneous neuron in the last temporally heterogeneous neuron layer.

[0029] In this way, by using a spiking neuron model with multiple temporally heterogeneous neuron layers for time-domain signal processing, we can better handle multi-scale time information with complex frequency domains and obtain higher accuracy, better robustness and generalization.

[0030] In one possible implementation, the spiking neuron model further includes an input layer and an output layer; the input layer is used to divide the time-domain signal to be processed to obtain the plurality of signals to be processed, and a plurality of first channels are provided between the input layer and the time-domain heterogeneous neuron layer, so that each signal to be processed is transmitted to each heterogeneous dendritic branch of the time-domain heterogeneous neuron through different first channels; a plurality of second channels are provided between the output layer and the time-domain heterogeneous neuron layer, so that the spiking output results of each heterogeneous neuron are transmitted to the output layer through different second channels, and the output layer is used to determine the target result based on the spiking output results of each heterogeneous neuron.

[0031] In this way, not only can the dimensionality reduction of high-dimensional signals be achieved, but it also helps to transmit the time-domain signal to be processed to the time-domain heterogeneous neuron layer through the first channel for signal processing. Furthermore, the output of the time-domain heterogeneous neuron layer, i.e. the pulse output result, is transmitted to the output layer through the second channel, so as to obtain the target result corresponding to the target task and realize the processing of the time-domain signal to be processed.

[0032] In one possible implementation, each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body, including: at each of the heterogeneous dendritic branches: the heterogeneous dendritic branch processes the input signal to be processed based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained by the heterogeneous dendritic branch at the previous time based on the first time factor to obtain a third output value, and uses the sum of the second output value at the current time and the third output value as the first output value.

[0033] Thus, a spiking neuron model is constructed based on the dendritic dynamics. Each heterogeneous neuron in this model is a complex neuron with a heterogeneous dendritic structure. Each heterogeneous dendritic branch has a different first time factor to process information of different frequencies. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0034] In one possible implementation, the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the second time factor to obtain a pulse output result, including: the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value, and uses the sum of the fourth output value at the current time and the fifth output value as the fourth output value at the current time; and determines the pulse output result based on the fourth output value at the current time and a preset threshold.

[0035] In this way, a spiking neuron model is constructed based on cell body dynamics. Each heterogeneous neuron in this model can have a different second time factor to process the information transmitted from the heterogeneous dendritic branches. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0036] In one possible implementation, the temporal heterogeneous neuron layer further includes a feedback channel, which transmits the pulse output result obtained by the cell body at the previous time to the heterogeneous dendritic branch through the feedback channel.

[0037] In this way, adding a feedback channel to the spiking neuron model can further enhance the spiking neuron model's ability to process signals, which is beneficial for better handling of multi-scale time information with complex frequency domains. It can achieve higher accuracy, better robustness and generalization in practical tasks involving multiple time information processing.

[0038] In one possible implementation, the target task includes any one of the following: image processing task, electroencephalogram (EEG) signal processing task, and speech signal processing task.

[0039] In this way, the spiking neuron model can be used to handle different signal processing tasks and obtain corresponding target results, making it more widely applicable.

[0040] According to another aspect of this disclosure, a time-domain signal processing apparatus is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described time-domain signal processing method when executing instructions stored in the memory.

[0041] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the above-described time-domain signal processing method.

[0042] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0043] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0044] Figure 1 A schematic diagram of a spiking neuron model provided according to an embodiment of the present disclosure is shown.

[0045] Figure 2 A schematic diagram of a spiking neuron model provided according to an embodiment of the present disclosure is shown.

[0046] Figure 3 A schematic diagram illustrating the calculation process of a spiking neuron model provided according to an embodiment of the present disclosure is shown.

[0047] Figure 4 A schematic diagram illustrating the calculation process of a spiking neuron model provided according to an embodiment of the present disclosure is shown.

[0048] Figure 5 A schematic diagram of a spiking neuron model provided according to an embodiment of the present disclosure is shown.

[0049] Figure 6 A flowchart is shown for a time-domain signal processing method provided according to an embodiment of the present disclosure.

[0050] Figure 7 A schematic diagram illustrating the processing of an EEG signal processing task according to an embodiment of the present disclosure is shown.

[0051] Figure 8 A schematic diagram illustrating the processing of a speech signal processing task according to an embodiment of the present disclosure is shown.

[0052] Figure 9 A block diagram of a time-domain signal processing apparatus provided according to an embodiment of the present disclosure is shown.

[0053] Figure 10 A block diagram of an apparatus for a time-domain signal processing method provided according to an embodiment of the present disclosure is shown. Detailed Implementation

[0054] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0055] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0056] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0057] To facilitate understanding of the technical solutions provided by the embodiments of this disclosure by those skilled in the art, the technical environment for implementing the technical solutions will be described below.

[0058] In biological research, brain neural networks possess a natural ability to process information across time scales, such as easily recognizing speech content at different speeds and moving objects at different speeds. However, in real-world environments, due to issues such as inappropriate modeling, high computational costs, and a lack of learning algorithms, existing spiking neural network models have failed to efficiently process complex temporal signals. They also suffer from drawbacks such as low computational efficiency, insufficient generalization and robustness, and difficulty in learning. Therefore, research on spiking neural networks is particularly important.

[0059] This disclosure provides a time-domain signal processing method. The method utilizes a spiking neuron model including a time-domain heterogeneous neuron layer for signal processing. The method inputs a time-domain signal to be processed, containing multi-timescale information for a target task, into each heterogeneous neuron in the time-domain heterogeneous neuron layer. At each heterogeneous neuron: each heterogeneous dendritic branch processes the input signal to be processed based on different first time factors to obtain a first output value and sends it to the cell body. The cell body then processes multiple first output values ​​from each heterogeneous dendritic branch based on a second time factor to obtain a spiking output result, thereby enabling signal processing based on each heterogeneous neuron. The pulse output results determine the target result for the target task. Each heterogeneous dendritic branch has a different first time factor, which is used to process the signal transmitted to the heterogeneous dendritic branch. Each cell body has a different second time factor, which is used to process the signal transmitted to the cell body. In this way, the novel brain-inspired spiking neuron model constructed based on sparse dendritic topology and the dynamic characteristics of dendrites and cell bodies can efficiently process multi-timescale information, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale temporal information, and achieve high comprehensive performance in actual temporal tasks.

[0060] The time-domain signal processing method provided in this disclosure can use a spiking neuron model for signal processing, that is, it can use a spiking neuron model to process the time-domain signal to be processed for the target task and obtain the target result for the target task.

[0061] The target task can be any one of image processing, EEG signal processing, or speech signal processing. Image processing tasks can be further divided into different types such as image super-resolution and image classification tasks, and the same applies to EEG and speech signal processing tasks. It should be noted that although image processing, EEG signal processing, and speech signal processing tasks are used as examples to illustrate the target task, those skilled in the art will understand that this disclosure is not limited to these. In fact, the target task can be flexibly set according to the actual application scenario, as long as the desired target result can be obtained.

[0062] The time-domain signal to be processed exhibits multi-time-scale characteristics; in other words, it can include information from multiple time scales. The time-domain signal to be processed can be divided into multiple signals, each with different time-scale information.

[0063] The spiking neuron model can be constructed based on the sparse dendritic topology and the dynamics of dendrites and cell bodies. Figure 1 A schematic diagram of a spiking neuron model provided according to an embodiment of this disclosure is shown. Figure 1 As shown, the spiking neuron model may include an input layer, a temporally heterogeneous neuron layer, and an output layer. A first channel (i.e., ...) may be provided between the input layer and the temporally heterogeneous neuron layer. Figure 1 The input layer synaptic connections are used to transmit information from the input layer to the temporal heterogeneous neuron layer; the temporal heterogeneous neuron layer may include multiple heterogeneous neurons, each of which may include a cell body with multiple heterogeneous dendritic branches; a second channel (i.e., ...) may be provided between the output layer and the temporal heterogeneous neuron layer. Figure 1 The output layer synaptic connections are used to transmit information from the temporally heterogeneous neuron layers to the output layer.

[0064] In the input layer, for time-domain signals that are typically high-dimensional, the signals can be divided into multiple signals according to their dimensions. Each signal can correspond to information in different dimensions, which may include different time scale information. Therefore, each signal can be used as an input in the input layer (i.e.,...). Figure 1 The purple dot (in the diagram) can transmit the input from the input layer to the temporally heterogeneous neuron layer through the first channel.

[0065] In the time-domain heterogeneous neuron layer, each heterogeneous neuron can receive all inputs from the upper layer. Within the same heterogeneous neuron, each heterogeneous dendritic branch receives a portion of the inputs from the upper layer. Each heterogeneous dendritic branch is sparsely and uniformly distributed with a certain number of synaptic connections, and each heterogeneous dendritic branch is designed as a small parallel "resistor-capacitor" circuit. Each heterogeneous neuron can have one output.

[0066] In the case where the spiking neuron model includes a layer of neurons with temporal heterogeneity, such as Figure 1 As shown, the upper layer of the temporal heterogeneous neuron layer is the input layer. Each heterogeneous neuron in the temporal heterogeneous neuron layer can receive all inputs from the input layer. For any given heterogeneous neuron, each heterogeneous dendritic branch can receive a portion of the input from the input layer, allowing the heterogeneous neuron to receive all inputs from the input layer. In other words, the temporal signal to be processed for the target task can be input to each heterogeneous neuron in the temporal heterogeneous neuron layer. Each signal to be processed can be transmitted to the heterogeneous dendritic branches of the temporal heterogeneous neuron through different first channels. Specifically, in one possible implementation, the input layer can be used to divide the temporal signal to be processed into multiple signals to be processed. Multiple first channels can be provided between the input layer and the temporal heterogeneous neuron layer, allowing each signal to be processed to be transmitted to the heterogeneous dendritic branches of the temporal heterogeneous neuron through different first channels. This not only achieves dimensionality reduction of high-dimensional signals but also facilitates the transmission of the temporal signal to be processed to the temporal heterogeneous neuron layer for signal processing via the first channels.

[0067] Now Figure 1 For example, to illustrate the signal transmission mechanism between the input layer and the temporally heterogeneous neuron layer: Figure 1 As shown, the input layer can include four inputs, two of which are inputs to the input layer (i.e. Figure 1 The first and fourth inputs from left to right in the input layer (hereinafter referred to as I1 and I4) are the same heterogeneous neuron in the temporal heterogeneous neuron layer (i.e., Figure 1 Signal transmission between one of the heterogeneous dendritic branches (D1') of the third heterogeneous neuron from the left in the temporal heterogeneous neuron layer (hereinafter referred to as C3') is through the first channel (i.e., the input layer synaptic connection). Figure 1 The purple lines in the diagram represent the sparsity of synaptic connections in the input layer, which can be set to 1 / the number of heterogeneous dendritic branches. This allows the signals to be processed corresponding to the two inputs I1 and I4 in the input layer to be transmitted through the first channel to the heterogeneous dendritic branch D1' of the heterogeneous neuron C3' in the time-domain heterogeneous neuron layer. Simultaneously, the remaining two inputs in the input layer (i.e., Figure 1The signal transmission between the second and third inputs from left to right in the input layer (hereinafter referred to as I2 and I3) and another heterogeneous dendritic branch of the heterogeneous neuron C3' (hereinafter referred to as D2') is through the first channel (i.e., the input layer synaptic connection). Figure 1 The purple lines in the diagram represent the sparsity of the input layer synaptic connections (which can be set to 1 / the number of heterogeneous dendritic branches). This allows the remaining inputs I2 and I3, corresponding to the signals to be processed, to be transmitted through the first channel to the heterogeneous dendritic branch D2'. This enables the heterogeneous neuron C3' in the temporal heterogeneous neuron layer to receive all the inputs (i.e., the temporal signals to be processed) from the input layer. The heterogeneous dendritic branches D1' and D2' of the heterogeneous neuron C3' each receive a portion of the temporal signals to be processed. The signal transmission between the input layer and the remaining heterogeneous neurons in the temporal heterogeneous neuron layer is similar to that between the heterogeneous neuron C3' and C3'. In this way, by constructing a spiking neuron model based on a sparse dendritic topology, the synaptic connections (i.e., the first channel) between the input layer and the temporal heterogeneous neuron layer are sparsely and uniformly distributed across the heterogeneous dendritic branches. This avoids an excessive increase in the number of parameters in the spiking neuron model and helps improve the efficiency of signal processing.

[0068] In the case where the spiking neuron model includes multiple temporally heterogeneous neuron layers, Figure 2 A schematic diagram of a spiking neuron model provided according to an embodiment of the present disclosure is shown, such as... Figure 2 As shown, the layer above the first temporal heterogeneous neuron layer is the input layer, the layer above the second temporal heterogeneous neuron layer is the first temporal heterogeneous neuron layer, and so on. Each heterogeneous neuron in the first temporal heterogeneous neuron layer can receive all inputs from the input layer. Specifically, for any heterogeneous neuron in the first temporal heterogeneous neuron layer, each heterogeneous dendritic branch on it can receive a portion of the input from the input layer, so that the heterogeneous neuron can receive all the inputs from the input layer. The signal transmission method between the input layer and the first temporal heterogeneous neuron layer is described above, and will not be repeated here. Each heterogeneous neuron in the second temporal heterogeneous neuron layer can receive all the outputs (i.e., pulse output results, see below) of the first temporal heterogeneous neuron layer. Specifically, for any heterogeneous neuron in the second temporal heterogeneous neuron layer, each heterogeneous dendritic branch can receive a portion of the outputs from the first temporal heterogeneous neuron layer, thus enabling that heterogeneous neuron to receive all the outputs from the first temporal heterogeneous neuron layer. A third channel (i.e., ...) can be established between the first and second temporal heterogeneous neuron layers. Figure 2The interlayer synaptic connections in the model are used to transmit the processing results corresponding to the outputs of the first temporal heterogeneous neuron layer to the second temporal heterogeneous neuron layer. In other words, all outputs of the first temporal heterogeneous neuron layer can be input to each heterogeneous neuron in the second temporal heterogeneous neuron layer. The outputs of the first temporal heterogeneous neuron layer can be transmitted to the heterogeneous dendritic branches of the second temporal heterogeneous neuron through different third channels, and so on for the remaining temporal heterogeneous neuron layers. For simplicity, this will not be elaborated further here. Therefore, in one possible implementation, when the spiking neuron model includes multiple temporal heterogeneous neuron layers, the pulse output results obtained by each heterogeneous neuron in the previous temporal heterogeneous neuron layer are used as the input to the next temporal heterogeneous neuron layer, ensuring that each heterogeneous neuron in the next temporal heterogeneous neuron layer receives all pulse output results. Thus, by performing time-domain signal processing using a spiking neuron model with multiple temporal heterogeneous neuron layers, it is possible to better handle multi-scale time information with complex frequency domains, achieving higher accuracy, better robustness, and better generalization.

[0069] Now Figure 2 To illustrate the signal transmission mechanism between the first and second temporally heterogeneous neuron layers, consider the following example: Figure 2 As shown, the first temporal heterogeneous neuron layer may include six heterogeneous neurons, each of which has one output (i.e., a pulse output result, as detailed below). Three of the outputs of the first temporal heterogeneous neuron layer (i.e.,...) Figure 2 The outputs of the first, third, and fifth heterogeneous neurons from left to right in the first temporal heterogeneous neuron layer (hereinafter referred to as O1', O3', and O5') and the same heterogeneous neuron in the second temporal heterogeneous neuron layer (i.e., Figure 2 Signal transmission between the third heterogeneous neuron from left to right in the temporal heterogeneous neuron layer (hereinafter referred to as C3") and one of its heterogeneous dendritic branches (hereinafter referred to as D1") is through the third channel (i.e., interlayer synaptic connection). Figure 2 The red lines in the diagram represent the sparsity of interlayer synaptic connections, which can be set to 1 / the number of heterogeneous dendritic branches. This allows the processing results corresponding to the three outputs O1', O3', and O5' of the first temporal heterogeneous neuron layer (see details below) to be transmitted through the third channel to the heterogeneous dendritic branch D1' of the heterogeneous neuron C3' in the second temporal heterogeneous neuron layer. Simultaneously, the remaining three outputs of the first temporal heterogeneous neuron layer (i.e., Figure 2The signal transmission between the outputs of the second, fourth, and sixth heterogeneous neurons from left to right in the first temporal heterogeneous neuron layer (hereinafter referred to as O2', O4', and O6') and another heterogeneous dendritic branch of heterogeneous neuron C3" (hereinafter referred to as D2") is through the third channel (i.e., interlayer synaptic connection). Figure 2 The red lines in the diagram represent the sparsity of interlayer synaptic connections (which can be 1 / the number of heterogeneous dendritic branches). This allows the processing results corresponding to the remaining three outputs O2', O4', and O6' of the first temporal heterogeneous neuron layer (see below) to be transmitted through the third channel to the heterogeneous dendritic branch D2' of the heterogeneous neuron C3" in the second temporal heterogeneous neuron layer. This enables the heterogeneous neuron C3" in the second temporal heterogeneous neuron layer to receive all the inputs from the first temporal heterogeneous neuron layer, and the heterogeneous dendritic branches D1' and D2' of the heterogeneous neuron C3" to receive part of the outputs from the first temporal heterogeneous neuron layer. The signal transmission between the output of the first temporal heterogeneous neuron layer and the remaining heterogeneous neurons in the second temporal heterogeneous neuron layer is similar to that of heterogeneous neuron C3". If the spiking neuron model includes more than two temporal heterogeneous neuron layers, the signal transmission between other adjacent temporal heterogeneous neuron layers is similar to that in this example, and will not be elaborated here for simplicity. In this way, by constructing the spiking neuron model based on a sparse tree-like topology, the interlayer synaptic connections (i.e., the third channel) between adjacent temporal heterogeneous neuron layers are sparsely and uniformly distributed on each heterogeneous dendritic branch, which can avoid an excessive increase in the number of parameters in the spiking neuron model and help improve the efficiency of signal processing.

[0070] In the time-domain heterogeneous neuron layer, each heterogeneous dendritic branch of a heterogeneous neuron can have different first time factors to process information at different time scales. The first time factor can be used to process the signals transmitted to the heterogeneous dendritic branches. At the heterogeneous neuron, each heterogeneous dendritic branch can process the input signal based on different first time factors to obtain a first output value and send it to the cell body. Thus, each heterogeneous dendritic branch of a heterogeneous neuron can process information at different time scales to extract important features at the corresponding time scale and send these features to the cell body to convert them into cell body membrane potentials.

[0071] The time scale can be understood as frequency. When heterogeneous dendritic branches have a small first time factor, such processing of high-frequency information that decays rapidly on heterogeneous dendritic branches is beneficial for rapid response to high-frequency information. When heterogeneous dendritic branches have a large first time factor, such processing of low-frequency information that decays slowly on heterogeneous dendritic branches is beneficial for preserving long-range information and improving the ability of the spiking neuron model to process complex multi-scale frequency domain temporal information.

[0072] In the case where the signals received by each heterogeneous neuron in the temporal heterogeneous neuron layer originate from the input layer, Figure 3 A schematic diagram illustrating the calculation process of a spiking neuron model provided according to an embodiment of the present disclosure is shown. Figure 3 To illustrate the processing of signals by the heterogeneous dendritic branches of heterogeneous neurons, consider the following example: Figure 3 As shown, a heterogeneous neuron may include a cell body and three heterogeneous dendritic branches, each with different first time factors α1, α2, and α3. Each of the three heterogeneous dendritic branches can receive partial input from the upper layer (the input layer in this example), meaning each heterogeneous dendritic branch can acquire different multi-scale temporal information, as... Figure 3 Heterogeneous dendritic branch 1 acquires multi-scale time information corresponding to the blue dots, heterogeneous dendritic branch 2 acquires multi-scale time information corresponding to the green dots, and heterogeneous dendritic branch 3 acquires multi-scale time information corresponding to the orange dots. Each multi-scale time information corresponds to a different signal to be processed, and all multi-scale time information constitutes the time-domain signal to be processed. Each heterogeneous dendritic branch can acquire multi-scale time information (i.e., the signal to be processed) in the form of current values; that is, heterogeneous dendritic branch 1 can acquire the current value at the current time t. ( Figure 3 Not shown, This represents the sum of currents acquired by each synaptic connection on heterogeneous dendritic branch 1 at the current time t. This sum of currents can indicate multi-scale time information, i.e., the signal to be processed. Heterogeneous dendritic branch 2 can acquire the current value at the current time t. ( Figure 3 Not shown, This represents the sum of currents acquired by each synaptic connection on heterogeneous dendritic branch 2 at the current time t. This sum of currents can indicate multi-scale time information (i.e., the signal to be processed). Heterogeneous dendritic branch 3 can acquire the current value at the current time t. (like Figure 3 As shown, This represents the sum of currents acquired by each synaptic connection on heterogeneous dendritic branch 3 at the current time t. This sum of currents can indicate multi-scale time information (i.e., the signal to be processed). At the heterogeneous neuron, each heterogeneous dendritic branch can process the input signal to be processed based on different first time factors to obtain a first output value. That is, in this example, the three heterogeneous dendritic branches can process the input signal based on first time factors α1, α2, and α3, respectively. The corresponding first output value is obtained through processing. Taking heterogeneous dendritic branch 3 as an example, such as... Figure 3 As shown, heterogeneous dendritic branches 3 can be based on the first time factor α3 on the input. The second output value at the current time t is obtained through processing. Furthermore, heterogeneous dendritic branches 3 can be based on the first time factor α3 and the second output value i′3 at the previous time ( Figure 3 (Not shown) is processed to obtain the third output value i′3×α3( Figure 3 (not shown), thus the heterogeneous dendritic branch 3 can be the second output value at the current time t. The sum of the first output value and the third output value i′3×α3 determines the first output value at the current time t. The processing of inputs by heterogeneous dendritic branches 1 and 2 is similar to that of heterogeneous dendritic branch 3, and will not be elaborated here for simplicity. Therefore, in one possible implementation, each heterogeneous dendritic branch processes the input signal based on a different first time factor to obtain a first output value and sends it to the cell body. This can include: at each heterogeneous dendritic branch: the heterogeneous dendritic branch processes the input signal based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained from the previous time at that heterogeneous dendritic branch based on the first time factor to obtain a third output value, using the sum of the second and third output values ​​at the current time as the first output value. Thus, a spiking neuron model is constructed based on the dendritic dynamics. Each heterogeneous neuron in this model is a complex neuron with a heterogeneous dendritic structure. Each heterogeneous dendritic branch has a different first time factor to process information of different frequencies. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0073] In cases where the signals received by the heterogeneous dendritic branches of heterogeneous neurons originate from another layer of heterogeneous neurons in the temporal domain, i.e. Figure 2As shown, each heterogeneous neuron in the second temporal heterogeneous neuron layer can receive all the outputs (i.e., pulse output results, see below for details) of the first temporal heterogeneous neuron layer. Similarly, these outputs can also be input to each heterogeneous neuron in the second temporal heterogeneous neuron layer in the form of current values. The processing of these outputs by each heterogeneous dendritic branch on each heterogeneous neuron in the second temporal heterogeneous neuron layer is similar to the processing of the signal to be processed by each heterogeneous dendritic branch of the heterogeneous neuron described above. For the sake of simplicity, it will not be repeated here.

[0074] In the time-domain heterogeneous neuron layer, the cell bodies of each heterogeneous neuron can have different second time factors to process the information transmitted by the corresponding heterogeneous dendritic branches. The second time factor can be used to process the signals transmitted to the cell body. At the heterogeneous neuron, the cell body can process multiple first output values ​​from each heterogeneous dendritic branch based on the second time factor to obtain the pulse output result (i.e., the output of the heterogeneous neuron).

[0075] Figure 4 A schematic diagram illustrating the calculation process of a spiking neuron model provided according to an embodiment of this disclosure is shown. Figure 4 Let's take an example to illustrate how the cell body of a heterogeneous neuron processes information transmitted from its corresponding heterogeneous dendritic branches: For example... Figure 4 As shown, heterogeneous neurons may include a cell body and D heterogeneous dendritic branches (i.e., Figure 4 The heterogeneous dendritic branches are 1, 2, ..., D, where each of the D heterogeneous dendritic branches has a different first time factor α1, α2, ..., α3. d The cell body possesses a second time factor β. The cell body can receive signals from D heterogeneous dendritic branches. The cell body can be based on the corresponding second time factor β for all first output values ​​from D heterogeneous dendritic branches. The fourth output value at the current time t is obtained through processing. Furthermore, the cell body can obtain the fourth output value u′ of the cell body at the previous time based on the second time factor β. soma ( Figure 4 (Not shown) is processed to obtain the fifth output value u′. soma ×β, thus the cell body can obtain the fourth output value at the current time t. With the fifth output value u′ soma The sum of ×β determines the fourth output value u at the current time t. soma u somaThis represents the cell body membrane potential at the current time t. Within the cell body, when the cell body membrane potential rises to a preset threshold, a pulse (i.e., the pulse output result) is generated through a pulse activation function and output to the next layer for subsequent calculations. Simultaneously, the cell body membrane potential is reset. The next layer can be the output layer or the next layer of heterogeneous neurons in the temporal heterogeneous neuron layer. The processing of information transmitted from the corresponding heterogeneous dendritic branches by the cell bodies of other heterogeneous neurons in the temporal heterogeneous neuron layer is similar to the above process, and will not be elaborated further here for simplicity. Therefore, in one possible implementation, the cell body processes multiple first output values ​​from various heterogeneous dendritic branches based on a second time factor to obtain a pulse output result. This can include: the cell body processes multiple first output values ​​from various heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value; the sum of the fourth output value and the fifth output value at the current time is used as the fourth output value at the current time; and the pulse output result is determined based on the fourth output value at the current time and a preset threshold. In this way, a spiking neuron model is constructed based on the cell body dynamics characteristics. Each heterogeneous neuron in this model can have different second time factors to process information transmitted from various heterogeneous dendritic branches. Using such a spiking neuron model can better process multi-scale temporal information with complex frequency domains, and can achieve higher accuracy, better robustness, and better generalization in practical tasks involving the processing of multiple temporal information.

[0076] In the temporal heterogeneous neuron layer, feedback channels can also be provided. Each cell body can feed back its pulse output result (i.e., pulse) determined at the previous time to the heterogeneous dendritic branches of all cells in the same layer through different feedback channels. The pulse output result determined at the previous time can be incorporated into the current value obtained by the heterogeneous dendritic branches at the current time. Thus, in one possible implementation, the temporal heterogeneous neuron layer also includes feedback channels, so that the pulse output result obtained by the cell body at the previous time is transmitted to the heterogeneous dendritic branches through the feedback channels. In this way, adding feedback channels to the spiking neuron model can further enhance the signal processing capability of the spiking neuron model, which is beneficial for better handling of multi-scale temporal information with complex frequency domains. It can achieve higher accuracy, better robustness, and better generalization in practical tasks involving multiple temporal information processing.

[0077] Figure 5 A schematic diagram of a spiking neuron model provided according to an embodiment of this disclosure is shown. Figure 5As shown, feedback channels are established between each cell body in the first temporal heterogeneous spiking neuron layer and one heterogeneous dendritic branch of each cell body in the first temporal heterogeneous spiking neuron layer. Taking the third heterogeneous neuron from left to right in the first temporal heterogeneous spiking neuron layer as an example, feedback channels are established between the cell body of this neuron and one heterogeneous dendritic branch of this neuron, and between the cell body of the other three heterogeneous neurons and one heterogeneous dendritic branch of each of the other three heterogeneous neurons (i.e., ... Figure 5 The yellow lines in the diagram indicate that the pulse output determined by the cell body of this neuron can be transmitted to the heterogeneous dendritic branches of all heterogeneous neurons in this layer through four feedback channels, and the same applies to the other three heterogeneous neurons. Therefore, the current value obtained by at least one heterogeneous dendritic branch of each heterogeneous neuron in this layer at the current time can also include the pulse output determined by the cell bodies of each heterogeneous neuron in this layer at the previous time. Thus, the information processed by the heterogeneous dendritic branches can also include the pulse output at the previous time. The feedback channels in the second time-domain heterogeneous spiking neuron layer are similar to those in the first time-domain heterogeneous spiking neuron layer, and will not be elaborated further for simplicity. In this way, adding feedback channels to the spiking neuron model can further enhance the model's ability to process signals, which is beneficial for better handling multi-scale time information with complex frequency domains. This can achieve higher accuracy, better robustness, and better generalization in practical tasks involving multiple time information processing.

[0078] In the output layer, the input is the output of the previous layer (i.e., the pulse output result). The pulse output results determined by each heterogeneous neuron in the temporal heterogeneous neuron layer can be transmitted to the output layer through a second channel, allowing the output layer to determine the target result for the target task based on the pulse output results of each heterogeneous neuron. Therefore, in one possible implementation, multiple second channels are provided between the output layer and the temporal heterogeneous neuron layer, allowing the pulse output results of each heterogeneous neuron to be transmitted to the output layer through different second channels. The output layer then determines the target result based on the pulse output results of each heterogeneous neuron. In this way, by transmitting the output of the temporal heterogeneous neuron layer, i.e., the pulse output result, to the output layer through the second channel, the target result corresponding to the target task can be obtained, realizing the processing of the temporal domain signal to be processed.

[0079] In a spiking neuron model comprising a single layer of time-domain heterogeneous neurons, the layer above the output layer is that same layer. In a model comprising multiple layers of time-domain heterogeneous neurons, the layer above the output layer is the last layer of time-domain heterogeneous neurons. Thus, in one possible implementation, determining the target result for the target task based on the spiking outputs of each heterogeneous neuron can include determining the target result based on the spiking outputs of each heterogeneous neuron in the last layer of time-domain heterogeneous neurons. In this way, using a spiking neuron model with multiple layers of time-domain heterogeneous neurons for time-domain signal processing allows for better handling of multi-scale temporal information with complex frequency domains, resulting in higher accuracy, better robustness, and better generalization.

[0080] Figure 6 A flowchart illustrating a time-domain signal processing method provided according to embodiments of the present disclosure is shown. Figure 6 As shown, the time-domain signal processing method may include:

[0081] S601. Input the time-domain signal to be processed for the target task into each heterogeneous neuron in the time-domain heterogeneous neuron layer.

[0082] S602. At each heterogeneous neuron: each heterogeneous dendritic branch processes the input signal to be processed based on different first time factors to obtain a first output value and sends it to the cell body. The cell body processes multiple first output values ​​from each heterogeneous dendritic branch based on a second time factor to obtain a pulse output result.

[0083] S603. Determine the target result for the target task based on the pulse output results of each heterogeneous neuron.

[0084] The temporal signal processing method provided in this disclosure inputs a time-domain signal to be processed, including multi-timescale information, for a target task into each heterogeneous neuron in a temporal heterogeneous neuron layer. At each heterogeneous neuron: each heterogeneous dendritic branch processes the input signal to be processed based on different first time factors to obtain a first output value and sends it to the cell body. The cell body processes multiple first output values ​​from each heterogeneous dendritic branch based on a second time factor to obtain a pulse output result. Thus, the target result for the target task is determined based on the pulse output results of each heterogeneous neuron. In this way, the novel brain-inspired spiking neuron model constructed based on sparse dendritic topology and dendritic and cell body dynamics can efficiently process multi-timescale information, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale temporal information, and achieve high comprehensive performance in actual temporal tasks.

[0085] In one possible implementation, the target task can include any one of image processing, EEG signal processing, or speech signal processing. This allows the spiking neuron model to handle different signal processing tasks and obtain corresponding target results, thus broadening its applicability.

[0086] When the target task is image processing, the time-domain signal to be processed is a signal indicating image information. This signal is processed using a spiking neuron model to obtain the target result for the image processing task. For example, if the image processing task is image classification, the corresponding target result is the image classification result.

[0087] When the target task is EEG signal processing, the time-domain signal to be processed is the EEG signal. Figure 7 This diagram illustrates processing a brainwave signal processing task according to an embodiment of the present disclosure. Figure 7 As shown, the acquired EEG signals can be preprocessed first, and then the preprocessed EEG signals can be input into a spiking neuron model for signal processing to obtain the target result for the EEG signal processing task. For example, if the EEG signal processing task is emotion recognition, the corresponding target result is the emotion recognition result (i.e., such as...). Figure 7 (As shown).

[0088] When the target task is speech signal processing, the time-domain signal to be processed is a speech command signal. Figure 8 A schematic diagram illustrating the processing of a speech signal processing task according to an embodiment of this disclosure is shown. Figure 8 As shown, the acquired speech signal can be preprocessed first, and then the preprocessed speech signal can be input into the spiking neuron model for signal processing to obtain the target result for the speech signal processing task. For example, if the speech signal processing task is speech command recognition, the corresponding target result is the speech command recognition result (i.e., ...). Figure 8 (As shown).

[0089] It should be noted that although image processing, EEG signal processing, and speech signal processing tasks have been used as examples to illustrate the target tasks, those skilled in the art will understand that the embodiments disclosed herein are not limited to these. In fact, users can flexibly set the target tasks according to their actual application scenarios.

[0090] In one possible implementation, the spiking neuron model used in the time-domain signal processing method can be a trained model. For example, the initial spiking neuron model can be trained using a high-performance spatiotemporal gradient descent method, i.e., the spatiotemporal gradient backpropagation algorithm (STBP), to obtain a trained spiking neuron model. Alternatively, a multi-peak Gaussian gradient substitution function can be used for derivative approximation to address the problem of non-differentiability of binary impulse activity when the spiking neuron model is used for signal processing. During the training process of the spiking neuron model, not only are the synaptic connection weights trained, but also the time factors of the dynamics of different heterogeneous dendritic branches in each heterogeneous neuron (i.e., the first time factor) and the time factor of the cell body membrane potential dynamics (i.e., the second time factor) are trained. This allows for automatic adaptation of the spiking neuron model to the multi-timescale information of the input signal, enabling efficient processing of multi-timescale information based on the trained spiking neuron model and achieving high comprehensive performance in practical time-series tasks.

[0091] In one possible implementation, the spiking neuron model utilized by the time-domain signal processing method can be a time-domain heterogeneous spiking feedforward neural network. Heterogeneous neurons can be arranged into a time-domain heterogeneous spiking feedforward neural network according to a feedforward structure, i.e., as shown below. Figure 2 As shown, the bottom layer of the temporal heterogeneous spiking feedforward neural network is the input layer, i.e., the external input. After passing through two temporal heterogeneous neuron layers, the external input is output at the top of the temporal heterogeneous spiking feedforward neural network, i.e., the output layer. The interlayer connections between the two temporal heterogeneous neuron layers follow a sparse tree-like topology. The sparsity of the connections is inversely proportional to the number of heterogeneous dendritic branches, keeping the number of network parameters almost constant.

[0092] In one possible implementation, the spiking neuron model utilized by the time-domain signal processing method can be a time-domain heterogeneous impulse feedback neural network. Time-domain heterogeneous neurons can be configured into a time-domain heterogeneous impulse feedback neural network according to feedforward and feedback structures, i.e., as shown below. Figure 5 As shown, based on the temporal heterogeneous spiking feedforward neural network, each temporal heterogeneous neuron layer contains feedback connections (i.e., the feedback channels mentioned above). Each temporal heterogeneous neuron layer not only receives the output of the previous layer but also receives the feedback inputs (i.e., the spiking output results mentioned above) from each heterogeneous neuron in the previous time step. The interlayer connections between two temporal heterogeneous neuron layers and the intralayer feedback connections within each temporal heterogeneous neuron layer all follow a sparse tree-like topology. The sparsity of the connections is inversely proportional to the number of heterogeneous dendritic branches, keeping the number of network parameters almost constant.

[0093] Therefore, this disclosure considers the computational storage capacity of dendrites in time-domain heterogeneous biological neurons, which has been previously neglected, when modeling the spiking neuron model. By simplifying the cable characteristics of dendrites, the neuronal dendritic current decay mechanism is introduced. Simultaneously, a multi-branched dendritic structure (i.e., a cell body can have multiple heterogeneous dendritic branches) is added, proposing a spiking neuron model based on heterogeneous neurons with heterogeneous dendritic structures and a sparse dendritic topology. This model has the ability to process multi-scale information. The time-domain signal processing method provided by this disclosure, through the time-domain heterogeneous spiking neuron model, can better handle multi-scale time information with complex frequency domains. It can achieve higher accuracy, better robustness, and generalization in practical tasks involving multiple time information processing. Furthermore, considering the sparse pulse and event-driven characteristics of spiking neural networks, it has lower computational load and lower energy consumption compared to artificial neural networks in actual operation. Therefore, the time-domain signal processing method provided in this disclosure can improve or even solve the poor performance of conventional spiking neural networks when processing multi-scale time information. It can not only process multi-scale time information but also save long-range information to avoid the loss of time information during the saving process due to mechanisms such as resetting. It also provides a reasonable biological dendritic dynamics modeling method and a high-performance training method for training novel spiking neural networks with sparse tree-like topology and dendritic and cell body dynamics structures (i.e., a high-performance learning algorithm based on spatiotemporal gradient backpropagation).

[0094] This disclosure also provides a time-domain signal processing apparatus. The time-domain signal processing apparatus utilizes a spiking neuron model for signal processing. The spiking neuron model includes a time-domain heterogeneous neuron layer, which comprises multiple heterogeneous neurons. Each heterogeneous neuron includes a cell body with multiple heterogeneous dendritic branches, and each heterogeneous dendritic branch has a different first time factor. The first time factor is used to process signals transmitted to the heterogeneous dendritic branches. Each cell body has a different second time factor, which is used to process signals transmitted to the cell body. Figure 9 A block diagram of a time-domain signal processing apparatus provided according to embodiments of the present disclosure is shown. Figure 9 As shown, the time-domain signal processing device 900 may include:

[0095] Input module 901 is configured to input a time-domain signal to be processed for a target task into each of the heterogeneous neurons in the time-domain heterogeneous neuron layer. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time scale information.

[0096] The processing module 902 is configured to: at each of the heterogeneous neurons, each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on a second time factor to obtain a pulse output result.

[0097] Output module 903 is configured to determine a target result for the target task based on the pulse output results of each of the heterogeneous neurons.

[0098] Thus, the novel brain-inspired spiking neuron model constructed using sparse tree-like topology and dendritic and cell body dynamics can efficiently process information at multiple time scales, which helps to develop matching high-performance learning algorithms, improve their ability to process multi-scale temporal information, and achieve high comprehensive performance in practical temporal tasks.

[0099] In one possible implementation, when the spiking neuron model includes multiple temporally heterogeneous neuron layers, the spiking output results obtained by each heterogeneous neuron in the previous temporally heterogeneous neuron layer are used as the input of the next temporally heterogeneous neuron layer, so that each heterogeneous neuron in the next temporally heterogeneous neuron layer receives each of the spiking output results; determining the target result for the target task based on the spiking output results of each heterogeneous neuron includes: determining the target result based on the spiking output results obtained by each heterogeneous neuron in the last temporally heterogeneous neuron layer.

[0100] In this way, by using a spiking neuron model with multiple temporally heterogeneous neuron layers for time-domain signal processing, we can better handle multi-scale time information with complex frequency domains and obtain higher accuracy, better robustness and generalization.

[0101] In one possible implementation, the spiking neuron model further includes an input layer and an output layer; the input layer is used to divide the time-domain signal to be processed to obtain the plurality of signals to be processed, and a plurality of first channels are provided between the input layer and the time-domain heterogeneous neuron layer, so that each signal to be processed is transmitted to each heterogeneous dendritic branch of the time-domain heterogeneous neuron through different first channels; a plurality of second channels are provided between the output layer and the time-domain heterogeneous neuron layer, so that the spiking output results of each heterogeneous neuron are transmitted to the output layer through different second channels, and the output layer is used to determine the target result based on the spiking output results of each heterogeneous neuron.

[0102] In this way, not only can the dimensionality reduction of high-dimensional signals be achieved, but it also helps to transmit the time-domain signal to be processed to the time-domain heterogeneous neuron layer through the first channel for signal processing. Furthermore, the output of the time-domain heterogeneous neuron layer, i.e. the pulse output result, is transmitted to the output layer through the second channel, so as to obtain the target result corresponding to the target task and realize the processing of the time-domain signal to be processed.

[0103] In one possible implementation, each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body, including: at each of the heterogeneous dendritic branches: the heterogeneous dendritic branch processes the input signal to be processed based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained by the heterogeneous dendritic branch at the previous time based on the first time factor to obtain a third output value, and uses the sum of the second output value at the current time and the third output value as the first output value.

[0104] Thus, a spiking neuron model is constructed based on the dendritic dynamics. Each heterogeneous neuron in this model is a complex neuron with a heterogeneous dendritic structure. Each heterogeneous dendritic branch has a different first time factor to process information of different frequencies. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0105] In one possible implementation, the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the second time factor to obtain a pulse output result, including: the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value, and uses the sum of the fourth output value at the current time and the fifth output value as the fourth output value at the current time; and determines the pulse output result based on the fourth output value at the current time and a preset threshold.

[0106] In this way, a spiking neuron model is constructed based on cell body dynamics. Each heterogeneous neuron in this model can have a different second time factor to process the information transmitted from the heterogeneous dendritic branches. Using such a spiking neuron model, we can better process multi-scale time information with complex frequency domains and achieve higher accuracy, better robustness and generalization in practical tasks that process multiple time information.

[0107] In one possible implementation, the temporal heterogeneous neuron layer further includes a feedback channel, which transmits the pulse output result obtained by the cell body at the previous time to the heterogeneous dendritic branch through the feedback channel.

[0108] In this way, adding a feedback channel to the spiking neuron model can further enhance the spiking neuron model's ability to process signals, which is beneficial for better handling of multi-scale time information with complex frequency domains. It can achieve higher accuracy, better robustness and generalization in practical tasks involving multiple time information processing.

[0109] In one possible implementation, the target task includes any one of the following: image processing task, electroencephalogram (EEG) signal processing task, and speech signal processing task.

[0110] In this way, the spiking neuron model can be used to handle different signal processing tasks and obtain corresponding target results, making it more widely applicable.

[0111] In some embodiments, the functions or modules of the time-domain signal processing apparatus provided in this disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above time-domain signal processing method embodiments, which will not be repeated here for the sake of brevity.

[0112] This disclosure also proposes a time-domain signal processing apparatus, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described time-domain signal processing method when executing the instructions stored in the memory.

[0113] In some embodiments, the functions or modules of the time-domain signal processing apparatus provided in this disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above time-domain signal processing method embodiments, which will not be repeated here for the sake of brevity.

[0114] This disclosure also proposes a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the aforementioned time-domain signal processing method. The computer-readable storage medium can be volatile or non-volatile.

[0115] In some embodiments, the functions or modules of the computer-readable storage medium provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above time-domain signal processing method embodiments, which will not be repeated here for the sake of brevity.

[0116] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described time-domain signal processing method.

[0117] In some embodiments, the computer program product provided in this disclosure may have functions or include modules that can be used to execute the methods described in the above method embodiments. The specific implementation of these methods may refer to the description of the above time-domain signal processing method embodiments, which will not be repeated here for the sake of brevity.

[0118] Figure 10 A block diagram of an apparatus for a time-domain signal processing method according to an embodiment of the present disclosure is shown. For example, apparatus 1900 may be provided as a server or terminal device. (Refer to...) Figure 10 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0119] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0120] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0121] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0122] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0123] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0124] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0125] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0126] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0127] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0129] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A time-domain signal processing method, characterized by, The method utilizes a spiking neuron model for signal processing. The spiking neuron model includes a temporally heterogeneous neuron layer, which comprises multiple heterogeneous neurons. Each heterogeneous neuron includes a cell body with multiple heterogeneous dendritic branches, and each heterogeneous dendritic branch has a different first time factor. The first time factor is used to process signals transmitted to the heterogeneous dendritic branches. Each cell body has a different second time factor, which is used to process signals transmitted to the cell body. The method includes: The time-domain signal to be processed for the target task is input to each of the heterogeneous neurons in the time-domain heterogeneous neuron layer. The target task includes any one of the following: image processing task, EEG signal processing task, speech signal processing task. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time scale information. At each of the heterogeneous neurons: each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on a second time factor to obtain a pulse output result. The target result for the target task is determined based on the pulse output results of each of the heterogeneous neurons. The process of processing the input signal to be processed by each heterogeneous dendritic branch based on different first time factors to obtain a first output value and sending it to the cell body includes: at each heterogeneous dendritic branch: the heterogeneous dendritic branch processes the input signal to be processed based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained by the heterogeneous dendritic branch at the previous time based on the first time factor to obtain a third output value, and the sum of the second output value at the current time and the third output value is used as the first output value; The cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the second time factor to obtain a pulse output result, including: the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value, and uses the sum of the fourth output value at the current time and the fifth output value as the fourth output value at the current time; the pulse output result is determined based on the fourth output value at the current time and a preset threshold.

2. The method of claim 1, wherein, In the case where the spiking neuron model includes multiple temporally heterogeneous neuron layers, the pulse output results obtained by each heterogeneous neuron in the previous temporally heterogeneous neuron layer are used as the input of the next temporally heterogeneous neuron layer, so that each heterogeneous neuron in the next temporally heterogeneous neuron layer receives each of the pulse output results. The determination of the target result for the target task based on the pulse output results of each of the heterogeneous neurons includes: determining the target result based on the pulse output results obtained from each heterogeneous neuron in the last temporal heterogeneous neuron layer.

3. The method according to claim 1, characterized in that, The spiking neuron model also includes an input layer and an output layer; The input layer is used to divide the time-domain signal to be processed to obtain the plurality of signals to be processed. A plurality of first channels are provided between the input layer and the time-domain heterogeneous neuron layer, so that each signal to be processed is transmitted to each heterogeneous dendritic branch of the time-domain heterogeneous neuron through different first channels. Multiple second channels are provided between the output layer and the temporal heterogeneous neuron layer, so that the pulse output results of each heterogeneous neuron are transmitted to the output layer through different second channels. The output layer is used to determine the target result based on the pulse output results of each heterogeneous neuron.

4. The method according to any one of claims 1 to 3, characterized in that, The time-domain heterogeneous neuron layer also includes a feedback channel, which transmits the pulse output result obtained by the cell body at the previous time to the heterogeneous dendritic branch through the feedback channel.

5. A time-domain signal processing device, characterized in that, The device utilizes a spiking neuron model for signal processing. The spiking neuron model includes a temporally heterogeneous neuron layer, which comprises multiple heterogeneous neurons. Each heterogeneous neuron includes a cell body with multiple heterogeneous dendritic branches, and each heterogeneous dendritic branch has a different first time factor. The first time factor is used to process signals transmitted to the heterogeneous dendritic branches. Each cell body has a different second time factor, which is used to process signals transmitted to the cell body. The device includes: An input module is configured to input a time-domain signal to be processed for a target task into each of the heterogeneous neurons in the time-domain heterogeneous neuron layer. The target task includes any one of the following: image processing task, EEG signal processing task, or speech signal processing task. The time-domain signal to be processed includes multiple signals to be processed, and different signals to be processed have different time scale information. The processing module is configured such that at each of the heterogeneous neurons: each of the heterogeneous dendritic branches processes the input signal to be processed based on a different first time factor to obtain a first output value and sends it to the cell body; the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on a second time factor to obtain a pulse output result. An output module, configured to determine a target result for the target task based on the pulse output results of each of the heterogeneous neurons; The process of processing the input signal to be processed by each heterogeneous dendritic branch based on different first time factors to obtain a first output value and sending it to the cell body includes: at each heterogeneous dendritic branch: the heterogeneous dendritic branch processes the input signal to be processed based on the corresponding first time factor to obtain a second output value at the current time, and processes the second output value obtained by the heterogeneous dendritic branch at the previous time based on the first time factor to obtain a third output value, and the sum of the second output value at the current time and the third output value is used as the first output value; The cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the second time factor to obtain a pulse output result, including: the cell body processes multiple first output values ​​from each of the heterogeneous dendritic branches based on the corresponding second time factor to obtain a fourth output value at the current time, and processes the fourth output value obtained by the cell body at the previous time based on the second time factor to obtain a fifth output value, and uses the sum of the fourth output value at the current time and the fifth output value as the fourth output value at the current time; the pulse output result is determined based on the fourth output value at the current time and a preset threshold.

6. A time-domain signal processing device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 4 when executing instructions stored in the memory.

7. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 4.

Images