Event-driven Pulse Convolutional Neural Network

Abstract

The present invention discloses an event-driven pulsed convolutional neural network, which includes: a kernel module configured to store and process kernel values of at least one convolutional kernel in an event-driven manner; a neuron module configured to store and update neuron states of neurons in the pulsed convolutional neural network in an event-driven manner and output output pulse events generated by the updated neurons; a memory mapper configured to determine neurons projected by input pulse events from a source layer through convolution with at least one convolutional kernel, and wherein the neuron states of the determined neurons will be updated with applicable kernel values of at least one convolutional kernel, wherein the memory mapper is configured to process the input pulse events in an event-driven manner. The technical solution disclosed by the present invention can execute the pulsed convolutional neural network sCNN in an efficient and fast manner and consume extremely low energy in neuromorphic hardware.

Description

Technical Field

[0001] The present invention relates to a neuromorphic chip, and more particularly to an electronic circuit for operating an event-driven pulsed convolutional neural network. Background Art

[0002] Convolutional neural networks (CNNs) for efficiently performing CNN tasks and corresponding chip architectures are known in the art.

[0003] A particular type of CNN is the spiking convolutional neural network (sCNN), which mimics the function of biological neurons, i.e., when a certain membrane potential of a neuron is reached, a discontinuous signal in the form of an electrical pulse is generated. In contrast, traditional CNNs provide a continuous output for any input, and thus have lower energy efficiency than sCNNs.

[0004] Subsequently, although sCNNs can be well described mathematically, it is not known how to implement them in dedicated hardware. sCNNs implemented on general-purpose processors lose the advantage of specific data processing patterns, and thus are relatively slow or have particularly high memory requirements.

[0005] However, so far, there has been no dedicated electronic circuit specifically designed to operate sCNNs.

[0006] General (not necessarily convolutional) neural networks are based on IBM's "True North" chip architecture (US2014 / 0032465A1), based on the so-called crossbar architecture, which provides a neural network network in which each neuron can be connected to other neurons in almost any way. However, this comes at the cost of extremely high storage requirements, because each connection between neurons has associated weights. Therefore, since each neuron is interconnected with all other neurons, the storage requirements of the weight matrix containing all weights are approximately proportional to the square of the number of neurons, and the speed is difficult to be very fast.

[0007] In addition, in addition to the "True North" chip architecture, known CNN processors operate in a frame-based manner, which is different from the signal processing mode (frame-based and event-based) that is advantageous when processing sCNNs. So far, there has been no electronic circuit architecture specifically designed for efficient and fast execution of sCNNs. Summary of the Invention

[0008] The object of the present invention is to provide a system and method for executing a spiking convolutional network, which can effectively save energy consumption and storage resources.

[0009] In the following, if not otherwise stated or implied, the term "spiking convolutional neural network" and similar terms refer to at least one electronic circuit configured and arranged to operate a spiking convolutional neural network.

[0010] An event-driven spiking convolutional neural network, comprising a plurality of layers, wherein each layer comprises the following features:

[0011] A kernel module, configured to store and process kernel values of at least one convolutional kernel in an event-driven manner;

[0012] A neuron module, configured to store and update the neuron states of neurons in the network in an event-driven manner, and output spike events generated by the processed, e.g., updated, neurons;

[0013] A memory mapper, configured to determine, by convolution with at least one convolutional kernel, the neurons onto which incoming spike events from a source layer are projected, and wherein the neuron states of the determined neurons will be updated with applicable kernel values of at least one convolutional kernel, wherein the memory mapper is configured to process incoming spike events in an event-driven manner.

[0014] The event-driven sCNN is in particular an electronic circuit and / or at least one computer chip, comprising components such as a memory mapper, a neuron module, and a kernel module. In particular, in the electronic circuit or at least one computer chip, each component has a hard-wired correspondence.

[0015] In other words, although some components of the sCNN are programmable, these components do not particularly reflect or implement in a computer program or software and then execute on, for example, a general-purpose multi-purpose chip, i.e., the components of the sCNN are not virtual and cannot be used in a general-purpose computer chip, they are physical entities of the sCNN electronic circuit. The term "layer" particularly refers to at least one electronic circuit representing a layer in the sCNN, especially a layer in the general terms of CNN.

[0016] The event-driven sCNN according to the present invention comprises a finite and predetermined number of layers.

[0017] Compared with a traditional CNN, the components of the event-driven sCNN are configured to process information in an event-driven manner.

[0018] The events driving the sCNN are given by the spike events received by a layer of the sCNN.

[0019] The spike events are digital data containing structured information about the events.

[0020] Thus, compared with other CNNs, the event-driven sCNN does not run based on frames or clocks, and it is suitable for conforming to data streams provided, for example, by event-based dynamic vision cameras and other event-generating devices.

[0021] The kernel module stores the kernel values of at least one convolution kernel (sometimes only referred to as "kernel" in this specification), and the kernel values are applicable to all input spike events.

[0022] This enables the kernel module to use a relatively small associated kernel memory to store the kernel values of at least one convolution kernel.

[0023] Compared with general neural networks, the characteristics of neuron connections in the sCNN are a set of relatively small weights, so the kernel reduces the memory requirements of the sCNN.

[0024] The event-driven sCNN generally stores multiple convolution kernels in the kernel module.

[0025] The neuron module basically stores all the neurons of the layer, and the neurons are specifically related to the output feature map of the layer.

[0026] Each neuron is specifically included in the output feature map, and the position of the neuron in the output feature map is specifically given by the neuron coordinates.

[0027] In particular, the output feature map is a two-dimensional array of values, where these values correspond to the neuron states of the neurons.

[0028] The term "neuron" refers to an entity of the sCNN, which is characterized by including an adjustable neuron state, and the neuron state has the function of characterizing when and how the neuron outputs spike events.

[0029] In addition, the neuron module operates in an event-driven fashion.

[0030] In particular, for each input spike event, at least some neurons of the neuron module will be updated (i.e., processed) using specific kernel values and output spike events according to their neuron states (sometimes referred to as membrane potential or membrane voltage in the art).

[0031] The update process specifically includes the addition or subtraction of the neuron state stored at each respective neuron address and the applicable kernel value, and particularly also involves the calculation of a bias value.

[0032] It should be noted that the neuron module is configured to store and process the neuron states of neurons in the network. According to the updated neuron states, the neuron module can output one or more spike events generated by the updated neurons.

[0033] Generally speaking, not every updated neuron will output a spike event.

[0034] According to another embodiment of the present invention, the kernel module includes a kernel interface that is connected to an associated kernel memory, where the associated kernel memory is configured to store the kernel values of at least one convolutional kernel in kernel addresses, particularly multiple convolutional kernels, and where the kernel module is electronically connected to the neuron module.

[0035] The associated kernel memory includes or particularly is a physical memory, such as a memory arranged on each layer. The advantage of this is that each layer can be manufactured identically and operate as an autonomous unit.

[0036] Alternatively or additionally, the kernel memory includes or is a memory allocated to the layer but not essential to the layer. The kernel memory can be included in a global, particularly external kernel memory that is connected to the kernel module of the layer.

[0037] According to one embodiment, all layers of the sCNN are connected to a global kernel memory, where the global kernel memory includes a kernel memory associated with each layer.

[0038] The kernel address particularly refers to a kernel memory address where a kernel value is stored. According to another embodiment of the present invention, the neuron module includes a neuron memory, where the neuron memory is configured to store the neuron state of a neuron in a neuron address.

[0039] According to another embodiment, the neuron memory includes a plurality of neuron memory units that can be accessed in parallel, and such neuron storage units are also referred to as associated neuron memories.

[0040] A neuron memory, particularly each associative neuron memory, includes or, in particular, physically includes (e.g., is arranged on) a memory on each layer, or a memory that is allocated and connected to the layer but is not essential to the layer. Additionally or alternatively, the associative neuron memory can be included in a global, particularly external neuron memory of a neuron module connected to the layer.

[0041] In one embodiment, all layers of the sCNN are connected to a global neuron memory, where the global neuron memory includes a neuron memory associated with each layer.

[0042] Furthermore, the global kernel and the global neuron memory can consist of a single memory component.

[0043] A neuron address particularly refers to the address of a neuron memory that stores the neuron state.

[0044] According to another embodiment of the present invention, a memory mapper is electrically connected to a kernel module, specifically a kernel interface, where the memory mapper is configured and arranged as follows:

[0045] a) Receives input pulses, particularly single-pulse events, from a source layer of the sCNN through an electrical connection; the input pulse events include the coordinates of neurons in the source layer, particularly the coordinates regarding a single neuron in the source layer; and in response to the received pulse events,

[0046] b) Determines neuron coordinates and corresponding neuron addresses; particularly, in the neuron memory associated with the neurons projected after the received input pulse events are convolved with at least one convolution kernel,

[0047] c) Determines kernel coordinates (determines kernel addresses through corresponding kernel coordinates), where the kernel addresses are the addresses of kernel values applicable to the neuron states corresponding to the determined neuron addresses; the neuron states corresponding to the determined neuron addresses are updated by applicable kernel values, particularly, the applicable kernel values are from at least one convolution kernel or from multiple convolution kernels,

[0048] d) Provides the determined kernel addresses for the applicable kernel values, particularly provides the determined neuron addresses to the kernel module, particularly to the kernel interface.

[0049] The memory mapper is particularly an electronic circuit or an electronic circuit system.

[0050] The memory mapper further calculates the position to which the input pulse event is projected. This calculation is based on a fully hard-wired formula that enables the memory mapper to determine the neuron address to be updated and the kernel address with the applicable kernel value.

[0051] To determine the neuron address and the kernel address, programmable register values can be provided to the memory mapper, which are calculated using a defined formula.

[0052] Furthermore, the memory mapper is event-driven, in particular an asynchronous serial interface circuit, which has a predefined bandwidth parallel interface, enabling at least one pulse event to be processed at a time.

[0053] Furthermore, the memory mapper is an SRAM module, flash memory, etc.

[0054] Furthermore, the source layer is included in the sCNN. For example, when the pulse event generated by the neuron module is rerouted to the memory mapper of this layer, the source layer can even be the current layer.

[0055] Furthermore, the term "projects to" and similar terms in particular refer to the inverse of the CNN receptive field.

[0056] Since the sCNN is event-driven, the convolution operation can be performed particularly efficiently on a single pulse event.

[0057] According to another embodiment of the present invention, the kernel module is configured and arranged to receive the determined kernel address, in particular from the associated kernel memory, and provide, in particular output, the applicable kernel value stored at the determined kernel address (in particular in combination with the determined neuron address) to the neuron module.

[0058] According to another embodiment of the present invention, the neuron module is configured and arranged to:

[0059] a) Receive the determined neuron address and the applicable kernel value;

[0060] b) For each received neuron address, determine the updated neuron state for the neuron according to the applicable kernel value;

[0061] c) Output outgoing pulse events for the updated neuron, in particular when the neuron state is updated, for example, especially when it exceeds or falls below at least one predetermined threshold.

[0062] That is, if the neuron state reaches a predetermined threshold, a pulse event is generated.

[0063] Furthermore, the neuron states are represented by numbers.

[0064] The term "reach" in particular refers to the neuron state being below or above a threshold.

[0065] According to another embodiment of the present invention, the memory mapper is configured and arranged to determine, for each received input pulse event, at least one output feature map, which is composed of neurons assigned to the output feature map, wherein the number of output feature maps is equal to the number of convolutional kernels of the current layer.

[0066] The feature map can be represented as a two-dimensional array of neurons, which can be addressed by their neuron coordinates, and each neuron has a neuron state.

[0067] The input pulse events are projected onto the same number of feature maps as the number of kernels in the layer.

[0068] According to this embodiment, each kernel generates its associated output feature map.

[0069] According to another embodiment of the present invention, each layer of the sCNN further includes a destination mapping, where the destination mapping is connected to the output, in particular to the bus of the neuron module, and where the destination mapping is configured and arranged to dump outgoing (also referred to as departing or departure in the present invention) pulse events and / or generate output destination information of the output pulse events received from the neuron module of the current layer and associate it, in particular append the said destination information to the output pulse events, wherein the output destination information includes information about at least one target layer included in multiple layers to which the outgoing pulse event will be transmitted, specifically, the target information includes output feature map information of the generated pulse event, the neuron coordinates of the pulse event in the output feature map, and / or an offset value relative to the feature map index.

[0070] This embodiment allows each layer to autonomously determine the destination information, so that the sCNN can be extended by adding more layers in a modular manner.

[0071] According to another embodiment of the present invention, the neuron module includes a plurality of neuron sub-interfaces operating in parallel, which are configured to process the received neuron addresses and kernel values, in particular the received bias values, and generate updated neuron states and pulse events, wherein each neuron sub-interface includes an associated neuron memory for reading the neuron state of the received neuron address and writing the updated neuron state to the received neuron address in the relevant neuron memory, so that the parallel processing of neuron states is implemented by the neuron sub-interface and its associated neuron memory.

[0072] Typically, since the processes of reading and writing on the memory are rather slow and time-consuming, the present application bypasses potential bottlenecks in network processing speed, allowing for efficient and rapid processing of pulse events from previous layers. According to this embodiment, the determined neuron addresses and applicable kernel values are assigned to multiple neuron sub-interfaces for parallel processing.

[0073] According to another embodiment of the present invention, the neuron module includes a router module that is configured, adapted, and connected to receive the determined neuron addresses, applicable kernel values, in particular, and / or bias values from a bias module, and transmit the received neuron addresses and kernel values, and / or bias values, to multiple neuron sub-interfaces operating in parallel, which are configured to process the received neuron addresses and kernel values, especially parallel bias values, in parallel. Among them, the neuron module further includes a merger module that is configured to receive the pulse events generated from multiple neuron sub-interfaces and funnel the pulse events for serial processing. Further, in a bus where the number of channels is less than the number of neuron sub-interfaces, and further, where the bus has only a single transmission channel. This embodiment allows adaptation to the serial bus of the neuron module and the serial bus connection from the neuron module, and at the neuron module, it facilitates parallel processing.

[0074] According to another embodiment of the present invention, each layer further includes a bias module that includes an associated bias memory, in particular a bias interface connected to the associated bias memory, where the associated bias memory is configured to store bias values at bias addresses, where the bias module is connected to the neuron module, and where the bias module is configured to provide bias values to the neuron module at particularly predetermined time intervals, and where the neuron module is configured to update the neuron states of all neurons in at least one output feature map according to the received bias values.

[0075] Similar to the kernel interface, the bias interface is configured to address the memory to receive, in particular, process bias values.

[0076] Furthermore, the bias module is not connected to the memory mapper. Thus, the bias memory provides bias values to the neuron module, in particular to at least one output feature map, that are independent of the information included in the input pulse events.

[0077] According to another embodiment of the present invention, the neuron module is configured to receive neuron addresses and kernel values, particularly bias values, from the kernel module and / or the bias module, and distribute the neuron addresses, kernel values, and bias values via one of the neuron sub-interfaces.

[0078] Read the neuron state of the received neuron address, particularly from the associated neuron memory, update the read neuron state using the received kernel value and / or the received bias value (particularly on the neuron sub-interface), compare the updated neuron state with at least one threshold value, which is stored in and accessible from a threshold register, and when the at least one threshold value is reached, the register can be programmed with the at least one threshold value to generate a pulse event provided to the destination mapping, and via a merging module, reset the updated neuron state to a reset neuron state to write the updated neuron state, particularly the reset neuron state, to the received neuron address of the relevant neuron memory.

[0079] For each neuron, the sCNN can include two threshold values, for example, a lower threshold value and an upper threshold value.

[0080] When the neuron generates a pulse event, the neuron state is set to the reset neuron state. For example, the reset neuron state can be zero or equal to the difference between the updated neuron state and the threshold value.

[0081] Therefore, the neuron module, particularly the neuron sub-interface, is configured to perform the necessary calculations to update the neuron states of the neurons projected by the input pulse events, particularly by applying them to the applicable kernel values in parallel. Thus, the neuron module includes the electronic circuits required to perform this task.

[0082] According to another embodiment of the present invention, the neuron module includes a plurality of neuron sub-interfaces having associated neuron memories for parallel access to the determined neuron addresses, wherein the neuron module is configured to distribute the plurality of received neuron addresses by connecting a neuron router module to the plurality of neuron sub-interfaces, where each neuron sub-interface and its associated memory sub-block are configured to process the received neuron addresses and kernel values.

[0083] This embodiment allows for parallel processing of neurons updated with kernel values.

[0084] Further, since reading from and writing to the associated neuron memory is a rather slow process, when using a conventional memory such as a random access memory, the processing time can be reduced by parallelizing this step.

[0085] This embodiment can process the input pulse events in real time even at high data rates.

[0086] According to another embodiment of the present invention, the kernel interface includes a plurality of kernel read / write interfaces for accessing kernel memory sub-blocks included in the associated kernel memory in parallel, wherein the kernel interface is configured to distribute the received kernel addresses to the plurality of kernel read / write interfaces through a kernel interface router system and collect the kernel values received from the kernel memory sub-blocks. This embodiment solves the problem of accelerating the similar slow read / write processes in conventional memories, so that sCNN can particularly perform real-time processing even at high data rates.

[0087] The specific manifestation of the term "real-time" processing or operation is that the average processing rate of sCNN for the input pulse events is equal to the average rate of the input pulse events in sCNN, thus avoiding memory overflow.

[0088] According to another embodiment of the present invention, the memory mapper is configured to execute the following operation sequence to determine the kernel address of the applicable kernel value and the neuron coordinates corresponding to the neuron address to be updated when a pulse event is determined. In the pulse event received by the memory mapper, the pulse event includes information of the coordinates (x, y) of the pulse event, or is composed of information of the coordinates (x, y) of the pulse event in the source feature map, and further includes a channel identifier (c), and the neuron coordinates include an x-identifier and a y-identifier.

[0089] Further, zero-padding or zero-filling (x + p x , y + p y ) the coordinates (x, y) of the pulse event with a predefined offset (p x , p y ) provided from and stored in the register for the coordinates (x, y), and further, from the zero-padded or zero-filled coordinates (x + p x , y + p y ), calculate the neuron anchor coordinates (x0, y0) to which the pulse event projects, and the kernel anchor coordinates (x0 k , y0 k ) corresponding to at least one kernel (f), wherein the kernel anchor coordinates (x0 k , y0 k) represents the kernel coordinates (f, x0 k , y0 k ) for each kernel in at least one kernel (f), and further, the neuron anchor coordinates are associated with the output feature map.

[0090] Starting from the neuron anchor coordinates and the kernel anchor coordinates, and through the size (size, also referred to as the dimension in this text) (H, W), stride size (stride size, also referred to as the stride in this text) (s x , s y ) and / or kernel size (size, also referred to as the dimension in this text) (H k , W k ) of the output feature map (430) provided by the register, all the neuron coordinates (f, x, y) to be updated and all the kernel coordinates (c, f, x k , y k ) of the applicable kernel values (142k) are determined.

[0091] Based on the determined neuron coordinates (coordinates) (f, x, y) and the kernel coordinates (coordinates) with applicable kernel values (values), compressed neuron addresses (addresses) and kernel addresses (addresses) are determined, and the determined (determined) neuron and kernel addresses (addresses) are provided to the kernel module and / or the neuron module.

[0092] The source feature map is the output feature map of the sCNN layer that has received the spike event.

[0093] Padding (or zero-padding, Zero Padding) the coordinates of the input spike event solves the problem of spike event convolution at the boundary of the feature map. Further, the offset (p x , p y ) is included in the programmable register.

[0094] In the context of the present specification, a register particularly refers to a memory device that provides faster read and write of the stored data compared to memories such as neuron memory or kernel memory. Therefore, the sCNN according to the present invention includes a register for storing information that is frequently accessed (or frequently visited).

[0095] Whenever a layer receives a spike event, the offset values are accessed. Therefore, the offset is stored in a register that is electrically connected to or included by the memory mapper.

[0096] The neuron anchor coordinates address neurons located at the projection window where the pulse events project onto in the output feature map, such as neurons at a corner of the projection window. The projection window is given by the kernel size, for example, its dimensions in x and y, the stride size of the convolution, and other possible factors. The neuron anchor coordinates are used to define the starting point of the convolution in the output feature map, especially for the scanning operation of scanning the kernel over the determined neuron addresses to update the neuron states of these neurons. The scanning of the kernel over the output feature map specifically depends on the stride size of the convolution.

[0097] The kernel anchor coordinates are further the coordinates of the kernel values located at a corner or near / close to a corner of the kernel. The kernel anchor coordinates are used to define the starting point to determine all applicable kernel values. This is particularly important for strides greater than 1.

[0098] The kernel anchor coordinates specifically depend on the stride size, the kernel size, and potentially other factors.

[0099] Once the neuron anchor coordinates and the kernel anchor coordinates are determined, a starting point for calculating the neurons affected by the convolution with applicable kernel values is generated. Starting from the starting point defined by the neuron anchor coordinates and the kernel anchor coordinates, all neuron coordinates (f, x, y) to be updated are determined, i.e., all neurons onto which the pulse events project, and all kernel coordinates (c, f, x k , y k ) with applicable kernel values are determined, specifically by processing information such as the size (H, W) of the output feature map, the stride size (s x , s y ) and / or the kernel size (H k , W k ) provided by the register.

[0100] The parameters required to determine the neuron coordinates to be updated are specifically stored in a register, which provides faster read and write speeds compared to traditional memory.

[0101] The neuron address is determined from the neuron coordinates, where the neuron coordinates specifically refer to the reference position in the output feature map, and the neuron address specifically refers to the storage address in the neuron memory.

[0102] Once the memory mapper determines the neuron address and the kernel address, these addresses are provided to the kernel module, specifically the kernel interface, where the applicable kernel values are received and sent to the neuron module, where the neuron state of the determined neuron will be updated using the applicable kernel values.

[0103] The memory mapper is configured and arranged to perform all these operations. To this end, the memory mapper is specifically configured and arranged to execute a formula that calculates the neuron address to be updated and the kernel address of the applicable kernel value. In particular, in the memory mapper, this formula is hardwired and not programmable. However, the values stored in registers, such as those used to provide the stride size, kernel size, etc., of the formula can be adjusted to appropriate values by programming the corresponding registers.

[0104] According to another embodiment of the present invention, the associated neuron memory is organized such that all neuron states are stored at consecutive neuron addresses, where the associated neuron memory has all neuron states stored at consecutive neuron addresses during operation, where the memory mapper is configured to generate consecutive neuron addresses for all neuron coordinates, such as generating compressed neuron addresses, and where the memory mapper is configured to provide the compressed neuron addresses to the neuron module.

[0105] This embodiment enables efficient utilization of the storage space of the associated neuron memory. According to another embodiment of the present invention, the associated kernel memory is organized such that all kernel values are stored at consecutive kernel addresses, in particular where the associated kernel memory has all kernel values stored at consecutive kernel addresses during operation, where the memory mapper is configured to generate consecutive kernel addresses for all kernel coordinates, such as generating compressed kernel addresses, and where the memory mapper is configured to provide the compressed kernel addresses to the kernel module.

[0106] This embodiment enables efficient use of the memory space of the associated kernel memory.

[0107] According to another embodiment of the present invention, an event-driven spiking convolutional network includes multiple layers connected to a spiking event router, which is configured to route output spiking events received from a source layer to at least one destination layer. Further, where the router accesses destination information associated with the output spiking event, and where the destination layer can be the source layer.

[0108] This embodiment provides an sCNN with multiple layers that are electrically connected and organized through a spiking event router. Preferably, the spiking event router is programmable to allow for various sCNN configurations and routing options.

[0109] A dynamic vision sensor is configured to generate event-driven spikes and can be used as the input or the first source layer of a spiking event router.

[0110] According to another embodiment of the present invention, an event-driven convolutional neural network is configured for event-driven, particularly asynchronous processing of pulsed events. Wherein, the sCNN is configured to receive asynchronous pulsed events and process each pulsed event upon reception; further, wherein the memory mapper is event-driven and is an asynchronous electronic circuit, the associated memory module is event-driven and is an asynchronous module, the kernel module is event-driven and is an asynchronous module, and the destination mapper is event-driven and is an asynchronous electronic circuit.

[0111] According to another embodiment of the present invention, the sCNN is configured to receive and process pulsed events from a dynamic vision sensor. Further, the sCNN includes a dynamic vision sensor. Additionally, the problem according to the present invention is solved by a computer program. The computer program is particularly configured to operate the sCNN, and more specifically, the components of the sCNN according to the present invention.

[0112] The terms and definitions introduced in the context of the embodiment of the sCNN also apply to the computer program.

[0113] The computer program includes instructions that, when the computer program is executed on at least one component of the sCNN, such as a memory mapper, neuron module, kernel module, destination mapper, and / or pulsed event router. The event-driven pulsed convolutional network according to the present invention uses the corresponding components to perform corresponding steps, and the components are configured and arranged to asynchronously process the input pulsed events.

[0114] Further, the computer program provides programmable register values for the registers of the sCNN.

[0115] According to another embodiment of the computer program, the computer program causes the memory mapper to:

[0116] a) Receive an input single pulsed event from the source layer of the network via an electrical connection. The input pulsed event includes information about the coordinates of a single neuron in the source layer, and in response to the received pulsed event,

[0117] b) Determine the neuron coordinates and corresponding neuron addresses in the neuron memory of the neuron to which the convolution projection of the received input pulsed event and at least one convolution kernel projects,

[0118] c) Determine the kernel coordinates of the corresponding kernel address of the kernel value having a neuron state applicable to the determined neuron address (determine the kernel address of the kernel value having a neuron state applicable to the determined neuron address according to the corresponding kernel coordinates), wherein the neuron state of the determined neuron address will be updated with the applicable kernel value. Further, the applicable kernel value is from at least one or more convolution kernels.

[0119] d) A kernel module, particularly a kernel interface, provides a determined kernel address for applicable kernel values, and further determines a neuron address.

[0120] According to another embodiment of the computer program, the computer program causes the neuron module to:

[0121] a) Receive a determined neuron address and an applicable kernel value at one of the neuron sub-interfaces,

[0122] b) Determine the updated neuron state of the neuron at each received neuron address according to the applicable kernel value,

[0123] c) Output a pulse event for neurons whose updated state reaches, exceeds, or is below at least one predetermined threshold, specifically after the neuron state has been updated.

[0124] According to another embodiment of the computer program, the computer program causes the destination mapping dump to output pulse events and / or generate output destination information for the output pulse events received from the neuron module, and associate the destination information with the output pulse events, where the output destination information includes information about at least one destination layer to which the output pulse event is to be transmitted, where the destination information includes information about the output feature map in which the pulse event is generated, the neuron coordinates of the pulse event in the output feature map, and / or an offset value relative to the feature map index.

[0125] According to another embodiment of the computer program, the computer program causes the neuron sub-interface to receive a neuron address, a kernel value, and a bias value, specifically received from the kernel module and / or the bias module, read the neuron state of the received neuron address, update the read neuron state with the received kernel value and / or the received bias value, compare the updated neuron state with at least one threshold, where the threshold is stored in a threshold register and accessible from the threshold register, where the register is programmable to have at least one threshold, and once the threshold is reached, a pulse event is generated specifically provided to the destination mapping, and reset the updated neuron state to a reset neuron state, and write the updated (i.e., specifically the reset neuron state) to the neuron memory associated with the received neuron address.

[0126] According to another embodiment of the computer program, the computer program causes the neuron module to allocate a plurality of received neuron addresses to a plurality of neuron sub-interfaces through a neuron router module, where the computer program also causes each neuron sub-interface to be associated with a corresponding memory to process the received neuron addresses and kernel values.

[0127] According to another embodiment of the computer program, the computer program causes the kernel interface to allocate / distribute the received kernel addresses to a plurality of kernel read / write interfaces by means of a kernel interface router system, and collect kernel values received from kernel memory sub-blocks.

[0128] According to another embodiment of the computer program, when the computer program performs the following operations, the computer program causes the memory mapper to perform the following sequence of operations to determine the kernel addresses of applicable kernel values and the neuron coordinates of the corresponding neuron addresses to be updated. The memory mapper receives a pulse event, which includes or consists of the following information: information on the coordinates (x, y) of the pulse event in the source feature map, further including a channel identifier (c), and the neuron coordinates include x and y identifiers.

[0129] Further, zero-padding or padding zeros (x + p x , y + p y ), the coordinates (x, y) of the pulse event have a predefined offset (p x , p y ) provided from the coordinates (x, y) and stored in a register;

[0130] Further, from the zero-padded or padded-zero coordinates (x + p x , y + p y ), calculate the neuron anchor coordinates (x0, y0) to which the pulse event projects, and the kernel anchor coordinates (x0 k , y0 k ) corresponding to at least one kernel (f), where the kernel anchor coordinates (x0 k , y0 k ) represent the kernel coordinates (f, x0 k , y0 k ) of each kernel in at least one kernel (f), especially in the case where the neuron anchor coordinates are associated with an output feature map;

[0131] Starting from the neuron anchor coordinates and the kernel anchor coordinates, by processing the information on the size (H, W) of the output feature map, the stride / step size (s x , s y ) and / or the kernel size (H k , W k ) provided by the register, determine all the neuron coordinates (f, x, y) to be updated, i.e., the neuron coordinates to which the pulse event projects, and all the kernel coordinates (c, f, x k , y k ).

[0132] Determine compressed neuron addresses and kernel addresses based on the determined neuron coordinates (f, x, y) and kernel coordinates with applicable kernel values;

[0133] Provide the determined neuron and kernel addresses to the kernel module and / or neuron module.

[0134] According to another embodiment of the computer program, the computer program organizes an associated neuron memory such that all neuron states are stored in consecutive neuron addresses, where the computer program causes a memory mapper to generate consecutive neuron addresses for all neuron coordinates, such as to generate compressed neuron addresses, and where the computer program also causes the memory mapper to provide the compressed neuron addresses to the neuron module.

[0135] According to another embodiment of the computer program, the computer program organizes an associated kernel memory such that all kernel values are stored in consecutive kernel addresses, where the computer program causes a memory mapper to generate consecutive kernel addresses for all kernel coordinates, such as to generate compressed kernel addresses, and where the computer program also causes the memory mapper to provide the compressed kernel addresses to the kernel module.

[0136] According to another embodiment of the computer program, the computer program causes a pulse event router to route output pulse events received from a source layer to at least one destination layer, in particular where the router accesses destination information associated with the output pulse events, and where the destination layer can be the source layer.

[0137] Furthermore, the problem according to the invention is solved by a computer-implemented method. The computer-implemented method is particularly configured to operate an sCNN, and furthermore, the components of the sCNN according to the invention. In addition, the computer-implemented method particularly includes at least some of the features and / or method steps disclosed for the computer program.

[0138] The problem is further solved by the memory mapper of the event-driven sCNN.

[0139] The terms and definitions introduced in the context of an embodiment of the sCNN also apply to the memory mapper. When the memory mapper receives a pulse event, the memory mapper according to the invention is configured to determine kernel addresses and neuron addresses that are updated in an event-driven manner, the pulse event including information about or consisting of coordinates (x, y), and further including a channel identifier (c), and where neuron coordinates include x and y identifiers, and where when the memory mapper receives a pulse event, the memory mapper performs the following operations:

[0140] - Zero-padding (x + p x , y + p y ), where the coordinates (x, y) of the pulse event have a predefined offset (p x , p y ) provided by and stored in a register;

[0141] - Calculate the neuron anchor coordinates (x0, y0) to which the pulse event projects from the coordinates (x, y), especially from the zero-padded coordinates (x + p x , y + p y ), and the corresponding kernel anchor coordinates (x0 k , y0 k ) of at least one kernel (f), where the kernel anchor coordinates (x0 k , y0 k ) indicate the kernel coordinates (f, x0 k , y0 k ) of each kernel in at least one kernel (f). Especially in the case where the neuron anchor coordinates are associated with the output feature map;

[0142] - Starting from the neuron anchor coordinates and the kernel anchor coordinates, determine all the neuron coordinates (f, x, y) (i.e., those to which the pulse event projects) and all the kernel coordinates (c, f, x k , y k ) to be updated by processing the information about the output feature map size (H, W), stride / step size (s x , s y ) and / or kernel size (H k , W k ) provided by the register to determine the applicable kernel values;

[0143] - Determine the compressed neuron addresses and kernel addresses according to the determined neuron coordinates (coordinates) (f, x, y) and the kernel coordinates (coordinates) with applicable kernel values (values);

[0144] - Provide the determined neuron and kernel addresses to the kernel module and / or the neuron module to process the neuron and kernel addresses.

[0145] According to another embodiment of the memory mapper, the memory mapper is configured to: generate consecutive neuron addresses for all neuron coordinates, such as to generate compressed neuron addresses, and wherein the memory mapper is configured to provide the compressed neuron addresses to the neuron module.

[0146] According to another embodiment of the memory mapper, the memory mapper is configured to: generate consecutive kernel addresses for all kernel coordinates, such as to generate compressed kernel addresses, and wherein the memory mapper is configured to provide the compressed kernel addresses to a kernel module.

[0147] Specifically, exemplary embodiments will be described below with reference to the accompanying drawings. The drawings are attached to the claims and are accompanied by text explaining the various features of the illustrated embodiments and aspects of the present invention. Each individual feature shown in the drawings and / or mentioned in the text of the drawings can be incorporated (also in a separate manner) into the claims related to the device according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0148] Figure 1 is a schematic diagram of a layer of the sCNN of the present invention.

[0149] Figure 2 is a schematic layout of an sCNN including multiple layers.

[0150] Figure 3 is the data flow in the memory mapper 130.

[0151] Figure 4 is a schematic diagram of how to determine neuron coordinates and applicable kernel values.

[0152] Figure 5 is a schematic data flow diagram illustrating the neuron update process.

[0153] Figure 6 is a parallelized neuron module architecture. DETAILED DESCRIPTION

[0154] Since it is not possible to exhaustively describe all alternatives, the key points of the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Based on the key points described in the specific embodiments of the present invention, those skilled in the art can completely apply means such as replacement, deletion, addition, combination, and rearrangement of the order of some technical features to obtain a technical solution that still follows the inventive concept of the present invention. These solutions that do not depart from the inventive concept of the present invention are also within the protection scope of the present invention.

[0155] Figure 1 Shows a schematic diagram of a layer 10 of an sCNN according to the present invention. Layer 10 is an electronic module containing various components and is usually integrated with several copies in the sCNN.

[0156] The boxed area refers to a unit or module, where the arrows indicate the data connections between components and the associated data flow directions.

[0157] The sCNN includes a memory mapper 130, which is configured to receive input pulse events 140 indicated by {c, x, y}. The input pulse events 140, particularly digital data, include location information of the input pulse events 140 and a channel index indicating the channel associated with the pulse events 140. The location information is particularly the x and y coordinates in the output feature map of the source layer or the x and y coordinates of a dynamic vision sensor (see Figure 2 ). The channel may indicate a color channel from a pulse generating device, such as a dynamic vision sensor. However, other channel markers are possible.

[0158] The task of the memory mapper is to determine all necessary information for the (current) layer 10 to perform convolution. Therefore, the memory mapper 130 needs to determine the neuron address 142n onto which the input pulse event 140 is projected, that is, which neuron in the layer is affected by the convolution of the pulse event 140 with at least one convolution kernel 410.

[0159] Once the memory mapper 130 receives the pulse event 140, the memory mapper 130 starts processing the pulse event 140. The memory mapper 130 is an event-driven memory mapper 130 and includes, for example, pipelined processing with multiple buffer levels and electronic combinational logic circuits. The combinational logic circuits add, multiply, and / or multiplex the respective bits of the pulse events c, x, and y.

[0160] The memory mapper 130 is connected to execute a predefined formula, which is configured to determine the neuron address 142n to be updated according to the applicable kernel value 142k. Due to the size of the feature map from the input pulse event 140, the size and number of output feature maps in the current layer 10, the number of kernels, the convolution stride (also known as the stride size), and other parameters may vary, so these values in the formula are provided by the connected registers or the registers 131 included in the memory mapper 130. Figure 1 All the registers 131, 163, 123 shown are similar fast memories, which are configured to perform fast read and write operations.

[0161] Note that throughout the specification, all registers are programmable, particularly configured to store non-trainable or un-trainable parameters of the sCNN.

[0162] Conversely, the kernel value 142k and the bias value are typically determined during the training process of the sCNN 1, as is well known in the art.

[0163] Register 131 is connected to memory mapper 130, which stores kernel values, strides, zero-padding of x and y, the number of kernels f, and the number of output feature maps 430 for at least one kernel.

[0164] Additionally, register 131 may store the size or dimensions of output feature maps 430, such as their height and width (e.g., measured in pixels).

[0165] Figure 3 An exemplary embodiment of memory mapper 130 is shown, as well as how neuron addresses and kernel values are determined by the memory mapper.

[0166] Further, the determined neuron address n comp and kernel address k comp are compressed so as not to waste storage space in the associated neuron and kernel memories. Memory mapper 130 outputs neuron address 142n and the kernel address to kernel module 110, which includes kernel interface 111 and associated kernel memory 112. Kernel interface 111 is a kernel memory interface.

[0167] At kernel interface 111, the kernel address is received, and kernel interface 111 reads the kernel address from the associated kernel memory 112.

[0168] Kernel memory 112 is a conventional memory, such as random access memory (RAM), flash memory, etc. The associated kernel memory 112 may be arranged on the same circuit board as the rest of layer 10, or may be connected to layer 10 and form an external storage device.

[0169] According to pulse event 140, multiple kernel values 142k need to be read from kernel memory 112, and since conventional storage devices such as SRAM are relatively slow, kernel module 110 is configured and arranged to read and write kernel values 142k from the associated kernel memory 112 in parallel (see Figure 6 ).

[0170] Figure 6 An exemplary embodiment is shown for solving this problem by parallelizing the read and write operations of the associated kernel and / or associated neuron memories 112, 122.

[0171] Assuming the number of convolutional kernels in the current layer 10 is N, kernel interface 111 outputs the kernel weights of N convolutional kernels associated with the neuron address, and updates the neuron state at the corresponding neuron address according to the values of the N convolutional kernels.

[0172] When each convolutional kernel 410 (number reference Figure 4)When projected onto the associated output feature map 430, the N convolutional kernels 410 are projected onto the N output feature maps 430 in the current layer 10. Assume the size of the kernel is N×M×H×W, where M is the number of input channels, H is the height of each kernel, and W is the width of each kernel. Thus, each output feature map 430 includes some neurons 420, and these neurons will be updated with the applicable kernel values (w k )to update. The neuron addresses of these neurons 420 are provided by the kernel interface 111 in the output dataset, and the output dataset includes the neuron addresses 142n associated with the applicable kernel values 142k. The neuron addresses of the neurons to be updated are provided by the kernel interface 111 and calculated by the memory mapper 130 as described above.

[0173] The kernel module 110 is electrically connected to the neuron module 120, which is configured and arranged to process the neuron states of the neurons. The neuron module 120 receives the output dataset from the kernel module 110. When the neuron module 120 receives the output dataset, it starts to process the dataset, that is, the neuron module 120 is event-driven.

[0174] In addition to the update of some neurons 420 in the output feature map 430, all neurons 431 in the output feature map 430 can be updated with the bias value 142b at repeated time intervals.

[0175] For this purpose, each layer 10, 10', 10” includes a bias module 160, and the bias module includes a bias interface 161 and an associated bias memory 162. The associated bias memory 162 is a conventional storage device and is configured to store the bias values to be applied to the neurons 431.

[0176] The bias interface 161 is connected to or integrated in the bias-related memory 162 and is configured to read and write bias values from the associated bias memory 162. In addition, the bias interface 161 is configured to output the bias values and provide the bias values and the neuron addresses to be updated by the bias values to the neuron module 120.

[0177] It is worth noting that generally, the bias values 142b are extended to all neurons 431 in a specific output feature map 430, while the neurons 420 to be updated with the kernel values 142k depend on the specific spike event 140 received by the memory mapper 130.

[0178] Therefore, the bias module 160 is not connected to the memory mapper 130 to exchange data and / or synchronize operations, and thus operates independently of the input event and in parallel with any event being processed. The operation of the bias module can be based on a clock device, but can also be operated by any trigger selected by the user.

[0179] The bias module 160 includes a bias register 163, which is a fast read / write memory component that stores the output feature map index indicating the output feature map 430 in the current layer 10 to be updated with the bias value 142b.

[0180] The neuron module 120 is connected to the bias module 160 and the kernel module 110, and is configured to receive the outputs from the modules 160, 110.

[0181] The neuron module 120 is connected to the neuron register 123. The neuron module 120 includes a plurality of neuron sub-interfaces 121s, where each neuron sub-interface 121s has an associated neuron memory 506 for storing the neuron state (see Figure 5 ), for reading, processing, and writing the neuron state (see Figure 6 ). Figure 5 For an embodiment of the neuron sub-interface. Whenever the neuron module 120 receives an output data set from the kernel module 110 or a bias value 142b from the bias module 160, the neuron module distributes the output data to one or more neuron sub-interfaces, which read the neuron state from the neurons to be updated with the applicable kernel value 142k and / or bias value 142b.

[0182] According to the pulse event 140 or the received bias value 142b, a plurality of neuron states must be read from the relevant neuron memory 506, and since conventional storage devices such as SRAM are relatively slow, the neuron module 120 is configured and arranged to read and write the neuron state in parallel from the associated neuron memory 506 (see Figure 6 ) via the associated neuron sub-interface and / or the associated neuron memory 506.

[0183] To update the neuron state using the kernel value 142k, the following formula can be used, for example:

[0184] s(n + 1) = mod(s(n) + w b , tr)

[0185] where s(n) represents the neuron state of the neuron stored in the associated neuron memory 506, and w bCorresponding to the applicable kernel value 142k or the bias value 142b. For example, the new neuron state s(n+1) is given by the modulo operation of the upper threshold and the lower threshold tr.

[0186] If s(n)+w b exceeds the threshold, the neuron state is reset to the reset neuron state through the module operation, and an output (outgoing) pulse event 150 is generated by the neuron sub-interface 121s. The neuron sub-interface 121s stores the new neuron state in the associated neuron memory 506 with the same receiving neuron address. This process is described in detail in Figure 5 . Note that Figure 5 provides a functional schematic diagram of the neuron module 120, while Figure 6 provides a structural schematic diagram of the neuron module 120.

[0187] Other defined ways of resetting the neuron state are also feasible and are elaborated in the context of Figure 5 .

[0188] For all neurons that have reached the lower threshold 502 or the upper threshold 502, the neuron interface generates an output pulse event 150. The output pulse event 150 is a data structure that includes the neuron coordinates of the neuron in the output feature map, including the neuron and the output feature map index for indicating the output feature map. This data structure is electrically transmitted to the destination mapper 170 (see Figure 1 ), which is also referred to as the target mapper in the context of the current specification.

[0189] The destination mapper is, for example, a merging buffer for merging the information received by the register 171 and the output pulse event 150, and the merging register 171 is connected to the destination mapper.

[0190] The destination mapper 170 is configured and arranged to append the information of the output (or outgoing) pulse event 150 so that it can be routed to the appropriate layer 10, 10', 10", which is also referred to as the destination layer in the context of the current specification.

[0191] To this end, the destination mapper 170 includes an associated register 171, which provides information about the destination layer 10, 10', 10" to which the output pulse event 150 will be routed.

[0192] The destination mapper 170 appends information of the output pulse event 150 such that it includes the x, y coordinates and index of the output feature map 430 where the pulsed neuron is located. Additionally, an output feature map index is included in the appended information, which indicates the position of the output feature map where the pulsed neuron is located. Moreover, the output pulse event 150 can be routed to multiple destination layers 10', 10". Thus, the destination mapper 170 appends this information such that it includes a destination layer index indicating another layer 10', 10" included by the sCNN and an offset value relative to the feature map index, thereby mapping the output pulse event to a predefined output feature map in the destination layer (or target layer). The appended offset value allows for layer stacking in the sCNN.

[0193] Subsequently, the output pulse event 150 (with the appended information as described above) can be provided to the pulse event router 200 of the sCNN, as Figure 2 shown.

[0194] Figure 2 The overall layout of the sCNN 1 according to the present invention is shown, which includes a pulse event router 200 and multiple layers 10, 10', 10" (as Figure 1 shown).

[0195] The pulse event router 200 receives pulse events 210 from the layers 10, 10', 10" or from an event generation device 220 (such as a dynamic vision sensor).

[0196] Then, the pulse event router 200 provides 230 such pulse events 210 based on the appended information of its destination layers 10', 10" for further processing of the pulse events 210.

[0197] The pulse event router 200 is, for example, a stream multiplexer / demultiplexer circuit. The pulse event router 200 is programmable and configured to route pulse events to the destination layers in a backward, forward, or cyclic manner, thereby providing a high degree of flexibility to the sCNN 1, especially in terms of deep learning networks.

[0198] The pulse event router 200 is configured to provide layer-to-layer data flow and chip-to-chip communication, represented as "chip output" and "chip input" ("chip-out" and "chip-in"), especially when multiple sCNNs 1 are arranged cooperatively into a network.

[0199] To this end, the pulse event router 200 includes an input port 240 for receiving signals and data from an external device between chips (which can be the pulse event router from a second sCNN). The pulse event router 200 is also configured to receive data, namely pulse events generated by the dynamic vision sensor 220 connected to the pulse event router 200. The dynamic vision sensor 220 can be regarded as the source layer of the sCNN, except that duplication or backpropagation is not allowed.

[0200] The pulse event router 200 has an output port 250 for outputting pulse events to another chip or as a result.

[0201] The sCNN 1 may further include more than 100,000 neurons, which are programmably assigned to the layers 10, 10', 10" of the sCNN 1 and the output feature map 430.

[0202] The layers 10, 10′, 10″, especially the registers 131, 163, 123, 171 of the layers 10, 10′, 10″ are programmed accordingly to reflect the specific layout of each layer 10, 10′, 10″, that is, how many convolutional kernels each layer contains, the output feature map size of each layer, etc.

[0203] The dynamic vision sensor 220 is, for example, a device with an array of light-receiving pixels. Each pixel is configured to output a pulse event when the detected light intensity changes. That is to say, the pixels of the dynamic vision sensor 220 are sensitive to changes in the detected light flux.

[0204] The dynamic vision sensor 220 is an event-driven device, that is, different from frame-based camera readout, when a pulse event occurs, the pixel outputs a pulse event. The advantage of such a dynamic vision sensor is dynamic imaging, which is very fast. Combined with the sCNN 1 of the present invention, the sCNN 1 can be utilized to fully exert the potential of real-time and ultra-fast data processing.

[0205] In Figure 3 the data flow in the memory mapper 130 is schematically shown. The purpose of the memory mapper 130 is to determine the neuron 420 to which the input pulse event 140 is projected. This is also shown in Figure 4 where the memory mapper 130 processes the input pulse event 140 based on the following steps:

[0206] The input pulse event 140 carries information {c, x, y} such as channels and coordinates. First, zero-padding or zero filling 300 is performed on the received input pulse event 140, where the coordinates of the pulse event 140 are offset by an offset value p x p y Conversion:

[0207] {c, x, y} → {c, x + p x , y + p y}}。

[0208] The purpose of zero-padding or zero-filling 300 is to avoid edge effects when the coordinates are too close to the boundary of the output feature map 430.

[0209] The offset values are provided by registers 300r, 131 associated with the memory mapper 130.

[0210] In the next step, the kernel mapper 301 module determines the kernel addresses of the applicable kernel values for at least one kernel.

[0211] The kernel mapper 301 is configured to calculate the neuron anchor coordinates onto which the input pulse event 140 projects for each output feature map 430.

[0212] The neuron anchor coordinates are given, for example, by the neuron at the lower right corner of the array located in the corresponding output feature map 430 onto which the event projects. This step involves handling the output feature map size, the convolutional kernel size, and the convolutional stride. In addition, the corresponding kernel anchor coordinates corresponding to the neuron anchor coordinates are calculated. The neuron anchor coordinates and the kernel anchor coordinates are used as the starting points for determining all the neurons 420 to be updated in the output feature map 430 and all the applicable kernel values 142k to be received from the kernel memory 112.

[0213] Obviously, the starting point based on the two anchor coordinates is not just the neuron coordinates of the neuron at the lower right corner, but can be any neuron as long as its relationship with the projection of the pulse event is known.

[0214] Then the neuron anchor coordinates and the kernel anchor coordinates are sent to the address scan module 302, which is configured to calculate the remaining neuron coordinates of the neurons 420 onto which the pulse event 140 projects and their associated neuron addresses 142n.

[0215] In addition, all the kernel addresses of the applicable kernel values 142k are determined by the address scan module 302. The calculation of the neuron coordinates 422 and the applicable kernel coordinates 411 is done by "scanning", that is, moving the convolutional kernel 410 according to the stride size. And the anchor coordinates determined on the output feature map 430, for example Figure 4 as shown.

[0216] Based on the neuron coordinates 422 and the kernel coordinates 411, the neuron addresses 142n and the kernel addresses are determined by the address scan module 302. For this purpose, the address scan module 302 can access the programmable register 302r.

[0217] As described above, compress the neuron addresses and kernel addresses so that the storage space is optimally reserved in the neuron memory 506 and the kernel memory 112.

[0218] As described above, provide the compressed neuron and kernel addresses from the memory mapper 130 to the kernel module 110.

[0219] In Figure 4 , the process of projecting the input pulse event 140 onto the neuron addresses and the applicable kernel values 142k is schematically shown.

[0220] The pulse event 140 comes from the source feature map 440, specifically from the output feature map 430 in the source layer, which is shown in the Figure 4 left column. Each square in the left column represents a neuron in the source layer feature map 440, SFM, with associated coordinates {x, y}. The neuron 421 that generates the pulse event 140 is black. For example Figure 4 in panel A of k , W k ), the current layer 10 that receives the pulse event 140 includes two kernels 410K1, K2, each kernel having a size / dimension of 3×3 kernel values (depicted as a 3×3 square matrix) (H

[0221] Stride (or step size) s x = s y is set to 2 in the x and y directions, i.e., the convolutional kernels K1, K2 move on the output feature maps OFM1 and OFM2 with a stride of 2.

[0222] Determine the neuron anchor coordinates x0, y0 and the kernel anchor coordinates corresponding to the first kernel K1 such that the neuron anchor coordinates x0, y0 are located at the lower right corner of the projection area onto which the pulse event 140 / 421 is projected, as shown in Figure 4 panel A of x Subsequently, the scanning module "scans" the first convolutional kernel K1 on the output feature map OFM1 according to the stride (or step size) size s Figure 4as shown in Panel B of, and determine another kernel coordinate and another neuron coordinate to which the pulse event 140 (represented as neuron 421) projects. Scan according to what is indicated by the framed area, which includes nine neurons in the source feature map 440 SFM (Source Feature Map) (and always includes neuron 421).

[0223] In the next step (Panel C), the first convolutional kernel K1 scans in the y direction (stride / step size is 2), and again determines the neuron coordinate 422 and the kernel coordinate in the first output feature map OFM1.

[0224] In Figure 4 Panel D of, the first convolutional kernel K1 moves along the x, and determines the last of the four neuron coordinates and the kernel coordinate of the first output feature map OFM1.

[0225] Then the same process is performed on the second kernel K2 (refer to Panels E to H). In this way, in the two output feature maps OFM1 and OFM2, a total of eight neuron coordinates and eight kernel coordinates from the two kernels K1, K2 are determined by the scanning module.

[0226] For the determined neuron coordinate 422 and kernel coordinate 411, determine the corresponding neuron addresses 142n (eight) and the kernel addresses with applicable kernel values 142k (eight) such that the neuron module 120 performs convolution.

[0227] Figure 5 Shows in detail and schematically how convolution is performed on the neuron module 120, particularly on the neuron sub-interface 121s.

[0228] When the neuron sub-interface receives, for example, the neuron address 142n and the applicable kernel value 142k from the neuron router module 601, the neuron sub-interface 121s reads 500 from the relevant neuron memory 506 the neuron state stored under the received neuron address 142n. Add 501 the applicable kernel value 142k to the neuron state. Then compare 502 the resulting neuron state with at least one threshold 502, typically two thresholds: a lower threshold and an upper threshold 502.

[0229] If the resulting neuron state exceeds one of the thresholds 502 (rises above the upper threshold or falls below the lower threshold), the neuron sub-interface 121s generates an output pulse event and resets the neuron state to the reset neuron state. In this example, the reset neuron state can be zero or the remainder of a modulo operation. The reset neuron state is written back 505 to the associated neuron memory 506 of the neuron sub-interface 121s. In the case where no pulse event is generated, the resulting neuron state is written back 505 to the associated neuron memory 506.

[0230] Figure 5 The schematic diagram of is applied in the same way to the bias value 142b and the corresponding neuron address received at the neuron sub-interface 121s, where, instead of the applicable kernel value 142k, the bias value 142b is added to the neuron state. The processing of the remaining operations is the same as that described for the reception of the kernel value 142k.

[0231] Figure 6 Schematically shows how to facilitate parallel access and pipelining of the associated kernel 112 or neuron memory 506. In Figure 6 it is shown that the neuron module 120 is configured to address a plurality of neuron sub-interfaces 121s, where each neuron sub-interface 121s is configured to read, write, and process neuron addresses and neuron states as described above. To this end, the neuron module 120 includes a neuron router module 601 that receives the applicable kernel value and the determined neuron address to be updated. The neuron module may also receive a bias value to be assigned to the determined neuron address. The router module 601 sends the received kernel value and the applicable neuron address to the corresponding neuron sub-interface 121s among the plurality of parallel-organized neuron sub-interfaces 121s. At the neuron sub-interface 121s, the applicable neuron address is read from the associated memory 506 and the kernel value 142k, in particular the bias value 142b (see Figure 5) is updated. Once the neuron state of the determined neuron address is updated, the updated neuron state is written back, i.e., stored in the associated memory 506. Based on the updated neuron state, the neuron sub-interface can generate pulse events and output the pulse events. Since the neuron interface is configured to operate in parallel, it compensates for the relatively slow read and write processes on the neuron memory, thus maintaining the processing speed. The neuron module also includes a merging module 602 that merges the generated pulse events from multiple neuron sub-interfaces 121s for further processing in a common, especially serial connection. The neuron module 120, where each neuron sub-interface 121s has its own accessible associated memory 506, allows for efficient parallel processing of reading, writing, and updating of multiple neuron states, thus increasing the processing speed of layer 10 compared to non-parallel. A similar architecture can also be implemented for the kernel module 110 such that the reading and writing of kernel values at the kernel module are parallelized accordingly.

[0232] Thus, the kernel module 100 includes a plurality of kernel sub-interfaces, each kernel sub-interface including an associated kernel memory for reading and writing kernel values. The plurality of kernel sub-interfaces are connected to a kernel router module that is configured to assign kernel addresses to kernel peer interfaces such that kernel values associated with the kernel addresses are read from the associated kernel memories. Additionally, the kernel module may include a kernel merging module that is configured to pool kernel values provided from the plurality of kernel sub-interfaces onto a serial bus.

[0233] The pulsed convolutional neural network according to the present invention provides dedicated electronic circuitry for operating modern pulsed convolutional neural networks in a memory and energy-efficient manner.

[0234]

[0235]

Claims

1. An electronic circuit, characterized in that: configured and arranged to operate an event-driven pulsed convolutional neural network (1), including multiple layers (10, 10', 10''), each layer including: a kernel module (110) configured to store and process kernel values of at least one convolutional kernel (410) in an event-driven manner; a neuron module (120) configured to store and update the neuron states of neurons in the event-driven pulsed convolutional neural network (1) in an event-driven manner, and output pulsed events (150) generated by the updated neurons (420); a memory mapper (130) configured to determine the neurons (420) projected by input pulsed events (140) from a source layer (10') through convolution with at least one convolutional kernel (410), and the neuron states of the determined neurons (420) will be updated with the applicable kernel values of at least one convolutional kernel (410); a destination mapper (170) configured and arranged to: attach information to outgoing pulsed events (150) such that they can be routed to appropriate layers; and, the memory mapper is configured to process input pulsed events in an event-driven manner.

2. The electronic circuit according to claim 1, characterized in that: the kernel module (110) includes a kernel interface (111) connected to an associated kernel memory (112); the associated kernel memory (112) is configured to store kernel values of at least one convolutional kernel in kernel addresses; and, the kernel module (110) is electrically connected to the neuron module (120).

3. The electronic circuit according to claim 1 or 2, characterized in that: the memory mapper (130) is electrically connected to the kernel module (110); furthermore, the memory mapper (130) is configured and arranged to: a) receive input pulsed events (140) from a source layer of the event-driven pulsed convolutional neural network (1), the input pulsed events (140) including information on neuron coordinates (421) in the source layer; and, in response to the received input pulsed events (140), b) determine the neuron coordinates (422) projected after convolution of the received input pulsed events (140) with at least one convolutional kernel and the neuron addresses of the corresponding neurons (420); c) determine a kernel address through corresponding kernel coordinates (411), the kernel address being the address of the kernel value applicable to the neuron state corresponding to the determined neuron address, wherein the neuron state corresponding to the determined neuron address is updated with the applicable kernel value; d) provide the determined neuron address to the kernel module.

4. The electronic circuit according to claim 3, characterized in that: the neuron module (120) is configured and arranged to: a) receive the determined neuron addresses (142n) and applicable kernel values (142k); b) for each received neuron address (142n), determine an updated neuron state for the neurons (420) based on the applicable kernel values (142k); c) Updated neurons (420) that reach at least one preset threshold (502) output departing pulse events (150).

5. The electronic circuit according to claim 3, wherein: The memory mapper (130) is further configured and arranged to: Provide the determined kernel address to the kernel module.

6. The electronic circuit according to claim 5, wherein: Provide the determined neuron address and the determined kernel address to the kernel module, specifically to the kernel interface (111).

7. The electronic circuit according to claim 6, wherein: The kernel module (110) is configured and arranged to: receive the determined kernel address and provide the applicable kernel value stored at the determined kernel address to the neuron module (120).

8. The electronic circuit according to claim 7, wherein: The kernel module (110) is configured and arranged to: provide the applicable kernel value stored at the determined kernel address, together with the determined neuron address, to the neuron module (120).

9. The electronic circuit according to claim 8, wherein: The neuron module includes a neuron memory, and the neuron memory is configured to store the neuron state of the neuron at the neuron address.

10. The electronic circuit according to any one of claims 4-9, wherein: The kernel address refers to the kernel memory address where the kernel value is stored; The neuron address refers to the neuron memory address where the neuron state is stored.

11. The electronic circuit according to claim 10, wherein: The memory mapper (130) is configured to: Determine at least one output feature map (430) for each received input pulse event (140), and the output feature map (430) consists of neurons assigned to the output feature map (430), wherein the number of output feature maps (430) is equal to the number of convolution kernels (410) of the current layer (10).

12. The electronic circuit according to claim 11, wherein: Each layer further includes a bias module (160), and the bias module (160) includes a bias memory (162) and a bias interface (161) connected to the bias memory (162); The bias memory (162) is configured to store bias values at the bias address; The bias module (160) is connected to the neuron module (120); The bias module (160) is configured to: provide bias values to the neuron module (120) for a predefined time interval; The neuron module (120) is configured to: update the neuron states of all neurons in at least one output feature map (430) based on the received bias values.

13. The electronic circuit according to claim 12, wherein: The neuron module (120) includes a plurality of neuron sub-interfaces (121s) operating in parallel; The multiple neuron sub-interfaces (121s) operating in parallel are configured to: process the received neuron address (142n) and kernel value (142k), and generate (501, 502) updated neuron states and pulse events (150); Wherein, each neuron sub-interface (121s) has an associated neuron memory (506), and the neuron memory (506) is used to read the neuron state according to the received neuron address, and write the updated neuron state into the associated neuron memory (506) according to the received neuron address, thereby realizing the parallel processing of neuron states through the neuron sub-interface and its associated neuron memory (506).

14. The electronic circuit according to claim 13, wherein: The multiple neuron sub-interfaces (121s) operating in parallel are further configured to: Process the received bias value (142b), and generate (501, 502) updated neuron states and pulse events (150).

15. The electronic circuit according to claim 13, wherein: The neuron module (120) includes a router module (601), which is adapted to: Receive the determined neuron address (142n) and kernel value (142k) from the kernel module (110), and transmit the received neuron address (142n) and kernel value (142k) to the multiple neuron sub-interfaces (121s) operating in parallel; The neuron sub-interfaces (121s) are configured to: process the received neuron address (142n) and kernel value (142k) in parallel.

16. The electronic circuit according to claim 15, wherein: The router module (601), which is further adapted to: Receive the bias value (142b) from the bias module (160), and transmit the received bias value (142b) to the multiple neuron sub-interfaces (121s) operating in parallel; The neuron sub-interfaces (121s) are further configured to: process the received bias value (142b) in parallel.

17. The electronic circuit according to claim 15 or 16, wherein: The neuron module (120) further includes a merging module (602), which is adapted to: Receive the pulse events (150) generated by the multiple neuron sub-interfaces (121s) operating in parallel, and aggregate the pulse events (150) for serial processing in a bus with fewer channels than the neuron sub-interfaces.

18. The electronic circuit according to claim 17, wherein: The bus has only one transmission channel.

19. The electronic circuit according to claim 4, wherein: The neuron module (120) is configured to: Receive the neuron address (142n) and kernel value (142k); Read (500) the neuron state from the associated neuron memory (506) according to the received neuron address (142n); Update (501) the read neuron state using the received kernel value (142k); Compare the updated neuron state with at least one threshold (502), generate a pulse event once the at least one threshold (502) is reached, and reset the updated neuron state to a reset neuron state; Write (505) the updated neuron state to the associated neuron memory (506) according to the received neuron address (142n).

20. The electronic circuit according to claim 19, wherein: The neuron module (120) is further configured to: Receive a bias value (142b); Update (501) the read neuron state using the received core value (142k) and the received bias value (142b).

21. The electronic circuit according to any one of claims 12-16, 18-20, wherein: The neuron module (120) is further configured to: Receive, via a router module, a neuron address (142n) and a core value (142k) from a core module (110), and receive a bias value (142b) from a bias module (160); Read (500) the neuron state from the associated neuron memory (506) according to the received neuron address (142n) via one of the neuron sub-interfaces (121s); Update (501) the read neuron state using the received core value (142k) and the received bias value (142b) via the neuron sub-interface (121s); Compare the updated neuron state with at least one threshold (502), the threshold being stored in a threshold register and the threshold register being accessed via the neuron sub-interface (121s); Compare the updated neuron state with at least one threshold (502), generate a pulse event once the at least one threshold (502) is reached, and reset the updated neuron state to a reset neuron state; In addition, the generated pulse event is provided to a destination mapper (170).

22. The electronic circuit according to claim 21, wherein: The destination mapper (170) includes an associated register (171).

23. The electronic circuit according to claim 22, wherein: The associated register (171) provides information about the destination layer (10, 10', 10”) to which the outgoing pulse event (150) will be routed.

24. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, wherein: The core module includes a core interface, and the core interface is connected to an associated core memory; in addition, The core memory is a memory arranged on each layer; Or, All layers are connected to a global core memory, where the global core memory includes core memories associated with each layer.

25. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, wherein: The neuron module includes a neuron memory, and the neuron memory includes a plurality of associated neuron memories that can be accessed in parallel; in addition, each associated neuron memory is a memory physically arranged on each layer; or, all layers are connected to a global neuron memory, where the global neuron memory includes neuron memories associated with each layer.

26. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, characterized in that: To determine the kernel address for the applicable kernel value (142k) and the neuron coordinates (421) corresponding to the neuron address (142n) to be updated, when the memory mapper (130) receives a pulse event (140), the memory mapper (130) is configured to perform the following operations, where the pulse event (140) includes or consists of information about the coordinates (x, y) of the neuron (421) and the channel identifier (c), and the pulse event (140) is generated by the neuron (421) with coordinates (x, y) in the source feature map (440), and the coordinates (x, y) include an x identifier and a y identifier: Based on the coordinates (x, y), calculate the neuron anchor coordinates (x0, y0) projected by the pulse event, and the kernel anchor coordinates (x0 k , y0 k ) corresponding to at least one kernel, where the kernel anchor coordinates (x0 k , y0 k ) represent the kernel coordinates (f, x0 k , y0 k ) of each kernel in the at least one kernel (f), and the neuron anchor coordinates are associated with the output feature map (430); Starting from the neuron anchor coordinates and the kernel anchor coordinates, determine all neuron coordinates (f, x, y) to be updated and all kernel coordinates (c, f, x k , y k ) for which the applicable kernel value (142k) is valid; Based on the determined neuron coordinates (f, x, y) and the kernel coordinates (c, f, x k , y k ) of the applicable kernel value (142k), determine the compressed neuron address and kernel address; Provide the determined neuron address and kernel address to the kernel module (110) and / or the neuron module (120).

27. The electronic circuit according to claim 26, characterized in that: The memory mapper (130) is further configured to perform the following operations: Determine all neuron coordinates (f, x, y) to be updated and all kernel coordinates (c, f, x k , y k ) for the applicable kernel values (142k), by processing the output feature map (430) dimensions (H, W) provided by the register with the stride (s x , s y ) and / or the kernel dimensions (H k , W k ) information.

28. The electronic circuit according to claim 27, characterized in that: The memory mapper (130) is further configured to perform the following operations: Zero-fill the coordinates (x, y) of the input pulse event (140) using a preset offset value (p x , p y ), where the preset offset value is provided by and stored in a register; Based on the coordinates (x + p x , y + p y ) after zero-padding, calculate the neuron anchor coordinates (x0, y0) projected by the pulse event, and the kernel anchor coordinates (x0 k , y0 k ) corresponding to at least one kernel.

29. The electronic circuit according to claim 25, characterized in that: The associated neuron memories are organized such that all neuron states are stored in consecutive neuron addresses; The memory mapper (130) is configured to: Generate consecutive neuron addresses for all neuron coordinates, and specifically, the generation of consecutive neuron addresses is to generate compressed neuron addresses; and, Provide the compressed neuron addresses to the neuron module (120).

30. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, characterized in that: It further includes a pulse event router (200), and the pulse event router (200) is connected to a plurality of layers (10, 10', 10"); The pulse event router (200) is configured to: route the outgoing pulse event (150) received from the source layer to at least one destination layer; wherein, the pulse event router (200) accesses the destination information associated with the outgoing pulse event (150).

31. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, characterized in that: The memory mapper (130) determines the neuron address to be updated and the kernel address with the applicable kernel value based on a fully hardwired formula.

32. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, characterized in that: The kernel module includes a kernel interface; The kernel interface includes a plurality of kernel read / write interfaces for accessing kernel memory sub-blocks included in an associated kernel memory in parallel.

33. The electronic circuit according to claim 32, characterized in that: The kernel interface is configured to: distribute the received kernel addresses to a plurality of kernel read / write interfaces through a kernel interface router system and collect kernel values received from the kernel memory sub-blocks.

34. The electronic circuit according to claim 33, characterized in that: The kernel module includes a kernel merging module; The kernel merging module is configured to: pool the kernel values provided from a plurality of kernel sub-interfaces to a serial bus.

35. The electronic circuit according to claim 32, characterized in that: The event-driven pulse convolutional neural network (1) is configured to: receive a data stream provided by a dynamic vision sensor or other event generation device.

36. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, 33-35, characterized in that: It further includes: A dynamic vision sensor.

37. The electronic circuit according to claim 36, characterized in that: The dynamic vision sensor is configured to: generate event-driven pulse events and serve as an input or a first source layer of a pulse event router (200).

38. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, 33-35, 37, characterized in that: The plurality of layers (10, 10', 10”) are electrically connected and organized through a pulse event router.

39. The electronic circuit according to claim 38, characterized in that: The pulse event router is programmable to allow various event-driven pulse convolutional neural network (1) configuration and routing options.

40. The electronic circuit according to claim 39, characterized in that: The pulse event router is configured to: Provide layer-to-layer data flow and chip-to-chip communication.

41. The electronic circuit according to claim 40, characterized in that: The pulse event router includes an input port and an output port and is further configured to: Route pulse events to a destination layer in a backward, forward or cyclic manner, The input port is used to receive signals and data from external devices between chips; The output port is used to output pulse events to another chip or as a result.

42. The electronic circuit according to any one of claims 1-2, 4-9, 11-16, 18-20, 22-23, 27-29, 33-35, 37, 39, characterized in that: The neuron module stores all neurons of the layer.

43. The electronic circuit according to claim 42, characterized in that: Each neuron is included in the output feature map, and the position of the neuron in the output feature map is specifically given by the neuron coordinates; and, The output feature map is a two-dimensional array of values that correspond to the neuron states of the neurons.

44. The electronic circuit according to claim 43, wherein: Updating the neuron states of the neurons in the event-driven spiking convolutional neural network (1) specifically includes: The addition or subtraction of the neuron state stored at the corresponding neuron address and the applicable kernel value.

45. The electronic circuit according to claim 44, wherein: Updating the neuron states of the neurons in the event-driven spiking convolutional neural network (1) is specifically based on the following formula: s(n + 1) = mod(s(n) + w b , tr), where s(n + 1) is the new neuron state, s(n) is the neuron state of the neuron stored in the associated neuron memory, w b is the applicable kernel value or bias value, and tr is the threshold value.

46. The electronic circuit according to claim 43, wherein: Each layer further includes a bias module (160), and the bias module (160) is configured to: provide a bias value (142b) for a predefined time interval to the neuron module (120); and, The bias value (142b) is extended to all neurons in a specific output feature map (430).

47. The electronic circuit according to claim 46, wherein: The neuron (420) to be updated with the kernel value (142k) depends on the specific spike event (140) received by the memory mapper (130).

48. The electronic circuit according to claim 46, wherein: The operation of the bias module is based on a clock device or based on a flip-flop.

49. The electronic circuit according to claim 43, wherein: The input spike event (140) is digital data, including the position information for generating the input spike event (140) and the channel index or channel identifier indicating the channel associated with the input spike event (140).

50. The electronic circuit according to claim 49, wherein: The position information is the coordinates (x, y) in the source layer output feature map or the coordinates (x, y) of a dynamic vision sensor.

51. The electronic circuit according to claim 50, wherein: The memory mapper (130) is configured to perform the following operations: According to the coordinates (x, y), calculate the neuron anchor coordinates projected by the spike event it receives, and the kernel anchor coordinates corresponding to at least one kernel, wherein the kernel anchor coordinates represent the kernel coordinates of each of the at least one kernel, and the neuron anchor coordinates are associated with the output feature map; Starting from the neuron anchor coordinates and the kernel anchor coordinates, determine all the neuron coordinates to be updated and all the kernel coordinates of the applicable kernel values.

52. The electronic circuit according to claim 51, wherein: The memory mapper (130) is further configured to perform the following operations: Zero-fill the coordinates (x, y) of the input pulse event (140) using a preset offset value (p x , p y ), where the preset offset value is provided by and stored in a register; According to the zero-padded coordinates, calculate the neuron anchor coordinates projected by the spike event, and the kernel anchor coordinates corresponding to at least one kernel.

53. The electronic circuit according to claim 52, wherein: The coordinates {c, x, y} of the input pulse event (140) are zero-padded by a preset offset value (p x , p y ) to {c, x + p x , y + p y}, where c is the channel identifier.

54. The electronic circuit according to claim 53, wherein: Whenever a layer receives a pulse event (140), it accesses a preset offset value; The preset offset value is stored in a register electrically connected to or included by the memory mapper (130).

55. The electronic circuit according to claim 46, wherein: The bias module (160) includes a bias register (163) that stores an output feature map index indicating the output feature map (430) in the current layer (10) to be updated with a bias value (142b).

56. The electronic circuit according to claim 43, wherein: The neuron module (120) outputs a pulse event (150) generated by the updated neuron (420) to the destination mapper (170); The destination mapper (170) attaches information to the output pulse event (150) such that it includes: the coordinates (x, y) of the updated neuron (420) that generated the output pulse event (150) in the output feature map (430) and the output feature map index.

57. The electronic circuit according to claim 56, wherein: The destination mapper (170) attaches information to the output pulse event (150) such that it includes: an index indicating the destination layer and an offset value relative to the feature map index, thereby mapping the output pulse event to a predefined output feature map in the destination layer.

58. The electronic circuit according to claim 51, wherein: The neuron anchor coordinates and the kernel anchor coordinates are used as a starting point for determining all the neurons (420) to be updated in the output feature map (430) and all the applicable kernel values (142k) to be received from the kernel memory (112).

59. The electronic circuit according to claim 58, wherein: The neuron anchor coordinates and the kernel anchor coordinates are sent to the address scanning module (302); The address scanning module (302) is configured to calculate the remaining neuron coordinates of the neurons (420) onto which the input pulse event (140) is projected and their associated neuron addresses (142n).

60. The electronic circuit according to claim 51, wherein: The neuron anchor coordinates are located at the neurons at the corners of the projection window onto which the pulse event is projected in the output feature map and are used to define the starting point of convolution in the output feature map; Determine the neuron anchor coordinates (x0, y0) and the kernel anchor coordinates corresponding to the first convolution kernel (K1); Scan the first convolution kernel (K1) sequentially on the output feature map along the x - direction and the y - direction according to the stride size, and determine another kernel coordinate and another neuron coordinate onto which the input pulse event (140) is projected; For the determined neuron coordinates and kernel coordinates, determine the corresponding neuron addresses and kernel addresses having applicable kernel values (142k) such that the neuron module (120) performs convolution.

61. The electronic circuit according to claim 58, wherein: All the kernel addresses of the applicable kernel values (142k) are determined by the address scanning module (302).

62. The electronic circuit according to claim 60, wherein: For other convolution kernels (K2), the same process as that for the first convolution kernel (K1) is performed.

63. The electronic circuit according to claim 60, wherein: The neuron module (120) includes a plurality of neuron sub-interfaces (121s) operating in parallel; When the neuron sub-interface receives a neuron address (142n) and an applicable kernel value (142k) from the neuron router module (601), the neuron sub-interface (121s) reads the neuron state stored at the received neuron address (142n) from the relevant neuron memory (506) and adds the applicable kernel value (142k) to the neuron state.

64. A memory mapper (130) configured in an electronic circuit that operates an event-driven pulse convolutional neural network (1), characterized in that, The memory mapper is configured to: When a pulse event (140) is received by the memory mapper (130), determine the kernel address and the neuron address (142n) to be updated in an event-driven manner, where the pulse event (140) includes or consists of information on the coordinates (x, y) and the channel identifier (c) of the pulse event, and the coordinates include an x identifier and a y identifier; and, The memory mapper (130) performs the following operations: Using a preset offset value (p x , p y ), zero-fill the coordinates (x + p x , y + p y ) of the pulse event (140), where the preset offset value is provided by a register and stored in the register; Calculate the neuron anchor coordinates (x0, y0) onto which the (301) pulse event is projected according to the coordinates (x, y) and the corresponding kernel anchor coordinates (x0 k , y0 k ) of at least one convolutional kernel (410), where the kernel anchor coordinates (x0 k , y0 k ) indicate the kernel coordinates (f, x0 k , y0 k ) of each kernel in at least one convolutional kernel (410), and the neuron anchor coordinates are associated with the output feature map (430); Starting from the neuron anchor coordinates and the kernel anchor coordinates (302), and using the information on the size of the output feature map (430), the stride, and / or the kernel size provided by the register, determine all the neuron coordinates (f, x, y) to be updated and all the kernel coordinates (c, f, x k , y k ) for the applicable kernel values (142k); Based on all determined neuron coordinates (f, x, y) and all kernel coordinates (c, f, x k , y k ) where the applicable kernel values are located, determine the compressed neuron address and kernel address; Provide the determined neuron address and kernel address to the kernel module (110) and / or the neuron module (120) to process the neuron address and the kernel address.

65. The memory mapper (130) according to claim 64, wherein: Provide the determined neuron address and kernel address to the kernel module (110) and the neuron module (120) to process the neuron address and the kernel address; and, The kernel module (110) is configured to store and process the kernel values of at least one convolution kernel (410) in an event-driven manner; The neuron module (120) is configured to store and update the neuron state of the neurons in the event-driven pulse convolutional neural network (1) in an event-driven manner and output a pulse event (150) generated by the updated neurons (420).

66. The memory mapper (130) according to claim 64 or 65, wherein: The neuron anchor coordinates are located at the corner of the projection window where the pulse event projects in the output feature map and are used to define the starting point of convolution in the output feature map; Determine the neuron anchor coordinates (x0, y0) and the kernel anchor coordinates corresponding to the first convolution kernel (K1); Scan the first convolution kernel (K1) sequentially on the output feature map along the x direction and the y direction according to the stride size, and determine another kernel coordinate and another neuron coordinate where the input pulse event (140) projects; For the determined neuron coordinates and kernel coordinates, determine the corresponding neuron address and the kernel address with the applicable kernel value (142k) such that the neuron module (120) performs convolution.

Images